A surface flow velocity measurement method and device based on multi-scale space-time image fusion

Abstract

The application provides a surface flow velocity measurement method and device based on multi-scale space-time image fusion, which comprises the following steps: acquiring a river video of a target measurement point; processing the river video to obtain a plurality of scale projection image sets; processing the plurality of scale projection image sets to obtain a plurality of scale space-time image graphs; processing the plurality of scale space-time image graphs to obtain initial interpreted flow velocities of the plurality of scale space-time image graphs and statistical characteristic parameters of the plurality of scale space-time image graphs; inputting the plurality of statistical characteristic parameters and the plurality of scale space-time image graphs into a measurement point flow velocity adaptive optimization model to obtain scale space-time image graph flow velocity weights; and obtaining a target surface flow velocity according to the initial interpreted flow velocities of the plurality of scale space-time image graphs and the scale space-time image graph flow velocity weights. The scheme of the application can solve the limitation of single projection resolution analysis of the traditional space-time image method and improve the accuracy and stability of river surface flow velocity measurement.

Description

Technical Field

[0001] This invention relates to the field of hydrological monitoring technology, and in particular to a method and device for measuring surface flow velocity based on multi-scale spatiotemporal image fusion. Background Technology

[0002] River flow velocity is one of the core basic parameters for hydrological monitoring, flood control and disaster reduction, and water conservancy engineering design. Its accurate measurement has important scientific value and engineering significance for flood evolution prediction, water resource allocation efficiency assessment, and water ecology research.

[0003] In recent years, computer vision-based spatiotemporal image-based flow velocity measurement technology has achieved groundbreaking applications in the field of hydrological monitoring. This technology characterizes the spatiotemporal coupled texture features generated by the movement of water surface tracers through aerial imagery, interprets the main texture direction of the spatiotemporal imagery using methods such as Fast Fourier Transform, and then calculates the flow velocity based on the velocity-displacement relationship. Compared to traditional contact measurement methods, it innovatively employs a monocular vision sensing system to achieve non-contact, large-area, high spatiotemporal resolution one-dimensional time-averaged surface flow field monitoring, effectively avoiding the interference that traditional contact measurements may cause to the water flow, and greatly improving the accuracy and convenience of the measurement.

[0004] However, traditional spatiotemporal imaging technology still faces significant technical bottlenecks in natural river flow velocity monitoring. One of its shortcomings stems from the fixed projection resolution parameter setting mechanism, which struggles to adapt to the strong spatiotemporal heterogeneity of water flow within natural river cross-sections. In actual monitoring, the flow characteristics (such as rapids, slow flows, and turbulent flows) at different measuring points often exhibit significant differences, making it difficult to achieve optimal adaptation across the entire area using traditional resolution parameter selection methods that rely on manual experience. When using high-resolution projection, the insufficient ability to capture sub-pixel-level motion features makes it difficult to guarantee the continuity of the tracer's displacement along the velocity normal, leading to fragmented spatiotemporal texture features, a significant decrease in signal-to-noise ratio, and increased errors in calculating the principal texture direction angle. Conversely, when the projection resolution is too low, the multi-target brightness superposition caused by the mixed pixel effect will significantly weaken texture contrast, resulting in blurred texture in the spatiotemporal image and causing a systematic shift in the principal direction angle, severely affecting the accuracy of the measurement results. Summary of the Invention

[0005] This invention provides a surface velocity measurement method and device based on multi-scale spatiotemporal image fusion, which can overcome the limitations of traditional spatiotemporal image methods with single projection resolution analysis and improve the accuracy and stability of river surface velocity measurement.

[0006] To solve the above-mentioned technical problems, the technical solution of the present invention is as follows:

[0007] A surface flow velocity measurement method based on multi-scale spatiotemporal image fusion includes:

[0008] Acquire river channel video at the target measurement point;

[0009] The river video is processed to obtain a multi-scale projection image atlas;

[0010] The multiple scale projection image atlases are processed to obtain multiple scale spatiotemporal image maps;

[0011] The spatiotemporal image maps at multiple scales are processed to obtain the initial interpretation flow rate and statistical characteristic parameters of the spatiotemporal image maps at multiple scales.

[0012] The statistical feature parameters of the multiple scale spatiotemporal image maps and the multiple scale spatiotemporal image maps are input into the measurement point flow velocity adaptive optimization model to obtain the flow velocity weights of the multiple scale spatiotemporal image maps corresponding to the initial interpreted flow velocities of the multiple scale spatiotemporal image maps; wherein, the measurement point flow velocity adaptive optimization model is trained according to the first preset network model;

[0013] The target surface velocity is obtained based on the initial interpretation velocity of the multiple scale spatiotemporal image maps and the velocity weights of the multiple scale spatiotemporal image maps.

[0014] Optionally, the river video information is processed to obtain a multi-scale projection image atlas, including:

[0015] The river video is projected and converted based on the preset calibration point information to obtain the projected video;

[0016] The projected video is downsampled in stages to obtain a multi-scale projection image atlas.

[0017] Optionally, the river video is projected and converted based on preset calibration point information to obtain a projected video, including:

[0018] pass as well as The river video is subjected to projection conversion processing to obtain a projected video;

[0019] Among them, X c This indicates the X coordinate of the calibration point in the geographic spatial coordinate system; Y... c This indicates the Y-coordinate of the calibration point in the geographic spatial coordinate system; Z-coordinate... c This represents the Z-coordinate of the calibration point in the geographic spatial coordinate system; x i This indicates the x-coordinate of the calibration point on the river video; y-coordinate... i M represents the y-coordinate of the calibration point on the river video; 变 Denotes the projection transformation matrix; m 11 m 12 …m 34 This represents the parameters of the projection transformation matrix.

[0020] Optionally, the projected video is downsampled at different scales to obtain a multi-scale projected image atlas, including:

[0021] pass and The projected video is downsampled in stages to obtain a multi-scale projection image atlas;

[0022] Where (x′,y′) represents the reference image coordinate mapping; W(k) represents the bicubic kernel function, k represents the distance between the current pixel and the target interpolation position; a represents the parameter, a=-0.5; δ represents the resolution, in m / pixel; I δ (x,y) represents an image projected as δm / pixel; I base Indicates the initial image, I base (x,y) represents the reference projected image with a resolution of 0.01m / pixel; W(i) represents the interpolation weight in the x-direction; W(j) represents the interpolation weight in the y-direction.

[0023] Optionally, the multiple scale projected image atlases are processed to obtain multiple scale spatiotemporal image maps, including:

[0024] The positions of the target measuring points are located on the multiple scale projection image atlases to obtain multiple target measuring point velocity lines;

[0025] Pixels on the velocity measurement lines of the multiple target measurement points are extracted frame by frame and stacked over time to obtain spatiotemporal image maps of multiple scales.

[0026] Optionally, the multiple scale spatiotemporal image maps are processed to obtain an initial interpretation flow rate for the multiple scale spatiotemporal image maps, including:

[0027] pass Domain transformation is performed on the spatiotemporal image maps at multiple scales to obtain spectral image maps at multiple scales;

[0028] Where F(u,v) represents a multi-scale spectral image; u,v represent frequency parameters; M represents the length of the multi-scale spatiotemporal image; N represents the width of the multi-scale spatiotemporal image; f(m,n) represents the pixel value of the multi-scale spatiotemporal image at (m,n); e represents the base of the natural logarithm; and j represents the imaginary number.

[0029] Spectral centering is performed on the multiple scale spectral image maps to obtain multiple scale spectral image processing maps;

[0030] In the polar coordinate system of the component within the multi-scale spectral image processing diagram, through The maximum energy direction of the spectrum of multiple scale spectral images is obtained; where α represents the angle in the polar coordinate system, with the counterclockwise direction as the positive direction, and the value range is [0,π]; R represents the frequency domain integration radius; F(rcosα,r sinα) represents the energy value located at (r cosα,r sinα);

[0031] pass The main texture direction of the multi-scale spectral image processing map is transformed to obtain the optimal main texture direction of the multi-scale spectral image processing map; where θ represents the optimal main texture direction of the spatiotemporal image map measured from the positive x-axis clockwise direction;

[0032] The initial interpretation flow rate of the spatiotemporal image map at multiple scales is obtained by using v1 = cotθ * dx * fps; where v1 represents the initial interpretation flow rate of the spatiotemporal image map at multiple scales; θ represents the optimal main texture direction of the spectral image processing map at multiple scales; dx represents the pixel resolution of the spatiotemporal image map at multiple scales, in m / pixel; and fps represents the frame rate of the river video, in frames / s.

[0033] Optionally, the spatiotemporal image maps at multiple scales are processed to obtain statistical feature parameters of the spatiotemporal image maps at multiple scales, including:

[0034] pass The average brightness of spatiotemporal imagery at multiple scales was obtained;

[0035] pass The variance of spatiotemporal imagery at multiple scales was obtained;

[0036] pass The signal-to-noise ratio of spatiotemporal imagery at multiple scales was obtained;

[0037] Where Mean represents the average brightness of the spatiotemporal image at multiple scales; I(i,j) represents the gray value at position (i,j) of the spatiotemporal image at multiple scales; Var represents the variance of the spatiotemporal image at multiple scales; SNR represents the signal-to-noise ratio of the spatiotemporal image at multiple scales; M represents the length of the spatiotemporal image at multiple scales; and N represents the width of the spatiotemporal image at multiple scales.

[0038] pass Obtain the entropy of spatiotemporal image maps at multiple scales;

[0039] pass Obtain the energy of spatiotemporal image maps at multiple scales;

[0040] Where Entropy represents the entropy of the spatiotemporal image at multiple scales; Energy represents the energy of the spatiotemporal image at multiple scales; P(i,j) is the gray value at position (i,j) of the normalized gray-level co-occurrence matrix; M represents the length of the spatiotemporal image at multiple scales; and N represents the width of the spatiotemporal image at multiple scales.

[0041] The present invention also provides a surface flow velocity measurement device based on multi-scale spatiotemporal image fusion, comprising:

[0042] The acquisition module is used to acquire river channel videos of the target measurement points;

[0043] The processing module is used to process the river video to obtain a multi-scale projected image atlas; process the multi-scale projected image atlas to obtain a multi-scale spatiotemporal image map; process the multi-scale spatiotemporal image map to obtain the initial interpreted flow velocity and statistical feature parameters of the multi-scale spatiotemporal image map; input the statistical feature parameters and the multi-scale spatiotemporal image map into a measuring point flow velocity adaptive optimization model to obtain the multi-scale spatiotemporal image map flow velocity weights corresponding to the initial interpreted flow velocity of the multi-scale spatiotemporal image map; wherein, the measuring point flow velocity adaptive optimization model is trained according to a first preset network model; and the target surface flow velocity is obtained according to the initial interpreted flow velocity and the multi-scale spatiotemporal image map flow velocity weights.

[0044] The present invention also provides a computing device, comprising: a processor and a memory storing a computer program, wherein the computer program, when executed by the processor, performs the method described above.

[0045] The present invention also provides a computer-readable storage medium storing instructions that, when executed on a computer, cause the computer to perform the method described above.

[0046] The above-described solution of the present invention has at least the following beneficial effects:

[0047] The above-described solution of the present invention acquires river channel video at the target measurement point; processes the river channel video to obtain a multi-scale projection image atlas; processes the multi-scale projection image atlas to obtain multi-scale spatiotemporal image maps; processes the multi-scale spatiotemporal image maps to obtain the initial interpreted flow velocity and statistical characteristic parameters of the multi-scale spatiotemporal image maps; inputs the statistical characteristic parameters of the multi-scale spatiotemporal image maps into the measurement point flow velocity adaptive optimization model to obtain the multi-scale spatiotemporal image map flow velocity weights corresponding to the initial interpreted flow velocity of the multi-scale spatiotemporal image maps; and obtains the target surface flow velocity based on the initial interpreted flow velocity and the multi-scale spatiotemporal image map flow velocity weights. This solution overcomes the limitations of traditional spatiotemporal image methods with single projection resolution analysis, and improves the accuracy and stability of river surface flow velocity measurement. Attached Figure Description

[0048] Figure 1 This is a flowchart of a surface velocity measurement method based on multi-scale spatiotemporal image fusion provided by an embodiment of the present invention;

[0049] Figure 2 This is a flowchart of the surface flow velocity measurement process based on multi-scale spatiotemporal image fusion provided by an embodiment of the present invention;

[0050] Figure 3 This is a schematic diagram of the surface velocity measurement system based on multi-scale spatiotemporal image fusion provided in an embodiment of the present invention;

[0051] Figure 4 This is a flowchart of the processing procedure of the surface velocity measurement system based on multi-scale spatiotemporal image fusion provided in an embodiment of the present invention;

[0052] Figure 5 A block diagram of a surface velocity measurement device based on multi-scale spatiotemporal image fusion provided in an embodiment of the present invention. Detailed Implementation

[0053] Exemplary embodiments of the invention will now be described in more detail with reference to the accompanying drawings. While exemplary embodiments of the invention are shown in the drawings, it should be understood that the invention may be implemented in various forms and should not be limited to the embodiments set forth herein. Rather, these embodiments are provided so that this invention will be thorough and complete, and will fully convey the scope of the invention to those skilled in the art.

[0054] like Figure 1 As shown, an embodiment of the present invention proposes a surface flow velocity measurement method based on multi-scale spatiotemporal image fusion, comprising:

[0055] Step 11: Obtain river channel video at the target measurement point;

[0056] Step 12: Process the river video to obtain a multi-scale projection image atlas;

[0057] Step 13: Process the multiple scale projection image atlas to obtain multiple scale spatiotemporal image maps;

[0058] Step 14: Process the spatiotemporal image maps at multiple scales to obtain the initial interpretation flow rate and statistical characteristic parameters of the spatiotemporal image maps at multiple scales.

[0059] Step 15: Input the statistical feature parameters of the multiple scale spatiotemporal image maps and the multiple scale spatiotemporal image maps into the measurement point flow velocity adaptive optimization model to obtain the flow velocity weights of the multiple scale spatiotemporal image maps corresponding to the initial interpreted flow velocities of the multiple scale spatiotemporal image maps; wherein, the measurement point flow velocity adaptive optimization model is trained according to the first preset network model;

[0060] Step 16: Obtain the target surface flow velocity based on the initial interpretation flow velocity of the multiple scale spatiotemporal image maps and the flow velocity weights of the multiple scale spatiotemporal image maps.

[0061] In this embodiment, river channel video of the target measuring point is acquired; the river channel video is processed to obtain a multi-scale projection image atlas; the multi-scale projection image atlas is processed to obtain a multi-scale spatiotemporal image map; the multi-scale spatiotemporal image map is processed to obtain the initial interpreted flow velocity and statistical feature parameters of the multi-scale spatiotemporal image map; the statistical feature parameters of the multi-scale spatiotemporal image map are input into the measuring point flow velocity adaptive optimization model to obtain the multi-scale spatiotemporal image map flow velocity weights corresponding to the initial interpreted flow velocity of the multi-scale spatiotemporal image map; based on the initial interpreted flow velocity and the multi-scale spatiotemporal image map flow velocity weights, the target surface flow velocity is obtained; this method can overcome the limitations of traditional spatiotemporal image methods with single projection resolution analysis, and improve the accuracy and stability of river surface flow velocity measurement.

[0062] In this embodiment, when acquiring river video of the target measurement point, the measurement section and equipment can be determined: the measurement section should be located in a straight section of the river, away from the influence of backwater and lateral inflow areas; the video measurement equipment should be installed, and the placement of the video measurement equipment should take into account the influence of solar glare, and the video shooting direction should be reasonably determined by combining the latitude and longitude information of the measurement point and the key measurement period; at the same time, the video image should be able to completely cover the measurement section, and the measurement section should be located in the center of the image as much as possible to reduce measurement errors caused by image distortion;

[0063] Set up calibration points. In the area covered by the video image, set up at least 6 sets of calibration points. The calibration points should not be collinear and should be evenly distributed on both sides of the river. Record the position of the calibration points in the three-dimensional geospatial system.

[0064] In an optional embodiment of the present invention, step 12 includes:

[0065] Step 121: Perform projection conversion processing on the river video according to the preset calibration point information to obtain the projected video;

[0066] Step 122: Perform hierarchical downsampling on the projected video to obtain a multi-scale projection image atlas.

[0067] Further, step 121 includes:

[0068] Step 1211, through as well as The river video is subjected to projection conversion processing to obtain a projected video;

[0069] Among them, X c This indicates the X coordinate of the calibration point in the geographic spatial coordinate system; Y... c This indicates the Y-coordinate of the calibration point in the geographic spatial coordinate system; Z-coordinate... c This represents the Z-coordinate of the calibration point in the geographic spatial coordinate system; x i This indicates the x-coordinate of the calibration point on the river video; y-coordinate... i M represents the y-coordinate of the calibration point on the river video; 变 Denotes the projection transformation matrix; m 11 m 12 …m 34 This represents the parameters of the projection transformation matrix.

[0070] Further, step 122 includes:

[0071] Step 1221, through and The projected video is downsampled in stages to obtain a multi-scale projection image atlas;

[0072] Where (x′,y′) represents the reference image coordinate mapping; W(k) represents the bicubic kernel function, k represents the distance between the current pixel and the target interpolation position; a represents the parameter, a = -0.5, used to suppress aliasing; δ represents the resolution, in m / pixel; I δ (x,y) represents an image projected as δm / pixel; I base Indicates the initial image, I base (x,y) represents the reference projected image with a resolution of 0.01m / pixel; W(i) represents the interpolation weight in the x-direction; W(j) represents the interpolation weight in the y-direction.

[0073] In this embodiment, the image coordinates are projected to the geographic space coordinates by combining the position of the calibration point in the three-dimensional geospatial system;

[0074] Specifically, through as well as

[0075] The river video is subjected to projection conversion processing to obtain a projected video;

[0076] The initial projection resolution δ0 is set to 0.01m / pixel; based on the reference projection result (0.01m / pixel), the "single projection, hierarchical downsampling" mode is adopted to reduce the amount of computation, and projection images of 0.02m / pixel, 0.03m / pixel, 0.04m / pixel...0.10m / pixel are generated respectively.

[0077] Specifically, through and The projected video is downsampled in stages to obtain multiple scale projection image sets, totaling 10 scale projection image sets.

[0078] In an optional embodiment of the present invention, step 13 includes:

[0079] Step 131: Locate the position of the target measuring point on the multiple scale projection image atlas to obtain multiple target measuring point velocity lines;

[0080] Step 132: Extract the pixels on the velocity measurement lines of the multiple target measurement points frame by frame, and stack them on a time scale to obtain spatiotemporal image maps of multiple scales.

[0081] In this embodiment, for the multiple scale projection image atlases, pixels on the multiple target measurement point velocity measurement lines are extracted frame by frame based on the target measurement point velocity measurement lines, and then stacked on a time scale to obtain multiple scale spatiotemporal image maps.

[0082] In an optional embodiment of the present invention, step 14 includes:

[0083] Step 1411, through Domain transformation is performed on the spatiotemporal image maps at multiple scales to obtain spectral image maps at multiple scales;

[0084] Where F(u,v) represents a multi-scale spectral image; u,v represent frequency parameters; M represents the length of the multi-scale spatiotemporal image; N represents the width of the multi-scale spatiotemporal image; f(m,n) represents the pixel value of the multi-scale spatiotemporal image at (m,n); e represents the base of the natural logarithm; and j represents the imaginary number.

[0085] Step 1412: Perform spectrum centering on the multiple scale spectrum image maps to obtain multiple scale spectrum image processing maps;

[0086] Step 1413, in the polar coordinate system of the component within the multiple scale spectral image processing diagram, through The maximum energy direction of the spectrum of multiple scale spectral images is obtained; where α represents the angle in the polar coordinate system, with the counterclockwise direction as the positive direction, and the value range is [0,π]; R represents the frequency domain integration radius; F(rcosα,r sinα) represents the energy value located at (r cosα,r sinα);

[0087] Step 1414, through The main texture direction of the multi-scale spectral image processing map is transformed to obtain the optimal main texture direction of the multi-scale spectral image processing map; where θ represents the optimal main texture direction of the spatiotemporal image map measured from the positive x-axis clockwise direction;

[0088] Step 1415: Obtain the initial interpretation flow rate of multiple scale spatiotemporal image maps by using v1 = cotθ * dx * fps; where v1 represents the initial interpretation flow rate of multiple scale spatiotemporal image maps; θ represents the optimal main texture direction of multiple scale spectral image processing maps; dx represents the pixel resolution of multiple scale spatiotemporal image maps, in m / pixel; fps represents the frame rate of the river video, in frames / s.

[0089] In this embodiment, the multiple scale spatiotemporal image maps are transformed from the image domain to the frequency domain; specifically, through... Domain transformation is performed on the spatiotemporal image maps at multiple scales to obtain spectral image maps at multiple scales;

[0090] The spectrum image maps at multiple scales are spectral centered to move high-energy low-frequency information to the center of the spectrum, resulting in a multi-scale spectrum image processing map.

[0091] A polar coordinate system is constructed with the image center of the multi-scale spectral image processing images as the origin and half the length of the shorter side of each image as the radius. The pixel brightness values ​​at each angle in the Fourier spectra of the multi-scale spectral image processing images are summed to determine the direction of maximum energy in the Fourier spectra. Specifically, this is achieved through... The maximum energy direction of the spectrum in spectral images at multiple scales was obtained;

[0092] Since the direction of maximum energy in the Fourier frequency domain of the multi-scale spectral image processing map is perpendicular to the texture direction of the original spatiotemporal image map, the main texture direction of the spatiotemporal image map is defined as clockwise as positive. Therefore, the texture angle needs to be transformed accordingly; specifically, through... The main texture direction of the multi-scale spectral image processing map is transformed to obtain the optimal main texture direction of the multi-scale spectral image processing map;

[0093] The initial interpretation flow rate of spatiotemporal image maps at multiple scales is obtained by using v1 = cotθ * dx * fps.

[0094] In an optional embodiment of the present invention, step 14 further includes:

[0095] Step 1421, through The average brightness of spatiotemporal imagery at multiple scales was obtained;

[0096] Step 1422, through The variance of spatiotemporal imagery at multiple scales was obtained;

[0097] Step 1423, through The signal-to-noise ratio of spatiotemporal imagery at multiple scales was obtained;

[0098] Where Mean represents the average brightness of the spatiotemporal image at multiple scales; I(i,j) represents the gray value at position (i,j) of the spatiotemporal image at multiple scales; Var represents the variance of the spatiotemporal image at multiple scales; SNR represents the signal-to-noise ratio of the spatiotemporal image at multiple scales; M represents the length of the spatiotemporal image at multiple scales; and N represents the width of the spatiotemporal image at multiple scales.

[0099] Step 1424, through Obtain the entropy of spatiotemporal image maps at multiple scales;

[0100] Step 1425, through Obtain the energy of spatiotemporal image maps at multiple scales;

[0101] Where Entropy represents the entropy of the spatiotemporal image at multiple scales; Energy represents the energy of the spatiotemporal image at multiple scales; P(i,j) is the gray value at position (i,j) of the normalized gray-level co-occurrence matrix; M represents the length of the spatiotemporal image at multiple scales; and N represents the width of the spatiotemporal image at multiple scales.

[0102] In this embodiment, the statistical characteristic parameters of the spatiotemporal image maps at multiple scales include: the average brightness, variance, signal-to-noise ratio, entropy, and energy of the spatiotemporal image maps at multiple scales;

[0103] pass The average brightness of spatiotemporal imagery at multiple scales was obtained;

[0104] pass The variance of spatiotemporal imagery at multiple scales was obtained;

[0105] pass The signal-to-noise ratio of spatiotemporal imagery at multiple scales was obtained;

[0106] Construct a normalized gray-level co-occurrence matrix and calculate the entropy and energy of spatiotemporal imagery at multiple scales;

[0107] Specifically, through Obtain the entropy of spatiotemporal image maps at multiple scales;

[0108] pass The energy of spatiotemporal imagery at multiple scales is obtained.

[0109] In an optional embodiment of the present invention, step 15 includes:

[0110] Step 151: Input the multiple scale spatiotemporal image maps into the semantic feature extraction module of the first processing layer of the adaptive optimization model for measuring point flow velocity for processing to obtain the first processing result;

[0111] Step 152: Input the statistical feature parameters of the multiple scale spatiotemporal image maps into the equivalent expansion module of the first processing layer of the adaptive optimization model of the flow velocity at the measurement point for processing, and obtain the second processing result;

[0112] Step 153: Input the first processing result and the second processing result into the second processing layer of the adaptive optimization model of the flow velocity at the measurement point for processing, and obtain the output result of the second processing layer;

[0113] Step 154: Input the output result of the second processing layer into the third processing layer of the adaptive optimization model of the flow velocity at the measurement point for processing, and obtain the output result of the third processing layer;

[0114] Step 155: Input the output result of the third processing layer into the output layer of the adaptive optimization model of the measurement point flow velocity for processing, and obtain the flow velocity weights of the spatiotemporal image maps corresponding to the initial interpreted flow velocities of the spatiotemporal image maps at multiple scales.

[0115] In this embodiment, the multiple scale spatiotemporal image maps are input into the semantic feature extraction module of the first processing layer of the adaptive optimization model for measuring point flow velocity for processing to obtain a first processing result. Specifically, the multiple scale spatiotemporal image maps are input into the semantic feature extraction module of the first processing layer of the adaptive optimization model for measuring point flow velocity. There are 10 sets of scale spatiotemporal image maps. The 10 sets of scale spatiotemporal image maps are scaled to 256×256, and then center-cropped to a size of 224×224 to obtain an input matrix with a dimension of 10×224×224. Here, 10 represents 10 sets of spatiotemporal image maps with different resolutions; 224×224 is the image size after center cropping.

[0116] The semantic feature extraction module has a five-layer structure:

[0117] The first layer uses a convolutional layer with a kernel size of 7, a stride of 2, a padding of 3, and 32 channels; its output matrix has a dimension of 10×32×112×112.

[0118] The second layer uses a convolutional layer with a kernel size of 5, a stride of 2, a padding of 2, and 64 channels; its output matrix has a dimension of 10×64×56×56.

[0119] The third layer uses a convolutional layer with a kernel size of 3, a stride of 2, and a padding of 1; its output matrix has a dimension of 10×128×28×28.

[0120] The fourth layer uses a convolutional layer with a kernel size of 3, a stride of 2, and a padding of 1; its output matrix has a dimension of 10×256×14×14.

[0121] The fifth layer uses a convolutional layer with a kernel size of 3, a stride of 2, and a boundary padding of 1; its output matrix contains deep semantic features with dimensions of 10×512×7×7.

[0122] The statistical feature parameters of the multiple scale spatiotemporal image maps are input into the equivalent expansion module of the first processing layer of the adaptive optimization model for flow velocity at the measurement point for processing to obtain the second processing result. Specifically, the average brightness, variance, signal-to-noise ratio, entropy, and energy of the multiple scale spatiotemporal image maps are normalized to ensure that each parameter is in the range of 0 to 1. The normalization adopts the maximum-minimum method and is performed according to the data statistical feature type. The equivalent expansion method is used to map the feature data with a dimension of 10×5×1×1 (10 represents 10 sets of spatiotemporal image maps at different scales, 5 represents 5 feature parameters, and 1×1 represents the data size) into a matrix with a dimension of 10×5×7×7 (10 represents 10 sets of spatiotemporal image maps at different scales, 5 represents 5 feature parameters, and 7×7 is the size of the feature parameters after the equivalent mapping).

[0123] The first processing result and the second processing result are input into the second processing layer of the adaptive optimization model for measuring point flow velocity to obtain the output result of the second processing layer. Specifically, the second processing layer of the adaptive optimization model for measuring point flow velocity is a feature stitching module. The deep semantic features extracted by the semantic feature module (first processing result) and the data matrix obtained by the equivalent expansion module (second processing result) are stitched together to generate a stitching matrix with a dimension of 10×517×7×7. Here, 10 represents 10 sets of spatiotemporal image maps at different scales, 517 represents the stitching of 512-dimensional channel data and 5-dimensional feature parameters, and 7×7 is the size of the feature parameters.

[0124] The output of the second processing layer is input into the third processing layer of the adaptive optimization model for measuring point flow velocity to obtain the output of the third processing layer. Specifically, the third processing layer of the adaptive optimization model for measuring point flow velocity is a feature fusion module. Taking the output of the feature stitching module (the output of the second processing layer) as input, the stitching matrix is ​​first copied. Then, the copied stitching matrix is ​​globally pooled along the spatial scale, transforming the 10×517×7×7 stitching matrix into a 10×517×1 matrix. Then, it is further globally averaged pooled along the channel scale to obtain a 10×1 weight matrix. Finally, the original stitching matrix and the weight matrix are multiplied and added according to the weighted addition method to obtain the fused multi-scale spatiotemporal image comprehensive feature (the output of the third processing layer), whose output matrix dimension is 517×7×7.

[0125] The output of the third processing layer is input into the output layer of the adaptive optimization model for measuring point flow velocity to obtain the flow velocity weights of multiple scale spatiotemporal image maps corresponding to the initial interpreted flow velocities of multiple scale spatiotemporal image maps. Specifically, the output layer of the adaptive optimization model for measuring point flow velocity is a weight output module. The output of the feature fusion module (the output of the third processing layer) is used as input, and the matrix is ​​compressed to a size of 517×1 by performing spatial scale global average pooling on the matrix. The result is further mapped to a size of 10×1 through a fully connected layer and activated by the Softmax activation function (normalized exponential function) to ensure that its sum is 1. Its physical meaning is the flow velocity weights of multiple scale spatiotemporal image maps corresponding to the initial interpreted flow velocities of multiple scale spatiotemporal image maps.

[0126] The training process of the adaptive optimization model for the flow velocity at the measurement point:

[0127] Obtain historical river channel videos;

[0128] The historical river channel videos were processed to obtain a collection of historical projection images at multiple scales;

[0129] The historical projection image atlases at multiple scales are processed to obtain historical spatiotemporal image maps at multiple scales.

[0130] The historical spatiotemporal image maps at multiple scales are processed to obtain the initial interpretation flow rate and statistical characteristic parameters of the historical spatiotemporal image maps at multiple scales.

[0131] The dataset is divided into training, validation, and test sets based on the historical spatiotemporal image maps at multiple scales, the initial interpretation flow rate of the historical spatiotemporal image maps at multiple scales, and the statistical feature parameters of the historical spatiotemporal image maps at multiple scales. For data with fewer than 100 cumulative comparative data points, the dataset is divided in an 8:1:1 ratio; for data with more than 100 cumulative comparative data points, the dataset is divided in a 6:2:2 ratio.

[0132] The spatiotemporal images of the training at multiple scales are input into the semantic feature extraction module of the first processing layer of the adaptive optimization model for flow velocity at the measurement point for processing to obtain the first training processing result.

[0133] The statistical feature parameters of the multi-scale training spatiotemporal image map are input into the equivalent expansion module of the first processing layer of the measurement point flow velocity adaptive optimization model for processing, and a second training processing result is obtained.

[0134] The first training result and the second training result are input into the second processing layer of the adaptive optimization model of the flow velocity at the measurement point for processing to obtain the training output result of the second processing layer.

[0135] The training output of the second processing layer is input into the third processing layer of the adaptive optimization model of the flow velocity at the measurement point for processing, so as to obtain the training output of the third processing layer.

[0136] The training output of the third processing layer is input into the output layer of the adaptive optimization model of the measurement point flow velocity for processing, so as to obtain the flow velocity weights of the multi-scale training spatiotemporal image map corresponding to the initial interpreted flow velocity of the spatiotemporal image map at multiple scales.

[0137] During model training, the training set is used to train the model parameters; the validation set is used to verify the model's reliability; and the test set is used to verify the model's generalization ability.

[0138] The loss function is defined as:

[0139]

[0140] In the formula, v′ is the measured value at the i-th measuring point; i Let K be the model prediction value for the i-th measurement point; K represents the number of measurement points.

[0141] During training, model performance is evaluated using MAE and RMSE.

[0142]

[0143] In the formula, MAE represents the mean absolute error; RMSE represents the root mean square error; N 总M represents the total number of samples. 高 This represents the number of predefined Gaussian distribution functions; y i p represents the measurement value of the i-th sample; ij μ represents the weight of the j-th Gaussian distribution function calculated under the i-th sample condition; ij This represents the mean of the j-th Gaussian resolution function calculated under the condition of sample i.

[0144] In an optional embodiment of the present invention, step 16 includes:

[0145] Step 161, through Obtain the flow velocity at the target surface;

[0146] Among them, v out Indicates the flow velocity at the target surface; v i This represents the initial interpretation flow rate of spatiotemporal imagery at multiple scales. For a single measurement point, there are s spatiotemporal imagery at different scales, corresponding to s initial interpretation flow rates. In this embodiment, s = 10; k i The flow velocity weights represent multiple scales of spatiotemporal image maps. In this embodiment, there are a total of 10 different scales of spatiotemporal image map flow velocity weights.

[0147] In this embodiment, the target surface flow velocity is obtained based on the initial interpretation flow velocity of the multiple scale spatiotemporal image maps and the flow velocity weights of the multiple scale spatiotemporal image maps, which can significantly improve the detection accuracy and stability of the target surface flow velocity.

[0148] like Figure 2 As shown, the process of measuring the surface velocity of a river is as follows:

[0149] Step 1, Determine the measurement section and arrange equipment: Determine the measurement section, which should be located in a straight section of the river channel, away from the influence of backwater and lateral inflow areas; Install video measurement equipment, taking into account the influence of solar glare, and rationally determine the video shooting direction based on the latitude and longitude information of the measurement points and the key measurement period; At the same time, the video image should completely cover the measurement section, and the measurement section should be located in the center of the image as much as possible to reduce measurement errors caused by image distortion; Set up calibration points, with at least 6 sets of calibration points in the area covered by the video image; Calibration points should not be collinear and should be evenly distributed on both sides of the river channel; Record the position of the calibration points in the three-dimensional geographic space system;

[0150] Step 2: Obtain river channel video at the target measurement point;

[0151] Step 3: Process the river video to obtain a multi-scale projection image atlas;

[0152] Specific through as well as

[0153] The river video is subjected to projection transformation processing to obtain projected video; the river video image is then projected onto a geospatial coordinate system.

[0154] pass and The projected video is downsampled in stages to obtain a multi-scale projection image atlas;

[0155] Step 4: Process the multiple scale projection image atlas to obtain multiple scale spatiotemporal image maps;

[0156] Specifically, the positions of target measurement points are located on the multiple scale projection image atlases to obtain multiple target measurement point velocity lines; the pixels on the multiple target measurement point velocity lines are extracted frame by frame and stacked on a time scale to obtain multiple scale spatiotemporal image maps.

[0157] Step 5: Process the spatiotemporal image maps at multiple scales to obtain the initial interpretation flow rate and statistical characteristic parameters of the spatiotemporal image maps at multiple scales.

[0158] Specifically, through Domain transformation is performed on the multiple scale spatiotemporal image maps to obtain multiple scale spectral image maps; spectral centering is performed on the multiple scale spectral image maps to obtain multiple scale spectral image processing maps; a polar coordinate system is constructed within the multiple scale spectral image processing maps, through... The maximum energy direction of the spectrum in multiple scale spectral images was obtained; through The main texture direction of the multi-scale spectral image processing map is transformed to obtain the optimal main texture direction of the multi-scale spectral image processing map; the initial interpretation flow rate of the multi-scale spatiotemporal image map is obtained by using v1=cotθ*dx*fps;

[0159] Passed The average brightness of spatiotemporal imagery at multiple scales was obtained; through The variance of spatiotemporal imagery at multiple scales was obtained; through... The signal-to-noise ratio of spatiotemporal imagery at multiple scales was obtained; through... Obtain the entropy of spatiotemporal imagery at multiple scales; through Obtain the energy of spatiotemporal image maps at multiple scales;

[0160] Step 6: Input the statistical feature parameters of the multiple scale spatiotemporal image maps and the multiple scale spatiotemporal image maps into the adaptive optimization model of the flow velocity at the measurement point to obtain the flow velocity weights of the multiple scale spatiotemporal image maps corresponding to the initial interpreted flow velocities of the multiple scale spatiotemporal image maps.

[0161] Specifically, the spatiotemporal images at multiple scales are input into the semantic feature extraction module of the first processing layer of the adaptive optimization model for flow velocity at the measurement point for processing to obtain the first processing result;

[0162] The statistical feature parameters of the multiple scale spatiotemporal image maps are input into the equivalent expansion module of the first processing layer of the adaptive optimization model for flow velocity at the measurement point for processing, and a second processing result is obtained.

[0163] The first processing result and the second processing result are input into the second processing layer of the adaptive optimization model of the flow velocity at the measuring point for processing, and the output result of the second processing layer is obtained.

[0164] The output result of the second processing layer is input into the third processing layer of the adaptive optimization model of the flow velocity at the measurement point for processing, and the output result of the third processing layer is obtained.

[0165] The output results of the third processing layer are input into the output layer of the adaptive optimization model of the measuring point flow velocity for processing, so as to obtain the flow velocity weights of multiple scale spatiotemporal image maps corresponding to the initial interpreted flow velocities of multiple scale spatiotemporal image maps.

[0166] Step 7: Obtain the target surface flow velocity based on the initial interpretation flow velocity of the multiple scale spatiotemporal image maps and the flow velocity weights of the multiple scale spatiotemporal image maps;

[0167] Specifically, through The flow velocity at the target surface is obtained.

[0168] The above-mentioned river surface velocity measurement process is based on, for example, Figure 3 The system shown is a surface flow velocity measurement system based on multi-scale spatiotemporal image fusion.

[0169] like Figure 3 and Figure 4 As shown, the surface flow velocity measurement system based on multi-scale spatiotemporal image fusion includes:

[0170] Video acquisition module: used to acquire video of the river channel;

[0171] Measurement point determination module: used to specify the distribution of speed measurement points and the length of the speed measurement line;

[0172] Multi-scale projection transformation module: used for projection transformation of video images, projecting video images from the image coordinate system to a multi-scale geospatial coordinate system;

[0173] Multi-scale spatiotemporal image generation module: used to traverse multi-scale projection images and obtain spatiotemporal image maps of each measurement point in each projection image;

[0174] The spatiotemporal image map solving module based on Fast Fourier Transform is used to solve the main texture direction of spatiotemporal image maps at various scales based on the Fast Fourier Transform method, and to perform flow velocity interpretation to obtain the initial flow velocity interpretation results.

[0175] Spatiotemporal image statistical feature extraction module: used to calculate the statistical features of spatiotemporal image maps at various scales, including mean, variance, signal-to-noise ratio, entropy, and energy;

[0176] Deep semantic feature extraction module: used to extract potential deep semantic features from various spatiotemporal image images;

[0177] Equivalence expansion module: Used to process the statistical features obtained by the spatiotemporal image graph statistical feature extraction module, and make its data size consistent with the deep semantic features through equivalence mapping;

[0178] Feature concatenation module: used to combine deep semantic features extracted by the deep semantic feature extraction module with statistical features mapped by the equivalent expansion module;

[0179] Feature fusion module: used to achieve deep fusion of deep semantic features and statistical features;

[0180] Feature output module: used to convert each fused feature into reliability weights for spatiotemporal image interpretation results, and realize weighted output;

[0181] Velocity mapping module: This module combines the initial interpreted velocity output by the spatiotemporal image map solving module based on Fast Fourier Transform with the reliability weight output by the feature output module to output the final surface velocity of the target at the measurement point.

[0182] This invention addresses the measurement bottleneck caused by the mismatch of fixed resolution parameters in traditional spatiotemporal imagery methods due to the spatiotemporal heterogeneity of natural river flow patterns. It proposes an innovative solution based on multi-scale spatiotemporal image fusion. A multi-scale projection image atlas is constructed based on a dynamic resolution resampling module, and a gradient spatiotemporal image sequence is generated through a flow-adaptive resolution optimization mechanism. Furthermore, an adaptive optimization method for measurement point flow velocity in multi-scale spatiotemporal imagery is developed, establishing a measurement point flow velocity optimization model that integrates image features and statistical characteristic parameters from multi-scale spatiotemporal images, enabling intelligent inversion of surface velocity fields in natural field environments. This technology overcomes the limitations of single-projection resolution analysis in traditional spatiotemporal imagery methods, significantly improving the accuracy and stability of in-situ monitoring in the field.

[0183] like Figure 5 As shown, embodiments of the present invention also provide a surface flow velocity measurement device 50 based on multi-scale spatiotemporal image fusion, comprising:

[0184] Module 51 is used to acquire river channel video of the target measurement point;

[0185] Processing module 52 is used to process the river video to obtain a multi-scale projection image atlas; process the multi-scale projection image atlas to obtain a multi-scale spatiotemporal image map; process the multi-scale spatiotemporal image map to obtain the initial interpreted flow velocity and statistical feature parameters of the multi-scale spatiotemporal image map; input the statistical feature parameters and the multi-scale spatiotemporal image map into a measuring point flow velocity adaptive optimization model to obtain the multi-scale spatiotemporal image map flow velocity weights corresponding to the initial interpreted flow velocity of the multi-scale spatiotemporal image map; wherein, the measuring point flow velocity adaptive optimization model is trained according to a first preset network model; and the target surface flow velocity is obtained according to the initial interpreted flow velocity and the multi-scale spatiotemporal image map flow velocity weights.

[0186] Optionally, the river video information is processed to obtain a multi-scale projection image atlas, including:

[0187] The river video is projected and converted based on the preset calibration point information to obtain the projected video;

[0188] The projected video is downsampled in stages to obtain a multi-scale projection image atlas.

[0189] Optionally, the river video is projected and converted based on preset calibration point information to obtain a projected video, including:

[0190] pass as well as

[0191] The river video is subjected to projection conversion processing to obtain a projected video;

[0192] Among them, X c This indicates the X coordinate of the calibration point in the geographic spatial coordinate system; Y... c This indicates the Y-coordinate of the calibration point in the geographic spatial coordinate system; Z-coordinate... c This represents the Z-coordinate of the calibration point in the geographic spatial coordinate system; x i This indicates the x-coordinate of the calibration point on the river video; y-coordinate... i M represents the y-coordinate of the calibration point on the river video; 变 Denotes the projection transformation matrix; m 11 m 12 …m 34 This represents the parameters of the projection transformation matrix.

[0193] Optionally, the projected video is downsampled at different scales to obtain a multi-scale projected image atlas, including:

[0194] pass and The projected video is downsampled in stages to obtain a multi-scale projection image atlas;

[0195] Where (x′,y′) represents the reference image coordinate mapping; W(k) represents the bicubic kernel function, k represents the distance between the current pixel and the target interpolation position; a represents the parameter, a=-0.5; δ represents the resolution, in m / pixel; I δ (x,y) represents an image projected as δm / pixel; I base Indicates the initial image, I base (x,y) represents the reference projected image with a resolution of 0.01m / pixel; W(i) represents the interpolation weight in the x-direction; W(j) represents the interpolation weight in the y-direction.

[0196] Optionally, the multiple scale projected image atlases are processed to obtain multiple scale spatiotemporal image maps, including:

[0197] The positions of the target measuring points are located on the multiple scale projection image atlases to obtain multiple target measuring point velocity lines;

[0198] Pixels on the velocity measurement lines of the multiple target measurement points are extracted frame by frame and stacked over time to obtain spatiotemporal image maps of multiple scales.

[0199] Optionally, the multiple scale spatiotemporal image maps are processed to obtain the initial interpretation flow rate of the multiple scale spatiotemporal image maps and the statistical characteristic parameters of the multiple scale spatiotemporal image maps, including:

[0200] pass Domain transformation is performed on the spatiotemporal image maps at multiple scales to obtain spectral image maps at multiple scales;

[0201] Where F(u,v) represents a multi-scale spectral image; u,v represent frequency parameters; M represents the length of the multi-scale spatiotemporal image; N represents the width of the multi-scale spatiotemporal image; f(m,n) represents the pixel value of the multi-scale spatiotemporal image at (m,n); e represents the base of the natural logarithm; and j represents the imaginary number.

[0202] Spectral centering is performed on the multiple scale spectral image maps to obtain multiple scale spectral image processing maps;

[0203] In the polar coordinate system of the component within the multi-scale spectral image processing diagram, through The maximum energy direction of the spectrum of multiple scale spectral images is obtained; where α represents the angle in the polar coordinate system, with the counterclockwise direction as the positive direction, and the value range is [0,π]; R5 represents the frequency domain integration radius; F(r cosα,r sinα) represents the energy value located at (r cosα,r sinα);

[0204] pass The main texture direction of the multi-scale spectral image processing map is transformed to obtain the optimal main texture direction of the multi-scale spectral image processing map; where θ represents the optimal main texture direction of the spatiotemporal image map measured from the positive x-axis clockwise direction;

[0205] The initial interpretation flow rate of the spatiotemporal image map at multiple scales is obtained by using v1 = cotθ * dx * fps; where v1 represents the initial interpretation flow rate of the spatiotemporal image map at multiple scales; θ represents the optimal main texture direction of the spectral image processing map at multiple scales; dx represents the pixel resolution of the spatiotemporal image map at multiple scales, in m / pixel; and fps represents the frame rate of the river video, in frames / s.

[0206] Optionally, processing the multiple scale spatiotemporal image maps to obtain the initial interpretation flow rate of the multiple scale spatiotemporal image maps and the statistical characteristic parameters of the multiple scale spatiotemporal image maps further includes:

[0207] pass The average brightness of spatiotemporal imagery at multiple scales was obtained;

[0208] pass The variance of spatiotemporal imagery at multiple scales was obtained;

[0209] pass The signal-to-noise ratio of spatiotemporal imagery at multiple scales was obtained;

[0210] Where Mean represents the average brightness of the spatiotemporal image at multiple scales; I(i,j) represents the gray value at position (i,j) of the spatiotemporal image at multiple scales; Var represents the variance of the spatiotemporal image at multiple scales; SNR represents the signal-to-noise ratio of the spatiotemporal image at multiple scales; M represents the length of the spatiotemporal image at multiple scales; and N represents the width of the spatiotemporal image at multiple scales.

[0211] pass Obtain the entropy of spatiotemporal image maps at multiple scales;

[0212] pass Obtain the energy of spatiotemporal image maps at multiple scales;

[0213] Where Entropy represents the entropy of the spatiotemporal image at multiple scales; Energy represents the energy of the spatiotemporal image at multiple scales; P(i,j) is the gray value at position (i,j) of the normalized gray-level co-occurrence matrix; M represents the length of the spatiotemporal image at multiple scales; and N represents the width of the spatiotemporal image at multiple scales.

[0214] It should be noted that this device is the same as the method described above. All implementations in the above method embodiments are applicable to the embodiments of this device and can achieve the same technical effect.

[0215] An embodiment of the present invention also provides a computing device, including: a processor and a memory storing a computer program, wherein the computer program, when executed by the processor, performs the method described in the above embodiments. All implementations in the above method embodiments are applicable to this embodiment and can achieve the same technical effect.

[0216] In this embodiment of the invention, a computer-readable storage medium is also provided, storing instructions that, when executed on a computer, cause the computer to perform the method described in the above embodiments. All implementations of the methods described in the above embodiments are applicable to this embodiment and can achieve the same technical effect.

[0217] Those skilled in the art will recognize that the units and algorithm steps of the various examples described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementations should not be considered beyond the scope of this invention.

[0218] Those skilled in the art will understand that, for the sake of convenience and brevity, the specific working processes of the systems, devices, and units described above can be referred to the corresponding processes in the foregoing method embodiments, and will not be repeated here.

[0219] In the embodiments provided by this invention, it should be understood that the disclosed apparatus and methods can be implemented in other ways. For example, the apparatus embodiments described above are merely illustrative. For instance, the division of units is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be through some interfaces; the indirect coupling or communication connection between devices or units may be electrical, mechanical, or other forms.

[0220] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0221] In addition, the functional units in the various embodiments of the present invention can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit.

[0222] If the aforementioned functions are implemented as software functional units and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this invention, essentially, or the part that contributes to the prior art, or a portion of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of this invention. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, ROM, RAM, magnetic disks, or optical disks.

[0223] Furthermore, it should be noted that in the apparatus and method of the present invention, it is obvious that the components or steps can be decomposed and / or recombined. These decompositions and / or recombinations should be considered equivalent solutions of the present invention. Moreover, the steps performing the above-described series of processes can naturally be executed in the order described, but are not necessarily required to be executed in chronological order; some steps can be executed in parallel or independently of each other. Those skilled in the art will understand that all or any step or component of the method and apparatus of the present invention can be implemented in any computing device (including processors, storage media, etc.) or network of computing devices, in hardware, firmware, software, or a combination thereof. This is something that those skilled in the art can achieve by using their basic programming skills after reading the description of the present invention.

[0224] Therefore, the object of the present invention can also be achieved by running a program or a set of programs on any computing device. The computing device can be a known general-purpose device. Therefore, the object of the present invention can also be achieved simply by providing a program product containing program code implementing the method or apparatus. That is, such a program product also constitutes the present invention, and the storage medium storing such a program product also constitutes the present invention. Obviously, the storage medium can be any known storage medium or any storage medium developed in the future. It should also be noted that in the apparatus and method of the present invention, it is obvious that the components or steps can be decomposed and / or recombined. These decompositions and / or recombinations should be considered equivalent to the present invention. Furthermore, the steps performing the above series of processes can naturally be performed in the order described, but are not necessarily required to be performed in chronological order. Some steps can be performed in parallel or independently of each other.

[0225] The above description represents the preferred embodiments of the present invention. It should be noted that those skilled in the art can make various improvements and modifications without departing from the principles of the present invention, and these improvements and modifications should also be considered within the scope of protection of the present invention.

Claims

1. A surface velocity measurement method based on multi-scale spatiotemporal image fusion, characterized in that, include: Acquire river channel video at the target measurement point; The river video is processed to obtain a multi-scale projection image atlas; The multiple scale projection image atlases are processed to obtain multiple scale spatiotemporal image maps; The spatiotemporal image maps at multiple scales are processed to obtain the initial interpretation flow rate and statistical characteristic parameters of the spatiotemporal image maps at multiple scales. The statistical feature parameters of the multiple scale spatiotemporal image maps and the multiple scale spatiotemporal image maps are input into the measurement point flow velocity adaptive optimization model to obtain the flow velocity weights of the multiple scale spatiotemporal image maps corresponding to the initial interpreted flow velocities of the multiple scale spatiotemporal image maps; wherein, the measurement point flow velocity adaptive optimization model is trained according to the first preset network model; The target surface velocity is obtained based on the initial interpretation velocity of the multiple scale spatiotemporal image maps and the velocity weights of the multiple scale spatiotemporal image maps.

2. The surface flow velocity measurement method based on multi-scale spatiotemporal image fusion according to claim 1, characterized in that, The video information of the river channel is processed to obtain a multi-scale projection image atlas, including: The river video is projected and converted based on the preset calibration point information to obtain the projected video; The projected video is downsampled in stages to obtain a multi-scale projection image atlas.

3. The surface flow velocity measurement method based on multi-scale spatiotemporal image fusion according to claim 2, characterized in that, The river video is projected and converted based on preset calibration point information to obtain a projected video, including: pass as well as The river video is subjected to projection conversion processing to obtain a projected video; Among them, X c This indicates the X coordinate of the calibration point in the geographic spatial coordinate system; Y... c This indicates the Y-coordinate of the calibration point in the geographic spatial coordinate system; Z-coordinate... c This represents the Z-coordinate of the calibration point in the geographic spatial coordinate system; x i This indicates the x-coordinate of the calibration point on the river video; y-coordinate... i M represents the y-coordinate of the calibration point on the river video; 变 Denotes the projection transformation matrix; m 11 m 12 …m 34 This represents the parameters of the projection transformation matrix.

4. The surface flow velocity measurement method based on multi-scale spatiotemporal image fusion according to claim 2, characterized in that, The projected video is downsampled at different scales to obtain a multi-scale projected image atlas, including: pass and The projected video is downsampled in stages to obtain a multi-scale projection image atlas; Where (x′, y′) represents the reference image coordinate mapping; W(k) represents the bicubic kernel function, k represents the distance between the current pixel and the target interpolation position; a represents the parameter, a = -0.5; δ represents the resolution, in m / pixel; I δ (x,y) represents an image projected as δm / pixel; I base Indicates the initial image, I base (x,y) represents the reference projected image with a resolution of 0.01m / pixel; W(i) represents the interpolation weight in the x-direction; W(j) represents the interpolation weight in the y-direction.

5. The surface flow velocity measurement method based on multi-scale spatiotemporal image fusion according to claim 1, characterized in that, The multiple scale projection image atlases are processed to obtain multiple scale spatiotemporal image maps, including: The positions of the target measuring points are located on the multiple scale projection image atlases to obtain multiple target measuring point velocity lines; Pixels on the velocity measurement lines of the multiple target measurement points are extracted frame by frame and stacked over time to obtain spatiotemporal image maps of multiple scales.

6. The surface flow velocity measurement method based on multi-scale spatiotemporal image fusion according to claim 1, characterized in that, The multiple spatiotemporal image maps at various scales are processed to obtain the initial interpretation flow rate of the multiple spatiotemporal image maps, including: pass Domain transformation is performed on the spatiotemporal image maps at multiple scales to obtain spectral image maps at multiple scales; Where F(u,v) represents a multi-scale spectral image; u,v represent frequency parameters; M represents the length of the multi-scale spatiotemporal image; N represents the width of the multi-scale spatiotemporal image; f(m,n) represents the pixel value of the multi-scale spatiotemporal image at (m,n); e represents the base of the natural logarithm; and j represents the imaginary number. Spectral centering is performed on the multiple scale spectral image maps to obtain multiple scale spectral image processing maps; In the polar coordinate system of the component within the multi-scale spectral image processing diagram, through The maximum energy direction of the spectrum of multiple scale spectral images is obtained; where α represents the angle in the polar coordinate system, with the counterclockwise direction as the positive direction, and the value range is [0,π]; R represents the frequency domain integration radius; F(rcosα,rsinα) represents the energy value located at (rcosα,rsinα); pass The main texture direction of the multi-scale spectral image processing map is transformed to obtain the optimal main texture direction of the multi-scale spectral image processing map; where θ represents the optimal main texture direction of the spatiotemporal image map measured from the positive x-axis clockwise direction; The initial interpretation flow rate of the spatiotemporal image map at multiple scales is obtained by using v1 = cotθ * dx * fps; where v1 represents the initial interpretation flow rate of the spatiotemporal image map at multiple scales; θ represents the optimal main texture direction of the spectral image processing map at multiple scales; dx represents the pixel resolution of the spatiotemporal image map at multiple scales, in m / pixel; and fps represents the frame rate of the river video, in frames / s.

7. The surface flow velocity measurement method based on multi-scale spatiotemporal image fusion according to claim 1, characterized in that, The spatiotemporal imagery at multiple scales is processed to obtain statistical characteristic parameters of the spatiotemporal imagery at multiple scales, including: pass The average brightness of spatiotemporal imagery at multiple scales was obtained; pass The variance of spatiotemporal imagery at multiple scales was obtained; pass The signal-to-noise ratio of spatiotemporal imagery at multiple scales was obtained; Where Mean represents the average brightness of the spatiotemporal image at multiple scales; I(i,j) represents the gray value at position (i,j) of the spatiotemporal image at multiple scales; Var represents the variance of the spatiotemporal image at multiple scales; SNR represents the signal-to-noise ratio of the spatiotemporal image at multiple scales; M represents the length of the spatiotemporal image at multiple scales; and N represents the width of the spatiotemporal image at multiple scales. pass Obtain the entropy of spatiotemporal image maps at multiple scales; pass Obtain the energy of spatiotemporal image maps at multiple scales; Where Entropy represents the entropy of the spatiotemporal image at multiple scales; Energy represents the energy of the spatiotemporal image at multiple scales; P(i,j) is the gray value at position (i,j) of the normalized gray-level co-occurrence matrix; M represents the length of the spatiotemporal image at multiple scales; and N represents the width of the spatiotemporal image at multiple scales.

8. A surface velocity measurement device based on multi-scale spatiotemporal image fusion, characterized in that, include: The acquisition module is used to acquire river channel videos of the target measurement points; The processing module is used to process the river video to obtain a multi-scale projection image atlas; The multiple scale projection image atlases are processed to obtain multiple scale spatiotemporal image maps; The spatiotemporal image maps at multiple scales are processed to obtain the initial interpretation flow rate and statistical characteristic parameters of the spatiotemporal image maps at multiple scales. The statistical feature parameters of the multiple scale spatiotemporal image maps and the multiple scale spatiotemporal image maps are input into the measurement point flow velocity adaptive optimization model to obtain the flow velocity weights of the multiple scale spatiotemporal image maps corresponding to the initial interpreted flow velocities of the multiple scale spatiotemporal image maps; wherein, the measurement point flow velocity adaptive optimization model is trained according to the first preset network model; the target surface flow velocity is obtained according to the initial interpreted flow velocities of the multiple scale spatiotemporal image maps and the flow velocity weights of the multiple scale spatiotemporal image maps.

9. A computing device, characterized in that, include: A processor, a memory storing a computer program, wherein the computer program, when executed by the processor, performs the method as described in any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that, The system stores instructions that, when executed on a computer, cause the computer to perform the method as described in any one of claims 1 to 7.

Images