Surface flow velocity measurement method and device based on multi-scale space-time image fusion

Abstract

The invention provides a surface flow velocity measurement method and device based on multi-scale space-time image fusion. The method comprises the following steps: acquiring a river channel video of a target measurement point; processing the river channel video to obtain a multi-scale projection image set; processing the multi-scale projection image set to obtain a multi-scale space-time image; processing the plurality of scale space-time images to obtain initial interpretation flow rates of the plurality of scale space-time images and statistical characteristic parameters of the plurality of scale space-time images; inputting the plurality of statistical characteristic parameters and the plurality of scale space-time images into a measurement point flow velocity adaptive optimization model to obtain flow velocity weights of the plurality of scale space-time images; and obtaining the target surface flow velocity according to the initial interpretation flow velocity of the multi-scale space-time images and the flow velocity weights of the multi-scale space-time images. According to the scheme, the limitation of single projection resolution analysis of a traditional space-time image method can be solved, and the river surface flow velocity measurement accuracy and stability are improved.

Description

Technical Field

[0001] The present invention relates to the field of hydrological monitoring technology, and in particular to a surface flow velocity measurement method and device based on multi-scale spatiotemporal image fusion. Background Art

[0002] River flow velocity is one of the core basic parameters for hydrological monitoring, flood prevention and disaster reduction, and water conservancy project design. Its precise measurement has important scientific value and engineering significance for flood evolution prediction, water resource scheduling efficiency evaluation, and water ecology research.

[0003] Computer vision-based spatiotemporal image velocity measurement technology has achieved groundbreaking applications in hydrological monitoring in recent years. This technology uses aerial imagery to characterize the spatiotemporal coupled texture features generated by the motion of surface tracers. It then uses methods such as fast Fourier transforms to interpret the dominant texture directions in the spatiotemporal imagery, and then infers flow velocity based on the velocity-displacement relationship. Compared to traditional contact measurement methods, this innovative monocular vision sensing system enables non-contact, large-scale, and high-spatiotemporal resolution one-dimensional time-averaged surface flow field monitoring. This effectively avoids the potential interference with water flow caused by traditional contact measurement, significantly improving measurement accuracy and convenience.

[0004] However, traditional spatiotemporal imaging technology still faces technical bottlenecks that need to be addressed in natural river flow velocity monitoring. One of its shortcomings is the fixed projection resolution parameter setting mechanism, which is difficult to adapt to the strong spatiotemporal heterogeneity of water flow in natural river sections. In actual monitoring, the flow characteristics (such as rapids, slow flows, turbulence, etc.) at different measuring points often vary significantly. The traditional resolution parameter selection method that relies on manual experience is difficult to achieve optimal adaptation across the entire domain. When using high-resolution projection, due to the limited ability to capture sub-pixel motion features, the displacement continuity of the tracer along the velocity measurement normal direction is difficult to guarantee, resulting in the fragmentation of spatiotemporal texture features, a significant attenuation of the signal-to-noise ratio, and an increase in the error in the main texture direction angle solution. On the contrary, when the projection resolution is too low, the multi-target brightness superposition caused by the mixed pixel effect will significantly weaken the texture contrast, resulting in texture blurring of the spatiotemporal image, causing a systematic shift in the main direction angle, and seriously affecting the accuracy of the measurement results. Summary of the Invention

[0005] The present invention provides a surface flow velocity measurement method and device based on multi-scale spatiotemporal image fusion, which can overcome the limitations of single projection resolution analysis of traditional spatiotemporal imaging methods and improve the accuracy and stability of river surface flow velocity measurement.

[0006] In order to solve the above technical problems, the technical solutions of the present invention are as follows:

[0007] A surface flow velocity measurement method based on multi-scale spatiotemporal image fusion, comprising:

[0008] Get the river channel video of the target measuring point;

[0009] Processing the river channel video to obtain a plurality of scale projection image atlases;

[0010] Processing the plurality of scale projection image atlases to obtain a plurality of scale spatiotemporal image maps;

[0011] Processing the multiple-scale space-time image graphs to obtain initial interpreted flow rates of the multiple-scale space-time image graphs and statistical characteristic parameters of the multiple-scale space-time image graphs;

[0012] Inputting the statistical characteristic parameters of the multi-scale spatiotemporal image graphs and the multi-scale spatiotemporal image graphs into a measuring point flow velocity adaptive optimization model to obtain the flow velocity weights of the multi-scale spatiotemporal image graphs corresponding to the initial interpreted flow velocities of the multi-scale spatiotemporal image graphs; wherein the measuring point flow velocity adaptive optimization model is trained based on a first preset network model;

[0013] The target surface flow velocity is obtained according to the initial interpreted flow velocities of the multiple-scale space-time image maps and the flow velocity weights of the multiple-scale space-time image maps.

[0014] Optionally, the river channel video information is processed to obtain a multi-scale projection image atlas, including:

[0015] Performing projection conversion processing on the river channel video according to preset calibration point information to obtain a projection video;

[0016] The projection video is downsampled in stages to obtain projection image atlases of multiple scales.

[0017] Optionally, performing projection conversion processing on the river channel video according to preset calibration point information to obtain a projected video includes:

[0018] pass as well as Performing projection conversion processing on the river channel video to obtain a projection video;

[0019] Among them, X c Indicates the X coordinate of the calibration point in the geographic space coordinate system; Y c Indicates the Y coordinate of the calibration point in the geographic space coordinate system; Z c Indicates the Z coordinate of the calibration point in the geographic space coordinate system; x i Indicates the x coordinate of the calibration point on the river video; y i Indicates the y coordinate of the calibration point on the river video; M 变 Represents the projection transformation matrix; m 11 、m 12 …m 34 Represents the projection transformation matrix parameters.

[0020] Optionally, the projection video is downsampled hierarchically to obtain projection image atlases at multiple scales, including:

[0021] pass and Performing hierarchical downsampling on the projected video to obtain projection image atlases at multiple scales;

[0022] Where (x′, y′) represents the reference image coordinate mapping; W(k) represents the bicubic kernel function, k represents the distance between the current pixel and the target interpolation position; a represents the parameter, a = -0.5; δ represents the resolution, the unit is m / pixel; I δ (x,y) represents the image projected as δm / pixel; I base represents the initial image, I base (x,y) represents the reference projection image with a resolution of 0.01 m / pixel; W(i) represents the interpolation weight in the x-direction; and W(j) represents the interpolation weight in the y-direction.

[0023] Optionally, processing the multiple-scale projection image atlases to obtain multiple-scale spatiotemporal image maps includes:

[0024] Positioning target measuring points on the plurality of scale projection image atlases to obtain a plurality of target measuring point speed measurement lines;

[0025] Pixels on the speed measurement lines of the multiple target measurement points are extracted frame by frame, and stacked on a time scale to obtain multi-scale spatiotemporal image maps.

[0026] Optionally, processing the multiple-scale spatiotemporal image maps to obtain initial interpreted flow rates of the multiple-scale spatiotemporal image maps includes:

[0027] pass Performing domain conversion on the multiple-scale spatiotemporal image graphs to obtain multiple-scale spectrum image graphs;

[0028] Wherein, F(u,v) represents the multi-scale spectrum image graph; u,v represent the frequency parameters; M represents the length of the multi-scale space-time image graph; N represents the width of the multi-scale space-time image graph; f(m,n) represents the pixel value of the multi-scale space-time image graph at (m,n); e represents the base of the natural logarithm; j represents the imaginary number;

[0029] Performing spectrum centering on the multiple scale spectrum image graphs to obtain multiple scale spectrum image processing graphs;

[0030] In the multiple scale spectrum image processing diagram, a polar coordinate system is constructed by The maximum energy direction of the spectrum of multiple scale spectrum images is obtained; where α represents the angle in the polar coordinate system, with the counterclockwise direction as the positive direction and the value range being [0,π]; R represents the frequency domain integration radius; F(rcosα,r sinα) represents the energy value at (r cosα,r sinα);

[0031] pass The main texture directions of the multi-scale spectrum image processing images are converted to obtain the optimal main texture directions of the multi-scale spectrum image processing images; wherein θ represents the optimal main texture direction of the spatiotemporal image image measured clockwise from the positive x-axis;

[0032] The initial interpreted flow rate of the multi-scale spatiotemporal image map is obtained by v1=cotθ*dx*fps; where v1 represents the initial interpreted flow rate of the multi-scale spatiotemporal image map; θ represents the optimal main texture direction of the multi-scale spectral image processing map; dx represents the pixel resolution of the multi-scale spatiotemporal image map, in m / pixel; fps represents the frame rate of the river video, in frames / s.

[0033] Optionally, processing the multiple-scale spatiotemporal image graphs to obtain statistical characteristic parameters of the multiple-scale spatiotemporal image graphs includes:

[0034] pass Get the average brightness of spatiotemporal images at multiple scales;

[0035] pass Obtain the variance of spatiotemporal image maps at multiple scales;

[0036] pass Get the signal-to-noise ratio of spatiotemporal images at multiple scales;

[0037] Among them, Mean represents the average brightness of the multi-scale spatiotemporal image graph; I(i,j) represents the grayscale value at the (i,j) position of the multi-scale spatiotemporal image graph; Var represents the variance of the multi-scale spatiotemporal image graph; SNR represents the signal-to-noise ratio of the multi-scale spatiotemporal image graph; M represents the length of the multi-scale spatiotemporal image graph; N represents the width of the multi-scale spatiotemporal image graph;

[0038] pass Obtain the entropy of spatiotemporal image graphs at multiple scales;

[0039] pass Obtain the energy of spatiotemporal image maps at multiple scales;

[0040] Among them, Entropy represents the entropy of the multi-scale spatiotemporal image graph; Energy represents the energy of the multi-scale spatiotemporal image graph; P(i, j) is the grayscale value at the position (i, j) of the normalized gray-level co-occurrence matrix; M represents the length of the multi-scale spatiotemporal image graph; N represents the width of the multi-scale spatiotemporal image graph.

[0041] The present invention also provides a surface flow velocity measurement device based on multi-scale spatiotemporal image fusion, comprising:

[0042] Acquisition module, used to obtain river channel video of target measuring point;

[0043] A processing module is used to process the river channel video to obtain a multi-scale projection image atlas; process the multi-scale projection image atlas to obtain a multi-scale space-time image map; process the multi-scale space-time image map to obtain the initial interpreted flow velocity of the multi-scale space-time image map and the statistical characteristic parameters of the multi-scale space-time image map; input the statistical characteristic parameters of the multi-scale space-time image map and the multi-scale space-time image map into the measuring point flow velocity adaptive optimization model to obtain the multi-scale space-time image map flow velocity weights corresponding to the initial interpreted flow velocity of the multi-scale space-time image map; wherein the measuring point flow velocity adaptive optimization model is obtained by training according to the first preset network model; and the target surface flow velocity is obtained according to the initial interpreted flow velocity of the multi-scale space-time image map and the flow velocity weights of the multi-scale space-time image map.

[0044] The present invention further provides a computing device, comprising: a processor and a memory storing a computer program, wherein the computer program executes the method described above when executed by the processor.

[0045] The present invention also provides a computer-readable storage medium storing instructions, which, when executed on a computer, enable the computer to execute the method described above.

[0046] The above solution of the present invention includes at least the following beneficial effects:

[0047] The above-mentioned scheme of the present invention obtains the river channel video of the target measuring point; processes the river channel video to obtain a multi-scale projection image atlas; processes the multi-scale projection image atlas to obtain a multi-scale space-time image map; processes the multi-scale space-time image map to obtain the initial interpreted flow velocity of the multi-scale space-time image map and the statistical characteristic parameters of the multi-scale space-time image map; inputs the statistical characteristic parameters of the multi-scale space-time image map into the measuring point flow velocity adaptive optimization model to obtain the multi-scale space-time image map flow velocity weights corresponding to the initial interpreted flow velocity of the multi-scale space-time image map; obtains the target surface flow velocity according to the initial interpreted flow velocity of the multi-scale space-time image map and the multi-scale space-time image map flow velocity weights; it can solve the limitations of the single projection resolution analysis of the traditional space-time imaging method and improve the accuracy and stability of river surface flow velocity measurement. BRIEF DESCRIPTION OF THE DRAWINGS

[0048] Figure 1 is a flow chart of a surface flow velocity measurement method based on multi-scale spatiotemporal image fusion provided by an embodiment of the present invention;

[0049] Figure 2 is a flow chart of a surface flow velocity measurement process based on multi-scale spatiotemporal image fusion provided by an embodiment of the present invention;

[0050] Figure 3 1 is a schematic structural diagram of a surface flow velocity measurement system based on multi-scale spatiotemporal image fusion provided by an embodiment of the present invention;

[0051] Figure 4 is a flowchart of the processing process of a surface flow velocity measurement system based on multi-scale spatiotemporal image fusion provided by an embodiment of the present invention;

[0052] Figure 5 A module diagram of a surface flow velocity measurement device based on multi-scale spatiotemporal image fusion provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0053] Exemplary embodiments of the present invention will be described in more detail below with reference to the accompanying drawings. Although exemplary embodiments of the present invention are shown in the accompanying drawings, it should be understood that the present invention can be implemented in various forms and should not be limited by the embodiments set forth herein. Rather, these embodiments are provided to enable a more thorough understanding of the present invention and to fully convey the scope of the present invention to those skilled in the art.

[0054] like Figure 1 As shown, an embodiment of the present invention proposes a surface flow velocity measurement method based on multi-scale spatiotemporal image fusion, comprising:

[0055] Step 11, obtaining the river channel video of the target measuring point;

[0056] Step 12: Process the river video to obtain a multi-scale projection image atlas;

[0057] Step 13, processing the plurality of scale projection image atlases to obtain a plurality of scale spatiotemporal image maps;

[0058] Step 14: Processing the multi-scale space-time image graphs to obtain initial interpreted flow rates and statistical characteristic parameters of the multi-scale space-time image graphs;

[0059] Step 15: Inputting the statistical characteristic parameters of the multi-scale spatiotemporal image graphs and the multi-scale spatiotemporal image graphs into a measuring point flow velocity adaptive optimization model to obtain the multi-scale spatiotemporal image graph flow velocity weights corresponding to the initial interpreted flow velocities of the multi-scale spatiotemporal image graphs; wherein the measuring point flow velocity adaptive optimization model is trained based on the first preset network model;

[0060] Step 16: Obtain the target surface flow velocity according to the initial interpreted flow velocities of the multi-scale space-time image graphs and the flow velocity weights of the multi-scale space-time image graphs.

[0061] In this embodiment, a river channel video of a target measuring point is obtained; the river channel video is processed to obtain a multi-scale projection image atlas; the multi-scale projection image atlas is processed to obtain a multi-scale space-time image map; the multi-scale space-time image map is processed to obtain the initial interpreted flow velocity of the multi-scale space-time image map and the statistical characteristic parameters of the multi-scale space-time image map; the statistical characteristic parameters of the multi-scale space-time image map are input into the measuring point flow velocity adaptive optimization model to obtain the multi-scale space-time image map flow velocity weights corresponding to the initial interpreted flow velocity of the multi-scale space-time image map; the target surface flow velocity is obtained based on the initial interpreted flow velocity of the multi-scale space-time image map and the multi-scale space-time image map flow velocity weights; the limitation of the single projection resolution analysis of the traditional space-time imaging method is solved, and the accuracy and stability of the river surface flow velocity measurement are improved;

[0062] In this embodiment, when acquiring a river channel video of a target measuring point, the measuring section and equipment layout can be determined: the measuring section should be located in a straight area of ​​the river channel, away from backwater influences and lateral inflow areas; the video measuring equipment should be installed, and the layout of the video measuring equipment should take into account the influence of solar glare. The video shooting direction should be reasonably determined in combination with the latitude and longitude information of the measuring point and the key measurement period; at the same time, the video image should be able to completely cover the measuring section, and the measuring section should be located in the center of the image as much as possible to reduce measurement errors caused by image distortion;

[0063] Lay out calibration points. In the area covered by the video image, lay out at least 6 groups of calibration points. The calibration points cannot be collinear and should be evenly distributed on both sides of the river. Record the positions of the calibration points in the 3D geographic space system.

[0064] In an optional embodiment of the present invention, step 12 includes:

[0065] Step 121, performing projection conversion processing on the river channel video according to preset calibration point information to obtain a projection video;

[0066] Step 122 : performing hierarchical downsampling on the projection video to obtain projection image atlases of multiple scales.

[0067] Furthermore, step 121 includes:

[0068] Step 1211, pass as well as Performing projection conversion processing on the river channel video to obtain a projection video;

[0069] Among them, X c Indicates the X coordinate of the calibration point in the geographic space coordinate system; Y c Indicates the Y coordinate of the calibration point in the geographic space coordinate system; Z c Indicates the Z coordinate of the calibration point in the geographic space coordinate system; x i Indicates the x coordinate of the calibration point on the river video; y i Indicates the y coordinate of the calibration point on the river video; M 变 Represents the projection transformation matrix; m 11 、m 12 …m 34 Represents the projection transformation matrix parameters.

[0070] Furthermore, step 122 includes:

[0071] Step 1221, pass and Performing hierarchical downsampling on the projected video to obtain projection image atlases at multiple scales;

[0072] Where (x′, y′) represents the coordinate mapping of the reference image; W(k) represents the bicubic kernel function, k represents the distance between the current pixel and the target interpolation position; a represents a parameter, a = -0.5, which is used to suppress the aliasing effect; δ represents the resolution, in m / pixel; I δ (x,y) represents the image projected as δm / pixel; I base represents the initial image, I base (x,y) represents the reference projection image with a resolution of 0.01 m / pixel; W(i) represents the interpolation weight in the x-direction; and W(j) represents the interpolation weight in the y-direction.

[0073] In this embodiment, the image coordinates are projected into the geographic space coordinates using the inverse projection transformation in combination with the position of the calibration point in the three-dimensional geographic space system;

[0074] Specifically, through as well as Performing projection conversion processing on the river channel video to obtain a projection video;

[0075] The initial projection resolution δ0 is set to 0.01m / pixel. Based on the baseline projection result (0.01m / pixel), the "one-time projection, hierarchical downsampling" mode is adopted to reduce the amount of calculation, and projection images with resolutions of 0.02m / pixel, 0.03m / pixel, 0.04m / pixel, and so on are generated. 0.10m / pixel

[0076] Specifically, through and The projection video is downsampled hierarchically to obtain multiple scale projection image atlases, with a total of 10 scale projection image atlases.

[0077] In an optional embodiment of the present invention, step 13 includes:

[0078] Step 131, locating the position of the target measuring point on the plurality of scale projection image atlases to obtain a plurality of target measuring point speed measurement lines;

[0079] Step 132 : extracting pixels on the speed measurement lines of the multiple target measurement points frame by frame, and stacking them on a time scale to obtain multi-scale spatiotemporal image maps.

[0080] In this embodiment, for the multiple scale projection image atlases, pixels on the multiple target measurement point speed measurement lines are extracted frame by frame based on the target measurement point speed measurement lines, and stacked on a time scale to obtain multiple scale spatiotemporal image maps.

[0081] In an optional embodiment of the present invention, step 14 includes:

[0082] Step 1411, pass Performing domain conversion on the multiple-scale spatiotemporal image graphs to obtain multiple-scale spectrum image graphs;

[0083] Wherein, F(u,v) represents the multi-scale spectrum image graph; u,v represent the frequency parameters; M represents the length of the multi-scale space-time image graph; N represents the width of the multi-scale space-time image graph; f(m,n) represents the pixel value of the multi-scale space-time image graph at (m,n); e represents the base of the natural logarithm; j represents the imaginary number;

[0084] Step 1412: performing spectrum centering on the multiple scale spectrum image graphs to obtain multiple scale spectrum image processing graphs;

[0085] Step 1413: construct a polar coordinate system within the multiple scale spectrum image processing graphs by The maximum energy direction of the spectrum of multiple scale spectrum images is obtained; where α represents the angle in the polar coordinate system, with the counterclockwise direction as the positive direction and the value range being [0,π]; R represents the frequency domain integration radius; F(rcosα,r sinα) represents the energy value at (r cosα,r sinα);

[0086] Step 1414, pass The main texture directions of the multi-scale spectrum image processing images are converted to obtain the optimal main texture directions of the multi-scale spectrum image processing images; wherein θ represents the optimal main texture direction of the spatiotemporal image image measured clockwise from the positive x-axis;

[0087] Step 1415, obtain the initial interpreted flow rate of the multi-scale spatiotemporal image map through v1=cotθ*dx*fps; wherein v1 represents the initial interpreted flow rate of the multi-scale spatiotemporal image map; θ represents the optimal main texture direction of the multi-scale spectral image processing map; dx represents the pixel resolution of the multi-scale spatiotemporal image map, the unit is m / pixel; fps represents the frame rate of the river video, the unit is frame / s.

[0088] In this embodiment, the multiple scale spatiotemporal image graphs are transformed from the image domain to the frequency domain; specifically, by Performing domain conversion on the multiple-scale spatiotemporal image graphs to obtain multiple-scale spectrum image graphs;

[0089] Performing spectrum centering on the multiple scale spectrum image graphs, moving high-energy low-frequency information to the spectrum center, and obtaining multiple scale spectrum image processing graphs;

[0090] A polar coordinate system is constructed with the center of the image of the multiple scale spectrum image processing images as the origin and half the length of the short side of the image of the multiple scale spectrum image processing images as the radius; the pixel brightness value of each angle in the Fourier spectrum of the multiple scale spectrum image processing images is added to determine the maximum energy direction of the Fourier spectrum of the multiple scale spectrum image processing images; specifically, by Obtain the maximum energy direction of the spectrum of multiple scale spectrum images;

[0091] Since the maximum energy direction in the Fourier frequency domain of the multi-scale spectrum image processing image is perpendicular to the texture direction of the original spatiotemporal image, the main texture direction of the spatiotemporal image is defined as clockwise. Therefore, the texture angle needs to be converted accordingly; specifically, by Converting the main texture directions of the multi-scale spectrum image processing images to obtain the optimal main texture directions of the multi-scale spectrum image processing images;

[0092] The initial interpretation flow rate of the multi-scale spatiotemporal image map is obtained by v1=cotθ*dx*fps.

[0093] In an optional embodiment of the present invention, step 14 further includes:

[0094] Step 1421, pass Get the average brightness of spatiotemporal images at multiple scales;

[0095] Step 1422, pass Obtain the variance of spatiotemporal image maps at multiple scales;

[0096] Step 1423, pass Get the signal-to-noise ratio of spatiotemporal images at multiple scales;

[0097] Among them, Mean represents the average brightness of the multi-scale spatiotemporal image graph; I(i,j) represents the grayscale value at the (i,j) position of the multi-scale spatiotemporal image graph; Var represents the variance of the multi-scale spatiotemporal image graph; SNR represents the signal-to-noise ratio of the multi-scale spatiotemporal image graph; M represents the length of the multi-scale spatiotemporal image graph; N represents the width of the multi-scale spatiotemporal image graph;

[0098] Step 1424, pass Obtain the entropy of spatiotemporal image graphs at multiple scales;

[0099] Step 1425, pass Obtain the energy of spatiotemporal image maps at multiple scales;

[0100] Among them, Entropy represents the entropy of the multi-scale spatiotemporal image graph; Energy represents the energy of the multi-scale spatiotemporal image graph; P(i, j) is the grayscale value at the position (i, j) of the normalized gray-level co-occurrence matrix; M represents the length of the multi-scale spatiotemporal image graph; N represents the width of the multi-scale spatiotemporal image graph.

[0101] In this embodiment, the statistical characteristic parameters of the multiple-scale spatiotemporal image graphs include: average brightness, variance, signal-to-noise ratio, entropy, and energy of the multiple-scale spatiotemporal image graphs;

[0102] pass Get the average brightness of spatiotemporal images at multiple scales;

[0103] pass Obtain the variance of spatiotemporal image maps at multiple scales;

[0104] pass Get the signal-to-noise ratio of spatiotemporal images at multiple scales;

[0105] Construct a normalized gray-level co-occurrence matrix and calculate the entropy and energy of spatiotemporal image graphs at multiple scales;

[0106] Specifically, through Obtain the entropy of spatiotemporal image graphs at multiple scales;

[0107] pass The energy of spatiotemporal image maps at multiple scales is obtained.

[0108] In an optional embodiment of the present invention, step 15 includes:

[0109] Step 151: input the multi-scale spatiotemporal image graphs into the semantic feature extraction module of the first processing layer of the measuring point flow velocity adaptive optimization model for processing to obtain a first processing result;

[0110] Step 152: Inputting the statistical characteristic parameters of the multi-scale spatiotemporal image graphs into the equivalent expansion module of the first processing layer of the measuring point velocity adaptive optimization model for processing to obtain a second processing result;

[0111] Step 153: input the first processing result and the second processing result into the second processing layer of the measuring point flow velocity adaptive optimization model for processing to obtain a second processing layer output result;

[0112] Step 154: input the output result of the second processing layer into the third processing layer of the measuring point flow velocity adaptive optimization model for processing to obtain the output result of the third processing layer;

[0113] Step 155 : Input the output result of the third processing layer into the output layer of the measuring point velocity adaptive optimization model for processing to obtain velocity weights of multiple scale spatiotemporal image maps corresponding to the initial interpreted velocity of the multiple scale spatiotemporal image maps.

[0114] In this embodiment, the multiple scaled spatiotemporal image maps are input into the semantic feature extraction module of the first processing layer of the measuring point flow velocity adaptive optimization model for processing to obtain a first processing result; specifically, the multiple scaled spatiotemporal image maps are input into the semantic feature extraction module of the first processing layer of the measuring point flow velocity adaptive optimization model, wherein there are 10 groups of scaled spatiotemporal image maps, the 10 groups of scaled spatiotemporal image maps are scaled to 256×256, and then centrally cropped to a size of 224×224, to obtain a matrix with an input dimension of 10×224×224; wherein 10 represents 10 groups of spatiotemporal image maps with different resolutions; and 224×224 is the image size after central cropping;

[0115] The semantic feature extraction module has a five-layer structure:

[0116] The first layer uses a convolutional layer with a kernel size of 7, a stride of 2, a border padding of 3, and a channel number of 32; its output matrix dimension is 10×32×112×112;

[0117] The second layer uses a convolutional layer with a kernel size of 5, a stride of 2, a border padding of 2, and 64 channels; its input and output matrix dimensions are 10×64×56×56;

[0118] The third layer uses a convolutional layer with a kernel size of 3, a stride of 2, and a border padding of 1; its input and output matrix dimensions are 10×128×28×28;

[0119] The fourth layer uses a convolutional layer with a kernel size of 3, a stride of 2, and a border padding of 1; its input and output matrix dimensions are 10×256×14×14;

[0120] The fifth layer uses a convolutional layer with a kernel size of 3, a stride of 2, and a border padding of 1; its output matrix dimension is 10×512×7×7 deep semantic features;

[0121] The statistical characteristic parameters of the multiple-scale spatiotemporal image maps are input into the equivalent expansion module of the first processing layer of the measuring point flow velocity adaptive optimization model for processing to obtain a second processing result; specifically, the average brightness of the multiple-scale spatiotemporal image maps, the variance of the multiple-scale spatiotemporal image maps, the signal-to-noise ratio of the multiple-scale spatiotemporal image maps, the entropy of the multiple-scale spatiotemporal image maps and the energy of the multiple-scale spatiotemporal image maps are normalized to ensure that each parameter is in the range of 0 to 1; wherein the normalization adopts the maximum and minimum value method and is normalized according to the statistical feature type of the data; using the equivalent expansion method, the feature data with a dimension size of 10×5×1×1 (10 represents 10 groups of spatiotemporal image maps of different scales, 5 represents 5 feature parameters, and 1×1 represents data size) is mapped into a matrix with a dimension size of 10×5×7×7 (10 represents 10 groups of spatiotemporal image maps of different scales, 5 represents 5 feature parameters, and 7×7 is the size of the feature parameters after equivalent mapping);

[0122] The first processing result and the second processing result are input into the second processing layer of the measuring point flow velocity adaptive optimization model for processing to obtain the output result of the second processing layer; specifically, the second processing layer of the measuring point flow velocity adaptive optimization model is a feature splicing module; a channel splicing method is used to splice the deep semantic features extracted by the semantic feature module (the first processing result) and the data matrix expanded by the equivalent expansion module (the second processing result) to generate a splicing matrix with a dimension size of 10×517×7×7; wherein 10 represents 10 groups of spatiotemporal image maps of different scales, 517 represents the splicing of 512-dimensional channel data and 5-dimensional feature parameters, and 7×7 is the size of the feature parameters;

[0123] The output result of the second processing layer is input into the third processing layer of the measuring point flow velocity adaptive optimization model for processing to obtain the output result of the third processing layer; specifically, the third processing layer of the measuring point flow velocity adaptive optimization model is a feature fusion module; the output of the feature splicing module (the output result of the second processing layer) is used as input, and the splicing matrix is ​​first copied; then, the copied splicing matrix is ​​globally pooled along the spatial scale, and the 10×517×7×7 splicing matrix is ​​transformed into a 10×517×1 matrix; then, global average pooling is further performed along the channel scale to obtain a 10×1 weight matrix; finally, the original splicing matrix and the weight matrix are multiplied and added according to the weighted addition method to obtain the fused multi-scale spatiotemporal image comprehensive features (the output result of the third processing layer), and the output matrix dimension size is 517×7×7;

[0124] The output result of the third processing layer is input into the output layer of the measuring point flow velocity adaptive optimization model for processing to obtain the flow velocity weights of the multiple-scale spatiotemporal image graphs corresponding to the initial interpreted flow velocities of the multiple-scale spatiotemporal image graphs; specifically, the output layer of the measuring point flow velocity adaptive optimization model is a weight output module; the output of the feature fusion module (the output result of the third processing layer) is used as input, and the matrix is ​​compressed to a size of 517×1 by performing spatial scale global average pooling on the matrix; the result is further mapped to a size of 10×1 through a fully connected layer, and activated by a Softmax activation function (normalized exponential function) to ensure that the cumulative sum is 1, which physically means the flow velocity weights of the multiple-scale spatiotemporal image graphs corresponding to the initial interpreted flow velocities of the multiple-scale spatiotemporal image graphs;

[0125] The training process of the measuring point flow velocity adaptive optimization model:

[0126] Get historical river videos;

[0127] Processing the historical river channel video to obtain a historical projection image atlas at multiple scales;

[0128] Processing the multiple-scale historical projection image atlas to obtain multiple-scale historical spatiotemporal image maps;

[0129] Processing the multiple-scale historical space-time image graphs to obtain initial interpretation flow rates of the multiple-scale historical space-time image graphs and statistical characteristic parameters of the multiple-scale historical space-time image graphs;

[0130] The multiple-scale historical space-time image maps, the initial interpreted flow rates of the multiple-scale historical space-time image maps, and the statistical characteristic parameters of the multiple-scale historical space-time image maps are divided into a data set, a validation set, and a test set. For data with less than 100 cumulative comparative measurement data points, the data set is divided in a ratio of 8:1:1; for data with more than 100 cumulative comparative measurement data points, the data set is divided in a ratio of 6:2:2;

[0131] Inputting the multiple scale training spatiotemporal image graphs into the semantic feature extraction module of the first processing layer of the measuring point flow velocity adaptive optimization model for processing to obtain a first training processing result;

[0132] Inputting the statistical characteristic parameters of the multiple scale training spatiotemporal image graphs into the equivalent expansion module of the first processing layer of the measuring point flow velocity adaptive optimization model for processing to obtain a second training processing result;

[0133] Inputting the first training processing result and the second training processing result into the second processing layer of the measuring point flow velocity adaptive optimization model for processing to obtain a second processing layer training output result;

[0134] Inputting the output result of the second processing layer training into the third processing layer of the measuring point flow velocity adaptive optimization model for processing to obtain the output result of the third processing layer training;

[0135] Inputting the training output result of the third processing layer into the output layer of the measuring point flow velocity adaptive optimization model for processing, thereby obtaining flow velocity weights of multiple scale training spatiotemporal image graphs corresponding to the initial interpreted flow velocities of the multiple scale spatiotemporal image graphs;

[0136] During the model training process, the training set is used to train the model parameters; the validation set is used to test the model reliability; and the test set is used to verify the model generalization.

[0137] The loss function is defined as:

[0138]

[0139] Where, is the measured value of the i-th measuring point; v′ i is the model prediction value of the i-th measuring point; K represents the number of measuring points;

[0140] During the training process, the model performance was evaluated by MAE and RMSE;

[0141]

[0142] Where, MAE represents mean absolute error; RMSE represents root mean square error; N 总Indicates the total number of samples, M 高 Indicates the number of preset Gaussian distribution functions; y i represents the measurement value of the i-th sample; p ij represents the weight of the j-th Gaussian distribution function calculated under the i-th sample condition; μ ij represents the mean of the j-th Gaussian resolution function calculated under the condition of sample i.

[0143] In an optional embodiment of the present invention, step 16 includes:

[0144] Step 161, pass Get the target surface velocity;

[0145] Among them, v out Indicates the target surface velocity; v i k represents the initial interpretation flow rate of multiple scales of spatiotemporal image maps. For a single measuring point, there are s spatiotemporal image maps of different scales, corresponding to s initial interpretation flow rates. In this embodiment, s = 10; k i Indicates the flow rate weights of multiple-scale spatiotemporal image graphs. In this embodiment, there are a total of 10 flow rate weights of spatiotemporal image graphs of different scales.

[0146] In this embodiment, the target surface flow velocity is obtained based on the initial interpreted flow velocities of the multiple-scale spatiotemporal image maps and the flow velocity weights of the multiple-scale spatiotemporal image maps, which can significantly improve the detection accuracy and stability of the target surface flow velocity.

[0147] like Figure 2 As shown in Figure 2, the process of measuring river surface velocity is as follows:

[0148] Step 1: Determine the survey section and arrange equipment: Determine the survey section. The survey section should be located in a straight area of ​​the river channel, away from backwater influences and lateral inflow areas. Install video measurement equipment. The layout of the video measurement equipment should take into account the influence of solar glare. The video shooting direction should be reasonably determined in combination with the latitude and longitude information of the measuring points and the key measurement period. At the same time, the video image should be able to completely cover the survey section and the survey section should be located in the center of the image as much as possible to reduce measurement errors caused by image distortion. Arrange calibration points. In the area covered by the video image, at least 6 groups of calibration points should be arranged. The calibration points cannot be collinear and should be evenly distributed on both sides of the river channel. Record the position of the calibration points in the 3D geographic space system.

[0149] Step 2: Obtain the river channel video of the target measuring point;

[0150] Step 3: Process the river video to obtain a multi-scale projection image atlas;

[0151] Specific through as well as Performing projection conversion processing on the river channel video to obtain a projection video; projecting the river channel video image into a geographic space coordinate system;

[0152] pass and Performing hierarchical downsampling on the projected video to obtain projection image atlases at multiple scales;

[0153] Step 4: Processing the multiple scale projection image atlases to obtain multiple scale spatiotemporal image maps;

[0154] Specifically, the target measuring point is located on the multi-scale projection image atlas to obtain a plurality of target measuring point speed measurement lines; pixels on the plurality of target measuring point speed measurement lines are extracted frame by frame, and stacked on a time scale to obtain a multi-scale spatiotemporal image map;

[0155] Step 5: Processing the multi-scale space-time image graphs to obtain initial interpreted flow rates and statistical characteristic parameters of the multi-scale space-time image graphs;

[0156] Specifically, through Perform domain conversion on the multiple scale spatiotemporal image graphs to obtain multiple scale spectrum image graphs; perform spectrum centering on the multiple scale spectrum image graphs to obtain multiple scale spectrum image processing graphs; construct a polar coordinate system within the multiple scale spectrum image processing graphs, by Get the maximum energy direction of the spectrum of multiple scale spectrum images; The main texture directions of the multi-scale spectrum image processing images are converted to obtain the optimal main texture directions of the multi-scale spectrum image processing images; the initial interpretation flow rate of the multi-scale space-time image images is obtained by v1=cotθ*dx*fps;

[0157] Get through Get the average brightness of multiple-scale spatiotemporal image maps; Get the variance of spatiotemporal image maps at multiple scales; Get the signal-to-noise ratio of multiple-scale spatiotemporal image maps; Get the entropy of multiple-scale spatiotemporal image graphs; Obtain the energy of spatiotemporal image maps at multiple scales;

[0158] Step 6: Inputting the statistical characteristic parameters of the multi-scale space-time image graphs and the multi-scale space-time image graphs into the measuring point flow velocity adaptive optimization model to obtain the multi-scale space-time image graph flow velocity weights corresponding to the initial interpreted flow velocities of the multi-scale space-time image graphs;

[0159] Specifically, the multiple-scale spatiotemporal image graphs are input into a semantic feature extraction module of a first processing layer of the measuring point flow velocity adaptive optimization model for processing to obtain a first processing result;

[0160] Inputting the statistical characteristic parameters of the multiple-scale spatiotemporal image graphs into the equivalent expansion module of the first processing layer of the measuring point velocity adaptive optimization model for processing to obtain a second processing result;

[0161] Inputting the first processing result and the second processing result into the second processing layer of the measuring point flow velocity adaptive optimization model for processing to obtain a second processing layer output result;

[0162] Inputting the output result of the second processing layer into the third processing layer of the measuring point flow velocity adaptive optimization model for processing to obtain the output result of the third processing layer;

[0163] Inputting the output result of the third processing layer into the output layer of the measuring point flow velocity adaptive optimization model for processing, thereby obtaining flow velocity weights of multiple scale space-time image maps corresponding to the initial interpreted flow velocities of the multiple scale space-time image maps;

[0164] Step 7: Obtain the target surface flow velocity based on the initial interpreted flow velocity of the multi-scale space-time image graph and the flow velocity weights of the multi-scale space-time image graph;

[0165] Specifically, through Get the target surface velocity.

[0166] The above-mentioned river surface velocity measurement process is based on the following Figure 3 The surface velocity measurement system based on multi-scale spatiotemporal image fusion is shown;

[0167] like Figure 3 and Figure 4 As shown in FIG, the surface velocity measurement system based on multi-scale spatiotemporal image fusion includes:

[0168] Video acquisition module: used to obtain river videos;

[0169] Measuring point determination module: used to specify the distribution of speed measuring points and the length of the speed measuring line;

[0170] Multi-scale projection transformation module: used for projection transformation of video images, projecting video images from the image coordinate system to the multi-scale geographic space coordinate system;

[0171] Multi-scale spatiotemporal image map generation module: used to traverse multi-scale projection images and obtain the spatiotemporal image map of each measurement point in each projection image;

[0172] Fast Fourier transform-based spatiotemporal image solving module: used to solve the main texture direction of spatiotemporal image maps of various scales based on the fast Fourier transform method, and perform flow velocity interpretation to obtain the initial flow velocity interpretation results;

[0173] Spatiotemporal image statistical feature extraction module: used to calculate the statistical features of spatiotemporal image maps at various scales, including mean, variance, signal-to-noise ratio, entropy and energy;

[0174] Deep semantic feature extraction module: used to extract the potential deep semantic features of each spatiotemporal image;

[0175] Equivalue expansion module: used to process the statistical features obtained by the spatiotemporal image statistical feature extraction module, and make its data size consistent with the deep semantic features through equivalue mapping;

[0176] Feature splicing module: used to combine the deep semantic features extracted by the deep semantic feature extraction module and the statistical features mapped by the equivalent expansion module;

[0177] Feature fusion module: used to achieve deep fusion of deep semantic features and statistical features;

[0178] Feature output module: used to convert each fusion feature into the reliability weight of the spatiotemporal image interpretation result to achieve weighted output;

[0179] Flow velocity mapping module: It is used to combine the initial interpreted flow velocity output by the spatiotemporal image solving module based on fast Fourier transform and the reliability weight output by the feature output module to output the final surface flow velocity of the target measuring point.

[0180] This invention addresses the measurement bottleneck problem of mismatch of fixed resolution parameters of traditional spatiotemporal imaging methods caused by spatiotemporal heterogeneity of natural river flow patterns, and proposes an innovative solution based on multi-scale spatiotemporal image fusion. A multi-scale projection image atlas is constructed based on a dynamic resolution resampling module, and a gradient spatiotemporal image sequence is generated through a resolution optimization mechanism that is adaptive to flow patterns. Then, a method for adaptive optimization of flow velocity at measuring points of multi-scale spatiotemporal image maps is developed, and a flow velocity optimization model for measuring points that integrates image features and statistical characteristic parameters of multi-scale spatiotemporal images is established to achieve intelligent inversion of surface velocity fields in natural outdoor environments. This technology breaks through the limitations of single projection resolution analysis of traditional spatiotemporal imaging methods, and significantly improves the accuracy and stability of in-situ monitoring in the field.

[0181] like Figure 5 As shown, an embodiment of the present invention further provides a surface flow velocity measurement device 50 based on multi-scale spatiotemporal image fusion, comprising:

[0182] An acquisition module 51 is used to acquire a river channel video of a target measuring point;

[0183] The processing module 52 is used to process the river channel video to obtain a multi-scale projection image atlas; process the multi-scale projection image atlas to obtain a multi-scale space-time image map; process the multi-scale space-time image map to obtain the initial interpreted flow velocity of the multi-scale space-time image map and the statistical characteristic parameters of the multi-scale space-time image map; input the statistical characteristic parameters of the multi-scale space-time image map and the multi-scale space-time image map into the measuring point flow velocity adaptive optimization model to obtain the multi-scale space-time image map flow velocity weights corresponding to the initial interpreted flow velocity of the multi-scale space-time image map; wherein the measuring point flow velocity adaptive optimization model is obtained by training according to the first preset network model; and the target surface flow velocity is obtained according to the initial interpreted flow velocity of the multi-scale space-time image map and the flow velocity weights of the multi-scale space-time image map.

[0184] Optionally, the river channel video information is processed to obtain a multi-scale projection image atlas, including:

[0185] Performing projection conversion processing on the river channel video according to preset calibration point information to obtain a projection video;

[0186] The projection video is downsampled in stages to obtain projection image atlases of multiple scales.

[0187] Optionally, performing projection conversion processing on the river channel video according to preset calibration point information to obtain a projected video includes:

[0188] pass as well as Performing projection conversion processing on the river channel video to obtain a projection video;

[0189] Among them, X c Indicates the X coordinate of the calibration point in the geographic space coordinate system; Y c Indicates the Y coordinate of the calibration point in the geographic space coordinate system; Z c Indicates the Z coordinate of the calibration point in the geographic space coordinate system; x i Indicates the x coordinate of the calibration point on the river video; y i Indicates the y coordinate of the calibration point on the river video; M 变 Represents the projection transformation matrix; m 11 、m 12 …m 34 Represents the projection transformation matrix parameters.

[0190] Optionally, the projection video is downsampled hierarchically to obtain projection image atlases at multiple scales, including:

[0191] pass and Performing hierarchical downsampling on the projected video to obtain projection image atlases at multiple scales;

[0192] Where (x′, y′) represents the reference image coordinate mapping; W(k) represents the bicubic kernel function, k represents the distance between the current pixel and the target interpolation position; a represents the parameter, a = -0.5; δ represents the resolution, the unit is m / pixel; I δ (x,y) represents the image projected as δm / pixel; I base represents the initial image, I base (x,y) represents the reference projection image with a resolution of 0.01 m / pixel; W(i) represents the interpolation weight in the x-direction; and W(j) represents the interpolation weight in the y-direction.

[0193] Optionally, processing the multiple-scale projection image atlases to obtain multiple-scale spatiotemporal image maps includes:

[0194] Positioning target measuring points on the plurality of scale projection image atlases to obtain a plurality of target measuring point speed measurement lines;

[0195] Pixels on the speed measurement lines of the multiple target measurement points are extracted frame by frame, and stacked on a time scale to obtain multi-scale spatiotemporal image maps.

[0196] Optionally, processing the multiple-scale spatiotemporal image maps to obtain initial interpreted flow rates and statistical characteristic parameters of the multiple-scale spatiotemporal image maps includes:

[0197] pass Performing domain conversion on the multiple-scale spatiotemporal image graphs to obtain multiple-scale spectrum image graphs;

[0198] Wherein, F(u,v) represents the multi-scale spectrum image graph; u,v represent the frequency parameters; M represents the length of the multi-scale space-time image graph; N represents the width of the multi-scale space-time image graph; f(m,n) represents the pixel value of the multi-scale space-time image graph at (m,n); e represents the base of the natural logarithm; j represents the imaginary number;

[0199] Performing spectrum centering on the multiple scale spectrum image graphs to obtain multiple scale spectrum image processing graphs;

[0200] In the multiple scale spectrum image processing diagram, a polar coordinate system is constructed by The maximum energy direction of the spectrum of multiple scale spectrum images is obtained; where α represents the angle in the polar coordinate system, with the counterclockwise direction as the positive direction and the value range being [0,π]; R5 represents the frequency domain integration radius; F(rcosα,r sinα) represents the energy value at (r cosα,r sinα);

[0201] pass The main texture directions of the multi-scale spectrum image processing images are converted to obtain the optimal main texture directions of the multi-scale spectrum image processing images; wherein θ represents the optimal main texture direction of the spatiotemporal image image measured clockwise from the positive x-axis;

[0202] The initial interpreted flow rate of the multi-scale spatiotemporal image map is obtained by v1=cotθ*dx*fps; where v1 represents the initial interpreted flow rate of the multi-scale spatiotemporal image map; θ represents the optimal main texture direction of the multi-scale spectral image processing map; dx represents the pixel resolution of the multi-scale spatiotemporal image map, in m / pixel; fps represents the frame rate of the river video, in frames / s.

[0203] Optionally, processing the multiple-scale space-time image maps to obtain initial interpreted flow rates and statistical characteristic parameters of the multiple-scale space-time image maps further includes:

[0204] pass Get the average brightness of spatiotemporal images at multiple scales;

[0205] pass Obtain the variance of spatiotemporal image maps at multiple scales;

[0206] pass Get the signal-to-noise ratio of spatiotemporal images at multiple scales;

[0207] Among them, Mean represents the average brightness of the multi-scale spatiotemporal image graph; I(i,j) represents the grayscale value at the (i,j) position of the multi-scale spatiotemporal image graph; Var represents the variance of the multi-scale spatiotemporal image graph; SNR represents the signal-to-noise ratio of the multi-scale spatiotemporal image graph; M represents the length of the multi-scale spatiotemporal image graph; N represents the width of the multi-scale spatiotemporal image graph;

[0208] pass Obtain the entropy of spatiotemporal image graphs at multiple scales;

[0209] pass Obtain the energy of spatiotemporal image maps at multiple scales;

[0210] Among them, Entropy represents the entropy of the multi-scale spatiotemporal image graph; Energy represents the energy of the multi-scale spatiotemporal image graph; P(i, j) is the grayscale value at the position (i, j) of the normalized gray-level co-occurrence matrix; M represents the length of the multi-scale spatiotemporal image graph; N represents the width of the multi-scale spatiotemporal image graph.

[0211] It should be noted that the device is a device corresponding to the above method, and all implementation methods in the above method embodiments are applicable to the embodiments of the device and can achieve the same technical effects.

[0212] An embodiment of the present invention further provides a computing device comprising: a processor and a memory storing a computer program, wherein the computer program, when executed by the processor, performs the method described in the above embodiment. All implementations of the above method embodiments are applicable to this embodiment and can achieve the same technical effects.

[0213] The present invention also provides a computer-readable storage medium storing instructions that, when executed on a computer, cause the computer to execute the method described in the above embodiment. All implementations of the above method embodiments are applicable to this embodiment and can achieve the same technical effects.

[0214] Those skilled in the art will appreciate that the units and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professionals and technicians can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of the present invention.

[0215] Those skilled in the art will clearly understand that, for the convenience and brevity of description, the specific working processes of the systems, devices and units described above can refer to the corresponding processes in the aforementioned method embodiments and will not be repeated here.

[0216] In the embodiments provided by the present invention, it should be understood that the disclosed devices and methods can be implemented in other ways. For example, the device embodiments described above are merely illustrative. For example, the division of the units is merely a logical function division. In actual implementation, there may be other division methods, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. In addition, the mutual coupling or direct coupling or communication connection shown or discussed can be through some interface, indirect coupling or communication connection of devices or units, which can be electrical, mechanical or other forms.

[0217] The units described as separate components may or may not be physically separate, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed across multiple network units. Some or all of these units may be selected to achieve the purpose of this embodiment according to actual needs.

[0218] In addition, each functional unit in each embodiment of the present invention may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit.

[0219] If the functions are implemented in the form of software functional units and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, or the part that contributes to the prior art, or part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions for enabling a computer device (which can be a personal computer, server, or network device, etc.) to execute all or part of the steps of the method described in each embodiment of the present invention. The aforementioned storage medium includes various media that can store program code, such as a USB flash drive, a mobile hard drive, ROM, RAM, a magnetic disk, or an optical disk.

[0220] In addition, it should be noted that, in the apparatus and method of the present invention, it is obvious that each component or each step can be decomposed and / or recombined. These decompositions and / or recombinations should be regarded as equivalent schemes of the present invention. Moreover, the steps of performing the above-mentioned series of processing can naturally be performed in chronological order according to the order of description, but it is not necessary to perform them in chronological order, and some steps can be performed in parallel or independently of each other. For those of ordinary skill in the art, it will be understood that all or any steps or components of the method and apparatus of the present invention can be implemented in any computing device (including processors, storage media, etc.) or a network of computing devices in hardware, firmware, software or a combination thereof, which can be achieved by those of ordinary skill in the art using their basic programming skills after reading the description of the present invention.

[0221] Therefore, the purpose of the present invention can also be achieved by running a program or a group of programs on any computing device. The computing device can be a well-known general-purpose device. Therefore, the purpose of the present invention can also be achieved simply by providing a program product containing program code that implements the method or device. That is to say, such a program product also constitutes the present invention, and the storage medium storing such a program product also constitutes the present invention. Obviously, the storage medium can be any well-known storage medium or any storage medium developed in the future. It should also be pointed out that in the device and method of the present invention, it is obvious that each component or each step can be decomposed and / or recombined. These decompositions and / or recombinations should be regarded as equivalent schemes of the present invention. In addition, the steps of performing the above-mentioned series of processing can naturally be performed in chronological order according to the order of description, but do not necessarily need to be performed in chronological order. Certain steps can be performed in parallel or independently of each other.

[0222] The above is a preferred embodiment of the present invention. It should be pointed out that for ordinary technicians in this technical field, several improvements and modifications can be made without departing from the principles of the present invention. These improvements and modifications should also be regarded as within the scope of protection of the present invention.

Claims

1. A surface flow velocity measurement method based on multi-scale spatiotemporal image fusion, characterized in that: include: Get the river channel video of the target measuring point; Processing the river channel video to obtain a plurality of scale projection image atlases; Processing the plurality of scale projection image atlases to obtain a plurality of scale spatiotemporal image maps; Processing the multiple-scale space-time image graphs to obtain initial interpreted flow rates of the multiple-scale space-time image graphs and statistical characteristic parameters of the multiple-scale space-time image graphs; Inputting the statistical characteristic parameters of the multi-scale spatiotemporal image graphs and the multi-scale spatiotemporal image graphs into a measuring point flow velocity adaptive optimization model to obtain the flow velocity weights of the multi-scale spatiotemporal image graphs corresponding to the initial interpreted flow velocities of the multi-scale spatiotemporal image graphs; wherein the measuring point flow velocity adaptive optimization model is trained based on a first preset network model; The target surface flow velocity is obtained according to the initial interpreted flow velocities of the multiple-scale space-time image maps and the flow velocity weights of the multiple-scale space-time image maps.

2. The surface flow velocity measurement method based on multi-scale spatiotemporal image fusion according to claim 1 is characterized in that: The river channel video information is processed to obtain a multi-scale projection image atlas, including: Performing projection conversion processing on the river channel video according to preset calibration point information to obtain a projection video; The projection video is downsampled in stages to obtain projection image atlases of multiple scales.

3. The surface flow velocity measurement method based on multi-scale spatiotemporal image fusion according to claim 2 is characterized in that: The river channel video is projected and converted according to the preset calibration point information to obtain a projection video, including: pass as well as Performing projection conversion processing on the river channel video to obtain a projection video; Among them, X c Indicates the X coordinate of the calibration point in the geographic space coordinate system; Y c Indicates the Y coordinate of the calibration point in the geographic space coordinate system; Z c Indicates the Z coordinate of the calibration point in the geographic space coordinate system; x i Indicates the x coordinate of the calibration point on the river video; y i Indicates the y coordinate of the calibration point on the river video; M 变 Represents the projection transformation matrix; m 11 、m 12 …m 34 Represents the projection transformation matrix parameters.

4. The surface flow velocity measurement method based on multi-scale spatiotemporal image fusion according to claim 2 is characterized in that: The projection video is downsampled in stages to obtain projection image atlases of multiple scales, including: pass and Performing hierarchical downsampling on the projected video to obtain projection image atlases at multiple scales; Where (x′, y′) represents the reference image coordinate mapping; W(k) represents the bicubic kernel function, k represents the distance between the current pixel and the target interpolation position; a represents the parameter, a = -0.5; δ represents the resolution, the unit is m / pixel; I δ (x,y) represents the image projected as δm / pixel; I base represents the initial image, I base (x,y) represents the reference projection image with a resolution of 0.01 m / pixel; W(i) represents the interpolation weight in the x-direction; and W(j) represents the interpolation weight in the y-direction.

5. The surface flow velocity measurement method based on multi-scale spatiotemporal image fusion according to claim 1 is characterized in that: Processing the multiple scale projection image atlases to obtain multiple scale spatiotemporal image maps, including: Positioning target measuring points on the plurality of scale projection image atlases to obtain a plurality of target measuring point speed measurement lines; Pixels on the speed measurement lines of the multiple target measurement points are extracted frame by frame, and stacked on a time scale to obtain multi-scale spatiotemporal image maps.

6. The surface flow velocity measurement method based on multi-scale spatiotemporal image fusion according to claim 1 is characterized in that: Processing the multiple-scale spatiotemporal image maps to obtain initial interpreted flow rates of the multiple-scale spatiotemporal image maps includes: pass Performing domain conversion on the multiple-scale spatiotemporal image graphs to obtain multiple-scale spectrum image graphs; Wherein, F(u,v) represents the multi-scale spectrum image graph; u,v represent the frequency parameters; M represents the length of the multi-scale space-time image graph; N represents the width of the multi-scale space-time image graph; f(m,n) represents the pixel value of the multi-scale space-time image graph at (m,n); e represents the base of the natural logarithm; j represents the imaginary number; Performing spectrum centering on the multiple scale spectrum image graphs to obtain multiple scale spectrum image processing graphs; In the multiple scale spectrum image processing diagram, a polar coordinate system is constructed by The maximum energy direction of the spectrum of multiple scale spectrum images is obtained; where α represents the angle in the polar coordinate system, with the counterclockwise direction as the positive direction and the value range being [0,π]; R represents the frequency domain integration radius; F(rcosα,rsinα) represents the energy value at (rcosα,rsinα); pass The main texture directions of the multi-scale spectrum image processing images are converted to obtain the optimal main texture directions of the multi-scale spectrum image processing images; wherein θ represents the optimal main texture direction of the spatiotemporal image image measured clockwise from the positive x-axis; The initial interpreted flow rate of the multi-scale spatiotemporal image map is obtained by v1=cotθ*dx*fps; where v1 represents the initial interpreted flow rate of the multi-scale spatiotemporal image map; θ represents the optimal main texture direction of the multi-scale spectral image processing map; dx represents the pixel resolution of the multi-scale spatiotemporal image map, in m / pixel; fps represents the frame rate of the river video, in frames / s.

7. The surface flow velocity measurement method based on multi-scale spatiotemporal image fusion according to claim 1 is characterized in that: Processing the multiple-scale spatiotemporal image graphs to obtain statistical characteristic parameters of the multiple-scale spatiotemporal image graphs includes: pass Get the average brightness of spatiotemporal images at multiple scales; pass Obtain the variance of spatiotemporal image maps at multiple scales; pass Get the signal-to-noise ratio of spatiotemporal images at multiple scales; Among them, Mean represents the average brightness of the multi-scale spatiotemporal image graph; I(i,j) represents the grayscale value at the (i,j) position of the multi-scale spatiotemporal image graph; Var represents the variance of the multi-scale spatiotemporal image graph; SNR represents the signal-to-noise ratio of the multi-scale spatiotemporal image graph; M represents the length of the multi-scale spatiotemporal image graph; N represents the width of the multi-scale spatiotemporal image graph; pass Obtain the entropy of spatiotemporal image graphs at multiple scales; pass Obtain the energy of spatiotemporal image maps at multiple scales; Among them, Entropy represents the entropy of the multi-scale spatiotemporal image graph; Energy represents the energy of the multi-scale spatiotemporal image graph; P(i, j) is the grayscale value at the position (i, j) of the normalized gray-level co-occurrence matrix; M represents the length of the multi-scale spatiotemporal image graph; N represents the width of the multi-scale spatiotemporal image graph.

8. A surface flow velocity measurement device based on multi-scale spatiotemporal image fusion, characterized in that: include: Acquisition module, used to obtain river channel video of target measuring point; A processing module, configured to process the river channel video to obtain a plurality of scale projection image atlases; Processing the plurality of scale projection image atlases to obtain a plurality of scale spatiotemporal image maps; Processing the multiple-scale space-time image graphs to obtain initial interpreted flow rates of the multiple-scale space-time image graphs and statistical characteristic parameters of the multiple-scale space-time image graphs; The statistical characteristic parameters of the multiple-scale space-time image maps and the multiple-scale space-time image maps are input into the measuring point flow rate adaptive optimization model to obtain the multiple-scale space-time image map flow rate weights corresponding to the initial interpreted flow rates of the multiple-scale space-time image maps; wherein, the measuring point flow rate adaptive optimization model is obtained by training according to the first preset network model; according to the initial interpreted flow rates of the multiple-scale space-time image maps and the multiple-scale space-time image map flow rate weights, the target surface flow rate is obtained.

9. A computing device, characterized in that include: A processor and a memory storing a computer program, wherein when the computer program is executed by the processor, the method according to any one of claims 1 to 7 is performed.

10. A computer-readable storage medium, characterized in that: The device stores instructions, which, when executed on a computer, enable the computer to execute the method according to any one of claims 1 to 7.

Images