Video flow measurement method, device, storage medium and equipment based on improved recursive full-field transform optical flow model

Abstract

The present invention discloses a video flow measurement method, device, storage medium and equipment based on an improved recursive full-field transformation optical flow model, belonging to the field of hydrological flow measurement technology. The method comprises: obtaining a river video and performing frame cutting processing to obtain video frames; calculating a transformation matrix based on the river video and pre-acquired survey results, transforming the on-map coordinates of each speed measurement point in each video frame in the river video into world coordinates through the transformation matrix, and obtaining the cross-sectional area between each speed measurement point; inputting the video frame into a trained improved recursive full-field transformation optical flow model, and outputting the pixel displacement between two adjacent frames of each speed measurement point; calculating the river surface flow velocity according to the pixel displacement, the world coordinates and the cross-sectional area; the present invention introduces a CBAM attention mechanism and deformable convolution, which can better detect complex river changes and subtle water surface movements, thereby improving the accuracy of the algorithm.

Description

Technical Field

[0001] The present invention relates to a video flow measurement method, device, storage medium and equipment based on an improved recursive full-field transformation optical flow model, and belongs to the technical field of hydrological flow measurement. Background Art

[0002] my country boasts abundant water resources and complex river systems. Floods and waterlogging disasters are common in southern China, where rainfall is plentiful, posing a threat to national and public security. Consequently, local governments have launched large-scale water conservancy projects focused on flood control and water supply, ensuring pre-disaster warnings and monitoring river flow rates. Traditional measurement methods, including flowmeters, radar, and acoustics, require personnel to travel back and forth to the site for measurements, which lacks real-time performance, impairs disaster warning capabilities, and poses a risk to personnel safety during inclement weather. Currently, deep learning technology has been widely applied in the field of computer vision, enabling the development of non-contact surface flow velocity measurement methods based on deep learning and images. For example, patent publication number CN106156734A proposes a water velocity measurement method based on convolutional neural network image recognition. This method classifies and trains images of rivers at different speeds to generate a model. During measurement, the image with the highest probability of matching the measured image with the model is identified, and the corresponding velocity is used as the flow velocity. However, this method suffers from limited accuracy and has certain limitations. The invention patent with patent publication number CN112149597A proposes a river surface flow velocity detection method based on deep learning. It uses an improved image classification method to improve measurement accuracy and speed, but it still has certain limitations when the surface flow velocity changes complexly.

[0003] Deep learning methods using image classification can improve the convenience and safety of flow measurement, but when measuring turbulent waters, they are easily disturbed by water splashes, and complex river surface changes can easily lead to large measurement errors. Summary of the Invention

[0004] The purpose of the present invention is to provide a video flow measurement method, device, storage medium and equipment based on an improved recursive full field transform optical flow model to solve the problem of large errors in the prior art.

[0005] To achieve the above objectives, the present invention is implemented by adopting the following technical solutions:

[0006] In a first aspect, the present invention provides a video flow measurement method based on an improved recursive full-field transform optical flow model, comprising:

[0007] Obtain river video and perform frame cutting to obtain video frames;

[0008] Calculating a transformation matrix based on the river video and pre-acquired survey results, transforming the on-map coordinates of each speed measurement point in each video frame in the river video into world coordinates using the transformation matrix, and obtaining the cross-sectional area between each speed measurement point;

[0009] Input the video frames into the trained improved recursive full-field transform optical flow model, and output the pixel displacement between two adjacent frames of each speed measurement point;

[0010] The river surface velocity is calculated based on the pixel displacement, the world coordinates and the cross-sectional area.

[0011] In combination with the first aspect, further, calculating the transformation matrix based on the river video and the pre-acquired survey results includes:

[0012] The coordinates of the calibration points on the map are obtained from the river video, and the world coordinates of the calibration points are obtained from the survey results. According to the coordinates of the calibration points on the map and the world coordinates, the transformation matrix is calculated through the projection transformation theory.

[0013] In combination with the first aspect, further, transforming the coordinates of each speed measurement point in each video frame in the river video into world coordinates through a transformation matrix includes:

[0014] Based on the projection transformation theory, the coordinates of each speed measurement point in each video frame in the river video are transformed into world coordinates through the transformation matrix.

[0015] In combination with the first aspect, further, the improved recursive full-field transform optical flow model includes:

[0016] The feature extractor operates on two adjacent video frames and obtains a feature map through convolution and deformable convolution operations. The resolution of the feature map is 1 / 8 of the input video frame.

[0017] a 4D correlation pyramid module, configured to calculate the complete correlation between all pairs of feature vectors of feature maps of two adjacent frames, perform a pooling operation on the correlation to obtain a 4D correlation pyramid, and use a local pixel grid to index the correlation in the 4D correlation pyramid to generate a first correlation;

[0018] A CBAM attention mechanism module, configured to convert the first correlation quantity and the optical flow of the previous iteration into a second correlation quantity;

[0019] an iterative update module, configured to perform a gated loop on the feature information of the first frame, the optical flow of the previous iteration, the second correlation quantity, and the hidden state, and output an updated optical flow;

[0020] The upsampling module is used to upsample the updated optical flow after the iteration and restore the updated optical flow to full resolution.

[0021] In combination with the first aspect, further, the CBAM attention mechanism module includes a channel attention mechanism module and a spatial attention module;

[0022] In the channel attention mechanism module, the input consisting of the first correlation quantity and the optical flow of the previous iteration is first subjected to average pooling and maximum pooling to obtain two different spatial context feature information. Then, through a shared multi-layer perceptron, sum operation and Sigmoid function activation, a channel attention feature map is generated. The calculation formula of the output of the channel attention mechanism module is as follows:

[0023] ;

[0024] in, is the output of the channel attention mechanism module, is the channel attention feature map, is the input, Represents the element-by-element multiplication symbol;

[0025] In the spatial attention mechanism module, the output of the channel attention mechanism module is first subjected to average pooling and maximum pooling to obtain two different feature maps and spliced. Then, it undergoes a 7×7 convolution, the number of channels is reduced to 1, and the sigmoid function is activated to generate a spatial attention feature map. The calculation formula for the output of the spatial attention mechanism module is as follows:

[0026] ;

[0027] in, is the output of the spatial attention mechanism module, is the spatial attention feature map.

[0028] In combination with the first aspect, further, the improved recursive full-field transform optical flow model is trained by the following method:

[0029] The sample size and number of iterations are set, the Sintel dataset is used as the training dataset, and the trained improved recursive full-field transform optical flow model is obtained through deep learning network training.

[0030] In combination with the first aspect, further, the river surface velocity is calculated according to the pixel displacement, the world coordinates and the cross-sectional area, using the following formula:

[0031] ;

[0032] ;

[0033] ;

[0034] ;

[0035] ;

[0036] ;

[0037] ;

[0038] ;

[0039] in, It is the interval between frame 1 and frame 2. The inter-frame pixel displacement of the speed measurement points, It is The horizontal coordinates of the speed measurement points on the graph are It is the second frame The horizontal coordinate of the graph after the displacement of the speed measurement point is It is The vertical coordinate of the speed measurement point on the graph is It is the second frame The vertical coordinate on the graph after the displacement of the speed measurement point is It is the interval between frame 1 and frame 2. The real displacement between frames of the speed measurement points, It is The world horizontal coordinate of the speed measurement point, It is the second frame The world horizontal coordinate after the displacement of the speed measurement point, It is The world vertical coordinate of the speed measurement point, It is the second frame The world vertical coordinate after the displacement of the speed measurement point, yes and The ratio of It is the interval between frame 1 and frame 2. The flow velocity at each measuring point, is the river video frame rate, is the surface velocity coefficient, It is the interval between frame 1 and frame 2. -1 and The average speed between the speed measurement points, It is the interval between frame 1 and frame 2. The flow velocity at each measuring point, It is the interval between frame 1 and frame 2. -1 and The cross-sectional flow between the speed measurement points, It is -1 and The cross-sectional area between the speed measurement points, is the total flow rate of the section, is the number of frames in the river video, is the number of speed measurement points, is the surface velocity of the river.

[0040] In a second aspect, the present invention further provides a video flow measurement device based on an improved recursive full-field transform optical flow model, comprising:

[0041] The video acquisition and frame cutting module is configured to: acquire river video and perform frame cutting processing to obtain video frames;

[0042] a data acquisition module configured to: calculate a transformation matrix based on the river video and pre-acquired survey results, transform the on-map coordinates of each speed measurement point in each video frame in the river video into world coordinates using the transformation matrix, and obtain the cross-sectional area between each speed measurement point;

[0043] The video frame processing module is configured to: input the video frame into the trained improved recursive full-field transform optical flow model, and output the pixel displacement between two adjacent frames at each speed measurement point;

[0044] The flow rate calculation module is configured to calculate the surface flow rate of the river according to the pixel displacement, the world coordinates and the cross-sectional area.

[0045] In a third aspect, the present invention further provides a computer-readable storage medium having a computer program stored thereon. When the computer program is executed by a processor, the video flow measurement method based on the improved recursive full field transform optical flow model as described in any one of the first aspects is implemented.

[0046] In a fourth aspect, the present invention further provides a device comprising:

[0047] a memory for storing instructions;

[0048] The processor is configured to execute the instructions so that the device performs operations to implement the video flow measurement method based on the improved recursive full field transform optical flow model as described in any one of the first aspects.

[0049] Compared with the prior art, the present invention has the following beneficial effects:

[0050] The present invention provides a video flow measurement method, device, storage medium and equipment based on an improved recursive full-field transformation optical flow model. The improved recursive full-field transformation optical flow model is adopted to improve the accuracy of detection. Specifically, a CBAM attention mechanism module is introduced into the existing recursive full-field transformation optical flow model, which can detect complex river changes and subtle water surface movements, and can improve the accuracy of the inter-frame displacement of the speed measurement point. Deformable convolution is introduced into the feature extraction part, which expands the receptive field and enhances the ability to extract water surface features, thereby obtaining a more accurate river surface speed. The present invention uses a pre-trained model, which can save measurement time and improve measurement efficiency. The present invention uses video for flow measurement, does not require manual on-site measurement, ensures the life safety of workers, and does not require the throwing of tracers into the water, avoiding environmental pollution. BRIEF DESCRIPTION OF THE DRAWINGS

[0051] Figure 1 This is one of the flow charts of a video flow measurement method based on an improved recursive full-field transform optical flow model provided by an embodiment of the present invention;

[0052] Figure 2 This is the second flow chart of a video flow measurement method based on an improved recursive full-field transform optical flow model provided by an embodiment of the present invention;

[0053] Figure 3 This is a schematic diagram of a natural river flow measurement scenario provided by an embodiment of the present invention;

[0054] Figure 4 Schematic diagram of the structure of the improved recursive full-field transform optical flow model provided by an embodiment of the present invention;

[0055] Figure 5 is a structural diagram of a feature extractor provided by an embodiment of the present invention;

[0056] Figure 6 Schematic diagram of the structure of the CBAM attention mechanism module provided by an embodiment of the present invention;

[0057] Figure 7 It is a structural diagram of the iterative update module provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0058] The present invention will be further described below in conjunction with the accompanying drawings. The following embodiments are only used to more clearly illustrate the technical solutions of the present invention and are not intended to limit the scope of protection of the present invention.

[0059] Example 1

[0060] like Figure 1 As shown, an embodiment of the present invention provides a video flow measurement method based on an improved recursive full field transform optical flow model, comprising the following steps:

[0061] S1. Obtain a river video and perform frame cutting to obtain video frames.

[0062] like Figure 2 As shown, in this embodiment, after obtaining the river video, a detection is performed to determine whether frame skipping exists. If the detection result shows that there is no frame skipping, frame cutting is performed to obtain video frames.

[0063] S2. Calculate a transformation matrix based on the river video and pre-acquired survey results, transform the on-map coordinates of each speed measurement point in each video frame in the river video into world coordinates through the transformation matrix, and obtain the cross-sectional area between each speed measurement point.

[0064] The coordinates of the speed measurement point on the map are , the world coordinates are .

[0065] by Figure 3 Taking the river shown in the figure as an example, the coordinates of the four calibration points A, B, C, and D on the map and in the world are (277,790), (1674,850), (1835,119), (1160,48) and (0,0), (9.49,0), (22.5,48.7), (-24.9,45.2) respectively. The transformation matrix is obtained by the projection transformation theory. Obtain the coordinates of the speed measurement points in the river section information (658,752), (912,597), (1119,468), (1274,374), (1384,306), (1470,253), (1542,208), (1604,170), and pass the projection matrix Converted to world coordinates (2.13, 0.35), (2.45, 1.75), (2.86, 3.68), (3.4, 6.03), (4, 8.75), (4.73, 12.05), (5.67, 16.45), (7.01, 22.5).

[0066] S3. Input the video frame into the trained improved recursive full-field transform optical flow model, and output the pixel displacement between two adjacent frames of each speed measurement point.

[0067] The pixel displacement between two adjacent frames of each speed measurement point is used to obtain the coordinates on the map after displacement. , the world coordinates of each speed measurement point after pixel displacement update are obtained through transformation matrix calculation ,in is the number of river video frames.

[0068] Taking the third speed measurement point as an example, the pixel coordinates of the second frame are , the world coordinates are .

[0069] The training method of the improved recursive full-field transform optical flow model is as follows: set the sample size batch size to 4 and the number of iterations iters to 12, use the Sintel dataset as the training dataset, and train the improved recursive full-field transform optical flow model through a deep learning network.

[0070] Improve the network structure of the recursive full field transformation optical flow model as follows Figure 4 As shown, including:

[0071] Feature extractor: The feature extractor is applied to two adjacent frames of images. Through a series of convolution and deformable convolution operations, an output feature map with a size of 1 / 8 of the original resolution is obtained. The number of feature map channels is set to 256.

[0072] 4D Correlation Pyramid Module: Calculates the complete correlation vectors between all pairs of feature vectors in the feature maps of two adjacent frames. Pooling is performed on the obtained correlations to obtain a 4D correlation pyramid. The correlations in the 4D correlation pyramid are indexed using the local pixel grid to generate the first correlation.

[0073] CBAM attention mechanism module: The first correlation quantity and the optical flow of the previous iteration are passed through the CBAM attention mechanism module to generate the second correlation quantity.

[0074] Iterative update module: Use the gated recurrent neural network module GRU to input the feature information of the first frame, the optical flow of the previous iteration, the second correlation value and the hidden state into the gated recurrent neural network module to obtain the updated optical flow and the updated hidden state;

[0075] Upsampling module: The optical flow obtained after the iteration is restored to full resolution through the upsampling module.

[0076] The structure of the feature extractor is as follows Figure 5 As shown, including:

[0077] Through a convolution of size 7×7 and number of channels 64;

[0078] Three groups of residual blocks are formed in groups of two. The number of channels of the three groups of residual blocks is 64, 96, and 128 respectively. Each residual block consists of two 3×3 convolutions, and the number of channels is consistent with the number of channels of each group of residual blocks.

[0079] Through a convolution of size 3×3 and number of channels 256;

[0080] Through a deformable convolution with a channel number of 256;

[0081] The resolution of two adjacent frames of images will be reduced to 1 / 2 of the original after passing through the first set of residual blocks, the resolution will be reduced to 1 / 4 of the original after passing through the second set of residual blocks, and the resolution will be reduced to 1 / 8 of the original after passing through the third set of residual blocks.

[0082] The structure of the 4D correlation pyramid module includes:

[0083] Calculate the complete correlation between all pairs of feature vectors of two adjacent frames using the following formula:

[0084] (1);

[0085] (2);

[0086] in, is the relevant quantity, is the first frame feature map, is the second frame feature map, Represents the feature map of two adjacent frames of images, Represents the feature information vector group of two adjacent frames of images, Represents the feature information of all pixels in the first frame feature map, Represents the feature information of all pixels in the feature map of the second frame, Represents dimension, and Represent the height and width of the feature map respectively;

[0087] The pooling operation of the correlation quantity to obtain a 4D correlation quantity pyramid includes: pooling the last two dimensions of the correlation quantity using convolution kernels of sizes 1, 2, 4, and 8 respectively to obtain a 4D correlation quantity pyramid , related quantity The size is , Indicates the number of layers in the 4D related quantity pyramid;

[0088] The use of the local pixel grid to index the correlation quantity in the 4D correlation quantity pyramid and generate the first correlation quantity is performed by the following formula:

[0089] (3);

[0090] in, is the first correlation quantity, Represents each pixel of the first frame feature map The value mapped to the feature map of the second frame after the optical flow update, is the radius of the local pixel grid, represents a set of integers, Represents the maximum sum of the absolute values of column vectors.

[0091] CBAM attention mechanism module see Figure 6 As shown, including:

[0092] Channel attention mechanism module: input feature image First, after average pooling and maximum pooling, two different spatial context feature information are obtained, and then through a shared multi-layer perceptron, and finally through the addition operation and Sigmoid function activation, a channel attention feature map is generated. . Channel attention mechanism module output The calculation of is shown in formula (4);

[0093] (4);

[0094] Spatial attention module: The input feature map of the spatial attention module is the output of the channel attention mechanism module After maximum pooling and average pooling, the two feature maps are spliced, and then a 7×7 convolution is performed, the number of channels is reduced to 1, and the spatial attention feature map is generated after Sigmoid function activation. . Output of the spatial attention module The calculation of is shown in formula (5);

[0095] (5).

[0096] Iterative update module see Figure 7 As shown, including:

[0097] Compute update gate , It is composed of the feature information of the first frame, the optical flow of the previous iteration, and the content of the second related quantity, as shown in formula (6);

[0098] (6);

[0099] Compute reset gate , as shown in formula (7);

[0100] (7);

[0101] Calculate the data after reset , as shown in formula (8);

[0102] (8);

[0103] Calculate the updated hidden state , as shown in formula (9);

[0104] (9).

[0105] S4. Calculate the river surface velocity according to the pixel displacement, the world coordinates, and the cross-sectional area.

[0106] It is done by the following formula:

[0107] ;

[0108] ;

[0109] ;

[0110] ;

[0111] ;

[0112] ;

[0113] ;

[0114] ;

[0115] in, It is the interval between frame 1 and frame 2. The inter-frame pixel displacement of the speed measurement points, It is The horizontal coordinates of the speed measurement points on the graph are It is the second frame The horizontal coordinate of the graph after the displacement of the speed measurement point is It is The vertical coordinate of the speed measurement point on the graph is It is the second frame The vertical coordinate on the graph after the displacement of the speed measurement point is It is the interval between frame 1 and frame 2. The real displacement between frames of the speed measurement points, It is The world horizontal coordinate of the speed measurement point, It is the second frame The world horizontal coordinate after the displacement of the speed measurement point, It is The world vertical coordinate of the speed measurement point, It is the second frame The world vertical coordinate after the displacement of the speed measurement point, yes and The ratio of It is the interval between frame 1 and frame 2. The flow velocity at each measuring point, is the river video frame rate, is the surface velocity coefficient, It is the interval between frame 1 and frame 2. -1 and The average speed between the speed measurement points, It is the interval between frame 1 and frame 2. The flow velocity at each measuring point, It is the interval between frame 1 and frame 2. -1 and The cross-sectional flow between the speed measurement points, It is -1 and The cross-sectional area between the speed measurement points, is the total flow rate of the section, is the number of frames in the river video, is the number of speed measurement points, is the surface velocity of the river.

[0116] Example 2

[0117] The embodiment of the present invention further provides a video flow measurement device based on an improved recursive full-field transform optical flow model, comprising:

[0118] The video acquisition and frame cutting module is configured to: acquire river video and perform frame cutting processing to obtain video frames;

[0119] a data acquisition module configured to: calculate a transformation matrix based on the river video and pre-acquired survey results, transform the on-map coordinates of each speed measurement point in each video frame in the river video into world coordinates using the transformation matrix, and obtain the cross-sectional area between each speed measurement point;

[0120] The video frame processing module is configured to: input the video frame into the trained improved recursive full-field transform optical flow model, and output the pixel displacement between two adjacent frames at each speed measurement point;

[0121] The flow rate calculation module is configured to calculate the surface flow rate of the river according to the pixel displacement, the world coordinates and the cross-sectional area.

[0122] Example 3

[0123] An embodiment of the present invention further provides a computer-readable storage medium having a computer program stored thereon. When the computer program is executed by a processor, the method for video flow measurement based on an improved recursive full-field transform optical flow model as provided in Example 1 is implemented:

[0124] Obtain river video and perform frame cutting to obtain video frames;

[0125] Calculating a transformation matrix based on the river video and pre-acquired survey results, transforming the on-map coordinates of each speed measurement point in each video frame in the river video into world coordinates using the transformation matrix, and obtaining the cross-sectional area between each speed measurement point;

[0126] Input the video frames into the trained improved recursive full-field transform optical flow model, and output the pixel displacement between two adjacent frames of each speed measurement point;

[0127] The river surface velocity is calculated based on the pixel displacement, the world coordinates and the cross-sectional area.

[0128] Example 4

[0129] An embodiment of the present invention further provides a device, including:

[0130] a memory for storing instructions;

[0131] A processor, configured to execute the instructions so that the device performs operations to implement the video flow measurement method based on the improved recursive full field transform optical flow model provided in Example 1:

[0132] Obtain river video and perform frame cutting to obtain video frames;

[0133] Calculating a transformation matrix based on the river video and pre-acquired survey results, transforming the on-map coordinates of each speed measurement point in each video frame in the river video into world coordinates using the transformation matrix, and obtaining the cross-sectional area between each speed measurement point;

[0134] Input the video frames into the trained improved recursive full-field transform optical flow model, and output the pixel displacement between two adjacent frames of each speed measurement point;

[0135] The river surface velocity is calculated based on the pixel displacement, the world coordinates and the cross-sectional area.

[0136] Those skilled in the art will appreciate that the embodiments of the present application may be provided as methods, systems, or computer program products. Therefore, the present application may take the form of an entirely hardware embodiment, an entirely software embodiment, or an embodiment combining software and hardware. Furthermore, the present application may take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to magnetic disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0137] The present application is described with reference to the flowcharts and / or block diagrams of the methods, devices (systems), and computer program products according to the embodiments of the present application. It should be understood that each process and / or block in the flowchart and / or block diagram, as well as the combination of processes and / or blocks in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to produce a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 a process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0138] These computer program instructions may also be stored in a computer readable memory that can direct a computer or other programmable data processing device to work in a specific manner, so that the instructions stored in the computer readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 a process or multiple processes and / or boxes Figure 1 The function specified in one or more boxes.

[0139] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operational steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing the instructions executed on the computer or other programmable device for implementing the process. Figure 1 a process or multiple processes and / or boxes Figure 1 The steps for the function specified in one or more boxes.

[0140] The above is only a preferred embodiment of the present invention. It should be pointed out that for ordinary technicians in this technical field, several improvements and modifications can be made without departing from the technical principles of the present invention. These improvements and modifications should also be regarded as the scope of protection of the present invention.

Claims

1. A video flow measurement method based on an improved recursive full-field transform optical flow model, characterized in that: include: Obtain river video and perform frame cutting to obtain video frames; Calculating a transformation matrix based on the river video and pre-acquired survey results, transforming the on-map coordinates of each speed measurement point in each video frame in the river video into world coordinates using the transformation matrix, and obtaining the cross-sectional area between each speed measurement point; Input the video frames into the trained improved recursive full-field transform optical flow model, and output the pixel displacement between two adjacent frames of each speed measurement point; Calculating a river surface velocity based on the pixel displacement, the world coordinates, and the cross-sectional area; The improved recursive full-field transformation optical flow model includes: The feature extractor operates on two adjacent video frames and obtains a feature map through convolution and deformable convolution operations. The resolution of the feature map is 1 / 8 of the input video frame. a 4D correlation pyramid module, configured to calculate the complete correlation between all pairs of feature vectors of feature maps of two adjacent frames, perform a pooling operation on the correlation to obtain a 4D correlation pyramid, and use a local pixel grid to index the correlation in the 4D correlation pyramid to generate a first correlation; A CBAM attention mechanism module, configured to convert the first correlation quantity and the optical flow of the previous iteration into a second correlation quantity; an iterative update module, configured to perform a gated loop on the feature information of the first frame, the optical flow of the previous iteration, the second correlation quantity, and the hidden state, and output an updated optical flow; The upsampling module is used to upsample the updated optical flow after the iteration and restore the updated optical flow to full resolution.

2. The video flow measurement method based on the improved recursive full field transform optical flow model according to claim 1 is characterized in that: The calculating of the transformation matrix based on the river video and the pre-acquired survey results includes: The coordinates of the calibration points on the map are obtained from the river video, and the world coordinates of the calibration points are obtained from the survey results. According to the coordinates of the calibration points on the map and the world coordinates, the transformation matrix is calculated through the projection transformation theory.

3. The video flow measurement method based on the improved recursive full field transform optical flow model according to claim 1 is characterized in that: The method of transforming the coordinates of each speed measurement point in each video frame of the river video into world coordinates by using a transformation matrix includes: Based on the projection transformation theory, the coordinates of each speed measurement point in each video frame in the river video are transformed into world coordinates through the transformation matrix.

4. The video flow measurement method based on the improved recursive full field transform optical flow model according to claim 1, characterized in that: The CBAM attention mechanism module includes a channel attention mechanism module and a spatial attention module; In the channel attention mechanism module, the input consisting of the first correlation quantity and the optical flow of the previous iteration is first subjected to average pooling and maximum pooling to obtain two different spatial context feature information. Then, through a shared multi-layer perceptron, sum operation and Sigmoid function activation, a channel attention feature map is generated. The calculation formula of the output of the channel attention mechanism module is as follows: ； in, is the output of the channel attention mechanism module, is the channel attention feature map, is the input, Represents the element-by-element multiplication symbol; In the spatial attention mechanism module, the output of the channel attention mechanism module is first subjected to average pooling and maximum pooling to obtain two different feature maps and spliced. Then, it undergoes a 7×7 convolution, the number of channels is reduced to 1, and the sigmoid function is activated to generate a spatial attention feature map. The calculation formula for the output of the spatial attention mechanism module is as follows: ； in, is the output of the spatial attention mechanism module, is the spatial attention feature map.

5. The video flow measurement method based on the improved recursive full field transform optical flow model according to claim 1, characterized in that: The improved recursive full field transform optical flow model is trained by the following method: The sample size and number of iterations are set, the Sintel dataset is used as the training dataset, and the trained improved recursive full-field transform optical flow model is obtained through deep learning network training.

6. The video flow measurement method based on the improved recursive full field transform optical flow model according to claim 1, characterized in that: The river surface velocity is calculated according to the pixel displacement, the world coordinates and the cross-sectional area by the following formula: ； ； ； ； ； ； ； ； in, It is the interval between frame 1 and frame 2. The inter-frame pixel displacement of the speed measurement points, It is The horizontal coordinates of the speed measurement points on the graph are It is the second frame The horizontal coordinate of the graph after the displacement of the speed measurement point is It is The vertical coordinate of the speed measurement point on the graph is It is the second frame The vertical coordinate on the graph after the displacement of the speed measurement point is It is the interval between frame 1 and frame 2. The real displacement between frames of the speed measurement points, It is The world horizontal coordinate of the speed measurement point, It is the second frame The world horizontal coordinate after the displacement of the speed measurement point, It is The world vertical coordinate of the speed measurement point, It is the second frame The world vertical coordinate after the displacement of the speed measurement point, yes and The ratio of It is the interval between frame 1 and frame 2. The flow velocity at each measuring point, is the river video frame rate, is the surface velocity coefficient, It is the interval between frame 1 and frame 2. -1 and The average speed between the speed measurement points, It is the interval between frame 1 and frame 2. The flow velocity at each measuring point, It is the interval between frame 1 and frame 2. -1 and The cross-sectional flow between the speed measurement points, It is -1 and The cross-sectional area between the speed measurement points, is the total flow rate of the section, is the number of frames in the river video, is the number of speed measurement points, is the surface velocity of the river.

7. A video flow measurement device based on an improved recursive full-field transform optical flow model, characterized in that: include: The video acquisition and frame cutting module is configured to: acquire river video and perform frame cutting processing to obtain video frames; a data acquisition module configured to: calculate a transformation matrix based on the river video and pre-acquired survey results, transform the on-map coordinates of each speed measurement point in each video frame in the river video into world coordinates using the transformation matrix, and obtain the cross-sectional area between each speed measurement point; The video frame processing module is configured to: input the video frame into the trained improved recursive full-field transform optical flow model, and output the pixel displacement between two adjacent frames at each speed measurement point; a flow velocity calculation module, configured to: calculate the surface flow velocity of the river according to the pixel displacement, the world coordinates and the cross-sectional area; The improved recursive full-field transform optical flow model includes: The feature extractor operates on two adjacent video frames and obtains a feature map through convolution and deformable convolution operations. The resolution of the feature map is 1 / 8 of the input video frame. a 4D correlation pyramid module, configured to calculate the complete correlation between all pairs of feature vectors of feature maps of two adjacent frames, perform a pooling operation on the correlation to obtain a 4D correlation pyramid, and use a local pixel grid to index the correlation in the 4D correlation pyramid to generate a first correlation; A CBAM attention mechanism module, configured to convert the first correlation quantity and the optical flow of the previous iteration into a second correlation quantity; an iterative update module, configured to perform a gated loop on the feature information of the first frame, the optical flow of the previous iteration, the second correlation quantity, and the hidden state, and output an updated optical flow; The upsampling module is used to upsample the updated optical flow after the iteration and restore the updated optical flow to full resolution.

8. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the video flow measurement method based on the improved recursive full field transform optical flow model according to any one of claims 1 to 6 is implemented.

9. A device, characterized in that include: a memory for storing instructions; The processor is configured to execute the instructions so that the device performs operations to implement the video flow measurement method based on the improved recursive full field transform optical flow model according to any one of claims 1 to 6.

Images