Water flow velocity measurement method, optical flow estimation network training method and device

The two-frame images in the water flow video are processed through the optical flow estimation network, which solves the problem of low detection accuracy under high flow velocity of mountain torrents, and realizes high-precision water flow velocity measurement, which is suitable for intelligent water conservancy projects.

CN118823076BActive Publication Date: 2024-11-22CHINA INST OF WATER RESOURCES & HYDROPOWER RES
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
CN202411306473.4
Authority / Receiving Office
CN · China
Patent Type
Patents(China)
Current Assignee / Owner
Filing Date
2024-09-19
Publication Date
2024-11-22
Estimated Expiration
2044-09-19

AI Technical Summary

Technical Problem

The traditional water flow rate measurement method has low detection accuracy under high flow rate of mountain torrents, which cannot meet the needs of intelligent water conservancy projects and is difficult to achieve large-scale deployment.

Method used

The optical flow estimation network is used to obtain two frames of images in the water flow video, determine the region of interest, and use the pre-trained optical flow estimation network to perform forward and backward optical flow calculations, combining sputtering processing and bidirectional hole filling to generate the target image. Calculate the flow rate of the water flow.

Benefits of technology

It improves the accuracy and speed of water flow velocity measurement, can accurately calculate flow velocity in complex environments, and is suitable for large-scale deployment and meets the needs of intelligent water conservancy projects.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN118823076B_ABST
    Figure CN118823076B_ABST
Patent Text Reader

Abstract

The present disclosure relates to a method for measuring the flow velocity of a water flow, a method for training an optical flow estimation network, and a device. The method comprises: obtaining two frames of water flow images of the same water flow in a water flow video; determining the region of interest in the two frames of water flow images; inputting the region of interest in each frame of the water flow image into a pre-trained optical flow estimation network, and outputting the pixel displacement between the two frames of water flow images through the optical flow estimation network; wherein the optical flow estimation network performs optical flow estimation on two frames of water flow training images of the same water flow, obtains forward optical flow and backward optical flow, performs sputtering processing and bidirectional hole filling based on the forward optical flow, backward optical flow, and two frames of water flow training images, generates a target water flow training image, and obtains it based on the target water flow training image and the forward optical flow training; calculates the flow velocity of the water flow based on the pixel displacement and the time between the two frames of water flow images. The present method can accurately calculate the flow velocity of the water flow.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present disclosure relates to the field of data processing technology, and in particular to a method for measuring water flow velocity, and a method and device for training an optical flow estimation network. Background Art

[0002] With the development of computer technology, more and more fields rely on computer computing resources for prediction or calculation. In the field of hydrological velocity measurement, key data such as velocity and flow in river water dispatch management are particularly important under the condition of high flow velocity of mountain torrents. These data provide effective information support for water situation control, water allocation and disaster prevention. Traditional water flow velocity measurement methods, such as using acoustic Doppler current profiler (ADCP) or satellite radar velocity measurement, have limited detection range and cannot be deployed over a large area. In addition, it is difficult for personnel to get involved in dangerous scenes such as mountain torrents, and it is difficult to meet the development needs of my country's water conservancy projects in the direction of intelligence. In recent years, with the continuous development of velocity detection technology in automation, modernization and informatization, the online velocity monitoring technology for some rivers has gradually matured, and its application prospects have become clear.

[0003] Traditional online monitoring technology needs to accurately estimate the velocity of high-speed water flow. Due to the high flow rate, traditional online monitoring technology will have a great impact on accuracy and cannot accurately calculate the flow rate of the water flow. Summary of the invention

[0004] Based on this, it is necessary to provide a water flow velocity measurement method, an optical flow estimation network training method and a device to address the above technical problems.

[0005] In a first aspect, the present disclosure provides a method for measuring water flow velocity, the method comprising:

[0006] Obtain two frames of water flow images of the same water flow in the water flow video;

[0007] Determining a region of interest in the two frames of water flow images, wherein the region of interest in each frame of the water flow image is the same;

[0008] The region of interest in each frame of the water flow image is input into a pre-trained optical flow estimation network, and the pixel displacement between the two frames of the water flow image is output through the optical flow estimation network; wherein the optical flow estimation network performs optical flow estimation on two frames of water flow training images of the same water flow to obtain forward optical flow and backward optical flow, and performs sputtering processing and bidirectional hole filling based on the forward optical flow, backward optical flow, and the two frames of water flow training images to generate a target water flow training image, which is obtained based on the target water flow training image and the forward optical flow training;

[0009] The flow velocity of the water flow is calculated based on the pixel displacement and the time between the two frames of water flow images.

[0010] In one embodiment, the two frames of water flow training images include a first water flow training image and a second water flow training image; the step of performing sputtering processing and bidirectional hole filling based on the forward optical flow, the backward optical flow, and the two frames of water flow training images to generate a target water flow training image includes:

[0011] Performing sputtering processing based on the forward optical flow, the first water flow training image, and the second water flow training image to reconstruct the second water flow training image;

[0012] Performing bidirectional hole filling based on the forward optical flow, the backward optical flow, the first water flow training image and the reconstructed second water flow training image to obtain a second target training image;

[0013] Mapping pixels in the first water flow training image to corresponding positions in the second target training image to generate a third water flow training image;

[0014] A target water flow training image is generated based on the first water flow training image and the third water flow training image.

[0015] In one embodiment, the reconstructing the second water flow training image by performing sputtering processing based on the forward optical flow, the first water flow training image and the second water flow training image comprises:

[0016] Determining a reconstruction weight based on the forward optical flow, the first water flow training image, the second water flow training image and a preset smoothing parameter;

[0017] Based on the reconstruction weight, the first water flow training image, and the second water flow training image, the second water flow training image is reconstructed.

[0018] In one embodiment, the step of performing bidirectional hole filling based on the forward optical flow, the backward optical flow, the first water flow training image and the reconstructed second water flow training image to obtain the second target training image includes:

[0019] Based on the forward optical flow, mapping pixels of the first water flow training image to the reconstructed second water flow training image to obtain a first mapped image;

[0020] Using the backward optical flow, mapping pixels in the first mapping image back to the first water flow training image to obtain a second mapping image;

[0021] Determine first hole information in the first mapped image and second hole information in the second mapped image;

[0022] Based on the forward optical flow and the backward optical flow, the first hole information and the second hole information are filled to obtain a second target training image.

[0023] In one embodiment, the method further comprises:

[0024] Constructing a first optical flow initial estimation network, wherein the first optical flow initial estimation network includes: a feature extraction network, a feature matching network, and a prediction network;

[0025] Adding relative position encoding to the feature extraction network and adding an attention mechanism to the feature matching network to obtain a second optical flow initial estimation network;

[0026] The second optical flow initial estimation network is trained using the target water flow training image and the forward optical flow to obtain an optical flow estimation network.

[0027] In one embodiment, the use of the target water flow training image and the forward optical flow to train the second optical flow initial estimation network to obtain the optical flow estimation network includes:

[0028] Determine the predicted optical flow obtained by the second optical flow initial estimation network during the training process;

[0029] Calculate endpoint error based on the forward optical flow and the predicted optical flow;

[0030] Determine the gradient of the predicted optical flow, and calculate a smoothness constraint result based on the gradient of the predicted optical flow, the forward optical flow, and the predicted optical flow;

[0031] Calculating a loss value based on the endpoint error and the smoothness constraint result;

[0032] The parameters of the second optical flow initial estimation network are updated based on the loss value to obtain an optical flow estimation network.

[0033] In a second aspect, the present disclosure further provides an optical flow estimation network training method, the method comprising:

[0034] Obtain at least two corresponding frames of water flow training images;

[0035] Performing optical flow estimation on the two frames of water flow training images to obtain forward optical flow and backward optical flow;

[0036] Based on the forward optical flow, the backward optical flow, and the two frames of water flow training images, sputtering processing and bidirectional hole filling are performed to generate a target water flow training image;

[0037] A second optical flow initial estimation network is trained based on the target water flow training image and the forward optical flow to obtain an optical flow estimation network, wherein the second optical flow initial estimation network includes: a feature extraction network, a feature matching network, and a prediction network; the feature extraction network contains relative position encoding, and the feature matching network contains an attention mechanism.

[0038] In a third aspect, the present disclosure further provides a water flow velocity measuring device. The device comprises:

[0039] An image acquisition module, used to acquire two frames of water flow images of the same water flow in a water flow video;

[0040] A region determination module, used for determining the region of interest in the two frames of water flow images, wherein the region of interest in each frame of water flow image is the same;

[0041] A network processing module, used for inputting the region of interest in each frame of water flow image into a pre-trained optical flow estimation network, and outputting the pixel displacement between the two frames of water flow images via the optical flow estimation network; wherein the optical flow estimation network performs optical flow estimation on two frames of water flow training images of the same water flow to obtain forward optical flow and backward optical flow, performs sputtering processing and bidirectional hole filling based on the forward optical flow, backward optical flow, and the two frames of water flow training images to generate a target water flow training image, which is obtained based on the training of the target water flow training image;

[0042] The flow rate calculation module is used to calculate the flow rate of the water flow based on the pixel displacement and the time between the two frames of water flow images.

[0043] In a fourth aspect, the present disclosure further provides an optical flow estimation network training device, the device comprising:

[0044] A training image acquisition module, used to acquire at least two corresponding frames of water flow training images;

[0045] An optical flow estimation module, used to perform optical flow estimation on the two frames of water flow training images to obtain forward optical flow and backward optical flow;

[0046] A training data generation module, used for performing sputtering processing and bidirectional hole filling based on the forward optical flow, the backward optical flow, and the two frames of water flow training images to generate a target water flow training image;

[0047] A training module is used to train a second optical flow initial estimation network based on the target water flow training image and the forward optical flow to obtain an optical flow estimation network, wherein the second optical flow initial estimation network includes: a feature extraction network, a feature matching network, and a prediction network; the feature extraction network contains relative position encoding, and the feature matching network contains an attention mechanism.

[0048] In a fifth aspect, the present disclosure further provides a computer device, wherein the computer device comprises a memory and a processor, wherein the memory stores a computer program, and when the processor executes the computer program, the steps in any of the above method embodiments are implemented.

[0049] In a sixth aspect, the present disclosure further provides a computer-readable storage medium having a computer program stored thereon, and when the computer program is executed by a processor, the steps in any of the above method embodiments are implemented.

[0050] In a seventh aspect, the present disclosure further provides a computer program product, which includes a computer program, and when the computer program is executed by a processor, the steps in any of the above method embodiments are implemented.

[0051] In the above embodiments, the region of interest in the two frames of water flow images is determined, and the region of interest is used for subsequent processing, so that the optical flow estimation network does not need to process all pixels of the water flow image, and only the region of interest is processed, which can improve the processing speed. In addition, the optical flow estimation network is trained using target water flow training images generated by sputtering processing and bidirectional hole filling. Sputtering processing can make image reconstruction more natural and continuous, and bidirectional hole filling can solve the problem of holes caused by inaccurate optical flow or occlusions in image reconstruction. Therefore, the accuracy of the data used in the training model can be ensured, thereby ensuring the accuracy of the pixel displacement determined by the optical flow estimation network. Finally, the flow rate of the water flow can be accurately calculated using the pixel displacement and the time between the two frames of water flow images. BRIEF DESCRIPTION OF THE DRAWINGS

[0052] In order to more clearly illustrate the specific embodiments of the present disclosure or the technical solutions in the prior art, the drawings required for use in the specific embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of the present disclosure. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0053] Figure 1 A schematic diagram of an application environment of a method for measuring water flow velocity in an embodiment;

[0054] Figure 2 A schematic diagram of a flow chart of a method for measuring water flow velocity in one embodiment;

[0055] Figure 3 A schematic flow chart of steps for generating a target water flow training image in one embodiment;

[0056] Figure 4 Schematic diagram of the process of step S302 in one embodiment;

[0057] Figure 5 Schematic diagram of the process of step S304 in one embodiment;

[0058] Figure 6 A schematic diagram of a process for training an optical flow estimation network in one embodiment;

[0059] Figure 7 is a schematic diagram of the process of step S606 in one embodiment;

[0060] Figure 8 A schematic diagram of a flow chart of an optical flow estimation network training method in one embodiment;

[0061] Fig. 9 It is a schematic block diagram of the structure of a water flow velocity measuring device in one embodiment;

[0062] Fig.10 A schematic block diagram of the structure of an optical flow estimation network training device in one embodiment

[0063] Fig.11 FIG. 4 is a diagram showing the internal structure of a computer device in one embodiment. DETAILED DESCRIPTION

[0064] In order to make the purpose, technical solution and advantages of the present disclosure more clear, the present disclosure is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present disclosure and are not used to limit the present disclosure.

[0065] It should be noted that the terms "first", "second", etc. in the specification and claims of this article and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the data used in this way can be interchanged where appropriate, so that the embodiments of this article described here can be implemented in an order other than those illustrated or described here. In addition, the terms "including" and "having" and any variations thereof are intended to cover non-exclusive inclusions, for example, a process, method, device, product or equipment that includes a series of steps or units is not necessarily limited to those steps or units clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or equipment.

[0066] In this article, the term "and / or" is only a description of the association relationship between related objects, indicating that there can be three relationships. For example, A and / or B can mean: A exists alone, A and B exist at the same time, and B exists alone. In addition, the character " / " in this article generally indicates that the related objects before and after are in an "or" relationship.

[0067] As described in the background technology, the current video water flow velocity measurement mainly includes four categories: particle image velocimetry (PIV), particle tracking velocimetry (PTV), space-time image velocimetry (STIV) and optical flow estimation velocimetry (OFV).

[0068] The main principle of the first three types of technologies is to detect and track image features through some traditional image processing methods, and then convert pixel displacement into actual displacement through affine transformation to finally obtain flow velocity. These methods are difficult to track the same feature point in flash floods, resulting in a significant impact on the accuracy of flow velocity detection. In addition, their detection accuracy fluctuates greatly when the parameter settings change, limiting their scope of application.

[0069] The fourth type of technology, optical flow estimation speed measurement technology OFV, is also divided into traditional optical flow estimation and neural network-based optical flow estimation. Among them, since the optical flow estimation based on neural network has higher robustness in high-velocity complex environments, it is more applicable and more suitable for actual flash flood application scenarios. However, the current technology still has some shortcomings, such as: the flash flood data set required by the neural network is difficult to obtain, and how to design the structure of the neural network to better track high-velocity characteristics. These problems need to be solved.

[0070] The present disclosure provides a method for measuring water flow velocity, which can be applied to Figure 1In the application environment shown. Among them, the terminal 102 communicates with the server 104 through the network. The data storage system can store the data that the server 104 needs to process. The data storage system can be integrated on the server 104, or it can be placed on the cloud or other network servers. The terminal 102 can obtain two frames of water flow images of the same water flow in the water flow video stored in the server 104. The terminal 102 can determine the region of interest in the two frames of water flow images, wherein the region of interest in each frame of the water flow image is the same. The terminal 102 can input the region of interest in each frame of the water flow image into the pre-trained optical flow estimation network, and output the pixel displacement between the two frames of the water flow image through the optical flow estimation network; wherein the optical flow estimation network is the terminal 102 or the server 104 performing optical flow estimation on two frames of water flow training images of the same water flow, obtaining forward optical flow and backward optical flow, performing sputtering processing and bidirectional hole filling based on the forward optical flow, backward optical flow, and the two frames of water flow training images, generating a target water flow training image, and training based on the target water flow training image. The terminal 102 can calculate the flow velocity of the water flow based on the pixel displacement and the time between the two frames of water flow images. The terminal 102 can be, but is not limited to, various personal computers, laptops, smart phones, tablet computers, etc. The server 104 can be implemented as an independent server or a server cluster consisting of multiple servers.

[0071] In one embodiment, Figure 2 As shown, a method for measuring water flow velocity is provided, and the method is applied to Figure 1 The terminal 102 in the example is used as an example to illustrate, and the following steps are included:

[0072] S202, obtaining two frames of water flow images of the same water flow in the water flow video.

[0073] The water flow video may be a video containing the corresponding states of water flow at a certain location at different times. In some embodiments of the present disclosure, the water flow video may be a video of a high-speed mountain torrent. Two frames of water flow images may generally be water flow images with a certain time interval between the two frames.

[0074] S204, determining the region of interest in the two frames of water flow images, wherein the region of interest in each frame of water flow image is the same.

[0075] Among them, Region of Interest (ROI) is an important concept in computer vision and image processing, which is used to refer to the specific parts of an image or video that the user is concerned about.

[0076] Specifically, an area of ​​interest can be determined from any one of the two water flow images using an algorithm or other method, and then an area corresponding to the area of ​​interest can be determined in the other water flow image as the area of ​​interest in the other water flow image. It should be noted that in some embodiments of the present disclosure, there is no limitation on how the area of ​​interest is determined.

[0077] S206, inputting the region of interest in each frame of the water flow image into a pre-trained optical flow estimation network, and outputting the pixel displacement between the two frames of the water flow image via the optical flow estimation network; wherein the optical flow estimation network performs optical flow estimation on two frames of water flow training images of the same water flow to obtain forward optical flow and backward optical flow, and performs sputtering processing and bidirectional hole filling based on the forward optical flow, backward optical flow, and the two frames of water flow training images to generate a target water flow training image, which is obtained based on the target water flow training image and the forward optical flow training.

[0078] Among them, pixel displacement refers to the amount of movement of pixels of an object or scene between adjacent frames in a video sequence or image sequence, which can be understood as optical flow. Pixel displacement can usually be represented by a two-dimensional vector, indicating changes in the horizontal and vertical directions. Optical flow estimation is a technology in computer vision that is used to estimate the pixel motion of objects or scenes in consecutive video frames. It infers the movement of objects in time by analyzing the brightness changes in the image sequence. Sputtering processing can be Softmax sputtering is an image reconstruction method used in optical flow estimation, which is mainly used to deal with artifacts caused by blur and uncertainty in optical flow estimation. It achieves smooth mapping of pixels by assigning pixel weights, making image reconstruction more natural and continuous. The core of Softmax sputtering is to use the Softmax function to normalize the weight of each pixel. Bidirectional hole filling is a method for solving the hole problem caused by inaccurate optical flow or occlusions in image reconstruction. It fills these holes by combining the information of forward and backward optical flow to ensure the integrity of the reconstructed image. The forward optical flow is usually the optical flow between two frames of images, such as image A to image B. The backward optical flow is usually the optical flow between two frames of images, such as image B to image A.

[0079] Specifically, at least two frames of water flow training images can be obtained first, and optical flow estimation can be performed on the two frames of water flow training images to obtain forward optical flow and backward optical flow. The forward optical flow, backward optical flow, and two frames of water flow training images are used for sputtering processing and bidirectional hole filling to generate a target water flow training image, and an optical flow estimation network is obtained based on the target water flow training image and forward optical flow training. Then, the region of interest in each frame of the water flow image is input into the pre-trained optical flow estimation network, and the pixel displacement (optical flow result) between the two frames of water flow images is output through the optical flow estimation network.

[0080] S208, calculating the flow velocity of the water flow based on the pixel displacement and the time between the two frames of water flow images.

[0081] Specifically, the pixel coordinates can be converted into camera coordinates. According to the camera projection principle, the depth value of the camera from the water surface can be used to convert the obtained pixel displacement into the distance in the real world, and then the flow rate of the water can be calculated based on the time between two frames.

[0082] In the above-mentioned water flow velocity measurement method, the region of interest in the two frames of water flow images is determined, and the region of interest is used for subsequent processing, so that the optical flow estimation network does not need to process all pixels of the water flow image, and only the region of interest is processed, which can improve the processing speed. In addition, the optical flow estimation network is trained using the target water flow training image generated by sputtering processing and bidirectional hole filling. The sputtering processing can make the image reconstruction more natural and continuous, and the bidirectional hole filling can solve the hole problem caused by inaccurate optical flow or occlusions in image reconstruction. Therefore, the accuracy of the data used in the training model can be ensured, thereby ensuring the accuracy of the pixel displacement determined by the optical flow estimation network. Finally, the pixel displacement and the time between the two frames of water flow images can be used to accurately calculate the flow velocity of the water flow.

[0083] In one embodiment, Figure 3 As shown, the two frames of water flow training images include a first water flow training image and a second water flow training image; the method of performing sputtering processing and bidirectional hole filling based on the forward optical flow, the backward optical flow, and the two frames of water flow training images to generate a target water flow training image includes:

[0084] S302, performing sputtering processing based on the forward optical flow, the first water flow training image and the second water flow training image to reconstruct the second water flow training image.

[0085] Specifically, weights can be calculated according to the forward optical flow and the Softmax function, and then the second water flow training image can be reconstructed using the weights, that is, pixels in the first water flow training image are mapped to corresponding positions in the second water flow training image according to the weights.

[0086] S304 , performing bidirectional hole filling based on the forward optical flow, the backward optical flow, the first water flow training image and the reconstructed second water flow training image to obtain a second target training image.

[0087] Specifically, due to holes caused by inaccurate optical flow or occlusions in the reconstruction, the integrity of the reconstructed second water flow training image is poor. Therefore, the forward optical flow and the backward optical flow can be used to perform bidirectional hole filling on the first water flow training image and the reconstructed second water flow training image, so as to obtain the hole-filled and reconstructed second water flow training image, and the second target training image can be obtained according to the hole-filled and reconstructed second water flow training image.

[0088] S306: Map pixels in the first water flow training image to corresponding positions in the second target training image to generate a third water flow training image.

[0089] Specifically, the pixels in the first water flow training image may be mapped again to corresponding positions in the second target training image to generate a third water flow training image.

[0090] S308: Generate a target water flow training image based on the first water flow training image and the third water flow training image.

[0091] In this embodiment, the Softmax sputtering process can handle the artifact problem caused by the blur and uncertainty in the optical flow estimation, making the image reconstruction more natural and continuous. The bidirectional hole filling process can fill the image holes caused by image deformation and inaccurate optical flow.

[0092] In one embodiment, Figure 4 As shown, the sputtering process is performed based on the forward optical flow, the first water flow training image and the second water flow training image to reconstruct the second water flow training image, including:

[0093] S402: Determine a reconstruction weight based on the forward optical flow, the first water flow training image, the second water flow training image, and a preset smoothing parameter.

[0094] Specifically, the reconstruction weight can be calculated using the following formula:

[0095]

[0096] in, is the reconstruction weight calculated by the Softmax function. It is a smoothing parameter that controls the distribution of reconstruction weights and is taken as 0.1 here. is a pixel position in the first water flow training image, is a pixel position of the second water flow training image, which can be obtained by forward optical flow Map To the second stream training image Location.

[0097] S404: reconstruct the second water flow training image based on the reconstruction weight, the first water flow training image, and the second water flow training image.

[0098] Specifically, after the reconstruction weights are determined, for the forward optical flow , from the first water flow training image To the second water flow training image The reconstruction process can be expressed as: , the second water flow training image can be reconstructed using the above formula.

[0099] In this embodiment, reconstruction weights are determined, and smooth mapping of pixels can be achieved by assigning reconstruction weights to pixels, making image reconstruction more natural and continuous, and improving the accuracy of water flow velocity determination.

[0100] In one embodiment, Figure 5 As shown, the bidirectional hole filling is performed based on the forward optical flow, the backward optical flow, the first water flow training image and the reconstructed second water flow training image to obtain the second target training image, including:

[0101] S502: Map pixels of the first water flow training image to a reconstructed second water flow training image based on the forward optical flow to obtain a first mapped image.

[0102] Specifically, using forward optical flow The first water flow training image The pixel position is mapped to the reconstructed second water flow training image The purpose of this step is to generate a preliminary first mapping image. , where each pixel position Through optical flow From the first water flow training images The corresponding position in Get the pixel value. Furthermore, the following formula can be used for pixel mapping:

[0103]

[0104] in is the first mapped image generated by the forward optical flow. is a pixel position in the reconstructed second water flow training image. is the pixel position in the first water flow training image, which is obtained by optical flow Mapped to the reconstructed second water flow training image Position. Through this mapping, each pixel value in the first mapping image comes from the first water flow training image .

[0105] S504: Map pixels in the first mapped image back to the first water flow training image using the backward optical flow to obtain a second mapped image.

[0106] Specifically, using backward optical flow Verify the effectiveness of the forward filling results. Backward optical flow The first mapping image The pixel positions are mapped back to the first water flow training image In this way, it is possible to check if there are holes or errors in the forward filling process. The following formula can be used for pixel mapping:

[0107]

[0108] in is the second mapped image generated by backward optical flow. is a pixel position in the first water flow training image. is the pixel position in the first mapped image or the reconstructed second water flow training image, which is obtained through the backward optical flow Mapping back to the first water flow training image Position. Backward verification checks the validity of the forward filling result, that is, checks the first mapping image Check for holes or errors in the

[0109] S506: Determine first hole information in the first mapped image and second hole information in the second mapped image.

[0110] Specifically, the first hole information and the second hole information may generally be information of an area without filled pixels. The first hole information may be determined based on the first mapped image. The second hole information may be determined based on the second mapped image.

[0111] S508: Fill the first hole information and the second hole information based on the forward optical flow and the backward optical flow to obtain a second target training image.

[0112] Specifically, the first hole information that appears during the forward filling process is filled by combining the forward optical flow and the backward optical flow. If the value at a position is invalid (such as a hole or an occlusion), the information of the backward optical flow is used to fill the position. This ensures the integrity and continuity of the image. In addition, the second hole information that appears during the backward filling process can also be filled.

[0113] In one embodiment, Figure 6 As shown, the method also includes:

[0114] S602, constructing a first optical flow initial estimation network, wherein the first optical flow initial estimation network includes: a feature extraction network, a feature matching network, and a prediction network.

[0115] Among them, the first optical flow initial estimation network The main task of the optical flow estimation network is to estimate the motion (optical flow) of each pixel from the two frames. In order to achieve this goal, the network structure is usually divided into feature extraction network, matching network and prediction network. The main task of the feature extraction network is to extract useful feature representations from the input image. These features should be able to capture important information in the image, such as edges, textures and other visual clues. Deep learning structures such as convolutional neural networks (CNN) are usually used to gradually extract high-level feature representations through multi-layer convolution and pooling operations. The matching network finds the corresponding relationship by comparing the features between different frames based on the features extracted by the feature extraction network. It looks for areas with similar features in the two frames to establish the motion relationship between pixels. Matching can be achieved by calculating the similarity between features (such as through distance measurement) and generating a matching map (or corresponding matching code) to represent the position of each pixel in the other frame. The prediction network is responsible for calculating the optical flow based on the information generated by the matching network. This part of the network combines features and matching information and finally outputs the optical flow vector of each pixel. The prediction network usually uses convolutional layers to process matching features and uses upsampling and other techniques to map the optical flow to the same resolution as the input image. During the entire optical flow estimation process, the various modules work together, feature extraction provides the basis for subsequent matching and prediction, and the matching network uses the extracted features to establish the correspondence between pixels. Finally, the prediction network uses this information to generate optical flow estimates. This modular design enables the optical flow estimation network to adaptively handle motion estimation tasks in complex scenes.

[0116] S604, adding relative position encoding to the feature extraction network and adding an attention mechanism to the feature matching network to obtain a second optical flow initial estimation network.

[0117] Specifically, relative position encoding can be added to the feature extraction network, which enables the network to better capture the relative position relationship between pixels in the image, which is particularly important when processing image registration and matching. The processing process of the feature extraction network includes: using a convolutional neural network (CNN) from an input image pair (such as two frames of water flow images) ( , ) to extract feature maps ( , ). For the feature map and Add relative position encoding. Assume that the size of the feature map is , the dimension of each feature vector is , generate the corresponding relative position code and add it to the feature map. The formula is as follows:

[0118]

[0119] in, is the position of the pixel coordinates in each feature map.

[0120] An attention mechanism can be added to the feature matching network. The attention mechanism can help the network focus on important areas during feature matching and improve the accuracy of optical flow estimation. The feature matching network processing process includes:

[0121] Calculate attention weights using feature maps and Calculate the attention weight. Here we use dot-product attention. The calculation formula is as follows:

[0122]

[0123] in , is a learnable weight matrix. The attention mechanism is introduced into the optical flow estimation for the prediction of optical flow, and the calculation is as follows:

[0124]

[0125] Finally, the new feature representation is used to predict the optical flow and obtain the final optical flow result. (This optical flow is usually the forward optical flow), which is calculated as follows:

[0126]

[0127] S606: Train the second optical flow initial estimation network using the target water flow training image and the forward optical flow to obtain an optical flow estimation network.

[0128] Specifically, the target water flow training image and the forward optical flow can be used as training data to train the second optical flow initial estimation network, thereby obtaining an optical flow estimation network.

[0129] In this embodiment, relative position encoding and attention mechanism are added to the optical flow estimation network. Relative position encoding enables the network to better capture the relative position relationship between pixels in the image, while the attention mechanism can help the network focus on important areas when matching features. Therefore, optical flow estimation can be performed more accurately, thereby improving the performance of the model in complex scenes and detail processing.

[0130] In one embodiment, Figure 7 As shown, the target water flow training image and the forward optical flow are used to train the second optical flow initial estimation network to obtain an optical flow estimation network, including:

[0131] S702, determining the predicted optical flow obtained by the second optical flow initial estimation network during the training process.

[0132] S704: Calculate endpoint errors based on the forward optical flow and the predicted optical flow.

[0133] Among them, End-Point Error (EPE) is an indicator for evaluating the accuracy of optical flow estimation or motion estimation methods. It measures the difference between the estimated optical flow (i.e., motion vector) and the true optical flow.

[0134] Specifically, the target water flow training image can be input into the second optical flow initial estimation network, and the predicted optical flow can be obtained through the output of the second optical flow initial estimation network. Then, the endpoint error can be calculated according to the Euclidean distance between the forward optical flow and the predicted optical flow.

[0135] In some exemplary embodiments, the endpoint error may be calculated using the following formula:

[0136] Among them, the predicted optical flow is And the forward optical flow is is the number of pixels.

[0137] S706, determining the gradient of the predicted optical flow, and calculating a smoothness constraint result based on the gradient of the predicted optical flow, the forward optical flow, and the predicted optical flow.

[0138] Among them, the gradient can include: The gradient of the direction and The gradient of the direction. Smoothness constraint is usually used to encourage the smoothness of the optical flow field and reduce the noise and discontinuity in the optical flow estimation. This constraint is based on the assumption that the movement of objects in the image should be continuous and smooth.

[0139] Specifically, the smoothness constraint result can be calculated according to the following formula: in, and Respectively represent the predicted optical flow pair and Direction gradient.

[0140] S708, calculating a loss value based on the endpoint error and the smoothness constraint result.

[0141] Specifically, the endpoint error and the smoothness constraint result can be added to obtain the loss value. In other implementations, the loss value can also be calculated using the following formula:

[0142] , It is a preset parameter, usually in the range of 0.1-1.

[0143] S710: Update the parameters of the second optical flow initial estimation network based on the loss value to obtain an optical flow estimation network.

[0144] Specifically, the loss value can be used to continuously update the parameters of the second optical flow initial estimation network until the loss value meets the requirements, and the optical flow estimation network is obtained based on the updated second optical flow initial estimation network.

[0145] In some exemplary embodiments, the parameters of the second optical flow initial estimation network can be updated by an Adam optimizer and a loss value. First, the gradient is calculated:

[0146] ,in, is at the time step The gradient of is the parameter of the previous time step, is the loss function. Then, update the first-order moment estimate and the second moment estimate :

[0147]

[0148] in, and is the decay rate, usually close to 1.

[0149] Next, the first-order moment estimate and the second-order moment estimate are bias corrected:

[0150] Finally, update the parameters: ,in is the learning rate, is a small constant used to prevent the denominator from being zero.

[0151] In this embodiment, by selecting the loss function as a combination of the endpoint error (End-Point Error, EPE) and the smoothness constraint (Smoothness Constraint), the effect of model training can be guaranteed and the accuracy of the model predicting optical flow can be improved.

[0152] In one embodiment, Figure 8 As shown, the present disclosure also provides an optical flow estimation network training method, comprising:

[0153] S802, obtaining at least two corresponding frames of water flow training images.

[0154] S804, performing optical flow estimation on the two frames of water flow training images to obtain forward optical flow and backward optical flow.

[0155] S806, performing sputtering processing and bidirectional hole filling based on the forward optical flow, the backward optical flow, and the two frames of water flow training images to generate a target water flow training image.

[0156] S808, training a second optical flow initial estimation network based on the target water flow training image and the forward optical flow to obtain an optical flow estimation network, wherein the second optical flow initial estimation network includes: a feature extraction network, a feature matching network, and a prediction network; the feature extraction network contains relative position encoding, and the feature matching network contains an attention mechanism.

[0157] For the specific implementation and limitation of this embodiment, please refer to the above embodiment, which will not be repeated here.

[0158] It should be understood that, although the various steps in the flowcharts involved in the above-mentioned embodiments are displayed in sequence according to the indication of the arrows, these steps are not necessarily executed in sequence according to the order indicated by the arrows. Unless there is a clear explanation in this article, the execution of these steps does not have a strict order restriction, and these steps can be executed in other orders. Moreover, at least a part of the steps in the flowcharts involved in the above-mentioned embodiments can include multiple steps or multiple stages, and these steps or stages are not necessarily executed at the same time, but can be executed at different times, and the execution order of these steps or stages is not necessarily to be carried out in sequence, but can be executed in turn or alternately with other steps or at least a part of the steps or stages in other steps.

[0159] Based on the same inventive concept, the embodiment of the present disclosure also provides an optical flow estimation network training device for implementing the optical flow estimation network training method involved above. The implementation solution provided by the device to solve the problem is similar to the implementation solution recorded in the above method, so the specific limitations in one or more optical flow estimation network training device embodiments provided below can refer to the limitations of the optical flow estimation network training method above, and will not be repeated here.

[0160] In one embodiment, Fig. 9 As shown, a water flow velocity measuring device 900 is provided, comprising: an image acquisition module 902, an area determination module 904, a network processing module 906 and a flow velocity calculation module 908, wherein:

[0161] An image acquisition module 902 is used to acquire two frames of water flow images of the same water flow in the water flow video;

[0162] A region determination module 904 is used to determine the region of interest in the two frames of water flow images, wherein the region of interest in each frame of water flow image is the same;

[0163] The network processing module 906 is used to input the region of interest in each frame of the water flow image into the pre-trained optical flow estimation network, and output the pixel displacement between the two frames of the water flow image through the optical flow estimation network; wherein the optical flow estimation network performs optical flow estimation on two frames of water flow training images of the same water flow to obtain forward optical flow and backward optical flow, and performs sputtering processing and bidirectional hole filling based on the forward optical flow, backward optical flow, and the two frames of water flow training images to generate a target water flow training image, which is obtained by training based on the target water flow training image;

[0164] The flow rate calculation module 908 is used to calculate the flow rate of the water flow based on the pixel displacement and the time between the two frames of water flow images.

[0165] In one embodiment of the device, the two frames of water flow training images include a first water flow training image and a second water flow training image; the network processing module 906 includes:

[0166] The sputtering processing module is used to perform sputtering processing based on the forward optical flow, the first water flow training image and the second water flow training image to reconstruct the second water flow training image.

[0167] The bidirectional hole filling module is used to perform bidirectional hole filling based on the forward optical flow, the backward optical flow, the first water flow training image and the reconstructed second water flow training image to obtain a second target training image.

[0168] The pixel mapping module is used to map pixels in the first water flow training image to corresponding positions in the second target training image to generate a third water flow training image.

[0169] The training image generation module is used to generate a target water flow training image based on the first water flow training image and the third water flow training image.

[0170] In one embodiment of the device, the sputtering processing module is also used to determine the reconstruction weight based on the forward optical flow, the first water flow training image, the second water flow training image and the preset smoothing parameter; and reconstruct the second water flow training image based on the reconstruction weight, the first water flow training image and the second water flow training image.

[0171] In one embodiment of the device, the bidirectional hole filling module is also used to map the pixels of the first water flow training image to the reconstructed second water flow training image based on the forward optical flow to obtain a first mapping image; use the backward optical flow to map the pixels in the first mapping image back to the first water flow training image to obtain a second mapping image; determine the first hole information in the first mapping image and the second hole information in the second mapping image; based on the forward optical flow and the backward optical flow, fill the first hole information and the second hole information to obtain a second target training image.

[0172] In one embodiment of the device, the device also includes: a model training module, used to construct a first optical flow initial estimation network, the first optical flow initial estimation network includes: a feature extraction network, a feature matching network, and a prediction network; relative position encoding is added to the feature extraction network, and an attention mechanism is added to the feature matching network to obtain a second optical flow initial estimation network; the second optical flow initial estimation network is trained using the target water flow training image and the forward optical flow to obtain an optical flow estimation network.

[0173] In one embodiment of the device, the model training module is also used to determine the predicted optical flow obtained by the second optical flow initial estimation network during the training process; calculate the endpoint error based on the forward optical flow and the predicted optical flow; determine the gradient of the predicted optical flow, and calculate the smoothness constraint result based on the gradient of the predicted optical flow, the forward optical flow and the predicted optical flow; calculate the loss value based on the endpoint error and the smoothness constraint result; and update the parameters of the second optical flow initial estimation network based on the loss value to obtain an optical flow estimation network.

[0174] Each module in the above-mentioned water flow velocity measuring device can be implemented in whole or in part by software, hardware and a combination thereof. Each of the above-mentioned modules can be embedded in or independent of a processor in a computer device in the form of hardware, or can be stored in a memory in a computer device in the form of software, so that the processor can call and execute the operations corresponding to each of the above modules.

[0175] In one embodiment, Fig.10 As shown, an optical flow estimation network training device 1000 is provided, and the device includes:

[0176] The training image acquisition module 1002 is used to acquire at least two corresponding frames of water flow training images;

[0177] An optical flow estimation module 1004 is used to perform optical flow estimation on the two frames of water flow training images to obtain forward optical flow and backward optical flow;

[0178] A training data generating module 1006 is used to perform sputtering processing and bidirectional hole filling based on the forward optical flow, the backward optical flow, and the two frames of water flow training images to generate a target water flow training image;

[0179] The training module 1008 is used to train a second optical flow initial estimation network based on the target water flow training image and the forward optical flow to obtain an optical flow estimation network, wherein the second optical flow initial estimation network includes: a feature extraction network, a feature matching network, and a prediction network; the feature extraction network contains relative position encoding, and the feature matching network contains an attention mechanism.

[0180] In one embodiment, a computer device is provided. The computer device may be a server, and its internal structure diagram may be as follows: Fig.11 As shown. The computer device includes a processor, a memory and a network interface connected through a system bus. Among them, the processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system, a computer program and a database. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The database of the computer device is used to store water flow image data. The network interface of the computer device is used to communicate with an external terminal through a network connection. When the computer program is executed by the processor, a water flow velocity measurement method is implemented.

[0181] Those skilled in the art will understand that Fig.11The structure shown in the figure is only a block diagram of a part of the structure related to the scheme of the present disclosure, and does not constitute a limitation on the computer device to which the scheme of the present disclosure is applied. The specific computer device may include more or fewer components than those shown in the figure, or combine certain components, or have a different arrangement of components.

[0182] In one embodiment, a computer device is provided, including a memory and a processor, wherein a computer program is stored in the memory, and the processor implements the steps in any of the above method embodiments when executing the computer program.

[0183] In one embodiment, a computer-readable storage medium is provided, on which a computer program is stored. When the computer program is executed by a processor, the steps in any of the above method embodiments are implemented.

[0184] In one embodiment, a computer program product is provided, including a computer program, which implements the steps in any of the above method embodiments when executed by a processor.

[0185] A person of ordinary skill in the art can understand that all or part of the processes in the above-mentioned embodiment method can be completed by instructing the relevant hardware through a computer program, and the computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, any reference to the memory, database or other medium used in the embodiments provided by the present disclosure can include at least one of non-volatile and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetoresistive random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory can include random access memory (RAM) or external cache memory, etc. As an illustration and not limitation, RAM can be in various forms, such as static random access memory (SRAM) or dynamic random access memory (DRAM). The database involved in each embodiment provided by the present disclosure may include at least one of a relational database and a non-relational database. Non-relational databases may include distributed databases based on blockchains, etc., but are not limited to this. The processor involved in each embodiment provided by the present disclosure may be a general-purpose processor, a central processing unit, a graphics processor, a digital signal processor, a programmable logic unit, a data processing logic unit based on quantum computing, etc., but are not limited to this.

[0186] The technical features of the above embodiments may be combined arbitrarily. To make the description concise, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0187] The above-described embodiments only express several implementation methods of the present disclosure, and the descriptions thereof are relatively specific and detailed, but they cannot be understood as limiting the scope of the present invention. It should be pointed out that, for a person of ordinary skill in the art, several modifications and improvements can be made without departing from the concept of the present disclosure, and these all belong to the protection scope of the present disclosure. Therefore, the protection scope of the present disclosure shall be subject to the attached claims.

Claims

1. A method for measuring water flow velocity, It is characterized in that The method comprises: Obtain two frames of water flow images of the same water flow in the water flow video; Determining a region of interest in the two frames of water flow images, wherein the region of interest in each frame of the water flow image is the same; The region of interest in each frame of the water flow image is input into a pre-trained optical flow estimation network, and the pixel displacement between the two frames of the water flow image is output through the optical flow estimation network; wherein the optical flow estimation network performs optical flow estimation on two frames of water flow training images of the same water flow to obtain forward optical flow and backward optical flow, and performs sputtering processing and bidirectional hole filling based on the forward optical flow, backward optical flow, and the two frames of water flow training images to generate a target water flow training image, which is obtained based on the target water flow training image and the forward optical flow training; Based on the pixel displacement and the time between the two frames of water flow images, the flow velocity of the water flow is calculated; wherein The two frames of water flow training images include a first water flow training image and a second water flow training image; the step of performing sputtering processing and bidirectional hole filling based on the forward optical flow, the backward optical flow, and the two frames of water flow training images to generate a target water flow training image includes: Performing sputtering processing based on the forward optical flow, the first water flow training image, and the second water flow training image to reconstruct the second water flow training image; Performing bidirectional hole filling based on the forward optical flow, the backward optical flow, the first water flow training image and the reconstructed second water flow training image to obtain a second target training image; Mapping pixels in the first water flow training image to corresponding positions in the second target training image to generate a third water flow training image; A target water flow training image is generated based on the first water flow training image and the third water flow training image.

2. The method according to claim 1, It is characterized in that The step of performing sputtering processing based on the forward optical flow, the first water flow training image, and the second water flow training image to reconstruct the second water flow training image includes: Determining a reconstruction weight based on the forward optical flow, the first water flow training image, the second water flow training image and a preset smoothing parameter; Based on the reconstruction weight, the first water flow training image, and the second water flow training image, the second water flow training image is reconstructed.

3. The method according to claim 1, It is characterized in that The step of performing bidirectional hole filling based on the forward optical flow, the backward optical flow, the first water flow training image and the reconstructed second water flow training image to obtain a second target training image includes: Based on the forward optical flow, mapping pixels of the first water flow training image to the reconstructed second water flow training image to obtain a first mapped image; Using the backward optical flow, mapping pixels in the first mapping image back to the first water flow training image to obtain a second mapping image; Determine first hole information in the first mapped image and second hole information in the second mapped image; Based on the forward optical flow and the backward optical flow, the first hole information and the second hole information are filled to obtain a second target training image.

4. The method according to claim 1, It is characterized in that The method further comprises: Constructing a first optical flow initial estimation network, wherein the first optical flow initial estimation network includes: a feature extraction network, a feature matching network, and a prediction network; Adding relative position encoding to the feature extraction network and adding an attention mechanism to the feature matching network to obtain a second optical flow initial estimation network; The second optical flow initial estimation network is trained using the target water flow training image and the forward optical flow to obtain an optical flow estimation network.

5. The method according to claim 4, It is characterized in that The method of training the second optical flow initial estimation network by using the target water flow training image and the forward optical flow to obtain an optical flow estimation network includes: Determine the predicted optical flow obtained by the second optical flow initial estimation network during the training process; Calculate endpoint error based on the forward optical flow and the predicted optical flow; Determine the gradient of the predicted optical flow, and calculate a smoothness constraint result based on the gradient of the predicted optical flow, the forward optical flow, and the predicted optical flow; Calculating a loss value based on the endpoint error and the smoothness constraint result; The parameters of the second optical flow initial estimation network are updated based on the loss value to obtain an optical flow estimation network.

6. A method for training an optical flow estimation network, It is characterized in that The method comprises: Obtain at least two corresponding frames of water flow training images; Performing optical flow estimation on the two frames of water flow training images to obtain forward optical flow and backward optical flow; Based on the forward optical flow, the backward optical flow, and the two frames of water flow training images, sputtering processing and bidirectional hole filling are performed to generate a target water flow training image; Based on the target water flow training image and the forward optical flow, a second optical flow initial estimation network is trained to obtain an optical flow estimation network, wherein the second optical flow initial estimation network includes: a feature extraction network, a feature matching network, and a prediction network; the feature extraction network contains relative position encoding, and the feature matching network contains an attention mechanism; wherein The two frames of water flow training images include a first water flow training image and a second water flow training image; the step of performing sputtering processing and bidirectional hole filling based on the forward optical flow, the backward optical flow, and the two frames of water flow training images to generate a target water flow training image includes: Performing sputtering processing based on the forward optical flow, the first water flow training image, and the second water flow training image to reconstruct the second water flow training image; Performing bidirectional hole filling based on the forward optical flow, the backward optical flow, the first water flow training image and the reconstructed second water flow training image to obtain a second target training image; Mapping pixels in the first water flow training image to corresponding positions in the second target training image to generate a third water flow training image; generating a target water flow training image based on the first water flow training image and the third water flow training image; and The method further comprises: Constructing a first optical flow initial estimation network, wherein the first optical flow initial estimation network includes: a feature extraction network, a feature matching network, and a prediction network; Adding relative position encoding to the feature extraction network and adding an attention mechanism to the feature matching network to obtain a second optical flow initial estimation network; The second optical flow initial estimation network is trained using the target water flow training image and the forward optical flow to obtain an optical flow estimation network; wherein The method of training the second optical flow initial estimation network by using the target water flow training image and the forward optical flow to obtain an optical flow estimation network includes: Determine the predicted optical flow obtained by the second optical flow initial estimation network during the training process; Calculate endpoint error based on the forward optical flow and the predicted optical flow; Determine the gradient of the predicted optical flow, and calculate a smoothness constraint result based on the gradient of the predicted optical flow, the forward optical flow, and the predicted optical flow; Calculating a loss value based on the endpoint error and the smoothness constraint result; The parameters of the second optical flow initial estimation network are updated based on the loss value to obtain an optical flow estimation network.

7. A water flow velocity measuring device, It is characterized in that The device comprises: An image acquisition module, used to acquire two frames of water flow images of the same water flow in a water flow video; A region determination module, used for determining the region of interest in the two frames of water flow images, wherein the region of interest in each frame of water flow image is the same; A network processing module, used for inputting the region of interest in each frame of water flow image into a pre-trained optical flow estimation network, and outputting the pixel displacement between the two frames of water flow images via the optical flow estimation network; wherein the optical flow estimation network performs optical flow estimation on two frames of water flow training images of the same water flow to obtain forward optical flow and backward optical flow, performs sputtering processing and bidirectional hole filling based on the forward optical flow, backward optical flow, and the two frames of water flow training images to generate a target water flow training image, which is obtained based on the training of the target water flow training image; A flow rate calculation module is used to calculate the flow rate of the water flow based on the pixel displacement and the time between the two frames of water flow images; wherein The two frames of water flow training images include a first water flow training image and a second water flow training image; the step of performing sputtering processing and bidirectional hole filling based on the forward optical flow, the backward optical flow, and the two frames of water flow training images to generate a target water flow training image includes: Performing sputtering processing based on the forward optical flow, the first water flow training image, and the second water flow training image to reconstruct the second water flow training image; Performing bidirectional hole filling based on the forward optical flow, the backward optical flow, the first water flow training image and the reconstructed second water flow training image to obtain a second target training image; Mapping pixels in the first water flow training image to corresponding positions in the second target training image to generate a third water flow training image; A target water flow training image is generated based on the first water flow training image and the third water flow training image.

8. An optical flow estimation network training device, It is characterized in that The device comprises: A training image acquisition module, used to acquire at least two corresponding frames of water flow training images; An optical flow estimation module, used to perform optical flow estimation on the two frames of water flow training images to obtain forward optical flow and backward optical flow; A training data generation module, used for performing sputtering processing and bidirectional hole filling based on the forward optical flow, the backward optical flow, and the two frames of water flow training images to generate a target water flow training image; A training module is used to train a second optical flow initial estimation network based on the target water flow training image and the forward optical flow to obtain an optical flow estimation network, wherein the second optical flow initial estimation network includes: a feature extraction network, a feature matching network, and a prediction network; the feature extraction network contains relative position encoding, and the feature matching network contains an attention mechanism; wherein The two frames of water flow training images include a first water flow training image and a second water flow training image; the step of performing sputtering processing and bidirectional hole filling based on the forward optical flow, the backward optical flow, and the two frames of water flow training images to generate a target water flow training image includes: Performing sputtering processing based on the forward optical flow, the first water flow training image, and the second water flow training image to reconstruct the second water flow training image; Performing bidirectional hole filling based on the forward optical flow, the backward optical flow, the first water flow training image and the reconstructed second water flow training image to obtain a second target training image; Mapping pixels in the first water flow training image to corresponding positions in the second target training image to generate a third water flow training image; generating a target water flow training image based on the first water flow training image and the third water flow training image; and The device also includes: A first optical flow initial estimation network, wherein the first optical flow initial estimation network includes: a feature extraction network, a feature matching network, and a prediction network; A second optical flow initial estimation network is obtained by adding relative position encoding to the feature extraction network and adding an attention mechanism to the feature matching network; The optical flow estimation network is obtained by training the second optical flow initial estimation network using the target water flow training image and the forward optical flow; wherein The step of training the second optical flow initial estimation network by using the target water flow training image and the forward optical flow includes: Determine the predicted optical flow obtained by the second optical flow initial estimation network during the training process; Calculate endpoint error based on the forward optical flow and the predicted optical flow; Determine the gradient of the predicted optical flow, and calculate a smoothness constraint result based on the gradient of the predicted optical flow, the forward optical flow, and the predicted optical flow; Calculating a loss value based on the endpoint error and the smoothness constraint result; The parameters of the second optical flow initial estimation network are updated based on the loss value to obtain an optical flow estimation network.

9. A computer device comprising a memory and a processor, wherein the memory stores a computer program. It is characterized in that When the processor executes the computer program, the steps of the method according to any one of claims 1 to 5 or claim 6 are implemented.