A method and device for calculating water flow displacement field based on initial optical flow estimation

Through the calculation method of water flow displacement field based on initial optical flow estimation, the problems of high cost and susceptibility to the environment of traditional water flow velocity measurement methods are solved, and the accurate displacement field calculation in complex water flow environments is realized, reducing costs and improving calculation accuracy sex.

CN118823073BActive Publication Date: 2024-11-22CHINA INST OF WATER RESOURCES & HYDROPOWER RES
View PDF 6 Cites 0 Cited by

Patent Information

Application Number
CN202411306537.0
Authority / Receiving Office
CN · China
Patent Type
Patents(China)
Current Assignee / Owner
Filing Date
2024-09-19
Publication Date
2024-11-22
Estimated Expiration
2044-09-19

AI Technical Summary

Technical Problem

Traditional water flow speed measurement methods rely on physical sensors, are costly and susceptible to environmental factors, and are prone to tracking failures or large errors in complex water flow environments.

Method used

The water flow displacement field calculation method based on initial optical flow estimation is adopted. The optical flow is realized by setting section segments in the water flow video, constructing interrogation windows and search windows, calculating optical flow displacement values, performing feature point detection and Lucas-Kanade method parameter construction. Track to calculate the water flow displacement vector.

Benefits of technology

It does not require relying on physical sensors, is low-cost, can adapt to complex water flow environments, provide reliable initial tracking information, and realizes accurate calculation of the surface displacement field of the water flow.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN118823073B_ABST
    Figure CN118823073B_ABST
Patent Text Reader

Abstract

The present invention provides a method and device for calculating water flow displacement field based on initial optical flow estimation, the method comprising: setting a cross-section line segment in each of any two consecutive frames of images; constructing an inquiry window in the previous frame of the image, and constructing a search window in the next frame of the image; intercepting an inquiry window image in the previous frame of the image, and sequentially moving the search window in the next frame of the image to intercept multiple search window images; determining a target search window image with the highest similarity to the inquiry window image; calculating an initial optical flow displacement estimation value, constructing an optical flow calculation function and performing optical flow tracking based on the initial optical flow displacement estimation value and the target feature point set of the previous frame of the image, and obtaining a water flow displacement vector. The present invention can realize accurate calculation of the displacement field of the water flow surface by combining image preprocessing, initial optical flow estimation and displacement field calculation.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present invention relates to the technical field of image processing, and in particular to a method and device for calculating a water flow displacement field based on initial optical flow estimation. Background Art

[0002] At present, traditional methods of measuring water flow velocity mostly rely on physical sensors, such as ultrasound and laser. Although these methods are accurate, they are costly and easily affected by environmental factors. In recent years, the optical flow method based on image processing has gradually become a research hotspot in the field of water flow velocity measurement due to its advantages such as non-contact, low cost, and wide application range. However, in complex water flow environments, especially when the water flow velocity varies greatly and the image noise is high, the traditional optical flow method is prone to tracking failure or large errors. Summary of the invention

[0003] In view of this, the present invention provides a method and device for calculating the water flow displacement field based on initial optical flow estimation, which is used to solve the problems that the current water flow velocity measurement methods either rely on physical sensors, which are high in cost and easily affected by environmental factors, or cannot adapt to complex water flow environments and are prone to tracking failures or large errors.

[0004] In order to solve the above technical problems, in a first aspect, the present invention provides a method for calculating a water flow displacement field based on initial optical flow estimation, the method comprising:

[0005] A cross-section line segment is set at the same position in any two consecutive frames of the water flow video;

[0006] Constructing an inquiry window at the location of the cross-section line segment in the previous frame image, and constructing a search window of the same size at the same location in the next frame image;

[0007] In the previous frame image, the area where the query window is located is intercepted to obtain a query window image, and in the next frame image, the search window is sequentially moved by a preset step length along the water flow direction until the search window moves to the edge of the next frame image, and each area where the search window is located is intercepted to obtain multiple search window images;

[0008] Determine, from the plurality of search window images, a target search window image having the highest similarity to the query window image;

[0009] Calculating a displacement value of the target search window image relative to the search window image located at the cross-section line segment, and using the displacement value as an initial optical flow displacement estimation value;

[0010] Performing full-image feature point detection on the previous frame image to obtain a target feature point set;

[0011] Constructing parameters in an optical flow calculation function based on the Lucas-Kanade method according to the initial optical flow displacement estimation value and the target feature point set;

[0012] The optical flow calculation function is used to perform optical flow tracing to obtain a water flow displacement vector.

[0013] Optionally, after the step of using the optical flow calculation function to perform optical flow tracing to obtain a water flow displacement vector, the method further includes:

[0014] According to the water flow displacement vector, a stable displacement field is constructed and the flow velocity value of the water flow is calculated.

[0015] Optionally, the step of determining, from the multiple search window images, a target search window image having the highest similarity to the query window image comprises:

[0016] Extracting feature points from the query window image to obtain a first feature point set;

[0017] Extracting feature points from each of the search window images to obtain multiple sets of second feature point sets;

[0018] Calculating the cluster center point of the first feature point set and the cluster center point of each group of the second feature point set;

[0019] The similarity between the query window image and each of the search window images is calculated based on the cluster center point of the first feature point set and each group of cluster center points of the second feature point set.

[0020] Optionally, the step of calculating the similarity between the query window image and each of the search window images according to the cluster center points of the first feature point set and the cluster center points of each group of the second feature point set comprises:

[0021] Calculating the Euclidean distance between the cluster center point of the first feature point set and each group of cluster center points of the second feature point set;

[0022] The Euclidean distance value is used as a similarity score between the query window image and each of the search window images.

[0023] Optionally, the step of constructing parameters in an optical flow calculation function based on a Lucas-Kanade method according to the initial optical flow displacement estimation value and the target feature point set includes:

[0024] The coordinates of each feature point in the target feature point set plus the initial optical flow displacement estimation value are used as the initial estimated position in the optical flow calculation function.

[0025] In a second aspect, the present invention further provides a water flow displacement field calculation device based on initial optical flow estimation, the device comprising:

[0026] A cross-section line segment setting module is used to set a cross-section line segment at the same position in any two consecutive frames of images in the water flow video;

[0027] A window construction module, used to construct an inquiry window at the location of the cross-section line segment in the previous frame image, and to construct a search window of the same size at the same location in the next frame image;

[0028] A capture module, configured to capture the area where the query window is located in the previous frame image to obtain a query window image, and to sequentially move the search window in the next frame image along the water flow direction by a preset step length until the search window moves to the edge of the next frame image, and to capture the area where each search window is located to obtain a plurality of search window images;

[0029] A determination module, configured to determine, from the plurality of search window images, a target search window image having the highest similarity to the query window image;

[0030] A displacement calculation module, used for calculating the displacement value of the target search window image relative to the search window image located at the cross-section line segment, and using the displacement value as an initial optical flow displacement estimation value;

[0031] A detection module, used to perform feature point detection on the entire previous frame image to obtain a target feature point set;

[0032] A construction module, used to construct parameters in an optical flow calculation function based on the Lucas-Kanade method according to the initial optical flow displacement estimation value and the target feature point set;

[0033] The optical flow tracing module is used to perform optical flow tracing using the optical flow calculation function to obtain a water flow displacement vector.

[0034] Optionally, the device further comprises:

[0035] The processing module is used to construct a stable displacement field and calculate the flow velocity value of the water flow according to the water flow displacement vector.

[0036] Optionally, the determining module includes:

[0037] A first extraction unit, configured to extract feature points from the query window image to obtain a set of first feature points;

[0038] A second extraction unit, configured to extract feature points from each of the search window images to obtain a plurality of second feature point sets;

[0039] A first calculation unit, used for calculating the cluster center point of the first feature point set and the cluster center point of each group of the second feature point set;

[0040] The second calculation unit is used to calculate the similarity between the query window image and each of the search window images according to the cluster center point of the first feature point set and the cluster center point of each group of the second feature point set.

[0041] Optionally, the second computing unit includes:

[0042] A Euclidean distance calculation subunit, used to calculate the Euclidean distance value between the cluster center point of the first feature point set and each group of cluster center points of the second feature point set;

[0043] The similarity scoring subunit is used to use the Euclidean distance value as a similarity score between the query window image and each of the search window images.

[0044] Optionally, the construction module includes:

[0045] A construction unit is used to add the coordinates of each feature point in the target feature point set to the initial optical flow displacement estimation value as the initial estimated position in the optical flow calculation function.

[0046] The beneficial effects of the above technical solution of the present invention are as follows:

[0047] In the embodiment of the present invention, the water flow displacement field calculation method based on initial optical flow estimation does not need to rely on physical sensors, is low-cost, and can adapt to complex water flow environments. By combining image preprocessing, initial optical flow estimation, and displacement field calculation, it can provide reliable initial tracking information for the optical flow tracking algorithm, thereby realizing accurate calculation of the displacement field of the water flow surface. BRIEF DESCRIPTION OF THE DRAWINGS

[0048] Figure 1 Schematic diagram of a flow chart of a method for calculating a water flow displacement field based on initial optical flow estimation in Embodiment 1 of the present invention;

[0049] Figure 2 A schematic diagram of an inquiry window and a search window provided in the first embodiment of the present invention;

[0050] Figure 3 A schematic diagram of a cross-sectional rendering effect provided by the first embodiment of the present invention;

[0051] Figure 4 Schematic diagram of the structure of a water flow displacement field calculation device based on initial optical flow estimation in Embodiment 2 of the present invention. DETAILED DESCRIPTION

[0052] In order to make the purpose, technical solution and advantages of the embodiment of the present invention clearer, the technical solution of the embodiment of the present invention will be clearly and completely described below in conjunction with the drawings of the embodiment of the present invention. Obviously, the described embodiment is a part of the embodiment of the present invention, not all of the embodiments. Based on the described embodiment of the present invention, all other embodiments obtained by ordinary technicians in this field belong to the scope of protection of the present invention.

[0053] Traditional methods of measuring water flow velocity mostly rely on physical sensors, such as ultrasound and laser. Although these methods are accurate, they are costly and easily affected by environmental factors. In recent years, the optical flow method based on image processing has gradually become a research hotspot in the field of water flow velocity measurement due to its advantages such as non-contact, low cost, and wide application range. However, in complex water flow environments, especially when the water flow velocity varies greatly and the image noise is high, the traditional optical flow method is prone to tracking failure or large errors.

[0054] In the process of making the present invention, the inventor of the present application discovered that cv2.calcOpticalFlowPyrLK() is a function in OpenCV for calculating optical flow, especially the pyramid version of the Lucas-Kanade method. The cv2.calcOpticalFlowPyrLK() function is very suitable for tracking the changes of points in an image over time. In the cv2.calcOpticalFlowPyrLK() function, the flags parameter is a very important parameter, which is used to control the behavior and performance of the algorithm. The flags parameter is a bit mask that can be set by combining multiple flags with the bitwise or (|) operator. In general, you only need to pay attention to the following two flags:

[0055] (1) cv2.OPTFLOW_USE_INITIAL_FLOW: If set to True, it means that the algorithm will use the optical flow estimate from the previous frame to the current frame as the initial guess for the pyramid iteration. This helps maintain tracking stability when there is large motion between frames.

[0056] (2) cv2.OPTFLOW_LK_GET_MIN_EIGENVALS: This flag is used to output the minimum eigenvalue of each point. This is useful for understanding the tracking quality, because small eigenvalues ​​may mean that the tracked points are not good feature points in the image, for example, they may be smooth without obvious texture.

[0057] When the flags parameter contains cv2.OPTFLOW_USE_INITIAL_FLOW, the custom nextPts parameter can store the initial optical flow estimate from the previous frame to the current frame. This helps maintain tracking stability when there is large motion between frames or poor image quality. In addition, the custom nextPts parameter can reduce the memory allocation, thereby further improving algorithm performance.

[0058] Therefore, this application proposes a method and device for calculating water flow displacement field based on initial optical flow estimation, which mainly consists of the following contents:

[0059] (1) Video acquisition and preprocessing: Use a high-resolution camera to capture video of the water surface to ensure that sufficient detail information is captured. The captured video is preprocessed, including denoising, grayscale, contrast enhancement, etc., to improve image quality and reduce the impact of lighting changes on optical flow calculation.

[0060] (2) Feature point extraction: For the preprocessed video frames, the feature point detection algorithm is used to extract stable feature points. The extracted feature points are screened to remove noise points and unstable points, and retain feature points that can accurately reflect the movement of water flow.

[0061] (3) Initial optical flow estimation method based on similarity analysis within the search box: For the extracted feature points, a local search is first performed within the search box. By analyzing the similarity of the feature points and pixels within the search box, an initial estimate of the optical flow displacement is made, thereby providing a reliable basis for the accurate calculation of the water flow displacement field.

[0062] (4) Displacement field calculation: Based on the initial optical flow estimation results, adaptively configure the optical flow parameters. Perform optical flow tracking and displacement field calculation based on the configured function.

[0063] (5) Visualization processing based on cross-section rendering: In view of the fact that the displacement field traced by optical flow is a sparse displacement field, a displacement field visualization processing based on cross-section rendering is designed.

[0064] The following example illustrates the method and device for automatically identifying calibration objects in the above-mentioned river speed measurement scenario.

[0065] See also Figure 1 , Figure 1 A flow chart of a method for calculating a water flow displacement field based on initial optical flow estimation provided in the first embodiment of the present invention, the method comprising the following steps:

[0066] Step 11: Set a cross-section line segment at the same position in any two consecutive frames of the water flow video.

[0067] In this embodiment, the water flow video can be acquired by a live camera. To improve the accuracy of subsequent processing, a high-resolution camera can be used to capture the water flow surface to ensure that sufficient detail information is captured. From the water flow video, any two consecutive video images are input, and then a cross-section line segment is set at the same position in each image. One function of the cross-section line segment is to visualize the water flow velocity result and the other function is to locate the subsequent window.

[0068] Step 12: construct an inquiry window at the location of the cross-section line segment in the previous frame image, and construct a search window of the same size at the same location in the next frame image.

[0069] In this embodiment, the so-called previous frame image and next frame image are the two consecutive frames of images mentioned above, which are respectively called the previous frame image and the next frame image, wherein the shooting time of the previous frame image is earlier than the shooting time of the next frame image.

[0070] In some embodiments of the present application, optionally, before performing step 12, the two frames of images may be preprocessed. Exemplarily, Gaussian filtering may be used to perform denoising and contrast enhancement on the two input frames of images, and then the enhanced images may be grayscaled to obtain grayscale frames. Preprocessing the two frames of images helps to improve image quality and reduce the effects of factors such as lighting changes on the optical flow calculation.

[0071] Please refer to Figure 2 , Figure 2 Schematic diagram of the query window and search window provided in the first embodiment of the present invention. Figure 2 As shown, Figure 2 The left side is the previous frame image. Figure 2 The right side is the next frame image. After image preprocessing is completed, an inquiry window is constructed at the location of the cross-section line segment in the previous frame image, and a search window is constructed at the same location in the next frame image. The shape and size of the search window are exactly the same as the inquiry window.

[0072] Exemplarily, the height of the query window may be set to 10 pixels.

[0073] Step 13: In the previous frame image, the area where the inquiry window is located is captured to obtain an inquiry window image, and in the subsequent frame image, the search window is moved in sequence by a preset step length along the water flow direction until the search window moves to the edge of the subsequent frame image, and each search window area is captured to obtain multiple search window images.

[0074] In this step, the query window image can be obtained by intercepting the area where the query window is located in the previous frame image. For the next frame image, the search window located at the cross-section line segment can be moved along the water flow direction (i.e. Figure 2 The preset step length is moved sequentially in the direction indicated by the arrow on the right until the search window moves to the edge of the next frame image. By intercepting the area where the search window is located each time in the next frame image, multiple search window images can be obtained.

[0075] In some embodiments, optionally, the preset step size of the search window movement is 5 pixels.

[0076] Step 14: Determine, from the multiple search window images, a target search window image having the highest similarity to the query window image.

[0077] In this embodiment, the similarity between each search window image and the query window image can be calculated to determine the search window image with the highest similarity to the query window image, that is, the target search window image.

[0078] Step 15: Calculate the displacement value of the target search window image relative to the search window image located at the cross-section line segment, and use the displacement value as the initial optical flow displacement estimation value.

[0079] After the target search window image is determined, the displacement value of the target search window image relative to the search window image located at the cross-section line segment can be calculated, which can also be considered as the distance of the coordinate displacement of the target search window image relative to the query window image on the original image. Therefore, the displacement value is used as the initial optical flow displacement estimation value.

[0080] Exemplarily, since the preset step size is known, for example, 5 pixels, the target search window image is obtained by taking a screenshot after the target search window is displaced for the Nth time, and the displacement distance at this time is 5N.

[0081] Step 16: Perform feature point detection on the entire previous frame image to obtain a target feature point set.

[0082] In this embodiment, when detecting feature points of the entire image of the previous frame, the cv2.goodFeaturesToTrack() function in Open CV can be used to detect stable feature points, wherein the corner point detection method is set to Shi-Tomasi.

[0083] Step 17: construct parameters in an optical flow calculation function based on the Lucas-Kanade method according to the initial optical flow displacement estimation value and the target feature point set.

[0084] After obtaining the initial optical flow displacement estimation value and the target feature point set obtained by feature extraction on the previous frame image, the parameter value of the optical flow calculation function with the initialized optical flow information (i.e., the initial optical flow displacement estimation value) can be constructed. The optical flow calculation function is a function used to calculate optical flow in OpenCV.

[0085] Step 18: Use the optical flow calculation function to perform optical flow tracking to obtain the water flow displacement vector.

[0086] After completing the construction of the parameters in the optical flow calculation function based on the Lucas-Kanade method, the function can be used to perform optical flow tracking to obtain the water flow displacement vector.

[0087] Through the above steps, the water flow displacement field calculation based on the initial optical flow estimation is finally realized.

[0088] The water flow displacement field calculation method based on initial optical flow estimation provided in an embodiment of the present invention does not need to rely on physical sensors, is low in cost, and can adapt to complex water flow environments. By combining image preprocessing, initial optical flow estimation, and displacement field calculation, it can provide reliable initial tracking information for the optical flow tracking algorithm, thereby realizing accurate calculation of the displacement field of the water flow surface.

[0089] The following example illustrates the above method for calculating the water flow displacement field based on initial optical flow estimation.

[0090] Exemplarily, the optical flow calculation function is cv2.calcOpticalFlowPyrLK().

[0091] The known parameters of cv2.calcOpticalFlowPyrLK() are as follows:

[0092] - prevImg: The previous frame. This is the reference frame for optical flow tracking.

[0093] - nextImg: the next frame image. This is the frame for feature point tracking.

[0094] - prevPts: the feature point array of the previous frame image, i.e. the target feature point set described in the present invention.

[0095] - nextPts: output parameter (optional). If you customize nextPts and configure the flags parameter, then this nextPts will be used as the initial estimated point position for optical flow tracking, that is, the initial position of the iterative search.

[0096] - status: Output parameter (optional), indicating the tracking status of nextPts. Customizable.

[0097] - err: Output parameter (optional), indicating the error vector of nextPts. Customizable.

[0098] - winSize: The size of the search window. Used to search for new locations of feature points in the image.

[0099] - maxLevel: The maximum number of levels of the pyramid.

[0100] - criteria: The termination condition of the iterative search algorithm.

[0101] - flags: Operation flag. If the flag is set to cv2.OPTFLOW_USE_INITIAL_FLOW, it means using the - nextPts value as the initial estimated point position.

[0102] - minEigThreshold: quality factor. The quality factor can be used to filter out feature points with poor quality.

[0103] In some embodiments of the present application, the step of constructing parameters in the optical flow calculation function based on the Lucas-Kanade method according to the initial optical flow displacement estimation value and the target feature point set includes:

[0104] The coordinates of each feature point in the target feature point set plus the initial optical flow displacement estimation value are used as the initial estimated position in the optical flow calculation function.

[0105] Exemplarily, to construct LK (Lucas-Kanade method) parameter values ​​with initialized optical flow information, the present invention constructs LK parameters as follows:

[0106] nextPts = P0 + [ESx, ESy]

[0107] status = np.zeros((len(nextPts), 1), np.uint8)

[0108] err = np.zeros_like(status)

[0109] flags=cv2.OPTFLOW_USE_INITIAL_FLOW|cv2.OPTFLOW_LK_GET_MIN_EIGENVALS.

[0110] Among them, P0 is the target feature point set mentioned above, [ESx, ESy] is the displacement value mentioned above, that is, the initial optical flow displacement estimation value.

[0111] The overall design ideas are as follows:

[0112] Including cv2.OPTFLOW_USE_INITIAL_FLOW in the flags parameter setting, and using the nextPts definition point set as the premise of the initial estimated position of the iterative search, each coordinate point of the target feature point set P0 detected in the previous frame plus the initial optical flow estimation value (ESx, ESy) obtained based on the similarity analysis in the search box is used as the value of nextPts, that is, the initial estimated position. Then the nextPts value is input into the optical flow tracking algorithm, guiding the algorithm to start iterative search from the position indicated by nextPts, which improves the stability of the tracking algorithm in complex scenes.

[0113] In one optional specific implementation manner, after the step of using the optical flow calculation function to perform optical flow tracing to obtain a water flow displacement vector, the method further includes:

[0114] According to the water flow displacement vector, a stable displacement field is constructed and the flow velocity value of the water flow is calculated.

[0115] In other words, based on the displacement vector of the tracked water flow, a stable displacement field can be constructed. Figure 3 , Figure 3 This is a schematic diagram of the cross-sectional rendering effect provided by the first embodiment of the present invention. Figure 3 As shown, at the same time, the cross-section rendering method can be used for visual drawing, that is, visual drawing is performed at the position of the cross-section line segment. Through the cross-section rendering method, the result can be made more intuitive and convenient to view. In addition, the flow velocity value of the water flow can be calculated.

[0116] In some embodiments of the present application, the step of determining the target search window image having the highest similarity to the query window image from the multiple search window images includes:

[0117] Extracting feature points from the query window image to obtain a first feature point set;

[0118] Extracting feature points from each of the search window images to obtain multiple sets of second feature point sets;

[0119] Calculating the cluster center point of the first feature point set and the cluster center point of each group of the second feature point set;

[0120] The similarity between the query window image and each of the search window images is calculated based on the cluster center point of the first feature point set and each group of cluster center points of the second feature point set.

[0121] Specifically, when determining the target search window image with the highest similarity to the query window image, feature point extraction can be performed on the query window image to obtain a set of first feature point sets, and then feature point extraction can be performed on each search window image to obtain a set of second feature point sets corresponding to each search window image. In this embodiment, optionally, when performing feature point detection / extraction, the cv2.goodFeaturesToTrack() function in Open CV can be used to detect stable feature points. Among them, the corner point detection method is set to Shi-Tomasi.

[0122] In this embodiment, the number of cluster center points can be set to 3, that is, 3 cluster center points are calculated for each set of feature points. Then, based on the cluster center points of the first feature point set and the cluster center points of each set of second feature point sets, the similarity between the query window image and each search window image can be calculated.

[0123] In some other embodiments of the present application, the step of calculating the similarity between the query window image and each of the search window images according to the cluster center point of the first feature point set and the cluster center point of each group of the second feature point set includes:

[0124] Calculating the Euclidean distance between the cluster center point of the first feature point set and each group of cluster center points of the second feature point set;

[0125] The Euclidean distance value is used as a similarity score between the query window image and each of the search window images.

[0126] That is to say, in this embodiment, the Euclidean distance value between the cluster center point of the first feature point set corresponding to the query window image and the cluster center point of the second feature point set corresponding to each search window image is calculated, and the Euclidean distance value is used as the similarity score between the query window image and each search window image. The higher the similarity score, the higher the similarity between the two window images, and the search window image corresponding to the highest similarity score is selected as the target search window image.

[0127] In summary, the main effects of this application are as follows:

[0128] (1) Improving calculation accuracy: Optical flow calculation is an important technology in the field of image processing and computer vision. It estimates the motion field by analyzing the motion of pixels or feature points in an image sequence. In the calculation of the water flow displacement field, this application can achieve accurate initial optical flow estimation, and accurate initial optical flow estimation can help the optical flow algorithm locate the best algorithm iteration starting point, thereby greatly improving the accuracy of displacement field calculation.

[0129] (2) Enhanced robustness: The water flow in the natural environment is complex and often accompanied by challenges such as illumination changes, occlusion, and weak textures. In this application, the collected video is preprocessed, including denoising, grayscale, and contrast enhancement, so that the algorithm can run stably in various complex environments and improve the robustness of the calculation.

[0130] (3) Solve the problem of low calculation accuracy of existing solutions: Existing optical flow calculation models may perform poorly under conditions such as large displacement motion and illumination changes. This application introduces the Lucas-Kanade optical flow tracking method based on initial optical flow estimation to improve the accuracy and robustness of optical flow estimation, thereby more accurately calculating the water flow displacement field.

[0131] (4) Broadening application scenarios: Accurate calculation of water flow displacement field is of great significance for studying fluid geodynamic characteristics, conducting water flow simulation and prediction, etc. This application provides a reliable water flow displacement field calculation method, which can strongly support scientific research and engineering applications in related fields.

[0132] See also Figure 4 , Figure 4 4 is a schematic diagram of a water flow displacement field calculation device based on initial optical flow estimation provided by Embodiment 2 of the present invention. The device 40 includes:

[0133] A cross-section line segment setting module 41 is used to set a cross-section line segment at the same position in any two consecutive frames of the water flow video;

[0134] A window construction module 42, configured to construct an inquiry window at the location of the cross-section line segment in the previous frame image, and to construct a search window of the same size at the same location in the next frame image;

[0135] The interception module 43 is used to intercept the area where the query window is located in the previous frame image to obtain an query window image, and in the next frame image, sequentially move the search window along the water flow direction by a preset step length until the search window moves to the edge of the next frame image, and intercept each area where the search window is located to obtain multiple search window images;

[0136] A determination module 44, configured to determine, from the plurality of search window images, a target search window image having the highest similarity to the query window image;

[0137] A displacement calculation module 45, used to calculate the displacement value of the target search window image relative to the search window image located at the cross-section line segment, and use the displacement value as an initial optical flow displacement estimation value;

[0138] A detection module 46 is used to perform feature point detection on the entire previous frame image to obtain a target feature point set;

[0139] A construction module 47, configured to construct parameters in an optical flow calculation function based on a Lucas-Kanade method according to the initial optical flow displacement estimation value and the target feature point set;

[0140] The optical flow tracing module 48 is used to perform optical flow tracing using the optical flow calculation function to obtain a water flow displacement vector.

[0141] Optionally, the device 40 further includes:

[0142] The processing module is used to construct a stable displacement field and calculate the flow velocity value of the water flow according to the water flow displacement vector.

[0143] Optionally, the determining module includes:

[0144] A first extraction unit, configured to extract feature points from the query window image to obtain a set of first feature points;

[0145] A second extraction unit, configured to extract feature points from each of the search window images to obtain a plurality of second feature point sets;

[0146] A first calculation unit, used for calculating the cluster center point of the first feature point set and the cluster center point of each group of the second feature point set;

[0147] The second calculation unit is used to calculate the similarity between the query window image and each of the search window images according to the cluster center point of the first feature point set and the cluster center point of each group of the second feature point set.

[0148] Optionally, the second computing unit includes:

[0149] A Euclidean distance calculation subunit, used to calculate the Euclidean distance value between the cluster center point of the first feature point set and each group of cluster center points of the second feature point set;

[0150] The similarity scoring subunit is used to use the Euclidean distance value as a similarity score between the query window image and each of the search window images.

[0151] Optionally, the construction module includes:

[0152] A construction unit is used to add the coordinates of each feature point in the target feature point set to the initial optical flow displacement estimation value as the initial estimated position in the optical flow calculation function.

[0153] In the embodiments of the present invention, there is no need to rely on physical sensors, the cost is low, and it can adapt to complex water flow environments. By combining image preprocessing, initial optical flow estimation and displacement field calculation, it can provide reliable initial tracking information for the optical flow tracking algorithm, thereby realizing accurate calculation of the displacement field of the water flow surface.

[0154] The embodiment of the present invention is a product embodiment corresponding to the above-mentioned method embodiment 1, so it will not be described in detail here. Please refer to the above-mentioned embodiment 1 for details.

[0155] The above is a preferred embodiment of the present invention. It should be pointed out that for ordinary technicians in this technical field, several improvements and modifications can be made without departing from the principles of the present invention. These improvements and modifications should also be regarded as the scope of protection of the present invention.

Claims

1. A method for calculating water flow displacement field based on initial optical flow estimation, characterized in that: include: A cross-section line segment is set at the same position in any two consecutive frames of the water flow video; Constructing an inquiry window at the location of the cross-section line segment in the previous frame image, and constructing a search window of the same size at the same location in the next frame image; In the previous frame image, the area where the query window is located is intercepted to obtain a query window image, and in the next frame image, the search window is sequentially moved by a preset step length along the water flow direction until the search window moves to the edge of the next frame image, and each area where the search window is located is intercepted to obtain multiple search window images; Determine, from the plurality of search window images, a target search window image having the highest similarity to the query window image; Calculating a displacement value of the target search window image relative to the search window image located at the cross-section line segment, and using the displacement value as an initial optical flow displacement estimation value; Performing full-image feature point detection on the previous frame image to obtain a target feature point set; Constructing parameters in an optical flow calculation function based on the Lucas-Kanade method according to the initial optical flow displacement estimation value and the target feature point set; Use the optical flow calculation function to perform optical flow tracking to obtain a water flow displacement vector; in After the step of using the optical flow calculation function to perform optical flow tracing to obtain a water flow displacement vector, the method further includes: According to the water flow displacement vector, a stable displacement field is constructed and the flow velocity value of the water flow is calculated; wherein The step of determining the target search window image having the highest similarity to the query window image from the multiple search window images comprises: Extracting feature points from the query window image to obtain a first feature point set; Extracting feature points from each of the search window images to obtain multiple sets of second feature point sets; Calculating the cluster center point of the first feature point set and the cluster center point of each group of the second feature point set; According to the cluster center point of the first feature point set and each group of cluster center points of the second feature point set, the similarity between the query window image and each search window image is calculated; wherein The step of calculating the similarity between the query window image and each of the search window images according to the cluster center point of the first feature point set and each group of cluster center points of the second feature point set comprises: Calculating the Euclidean distance between the cluster center point of the first feature point set and each group of cluster center points of the second feature point set; The Euclidean distance value is used as a similarity score between the query window image and each of the search window images; wherein The step of constructing parameters in the optical flow calculation function based on the Lucas-Kanade method according to the initial optical flow displacement estimation value and the target feature point set comprises: The coordinates of each feature point in the target feature point set plus the initial optical flow displacement estimation value are used as the initial estimated position in the optical flow calculation function.

2. A water flow displacement field calculation device based on initial optical flow estimation, characterized in that: include: A cross-section line segment setting module is used to set a cross-section line segment at the same position in any two consecutive frames of images in the water flow video; A window construction module, used to construct an inquiry window at the location of the cross-section line segment in the previous frame image, and to construct a search window of the same size at the same location in the next frame image; A capture module, configured to capture the area where the query window is located in the previous frame image to obtain a query window image, and to sequentially move the search window in the next frame image along the water flow direction by a preset step length until the search window moves to the edge of the next frame image, and to capture the area where each search window is located to obtain a plurality of search window images; A determination module, configured to determine, from the plurality of search window images, a target search window image having the highest similarity to the query window image; A displacement calculation module, used for calculating the displacement value of the target search window image relative to the search window image located at the cross-section line segment, and using the displacement value as an initial optical flow displacement estimation value; A detection module, used to perform feature point detection on the entire previous frame image to obtain a target feature point set; A construction module, used to construct parameters in an optical flow calculation function based on the Lucas-Kanade method according to the initial optical flow displacement estimation value and the target feature point set; An optical flow tracking module, used to perform optical flow tracking using the optical flow calculation function to obtain a water flow displacement vector; in The device also includes: A processing module is used to construct a stable displacement field and calculate the flow velocity value of the water flow according to the water flow displacement vector; wherein The determination module comprises: A first extraction unit, configured to extract feature points from the query window image to obtain a set of first feature points; A second extraction unit, configured to extract feature points from each of the search window images to obtain a plurality of second feature point sets; A first calculation unit, used for calculating the cluster center point of the first feature point set and the cluster center point of each group of the second feature point set; The second calculation unit is used to calculate the similarity between the query window image and each of the search window images according to the cluster center point of the first feature point set and the cluster center point of each group of the second feature point set; wherein The second computing unit comprises: A Euclidean distance calculation subunit, used to calculate the Euclidean distance value between the cluster center point of the first feature point set and each group of cluster center points of the second feature point set; A similarity scoring subunit is used to use the Euclidean distance value as a similarity score between the query window image and each of the search window images; wherein The construction module comprises: A construction unit is used to add the coordinates of each feature point in the target feature point set to the initial optical flow displacement estimation value as the initial estimated position in the optical flow calculation function.

Citation Information

Patent Citations

  • River surface flow velocity estimation method based on optical flow calculation and readable storage medium

    CN113012195A

  • Sewage draining exit water flow speed detection method and device based on video monitoring

    CN115272422A

  • River surface water velocity detection method based on video stream image processing

    CN117218546A

  • Water flow velocity estimation method and system based on traditional optical flow calculation

    CN117576168A

  • Water flow velocity measurement method and device based on tracer identification, medium and electronic equipment

    CN118097120A