A method of flow rate measurement and associated products

Abstract

This invention provides a method for measuring water flow velocity and related products. The method includes: acquiring a standardized image sequence of a target water area; extracting candidate feature points from every two adjacent real-time acquired images in the standardized image sequence using the SIFT algorithm, and filtering them according to preset screening criteria to obtain a set of candidate feature points; performing optical flow field analysis based on every two adjacent real-time acquired images using the RAFT optical flow model to obtain a dense optical flow field; determining a set of candidate optical flow vectors from the dense optical flow field based on the coordinates of each candidate feature point in the candidate feature point set; and determining the water flow velocity of the target water area based on the set of candidate optical flow vectors and a preset pixel-to-actual distance conversion relationship. This invention achieves accurate capture of the continuous motion trajectory of water flow and reliable matching of stable feature points, effectively improving velocity measurement accuracy and anti-interference capability.

Description

Technical Field

[0001] This invention relates to the field of image processing technology, and in particular to a method for measuring water flow velocity and related products. Background Technology

[0002] Current non-contact water flow velocity measurement technologies mainly include image feature matching-based methods and model-based methods. Image feature matching-based methods calculate flow velocity by inter-frame displacement of feature points, but they struggle to capture the continuous motion trajectory of water flow. Furthermore, the uneven distribution and overlapping of feature points on the water surface can lead to mismatches and inaccurate displacement calculations. Model-based methods, when dealing with water flow scenarios, are susceptible to interference from water surface reflections, ripple disturbances, and changes in illumination. This results in low accuracy in optical flow field calculations, the generation of numerous invalid optical flow vectors, and consequently, excessive flow velocity calculation errors, making them unsuitable for dynamic water flow scenarios in complex outdoor rivers. Summary of the Invention

[0003] One objective of this invention is to improve the speed measurement accuracy and anti-interference capability of non-contact water flow velocity measurement technology in complex outdoor river scenarios.

[0004] A further objective of this invention is to achieve accurate capture of the continuous motion trajectory of water flow and reliable matching of stable feature points.

[0005] Specifically, the present invention provides a method for measuring water flow velocity, comprising: A standardized image sequence of the target water area is obtained. The standardized image sequence contains multiple real-time acquired images arranged in chronological order. The time interval between each adjacent real-time acquired image is equal, and each real-time acquired image has the same pixel size. The images are also corrected to the same coordinate system by perspective transformation. The SIFT algorithm is used to extract candidate feature points from every two adjacent real-time acquired images in the standardized image sequence, and the candidate feature points are filtered according to preset screening criteria to obtain a set of candidate feature points. The RAFT optical flow model is used to perform optical flow field analysis based on every two adjacent real-time acquired images to obtain a dense optical flow field. The RAFT optical flow model is obtained by machine learning training on the historical optical flow dataset of the target water area. The historical optical flow dataset contains multiple sets of historical acquired images and their corresponding optical flow field annotations. The optical flow field annotations are the displacement vectors of each pixel in each set of historical acquired images. Based on the coordinates of each candidate feature point in the candidate feature point set, the optical flow vector corresponding to the candidate feature point is determined from the dense optical flow field, and the optical flow vector of the non-candidate feature point region in the dense optical flow field is set to zero vector, thus obtaining the candidate optical flow vector set corresponding to the candidate feature point set. The water flow velocity of the target water area is determined based on the candidate optical flow vector set and the preset pixel-actual distance conversion relationship.

[0006] Optionally, the steps of obtaining a standardized image sequence of the target water area include: The original video stream of the target water area is obtained according to a preset frame rate; Multiple original captured images are selected from the original video stream according to a preset time interval; The dimensions of each of the original acquired images are normalized to a preset target size; Perspective correction is performed on the original acquired image after the size normalization is completed, transforming the river region in the original acquired image into a rectangle; The original acquired image after perspective correction is denoised to obtain the standardized image sequence containing multiple real-time acquired images.

[0007] Optionally, the step of extracting candidate feature points from every two adjacent real-time acquired images in the standardized image sequence using the SIFT algorithm, and filtering them according to preset screening criteria to obtain a set of candidate feature points includes: For each of the real-time acquired images, a Gaussian pyramid and a difference-of-Gaussian pyramid are constructed respectively. Extreme point detection and key point localization are performed based on the difference-of-Gaussian pyramid. Key point orientation assignment and feature descriptor generation are performed based on the Gaussian pyramid. Initial feature points of each of the real-time acquired images are extracted, wherein each initial feature point contains a feature point response value. The Euclidean distance matching method is used to match the initial feature points of two adjacent real-time acquired images to determine the correspondence between adjacent real-time acquired images and the distance ratio of each pair of correspondences; The preset filtering criteria include a distance ratio threshold and a response value threshold. Correspondences that meet any of the following conditions are eliminated: the distance ratio is greater than the distance ratio threshold, or the feature point response value is less than the response value threshold. The initial feature points in the remaining correspondences are used as candidate feature points, and the candidate feature points are classified according to the real-time acquired images to obtain a set of candidate feature points for each real-time acquired image.

[0008] Optionally, determining the optical flow vector corresponding to each candidate feature point from the dense optical flow field based on the coordinates of each candidate feature point in the candidate feature point set includes: If the coordinates of the candidate feature point are integer coordinates, a neighborhood window of a preset size is defined in the dense optical flow field with the integer coordinates as the center, the optical flow vectors of all pixels in the neighborhood window are extracted, and the median or mean vector of the optical flow vectors of all pixels is used as the optical flow vector corresponding to the candidate feature point. If the coordinates of the candidate feature point are non-integer coordinates, the optical flow vector corresponding to the candidate feature point is obtained by interpolating from the optical flow vectors of the adjacent integer pixels around the non-integer coordinate position using bilinear interpolation.

[0009] Optionally, before setting the optical flow vector of the non-candidate feature point region in the dense optical flow field to a zero vector, the method further includes: Determine the local neighborhood variance of the optical flow vector of each pixel in the dense optical flow field, and identify pixels whose local neighborhood variance exceeds a preset fluctuation threshold as abnormal optical flow points. For each candidate feature point in the dense optical flow field, determine the pixel point on which the optical flow vector corresponding to the candidate feature point depends. If the pixel point on which the candidate feature point depends contains the abnormal optical flow point, remove the optical flow vector corresponding to the candidate feature point.

[0010] Optionally, the step of determining the water flow velocity of the target water area based on the candidate optical flow vector set and the preset pixel-actual distance conversion relationship includes: The average optical flow vector is obtained by averaging all candidate optical flow vectors in the candidate optical flow vector set. The average optical flow vector includes a first pixel displacement component along the horizontal direction of the image and a second pixel displacement component along the vertical direction of the image. Based on the pixel-actual distance conversion relationship, the first pixel displacement component and the second pixel displacement component are respectively converted into a first actual spatial displacement component and a second actual spatial displacement component. Based on the first actual spatial displacement component, the second actual spatial displacement component, and the time interval between adjacent real-time acquired images, the first velocity component along the horizontal direction of the image and the second velocity component along the vertical direction of the image are determined respectively. The water flow velocity of the target water area is determined based on the first velocity component and the second velocity component.

[0011] Optionally, after determining the water flow velocity of the target water area, the method further includes: The image coordinates of each candidate feature point in the candidate feature point set are transformed into spatial coordinates through perspective transformation, and the spatial coordinates are associated with their corresponding candidate optical flow vectors to generate a discrete vector field. The discrete vector field is spatially interpolated using the radial basis function interpolation method to obtain a continuous velocity distribution map of the entire target water area, and the continuous velocity distribution map is output as a pseudo-color image superimposed with arrow vectors.

[0012] According to another aspect of the present invention, a computer-readable storage medium is also provided, on which a computer program is stored, wherein the computer program, when executed by a processor, implements the steps of any of the above-described water flow velocity measurement methods.

[0013] According to another aspect of the present invention, a computer program product is also provided, comprising a computer program that, when executed by a processor, implements the steps of any of the above-described water flow velocity measurement methods.

[0014] According to another aspect of the present invention, a computer device is also provided, which includes a memory, a processor, and a computer program stored in the memory, wherein the processor executes the computer program to implement the steps of any water flow velocity measurement method.

[0015] The water flow velocity measurement method of this invention obtains standardized image sequences, unifying the pixel size and coordinate system of each real-time acquired image, providing high-quality and consistent input data for subsequent feature extraction and optical flow calculation, effectively eliminating calculation errors introduced by differences in image size and viewpoint distortion. By using the SIFT algorithm to extract and filter candidate feature points from adjacent real-time acquired images, a stable and reliable set of candidate feature points is obtained, effectively avoiding mismatch problems caused by uneven distribution and easy overlap of feature points on the water flow surface. By using the RAFT optical flow model to analyze the optical flow field of adjacent real-time acquired images and combining it with historical optical flow datasets for machine learning training, the optical flow model can adapt to complex water flow motion patterns, effectively capturing the global continuous motion trajectory of the water flow. By determining the corresponding optical flow vector from the dense optical flow field based on the coordinates of candidate feature points and setting the optical flow vectors in non-candidate feature point regions to zero vectors, the method utilizes both the stable feature point extraction capability of the SIFT algorithm and the global continuous motion capture capability of the RAFT optical flow model, while eliminating invalid optical flow interference in non-feature point regions, significantly improving the effectiveness and accuracy of the optical flow vector. By pre-setting the pixel-to-actual distance conversion relationship, the pixel-level displacement in the candidate optical flow vector set is converted into the physical displacement in the actual space, realizing the accurate mapping between the image coordinate system and the real-world coordinate system, eliminating the measurement deviation caused by the difference in camera angle and image resolution, ensuring that the flow velocity calculation results strictly correspond to the actual physical scene, and improving the authenticity and reliability of the velocity measurement results.

[0016] Furthermore, the water flow velocity measurement method of the present invention constructs a Gaussian pyramid and a difference-of-Gaussian pyramid, extracts initial feature points based on extreme point detection and key point localization, and assigns key point orientation values ​​and generates feature descriptors based on the Gaussian pyramid. This ensures that the extracted feature points possess scale invariance and rotation invariance, guaranteeing the extraction of stable and matchable feature points under different water flow conditions. The method employs Euclidean distance matching to match initial feature points in adjacent frames, and combines a dual screening mechanism of distance ratio threshold and response value threshold to eliminate poor matching quality and unstable feature point correspondences. This effectively filters weak feature points and mismatched points caused by factors such as water surface reflection, ripple disturbance, and illumination changes, ensuring that the retained candidate feature points possess high stability and matching reliability.

[0017] Furthermore, the water flow velocity measurement method of this invention obtains an average optical flow vector by averaging the candidate optical flow vector set, effectively eliminating displacement deviations caused by mismatches or local disturbances at individual feature points. By processing pixel displacement components along the horizontal and vertical directions of the image separately, converting them into actual spatial displacement components, and independently calculating the flow velocity component, the final water flow velocity is synthesized. This method can accurately capture the differences in water flow motion in the horizontal and vertical directions, avoiding errors that may be introduced by displacement calculation in a single direction, and significantly improving the accuracy of flow velocity calculation. Through the pixel-to-actual distance conversion relationship, the horizontal and vertical pixel displacement components are converted into actual spatial displacement components, achieving a precise mapping between the image coordinate system and the real-world coordinate system. It can use the same or different conversion coefficients for different directions, effectively eliminating scale differences in different directions caused by hardware devices, and ensuring that the flow velocity calculation results strictly correspond to the actual physical scene.

[0018] The above and other objects, advantages and features of the present invention will become more apparent to those skilled in the art from the following detailed description of specific embodiments of the invention in conjunction with the accompanying drawings. Attached Figure Description

[0019] The following sections will describe some specific embodiments of the invention in detail by way of example and not limitation, with reference to the accompanying drawings. The same reference numerals in the drawings denote the same or similar parts or portions. Those skilled in the art should understand that these drawings are not necessarily drawn to scale. In the drawings: Figure 1 This is a schematic flowchart of a water flow velocity measurement method according to an embodiment of the present invention; Figure 2 This is a schematic flowchart of a water flow velocity measurement method according to an embodiment of the present invention for obtaining a standardized image sequence; Figure 3 This is a schematic flowchart of a water flow velocity measurement method according to an embodiment of the present invention for obtaining a set of candidate feature points; Figure 4 This is a schematic flowchart of abnormal optical flow point processing in a water flow velocity measurement method according to an embodiment of the present invention; Figure 5 This is a schematic flowchart of a water flow velocity determination method according to an embodiment of the present invention. Figure 6 This is a schematic diagram of a computer program product according to an embodiment of the present invention; Figure 7 This is a schematic diagram of a computer-readable storage medium according to an embodiment of the present invention; Figure 8 This is a schematic diagram of a computer device according to an embodiment of the present invention. Detailed Implementation

[0020] Current non-contact water flow velocity measurement technologies mainly fall into two categories: image feature matching-based methods and model-based methods. Image feature matching-based methods extract naturally occurring feature points on the water flow surface (such as ripples, floating objects, and bubbles), calculate the pixel displacement of feature points between adjacent frames, and then combine this with the physical scale conversion relationship of the image to obtain the actual water flow velocity. However, this type of method can only calculate the velocity through the inter-frame displacement of discrete feature points, and cannot capture the continuous motion trajectory of the water flow, making it difficult to characterize the motion law of the entire flow field, resulting in a lack of global representativeness in the velocity calculation results. At the same time, the uneven distribution, easy overlap, and easy change of feature points on the water flow surface, as well as the problems of mismatch and missed match between feature points, can cause displacement calculation deviations. Furthermore, it is sensitive to environmental interferences such as changes in illumination and water surface reflection, which further aggravates the matching error and seriously affects the accuracy of velocity calculation.

[0021] Model-based velocity measurement methods construct optical flow field models and perform dense calculations on the global pixel motion between adjacent frames to obtain the motion vector of each pixel, thereby acquiring global motion information of the water flow. However, these methods are susceptible to natural interferences such as water surface reflection, ripple disturbances, and changes in illumination. The complex optical characteristics of the water surface lead to the violation of the assumption of constant grayscale, resulting in a large number of invalid and anomalous vectors in the optical flow field calculation results. At the same time, these methods have high requirements for the continuity of motion between frames. When the water flow velocity is uneven, or when there are eddies or turbulence, the sudden change in local motion patterns exceeds the range of model assumptions, causing deviations in optical flow vectors. In addition, the water flow state in outdoor rivers is complex and variable, and existing optical flow models are difficult to uniformly adapt to various motion patterns such as steady laminar flow, turbulent flow, eddies, and backflow, lacking stable and reliable scene adaptability.

[0022] Therefore, how to achieve non-contact water flow velocity measurement that is anti-interference, highly accurate, and in real-time has become a technical problem that urgently needs to be solved in this field.

[0023] This embodiment provides a water flow velocity calculation scheme that integrates the RAFT optical flow model and the SIFT algorithm. Without changing the existing hardware deployment, it achieves accurate capture of the continuous water flow trajectory and reliable matching of stable feature points, effectively improving velocity measurement accuracy and anti-interference capability.

[0024] Figure 1 This is a schematic flowchart of a water flow velocity measurement method according to an embodiment of the present invention, as follows: Figure 1 As shown, this water flow velocity measurement method generally includes: Step S101: Obtain a standardized image sequence of the target water area. The standardized image sequence contains multiple real-time acquired images arranged in chronological order. The time interval between each adjacent real-time acquired image is equal, and each real-time acquired image has the same pixel size. The images are then corrected to the same coordinate system through perspective transformation. Step S102: Using the SIFT algorithm, candidate feature points are extracted from every two adjacent real-time acquired images in the standardized image sequence, and then filtered according to preset screening criteria to obtain a set of candidate feature points. Step S103: The RAFT optical flow model is used to perform optical flow field analysis based on every two adjacent real-time acquired images to obtain a dense optical flow field. The RAFT optical flow model is obtained by machine learning training on the historical optical flow dataset of the target water area. The historical optical flow dataset contains multiple sets of historical acquired images and their corresponding optical flow field annotations. The optical flow field annotation is the displacement vector of each pixel in each set of historical acquired images. Step S104: Based on the coordinates of each candidate feature point in the candidate feature point set, determine the optical flow vector corresponding to the candidate feature point from the dense optical flow field, and set the optical flow vector of the non-candidate feature point region in the dense optical flow field to the zero vector to obtain the candidate optical flow vector set corresponding to the candidate feature point set. Step S105: Determine the water flow velocity of the target water area based on the candidate optical flow vector set and the preset pixel-actual distance conversion relationship.

[0025] In this embodiment, camera equipment deployed above the target water area includes fixed mounting equipment, gimbal camera equipment mounted on a rotary-wing UAV, multispectral imaging equipment, thermal imaging equipment, and lidar. The camera equipment continuously acquires images of the water surface at a preset frame rate, obtaining a raw video stream containing a time dimension. Frame extraction is performed on the acquired raw video stream to obtain a real-time image sequence arranged in chronological order. Each real-time image is normalized to a preset pixel size, eliminating scale inconsistencies caused by differences in camera resolution. Simultaneously, perspective correction is performed on each real-time image based on a pre-calibrated perspective transformation matrix, mapping the image coordinate system to a unified world coordinate system, eliminating geometric distortion introduced by the camera installation angle, and obtaining a standardized image sequence. In this standardized image sequence, the time interval between adjacent real-time images is constant and equal to the reciprocal of the preset frame rate.

[0026] In this embodiment, the SIFT (Scale-Invariant Feature Transform) algorithm is used to extract initial feature points from each real-time acquired image. Each initial feature point includes a response value characterizing the stability of the feature point and a feature descriptor characterizing the local image features of the feature point. A feature matching algorithm is used to match the initial feature points between adjacent real-time acquired images to determine the correspondence between feature points in adjacent real-time acquired images, and the matching confidence of each correspondence is obtained. The above correspondences are filtered according to a preset threshold, eliminating correspondences with matching confidence below the preset threshold and correspondences with feature point response values ​​below the preset threshold. The feature points in the retained correspondences after filtering are used as candidate feature points and categorized according to their respective real-time acquired images to form a candidate feature point set. It should be noted that, in addition to the SIFT algorithm, the SURF algorithm, LoFTR algorithm, Kanade-Lucas-Tomasi algorithm, PatchMatch algorithm, and other feature extraction algorithms can also be used for the initial feature point extraction operation. The specific extraction operation is the same as that of the SIFT algorithm, and will not be described in detail here.

[0027] In this embodiment, a pre-trained RAFT (Recurrent All-Pairs FieldTransforms) optical flow model is loaded. The training process of this RAFT optical flow model includes: acquiring a historical optical flow dataset of the target water area; the historical optical flow dataset contains multiple sets of training samples, each set of training samples consisting of two consecutive historical acquisition images and their corresponding optical flow field annotations, wherein the optical flow field annotations are the displacement vectors of each pixel in the two corresponding historical acquisition images (i.e., the motion displacement of each pixel from the first frame to the second frame). The RAFT model is trained using the above training samples, enabling the RAFT model to learn the mapping relationship from image pairs to dense optical flow fields. After training, the RAFT model has the ability to accurately estimate the optical flow of the target water area. The two adjacent real-time acquisition images obtained in step S101 are input into the loaded RAFT optical flow model. The RAFT optical flow model performs forward inference calculations on the input real-time acquired images based on the mapping relationship it has learned, and outputs a dense optical flow field with the same resolution as the input image. The optical flow vector of each pixel in the dense optical flow field represents the motion displacement of that pixel between two adjacent real-time acquired images (in pixels / frame).

[0028] In this embodiment, the candidate feature point set obtained in step S102 contains the coordinate information of all candidate feature points in the current real-time acquired image. The dense optical flow field obtained in step S103 has the same spatial resolution as the current real-time acquired image. Each pixel corresponds to a two-dimensional optical flow vector, representing the horizontal and vertical displacements of the pixel. Each candidate feature point in the candidate feature point set is traversed, and the corresponding optical flow vector is retrieved from the dense optical flow field based on its coordinate position in the image. An optical flow mask matrix of the same size as the dense optical flow field is generated, preserving the optical flow vectors at the locations of the candidate feature points and setting all optical flow vectors in non-candidate feature point regions to zero. The optical flow vectors corresponding to all the retained candidate feature points are organized according to the feature point order to form a candidate optical flow vector set that corresponds one-to-one with the candidate feature point set.

[0029] In this embodiment, the optical flow vectors in the candidate optical flow vector set represent the pixel-level motion displacement of the corresponding candidate feature point in two adjacent real-time acquired images. The pre-establishment of the pixel-to-actual distance conversion relationship includes: deploying reference objects with known actual distances in the target water area; acquiring standardized images containing the reference objects and measuring the pixel distances corresponding to the reference objects in the standardized images; dividing the actual distance by the pixel distance to obtain the pixel-to-actual distance conversion coefficient, which represents the actual spatial distance corresponding to a unit pixel. Based on the pixel-to-actual distance conversion relationship, each optical flow vector in the candidate optical flow vector set is converted into an actual spatial displacement vector. The time interval between adjacent real-time acquired images is obtained, which is determined by the reciprocal of the preset frame rate. Based on the ratio of the actual spatial displacement vector to the time interval, the instantaneous flow velocity corresponding to each candidate feature point is determined. Statistical analysis is performed on the instantaneous flow velocities corresponding to each candidate feature point to obtain the water flow velocity of the target water area.

[0030] Based on the above steps, by acquiring standardized image sequences and unifying the pixel size and coordinate system of each real-time acquired image, high-quality and consistent input data is provided for subsequent feature extraction and optical flow calculation, effectively eliminating calculation errors introduced by differences in image size and viewpoint distortion. By using the SIFT algorithm to extract and filter candidate feature points from adjacent real-time acquired images, a stable and reliable set of candidate feature points is obtained, effectively avoiding mismatch problems caused by uneven distribution and easy overlap of feature points on the water flow surface. By using the RAFT optical flow model to analyze the optical flow field of adjacent real-time acquired images and combining it with historical optical flow datasets for machine learning training, the optical flow model can adapt to complex water flow motion patterns and effectively capture the global continuous motion trajectory of the water flow. By determining the corresponding optical flow vector from the dense optical flow field based on the coordinates of candidate feature points and setting the optical flow vectors in non-candidate feature point regions to zero, the stable feature point extraction capability of the SIFT algorithm and the global continuous motion capture capability of the RAFT optical flow model are utilized, while eliminating invalid optical flow interference from non-feature point regions, significantly improving the effectiveness and accuracy of the optical flow vectors. By pre-setting the pixel-to-actual distance conversion relationship, the pixel-level displacement in the candidate optical flow vector set is converted into the physical displacement in the actual space, realizing the accurate mapping between the image coordinate system and the real-world coordinate system, eliminating the measurement deviation caused by the difference in camera angle and image resolution, ensuring that the flow velocity calculation results strictly correspond to the actual physical scene, and improving the authenticity and reliability of the velocity measurement results.

[0031] Figure 2 This is a schematic flowchart illustrating the acquisition of a standardized image sequence in a water flow velocity measurement method according to an embodiment of the present invention, as shown below. Figure 2 As shown, the steps for obtaining a standardized image sequence of the target water area include: Step S201: Obtain the original video stream of the target water area according to the preset frame rate; Step S202: Select multiple original captured images from the original video stream according to a preset time interval; Step S203: Normalize the size of each original acquired image to a preset target size; Step S204: Perform perspective correction on the original acquired image after size normalization, transforming the river area in the original acquired image into a rectangle; Step S205: Denoise reduction is performed on the original acquired image after perspective correction to obtain a standardized image sequence containing multiple real-time acquired images.

[0032] In this embodiment, the time interval between adjacent frames is determined according to a preset frame rate. This time interval is always equal to the reciprocal of the preset frame rate (for example, when the preset frame rate is 25 frames / second, the time interval is 0.04 seconds). According to this constant time interval, multiple original captured images are uniformly selected from the original video stream to ensure that the selected images are uniformly distributed in the time dimension and that the time interval between adjacent selected images is consistent.

[0033] In this embodiment, for each original acquired image, an interpolation algorithm (such as bilinear interpolation) is used to scale it to a preset target size (e.g., 1280×720 pixels). The interpolation algorithm determines the new pixel value by calculating the weighted average of the four neighboring pixels around the target pixel in the original acquired image, maintaining image smoothness while considering computational efficiency. After size normalization, all original acquired images have the same width and height.

[0034] In this embodiment, the river region under the current field of view is identified in the original acquired image, and the upper edge point of the left bank, the upper edge point of the right bank, the lower edge point of the right bank, and the lower edge point of the left bank are selected as source control points. According to the preset target size, four corresponding target control points are set. Based on the source control points and target control points, a perspective transformation matrix is ​​calculated. This perspective transformation matrix describes the projection mapping relationship from a two-dimensional image plane to another two-dimensional image plane. The original acquired image with normalized size is transformed through the perspective transformation matrix to correct the river region into a rectangle, and the non-river region is filled with a preset value (such as black), so that the river flow direction is parallel to the horizontal direction of the transformed image.

[0035] In this embodiment, the original acquired image after perspective correction is denoised using filtering algorithms, including Gaussian filtering, median filtering, and bilateral filtering. As an example, the image is smoothed by calculating the weighted average of each pixel and its neighboring pixels. The weights are determined by a Gaussian function, with pixels closer to the center pixel having a higher weight. The preset kernel size of the Gaussian filter is N×N pixels, where N is an odd number not less than 3 and does not exceed a preset proportion of the number of pixels on the shorter side of the image. The standard deviation σ of the preset Gaussian function and the kernel size N satisfy σ = N / c, where c is a positive integer greater than 3. After denoising, an enhanced, standardized image sequence is obtained.

[0036] Based on the above steps, size normalization ensures all images have a uniform width and height, eliminating scale inconsistencies caused by differences in camera resolution or digital zoom. Perspective correction transforms the river region in the original acquired image into a rectangle, eliminating perspective distortion introduced by geometric factors such as camera installation height, pitch angle, and yaw angle. Noise reduction eliminates various noise interferences introduced during image acquisition, resulting in smoother grayscale distribution and clearer edge features, reducing the probability of mismatches caused by noise interference.

[0037] Figure 3 This is a schematic flowchart illustrating the acquisition of a candidate feature point set in a water flow velocity measurement method according to an embodiment of the present invention, as shown below. Figure 3 As shown, the steps for extracting candidate feature points from every two adjacent real-time acquired images in a standardized image sequence using the SIFT algorithm, and then filtering them according to preset criteria to obtain a set of candidate feature points include: Step S301: Construct a Gaussian pyramid and a difference-of-Gaussian pyramid for each real-time acquired image. Perform extreme point detection and key point localization based on the difference-of-Gaussian pyramid. Assign key point orientation and generate feature descriptors based on the Gaussian pyramid. Extract initial feature points for each real-time acquired image, where each initial feature point contains a feature point response value. Step S302: Using the Euclidean distance matching method, the initial feature points of two adjacent real-time acquired images are matched to determine the correspondence between adjacent real-time acquired images and the distance ratio of each pair of correspondences. Step S303: The preset screening indicators include distance ratio threshold and response value threshold. The corresponding relationships that meet any of the following conditions are eliminated: the distance ratio is greater than the distance ratio threshold, or the feature point response value is less than the response value threshold. Step S304: The initial feature points in the remaining correspondences are taken as candidate feature points, and the candidate feature points are classified according to the real-time acquired images to obtain the candidate feature point set for each real-time acquired image.

[0038] In this embodiment, the real-time acquired image is subjected to continuous Gaussian filtering and downsampling to generate a multi-layer, multi-scale image representation. Each layer of the image is obtained by convolving the image with a Gaussian kernel of different scale parameters. The images at adjacent scales in the Gaussian pyramid are subtracted to obtain a Gaussian difference pyramid. In the Gaussian difference pyramid, each pixel is compared with its 8 spatial neighbors at the same scale and 9 corresponding points at adjacent scales (a total of 26 points) to detect local extrema. The detected extrema are precisely located at the sub-pixel level using Taylor expansion, and low-contrast extrema and those with strong responses located at image edges are removed to obtain stable keypoints. Based on the Gaussian-smoothed image at the scale of the keypoint, the gradient magnitude and direction of each pixel in the keypoint's neighborhood are calculated, and a gradient direction histogram is constructed. The main peak direction of the histogram is used as the main direction of the keypoint. If there are other peaks in the histogram whose magnitudes exceed a preset proportion (e.g., 80%) of the main peak, multiple directions are assigned to the keypoint to enhance the robustness of the matching. Centered on a keypoint, its neighborhood is divided into several sub-regions. A gradient direction histogram is calculated in each sub-region. The gradient direction histograms of each sub-region are concatenated and stitched together in a preset order to form a feature descriptor vector. This descriptor has illumination invariance through gradient magnitude normalization and rotation invariance by rotating the coordinates and gradient direction to the principal direction of the keypoint. It can uniquely represent the local image features around the keypoint. Each keypoint includes its coordinates, scale, principal direction, feature descriptor, and contrast response value in the image.

[0039] In this embodiment, initial feature points and their corresponding feature descriptors are obtained from two real-time acquired images. Euclidean distance is used as a similarity measure between feature descriptors. For each initial feature point in the previous real-time acquired image, the Euclidean distance between its feature descriptor and the feature descriptors of all initial feature points in the subsequent real-time acquired image is calculated. The nearest and second nearest initial feature points are found in the subsequent real-time acquired image, and the ratio of the nearest distance to the second nearest distance is recorded. This ratio is represented as the distance ratio. The smaller the distance ratio, the more significant the advantage of nearest neighbor matching over second nearest neighbor matching, and the higher the uniqueness of the match. The larger the distance ratio, the lower the distinguishability between the nearest and second nearest neighbors, and the greater the possibility of ambiguity in the match. The feature point response value reflects the salience of the local image structure of the initial feature point. Initial feature points with excessively small response values ​​usually correspond to weak texture regions in the image and have poor stability.

[0040] Based on the above steps, Gaussian pyramids and difference-of-Gaussian pyramids are constructed. Initial feature points are extracted based on extreme point detection and keypoint localization. Keypoint orientation assignment and feature descriptor generation are then performed based on the Gaussian pyramids, ensuring that the extracted feature points possess scale invariance and rotation invariance. This guarantees the extraction of stable and matchable feature points under different water flow conditions. Euclidean distance matching is used to match the initial feature points of adjacent frames. A dual screening mechanism combining distance ratio thresholds and response value thresholds is employed to eliminate poorly matched and unstable correspondences. This effectively filters out weak and mismatched feature points caused by factors such as water surface reflection, ripple disturbance, and lighting changes, ensuring that the retained candidate feature points possess high stability and matching reliability.

[0041] In the implementation of step S104, determining the optical flow vector corresponding to each candidate feature point from the dense optical flow field based on the coordinates of each candidate feature point in the candidate feature point set includes: If the coordinates of the candidate feature point are integer coordinates, a neighborhood window of a preset size is defined in the dense optical flow field with the integer coordinates as the center. The optical flow vectors of all pixels in the neighborhood window are extracted, and the median or mean vector of the optical flow vectors of all pixels is used as the optical flow vector corresponding to the candidate feature point. If the coordinates of the candidate feature point are non-integer coordinates, the optical flow vector corresponding to the candidate feature point is obtained by interpolating from the optical flow vectors of the neighboring integer pixels around the non-integer coordinate position using bilinear interpolation.

[0042] Based on the above implementation, differentiated optical flow vector determination strategies are adopted for different types of candidate feature point coordinates (integer coordinates and non-integer coordinates). For integer coordinate positions, by defining a neighborhood window and calculating the median or mean vector of the optical flow vectors of all pixels within the window, local optical flow calculation noise is effectively suppressed, avoiding the influence of outlier optical flow values ​​of a single pixel on the results. For non-integer coordinate positions, bilinear interpolation is used to interpolate the corresponding optical flow vector from the optical flow vectors of surrounding integer pixels, ensuring the accuracy of optical flow calculation for sub-pixel-level feature points. The differentiated strategy ensures that accurate and stable optical flow vectors can be obtained regardless of the type of candidate feature point coordinates.

[0043] Figure 4 This is a schematic flowchart of abnormal optical flow point processing in a water flow velocity measurement method according to an embodiment of the present invention, as shown below. Figure 4 As shown, before setting the optical flow vector of the non-candidate feature point region in the dense optical flow field to zero, the process also includes: Step S401: Determine the local neighborhood variance of the optical flow vector of each pixel in the dense optical flow field, and identify the pixels whose local neighborhood variance exceeds the preset fluctuation threshold as abnormal optical flow points. Step S402: For each candidate feature point in the dense optical flow field, determine the pixel point on which the optical flow vector corresponding to the candidate feature point depends. If the dependent pixel point contains an abnormal optical flow point, remove the optical flow vector corresponding to the candidate feature point.

[0044] Based on the above steps, anomaly detection is performed using the statistical characteristics of the optical flow field itself. This method eliminates the need for external reference data and effectively identifies outliers, noise points, and abnormal optical flow regions caused by factors such as water surface reflection and ripple disturbance during optical flow calculation, thus eliminating unreliable optical flow data at its source. For each candidate feature point, the corresponding pixel on which the optical flow vector depends is determined, and it is judged whether the dependent pixel contains an abnormal optical flow point. If it does, the optical flow vector corresponding to that candidate feature point is removed. This effectively eliminates the influence of local optical flow calculation errors (such as incorrect optical flow estimation in water surface reflection areas or sudden changes in local optical flow caused by ripple disturbance) on the flow velocity calculation results. The retained candidate optical flow vectors all originate from stable and reliable regions in the optical flow field, making the subsequent water flow velocity determined based on the candidate optical flow vector set more accurate and stable.

[0045] Figure 5 This is a schematic flowchart illustrating the determination of water flow velocity in a water flow velocity measurement method according to an embodiment of the present invention, as shown below. Figure 5 As shown, the steps for determining the water flow velocity of the target water area based on the candidate optical flow vector set and the preset pixel-to-actual distance conversion relationship include: Step S501: Perform vector averaging on all candidate optical flow vectors in the candidate optical flow vector set to obtain an average optical flow vector. The average optical flow vector includes a first pixel displacement component along the horizontal direction of the image and a second pixel displacement component along the vertical direction of the image. Step S502: Based on the pixel-actual distance conversion relationship, the first pixel displacement component and the second pixel displacement component are converted into the first actual spatial displacement component and the second actual spatial displacement component, respectively. Step S503: Based on the first actual spatial displacement component, the second actual spatial displacement component, and the time interval between adjacent real-time acquired images, determine the first velocity component along the horizontal direction of the image and the second velocity component along the vertical direction of the image, respectively. Step S504: Determine the water flow velocity of the target water area based on the first velocity component and the second velocity component.

[0046] Based on the above steps, an average optical flow vector is obtained by averaging the candidate optical flow vector set, effectively eliminating displacement deviations caused by mismatches or local disturbances at individual feature points. By processing the pixel displacement components along the horizontal and vertical directions of the image separately, converting them into actual spatial displacement components, and then independently calculating the flow velocity components, the final water flow velocity is synthesized. This accurately captures the differences in water flow motion in the horizontal and vertical directions, avoiding errors that may be introduced by displacement calculation in a single direction, and significantly improving the accuracy of flow velocity calculation. Through the pixel-to-actual distance conversion relationship, the horizontal and vertical pixel displacement components are converted into actual spatial displacement components, achieving a precise mapping between the image coordinate system and the real-world coordinate system. The same or different conversion coefficients can be used for different directions, effectively eliminating scale differences in different directions caused by hardware devices, and ensuring that the flow velocity calculation results strictly correspond to the actual physical scene.

[0047] It should be noted that after determining the water flow velocity of the target water area, the process also includes: converting the image coordinates of each candidate feature point in the candidate feature point set into spatial coordinates through perspective transformation, associating the spatial coordinates with their corresponding candidate optical flow vectors to generate a discrete vector field; using the radial basis function interpolation method to perform spatial interpolation on the discrete vector field to obtain a continuous velocity distribution map of the entire target water area, and outputting the continuous velocity distribution map by superimposing arrow vectors on a pseudo-color image.

[0048] By expanding the discrete and sparse flow velocity information of candidate feature points into a continuous flow velocity distribution map covering the entire target water area through spatial coordinate transformation and radial basis function interpolation, the flow velocity at any location within the target water area can be estimated, providing a data foundation for comprehensive analysis of water flow characteristics. By converting the image coordinates of candidate feature points into actual spatial coordinates through perspective transformation, a precise mapping between the image coordinate system and the real-world coordinate system is achieved. This ensures that each vector in the discrete vector field has a clear physical location meaning, enabling the subsequently interpolated full-area flow velocity distribution map to accurately reflect the actual spatial layout of the target water area and avoiding spatial position deviations caused by factors such as camera installation angle and perspective distortion.

[0049] The flowchart provided in this embodiment is not intended to indicate that the operations of the method will be performed in any particular order, or that all operations of the method are included in every case. Furthermore, the method may include additional operations. Within the scope of the technical concept provided by the method in this embodiment, additional variations can be made to the above method.

[0050] It should be understood that in some embodiments, the components may be implemented using hardware, software, firmware, or a combination thereof. In the above embodiments, multiple steps or methods may be implemented using software or firmware stored in memory and executed by a suitable instruction execution system.

[0051] This embodiment also provides a computer program product 10, a computer-readable storage medium 20, and a computer device 30. Figure 6 This is a schematic diagram of a computer program product 10 according to an embodiment of the present invention. Figure 7 This is a schematic diagram of a computer-readable storage medium 20 according to an embodiment of the present invention. Figure 8 This is a schematic diagram of a computer device 30 according to an embodiment of the present invention. Figure 6 As shown, the computer program product 10 includes a computer program 11, which, when executed by the processor 32, implements the steps of any of the above-described water flow velocity measurement methods. For example... Figure 7 As shown, a computer-readable storage medium 20 stores the aforementioned computer program 11, which, when executed by the processor 32, implements the steps of the water flow velocity measurement method of any of the above embodiments. Figure 8 As shown, the computer device 30 may include a memory 31, a processor 32, and a computer program 11 stored on the memory 31 and running on the processor 32.

[0052] The computer program 11 used to perform the operations of this invention may be assembly instructions, Instruction Set Architecture (ISA) instructions, machine instructions, machine-dependent instructions, microcode, firmware instructions, status setting data, integrated circuit configuration data, or source code or object code written in any combination of one or more programming languages ​​and procedural programming languages. The computer program 11 may execute entirely on the user's computer, partially on the user's computer, as a standalone software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In the latter case, the remote computer may be connected to the user's computer via any type of network, including a Local Area Network (LAN) or a Wide Area Network (WAN), or may be connected to an external computer. In some embodiments, to perform aspects of this invention, electronic circuits, including, for example, programmable logic circuits, field-programmable gate arrays (FPGAs), or programmable logic arrays (PLAs), may execute computer-readable program instructions to personalize the electronic circuits by utilizing status information of the computer-readable program instructions.

[0053] For the purposes of this embodiment, computer program product 10 is a related product that includes computer program 11.

[0054] For the purposes of this embodiment, computer-readable storage medium 20 is a tangible device capable of holding and storing a computer program 11. It can be any device capable of containing, storing, communicating, propagating, or transmitting the program 11 for use by or in conjunction with an instruction execution system, apparatus, or device. More specific examples (a non-exhaustive list) of computer-readable storage medium 20 include: portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), static random access memory (SRAM), portable optical disc read-only memory (CD-ROM), digital versatile disc (DVD), memory stick, floppy disk, mechanical encoding device, and any suitable combination thereof.

[0055] Computer device 30 can be, for example, a server, desktop computer, laptop computer, tablet computer, or smartphone. In some examples, computer device 30 can be a cloud computing node. Computer device 30 can be described in the general context of computer system executable instructions (such as program modules) executed by a computer system. Typically, program modules can include routines, programs, object programs, components, logic, data structures, etc., that perform specific tasks or implement specific abstract data types. Computer device 30 can be implemented in a distributed cloud computing environment where tasks are performed by remote processing devices linked through a communication network. In a distributed cloud computing environment, program modules can reside on local or remote computing system storage media, including storage devices.

[0056] Computer device 30 may include a processor 32 adapted to execute stored instructions and a memory 31 that provides temporary storage space for the operation of said instructions during operation. The processor 32 may be a single-core processor, a multi-core processor, a computing cluster, or any other configuration. The memory 31 may include random access memory (RAM), read-only memory, flash memory, or any other suitable storage system.

[0057] Computer device 30 may also include a network adapter / interface and an input / output (I / O) interface. The I / O interface allows external devices that can be connected to the computer device to input and output data. The network adapter / interface provides communication between the computer device and a network, typically represented as a communication network.

[0058] Therefore, those skilled in the art should recognize that although numerous exemplary embodiments of the present invention have been shown and described in detail herein, many other variations or modifications conforming to the principles of the present invention can be directly determined or derived from the disclosure of the present invention without departing from the spirit and scope of the invention. Thus, the scope of the present invention should be understood and construed as covering all such other variations or modifications.

Claims

1. A method for measuring water flow velocity, characterized in that, include: A standardized image sequence of the target water area is obtained. The standardized image sequence contains multiple real-time acquired images arranged in chronological order. The time interval between each adjacent real-time acquired image is equal, and each real-time acquired image has the same pixel size. The images are also corrected to the same coordinate system by perspective transformation. The SIFT algorithm is used to extract candidate feature points from every two adjacent real-time acquired images in the standardized image sequence, and the candidate feature points are filtered according to preset screening criteria to obtain a set of candidate feature points. The RAFT optical flow model is used to perform optical flow field analysis based on every two adjacent real-time acquired images to obtain a dense optical flow field. The RAFT optical flow model is obtained by machine learning training on the historical optical flow dataset of the target water area. The historical optical flow dataset contains multiple sets of historical acquired images and their corresponding optical flow field annotations. The optical flow field annotations are the displacement vectors of each pixel in each set of historical acquired images. Based on the coordinates of each candidate feature point in the candidate feature point set, the optical flow vector corresponding to the candidate feature point is determined from the dense optical flow field, and the optical flow vector of the non-candidate feature point region in the dense optical flow field is set to zero vector, thus obtaining the candidate optical flow vector set corresponding to the candidate feature point set. The water flow velocity of the target water area is determined based on the candidate optical flow vector set and the preset pixel-actual distance conversion relationship.

2. The water flow velocity measurement method according to claim 1, characterized in that, The steps to obtain a standardized image sequence of the target water area include: The original video stream of the target water area is obtained according to a preset frame rate; Multiple original captured images are selected from the original video stream according to a preset time interval; The dimensions of each of the original acquired images are normalized to a preset target size; Perspective correction is performed on the original acquired image after the size normalization is completed, transforming the river region in the original acquired image into a rectangle; The original acquired image after perspective correction is denoised to obtain the standardized image sequence containing multiple real-time acquired images.

3. The water flow velocity measurement method according to claim 1, characterized in that, The steps of extracting candidate feature points from every two adjacent real-time acquired images in the standardized image sequence using the SIFT algorithm, and filtering them according to preset screening criteria to obtain a set of candidate feature points include: For each of the real-time acquired images, a Gaussian pyramid and a difference-of-Gaussian pyramid are constructed respectively. Extreme point detection and key point localization are performed based on the difference-of-Gaussian pyramid. Key point orientation assignment and feature descriptor generation are performed based on the Gaussian pyramid. Initial feature points of each of the real-time acquired images are extracted, wherein each initial feature point contains a feature point response value. The Euclidean distance matching method is used to match the initial feature points of two adjacent real-time acquired images to determine the correspondence between adjacent real-time acquired images and the distance ratio of each pair of correspondences; The preset filtering criteria include a distance ratio threshold and a response value threshold. Correspondences that meet any of the following conditions are eliminated: the distance ratio is greater than the distance ratio threshold, or the feature point response value is less than the response value threshold. The initial feature points in the remaining correspondences are used as candidate feature points, and the candidate feature points are classified according to the real-time acquired images to obtain a set of candidate feature points for each real-time acquired image.

4. The water flow velocity measurement method according to claim 1, characterized in that, Determining the optical flow vector corresponding to each candidate feature point from the dense optical flow field based on the coordinates of each candidate feature point in the candidate feature point set includes: If the coordinates of the candidate feature point are integer coordinates, a neighborhood window of a preset size is defined in the dense optical flow field with the integer coordinates as the center, the optical flow vectors of all pixels in the neighborhood window are extracted, and the median or mean vector of the optical flow vectors of all pixels is used as the optical flow vector corresponding to the candidate feature point. If the coordinates of the candidate feature point are non-integer coordinates, the optical flow vector corresponding to the candidate feature point is obtained by interpolating from the optical flow vectors of the adjacent integer pixels around the non-integer coordinate position using bilinear interpolation.

5. The water flow velocity measurement method according to claim 1, characterized in that, Before setting the optical flow vector of the non-candidate feature point region in the dense optical flow field to zero, the method further includes: Determine the local neighborhood variance of the optical flow vector of each pixel in the dense optical flow field, and identify pixels whose local neighborhood variance exceeds a preset fluctuation threshold as abnormal optical flow points. For each candidate feature point in the dense optical flow field, determine the pixel point on which the optical flow vector corresponding to the candidate feature point depends. If the pixel point on which the candidate feature point depends contains the abnormal optical flow point, remove the optical flow vector corresponding to the candidate feature point.

6. The water flow velocity measurement method according to claim 1, characterized in that, The steps for determining the water flow velocity of the target water area based on the candidate optical flow vector set and the preset pixel-actual distance conversion relationship include: The average optical flow vector is obtained by averaging all candidate optical flow vectors in the candidate optical flow vector set. The average optical flow vector includes a first pixel displacement component along the horizontal direction of the image and a second pixel displacement component along the vertical direction of the image. Based on the pixel-actual distance conversion relationship, the first pixel displacement component and the second pixel displacement component are respectively converted into a first actual spatial displacement component and a second actual spatial displacement component. Based on the first actual spatial displacement component, the second actual spatial displacement component, and the time interval between adjacent real-time acquired images, the first velocity component along the horizontal direction of the image and the second velocity component along the vertical direction of the image are determined respectively. The water flow velocity of the target water area is determined based on the first velocity component and the second velocity component.

7. The water flow velocity measurement method according to claim 1, characterized in that, After determining the water flow velocity of the target water area, the method further includes: The image coordinates of each candidate feature point in the candidate feature point set are transformed into spatial coordinates through perspective transformation, and the spatial coordinates are associated with their corresponding candidate optical flow vectors to generate a discrete vector field. The discrete vector field is spatially interpolated using the radial basis function interpolation method to obtain a continuous velocity distribution map of the entire target water area, and the continuous velocity distribution map is output as a pseudo-color image superimposed with arrow vectors.

8. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by the processor, it implements the steps of the water flow velocity measurement method according to any one of claims 1 to 7.

9. A computer program product, comprising a computer program, characterized in that, When the computer program is executed by the processor, it implements the steps of the water flow velocity measurement method according to any one of claims 1 to 7.

10. A computer device, characterized in that, The method includes a memory, a processor, and a computer program stored in the memory, wherein the processor executes the computer program to implement the steps of the water flow velocity measurement method according to any one of claims 1 to 7.

Images