A method and system for calculating river surface velocity based on optical flow measurement

Abstract

The present invention discloses a method and system for calculating river surface velocity based on optical flow measurement. The method includes: using an optical flow algorithm to calculate the optical flow value of an image, calculating the offset of feature points in a flow measurement image matrix based on the optical flow value, and filtering the offset value; filtering the optical flow value of the flow measurement image matrix; calculating the river surface velocity in a pixel coordinate system, converting the pixel coordinates of each video frame image in a river flow video captured by a camera into world coordinates; calculating the spatial resolution of the image based on the pixel coordinates of two consecutive frames in the flow measurement image matrix after filtering the optical flow value and the world coordinates of the two consecutive frames; and calculating the river surface velocity in a world coordinate system using the spatial resolution of the image and the river velocity of the filtered feature points in the pixel coordinate system. The present invention can improve calculation efficiency while also improving calculation accuracy.

Description

Technical Field

[0001] The present invention relates to the technical field of flow field measurement, and in particular to a method and system for calculating river surface flow velocity based on optical flow measurement. Background Art

[0002] Flow monitoring plays a vital role in hydrological modeling and hydrological engineering. Traditional flow velocity measurement relies on instrumental methods. Most of these instruments are expensive, require a lot of time to perform, and can only perform point measurements, not comprehensive measurements.

[0003] With the advancement of computer vision, methods for extracting hydraulic information from image data are increasingly being applied to measuring river surface flow. Existing image velocimetry methods include large-scale particle image velocimetry (LSPIV), particle tracking velocimetry (PTV), and optical flow. There are two approaches to measuring river surface flow using optical flow: directly applying various optical flow methods to flow measurement, and combining them with deep learning methods.

[0004] During the implementation of the present invention, the inventors of this application discovered that the prior art methods have at least the following technical problems:

[0005] Due to the influence of vegetation, sediment, and floating objects on the river surface, rivers experience a lot of turbulence. The optical flow method shows and calculates the movement of every part of the river. The presence of a lot of turbulence leads to a large amount of calculation. Although the LK optical flow method alone can calculate optical flow, it is not sensitive to noise. When the target motion scale is too large or there is occlusion, it will produce large calculation errors due to the large motion scale, which not only affects the accuracy of the algorithm but also reduces the overall operation speed. While the incorporation of deep learning methods can improve the algorithm's calculation accuracy to a certain extent, it has extremely high requirements for equipment and the running time will increase again on the original basis, making it impossible to achieve the effect of real-time flow monitoring. Summary of the Invention

[0006] The present invention provides a method and system for calculating river surface flow velocity based on optical flow measurement, which are used to solve or at least partially solve the technical problem of low calculation efficiency existing in the prior art.

[0007] In order to solve the above technical problems, the first aspect of the present invention provides a method for calculating river surface velocity based on optical flow measurement, comprising:

[0008] S1: Use a camera to collect river flow video;

[0009] S2: extracting video frames from the collected river flow video according to a preset time interval and frame interval, and all the extracted video frames constitute a flow measurement image matrix;

[0010] S3: preprocessing the flow measurement image matrix;

[0011] S4: Use the optical flow algorithm combined with the feature detection algorithm to perform feature detection and optical flow value calculation on the image in the preprocessed flow measurement image matrix to obtain the optical flow value g of the measurement image matrix L =[l x ,l y ] T , the optical flow value of the measured image matrix represents the movement of feature points between image frames in the measured flow image matrix, where is the component parallel to the river section, is the component perpendicular to the river section;

[0012] S5: Based on the optical flow value of the flow measurement image matrix, the offset of the feature points in the flow measurement image matrix is obtained Then, the offset value ||l|| of the feature point in the flow measurement image matrix is obtained based on the Euclidean distance calculation.

[0013]

[0014] Sort the offset values from high to low, and filter the offset values of a preset proportion that are ranked low, wherein the offset value of a feature point in the flow measurement image matrix is the movement distance of the feature point between image frames in the flow measurement image matrix;

[0015] S6: filtering the optical flow value of the flow measurement image matrix;

[0016] S7: Calculate the river velocity of the filtered feature points in the pixel coordinate system based on the offset value obtained after filtering and the time of the video corresponding to the flow measurement image matrix, and select a preset portion of the river velocities of the filtered feature points in the pixel coordinate system to obtain an average value to obtain the river surface velocity in the pixel coordinate system;

[0017] S8: converting the pixel coordinates of each frame of the image in the flow measurement image matrix into world coordinates;

[0018] S9: Calculating the spatial resolution of the image according to the pixel coordinates of two consecutive frames of images in the flow measurement image matrix after filtering the optical flow value and the world coordinates of the two consecutive frames of images;

[0019] S10: Calculate the river surface velocity in the world coordinate system using the spatial resolution of the image and the river surface velocity in the pixel coordinate system.

[0020] In one embodiment, S3 pre-processes the flow measurement image matrix, including removing hue and saturation information, adjusting the hue of the image, and cropping the image boundary.

[0021] In one embodiment, an optical flow algorithm is combined with a feature detection algorithm to perform feature point detection and optical flow value calculation on the image in the preprocessed flow measurement image matrix, including:

[0022] Each frame of the flow measurement image matrix is scaled layer by layer in a pyramid layered manner, with the bottom image having the largest resolution and the top image having the smallest resolution. Starting from the top image, a feature detection algorithm is used to obtain feature points of the kth and k+1th frames, and the optical flow value of each feature point in the top image is obtained by feature matching, thereby obtaining the optical flow value of the top image. The optical flow value of the next layer is estimated based on the optical flow value in the top image, until the optical flow value of the bottom layer is calculated and used as the optical flow value of the flow measurement image matrix.

[0023] In one embodiment, S6 includes: filtering the optical flow values by using a statistical histogram.

[0024] In one embodiment, when the river flow direction is perpendicular to the cross section, S6 includes: filtering the optical flow value according to the angle range between the initial offset and the final offset of the feature point in the measurement image matrix,

[0025]

[0026] in is the initial offset of the feature points in the flow measurement image matrix, is the final offset of the feature points in the selected flow measurement image matrix.

[0027] In one embodiment, step S7 includes:

[0028] The river speed of each filtered feature point in the pixel coordinate system is calculated based on the offset value obtained after filtering and the time of the video corresponding to the flow measurement image matrix.

[0029]

[0030] Among them, v is the river speed of each filtered feature point in the pixel coordinate system, w is the time of the video corresponding to the flow measurement image matrix, and ||l|| is the offset value;

[0031] The river surface velocity in the pixel coordinate system is obtained by selecting the average value of the river velocity of the filtered feature points in the preset part:

[0032]

[0033] Where N is the number of filtered feature points in the selected preset part, V is the surface velocity of the river in the pixel coordinate system, Indicates the sum of the river speed at each selected feature point.

[0034] In one embodiment, S9 is calculated as follows:

[0035]

[0036]

[0037] Among them, (u k ,v k ) is the pixel coordinate of the k-th frame image, (u k+1 ,v k+1 ) is the pixel coordinate of the k+1 frame image, (X k ,Y k ) is the x-direction and y-direction coordinates of the world coordinates of the k-th frame image, (X k+1 ,Y k+1 ) is the x-direction and y-direction coordinates of the k+1th frame image world coordinates, λ x and λ y are the spatial resolution in the x and y directions respectively.

[0038] In one embodiment, S10 is calculated as follows:

[0039]

[0040] Where V is the river surface velocity in the pixel coordinate system, λ x and λ y are the spatial resolutions in the x and y directions respectively, V′ is the surface velocity of the river in the world coordinate system, and fps is the frame rate.

[0041] Based on the same inventive concept, the second aspect of the present invention provides a system for calculating river surface flow velocity based on optical flow measurement, comprising:

[0042] Video acquisition module, used to capture river flow videos using a camera;

[0043] A video frame extraction module is used to extract video frames from the collected river flow video according to a preset time interval and frame interval, and all the extracted video frames constitute a flow measurement image matrix;

[0044] A preprocessing module, used for preprocessing the flow measurement image matrix;

[0045] The optical flow value calculation module is used to use the optical flow algorithm combined with the feature detection algorithm to perform feature detection and optical flow value calculation on the image in the preprocessed flow measurement image matrix to obtain the optical flow value g of the measurement image matrix. L =[l x ,l y ] T , the optical flow value of the measured image matrix represents the movement of feature points between image frames in the measured flow image matrix, where is the component parallel to the river section, is the component perpendicular to the river section;

[0046] The offset calculation module is used to obtain the offset of the feature points in the flow measurement image matrix based on the optical flow value of the flow measurement image matrix Then, the offset value ||l|| of the feature point in the flow measurement image matrix is obtained based on the Euclidean distance calculation.

[0047]

[0048] Sort the offset values from high to low, and filter the offset values of a preset proportion that are ranked low, wherein the offset value of a feature point in the flow measurement image matrix is the movement distance of the feature point between image frames in the flow measurement image matrix;

[0049] Optical flow value filtering module, used to filter the optical flow value of the flow measurement image matrix;

[0050] The river surface velocity calculation module in the pixel coordinate system is used to calculate the river velocity of each filtered feature point in the pixel coordinate system based on the offset value obtained after filtering and the time of the video corresponding to the flow measurement image matrix, and select the river velocity of a preset part of the filtered feature points in the pixel coordinate system to find the average value to obtain the river surface velocity in the pixel coordinate system;

[0051] A coordinate conversion module is used to convert the pixel coordinates of each frame of the flow measurement image matrix into world coordinates;

[0052] A spatial resolution calculation module is used to calculate the spatial resolution of the image based on the pixel coordinates of two consecutive frames of images in the flow measurement image matrix after filtering the optical flow value and the world coordinates of the two consecutive frames of images;

[0053] The river surface velocity calculation module in the world coordinate system is used to calculate the river surface velocity in the world coordinate system by using the spatial resolution of the image and the river surface velocity in the pixel coordinate system.

[0054] Based on the same inventive concept, the third aspect of the present invention provides a computer-readable storage medium having a computer program stored thereon, which implements the method described in the first aspect when the program is executed.

[0055] Compared with the prior art, the advantages and beneficial technical effects of the present invention are as follows:

[0056] In the prior art, the optical flow method calculates the optical flow values for all captured feature points on an image, which is very computationally intensive and complex. However, rivers may experience turbulence or be affected by shore vegetation, causing the water to flow in a direction other than the mainstream. Calculating the optical flow values for non-mainstream directions affects data accuracy and is computationally intensive. The method provided by the present invention combines an optical flow algorithm with a feature detection algorithm to process and calculate images in a preprocessed flow measurement image matrix. After obtaining the optical flow values of the measurement image matrix, an offset value is calculated and filtered. Furthermore, the optical flow values of the flow measurement image matrix are filtered to filter out the optical flow values for non-mainstream directions, thereby reducing calculation time and improving calculation accuracy. BRIEF DESCRIPTION OF THE DRAWINGS

[0057] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the following briefly introduces the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings described below are some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0058] Figure 1 is a flow chart of a method for calculating river surface velocity based on optical flow measurement provided by an embodiment of the present invention;

[0059] Figure 2 This is a schematic diagram of optical flow grid division provided by an embodiment of the present invention;

[0060] Figure 3 Schematic diagram of histogram statistics provided by an embodiment of the present invention;

[0061] Figure 4 It is a schematic diagram of the test results of the first embodiment of the present invention.

[0062] Figure 5 2 is a schematic diagram of test results of the second embodiment of the present invention.

[0063] Figure 6 1 is a schematic diagram of test results of the third embodiment of the present invention.

[0064] Figure 7 It is a structural block diagram of a computer-readable storage medium provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0065] Through extensive research and practice, the inventors of this application have discovered that: since the optical flow on the same moving target is similar and its distribution shows a certain regularity, the optical flow on different targets is different, the angle deviation from the river cross-section is too large, and the length is too short, it can be considered that the optical flow is formed by turbulence caused by sediment, vegetation, and floating branches. Therefore, a river surface velocity calculation method based on optical flow measurement is provided. By filtering the optical flow field in terms of angle and length respectively and removing such optical flows, the algorithm calculation amount can be reduced, the algorithm's running speed can be improved, and the monitoring effect of real-time calculation of the river surface velocity can be achieved.

[0066] To make the objectives, technical solutions, and advantages of the embodiments of the present invention more clear, the technical solutions in the embodiments of the present invention will be clearly and completely described below in conjunction with the accompanying drawings in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative efforts shall fall within the scope of protection of the present invention.

[0067] Example 1

[0068] An embodiment of the present invention provides a method for calculating river surface velocity based on optical flow measurement, comprising:

[0069] S1: Use a camera to collect river flow video;

[0070] S2: extracting video frames from the collected river flow video according to a preset time interval and frame interval, and all the extracted video frames constitute a flow measurement image matrix;

[0071] S3: preprocessing the flow measurement image matrix;

[0072] S4: Use the optical flow algorithm combined with the feature detection algorithm to perform feature detection and optical flow value calculation on the image in the preprocessed flow measurement image matrix to obtain the optical flow value g of the measurement image matrix L =[l x ,l y ] T , the optical flow value of the measured image matrix represents the movement of feature points between image frames in the measured flow image matrix, where is the component parallel to the river section, is the component perpendicular to the river section;

[0073] S5: Based on the optical flow value of the flow measurement image matrix, the offset of the feature points in the flow measurement image matrix is obtained Then, the offset value ||l|| of the feature point in the flow measurement image matrix is obtained based on the Euclidean distance calculation.

[0074]

[0075] Sort the offset values from high to low, and filter the offset values of a preset proportion that are ranked low, wherein the offset value of a feature point in the flow measurement image matrix is the movement distance of the feature point between image frames in the flow measurement image matrix;

[0076] S6: filtering the optical flow value of the flow measurement image matrix;

[0077] S7: Calculate the river velocity of the filtered feature points in the pixel coordinate system based on the offset value obtained after filtering and the time of the video corresponding to the flow measurement image matrix, and select a preset portion of the river velocities of the filtered feature points in the pixel coordinate system to obtain an average value to obtain the river surface velocity in the pixel coordinate system;

[0078] S8: converting the pixel coordinates of each frame of the image in the flow measurement image matrix into world coordinates;

[0079] S9: Calculating the spatial resolution of the image according to the pixel coordinates of two consecutive frames of images in the flow measurement image matrix after filtering the optical flow value and the world coordinates of the two consecutive frames of images;

[0080] S10: Calculate the river surface velocity in the world coordinate system using the spatial resolution of the image and the river surface velocity in the pixel coordinate system.

[0081] See Figure 1 , is a flowchart of a method for calculating river surface flow velocity based on optical flow measurement provided by an embodiment of the present invention.

[0082] During the specific implementation process, video collection can be achieved by using a drone equipped with RTK and a camera, with the camera perpendicular to the river surface.

[0083] In step S2, the preset interval time Δt and the frame interval s can be set according to the actual situation, and one frame is extracted every certain time and frame interval, so as to obtain the flow measurement image matrix I composed of all the extracted video frames. i (i=1, 2, ..., n), i represents the serial number of the video frame (image), and n is the total number of video frames.

[0084] The feature detection algorithm is combined with the optical flow algorithm for calculation. The feature detection algorithm is mainly used for feature point detection. Feature points are points that contain image information. Feature matching is also used in the combination of the two algorithms. The optical flow algorithm in S4 can be the LK optical flow method or other optical flow algorithms. The main steps of the LK optical flow method can be divided into three steps: pyramid establishment, pyramid tracking, and iteration. Therefore, the feature matching process is performed in the pyramid tracking step. The feature detection algorithm is used to detect feature points in the first layer of the first frame. Since the first layer of the image is reduced, the range of selected feature points is actually very large, so it needs to be expanded through iterative calculation. Specifically, the result of the first layer is used as the initial optical flow value of the next layer. At this time, the size of the image is enlarged a little. Therefore, the feature points are detected again to make the result more detailed. In other words, each layer selects feature points within a given range and gradually refines the optical flow value itself. It is a vector value with a direction. The calculation process of the feature vector of the feature point in the previous and next two frames (the kth frame and the k+1th frame) is actually a feature matching process. Through this process, the displacement of the feature point in the previous and next two frames can be obtained. The displacement includes distance and direction. The component (l x 、l y ) is the data of feature points containing distance and direction.

[0085] The preset proportion of the bottom ranking can be set according to actual conditions, for example, set to 10%.

[0086] In one embodiment, S3 pre-processes the flow measurement image matrix, including removing hue and saturation information, adjusting the hue of the image, and cropping the image boundary.

[0087] Specifically, the hue and saturation information of the image in the flow measurement image matrix is eliminated, the original color image is converted into a grayscale image, and the hue of the image is adjusted using the log correction method (or other correction methods such as the Gamma correction method), and the image boundary is cropped to obtain the preprocessed flow measurement image matrix I′ i (i=1,2,...,n).

[0088] In one embodiment, an optical flow algorithm is combined with a feature detection algorithm to perform feature detection and optical flow value calculation on the images in the preprocessed flow measurement image matrix, including:

[0089] Each frame of the flow measurement image matrix is scaled layer by layer in a pyramid layered manner, with the bottom image having the largest resolution and the top image having the smallest resolution. Starting from the top image, a feature detection algorithm is used to obtain feature points of the kth and k+1th frames, and the optical flow value of each feature point in the top image is obtained by feature matching, thereby obtaining the optical flow value of the top image. The optical flow value of the next layer is estimated based on the optical flow value in the top image, until the optical flow value of the bottom layer is calculated and used as the optical flow value of the flow measurement image matrix.

[0090] Specifically, the optical flow algorithm may be a Lucas–Kanade optical flow algorithm or other optical flow algorithms (Farneback algorithm).

[0091] The optical flow value of the next layer is estimated according to the optical flow value in the top image, and the estimated optical flow value is used as the initial optical flow value of the next layer. Then, according to the initial optical flow value of the next layer, the actual optical flow value of the next layer of the next layer is calculated using the method of calculating the optical flow value of the top image, and so on, until the optical flow value of the bottom layer (that is, the optical flow value of the original image) is calculated.

[0092] When calculating the optical flow value of the top image, the optical flow value g of each point in the top image is obtained by minimizing the sum of the matching errors within the neighborhood of each corner point. L , the image size is scaled to half of its original size each time, for a total of L m Layer, the 0th layer is the original image, assuming that the displacement of the original image corner is d, then the displacement of each layer is The optical flow value in the top layer image is used as the estimated value g of the initial optical flow value of the next layer. L-1 =2(g L +d L ).

[0093] In one embodiment, the feature detection algorithm used includes but is not limited to the FAST algorithm.

[0094] Specifically, the feature detection algorithm is used to obtain feature points during the optical flow calculation process. To improve detection efficiency, the FAST algorithm is used to obtain feature points in the previous and next frames, and then the optical flow value is obtained based on the pixel coordinates of the feature points in the previous and next frames.

[0095] The specific steps of the FAST algorithm are as follows: compare the grayscale values of pixels in a certain area of the image with the center point. The specific method is as follows: compare the grayscale values of the center point with 12 consecutive points in the neighborhood passed by a circle with a radius of 3.4. If the grayscale values of these points are greater than or less than the center point, the center point can be considered as a candidate corner point. After obtaining the candidate corner point, use the ID3 classifier to determine whether the candidate corner point is a corner point.

[0096] In one embodiment, a filtering method in S6 is to filter the optical flow values using a statistical histogram.

[0097] Filtering the optical flow values by using histogram statistics can be applied to various river directions. This method uses histogram statistics to calculate the direction of the optical flow values in the image. Figure 2 , the optical flow map is divided into grids, each optical flow is included in a grid, the optical flow direction in the grid is counted, and the gradient histogram divides the direction range of 0 to 360 degrees into 12 bins, each bin is 30 degrees. Figure 3 As shown in the figure, the peak direction of the histogram represents the primary direction of the keypoint. The peak of the direction histogram represents the direction of the keypoint, and the maximum value in the histogram is considered the primary direction of the corresponding keypoint. The peak value can be considered as the value of each acquired optical flow direction, and the maximum value is the largest value among all optical flow directions. To enhance the robustness of the matching, directions with peak values greater than 80% of the peak value of the primary direction are retained as the secondary directions of the keypoint, and the rest are filtered out.

[0098] When the river flow direction is perpendicular to the cross section, one filtering method in S6 is to filter the optical flow value according to the angle range between the initial offset and the final offset of the feature point in the measurement image matrix.

[0099]

[0100] in is the initial offset of the feature points in the flow measurement image matrix, is the final offset of the feature points in the selected flow measurement image matrix.

[0101] The initial offset represents the state of the feature point in the first frame (including position and direction), and the final offset represents the state of the feature point in the last frame. This change process is called optical flow.

[0102] During the specific implementation process, if the river flows smoothly, there are no floating branches, leaves, etc. on the river surface to affect the river, and the river bottom roughness is low and there is no sediment, the probability of turbulence in the river is small. In order to reduce the running time of the program, the above two methods can both select 80% of the filtered optical flow values after filtering the optical flow to calculate the river surface flow velocity. If it is not the above situation, but you want to achieve a faster running speed, you can also select 80% of the filtered optical flow values after filtering the optical flow to calculate the river surface flow velocity.

[0103] In one embodiment, step S7 includes:

[0104] The river speed of each filtered feature point in the pixel coordinate system is calculated based on the offset value obtained after filtering and the time of the video corresponding to the flow measurement image matrix:

[0105]

[0106] Where v is the river speed for each offset, w is the time of the video corresponding to the flow measurement image matrix, and ||l|| is the offset value;

[0107] The river surface velocity in the pixel coordinate system is obtained by selecting the average value of the river velocity of the filtered feature points in the preset part:

[0108]

[0109] Where N is the number of filtered feature points in the selected preset part, V is the surface velocity of the river in the pixel coordinate system, Indicates the sum of the river speeds for each offset value chosen.

[0110] Specifically, the offset value is the distance that the feature point moves between image frames in the flow measurement image matrix, and this distance is used to calculate the river speed in the pixel coordinate system. Figure 2 The arrows in the rectangular box are the offsets corresponding to the feature points of the selected preset part.

[0111] In one embodiment, S9 is calculated as follows:

[0112]

[0113]

[0114] Among them, (u k ,v k ) is the pixel coordinate of the k-th frame image, (u k+1 ,v k+1 ) is the pixel coordinate of the k+1 frame image, (X k ,Y k ) is the x-direction and y-direction coordinates of the world coordinates of the k-th frame image, (X k+1 ,Y k+1 ) is the x-direction and y-direction coordinates of the k+1th frame image world coordinates, λ x and λ y are the spatial resolution in the x and y directions respectively.

[0115] In the specific implementation process, the coordinate rotation of S8 is achieved by the following method:

[0116] For the pixel coordinates of the i-th frame (u i,v i ), the corresponding world coordinate is (X i ,Y i ,Z i ), camera external parameters K1, camera internal parameters K, K2, focal length f, pixel width dx, pixel length dy, center point of pixel coordinate system (u0, z0), camera height Z wi , camera pose angle θ, rotation matrix R, and translation vector T are known parameters.

[0117]

[0118]

[0119]

[0120] A point on the pixel coordinates (u i ,v i ) is converted to image coordinates as (x i ,y i ),

[0121]

[0122]

[0123]

[0124] Z c is the camera scale variable.

[0125] Step S9 measures the pixel coordinates (u k ,v k ) and (u k+1 ,v k+1 ) and the world coordinates G(X k ,Y k ,Z k ) and G(X k+1 ,Y k+1 ,Z k+1 ) to calculate the image spatial resolution.

[0126] In one embodiment, S10 is calculated as follows:

[0127]

[0128] Where V is the river surface velocity in the pixel coordinate system, λ x and λ y are the spatial resolutions in the x and y directions respectively, V′ is the surface velocity of the river in the world coordinate system, and fps is the frame rate.

[0129] There are methods in the prior art that directly apply optical flow algorithms to flow rate measurement. The difference between the present invention and these methods is that the LK optical flow method is only used as a basic algorithm, and it is also feasible to use other optical flow algorithms. The focus of the present invention is on filtering the optical flow values. All steps of the existing methods are how to apply the LK optical flow method to flow rate measurement. This solution only exists in step S3 for the LK optical flow method. The present invention can solve the problems of computational complexity and large amount of computation of the optical flow method in the prior art. This is because the optical flow method calculates the optical flow values for all captured feature points on an image, and the amount of computation is very large and complex. However, there may be turbulence in the river, or the influence of vegetation on the shore, resulting in the water not flowing in the mainstream direction. Calculating the optical flow values of non-mainstream directions affects the accuracy of the data on the one hand, and the amount of computation is also large on the other hand. Filtering out the optical flow values of non-mainstream directions can reduce the calculation time and improve the accuracy of the calculation.

[0130] In order to illustrate the effectiveness of the method proposed in this invention, the upper reaches of Yangzigou were used as the test river object, and 3 seconds, 5 seconds, and 7 seconds of the video were selected with an interval of 1 frame for testing. The video frame rate was 24 frames / s, and the actual speed of the river was 0.8m / s. The test results are shown in Figure 2. Figures 4 to 6 As shown, Figure 4 This is a schematic diagram of the test results for selecting 3 seconds of the video. Figure 5 This is a diagram showing the results of testing a video of 5 seconds. Figure 6 The diagram below shows the results of testing a video of 7 seconds. The algorithm of the present invention is the one that adds the optical flow value filtering algorithm (i.e. LK+) based on the LK optical flow algorithm. The LK and Farneback algorithms are the algorithms used in the prior art. Figures 4 to 6 It can be seen from the results that the speed and time of the present invention are better than those of the existing algorithms.

[0131] Example 2

[0132] Based on the same inventive concept, this embodiment provides a system for calculating river surface flow velocity based on optical flow measurement, comprising:

[0133] Video acquisition module, used to capture river flow videos using a camera;

[0134] A video frame extraction module is used to extract video frames from the collected river flow video according to a preset time interval and frame interval, and all the extracted video frames constitute a flow measurement image matrix;

[0135] A preprocessing module, used for preprocessing the flow measurement image matrix;

[0136] The optical flow value calculation module is used to use the optical flow algorithm combined with the feature detection algorithm to perform feature detection and optical flow value calculation on the image in the preprocessed flow measurement image matrix to obtain the optical flow value g of the measurement image matrix. L =[l x ,l y ] T , the optical flow value of the measured image matrix represents the movement of feature points between image frames in the measured flow image matrix, where is the component parallel to the river section, is the component perpendicular to the river section;

[0137] The offset calculation module is used to obtain the offset of the feature points in the flow measurement image matrix based on the optical flow value of the flow measurement image matrix Then, the offset value ||l|| of the feature point in the flow measurement image matrix is obtained based on the Euclidean distance calculation.

[0138]

[0139] Sort the offset values from high to low, and filter the offset values of a preset proportion that are ranked low, wherein the offset value of a feature point in the flow measurement image matrix is the movement distance of the feature point between image frames in the flow measurement image matrix;

[0140] Optical flow value filtering module, used to filter the optical flow value of the flow measurement image matrix;

[0141] The river surface velocity calculation module in the pixel coordinate system is used to calculate the river velocity of each filtered feature point in the pixel coordinate system based on the offset value obtained after filtering and the time of the video corresponding to the flow measurement image matrix, and select the river velocity of a preset part of the filtered feature points in the pixel coordinate system to calculate the average value to obtain the river surface velocity in the pixel coordinate system:

[0142] A coordinate conversion module is used to convert the pixel coordinates of each frame of the flow measurement image matrix into world coordinates;

[0143] A spatial resolution calculation module is used to calculate the spatial resolution of the image based on the pixel coordinates of two consecutive frames of images in the flow measurement image matrix after filtering the optical flow value and the world coordinates of the two consecutive frames of images;

[0144] The river surface velocity calculation module in the world coordinate system is used to calculate the river surface velocity in the world coordinate system by using the spatial resolution of the image and the river surface velocity in the pixel coordinate system.

[0145] Since the system described in Example 2 of the present invention is the system used to implement the method for calculating river surface flow velocity based on optical flow measurement in Example 1 of the present invention, those skilled in the art will be able to understand the specific structure and variations of this system based on the method described in Example 1 of the present invention, and therefore will not be described in detail here. All systems used in the method of Example 1 of the present invention are within the scope of protection of the present invention.

[0146] Example 3

[0147] Based on the same inventive concept, see Figure 7 The present invention further provides a computer-readable storage medium 300 on which a computer program 311 is stored. When the program is executed, the method described in the first embodiment is implemented.

[0148] Since the computer-readable storage medium described in Example 3 of the present invention is used to implement the method for calculating river surface flow velocity based on optical flow measurement in Example 1 of the present invention, the specific structure and variations of the computer-readable storage medium are readily understood by those skilled in the art based on the method described in Example 1 of the present invention, and thus will not be further described here. All computer-readable storage media used in the method of Example 1 of the present invention fall within the scope of protection of the present invention.

[0149] It will be understood by those skilled in the art that embodiments of the present invention may be provided as methods, systems, or computer program products. Thus, the present invention may take the form of an entirely hardware embodiment, an entirely software embodiment, or an embodiment combining software and hardware. Furthermore, the present invention may take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to magnetic disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0150] The present invention is described with reference to flowcharts and / or block diagrams of methods, devices (systems), and computer program products according to embodiments of the present invention. It should be understood that each process and / or block in the flowcharts and / or block diagrams, as well as combinations of processes and / or blocks in the flowcharts and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to produce a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowcharts and / or block diagrams. Figure 1 a process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0151] Although the preferred embodiments of the present invention have been described, those skilled in the art may make additional changes and modifications to these embodiments once they have learned the basic creative concept. Therefore, the appended claims are intended to be interpreted as including the preferred embodiments and all changes and modifications that fall within the scope of the present invention.

[0152] Obviously, those skilled in the art may make various changes and modifications to the embodiments of the present invention without departing from the spirit and scope of the embodiments of the present invention. Thus, if such changes and modifications of the embodiments of the present invention fall within the scope of the claims and their equivalents, the present invention is intended to include such changes and modifications.

Claims

1. A method for calculating river surface velocity based on optical flow measurement, characterized in that: include: S1: Use a camera to collect river flow video; S2: extracting video frames from the collected river flow video according to a preset time interval and frame interval, and all the extracted video frames constitute a flow measurement image matrix; S3: preprocessing the flow measurement image matrix; S4: Use the optical flow algorithm combined with the feature detection algorithm to perform feature detection and optical flow value calculation on the image in the preprocessed flow measurement image matrix to obtain the optical flow value g of the measurement image matrix L =[l x ,l y ] T , the optical flow value of the measured image matrix represents the movement of feature points between image frames in the measured flow image matrix, where is the component parallel to the river section, is the component perpendicular to the river section; S5: Based on the optical flow value of the flow measurement image matrix, the offset of the feature points in the flow measurement image matrix is obtained Then, the offset value ||l|| of the feature point in the flow measurement image matrix is obtained based on the Euclidean distance calculation. Sort the offset values from high to low, and filter the offset values of a preset proportion that are ranked low, wherein the offset value of a feature point in the flow measurement image matrix is the movement distance of the feature point between image frames in the flow measurement image matrix; S6: filtering the optical flow value of the flow measurement image matrix; Step S6 also includes: filtering the optical flow values by using a statistical histogram, wherein when the river flow direction is perpendicular to the cross section, the optical flow values are filtered according to the angle range between the initial offset and the final offset of the feature point in the measurement image matrix. in is the initial offset of the feature points in the flow measurement image matrix, is the final offset of the feature points in the selected flow measurement image matrix; S7: Calculate the river velocity of the filtered feature points in the pixel coordinate system based on the offset value obtained after filtering and the time of the video corresponding to the flow measurement image matrix, and select a preset portion of the river velocities of the filtered feature points in the pixel coordinate system to obtain an average value to obtain the river surface velocity in the pixel coordinate system; The S7 also includes: The river speed of each filtered feature point in the pixel coordinate system is calculated based on the offset value obtained after filtering and the time of the video corresponding to the flow measurement image matrix. Among them, v is the river speed of each filtered feature point in the pixel coordinate system, w is the time of the video corresponding to the flow measurement image matrix, and ||l|| is the offset value; The river surface velocity in the pixel coordinate system is obtained by selecting the average value of the river velocity of the filtered feature points in the preset part: Where N is the number of filtered feature points in the selected preset part, V is the surface velocity of the river in the pixel coordinate system, Indicates the sum of the river speed of each selected feature point; S8: converting the pixel coordinates of each frame of the image in the flow measurement image matrix into world coordinates; S9: Calculating the spatial resolution of the image according to the pixel coordinates of two consecutive frames of images in the flow measurement image matrix after filtering the optical flow value and the world coordinates of the two consecutive frames of images; S10: Calculate the river surface velocity in the world coordinate system using the spatial resolution of the image and the river surface velocity in the pixel coordinate system.

2. The method for calculating river surface velocity based on optical flow measurement according to claim 1, characterized in that: S3 preprocesses the flow measurement image matrix, including eliminating hue and saturation information, adjusting the hue of the image, and cropping the image boundary.

3. The method for calculating river surface velocity based on optical flow measurement according to claim 1, characterized in that: The optical flow algorithm is combined with the feature detection algorithm to perform feature point detection and optical flow value calculation on the image in the preprocessed flow measurement image matrix, including: Each frame of the flow measurement image matrix is scaled layer by layer in a pyramid layered manner, with the bottom image having the largest resolution and the top image having the smallest resolution. Starting from the top image, a feature detection algorithm is used to obtain feature points of the kth and k+1th frames, and the optical flow value of each feature point in the top image is obtained by feature matching, thereby obtaining the optical flow value of the top image. The optical flow value of the next layer is estimated based on the optical flow value in the top image, until the optical flow value of the bottom layer is calculated and used as the optical flow value of the flow measurement image matrix.

4. The method for calculating river surface velocity based on optical flow measurement according to claim 1, wherein: The calculation method of S9 is: Among them, (u k ,v k ) is the pixel coordinate of the k-th frame image, (u k+1 ,v k+1 ) is the pixel coordinate of the k+1 frame image, (X k ,Y k ) is the x-direction and y-direction coordinates of the world coordinates of the k-th frame image, (X k+1 ,Y k+1 ) is the x-direction and y-direction coordinates of the k+1th frame image world coordinates, λ x and λ y are the spatial resolution in the x and y directions respectively.

5. The method for calculating river surface velocity based on optical flow measurement according to claim 1, wherein: S10 is calculated as: Where V is the river surface velocity in the pixel coordinate system, λ x and λ y are the spatial resolutions in the x and y directions respectively, V′ is the surface velocity of the river in the world coordinate system, and fps is the frame rate.

6. A river surface velocity calculation system based on optical flow measurement, characterized in that: include: Video acquisition module, used to capture river flow videos using a camera; A video frame extraction module is used to extract video frames from the collected river flow video according to a preset time interval and frame interval, and all the extracted video frames constitute a flow measurement image matrix; A preprocessing module, used for preprocessing the flow measurement image matrix; The optical flow value calculation module is used to use the optical flow algorithm combined with the feature detection algorithm to perform feature detection and optical flow value calculation on the image in the preprocessed flow measurement image matrix to obtain the optical flow value g of the measurement image matrix. L =[l x ,l y ] T , the optical flow value of the measured image matrix represents the movement of feature points between image frames in the measured flow image matrix, where is the component parallel to the river section, is the component perpendicular to the river section; The offset calculation module is used to obtain the offset of the feature points in the flow measurement image matrix based on the optical flow value of the flow measurement image matrix Then, the offset value ||l|| of the feature point in the flow measurement image matrix is obtained based on the Euclidean distance calculation. Sort the offset values from high to low, and filter the offset values of a preset proportion that are ranked low, wherein the offset value of a feature point in the flow measurement image matrix is the movement distance of the feature point between image frames in the flow measurement image matrix; Optical flow value filtering module, used to filter the optical flow value of the flow measurement image matrix; The optical flow value filtering module is also used to filter the optical flow value using a statistical histogram. When the river flow direction is perpendicular to the cross section, the optical flow value is filtered according to the angle range between the initial offset and the final offset of the feature point in the measurement image matrix. in is the initial offset of the feature points in the flow measurement image matrix, is the final offset of the feature points in the selected flow measurement image matrix; The river surface velocity calculation module in the pixel coordinate system is used to calculate the river velocity of each filtered feature point in the pixel coordinate system based on the offset value obtained after filtering and the time of the video corresponding to the flow measurement image matrix, and select the river velocity of a preset part of the filtered feature points in the pixel coordinate system to find the average value to obtain the river surface velocity in the pixel coordinate system; The river surface velocity calculation module in pixel coordinate system is also used for: The river speed of each filtered feature point in the pixel coordinate system is calculated based on the offset value obtained after filtering and the time of the video corresponding to the flow measurement image matrix. Among them, v is the river speed of each filtered feature point in the pixel coordinate system, w is the time of the video corresponding to the flow measurement image matrix, and ||l|| is the offset value; The river surface velocity in the pixel coordinate system is obtained by selecting the average value of the river velocity of the filtered feature points in the preset part: Where N is the number of filtered feature points in the selected preset part, V is the surface velocity of the river in the pixel coordinate system, Indicates the sum of the river speed of each selected feature point; A coordinate conversion module is used to convert the pixel coordinates of each frame of the flow measurement image matrix into world coordinates; A spatial resolution calculation module is used to calculate the spatial resolution of the image based on the pixel coordinates of two consecutive frames of images in the flow measurement image matrix after filtering the optical flow value and the world coordinates of the two consecutive frames of images; The river surface velocity calculation module in the world coordinate system is used to calculate the river surface velocity in the world coordinate system by using the spatial resolution of the image and the river surface velocity in the pixel coordinate system.

7. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the program is executed, the method according to any one of claims 1 to 5 is implemented.

Images