A method, device, equipment and medium for measuring water flow velocity

Abstract

The present application relates to a method, device, equipment and medium for measuring water flow velocity, and relates to the field of hydrological flow measurement technology. The method comprises: obtaining a water flow video and selecting a velocity measurement area; the water flow video comprises a continuous multi-frame water flow image; executing a first corner point detection algorithm on the N+(n‑1)K frame area image to obtain a first corner point set; executing a second corner point detection algorithm on the N+(n‑1)K frame area image to obtain a second corner point set; the N+(n‑1)K frame area image is an image of the velocity measurement area intercepted from the N+(n‑1)K frame water flow image; the number of corner points in the first corner point set is greater than the number of corner points in the second corner point set; K represents a preset frame extraction interval; the first corner point set is screened based on the second corner point set to obtain a third corner point set; according to the third corner point set and the multi-frame water flow image, the water flow distance of the velocity measurement area is obtained; according to the water flow distance of the velocity measurement area, the water flow velocity of the velocity measurement area is obtained. The present application has the effect of improving the accuracy of the measurement result of the water flow velocity.

Description

Technical Field

[0001] The present application relates to the technical field of hydrological flow measurement, and in particular to a method, device, equipment and medium for measuring water flow velocity. Background Art

[0002] One of the main goals of water conservancy and flood control is to improve my country's capabilities in real-time monitoring, forecasting, early warning, dispatching and control, and intelligent decision-making in flood and drought disaster prevention by promoting the application of the "four predictions" of flood control based on digital twin technology. However, there are problems such as insufficient monitoring methods and low data accuracy in extreme floods and debris flows in hilly areas of my country, as well as low efficiency and insufficient refinement of the "preview" simulation calculation in the "four predictions" of digital twin flood control. Therefore, measuring water flow velocity is crucial for water conservancy and flood control.

[0003] The traditional sparse optical flow velocity measurement algorithm first detects corner points in the video area frame by frame to obtain the key corner points of the water flow image, and then uses the sparse optical flow algorithm to track the corner points. Although the algorithm process is simple and time-saving, the corner point detection stage has extremely high requirements on the obviousness of image features. When the image features are not obvious due to factors such as the environment and camera distance, the quality of the detected corner points has no reference value, resulting in a large error in the final flow velocity measurement result. Summary of the invention

[0004] In order to solve the problem of how to accurately measure the flow velocity of water, the present application provides a method, device, equipment and medium for measuring the flow velocity of water.

[0005] In a first aspect, the present application provides a method for measuring water flow velocity, which adopts the following technical solution:

[0006] A method for measuring water flow velocity, comprising:

[0007] Acquire a water flow video and select a speed measurement area; the water flow video includes a continuous multi-frame water flow image;

[0008] A first corner point detection algorithm is executed on the N+(n-1)Kth frame area image to obtain a first corner point set; a second corner point detection algorithm is executed on the N+(n-1)Kth frame area image to obtain a second corner point set; the N+(n-1)Kth frame area image is an image of a speed measurement area intercepted from the N+(n-1)Kth frame water flow image; the number of corner points in the first corner point set is greater than the number of corner points in the second corner point set; N, n and K are all positive integers greater than 0; K represents a preset frame extraction interval;

[0009] Filtering the first corner point set based on the second corner point set to obtain a third corner point set;

[0010] Obtaining a water flow distance in the speed measurement area according to the third corner point set and the multiple frames of water flow images;

[0011] The water flow velocity in the velocity measuring area is obtained according to the water flow distance in the velocity measuring area.

[0012] By adopting the above technical solution, different first corner point detection algorithms and second corner point detection algorithms are used to perform corner point detection on the regional image, and a first corner point set and a second corner point set with different numbers of corner points are obtained. Further screening is performed based on the two corner point sets to obtain a third corner point set, thereby avoiding the problem of inaccurate corner points output by a single corner point algorithm. According to the third corner point set and multiple frames of water flow images, the water flow distance of the speed measurement area is obtained, and according to the water flow distance of the speed measurement area, the water flow velocity of the speed measurement area is obtained. The corner points in the third corner point set have more obvious features, which is conducive to the subsequent tracking of the corner points in the third corner point set, thereby improving the accuracy of the measurement results of the water flow velocity.

[0013] Optionally, filtering the first corner point set based on the second corner point set to obtain a third corner point set includes:

[0014] Determine, from the second set of corner points, corresponding corner points whose distances to each corner point in the first set of corner points are less than a first threshold, and obtain a list of multiple corner points whose distances are less than the first threshold;

[0015] The average of the corner point coordinates in each corner point list is determined as the coordinates of the corner points in the third corner point set to obtain the third corner point set.

[0016] By adopting the above technical solution, the second corner point set is used as an outer loop, the first corner point set is traversed in sequence, corner points with similar distances are found and averaged to obtain the third corner point set.

[0017] Optionally, obtaining the water flow distance of the speed measurement area according to the third corner point set and the multiple frames of water flow images includes:

[0018] Execute a sparse optical flow tracking algorithm on the third corner point set, the N+(n-1)Kth frame area image, and the N+nKth frame area image to obtain a fourth corner point set and a tracking state; the fourth corner point set represents the tracking results of each corner point in the third corner point set; the tracking state is used to indicate whether the tracking of the corner point is successful;

[0019] Filtering successfully tracked corner points from the fourth corner point set according to the tracking state to obtain a fifth corner point set; filtering successfully tracked corner points from the third corner point set according to the tracking state to obtain a sixth corner point set;

[0020] If the number of corner points in the fifth corner point set is greater than or equal to a second threshold, calculating pixel displacements between corresponding corner points in the fifth corner point set and the sixth corner point set;

[0021] The water flow distance of the velocity measurement area is obtained according to the pixel displacement between the two corresponding corner points.

[0022] By adopting the above technical solution, the number of corner points in the tracked corner point set is screened to ensure that the number of tracked corner points is greater than or equal to the second threshold, thereby avoiding the disappearance of tracked corner points, which leads to inaccurate subsequent water flow velocity detection results.

[0023] Optionally, the method further comprises:

[0024] If the number of corner points in the fifth corner point set is less than the second threshold, adjusting parameters of the first corner point detection algorithm and the second corner point detection algorithm;

[0025] Executing the adjusted first corner point detection algorithm on the N+(n-1)Kth frame area image to obtain an adjusted first corner point set; executing the adjusted second corner point detection algorithm on the N+(n-1)Kth frame area image to obtain an adjusted second corner point set;

[0026] The adjusted first corner point set is screened based on the adjusted second corner point set to obtain the third corner point set.

[0027] By adopting the above-mentioned technical solution, when the number of corner points in the fifth corner point set is less than the second threshold, the parameters of the first corner point detection algorithm and the second corner point detection algorithm are adjusted, and the double corner point detection is re-executed to ensure that the number of tracked corner points is greater than or equal to the second threshold, thereby improving the accuracy of subsequent water flow velocity measurement results.

[0028] Optionally, calculating pixel displacements between corresponding corner points in the fifth corner point set and the sixth corner point set includes:

[0029] If the pixel displacement between the first corner point and the second corner point in the fifth corner point set is greater than or equal to a third threshold, the pixel displacement between the first corner point and the second corner point is determined as the pixel displacement between the corresponding corner points; the first corner point is any corner point in the sixth corner point set, and the second corner point is the tracking result of the third corner point.

[0030] By adopting the above technical solution, the pixel displacement between the tracked corner points is threshold screened, and the corner points with pixel displacement less than the third threshold are screened out to ensure that the pixel displacement between the corner points is greater than or equal to the third threshold. The water flow velocity is subsequently obtained based on the pixel displacement, which can improve the accuracy of the measurement results of the water flow velocity.

[0031] Optionally, obtaining the water flow distance of the velocity measurement area according to the pixel displacement between the two corresponding corner points includes:

[0032] For different values ​​of n, a plurality of real distances between the N+(n-1)Kth frame region image and the N+nKth frame region image are obtained according to the pixel displacement between the corresponding two corner points and the perspective matrix; the perspective matrix is ​​used to indicate the mapping relationship between the pixel displacement and the real distance;

[0033] Sorting the multiple real distances according to their numerical values, filtering out the first M real distances and the last M real distances, and obtaining the remaining multiple real distances; M is a positive integer greater than 0;

[0034] The water flow distance in the speed measurement area is obtained according to the average of the remaining multiple real distances.

[0035] By adopting the above technical solution and the frame extraction detection method, the large measurement error caused by the small displacement between the two frames of images can be avoided, and multiple frame extraction detection methods can be used to obtain multiple real distances. According to the average of multiple real distances, the water flow distance in the speed measurement area can be obtained more accurately.

[0036] Optionally, obtaining the water flow distance of the speed measurement area according to the average of the remaining multiple real distances includes:

[0037] Calculating the true distance between the N+nKth frame area image and the N+nK+1th frame area image;

[0038] Adding the multiple real distances and the real distance between the N+nKth frame area image and the N+nK+1th frame area image to a distance list;

[0039] The water flow distance of the speed measurement area is determined according to the average of the remaining multiple real distances and the distance list; the calculation formula is as follows:

[0040]

[0041] Wherein, Dis is the water flow distance in the velocity measurement area, is the mean of multiple distances in the distance list, Dis mean is the average of the remaining multiple true distances.

[0042] In a second aspect, the present application provides a device for measuring water flow velocity, which adopts the following technical solution:

[0043] A device for measuring water flow velocity, comprising:

[0044] An acquisition module is used to: acquire a water flow video and select a speed measurement area; the water flow video includes a continuous multi-frame water flow image; a corner point detection module is used to: execute a first corner point detection algorithm on the N+(n-1)K frame area image to obtain a first corner point set; execute a second corner point detection algorithm on the N+(n-1)K frame area image to obtain a second corner point set; the N+(n-1)K frame area image is an image of the speed measurement area intercepted from the N+(n-1)K frame water flow image; the number of corner points in the first corner point set is greater than the number of corner points in the second corner point set; N, n and K are all positive integers greater than 0; K represents a preset frame extraction interval;

[0045] A corner point screening module, used to: screen the first corner point set based on the second corner point set to obtain a third corner point set;

[0046] A distance acquisition module, used to: obtain the water flow distance of the speed measurement area according to the third corner point set and the multiple frames of water flow images;

[0047] The flow velocity acquisition module is used to obtain the water flow velocity in the speed measurement area according to the water flow distance in the speed measurement area.

[0048] In a third aspect, the present application provides a computer device, which adopts the following technical solution:

[0049] A computer device comprises a memory, a processor and a computer program stored in the memory and executable on the processor, wherein the processor implements the method described in the first aspect when executing the program.

[0050] In a fourth aspect, the present application provides a computer-readable storage medium, which adopts the following technical solution:

[0051] A computer-readable storage medium stores a computer program that can be loaded by a processor and execute any one of the methods in the first aspect.

[0052] In summary, the present application includes at least one of the following beneficial technical effects:

[0053] The present application provides a method for measuring water flow velocity, which adopts a double corner point detection algorithm, screens the corner points twice, obtains a third corner point set, and obtains corner points with more obvious features, thereby avoiding the inaccuracy of the corner points output by the single corner point detection algorithm, and is conducive to better tracking of the sparse optical flow tracking algorithm based on the third corner point set. In addition, the frame extraction detection method is adopted to avoid the problem that the displacement of consecutive frames does not change or changes very little due to the short time difference, thereby improving the accuracy of the output result. In addition, the pixel displacement between the corner points after tracking is threshold screened to improve the accuracy of the output average pixel displacement, and the number of corner points in the fifth corner point set is judged. When the number of corner points is less than the preset threshold, the corner points are re-detected, thereby avoiding the problem that the output result error is large due to the small number of corner points each time the corner points are tracked, thereby improving the accuracy of the water flow velocity measurement result. BRIEF DESCRIPTION OF THE DRAWINGS

[0054] Figure 1 This is an application scenario diagram of a method for measuring water flow velocity provided in an embodiment of the present application.

[0055] Figure 2 This is a flow chart of a method for measuring water flow velocity provided in an embodiment of the present application. Figure 1 .

[0056] Figure 3 This is a flow chart of a method for measuring water flow velocity provided in an embodiment of the present application. Figure 2 .

[0057] Figure 4 It is a structural diagram of a water flow velocity measuring device provided in an embodiment of the present application.

[0058] Explanation of the reference numerals: 101, shooting device; 102, electronic device; 401, acquisition module; 402, corner point detection module; 403, corner point screening module; 404, distance acquisition module; 405, flow rate acquisition module; 406, pixel displacement determination module. DETAILED DESCRIPTION

[0059] In order to make the purpose, technical solutions and advantages of this application more clear, the following Figure 1-3 It should be understood that the specific embodiments described herein are only used to explain the present application and are not used to limit the present application.

[0060] There are two major problems with the traditional sparse optical flow tracking algorithm: First, the initial corner detection stage has extremely high requirements on the obviousness of image features. When the image features are not obvious due to environmental and camera distance factors, the quality of the detected corners has no reference value, resulting in large errors in the final measurement results. Second, in the sparse optical flow tracking stage, the positions and numbers of the tracked corners vary, resulting in a continuously growing error in subsequent tracking. Therefore, the traditional sparse optical flow tracking algorithm has a large error in measuring water flow velocity.

[0061] In view of this, an embodiment of the present application provides a method for measuring water flow velocity. The application scenarios involved in the embodiment of the present application are introduced below.

[0062] Please refer to Figure 1 , is a schematic diagram of an application scenario of a method for measuring water flow velocity provided in an embodiment of the present application, the application scenario includes a shooting device 101 and an electronic device 102 communicating therewith. The shooting device 101 is, for example, a terminal device or a camera, etc. The terminal device includes a camera, a mobile phone or a tablet computer, etc., and may also include other devices with shooting functions.

[0063] The electronic device 102 may be implemented by a terminal or a server, such as a mobile terminal, a fixed terminal or a portable terminal, such as a mobile phone, a multimedia computer, a multimedia tablet, a desktop computer, a notebook computer, a tablet computer, a camera, etc. The server may be an independent physical server, or a server cluster or a distributed system composed of multiple physical servers, or a cloud server that provides basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communications, and middleware services, but is not limited thereto.

[0064] Specifically, the shooting device 101 captures a water flow video and sends the water flow video to the electronic device 102. The electronic device 102 processes the water flow video to obtain the water flow velocity corresponding to the water flow video. The specific process of obtaining the water flow velocity will be described below.

[0065] based on Figure 1 The application scenario discussed below introduces a method for measuring water flow velocity involved in an embodiment of the present application. Figure 2 , is a flow chart of a method for measuring water flow velocity provided in an embodiment of the present application Figure 1 .

[0066] S201, obtaining water flow video and selecting a speed measurement area.

[0067] Specifically, the electronic device 102 obtains the water flow video sent by the shooting device 101. The water flow video includes multiple frames of continuous water flow images, and selects the speed measurement area, also known as the region of interest (ROI). There may be multiple shooting devices 101 and multiple speed measurement areas. i represents the index of the shooting device 101, and j represents the index of the speed measurement area. Then C ij It represents the jth speed measurement area in the water flow video captured by the i-th shooting device 101.

[0068] S202, executing a first corner point detection algorithm on the N+(n-1)Kth frame area image to obtain a first corner point set; executing a second corner point detection algorithm on the N+(n-1)Kth frame area image to obtain a second corner point set.

[0069] Specifically, N represents the starting frame, K represents the preset frame extraction interval, n represents the number of frame extractions, and N, n and K are all positive integers greater than 0. The electronic device 102 takes the Nth frame of the water flow image as the starting frame, and extracts the Nth frame of the water flow image from the water flow video for the first time according to the preset frame extraction interval K and the number of frame extractions n, and extracts the N+Kth frame of the water flow image from the water flow video for the second time, and so on, and extracts the N+(n-1)Kth frame of the water flow image from the water flow video for the nth time. Preferably, the maximum value of n is in the range of [25,35]. In other words, it can be extracted 25 to 35 times at most.

[0070] For example, N=1, K=5, the maximum value of n is 25, the first frame of water flow image is extracted from the water flow video for the first time, the sixth frame of water flow image is extracted from the water flow video for the second time, the 11th frame of water flow image is extracted from the water flow video for the third time, and so on, the 121st frame of water flow image is extracted from the water flow video for the 25th time.

[0071] Among them, the N+(n-1)Kth frame area image is an image of the speed measurement area intercepted from the N+(n-1)Kth frame water flow image. The electronic device 102 intercepts the image of the speed measurement area from the N+(n-1)Kth frame water flow image to obtain the N+(n-1)Kth frame intercepted image. The intercepted image can be directly used as the N+(n-1)Kth frame area image, or the intercepted image is enhanced by histogram equalization to obtain the N+(n-1)Kth frame area image. The specific steps of histogram equalization are described below.

[0072] The electronic device 102 divides the N+(n-1)Kth frame captured image into multiple grids, performs grayscale processing on each grid to obtain a grayscale histogram of each grid, accumulates the grayscale histograms of multiple grids to obtain an accumulated histogram, obtains the grayscale values ​​of multiple reconstructed pixels based on the mapping relationship and the accumulated histogram, and obtains the N+(n-1)Kth frame regional image based on the grayscale values ​​of the multiple reconstructed pixels. The mapping relationship is as follows:

[0073]

[0074] Among them, h(k) represents the kth grid area, q represents the number of grids, and preferably, q=9, indicating division into 9 grids, H and W represent the height and width of the N+(n-1)Kth frame area image respectively, and p is the grayscale value of the reconstructed pixel.

[0075] Further, after the electronic device 102 obtains the N+(n-1)Kth frame regional image, it uses the first corner point detection algorithm and the second corner point detection algorithm to detect the N+(n-1)Kth frame regional image, respectively, to obtain a first corner point set and a second corner point set. The first corner point detection algorithm and the second corner point detection algorithm are different algorithms, and the embodiment of the present application does not specifically limit the corner point detection algorithm. The corner point threshold of the first corner point detection algorithm is less than the corner point threshold of the second corner point detection algorithm, so that the number of corner points in the first corner point set is greater than the number of corner points in the second corner point set.

[0076] For example, the first corner point detection algorithm is the Shi-Tomasi corner point algorithm. As a preferred method, the maximum number of corner points is set to [6, 10], the corner point quality factor is set to [0.1, 0.5], the minimum distance for corner point deletion is set to [3, 7], the window size is set to [5, 7], and the first corner point set obtained is P s ={C0,C1,C2,…,C s The second corner point detection algorithm is the Fast corner point algorithm. As a preferred method, the threshold of the Fast corner point is set to 50, and the second corner point set obtained is P F ={C0,C1,C2,…,C F}.

[0077] S203: Filter the first corner point set based on the second corner point set to obtain a third corner point set.

[0078] There are many specific screening methods, which are introduced below.

[0079] In the first method, the electronic device 102 can determine the corresponding corner point closest to each corner point in the first corner point set from the second corner point set, and obtain multiple groups of corner point pairs with the closest distance; determine the mean value of the coordinates of each group of corner point pairs as the coordinates of the corner point in the third corner point set, and obtain the third corner point set.

[0080] In the second method, the electronic device 102 can determine, from the second corner point set, corresponding corner points whose distances to each corner point in the first corner point set are less than the first threshold, to obtain multiple lists of corner points whose distances are less than the first threshold; determine the average of the coordinates of the corner points in each corner point list as the coordinates of the corner points in the third corner point set, to obtain the third corner point set.

[0081] Specifically, take the first corner point set as the outer loop, traverse the second corner point set of the inner loop in sequence, find the corner points with similar distances and average them. The formula is as follows:

[0082]

[0083] P old =Mean(MD(P s ,P F ,δ1))

[0084] Among them, img represents the resolution size of the N+(n-1)Kth frame image, P s represents the first corner point set, P F represents the second corner point set, δ1 represents the distance screening factor, MD represents the Manhattan distance, Mean represents the average, P old Represents the third corner point set.

[0085] S204: Obtain the water flow distance in the speed measurement area according to the third corner point set and multiple frames of water flow images.

[0086] Specifically, the electronic device 102 can use a sparse optical flow tracking algorithm on the third corner point set and multiple frames of water flow images to obtain tracking results of each corner point in the third corner point set, and then obtain the water flow distance in the speed measurement area according to the displacement of the tracked corner point. There are many sparse optical flow tracking algorithms, which are not specifically limited in the embodiments of the present application. For example, the pyramid-improved LK optical flow algorithm, as a preferred embodiment, the sliding window size is set to 10-15, and the number of pyramid layers is 2-4.

[0087] S205: Obtain the water flow velocity in the velocity measurement area according to the water flow distance in the velocity measurement area.

[0088] Specifically, the ratio between the water flow distance and the time interval in the speed measurement area is determined as the water flow velocity in the speed measurement area. The time interval refers to the running time of the algorithm, or the actual time between two corresponding frames of water flow images in the water flow video.

[0089] In an embodiment of the present application, a dual corner point detection algorithm is proposed, which uses the Fast corner point algorithm and the Shi-Tomasi corner point algorithm to perform dual corner point detection, overcoming the problem of inaccurate corner point output by the traditional single corner point detection algorithm. By relaxing the threshold of the Shi-Tomasi corner point detection algorithm, the corner points of the first filter are obtained, and then the Fast corner point detection algorithm is used to finely screen the corner points based on the first filter to obtain corner points with more obvious features, which is conducive to better tracking by the subsequent optical flow tracking algorithm.

[0090] In a possible embodiment, obtaining the water flow distance of the speed measurement area according to the third corner point set and multiple frames of water flow images includes the following steps S1.1-S1.4:

[0091] S1.1. Execute a sparse optical flow tracking algorithm on the third corner point set, the N+(n-1)Kth frame region image, and the N+nKth frame region image to obtain a fourth corner point set and a tracking state.

[0092] Specifically, the fourth corner point set represents the tracking results of each corner point in the third corner point set, and the corner points in the fourth corner point set correspond to the corner points in the third corner point set one by one. During the tracking process, the tracking of each corner point may not be successful, and there may be corner points that fail to be tracked. The tracking status is used to indicate whether the tracking of the corner point is successful. The tracking status has a value of 0 or 1. When the tracking status has a value of 0, it indicates that the tracking failed, and when the tracking status has a value of 1, it indicates that the tracking was successful.

[0093] S1.2. Filter out successfully tracked corner points from the fourth corner point set according to the tracking status to obtain a fifth corner point set; filter out successfully tracked corner points from the third corner point set according to the tracking status to obtain a sixth corner point set.

[0094] For example:

[0095]

[0096] Among them, ST represents the tracking state, P new represents the fourth corner point set, P′ new represents the fifth corner point set, P old represents the third corner point set, P′ old Represents the sixth corner point set.

[0097] S1.3. If the number of corner points in the fifth corner point set is greater than or equal to the second threshold, then calculating the pixel displacement between corresponding corner points in the fifth corner point set and the sixth corner point set.

[0098] Specifically, after obtaining the fifth corner point set, the electronic device 102 can determine whether the number of corner points in the fifth corner point set is greater than or equal to the second threshold. If so, the electronic device 102 can screen the corner points based on the size of the pixel displacement, and then determine the pixel displacement between the corresponding corner points.

[0099] In a possible embodiment, if the pixel displacement between the first corner point and the second corner point in the fifth corner point set is greater than or equal to the third threshold, the pixel displacement between the first corner point and the second corner point is determined as the pixel displacement between corresponding corner points. The first corner point is any corner point in the sixth corner point set, and the second corner point is a tracking result of the third corner point.

[0100] For example:

[0101]

[0102]

[0103] Among them, δ3 is the third threshold, P′ old is the sixth corner point set, P′ new is the fifth corner point set, The sixth set of corner points after filtering out the corner points whose pixel displacement is less than the third threshold value, The fifth corner point set is obtained by filtering out the corner points whose pixel displacement is less than the third threshold.

[0104] For example, A1 is the tracking result of B1, A2 is the tracking result of B2, and A3 is the tracking result of B3. Calculate the pixel displacement S1 between A1 and B1, the pixel displacement S2 between A2 and B2, and the pixel displacement S3 between A3 and B3 respectively. If S1<δ3, S2≥δ3, S3≥δ3, then calculate ... new Eliminate A1 and obtain From P′ old Eliminate B1 and obtain S2 and S3 are determined as the pixel displacement Dis = {S2, S3} between corresponding corner points.

[0105] In a possible embodiment, if the number of corner points in the fifth corner point set is less than the first threshold, double corner point detection is performed again to obtain the third corner point set.

[0106] Specifically, the electronic device 102 adjusts the parameters of the first corner point detection algorithm and the second corner point detection algorithm, executes the adjusted first corner point detection algorithm on the N+(n-1)Kth frame area image to obtain an adjusted first corner point set; executes the adjusted second corner point detection algorithm on the N+(n-1)Kth frame area image to obtain an adjusted second corner point set. For example, the corner point thresholds of the first corner point detection algorithm and the second corner point detection algorithm are adjusted to be smaller, so that the number of corner points in the adjusted first corner point set is greater than the number of corner points in the first corner point set, and the number of corner points in the adjusted second corner point set is greater than the number of corner points in the second corner point set. Then, the adjusted first corner point set is screened based on the adjusted second corner point set to obtain a third corner point set, wherein the screening process refers to the process of screening the first corner point set based on the second corner point set discussed above, which will not be repeated here.

[0107] S1.4. Obtain the water flow distance of the velocity measurement area according to the pixel displacement between the two corresponding corner points.

[0108] Specifically, the electronic device 102 can obtain the true distance between the N+(n-1)Kth frame area image and the N+nKth frame area image based on the pixel displacement and perspective matrix between the corresponding two corner points, and obtain the water flow distance of the speed measurement area based on the true distance between the N+(n-1)Kth frame area image and the N+nKth frame area image.

[0109] Among them, the perspective matrix is ​​used to indicate the mapping relationship between pixel displacement and real distance. The following describes how to map pixel displacement to real distance.

[0110] Select four points A, B, C, and D in the flow measurement area, map the pixel distance to the real distance, calculate the pixel displacement between ABCD, and use point A as the origin of the world coordinate system, point B as the X-axis direction, and point D as the Y-axis direction to calculate the coordinates of points B, C, and D according to the cosine theorem. Then calculate the pixel coordinates of A, B, C, and D corresponding to the original image, calculate the perspective transformation matrix, and complete the mapping of pixel distance to real distance. The formula is as follows:

[0111] Real x =cl(y),y≠x

[0112] Trans=f(Pix,Real)

[0113] Among them, cl(y) represents the real distance between points A, B, C, and D. The real coordinates of points A, B, C, and D are obtained by the cosine theorem. xRepresents the real-world coordinates of two points, Pix represents the pixel coordinates of the image corresponding to each point A, B, C, and D, Real represents the real-world coordinates of each point A, B, C, and D, f represents the perspective matrix, and Trans is the perspective matrix used to indicate the mapping relationship between Pix and Real.

[0114] In a possible embodiment, S1.4 includes the following steps:

[0115] S2.1. For different values ​​of n, multiple real distances between the N+(n-1)Kth frame area image and the N+nKth frame area image are obtained according to the pixel displacement between the corresponding two corner points and the perspective matrix.

[0116] Specifically, please refer to the previous discussion on how to map pixel displacement to real distance, which will not be repeated here. Preferably, the maximum value of n is in the range of [25,35], that is, it can be extracted 25 to 35 times at most. When n=1, the real distance between the Nth frame area image and the N+Kth frame area image is calculated. When n=2, the real distance between the N+Kth frame area image and the N+2Kth frame area image is calculated. For different values ​​of n, multiple real distances between the N+(n-1)Kth frame area image and the N+nKth frame area image are obtained.

[0117] S2.2. Sort the multiple real distances according to their numerical values, filter out the first M real distances and the last M real distances, and obtain the remaining multiple real distances.

[0118] Specifically, the multiple real distances are sorted according to their numerical values, which can be sorted from large to small or from small to large, and the first M real distances and the last M real distances are screened out, where M is a positive integer greater than 0, and M is the number of screened out, which is specifically set according to the number of the multiple real distances.

[0119] For example, the sorted real distances are {D1,D2,D3,…,D9,D 10}, M = 2, then D1 and D2 at the front, and D9 and D at the back are eliminated. 10 , the remaining multiple true distances are {D3,D4,…,D7,D8}.

[0120] S2.3. Obtain the water flow distance in the velocity measurement area according to the average of the remaining multiple true distances.

[0121] Specifically, the electronic device 102 may calculate the average of the remaining multiple real distances, and obtain the water flow distance in the speed measurement area according to the average.

[0122] For example: Dis mean =mean(Dislist [β,-β]), where β represents the number of minimum and maximum outliers removed after sorting, Dis list [β,-β] represents the remaining multiple true distances, Dis mean is the mean of the remaining multiple true distances.

[0123] In a possible embodiment, S2.3 includes the following steps:

[0124] S3.1. Calculate the true distance between the N+nKth frame region image and the N+nK+1th frame region image.

[0125] Specifically, how to calculate the true distance between the N+nKth frame area image and the N+nK+1th frame area image, please refer to the calculation process of the true distance between the N+(n-1)Kth frame area image and the N+nKth frame area image discussed in the previous article, which will not be repeated here.

[0126] S3.2. Add multiple real distances and the real distance between the N+nKth frame area image and the N+nK+1th frame area image to the distance list.

[0127] S3.3. Determine the water flow distance of the speed measurement area based on the average of the remaining multiple real distances and the distance list.

[0128] For example, the calculation formula is as follows:

[0129]

[0130] Where Dis is the water flow distance in the velocity measurement area, is the mean of multiple distances in the distance list, Dis mean is the mean of the remaining multiple true distances.

[0131] To more clearly understand the method for measuring water flow velocity provided in the present application, please refer to Figure 3 , a flow chart of a method for measuring water flow velocity is provided in the embodiment of the present application Figure 2 , the following combination Figure 3 The overall process of the water flow velocity measurement method is introduced.

[0132] S301, obtaining water flow video and selecting a speed measurement area.

[0133] For the specific process, please refer to the content discussed in S201 above, which will not be repeated here.

[0134] S302, executing a first corner point detection algorithm on the N+(n-1)Kth frame area image to obtain a first corner point set; executing a second corner point detection algorithm on the N+(n-1)Kth frame area image to obtain a second corner point set.

[0135] For the specific process, please refer to the content discussed in S202 above, which will not be repeated here.

[0136] S303: Filter the first corner point set based on the second corner point set to obtain a third corner point set.

[0137] For the specific process, please refer to the content discussed in S203 above, which will not be repeated here.

[0138] S304, executing a sparse optical flow tracking algorithm on the third corner point set, the N+(n-1)Kth frame region image, and the N+nKth frame region image to obtain a fourth corner point set and a tracking state.

[0139] For the specific process, please refer to the content discussed in S1.1 above, which will not be repeated here.

[0140] S305. Filter out successfully tracked corner points from the fourth corner point set according to the tracking status to obtain a fifth corner point set; and filter out successfully tracked corner points from the third corner point set according to the tracking status to obtain a sixth corner point set.

[0141] For the specific process, please refer to the content discussed in S1.2 above, which will not be repeated here.

[0142] S306: Determine whether the number of corner points in the fifth corner point set is greater than or equal to a second threshold.

[0143] If yes, execute S308; if no, execute S307 and then continue to execute S302.

[0144] S307: Adjust parameters of the first corner point detection algorithm and the second corner point detection algorithm.

[0145] S308: Calculate pixel displacements between corresponding corner points in the fifth corner point set and the sixth corner point set.

[0146] For the specific process, please refer to the content discussed in S1.3 above, which will not be repeated here.

[0147] S309 . For different values ​​of n, obtain multiple real distances between the N+(n-1)Kth frame region image and the N+nKth frame region image according to the pixel displacement between the corresponding two corner points and the perspective matrix.

[0148] For the specific process, please refer to the content discussed in S2.1 above, which will not be repeated here.

[0149] S310 , sorting the multiple real distances according to their numerical values, filtering out the first M real distances and the last M real distances, and obtaining the remaining multiple real distances.

[0150] For the specific process, please refer to the content discussed in S2.2 above, which will not be repeated here.

[0151] S311 , calculating the real distance between the N+nKth frame region image and the N+nK+1th frame region image.

[0152] For the specific process, please refer to the content discussed in S3.1 above, which will not be repeated here.

[0153] S312: Add multiple real distances and the real distance between the N+nKth frame area image and the N+nK+1th frame area image to a distance list.

[0154] For the specific process, please refer to the content discussed in S3.2 above, which will not be repeated here.

[0155] S313: Determine the water flow distance of the speed measurement area according to the average of the remaining multiple real distances and the distance list.

[0156] For the specific process, please refer to the content discussed in S3.3 above, which will not be repeated here.

[0157] S314. Obtain the water flow velocity in the velocity measurement area according to the water flow distance in the velocity measurement area.

[0158] For the specific process, please refer to the content discussed in S204 above, which will not be repeated here.

[0159] Combine the following Figure 3 An example is given to illustrate the overall process of the method for measuring water flow velocity.

[0160] For example, N=1, K=5, and the maximum value of n is 25, that is, 25 times are extracted from the water flow video. The first time, the first frame of water flow image is extracted from the water flow video, the second time, the sixth frame of water flow image is extracted from the water flow video, the third time, the 11th frame of water flow image is extracted from the water flow video, and so on. The 24th time, the 116th frame of water flow image is extracted from the water flow video, and the 25th time, the 121st frame of water flow image is extracted from the water flow video.

[0161] Execute S301, intercept the image of the speed measurement area from the 1st frame of the water flow image to obtain the 1st frame of the area image, intercept the image of the speed measurement area from the 6th frame of the water flow image to obtain the 6th frame of the area image, and so on, intercept the image of the speed measurement area from the 116th frame of the water flow image to obtain the 116th frame of the area image, intercept the image of the speed measurement area from the 121st frame of the water flow image to obtain the 121st frame of the area image.

[0162] For the 1st frame area image and the 6th frame area image, S302-S309 are executed to obtain the real distance between the 1st frame area image and the 6th frame area image. Similarly, for the 116th frame area image and the 121st frame area image, S302-S309 are executed to obtain the real distance between the 116th frame area image and the 121st frame area image, thereby obtaining 24 real distances.

[0163] Execute S310, if M=3, sort the 24 real distances according to their numerical values, filter out the first 3 real distances and the last 3 real distances, and obtain the remaining 18 real distances.

[0164] Execute S311 to calculate the real distance between the 121st frame area image and the 122nd frame area image, and execute S302-S309 for the 121st frame area image and the 122nd frame area image to obtain the real distance between the 121st frame area image and the 122nd frame area image.

[0165] Execute S313 to determine the water flow distance of the speed measurement area according to the average of the remaining 18 real distances and the distance list.

[0166] Execute S314 to obtain the water flow velocity in the speed measurement area according to the water flow distance in the speed measurement area.

[0167] In summary, due to the adoption of the above technical solution, this application has the following beneficial effects:

[0168] 1. The interval frame calibration algorithm is used to avoid the problem of no or very small displacement changes due to short time difference between consecutive frames, thus improving the accuracy of the output results.

[0169] 2. A dual corner point detection algorithm is used to avoid inaccurate corner points output by a single corner point algorithm. By relaxing the threshold of the Shi-Tomasi corner point detection algorithm, the corner points of the first filter are obtained. Then, the Fast corner point algorithm is used to finely screen the corner points of the first filter to obtain corner points with more obvious features, which is conducive to the subsequent optical flow tracking of the corner points.

[0170] 3. The tracking optimization algorithm is used to perform threshold screening on the pixel displacement between the corner points after tracking to improve the accuracy of the output average pixel displacement, and the number of corner points is judged. When the number of corner points is less than the preset threshold, double corner point detection is re-executed to avoid the problem of large output error caused by too few corner points each time the corner points are tracked, thereby improving the accuracy of the water flow velocity measurement results.

[0171] The present application also provides a device for measuring water flow velocity, which can be arranged in the electronic device 102 discussed above.

[0172] Please refer to Figure 4 , a water flow velocity measuring device comprises:

[0173] The acquisition module 401 is used to: acquire a water flow video and select a speed measurement area; the water flow video includes a continuous multi-frame water flow image;

[0174] The corner point detection module 402 is used to: execute a first corner point detection algorithm on the N+(n-1)Kth frame area image to obtain a first corner point set; execute a second corner point detection algorithm on the N+(n-1)Kth frame area image to obtain a second corner point set; the N+(n-1)Kth frame area image is an image of the speed measurement area intercepted from the N+(n-1)Kth frame water flow image; the number of corner points in the first corner point set is greater than the number of corner points in the second corner point set; N, n and K are all positive integers greater than 0; K represents a preset frame extraction interval;

[0175] The corner point screening module 403 is used to screen the first corner point set based on the second corner point set to obtain a third corner point set; the distance acquisition module 404 is used to obtain the water flow distance in the speed measurement area according to the third corner point set and multiple frames of water flow images; the flow velocity acquisition module 405 is used to obtain the water flow velocity in the speed measurement area according to the water flow distance in the speed measurement area.

[0176] Optionally, the corner point screening module 403 is specifically used for:

[0177] Determine, from the second set of corner points, corresponding corner points whose distances to each corner point in the first set of corner points are less than a first threshold, and obtain a list of multiple corner points whose distances are less than the first threshold;

[0178] The average of the corner point coordinates in each corner point list is determined as the coordinates of the corner points in the third corner point set to obtain the third corner point set.

[0179] Optionally, the distance obtaining module 404 is specifically used for:

[0180] A sparse optical flow tracking algorithm is performed on the third corner point set, the N+(n-1)Kth frame area image, and the N+nKth frame area image to obtain a fourth corner point set and a tracking state; the fourth corner point set represents the tracking results of each corner point in the third corner point set; the tracking state is used to indicate whether the tracking of the corner point is successful;

[0181] Filtering the successfully tracked corner points from the fourth corner point set according to the tracking state to obtain a fifth corner point set; filtering the successfully tracked corner points from the third corner point set according to the tracking state to obtain a sixth corner point set;

[0182] If the number of corner points in the fifth corner point set is greater than or equal to the second threshold, then calculating the pixel displacement between corresponding corner points in the fifth corner point set and the sixth corner point set;

[0183] According to the pixel displacement between the two corresponding corner points, the water flow distance in the velocity measurement area is obtained.

[0184] Optionally, the corner point detection module 402 is further used for:

[0185] If the number of corner points in the fifth corner point set is less than the second threshold, adjusting parameters of the first corner point detection algorithm and the second corner point detection algorithm;

[0186] The adjusted first corner point detection algorithm is executed on the N+(n-1)Kth frame area image to obtain an adjusted first corner point set; the adjusted second corner point detection algorithm is executed on the N+(n-1)Kth frame area image to obtain an adjusted second corner point set; the corner point screening module 403 is also used to: screen the adjusted first corner point set based on the adjusted second corner point set to obtain a third corner point set.

[0187] Optionally, the device further includes a pixel displacement determination module 406, and the pixel displacement determination module 406 is used to:

[0188] If the pixel displacement between the first corner point and the second corner point in the fifth corner point set is greater than or equal to the third threshold, the pixel displacement between the first corner point and the second corner point is determined as the pixel displacement between corresponding corner points; the first corner point is any corner point in the sixth corner point set, and the second corner point is the tracking result of the third corner point.

[0189] Optionally, the distance obtaining module 404 is specifically used for:

[0190] For different values ​​of n, multiple real distances between the N+(n-1)Kth frame region image and the N+nKth frame region image are obtained according to the pixel displacement between the corresponding two corner points and the perspective matrix; the perspective matrix is ​​used to indicate the mapping relationship between the pixel displacement and the real distance;

[0191] Sort the multiple real distances according to their numerical values, filter out the first M real distances and the last M real distances, and obtain the remaining multiple real distances; M is a positive integer greater than 0;

[0192] The water flow distance in the velocity measurement area is obtained based on the average of the remaining multiple true distances.

[0193] Optionally, the distance obtaining module 404 is specifically used for:

[0194] Calculate the true distance between the N+nKth frame area image and the N+nK+1th frame area image;

[0195] Adding a plurality of real distances and a real distance between the N+nKth frame region image and the N+nK+1th frame region image to a distance list;

[0196] According to the average of the remaining multiple real distances and the distance list, the water flow distance of the speed measurement area is determined; the calculation formula is as follows:

[0197]

[0198] Where Dis is the water flow distance in the velocity measurement area, is the mean of multiple distances in the distance list, Dis mean is the mean of the remaining multiple true distances.

[0199] The water flow velocity measuring device of the embodiment of the present application can implement any of the above-mentioned water flow velocity measuring methods, and the specific working process of each module in the water flow velocity measuring device can refer to the corresponding process in the above-mentioned method embodiment.

[0200] In the several embodiments provided in this application, it should be understood that the provided methods and devices can be implemented in other ways. For example, the device embodiments described above are only illustrative; for example, the division of a module is only a logical function division, and there may be other division methods in actual implementation, such as multiple modules can be combined or integrated into another device, or some features can be ignored or not executed.

[0201] The embodiment of the present application also discloses a computer device, which is equivalent to the electronic device 102 discussed above.

[0202] The computer device comprises a memory, a processor and a computer program stored in the memory and executable on the processor. When the processor executes the computer program, the water flow velocity measurement method as described above is implemented.

[0203] The embodiment of the present application also discloses a computer-readable storage medium.

[0204] A computer-readable storage medium stores a computer program that can be loaded by a processor and execute any of the above-mentioned methods for measuring water flow velocity.

[0205] Among them, computer-readable storage media can be any tangible medium that contains or stores a program, which can be used by or in conjunction with an instruction execution system, apparatus or device; the program code contained on the computer-readable medium can be transmitted using any appropriate medium, including but not limited to wireless, wire, optical cable, RF, etc., or any suitable combination of the above.

[0206] It should be noted that in the above embodiments, the description of each embodiment has different emphases, and for parts that are not described in detail in a certain embodiment, reference can be made to the relevant descriptions of other embodiments.

[0207] The above are all preferred embodiments of the present application, and are not intended to limit the protection scope of the present application. Any feature disclosed in this specification (including the abstract and drawings), unless otherwise stated, can be replaced by other equivalent or alternative features with similar purposes. That is, unless otherwise stated, each feature is only an example of a series of equivalent or similar features.

Claims

1. A method for measuring water flow velocity, characterized in that: include: Get water flow video and select the speed measurement area; The water flow video includes multiple frames of continuous water flow images; Execute a first corner point detection algorithm on the N+(n-1)Kth frame region image to obtain a first corner point set; A second corner point detection algorithm is executed on the N+(n-1)Kth frame area image to obtain a second corner point set; the N+(n-1)Kth frame area image is an image of a speed measurement area intercepted from the N+(n-1)Kth frame water flow image; the number of corner points in the first corner point set is greater than the number of corner points in the second corner point set; N, n and K are all positive integers greater than 0; K represents a preset frame extraction interval; Filtering the first corner point set based on the second corner point set to obtain a third corner point set; Obtaining a water flow distance in the speed measurement area according to the third corner point set and the multiple frames of water flow images; Obtaining the water flow velocity in the velocity measurement area according to the water flow distance in the velocity measurement area; The first corner point set is screened based on the second corner point set to obtain a third corner point set, including: Determine, from the second set of corner points, corresponding corner points whose distances to each corner point in the first set of corner points are less than a first threshold, and obtain a list of multiple corner points whose distances are less than the first threshold; The average of the corner point coordinates in each corner point list is determined as the coordinates of the corner points in the third corner point set to obtain the third corner point set.

2. A method for measuring water flow velocity according to claim 1, characterized in that: Obtaining the water flow distance of the speed measurement area according to the third corner point set and the multiple frames of water flow images, including: Execute a sparse optical flow tracking algorithm on the third corner point set, the N+(n-1)Kth frame area image, and the N+nKth frame area image to obtain a fourth corner point set and a tracking state; the fourth corner point set represents the tracking results of each corner point in the third corner point set; the tracking state is used to indicate whether the tracking of the corner point is successful; Filtering successfully tracked corner points from the fourth corner point set according to the tracking state to obtain a fifth corner point set; filtering successfully tracked corner points from the third corner point set according to the tracking state to obtain a sixth corner point set; If the number of corner points in the fifth corner point set is greater than or equal to a second threshold, calculating pixel displacements between corresponding corner points in the fifth corner point set and the sixth corner point set; The water flow distance of the velocity measurement area is obtained according to the pixel displacement between the two corresponding corner points.

3. A method for measuring water flow velocity according to claim 2, characterized in that: The method further comprises: If the number of corner points in the fifth corner point set is less than the second threshold, adjusting parameters of the first corner point detection algorithm and the second corner point detection algorithm; Executing the adjusted first corner point detection algorithm on the N+(n-1)Kth frame area image to obtain an adjusted first corner point set; executing the adjusted second corner point detection algorithm on the N+(n-1)Kth frame area image to obtain an adjusted second corner point set; The adjusted first corner point set is screened based on the adjusted second corner point set to obtain the third corner point set.

4. A method for measuring water flow velocity according to claim 2, characterized in that: Calculating pixel displacements between corresponding corner points in the fifth corner point set and the sixth corner point set includes: If the pixel displacement between the first corner point and the second corner point in the fifth corner point set is greater than or equal to a third threshold, the pixel displacement between the first corner point and the second corner point is determined as the pixel displacement between the corresponding corner points; the first corner point is any corner point in the sixth corner point set, and the second corner point is the tracking result of the third corner point.

5. A method for measuring water flow velocity according to claim 2, characterized in that: According to the pixel displacement between the two corresponding corner points, the water flow distance of the speed measurement area is obtained, including: For different values ​​of n, a plurality of real distances between the N+(n-1)Kth frame region image and the N+nKth frame region image are obtained according to the pixel displacement between the two corresponding corner points and the perspective matrix; the perspective matrix is ​​used to indicate the mapping relationship between the pixel displacement and the real distance; Sorting the multiple real distances according to their numerical values, filtering out the first M real distances and the last M real distances, and obtaining the remaining multiple real distances; M is a positive integer greater than 0; The water flow distance in the speed measurement area is obtained according to the average of the remaining multiple real distances.

6. A method for measuring water flow velocity according to claim 5, characterized in that: Obtaining the water flow distance of the speed measurement area according to the average of the remaining multiple real distances includes: Calculating the true distance between the N+nKth frame area image and the N+nK+1th frame area image; Adding the multiple real distances and the real distance between the N+nKth frame area image and the N+nK+1th frame area image to a distance list; The water flow distance of the speed measurement area is determined according to the average of the remaining multiple real distances and the distance list; the calculation formula is as follows: Wherein, Dis is the water flow distance in the velocity measurement area, is the mean of multiple distances in the distance list, Dis mean is the average of the remaining multiple true distances.

7. A device for measuring water flow velocity, characterized in that: include: An acquisition module is used to: acquire a water flow video and select a speed measurement area; the water flow video includes a continuous multi-frame water flow image; A corner point detection module, used for: executing a first corner point detection algorithm on the N+(n-1)Kth frame area image to obtain a first corner point set; executing a second corner point detection algorithm on the N+(n-1)Kth frame area image to obtain a second corner point set; the N+(n-1)Kth frame area image is an image of a speed measurement area intercepted from the N+(n-1)Kth frame water flow image; the number of corner points in the first corner point set is greater than the number of corner points in the second corner point set; N, n and K are all positive integers greater than 0; K represents a preset frame extraction interval; A corner point screening module is used to: determine, from the second corner point set, corresponding corner points whose distances to each corner point in the first corner point set are less than a first threshold, and obtain multiple lists of corner points whose distances are less than the first threshold; determine the average of the corner point coordinates in each corner point list as the coordinates of the corner points in the third corner point set, and obtain the third corner point set; A distance acquisition module, used to: obtain the water flow distance of the speed measurement area according to the third corner point set and the multiple frames of water flow images; The flow velocity acquisition module is used to obtain the water flow velocity in the speed measurement area according to the water flow distance in the speed measurement area.

8. A computer device, characterized in that: The method comprises a memory and a server, wherein the memory stores a computer program according to any one of the methods of claims 1 to 6 which is loaded and executed by the server.

9. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores instructions, and when the instructions are executed, the method steps according to any one of claims 1 to 6 are performed.

Images