A method and device for determining flow velocity based on optical flow estimation and feature filtering
Through the methods of optical flow estimation and feature filtering, the problem of low accuracy in measuring the flow velocity of optical flow estimation in high flow velocity scenarios is solved. Through feature filtration and displacement calculation, the accuracy of water flow velocity measurement is improved.
Patent Information
- Application Number
- CN202411306272.4
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2024-09-19
- Publication Date
- 2024-12-03
- Estimated Expiration
- 2044-09-19
AI Technical Summary
In high flow velocity scenarios, the surface conditions of the water body are complex, resulting in low accuracy of the flow velocity determined by the velocity measurement algorithm based on optical flow estimation.
The flow rate determination method based on optical flow estimation and feature filtering is adopted. By obtaining continuous image frames on the surface of the water body, the motion duration and trajectory of the motion characteristics are determined based on the optical flow estimation calculation method, the interference characteristics are cleared, the effective characteristics are retained, and the effective characteristics are based on the actual situation of the effective characteristics. Displacement and movement time determine the flow rate of the water body.
It improves the accuracy of flow velocity measurement, effectively reduces the error of optical flow estimation in high flow velocity scenarios, and ensures the reliability of flow velocity detection.
Smart Images

Figure CN118823070B_ABST
Abstract
Description
Technical Field
[0001] The present application relates to the field of flow velocity measurement, and in particular to a flow velocity determination method and device based on optical flow estimation and feature filtering. Background Art
[0002] In the management of river water scheduling, river flow velocity measurement provides effective data support for important decisions such as understanding water conditions, allocating water volume, and disaster prevention. In river scenarios, common flow velocity detection methods are divided into contact measurement methods and non-contact measurement methods. The contact measurement method requires the sensor device to be placed in the water, and requires human wading participation, which is difficult to meet the requirements of large-scale deployment. The non-contact flow velocity detection methods are mainly ultrasonic velocity measurement methods and image velocity measurement methods. However, the ultrasonic measurement method cannot obtain a large range of river surface flow field information in a short time, while the surface velocity measurement method based on image algorithm has the advantages of easy installation of measurement equipment, low cost, less human intervention, and the ability to measure a large range of river surface flow field information. The image velocity measurement method includes a velocity measurement algorithm based on optical flow estimation.
[0003] However, in high-flow scenarios, the water surface conditions are complex, and the velocity measurement algorithm based on optical flow estimation is easily affected by the water surface conditions, resulting in low accuracy of the determined flow velocity. Summary of the invention
[0004] To this end, the present application provides a flow velocity determination method and device based on optical flow estimation and feature filtering to solve the problem in the prior art of low accuracy of flow velocity determined by a velocity measurement algorithm based on optical flow estimation due to the complex surface conditions of water bodies in high flow velocity scenarios.
[0005] In order to achieve the above-mentioned objectives, the first aspect of the present application provides a flow velocity determination method based on optical flow estimation and feature filtering, the method comprising: acquiring continuous image frames of the water body surface; determining the movement duration and movement trajectory of each motion feature on the water body surface based on the optical flow estimation algorithm and the continuous image frames; based on the movement duration and movement trajectory, removing interference features in the motion features to retain effective features; determining the actual displacement of the effective features based on the movement trajectory of the effective features; and determining the flow velocity of the water body based on the actual displacement and movement time of the effective features.
[0006] According to the first aspect, based on the optical flow estimation algorithm and continuous image frames, the motion duration and motion trajectory of each motion feature on the water surface are determined, including: based on the dense optical flow estimation algorithm and continuous image frames, an optical flow estimation image sequence is obtained; based on the optical flow estimation image sequence, the motion duration and motion trajectory of each motion feature is determined.
[0007] According to the first aspect, or any one of the implementations of the first aspect above, based on a dense optical flow estimation algorithm and continuous image frames, an optical flow estimation image sequence is obtained, including: determining multiple groups of adjacent image frames from continuous image frames; inputting adjacent image frames into a dense optical flow estimation algorithm model in sequence according to the acquisition order of the previous image frame in the adjacent image frames to obtain a corresponding output optical flow estimation image; the optical flow estimation image includes motion displacement information corresponding to each position in the previous image frame in the subsequent image frame; and multiple frames of optical flow estimation images constitute an optical flow estimation image sequence in the output order.
[0008] According to the first aspect, based on the motion duration and motion trajectory, interference features in the motion features are removed to retain valid features, including: from all motion features, removing motion features whose motion duration is less than a preset duration threshold, and removing motion features corresponding to motion trajectories that are inconsistent with the flow direction of the water body to retain valid features.
[0009] According to the first aspect, or any one of the implementation methods of the first aspect above, the motion features corresponding to the motion trajectory inconsistent with the flow direction of the water body are cleared, including: determining the flow field direction vector based on the vertex coordinates of the flow field area in the image frame; determining the motion direction vector of the motion feature based on the motion trajectory of the motion feature; and clearing the motion features whose vector inner product of the motion direction vector and the flow field direction vector is less than 0.
[0010] According to the first aspect, or any implementation of the first aspect above, the actual displacement of the effective feature is determined according to the motion trajectory of the effective feature, including: determining the correspondence between the pixel coordinates and the actual coordinates of the flow field area; and determining the actual displacement of the effective feature according to the correspondence.
[0011] According to the first aspect, or any one of the implementation methods of the first aspect above, the flow velocity of the water body is determined based on the actual displacement and movement time of the effective feature, including: determining the flow velocity of the effective feature based on the actual displacement and movement time of the effective feature; determining the flow velocity interval of the effective feature with the lowest effective feature flow velocity and the highest effective feature flow velocity as the boundary; dividing the flow velocity interval into a preset number of sub-flow velocity intervals; determining the number of effective features contained in each sub-flow velocity interval; determining the average flow velocity of the effective features contained in the sub-flow velocity interval with the largest number; and determining the average flow velocity as the flow velocity of the water body.
[0012] The second aspect of the present application provides a flow rate determination device for optical flow estimation and feature filtering, the device comprising: an image acquisition module, used to acquire continuous image frames of the water body surface; an optical flow estimation module, used to determine the motion duration and motion trajectory of each motion feature on the water body surface based on the optical flow estimation algorithm and the continuous image frames; an interference removal module, used to remove interference features in the motion features based on the motion duration and motion trajectory to retain effective features; a perspective transformation module, used to determine the actual displacement of the effective feature according to the motion trajectory of the effective feature; a flow rate determination module, used to determine the flow rate of the water body based on the actual displacement and motion time of the effective feature.
[0013] The third aspect of the present application provides an electronic device, which includes: a camera, which is used to obtain continuous image frames of the surface of a water body; one or more processors; a storage device, on which one or more programs are stored, and when the one or more programs are executed by one or more processors, the one or more processors implement any one of the methods in the first aspect; one or more I / O interfaces, connected between the processor and the memory, configured to implement information interaction between the processor and the memory.
[0014] A fourth aspect of the present application provides a computer-readable medium having a computer program stored thereon, which implements any one of the methods of the first aspect when the program is executed by a processor.
[0015] The present application has the following advantages: The embodiment of the present application provides a flow velocity determination method and device based on optical flow estimation and feature filtering. In this method, firstly, continuous image frames of the water body surface are obtained, and based on the optical flow estimation algorithm and the continuous image frames, the motion duration and motion trajectory of each motion feature on the water body surface are determined; secondly, based on the motion duration and motion trajectory, the interference features in the motion features are removed to retain the effective features; then, according to the motion trajectory of the effective features, the actual displacement of the effective features is determined, and based on the actual displacement and motion time of the effective features, the flow velocity of the water body is determined. In this flow velocity determination method, the optical flow estimation is first used to improve the recognition rate of the tracer features, and then the interference features are removed to retain the feature trajectory with higher credibility. Finally, the flow velocity of the water body is determined based on the effective features, which can effectively improve the accuracy of the determined water body flow velocity. BRIEF DESCRIPTION OF THE DRAWINGS
[0016] The accompanying drawings are used to provide further understanding of the present application and constitute a part of the specification. Together with the following specific embodiments, they are used to explain the present application, but do not constitute a limitation to the present application.
[0017] Figure 1 A flow chart of a method for determining flow velocity based on optical flow estimation and feature filtering provided in an embodiment of the present application;
[0018] Figure 2A flow chart of a method for determining the motion duration and motion trajectory of various motion features on the surface of a water body provided in an embodiment of the present application;
[0019] Figure 3 A flowchart of a method for acquiring an optical flow estimation image sequence is provided for an embodiment of the present application;
[0020] Figure 4 A flowchart of an interference feature removal method provided in an embodiment of the present application;
[0021] Figure 5 An example diagram of an image frame provided in an embodiment of the present application;
[0022] Figure 6 A perspective comparison schematic diagram provided for an embodiment of the present application;
[0023] Figure 7 A schematic diagram of a flow chart of a method for determining the flow velocity of a water body provided in an embodiment of the present application;
[0024] Figure 8 A flow rate determination device for optical flow estimation and feature filtering provided in an embodiment of the present application;
[0025] Fig. 9 A schematic diagram of the structure of an electronic device provided in an embodiment of the present application. DETAILED DESCRIPTION
[0026] In order to enable those skilled in the art to better understand the technical solutions of the present disclosure, the exemplary embodiments of the present disclosure are described below in conjunction with the accompanying drawings, including various details of the embodiments of the present disclosure to facilitate understanding, which should be considered as merely exemplary. Therefore, those of ordinary skill in the art should recognize that various changes and modifications can be made to the embodiments described herein without departing from the scope and spirit of the present disclosure. Similarly, for the sake of clarity and conciseness, the description of well-known functions and structures is omitted in the following description.
[0027] In the absence of conflict, the various embodiments of the present disclosure and the various features therein may be combined with each other.
[0028] As used herein, the term "and / or" includes any and all combinations of one or more of the associated listed items.
[0029] The terms used in this application are only used to describe specific embodiments and are not intended to limit the present application.As used in this application, the singular forms "a", "an" and "the" are intended to include the plural forms as well, unless the context clearly indicates otherwise.
[0030] When the terms “comprising” and / or “made of…” are used in this application, it specifies the existence of the stated features, integers, steps, operations, elements and / or components, but does not exclude the existence or addition of one or more other features, integers, steps, operations, elements, components and / or groups thereof.
[0031] The embodiments described in the present application may be described with reference to plan views and / or cross-sectional views by means of ideal schematic diagrams of the present application. Therefore, the exemplary illustrations may be modified according to manufacturing techniques and / or tolerances.
[0032] Unless otherwise defined, all terms (including technical and scientific terms) used in this application have the same meaning as those commonly understood by those of ordinary skill in the art. It will also be understood that terms such as those defined in commonly used dictionaries should be interpreted as having a meaning consistent with their meaning in the context of the relevant art and this application, and will not be interpreted as having an idealized or overly formal meaning unless explicitly defined in this application.
[0033] In the management of river water dispatch, river flow velocity measurement provides effective data support for important decisions such as understanding water conditions, allocating water, and disaster prevention. In most rivers, the flow rate of the river section can be calculated by velocity distribution and cross-sectional area. However, the traditional measurement method using flow velocity instruments has a small detection range, often can only measure a single point, and requires manual wading. This measurement method cannot guarantee the personal safety of measurement personnel during the flood season.
[0034] In river scenarios, common flow velocity detection methods are divided into contact measurement methods and non-contact measurement methods. The contact measurement method places the measuring instrument underwater and transmits flow velocity information through a specific sensor. Commonly used contact velocimeters include propeller velocimeters and acoustic Doppler velocimeters. The propeller velocimeter requires the sensor to be consistent with the actual flow direction. The measurement implementation is relatively difficult, and the measurement accuracy is greatly affected by human factors and is difficult to guarantee. The Doppler velocimeter can measure three-dimensional flow velocity more accurately, but the Doppler velocimeter is expensive and easily affected by noise in the water body. Especially during the flood season, floating particles in the water have a greater impact on the measurement accuracy. In general, the contact measurement method requires the sensor device to be placed in the water, and requires human participation in wading, which is difficult to meet the requirements of large-scale deployment.
[0035] The non-contact flow velocity detection methods mainly include ultrasonic velocity measurement method and image velocity measurement method. The ultrasonic velocity meter uses the Doppler effect to transmit microwaves with energy to the river surface from a certain angle, and converts the velocity at a certain point on the river surface after the sensor obtains the echo. However, the ultrasonic measurement method can often only detect the velocity of a small area of the water surface, and cannot obtain a large range of river surface flow field information in a short time. With the development of computer vision technology, the surface velocity measurement method based on image algorithm has the advantages of easy installation of measurement equipment, low cost, less manual intervention, and the ability to measure a wide range of river surface flow field information. Image velocity measurement methods include velocity measurement algorithms based on optical flow estimation.
[0036] The velocity measurement algorithm based on optical flow estimation follows three basic assumptions: 1) the brightness between adjacent frames is constant; 2) the displacement of objects moving between adjacent frames is relatively small; 3) the optical flow of a pixel is consistent with the optical flow of its neighborhood. However, in the case of high flow fields, there are noises such as ripple disturbances, shadow reflections, and strong light flickering on the river surface, and the tracer features are difficult to capture due to the high flow rate. These factors lead to the three basic assumptions of optical flow estimation not being fully established, which greatly affects the accuracy of the optical flow estimation algorithm and leads to large errors in the accuracy of velocity detection.
[0037] To this end, this application proposes a flow velocity measurement algorithm under high flow field based on optical flow estimation and trajectory filtering algorithm to reduce the flow velocity measurement error under complex environmental interference.
[0038] Figure 1 A flow chart of a flow velocity determination method based on optical flow estimation and feature filtering provided in an embodiment of the present application. The flow velocity determination method based on optical flow estimation and feature filtering is applied to a flow velocity determination device, which is deployed in an electronic device. Figure 1 As shown, the flow rate determination method based on optical flow estimation and feature filtering includes: steps S101 to S105.
[0039] Step S101: Acquire continuous image frames of the water surface.
[0040] The continuous image frames may be video frames in a video.
[0041] In one embodiment, continuous image frames of the surface of the water body are collected using an image acquisition device installed above or to the side of the water body.
[0042] The flow velocity determination method based on optical flow estimation and feature filtering in the embodiment of the present application is applied to high flow velocity scenarios. Since the motion features (tracers) on the surface of the water body in high flow velocity scenarios move extremely fast, the image acquisition device can be a high-definition camera, and the acquisition frame rate of the camera is greater than or equal to a preset frame rate value, and the preset frame rate value is set to at least 30FPS.
[0043] Step S102: Based on the optical flow estimation algorithm and the continuous image frames, the motion duration and motion trajectory of each motion feature on the water surface are determined.
[0044] Among them, the optical flow estimation algorithm is an algorithm used to estimate the displacement between pixels between adjacent frames.
[0045] The motion characteristics of the water surface refer to the characteristics of the water surface that can be captured by the image acquisition equipment, including artificially placed tracers, leaves, branches, etc. It should be noted that the shadows, flickering lights, splashing water, ripple disturbances on the river surface, etc. on the water surface also have motion behaviors, so these characteristics will also be considered as the motion characteristics of the water surface.
[0046] Figure 2 A flow chart of a method for determining the motion duration and motion trajectory of various motion features on the surface of a water body provided in an embodiment of the present application. In one embodiment, Figure 2 As shown, the step of determining the motion duration and motion trajectory of each motion feature on the water surface based on the optical flow estimation algorithm and continuous image frames includes: step S201 and step S202.
[0047] Step S201: Obtain an optical flow estimation image sequence based on a dense optical flow estimation algorithm and continuous image frames.
[0048] Among them, the dense optical flow estimation algorithm is, for example, the Farneback dense optical flow algorithm.
[0049] It should be noted that in a high flow environment, the feature tracking algorithm based on the sparse optical flow algorithm cannot capture the characteristics of the high-speed moving tracer, which will result in the loss of a large number of motion features and the poor stability of the flow velocity estimation algorithm. For this reason, the Farneback dense optical flow estimation algorithm is used in the embodiment of the present application. The Farneback is a gradient-based optical flow estimation method that uses a polynomial fitting plane to evaluate the motion of each point in the image.
[0050] Figure 3 A flowchart of a method for acquiring an optical flow estimation image sequence is provided for an embodiment of the present application. Figure 3 , based on the dense optical flow estimation algorithm and continuous image frames, obtaining the optical flow estimation image sequence (the above step S201) includes: the following steps S201a-S201c.
[0051] Step S201a: determining a plurality of groups of adjacent image frames from the continuous image frames.
[0052] In one example, continuous image frames include frame 1, frame 2, frame 3, frame 4, ..., frame n-1, frame n; then multiple groups of adjacent image frames are determined to include: frame 1 frame 2, frame 2 frame 3, frame 3 frame 4, ..., frame n-1 frame n.
[0053] Step S201b: input the adjacent image frames into the dense optical flow estimation algorithm model in sequence according to the acquisition order of the previous image frame in the adjacent image frames to obtain the corresponding output optical flow estimation image.
[0054] The optical flow estimation image is a two-channel image of the same size as the flow field area, including the corresponding motion displacement information of each position in the previous frame image in the next frame image. The position in the image frame refers to the pixel position, and the relative displacement can be represented by a coordinate value, for example, by the coordinate value (x, y) in the coordinate axis with the bottom edge of the image frame as the x-axis and the left edge as the y-axis.
[0055] The dense optical flow estimation algorithm model is a model for outputting an optical flow estimation image based on adjacent image frames.
[0056] Taking the aforementioned multiple groups of adjacent image frames determined: frame 1 frame 2, frame 2 frame 3, frame 3 frame 4, ..., frame n-1 frame n as an example, step S201b is explained: in the group of adjacent image frames of frame 1 frame 2, the previous frame image is frame 1; in the group of adjacent image frames of frame 2 frame 3, the previous frame image is frame 2... Then the acquisition sequence of the previous frame image in the adjacent image frames is: frame 1, frame 2, frame 3, frame 4, ..., frame n-1, which is the input sequence of the adjacent image frames. First, input the dense optical flow estimation algorithm model of frame 1 and frame 2 to obtain the corresponding output optical flow estimation image; then, input the dense optical flow estimation algorithm model of frame 2 frame 3 to obtain the corresponding output optical flow estimation image; then, input the dense optical flow estimation algorithm model of frame 3 frame 4 to obtain the corresponding output optical flow estimation image; ...; input the dense optical flow estimation algorithm model of frame n-1 frame n to obtain the corresponding output optical flow estimation image, and obtain a total of n-1 frames of optical flow estimation images.
[0057] Step S201c: multiple frames of optical flow estimation images form an optical flow estimation image sequence in an output order.
[0058] Taking the aforementioned multiple groups of adjacent image frames: frame 1 frame 2, frame 2 frame 3, frame 3 frame 4, ..., frame n-1 frame n, which are sequentially input into the dense optical flow estimation algorithm model to obtain the corresponding output optical flow estimation images as an example, the optical flow estimation image sequence constructed in the output order includes: the optical flow estimation image corresponding to frame 1 frame 2, the optical flow estimation image corresponding to frame 2 frame 3, the optical flow estimation image corresponding to frame 3 frame 4, ..., the optical flow estimation image corresponding to frame n-1 frame n.
[0059] Step S202: Determine the motion duration and motion trajectory of each motion feature based on the optical flow estimation image sequence.
[0060] Since the optical flow estimation image includes the motion displacement information corresponding to each position in the previous frame image in the next frame image, the motion trajectory of each motion feature can be determined based on the optical flow estimation image sequence.
[0061] In addition, the image frames captured by the image acquisition device contain a timestamp, and the interval between the image acquisition device and the image frames is fixed. Therefore, based on the optical flow estimation image sequence, the motion duration of each motion feature can be determined. For example, if the interval between the image acquisition device and the image frames is 1 / 30s, and the optical flow estimation image containing motion feature A is 30 frames, then the motion duration of the motion feature A is 1s.
[0062] Step S103: based on the motion duration and motion trajectory, remove interference features in the motion features to retain valid features.
[0063] It should be noted that the movement characteristics of the water surface include not only artificially placed tracers, leaves, branches, etc., but also interference characteristics such as shadows on the water surface, flickering light and shadows, splashing water on the water surface, and ripple disturbances on the river surface.
[0064] In high-velocity scenarios, the surface of the water body is complex and there are more interference features. In order to ensure the accuracy of the determined flow rate, it is necessary to filter out the interference features from the motion features, thereby retaining the effective features that help determine the water body flow rate.
[0065] Figure 4 A flowchart of an interference feature removal method provided in an embodiment of the present application. Figure 4 The step of removing interference features in motion features based on motion duration and motion trajectory to retain valid features (the above-mentioned step S103) includes: the following steps S401-S402.
[0066] Step S401: Remove the motion features whose motion duration is less than a preset duration threshold from all motion features.
[0067] The preset duration threshold may be set according to the general motion duration of the interference feature in actual applications, for example, it may be set to 0.1 seconds, or to 0.2 seconds.
[0068] It should be noted that the motion trajectory of interference features formed by shadows, flickering lights, etc. has the characteristic of short duration, while the motion of the tracer has a long continuity in time. In the embodiment of the present application, from all motion features, motion features with a motion duration less than a preset duration threshold are removed to filter out such interference features with short motion duration.
[0069] Step S402: Remove the motion features corresponding to the motion trajectories inconsistent with the flow direction of the water body from all the motion features.
[0070] It should be noted that some interference features may produce motion trajectories that are contrary to the direction of the flow field, such as splashing water on the water surface, ripple disturbances on the surface of the river, etc. In the embodiment of the present application, the motion features corresponding to the motion trajectories that are inconsistent with the flow direction of the water body are removed from all the motion features, and the features whose motion trajectories are contrary to the direction of the flow field can be filtered out.
[0071] In one embodiment, removing the motion features corresponding to the motion trajectory inconsistent with the water flow direction from all the motion features includes: the following steps 1 to 3.
[0072] Step 1: Determine the flow field direction vector based on the vertex coordinates of the flow field area in the image frame.
[0073] Figure 5 This is an example of an image frame provided in an embodiment of the present application. Figure 5 The image frame shown in FIG. 1 shows a flow field region in which the edges AB, BC, CD, and DA are formed, and the vertex coordinates of the flow field region are the pixel coordinates of A, B, C, and D. The flow field direction vector is , where M and N are the midpoints of sides AB and CD respectively.
[0074] Step 2: Based on the motion trajectory of the motion feature, determine the motion direction vector of the motion feature.
[0075] The starting point and the end point of the motion feature are taken as the endpoints, and the direction from the starting point to the end point is taken as the motion direction to determine the motion direction vector of the motion feature. Figure 5 As shown, the starting point of the motion feature is S, the end point is S', and the motion direction vector of the motion feature is The motion direction vector of this motion feature is The flow field direction vector is The angle between .
[0076] Step 3: Clear the motion features whose vector inner product between the motion direction vector and the flow field direction vector is less than 0.
[0077] Among them, when the vector inner product of the motion direction vector of the motion feature and the flow field direction vector is less than 0, it means that the angle θ between the motion direction of the motion feature and the flow field direction is greater than 90°, that is, the motion feature cannot represent the flow velocity of the water body, and it is highly likely to be an interference feature generated by the water surface. By removing the motion features whose vector inner product of the motion direction vector and the flow field direction vector is less than 0, the interference features corresponding to the motion trajectory inconsistent with the water flow direction can be removed.
[0078] Through the above step S103, the embodiment of the present application can remove interference features from all motion features, which can effectively avoid the influence of interference features on flow rate determination, thereby improving the accuracy of flow rate determined in subsequent processes.
[0079] Step S104: Determine the actual displacement of the effective feature according to the motion trajectory of the effective feature.
[0080] Among them, the motion trajectory of the effective feature is the pixel trajectory determined according to the image frame, and the determination of the water flow rate is based on the actual scene. Therefore, the motion trajectory of the effective feature needs to be converted into the actual displacement in the actual scene.
[0081] In one embodiment, determining the actual displacement of the effective feature according to the motion trajectory of the effective feature includes: the following steps 1 and 2.
[0082] Step 1: Determine the correspondence between the pixel coordinates of the flow field area and the actual coordinates.
[0083] The pixel coordinates of the flow field area refer to the coordinates in the image frame, and the actual coordinates of the flow field area refer to the coordinates of the flow field area in the real scene, which can be represented by longitude and latitude.
[0084] Figure 6 A schematic diagram of a viewing angle comparison provided in an embodiment of the present application. Figure 6 Due to different viewpoints, the flow field area surrounded by the four points A', B', C', and O' is a rectangle in the real world (see Figure 6 (1) in the figure), but in the image it appears as an irregular trapezoid ABCO (see Figure 6 To determine the actual displacement of the effective feature, it is necessary to first determine the correspondence between the pixel coordinates of the flow field area and the actual coordinates.
[0085] In one embodiment, determining the correspondence between the pixel coordinates and the actual coordinates of the flow field area includes: step a and step b.
[0086] Step a: Obtain the actual coordinates of the vertices in the flow field area.
[0087] based on Figure 5 and Figure 6 , to explain how to obtain the actual coordinates of the vertices in the flow field area. Figure 5 Point D in is taken as the plane origin O' of the actual coordinates. , The directions indicated are the x and y directions in the actual coordinate plane. The actual plane coordinates corresponding to the four points can be obtained through on-site field measurements.
[0088] Step b: Based on the vertex pixel coordinates of the flow field area in the image frame and the actual coordinates of the vertex of the flow field area, the corresponding relationship between the pixel coordinates of the flow field area and the actual coordinates is determined through homography plane coordinate transformation.
[0089] Step 2: Determine the actual displacement of the effective feature based on the correspondence between the pixel coordinates of the flow field area and the actual coordinates.
[0090] In an embodiment of the present application, the homography plane coordinate transformation can utilize the image coordinates and several groups of corresponding coordinate points in the actual coordinate plane to solve the perspective transformation matrix that converts the image coordinate system into the actual coordinate system. Utilizing this nonlinear transformation, the points in the two-dimensional image coordinates can be mapped to the actual coordinates, thereby determining the correspondence between the pixel coordinates of the flow field area and the actual coordinates, and then determining the actual displacement of the motion feature based on the correspondence.
[0091] Step S105: Determine the flow velocity of the water body based on the actual displacement and movement time of the effective features.
[0092] In the embodiment of the present application, the flow velocity of an effective feature in the water body can be determined based on the actual displacement and movement time of the effective feature. However, in practical applications, the surface of the water body generally contains multiple effective features, and the flow velocity of the water body needs to be determined based on the flow velocities of the multiple effective features.
[0093] Figure 7 A schematic diagram of a flow chart of a method for determining the flow velocity of a water body provided in an embodiment of the present application. Figure 7 The step of determining the flow velocity of the water body based on the actual displacement and movement time of the effective features (the above-mentioned step S105) includes: steps S701 to S706.
[0094] Step S701: Determine the flow velocity of the effective feature based on the actual displacement and movement time of the effective feature.
[0095] The flow velocity of the effective feature is the ratio of the actual displacement of the effective feature to the movement time.
[0096] It should be noted that the surface of a water body generally contains multiple effective features. Since the multiple effective features are located at different positions on the surface of the water body, and the flow velocities at different positions on the surface of the water body are different, the flow velocities of the effective features on the surface of the water body are also different. In addition, there may be very few interference features in the effective features that have not been removed through the above steps. The flow velocity of the interference features may be different from the flow of the tracer, which leads to differences in the flow velocity of the effective features on the surface of the water body.
[0097] After determining the flow rates of the effective characteristics, the determined flow rates of the effective characteristics are sorted to determine the lowest effective characteristic flow rate and the highest effective characteristic flow rate.
[0098] Step S702: determine the effective characteristic flow rate interval with the lowest effective characteristic flow rate and the highest effective characteristic flow rate as the boundary.
[0099] Step S703: Divide the flow rate interval into a preset number of sub-flow rate intervals.
[0100] The value of the preset number can be set according to the actual situation. According to the actual test results, the preferred value range of the preset number is [5, 10], that is, any one of 5, 6, 7, 8, 9, 10. When the value range of the preset number is between [5, 10], a relatively stable speed measurement effect can be finally achieved.
[0101] Step S704: Determine the number of valid features contained in each sub-flow rate interval.
[0102] After determining the number of valid features contained in each sub-flow rate interval, the sub-flow rate intervals may be sorted from large to small according to the number of valid features contained, so as to determine the sub-flow rate interval containing the largest number of valid features.
[0103] Step S705: Determine the average flow rate of the valid features contained in the sub-flow rate interval containing the largest number of valid features.
[0104] Taking the sub-flow rate interval with the largest number of valid features, the number of valid features is 5, and the flow velocities of the 5 valid features are 20m / s, 21m / s, 21m / s, 20m / s, and 20m / s respectively. The average flow velocity of the valid feature is (20+20+21+21+20+20) / 6=20.4m / s.
[0105] Step S706: Determine the average flow velocity as the flow velocity of the water body.
[0106] In an embodiment of the present application, the average flow velocity of the effective features in the sub-flow velocity interval containing the largest number of effective features is used as the flow velocity of the water body, which can further reduce the influence of the interfering features that have not been removed through the aforementioned steps, thereby further improving the accuracy of the determined water body flow velocity.
[0107] The embodiment of the present application provides a flow velocity determination method based on optical flow estimation and feature filtering. In this method, firstly, continuous image frames of the water body surface are obtained, and based on the optical flow estimation algorithm and the continuous image frames, the motion duration and motion trajectory of each motion feature on the water body surface are determined; secondly, based on the motion duration and motion trajectory, the interference features in the motion features are removed to retain the effective features; then, according to the motion trajectory of the effective features, the actual displacement of the effective features is determined, and based on the actual displacement and motion time of the effective features, the flow velocity of the water body is determined. In this flow velocity determination method, the optical flow estimation is first used to improve the recognition rate of the tracer features, and then the interference features are removed to retain the feature trajectory with higher credibility. Finally, the flow velocity of the water body is determined based on the effective features, which can effectively improve the accuracy of the determined water body flow velocity.
[0108] The step division of the above methods is only for clear description. When implemented, they can be combined into one step or some steps can be split and decomposed into multiple steps. As long as they include the same logical relationship, they are all within the protection scope of this application; adding insignificant modifications to the algorithm or process or introducing insignificant designs without changing the core design of the algorithm and process are all within the protection scope of this patent.
[0109] In a second aspect, an embodiment of the present application provides a flow rate determination device for optical flow estimation and feature filtering.
[0110] Figure 8 A flow rate determination device for optical flow estimation and feature filtering is provided in an embodiment of the present application. Figure 8 The flow rate determination device includes: an image acquisition module 81, an optical flow estimation module 82, an interference removal module 83, a perspective transformation module 84 and a flow rate determination module 85.
[0111] The image acquisition module 81 is used to acquire continuous image frames of the water surface.
[0112] The optical flow estimation module 82 is used to determine the motion duration and motion trajectory of each motion feature on the water surface based on the optical flow estimation algorithm and continuous image frames.
[0113] The interference removal module 83 is used to remove interference features in the motion features based on the motion duration and motion trajectory to retain effective features.
[0114] The perspective transformation module 84 is used to determine the actual displacement of the effective feature according to the motion trajectory of the effective feature.
[0115] The flow rate determination module 85 is used to determine the flow rate of the water body based on the actual displacement and movement time of the effective features.
[0116] The embodiment of the present application provides a flow velocity determination device based on optical flow estimation and feature filtering. In the device, the image acquisition module is used to acquire continuous image frames of the water body surface; the optical flow estimation module is used to determine the motion duration and motion trajectory of each motion feature on the water body surface based on the optical flow estimation algorithm and the continuous image frames; the interference removal module is used to remove the interference features in the motion features based on the motion duration and motion trajectory to retain the effective features; the perspective transformation module is used to determine the actual displacement of the effective features according to the motion trajectory of the effective features; the flow velocity determination module is used to determine the flow velocity of the water body based on the actual displacement and movement time of the effective features. In the flow velocity determination device, the optical flow estimation is first used to improve the recognition rate of the tracer features, then the interference features are removed, and the feature trajectories with higher credibility are retained. Finally, the flow velocity of the water body is determined based on the effective features, which can effectively improve the accuracy of the determined water body flow velocity.
[0117] It should be clear that the present application is not limited to the specific configurations and processes described in the above embodiments and shown in the figures. For the convenience and brevity of description, a detailed description of the known methods is omitted here, and the specific working processes of the systems, modules and units described above can refer to the corresponding processes in the aforementioned method embodiments, which will not be repeated here.
[0118] Fig. 9 This is a schematic diagram of the structure of an electronic device provided in an embodiment of the present application. Fig. 9 The electronic device includes: a camera, which is used to obtain continuous image frames of the water surface; one or more processors 901; a memory 902, on which one or more programs are stored, and when the one or more programs are executed by one or more processors, the one or more processors implement any one of the above-mentioned methods for determining flow velocity based on optical flow estimation and feature filtering; one or more I / O interfaces 903, connected between the processor and the memory, and configured to implement information interaction between the processor and the memory.
[0119] Among them, the processor 901 is a device with data processing capabilities, including but not limited to a central processing unit (CPU); the memory 902 is a device with data storage capabilities, including but not limited to random access memory (RAM, more specifically SDRAM, DDR, etc.), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory (FLASH); the I / O interface (read-write interface) 903 is connected between the processor 901 and the memory 902, and can realize information interaction between the processor 901 and the memory 902, including but not limited to a data bus (Bus), etc.
[0120] In some embodiments, the processor 901 , the memory 902 , and the I / O interface 903 are connected to each other through a bus, and further connected to other components of the computing device.
[0121] The present embodiment also provides a computer-readable medium having a computer program stored thereon. When the program is executed by a processor, the flow rate determination method for optical flow estimation and feature filtering provided in the present embodiment is implemented. To avoid repeated description, the specific steps of the flow rate determination method for optical flow estimation and feature filtering are not repeated here.
[0122] It will be appreciated by those skilled in the art that all or some of the steps, systems, and functional modules / units in the methods invented above may be implemented as software, firmware, hardware, and appropriate combinations thereof. In hardware implementations, the division between the functional modules / units mentioned in the above description does not necessarily correspond to the division of physical components; for example, a physical component may have multiple functions, or a function or step may be performed by several physical components in cooperation. Some or all physical components may be implemented as software executed by a processor, such as a central processing unit, a digital signal processor, or a microprocessor, or implemented as hardware, or implemented as an integrated circuit, such as an application-specific integrated circuit. Such software may be distributed on a computer-readable medium, which may include a computer storage medium (or non-transitory medium) and a communication medium (or transient medium). As is well known to those skilled in the art, the term computer storage medium includes volatile and non-volatile, removable and non-removable media implemented in any method or technology for storing information (such as computer-readable instructions, data structures, program modules, or other data). Computer storage media include, but are not limited to, RAM, ROM, EEPROM, flash memory or other memory technology, CD-ROM, digital versatile disks (DVD) or other optical disk storage, magnetic cassettes, magnetic tapes, magnetic disk storage or other magnetic storage devices, or any other medium that can be used to store the desired information and can be accessed by a computer. In addition, it is well known to those skilled in the art that communication media typically contain computer-readable instructions, data structures, program modules, or other data in a modulated data signal such as a carrier wave or other transport mechanism, and may include any information delivery media.
[0123] It should be noted that, in this article, the terms "include", "comprises" or any other variations thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, an element defined by the sentence "comprises a ..." does not exclude the existence of other identical elements in the process, method, article or device including the element.
[0124] Those skilled in the art will appreciate that although some embodiments described herein include certain features included in other embodiments but not other features, the combination of features from different embodiments is meant to be within the scope of the present embodiment and to form different embodiments.
[0125] It is to be understood that the above embodiments are merely exemplary embodiments used to illustrate the principles of the present application, but the present application is not limited thereto. For those skilled in the art, various modifications and improvements can be made without departing from the spirit and substance of the present application, and these modifications and improvements are also considered to be within the scope of protection of the present application.
Claims
1. A flow velocity determination method based on optical flow estimation and feature filtering, characterized in that: The method comprises: Acquire continuous image frames of the water surface; Based on the optical flow estimation algorithm and the continuous image frames, determining the motion duration and motion trajectory of each motion feature on the water surface; Based on the motion duration and the motion trajectory, removing interference features in the motion features to retain valid features, including removing motion features whose motion duration is less than a preset duration threshold from all motion features, and, Remove the motion features corresponding to the motion trajectory inconsistent with the water flow direction to retain the valid features; The motion characteristics corresponding to the motion trajectory inconsistent with the flow direction of the water body are removed, including: Determining a flow field direction vector based on vertex coordinates of a flow field region in the image frame; Determining a motion direction vector of the motion feature based on the motion trajectory of the motion feature; Clearing the motion features whose vector inner product between the motion direction vector and the flow field direction vector is less than 0; Determining the actual displacement of the effective feature according to the motion trajectory of the effective feature; Based on the actual displacement and motion time of the effective features, the flow velocity of the water body is determined.
2. The method according to claim 1, characterized in that The step of determining the motion duration and motion trajectory of each motion feature on the water surface based on the optical flow estimation algorithm and the continuous image frames includes: Based on a dense optical flow estimation algorithm and the continuous image frames, obtaining an optical flow estimation image sequence; Based on the optical flow estimation image sequence, the motion duration and motion trajectory of each of the motion features are determined.
3. The method according to claim 2, characterized in that The step of obtaining an optical flow estimation image sequence based on a dense optical flow estimation algorithm and the continuous image frames includes: Determining a plurality of groups of adjacent image frames from the continuous image frames; According to the acquisition order of the previous frame of the adjacent image frames, the adjacent image frames are sequentially input into the dense optical flow estimation algorithm model to obtain the corresponding output optical flow estimation image; the optical flow estimation image includes the motion displacement information corresponding to each position in the previous frame of the image in the next frame of the image; Multiple frames of the optical flow estimation images constitute the optical flow estimation image sequence in an output order.
4. The method according to claim 1, characterized in that Determining the actual displacement of the effective feature according to the motion trajectory of the effective feature includes: Determine the correspondence between the pixel coordinates and the actual coordinates of the flow field area; According to the corresponding relationship, the actual displacement of the effective feature is determined.
5. The method according to claim 1, characterized in that The determining of the flow velocity of the water body based on the actual displacement and movement time of the effective feature comprises: Determining a flow rate of the effective feature based on the actual displacement and movement time of the effective feature; Determine the flow rate interval of the effective characteristic by taking the lowest effective characteristic flow rate and the highest effective characteristic flow rate as boundaries; Dividing the flow rate interval into a preset number of sub-flow rate intervals; Determine the number of valid features contained in each sub-flow rate interval; Determine the average flow velocity of the valid features contained in the sub-flow velocity interval containing the largest number of valid features; The average flow velocity is determined as the flow velocity of the body of water.
6. A flow velocity determination device for optical flow estimation and feature filtering, characterized in that: The device comprises: An image acquisition module, used for acquiring continuous image frames of the water body surface; An optical flow estimation module, used to determine the motion duration and motion trajectory of each motion feature on the water surface based on the optical flow estimation algorithm and the continuous image frames; An interference removal module is used to remove interference features in the motion features based on the motion duration and the motion trajectory to retain valid features, including removing motion features whose motion duration is less than a preset duration threshold from all motion features, and, Remove the motion features corresponding to the motion trajectory inconsistent with the water flow direction to retain the valid features; The motion characteristics corresponding to the motion trajectory inconsistent with the flow direction of the water body are removed, including: Determining a flow field direction vector based on vertex coordinates of a flow field region in the image frame; Determining a motion direction vector of the motion feature based on the motion trajectory of the motion feature; Clearing the motion features whose vector inner product between the motion direction vector and the flow field direction vector is less than 0; A perspective transformation module, used to determine the actual displacement of the effective feature according to the motion trajectory of the effective feature; The flow rate determination module is used to determine the flow rate of the water body based on the actual displacement and movement time of the effective feature.
7. An electronic device, characterized in that: include: A camera, wherein the camera is used to obtain continuous image frames of the surface of the water body; one or more processors; A storage device having one or more programs stored thereon, wherein when the one or more programs are executed by the one or more processors, the one or more processors implement the method according to any one of claims 1 to 5; One or more I / O interfaces are connected between the processor and the memory and are configured to implement information interaction between the processor and the memory.
8. A computer-readable medium having a computer program stored thereon, wherein the computer program, when executed by a processor, implements the method according to any one of claims 1 to 5.
