Method, device and computer equipment for determining flow velocity of flow field

Through preprocessing of continuous video frames of river channels and filtering of optical flow networks, the river surface flow velocity is calculated, which solves the problems of low accuracy and noise interference in complex environments in traditional measurement technology, and achieves high-precision flow velocity measurement.

CN118823072BActive Publication Date: 2024-12-03CHINA INST OF WATER RESOURCES & HYDROPOWER RES
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
CN202411306498.4
Authority / Receiving Office
CN · China
Patent Type
Patents(China)
Current Assignee / Owner
Filing Date
2024-09-19
Publication Date
2024-12-03
Estimated Expiration
2044-09-19

AI Technical Summary

Technical Problem

Traditional river flow rate measurement technology is difficult to achieve long-term real-time measurement in complex environments, and is susceptible to noise interference, so it is impossible to obtain flow rate information on a large scale, affecting flood prevention work.

Method used

By obtaining continuous video frames of the river channel, preprocessing is performed to determine the guidance direction and perspective transformation matrix of the sub-flow field, the initial optical flow map is generated using the optical flow network, and filtering it with the timing filter model to obtain the target optical flow map, and finally according to The perspective transformation matrix calculates the target flow velocity in the subflow field.

Benefits of technology

It improves the accuracy and anti-interference ability of river surface flow velocity prediction in complex environments, realizes accurate measurement of river surface flow velocity, reduces noise interference, and is suitable for on-site measurement in harsh environments.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN118823072B_ABST
    Figure CN118823072B_ABST
Patent Text Reader

Abstract

The present application relates to a method, device and computer equipment for determining the flow velocity of a flow field, including: extracting an initial optical flow map with dense motion characteristics of the water surface through an optical flow network, filtering invalid motion characteristics by guiding the direction, and improving the stability of the generation of time series feature trajectories by high-speed filtering, thereby retaining feature trajectories with high credibility, and finally using perspective transformation to accurately estimate the flow velocity of the flow field in the velocity measurement area.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present application relates to the technical field of non-contact river surface velocity measurement based on machine vision, and in particular to a method, device and computer equipment for determining the velocity of a flow field. Background Art

[0002] Hydrological construction is one of the pillars of national economic construction. Comprehensive hydrological data can be obtained through long-term observation of indicators such as flow, water level, and precipitation. Further research and analysis of hydrological data can provide important decision-making basis for water conservancy project construction, drought and flood disaster prevention, comprehensive basin management, and protection and utilization of water resources. my country has a large land area, and significant regional and climatic differences between the east and west and the north and south, resulting in uneven distribution of water resources in space and seasons. The interannual variation of precipitation in the flood season is large, which is particularly obvious in the south. Therefore, water conservancy projects of different scales are widely distributed, which increases the difficulty of hydrological measurement.

[0003] As an important hydrological information, river velocity is the basis for studying river flow estimation. Traditional velocity measurement relies on various instruments, such as rotor flowmeters, Doppler laser velocimeters, radar velocimeters, etc. These instruments are widely used in river section velocity measurement and cross-sectional flow calculation due to their simple structure, easy operation, stable and accurate velocity measurement. However, most of these instruments require manual regular and fixed-length operation to complete the measurement. Therefore, their measurement operations are mostly carried out during the day, and the measurement activities are limited by the scope of human activities. This means that on-site measurements in harsh environments are extremely difficult, and the personal safety of measurement personnel cannot be guaranteed. In terms of disaster prevention, the flow characteristics of rivers in natural environments are relatively complex, and point measurement technology cannot obtain velocity information on a large scale, which is not conducive to the development of modern flood prevention work.

[0004] Image flow measurement technology has been gradually applied to hydrological monitoring sites at home and abroad due to its advantages such as simplicity, efficiency and safety. Traditional image flow measurement technologies mainly include particle velocimetry technology, large-scale particle image velocimetry, space-time image velocimetry, traditional optical flow velocimetry, etc. Methods based on particle velocimetry technology require artificial scattering of tracers, space-time image velocimetry can only obtain flow velocity information on the one-dimensional space of the velocity measurement line, and traditional optical flow velocimetry requires setting multiple parameters according to different scenarios. These image velocimetry methods have the problems of being unable to achieve long-term real-time measurement, and the system deployment is relatively difficult. The image is easily disturbed by external noise during the processing process.

[0005] With the continuous development of deep learning technology, the image velocity measurement algorithm based on deep optical flow network has higher velocity measurement accuracy and anti-interference ability than the traditional image velocity measurement method, and still has good prediction performance in the complex environment of natural rivers. However, the image noise formed by the ripple disturbance, shadow, reflection and strong light reflection on the river surface still leads to large errors in the prediction accuracy of river surface velocity. Summary of the invention

[0006] Based on this, the purpose of this application is to provide a flow field velocity determination method, device, computer equipment and computer-readable storage medium that improves the surface flow velocity prediction accuracy in complex environments, so as to solve the technical problems mentioned in the above background technology.

[0007] In a first aspect, the present application provides a method for determining flow velocity of a flow field, comprising:

[0008] Acquire continuous video frames of a river channel; the continuous video frames include sub-video frames of a plurality of sub-flow fields;

[0009] Preprocessing the sub-video frame to obtain the guiding direction and perspective transformation matrix of the sub-flow field;

[0010] Generate an initial optical flow map corresponding to the sub-video frame through an optical flow network, and filter the initial optical flow map according to the guiding direction and the temporal filtering model to obtain a target optical flow map;

[0011] The target flow velocity in the sub-flow field is obtained according to the perspective transformation matrix and the target optical flow map.

[0012] In one embodiment, the sub-video frame is preprocessed to obtain the guiding direction and perspective transformation matrix of the sub-flow field, including: determining the calibrated image coordinates and the actual world coordinates of each boundary point in the sub-flow field; determining multiple boundary vectors based on multiple calibrated image coordinates, and determining the guiding direction of the sub-flow field based on the multiple boundary vectors; determining the perspective transformation matrix of the sub-flow field based on the multiple calibrated image coordinates and the multiple actual world coordinates.

[0013] In one embodiment, determining the guiding direction of the sub-flow field based on multiple boundary vectors includes: determining the flow field direction of the sub-flow field through an optical flow network; searching for a target boundary vector having the smallest vector angle with the flow field direction from the multiple boundary vectors, and determining the guiding direction of the sub-flow field based on the target boundary vector and the flow field direction.

[0014] In one embodiment, an initial optical flow map corresponding to the sub-video frame is generated through an optical flow network, including: using a convolutional neural network to extract features of multiple adjacent sub-video frames to obtain multiple feature maps; determining key information of pixel displacement in any two feature maps, and performing similarity association; iterating the feature maps, the key information and the similarity to obtain an initial optical flow map corresponding to the sub-video frame.

[0015] In one embodiment, the initial optical flow map includes multiple feature points; the temporal filtering model includes a Gaussian model; the initial optical flow map is filtered according to the guiding direction and the temporal filtering model to obtain a target optical flow map, including: for each feature point among the multiple feature points, a motion direction vector and a motion distance of the current feature point are determined; a vector product between the motion direction vector and the guiding direction is determined, and when the vector product is greater than a preset value, the current feature point is retained; when the motion distance of the current feature point does not conform to the distribution characteristics of the Gaussian model, the current feature point is filtered.

[0016] In one embodiment, the target flow rate in the sub-flow field is obtained according to the perspective transformation matrix and the target optical flow map, including: based on the perspective transformation matrix, converting the pixel displacement of each feature point in the target optical flow map into an actual movement distance; determining the initial flow rate of each feature point according to the actual movement distance and the movement duration; screening out the maximum flow rate from a plurality of the initial flow rates, and determining the target flow rate in the sub-flow field according to the maximum flow rate.

[0017] In one embodiment, the target flow rate in the sub-flow field is determined according to the maximum flow rate, including: dividing the multiple initial flow rates into intervals based on the maximum flow rate; determining the number of the characteristic points in each interval, and selecting the interval with the largest number of characteristic points as the target interval; determining the target flow rate of the sub-flow field according to the initial flow rates of the multiple characteristic points in the target interval.

[0018] In a second aspect, the present application also provides a flow velocity determination device for a flow field, comprising:

[0019] A preprocessing module is used to obtain continuous video frames of the river; the continuous video frames include sub-video frames of multiple sub-flow fields; the sub-video frames are preprocessed to obtain the guiding direction and perspective transformation matrix of the sub-flow field;

[0020] An optical flow network extraction module, used to generate an initial optical flow map corresponding to a sub-video frame through an optical flow network;

[0021] A feature filtering module, used for filtering the initial optical flow map according to the guiding direction and the temporal filtering model to obtain a target optical flow map;

[0022] A perspective transformation module is used to obtain a target flow velocity in the sub-flow field according to the perspective transformation matrix and the target optical flow map.

[0023] In a third aspect, the present application further provides a computer device, which includes a memory and a processor, wherein the memory stores a computer program, and the processor implements the steps in the above flow field velocity determination method when executing the computer program.

[0024] In a fourth aspect, the present application further provides a computer-readable storage medium. The computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the following steps in the above flow field flow velocity determination method are implemented:

[0025] The above-mentioned flow field velocity determination method, device, computer equipment and readable storage medium obtain continuous video frames of the river channel and pre-process the sub-video frames to obtain the guiding direction and perspective transformation matrix of the sub-flow field, then generate the initial optical flow map corresponding to the sub-video frame through the optical flow network, and filter the initial optical flow map according to the guiding direction and the timing filtering model to obtain the target optical flow map, so that the target flow velocity in the sub-flow field can be obtained according to the perspective transformation matrix and the target optical flow map. Compared with the traditional optical flow network measurement method, which is easily affected by factors such as light changes, shadows, and wave reflections on the water surface, resulting in large surface flow velocity measurement errors, the present application first extracts the initial optical flow map with dense surface feature motion of the water body through the optical flow network, and uses the guiding direction to filter invalid motion features and uses high-speed filtering to improve the stability of the generation of the timing feature trajectory, so as to retain the feature trajectory with higher credibility, and finally uses the perspective transformation to accurately realize the estimation of the flow field velocity in the velocity measurement area. BRIEF DESCRIPTION OF THE DRAWINGS

[0026] Figure 1 A diagram showing an application environment of a method for determining flow velocity of a flow field in one embodiment;

[0027] Figure 2 A schematic flow chart of a method for determining flow velocity of a flow field in one embodiment;

[0028] Figure 3 A scene diagram of the sub-flow field area division in one embodiment;

[0029] Figure 4 Schematic diagram of sub-flow fields in different coordinate systems in one embodiment;

[0030] Figure 5 A schematic diagram of the guiding direction of a sub-flow field in an embodiment;

[0031] Figure 6 A schematic diagram of an abnormal trajectory of a motion direction vector in an embodiment;

[0032] Figure 7 is a schematic flow chart of a method for determining flow velocity of a flow field in another embodiment;

[0033] Figure 8 FIG. 4 is a diagram showing the internal structure of a computer device in one embodiment. DETAILED DESCRIPTION

[0034] In order to make the purpose, technical solution and advantages of the present application more clearly understood, the present application is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application and are not used to limit the present application.

[0035] The flow velocity determination method of the flow field provided in the embodiment of the present application can be applied to Figure 1 In the application environment shown. Among them, the terminal 102 communicates with the server 104 through the network. The data storage system can store the data that the server 104 needs to process. The data storage system can be integrated on the server 104, or it can be placed on the cloud or other network servers. The terminal 102 can be a different image acquisition device. The server 104 is used to obtain continuous video frames of the river channel collected by the terminal 102, and pre-process the sub-video frames to obtain the guiding direction and perspective transformation matrix of the sub-flow field. The server 104 is also used to generate an initial optical flow map corresponding to the sub-video frame through the optical flow network, and filter the initial optical flow map according to the guiding direction and the timing filtering model to obtain a target optical flow map; the server 104 is also used to obtain the target flow rate in the sub-flow field according to the perspective transformation matrix and the target optical flow map. Among them, server 104 can be implemented by an independent server or a server cluster composed of multiple servers, or it can be a cloud server that provides basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communications, middleware services, domain name services, security services, CDN (Content Delivery Network), as well as big data and artificial intelligence platforms.

[0036] In one embodiment, Figure 2 As shown, a method for determining the flow velocity of a flow field is provided, and the method can be integrated in a flow field flow velocity determination system, and the method is applied to Figure 1 The server in the example is used to illustrate, including the following steps:

[0037] Step 202, acquiring continuous video frames of the river.

[0038] The continuous video frames include sub-video frames of multiple sub-flow fields.

[0039] Specifically, first, an image acquisition device installed above or on the side of the river is used to capture continuous video frames at fixed intervals, such as a high-definition camera. After acquiring the continuous video frames sent by the image acquisition device, the server divides the river flow area into N sub-flow areas, each of which is an irregular quadrilateral in the image. Figure 3 As shown, Figure 3 Scene graph divided into sub-flow field areas.

[0040] Step 204 , preprocessing the sub-video frame to obtain the guiding direction and perspective transformation matrix of the sub-flow field.

[0041] Among them, the preprocessing includes flow field direction estimation and flow measurement area coordinate calibration.

[0042] Specifically, the server uses the optical flow network to calculate the approximate flow field direction and determines the calibration image coordinates of each boundary point in the sub-flow field, and then automatically generates the guidance direction of the sub-flow field area by combining the calibration image coordinates and flow field direction of the sub-flow field area. Due to different viewpoints, the sub-flow field area is a rectangle in the real world coordinate system, such as Figure 4 As shown in (a) in the figure, it is an irregular quadrilateral in the image coordinate system, as shown in Figure 4 As shown in (b), the server uses the calibrated image coordinates of each boundary point in the sub-flow field and the corresponding actual world coordinates obtained through on-site field measurements to solve the perspective transformation matrix from the image coordinate system to the world coordinate system.

[0043] Step 206, generating an initial optical flow map corresponding to the sub-video frame through the optical flow network, and filtering the initial optical flow map according to the guiding direction and the temporal filtering model to obtain a target optical flow map.

[0044] The optical flow network includes a visual feature extraction module, a visual association module, and an iterative update module. The optical flow graph uses the changes in the time domain of pixels in the image sequence and the correlation between adjacent frames to find the corresponding relationship between the previous frame and the current frame, thereby calculating a visual representation of the motion information of objects between adjacent frames.

[0045] Specifically, the server uses the images of two adjacent sub-video frames as the input of the optical flow network, and can obtain the initial optical flow map of the sub-flow field area after feature extraction by the visual feature extraction module, similarity association by the visual association module, and data iteration by the iterative update module. Since the value of each point in the feature map obtained by converting the initial optical flow map represents the horizontal and vertical displacement of the motion feature at a certain position in the previous frame of the image in the corresponding next frame of the image, the position of the motion feature on the next frame of the image can be calculated through the relative displacement. In this way, the motion trajectory of the tracked object features in continuous video frames can be calculated and recorded, and the 2D motion between objects can be estimated more efficiently and accurately.

[0046] Since the flow velocity measurement algorithm based on optical flow network estimation is easily affected by factors such as light changes, shadows, and wave reflections on the water surface, it leads to large errors in surface flow velocity measurement. In addition, the optical flow network can only make predictions for two consecutive frames of images with a very small interval, and its prediction results are very unstable. Therefore, the server also needs to perform noise filtering based on the guidance direction and temporal noise filtering based on the temporal filtering model.

[0047] Some interfering feature points in the initial optical flow map will produce motion trajectories that are contrary to the guiding direction of the sub-flow field, thereby increasing the prediction error. The server determines the vector product associated with the feature point and the guiding direction, and uses the similarities and differences between the vector product and the guiding direction when representing the motion trajectory to determine whether to filter the feature point. After the above filtering, there will still be some interfering features, such as reflected light and shadow on the river surface, shadow jitter, etc. These noise characteristics are short motion trajectories. The server uses the temporal filtering model to continue filtering the filtered initial optical flow map to obtain the target optical flow map.

[0048] Step 208: Obtain the target flow velocity in the sub-flow field according to the perspective transformation matrix and the target optical flow map.

[0049] In one embodiment, a target flow rate in a sub-flow field is obtained according to a perspective transformation matrix and a target optical flow map, including: based on the perspective transformation matrix, converting the pixel displacement of each feature point in the target optical flow map into an actual movement distance; determining an initial flow rate of each feature point according to the actual movement distance and the movement duration; screening out a maximum flow rate from multiple initial flow rates, and determining the target flow rate in the sub-flow field according to the maximum flow rate.

[0050] The maximum flow rate includes the maximum initial flow rate and the minimum initial flow rate.

[0051] Specifically, the server converts the pixel displacement in the corresponding sub-flow field area into the actual movement distance in the world coordinate system according to the perspective transformation matrix, and determines the true initial flow velocity of each feature point in the target optical flow map based on the recorded movement duration. The server divides multiple flow velocity intervals based on the maximum initial flow velocity and the minimum initial flow velocity. In order to obtain a relatively stable flow velocity, the server needs to further calculate the average flow velocity of multiple initial flow velocities in different flow velocity intervals until the target flow velocity in the sub-flow field is obtained.

[0052] In the above-mentioned flow field velocity determination method, the initial optical flow map with dense characteristic motion of the water surface is first extracted through the optical flow network, and the invalid motion features are filtered out by guiding the direction and the stability of the generation of the time series feature trajectory is improved by high-speed filtering, so as to retain the characteristic trajectory with high credibility. Therefore, the perspective transformation can be used to accurately estimate the flow field velocity in the velocity measurement area. In addition, this application combines optical flow network, camera calibration, perspective transformation, Gaussian filtering algorithm and other technologies to realize the river surface velocity measurement algorithm in complex environments. Through a large number of experimental verifications, this method can achieve high-precision velocity measurement performance only through high-definition cameras and simple distance calibration, and has good engineering application prospects.

[0053] It is easy to understand that the application scenarios of the above-mentioned flow field velocity determination method include but are not limited to the determination of river flow velocity in river flow fields, and can also be applied to scenarios such as vehicle flow, product raw material production flow, etc. that require feature tracking based on video frames to determine the motion trajectory.

[0054] In one embodiment, the sub-video frames are preprocessed to obtain the guiding direction and perspective transformation matrix of the sub-flow field, including: determining the calibrated image coordinates and the actual world coordinates of each boundary point in the sub-flow field; determining multiple boundary vectors based on the multiple calibrated image coordinates, and determining the guiding direction of the sub-flow field based on the multiple boundary vectors; determining the perspective transformation matrix of the sub-flow field based on the multiple calibrated image coordinates and the multiple actual world coordinates.

[0055] Specifically, the server connects the four boundary points of each sub-flow field area in a clockwise or counterclockwise direction, obtains four boundary vectors αi1, αi2, αi3 and αi4, and the calibrated image coordinates of each boundary point, and measures the actual world coordinates of the corresponding boundary point. Figure 5 As shown, Figure 5 is a schematic diagram of the guiding direction of the sub-flow field. Then the server inputs the sub-video frames corresponding to different sub-flow fields into the optical flow network to obtain the guiding direction Gi of the corresponding sub-flow field. The server uses the calibrated image coordinates of each boundary point in the sub-flow field and the corresponding actual world coordinates obtained through on-site field measurement to solve the perspective transformation matrix of the image coordinate system to the world coordinate system.

[0056] In this embodiment, the guiding direction of the sub-flow field is determined according to multiple boundary vectors, so that the movement direction and speed of the sub-flow field can be better determined to make it more in line with actual needs. The perspective effect of the sub-flow field is adjusted through camera calibration, which enhances the realism and visual effect of the sub-flow field.

[0057] In one embodiment, the guiding direction of the sub-flow field is determined according to multiple boundary vectors, including: determining the flow field direction of the sub-flow field through an optical flow network; searching for a target boundary vector having the smallest vector angle with the flow field direction from multiple boundary vectors, and determining the guiding direction of the sub-flow field according to the target boundary vector and the flow field direction.

[0058] Specifically, the server intercepts a small segment of continuous video frames from the sub-video frames corresponding to different sub-flow fields, and uses the optical flow network to calculate the approximate flow field direction di of the ith sub-flow field area Ri in this segment of continuous video frames, and finds the target boundary vector with the smallest vector angle with the flow field direction di among the boundary vectors αi1, αi2, αi3, and αi4. The server takes the midpoints of the two vectors adjacent to the target boundary vector as the starting point and end point of the guidance direction to be generated, and determines the guidance direction Gi of the sub-flow field with reference to the actual flow direction of the flow field direction di, as shown in the example below: Figure 5 shown.

[0059] In this embodiment, the flow field direction determined by the optical flow network can more accurately capture the motion of different areas in the sub-flow field, thereby improving the precision and accuracy of the sub-flow field. The guidance direction is determined according to the vector angle between the flow field direction and the target boundary vector, so that the sub-flow field can better adapt to environmental changes, thereby enhancing the adaptability and flexibility of the sub-flow field.

[0060] In one embodiment, an initial optical flow map corresponding to a sub-video frame is generated through an optical flow network, including: using a convolutional neural network to extract features of multiple adjacent sub-video frames to obtain multiple feature maps; determining key information of pixel displacement in any two feature maps, and performing similarity association; iterating the feature maps, key information and similarity to obtain an initial optical flow map corresponding to the sub-video frame.

[0061] Specifically, the server extracts features of multiple adjacent sub-video frames through the convolutional neural network in the visual feature extraction module to obtain multiple feature maps. The server determines the key information of pixel displacement in any two feature maps through the visual association module. The visual similarity is calculated as the inner product of all feature map pairs, which is used to calculate the full pairwise correlation of two feature maps, and does not limit the size of the feature map comparison window. The server iterates all data such as feature maps, key information, and similarity through the iterative update module, where the iterative update module is a series of gated recurrent units. The gated recurrent unit simulates the iterative optimization algorithm and has a trainable convolutional layer, which contains shared weights. One update iteration will generate a new optical flow update.

[0062] In this embodiment, by extracting feature information through a convolutional neural network and combining key information for determining pixel displacement and similarity association, the optical flow between video frames can be calculated more accurately, thereby improving the accuracy of optical flow calculation. By iteratively optimizing feature maps, key information, and similarity, the matching accuracy between images can be improved, so that the optical flow map can more accurately reflect the changes and motion between video frames.

[0063] In one embodiment, the initial optical flow map is filtered according to the guiding direction and the timing filtering model to obtain the target optical flow map, including: for each feature point among multiple feature points, determining the motion direction vector and motion distance of the current feature point; determining the vector product between the motion direction vector and the guiding direction, and retaining the current feature point when the vector product is greater than a preset value; filtering the current feature point when the motion distance of the current feature point does not conform to the distribution characteristics of the Gaussian model.

[0064] The initial optical flow map includes multiple feature points; and the temporal filtering model includes a Gaussian model.

[0065] Specifically, for each feature point among the multiple feature points, the server uses the starting and ending points of the motion feature of the current feature point as endpoints to obtain the motion direction vector v of the motion feature. At this time, the motion direction vector v is divided into the corresponding sub-flow field according to the area where the center point coordinates of the motion direction vector v are located. The server calculates the vector product β of the motion direction vector v and the guiding direction Gi in combination with the guiding direction Gi of each sub-flow field area. When the vector product β is less than the preset value, it means that the motion direction of the feature point is significantly different from the guiding direction of the sub-flow field. Therefore, the feature point is very likely to be an interference feature generated by the water surface. In order to reduce the error in flow velocity estimation, it can be further removed. Figure 6 As shown, Figure 6 is a schematic diagram of an abnormal trajectory of the motion direction vector, for example, the gray vector n is an interference trajectory, and v is a normal trajectory. When the vector product is greater than a preset value, the server retains the current feature point. The preset value is, for example, 0.

[0066] After the noise filtering based on the guidance direction, there will still be some interference features in the sub-flow field, such as branches and leaves shaking with the wind, and the shadow area formed on the river surface will also produce pixel changes to form motion features. These noise characteristics are short motion trajectories. The pixel distance from the starting point to the end point of the feature point trajectory of continuous video frames is used as the motion distance of the current feature point. If the motion distance of the interference feature and the water flow feature in the continuous video frame does not obey the distribution characteristics of the one-dimensional Gaussian model, the server will filter the current feature point.

[0067] In this embodiment, by determining the motion direction vector and comparing it with the guiding direction, feature points whose motion is consistent with the guiding direction can be screened out, which helps to reduce the interference of non-moving objects or noise on the feature points and improves the accuracy of the feature points. Considering the relationship between the motion distance and the Gaussian model distribution characteristics, feature points that do not conform to the Gaussian model distribution characteristics, that is, points with abnormal motion trajectories or unclear motion directions, can be filtered out, thereby optimizing the detection effect of feature points and improving the sensitivity to key information. By effectively screening and filtering the motion characteristics and distribution of feature points, the robustness and stability of the algorithm can be improved, making the extraction and tracking process of feature points more reliable and accurate.

[0068] In one embodiment, the probability distribution of the Gaussian model is as follows:

[0069]

[0070] According to the Gaussian distribution characteristics, the server can further filter the motion trajectories corresponding to different feature points. The calculation steps are as follows: count the feature trajectories within a certain time window T and calculate the motion distance of its pixels; calculate the mean motion distance of all feature points ; Calculate the standard deviation of the movement distance of all feature points Therefore, due to the movement distance The feature trajectories outside the range are more likely to be interference trajectories. To reduce the impact of interference features, the trajectories of these feature points can be filtered out.

[0071] In one embodiment, a target flow rate in a sub-flow field is determined based on a maximum flow rate, including: dividing a plurality of initial flow rates into intervals based on the maximum flow rate; determining the number of feature points in each interval, and selecting the interval with the largest number of feature points as the target interval; and determining the target flow rate of the sub-flow field based on the initial flow rates of the plurality of feature points in the target interval.

[0072] Specifically, the server first sorts the actual initial flow velocities of all feature points in the sub-flow field area, obtains the minimum initial flow velocity min and the maximum initial flow velocity max in the sub-flow field flow velocity distribution, and divides M sub-flow velocity intervals with (min, max) as the boundary. Then the server counts the number of feature points in each sub-flow velocity interval. In the high flow velocity field, when the value range of M is (5~10), a relatively stable speed measurement effect can be achieved. The server sorts the number of feature points in each interval, selects the interval with the largest number of feature points as the target interval, and calculates the average value of the initial flow velocity corresponding to each feature point in the target interval, and uses the average value as the target flow velocity of the sub-flow field.

[0073] In this embodiment, by dividing the initial flow velocity into intervals and selecting the interval with the largest number of characteristic points as the target interval, the characteristics and variation patterns of the flow field can be better captured, thereby improving the accuracy and reliability of the flow field analysis.

[0074] In one embodiment, Figure 7 As shown, Figure 7 The flow diagram of the flow field velocity determination method based on the optical flow network and the Gaussian filtering of the time series characteristics in another embodiment is shown. The server obtains continuous video frames of the river channel and measures the actual world coordinates corresponding to the sub-flow field area when dividing the multiple sub-flow field areas of the river channel. The server determines the calibrated image coordinates of each boundary point in the sub-flow field, and estimates the flow field direction according to the calibrated image coordinates to obtain the guiding direction of the sub-flow field, and calibrates the coordinates of the flow measurement area according to the calibrated image coordinates and multiple actual world coordinates to obtain the perspective transformation matrix of the sub-flow field. The server generates an initial optical flow map corresponding to the sub-video frame through the optical flow network, and successively obtains the target optical flow map through noise filtering based on the guiding direction and temporal noise filtering based on the temporal filtering model. The server performs perspective transformation on the target optical flow map according to the perspective transformation matrix until the target flow velocity in the sub-flow field is obtained, and determines the flow velocity of the river flow field when the target flow velocities corresponding to each sub-flow field are integrated.

[0075] It should be understood that, although the steps in the flowcharts involved in the above embodiments are displayed in sequence according to the indication of the arrows, these steps are not necessarily executed in sequence according to the order indicated by the arrows. Unless there is a clear explanation in this article, the execution of these steps is not strictly limited in order, and these steps can be executed in other orders. Moreover, at least a part of the steps in the flowcharts involved in the above embodiments may include multiple steps or multiple stages, and these steps or stages are not necessarily executed at the same time, but can be executed at different times, and the execution order of these steps or stages is not necessarily carried out in sequence, but can be executed in turn or alternately with other steps or at least a part of the steps or stages in other steps.

[0076] Based on the same inventive concept, the embodiment of the present application also provides a flow velocity determination device for a flow field for implementing the flow velocity determination method of the flow field involved above. The implementation scheme for solving the problem provided by the device is similar to the implementation scheme recorded in the above method, so the specific limitations in the embodiments of the flow velocity determination device for one or more flow fields provided below can refer to the limitations of the flow velocity determination method for the flow field above, and will not be repeated here.

[0077] In one embodiment, a flow velocity determination device for a flow field is provided, comprising: a preprocessing module, an optical flow network extraction module, a feature filtering module and a perspective transformation module, wherein:

[0078] The preprocessing module is used to obtain continuous video frames of the river; the continuous video frames include sub-video frames of multiple sub-flow fields; the sub-video frames are preprocessed to obtain the guiding direction and perspective transformation matrix of the sub-flow field.

[0079] The optical flow network extraction module is used to generate an initial optical flow map corresponding to the sub-video frame through the optical flow network.

[0080] The feature filtering module is used to filter the initial optical flow map according to the guiding direction and the temporal filtering model to obtain the target optical flow map.

[0081] The perspective transformation module is used to obtain the target flow velocity in the sub-flow field according to the perspective transformation matrix and the target optical flow map.

[0082] Each module in the flow velocity determination of the flow field can be implemented in whole or in part by software, hardware, or a combination thereof. Each module can be embedded in or independent of a processor in a computer device in the form of hardware, or can be stored in a memory in a computer device in the form of software, so that the processor can call and execute operations corresponding to each module.

[0083] In one embodiment, a computer device is provided. The computer device may be a server, and its internal structure diagram may be as follows: Figure 8As shown. The computer device includes a processor, a memory, an input / output interface (Input / Output, referred to as I / O) and a communication interface. The processor, the memory and the input / output interface are connected through a system bus, and the communication interface is connected to the system bus through the input / output interface. The processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system, a computer program and a database. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The database of the computer device is used to store optical flow maps. The input / output interface of the computer device is used to exchange information between the processor and an external device. The communication interface of the computer device is used to communicate with an external terminal through a network connection. When the computer program is executed by the processor, a method for determining the flow velocity of a flow field is implemented.

[0084] Those skilled in the art will understand that Figure 8 The structure shown in the figure is only a block diagram of a part of the structure related to the solution of the present application, and does not constitute a limitation on the computer device to which the solution of the present application is applied. The specific computer device may include more or fewer components than those shown in the figure, or combine certain components, or have a different arrangement of components.

[0085] In one embodiment, a computer device is further provided, including a memory and a processor, wherein a computer program is stored in the memory, and the processor implements the steps in the above method embodiments when executing the computer program.

[0086] In one embodiment, a computer-readable storage medium is provided, storing a computer program, which implements the steps in the above method embodiments when executed by a processor.

[0087] In one embodiment, a computer program product or computer program is provided, the computer program product or computer program includes computer instructions, the computer instructions are stored in a computer-readable storage medium. A processor of a computer device reads the computer instructions from the computer-readable storage medium, and the processor executes the computer instructions, so that the computer device performs the steps in the above-mentioned method embodiments.

[0088] A person of ordinary skill in the art can understand that all or part of the processes in the above-mentioned embodiment methods can be completed by instructing the relevant hardware through a computer program, and the computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, any reference to the memory, database or other medium used in the embodiments provided in the present application can include at least one of non-volatile and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive memory (ReRAM), magnetic random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory can include random access memory (RAM) or external cache memory, etc. As an illustration and not limitation, RAM can be in various forms, such as static random access memory (SRAM) or dynamic random access memory (DRAM). The database involved in each embodiment provided in this application may include at least one of a relational database and a non-relational database. Non-relational databases may include distributed databases based on blockchains, etc., but are not limited to this. The processor involved in each embodiment provided in this application may be a general-purpose processor, a central processing unit, a graphics processor, a digital signal processor, a programmable logic device, a data processing logic device based on quantum computing, etc., but are not limited to this.

[0089] The technical features of the above embodiments may be combined arbitrarily. To make the description concise, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0090] The above embodiments only express several implementation methods of the present application, and the descriptions thereof are relatively specific and detailed, but they cannot be understood as limiting the scope of the present application. It should be pointed out that, for a person of ordinary skill in the art, several variations and improvements can be made without departing from the concept of the present application, and these all belong to the protection scope of the present application. Therefore, the protection scope of the present application shall be subject to the attached claims.

Claims

1. A method for determining flow velocity of a flow field, characterized in that: The method comprises: Acquire continuous video frames of a river channel; the continuous video frames include sub-video frames of a plurality of sub-flow fields; Preprocessing the sub-video frame to obtain the guiding direction and perspective transformation matrix of the sub-flow field; Generate an initial optical flow map corresponding to the sub-video frame through an optical flow network, and filter the initial optical flow map according to the guiding direction and the temporal filtering model to obtain a target optical flow map; Generating the initial optical flow graph corresponding to the sub-video frame through the optical flow network includes: A convolutional neural network is used to extract features of multiple adjacent sub-video frames to obtain multiple feature maps; Determine the key information of pixel displacement in any two feature maps and perform similarity correlation; Iterating the feature map, the key information and the similarity to obtain an initial optical flow map corresponding to the sub-video frame; the iterative update is based on a series of gated recurrent units, the gated recurrent units simulate an iterative optimization algorithm and have a trainable convolutional layer, which includes shared weights; The initial optical flow map includes a plurality of feature points; the temporal filtering model includes a Gaussian model; and filtering the initial optical flow map according to the guiding direction and the temporal filtering model to obtain a target optical flow map includes: For each feature point among the multiple feature points, a motion direction vector and a motion distance of the current feature point are determined; Determine a vector product between the motion direction vector and the guiding direction, and retain the current feature point when the vector product is greater than a preset value; When the movement distance of the current feature point does not conform to the distribution characteristics of the Gaussian model, filtering the current feature point; The target flow velocity in the sub-flow field is obtained according to the perspective transformation matrix and the target optical flow map.

2. The method according to claim 1, characterized in that The preprocessing of the sub-video frame to obtain the guiding direction and perspective transformation matrix of the sub-flow field includes: Determining the calibrated image coordinates and the actual world coordinates of each boundary point in the sub-flow field; Determining a plurality of boundary vectors according to the plurality of calibration image coordinates, and determining a guiding direction of the sub-flow field according to the plurality of boundary vectors; The perspective transformation matrix of the sub-flow field is determined according to the plurality of calibration image coordinates and the plurality of actual world coordinates.

3. The method according to claim 2, characterized in that The step of determining the guiding direction of the sub-flow field according to the plurality of boundary vectors comprises: Determine the flow field direction of the sub-flow field through an optical flow network; A target boundary vector having the smallest vector angle with the flow field direction is searched from the plurality of boundary vectors, and a guiding direction of the sub-flow field is determined according to the target boundary vector and the flow field direction.

4. The method according to claim 1, characterized in that: The step of obtaining the target flow velocity in the sub-flow field according to the perspective transformation matrix and the target optical flow map comprises: Based on the perspective transformation matrix, converting the pixel displacement of each feature point in the target optical flow map into an actual movement distance; Determine the initial flow velocity of each of the feature points according to the actual movement distance and movement duration; A maximum flow rate is screened out from the multiple initial flow rates, and a target flow rate in the sub-flow field is determined according to the maximum flow rate.

5. The method according to claim 4, characterized in that Determining the target flow rate in the sub-flow field according to the maximum flow rate includes: Dividing the multiple initial flow rates into intervals based on the maximum flow rate as a boundary; Determine the number of the characteristic points in each interval, and select the interval with the largest number of the characteristic points as the target interval; The target flow velocity of the sub-flow field is determined according to the initial flow velocities of a plurality of characteristic points within the target interval.

6. A flow velocity determination device for a flow field, characterized in that: The device comprises: A preprocessing module is used to obtain continuous video frames of the river; the continuous video frames include sub-video frames of multiple sub-flow fields; the sub-video frames are preprocessed to obtain the guiding direction and perspective transformation matrix of the sub-flow field; An optical flow network extraction module, used to generate an initial optical flow map corresponding to a sub-video frame through an optical flow network; Generating the initial optical flow graph corresponding to the sub-video frame through the optical flow network includes: A convolutional neural network is used to extract features of multiple adjacent sub-video frames to obtain multiple feature maps; Determine the key information of pixel displacement in any two feature maps and perform similarity correlation; Iterating the feature map, the key information and the similarity to obtain an initial optical flow map corresponding to the sub-video frame; the iterative update is based on a series of gated recurrent units, the gated recurrent units simulate an iterative optimization algorithm and have a trainable convolutional layer, which includes shared weights; The initial optical flow map includes a plurality of feature points; A feature filtering module, configured to filter the initial optical flow map according to the guide direction and the temporal filtering model to obtain a target optical flow map, including, for each feature point in a plurality of feature points, determining a motion direction vector and a motion distance of the current feature point; the temporal filtering model includes a Gaussian model; Determine a vector product between the motion direction vector and the guiding direction, and retain the current feature point when the vector product is greater than a preset value; When the movement distance of the current feature point does not conform to the distribution characteristics of the Gaussian model, filtering the current feature point; A perspective transformation module is used to obtain a target flow velocity in the sub-flow field according to the perspective transformation matrix and the target optical flow map.

7. A computer device comprising a memory and a processor, wherein the memory stores a computer program, characterized in that: When the processor executes the computer program, the steps of the method according to any one of claims 1 to 5 are implemented.

8. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 5 are implemented.