A channel flow measuring method and device based on bionic eagle eye vision

Abstract

The application provides a channel flow measurement method and device based on bionic eagle eye vision, which comprises the following steps: acquiring a channel flow video, and extracting image frames of the channel flow video at intervals to obtain a target observation image; calculating an optical flow value of the target observation image, and based on the position change information of the optical flow value between image frames and the spatial resolution of the target observation image, calculating a surface flow speed value of the target observation image in a world coordinate system; acquiring a cross-section average flow speed based on a pre-trained surface cross-section flow speed coupling model, and based on the cross-section average flow speed and a cross-section water level value, calculating a channel flow; wherein the surface cross-section flow speed coupling model is obtained by fitting the surface flow speed value in the world coordinate system based on a flow speed distribution formula, and is trained according to a surface flow speed distribution law and a median line flow speed distribution law. The method provided by the application for calculating the channel flow is simple and has high measurement precision.

Description

Technical Field

[0001] This invention relates to the field of channel flow measurement technology, and in particular to a method and apparatus for measuring channel flow based on biomimetic eagle eye vision. Background Technology

[0002] Food security is a fundamental guarantee of national security, and water security is the foundation of food security. Canal flow monitoring is crucial for the quantitative management of water resources. Real-time online monitoring of canal flow allows for real-time understanding of water consumption in the irrigation area, which is highly significant for controlling water use and diversion, and facilitates the overall management of the irrigation area.

[0003] Currently, water measurement methods are mainly divided into five categories: flow velocity meter measurement, standard cross-section measurement, culvert and inverted siphon measurement, weir and flume measurement (such as Parshall flume and measuring sill), and instrumental measurement of water level and flow rate. However, due to the discrepancy between the actual conditions of the structures and theoretical conditions, flow deviations are easily generated. Furthermore, the cost of structural water measurement equipment is high, resulting in high maintenance costs, and there are requirements regarding the flow regime during measurement. Existing non-contact flow measurement methods may have their measurement accuracy affected by factors such as weather, temperature, objects in the river, and water quality, leading to inaccurate data.

[0004] Improving the accuracy of channel traffic measurement is a technical problem that needs to be solved. Summary of the Invention

[0005] This invention provides a method and apparatus for measuring channel flow based on biomimetic eagle eye vision, in order to overcome the deficiencies in the prior art.

[0006] This invention provides a method for measuring channel flow based on biomimetic eagle-eye vision, comprising:

[0007] Acquire channel flow video and extract image frame intervals from the channel flow video to obtain target observation image;

[0008] The optical flow value of the target observation image is calculated. Based on the positional change information of the optical flow value of the target observation image between image frames and the spatial resolution of the target observation image, the surface velocity value of the target observation image in the world coordinate system is calculated. The positional change information is used to indicate the change of pixels in the target observation image in the time domain.

[0009] The cross section average flow rate is obtained based on a pre-trained surface cross section flow rate coupling model, and the channel flow rate is calculated based on the cross section average flow rate and a cross section water level value.

[0010] According to the channel flow rate determination method based on the bionic eagle eye vision provided in the application, the channel flowing video is extracted at an image frame interval to obtain a target observation image, which comprises:

[0011] The channel flowing video is extracted at an image frame interval and image enhancement preprocessed to obtain an initial observation image.

[0012] The initial observation image is subjected to background segmentation processing to obtain the target observation image.

[0013] According to the channel flow rate determination method based on the bionic eagle eye vision provided in the application, the light flow value of the target observation image is calculated, the surface flow rate value of the target observation image in a world coordinate system is calculated based on the position change information of the light flow value of the target observation image between image frames and the spatial resolution of the target observation image, which comprises:

[0014] The light flow value of the target observation image is calculated, and the surface flow rate value of the target observation image in a pixel coordinate system is calculated based on the position change information of the light flow value of the target observation image between image frames.

[0015] The surface flow rate value of the target observation image in the world coordinate system is calculated based on the spatial resolution of the target observation image and the surface flow rate value of the target observation image in the pixel coordinate system.

[0016] According to the channel flow rate determination method based on the bionic eagle eye vision provided in the application, before the cross section average flow rate is obtained based on the pre-trained surface cross section flow rate coupling model, the method further comprises:

[0017] The optimal frame image and the reference line position in the target observation image are determined based on the surface flow rate value in the world coordinate system, and the local channel flow field is obtained based on the optimal frame image and the reference line position, the flow rate distribution formula is obtained by performing flow rate fitting on the local channel flow field.

[0018] The application provides a channel flow measurement method based on bionic eagle eye vision.

[0019] A plurality of target normal points are determined based on the reference line position, and the channel is segmented by taking a plurality of vertical lines where the plurality of target normal points are located as symmetry axes of a segmentation surface.

[0020] The average flow rate of each vertical line is calculated based on the surface section flow rate coupling model, and the average flow rate of each vertical line is calculated by a weighted average method to obtain the section average flow rate.

[0021] The application provides a channel flow measurement method based on bionic eagle eye vision, before calculating the channel flow based on the section average flow rate and the section water level value, the method further comprises:

[0022] A historical water area shoreline video is obtained, and images extracted from the historical water area shoreline video are spliced and integrated to obtain a historical water area shoreline image.

[0023] The bionic eagle eye vision model is trained through the historical water area shoreline image to obtain a water level recognition model; wherein the water level recognition model comprises a backbone network and a neck network, the backbone network is used to extract feature information in the water area shoreline image and provide the feature information to the neck network, and the neck network is used to perform feature fusion on the feature information.

[0024] The application provides a channel flow measurement method based on bionic eagle eye vision, before calculating the channel flow based on the section average flow rate and the section water level value, the method further comprises:

[0025] A water area shoreline video is obtained, and the water area shoreline video is input into the water level recognition model to obtain the section water level value.

[0026] The application further provides a channel flow measurement device based on bionic eagle eye vision, comprising:

[0027] The extraction module is used to obtain a channel flow video, and image frame interval extraction is performed on the channel flow video to obtain a target observation image.

[0028] The first calculation module is used to calculate the optical flow value of the target observation image, and based on the position change information of the target observation image between image frames and the spatial resolution of the target observation image, the surface flow rate value of the target observation image in the world coordinate system is calculated; wherein the position change information is used to indicate the change of a pixel point in the target observation image in the time domain.

[0029] The second calculation module is configured to obtain a cross-section average flow rate based on a pre-trained surface cross-section flow rate coupling model, and calculate a channel flow rate based on the cross-section average flow rate and a cross-section water level value.

[0030] The application further provides an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, and the processor implements the method for determining a channel flow rate based on bionic eagle eye vision when executing the program.

[0031] The application further provides a non-transitory computer readable storage medium, which stores a computer program executable by a processor to implement the method for determining a channel flow rate based on bionic eagle eye vision.

[0032] The application further provides a computer program product, which includes a computer program executable by a processor to implement the method for determining a channel flow rate based on bionic eagle eye vision.

[0033] The application provides a method and device for determining a channel flow rate based on bionic eagle eye vision, which obtains a channel flow video, extracts a target observation image from the channel flow video, calculates a flow value of the target observation image, calculates a surface flow rate value of the target observation image in a world coordinate system based on the position change information of the target observation image between image frames and the spatial resolution of the target observation image, obtains a cross-section average flow rate based on a pre-trained surface cross-section flow rate coupling model, and calculates a channel flow rate based on the cross-section average flow rate and a cross-section water level value. The surface cross-section flow rate coupling model is constructed by fitting the surface flow rate value in the world coordinate system based on a flow rate distribution formula, constructing a surface water flow rate distribution rule and a centerline flow rate distribution rule, and training the surface water flow rate distribution rule and the centerline flow rate distribution rule. The cross-section water level value is calculated based on a pre-obtained water area shoreline image. Therefore, the application can obtain a cross-section average flow rate by collecting a local channel flow video to infer a global area, and calculate a channel flow rate based on the cross-section average flow rate, so that the method is simple and has high measurement accuracy. BRIEF DESCRIPTION OF DRAWINGS

[0034] In order to make the technical solutions in the present application or the prior art clearer, the accompanying drawings needed in the embodiments or the prior art description will be briefly introduced. Obviously, the accompanying drawings in the following description are only some embodiments of the present application, and other accompanying drawings can be obtained by those skilled in the art without any creative effort.

[0035] Figure 1 is a flowchart of the channel flow measurement method based on the bionic eagle eye vision provided by the present application;

[0036] Figure 2 is a schematic diagram of the bionic eagle eye vision function provided by the present application;

[0037] Figure 3 is a schematic diagram of the channel video flow measurement frame image segmentation provided by the present application;

[0038] Figure 4 is a schematic diagram of the channel cross-section selection point normal line and area segmentation provided by the present application;

[0039] Figure 5 is a schematic diagram of the comparison between the channel optical flow value speed and the measured speed provided by the present application;

[0040] Figure 6 is a schematic diagram of the comparison between the cross-section median line flow rate measured value and the calculated value provided by the present application;

[0041] Figure 7 is a schematic diagram of the cross-section normal line flow rate distribution formula provided by the present application;

[0042] Figure 8 is a schematic diagram of the comparison between the optical flow method reference line predicted flow rate and the real position flow rate provided by the present application;

[0043] Figure 9 is a schematic diagram of the comparison between the global reference line flow rate measured by the optical flow method and the real value flow rate provided by the present application;

[0044] Figure 10 is a schematic diagram of the water level recognition model recognizing the water level line provided by the present application;

[0045] Figure 11 is a schematic diagram of the comparison between the calculated value and the measured value of the channel water level based on the bionic eagle eye vision provided by the present application;

[0046] Figure 12 is a complete flowchart of the channel flow measurement method based on the bionic eagle eye vision provided by the present application;

[0047] Figure 13 is a schematic diagram of the structure of the channel flow measurement device based on the bionic eagle eye vision provided by the present application;

[0048] Figure 14 is a structural schematic diagram of an electronic device provided by the present application. DETAILED DESCRIPTION

[0049] To make the objectives, technical solutions and advantages of the present application clearer, the technical solutions of the present application will be described below in conjunction with the accompanying drawings in the present application. Obviously, the described embodiments are only some of the embodiments of the present application, rather than all the embodiments. Based on the embodiments in the present application, all other embodiments obtained by those of ordinary skill in the art without creative work fall within the protection scope of the present application.

[0050] The present application will be described below in conjunction with the accompanying drawings. Figures 1-14 A channel flow measuring method and device based on bionic eagle eye vision are described.

[0051] Figure 1 is a flowchart of the channel flow measuring method based on bionic eagle eye vision provided by the present embodiment, as shown in Figure 1 The channel flow measuring method based on bionic eagle eye vision provided by the present embodiment includes:

[0052] Step 100: acquiring a channel flow video and performing image frame interval extraction on the channel flow video to obtain a target observation image.

[0053] It should be noted that at the same time, the human eye can only pay attention to a small part of the field of view, and the eagle can quickly find multiple targets in the case of divided attention, and use its visual advantage to lock the target at a super long distance and estimate the distance for hunting. Figure 2 is a schematic diagram of the bionic eagle eye vision function provided by the present embodiment, as shown in Figure 2 The present embodiment simulates the lateral foveal structure of the eagle eye to perform video image acquisition, and the acquired channel flow video is a local video of the channel.

[0054] Specifically, step 100 specifically includes:

[0055] Step 110: performing image frame interval extraction and image enhancement preprocessing on the channel flow video to obtain an initial observation image.

[0056] Step 120: performing background segmentation processing on the initial observation image to obtain the target observation image.

[0057] In one embodiment, a monocular camera is used to locally capture the channel surface flow, and the captured channel local video is preprocessed to generate a data set. The channel local video obtained is subjected to frame image extraction, and the extracted image is subjected to image enhancement processing such as denoising, and the channel flow in the image is subjected to background segmentation from the background. Figure 3 is a frame image segmentation diagram of the channel video flow measurement provided by the embodiment, as Figure 3 shown, each frame image is segmented into channel flow and background by dynamic threshold. Because the water flow pixels in the channel are different from the background pixels, by setting different feature thresholds, the image pixels are divided into several categories, and a set of threshold values related to the pixel position (i.e. the threshold value is a function of the coordinate) is used to segment each part of the image.

[0058] Step 200, the optical flow value of the target observation image is calculated, and based on the position change information of the optical flow value of the target observation image between image frames and the spatial resolution of the target observation image, the surface flow velocity value of the target observation image in the world coordinate system is calculated; wherein the position change information is used to indicate the change of the pixel points in the target observation image in the time domain.

[0059] Specifically, step 200 specifically includes:

[0060] Step 210, the optical flow value of the target observation image is calculated, and based on the position change information of the optical flow value of the target observation image between image frames, the surface flow velocity value of the target observation image in the pixel coordinate system is calculated;

[0061] Step 220, based on the spatial resolution of the target observation image and the surface flow velocity value of the target observation image in the pixel coordinate system, the surface flow velocity value of the target observation image in the world coordinate system is calculated.

[0062] In one embodiment, the optical flow method is used to measure the channel surface flow, and the optical flow of all points in the target observation image is detected, and the flow velocity is measured by the movement of the optical flow points between the surface frame image and the second frame image. First, a Gaussian pyramid is established for each frame of the image, the lowest resolution image is at the top layer, and the original image is at the bottom layer. The optical flow on the top layer of the pyramid is calculated from the top layer. Then, the initial value of the next top layer optical flow is estimated according to the calculation result of the top layer (Lm-1) optical flow, and the accurate value of the optical flow on the next top layer image is calculated. Finally, the initial value of the next layer (Lm-2) optical flow is estimated according to the calculation result of the next top layer optical flow, and the accurate value is calculated and then fed back to the next layer until the optical flow of the original image at the bottom layer is calculated.

[0063] Wherein, the optical flow method is used to calculate the optical flow value of the target observation image, the flow velocity is calculated according to the movement of the optical flow value of the flow measurement image between the image frames, and the flow velocity value in the pixel coordinate system is u. After that, the pixel coordinate conversion is carried out, first, the collected data set is converted into the hsv format to calculate the flow velocity, the flow velocity measured by the optical flow method is converted through the angle and the pixel, then the real flow velocity of the test point of the channel in the world coordinate is selected, the flow velocity measured by the optical flow method is calculated with the flow velocity in the world coordinate to obtain the space conversion rate of the image, and the space conversion rate of the image is used to convert the channel surface flow velocity value in the pixel coordinate system into the channel surface flow velocity value in the world coordinate system, and the hsv format is converted into the bgr format to display the surface flow velocity value.

[0064] Step 300, obtaining the cross-section average flow velocity based on the pre-trained surface cross-section flow velocity coupling model, and calculating the channel flow based on the cross-section average flow velocity and the cross-section water level value; wherein, the surface cross-section flow velocity coupling model is: based on the flow velocity distribution formula, fitting the surface flow velocity value in the world coordinate system, constructing the surface flow velocity distribution law and the midline flow velocity distribution law, and training to obtain the surface flow velocity distribution law and the midline flow velocity distribution law; the cross-section water level value is calculated based on the pre-acquired water area shoreline image.

[0065] It should be noted that before step 300, the method further comprises:

[0066] Determine the optimal frame image and reference line position in the target observation image based on the surface flow velocity value.

[0067] Based on the optimal frame image and the reference line position, a local channel flow field is obtained, and the flow velocity fitting is performed on the local channel flow field to obtain the flow velocity distribution formula.

[0068] Specifically, the optimal frame image and the reference line position are extracted from all the surface flow velocity values in the world coordinate system by the formula, and the calculation formula is as follows:

[0069] y=ax+b

[0070] Wherein, b=optical flow point start array-a*real position start one bit array, x is the real point position, and y is the predicted point position.

[0071] It should be noted that due to different channel widths and limited camera acquisition range, it is difficult to obtain a global flow field, therefore, the optimal frame image and the reference line position obtained above are output to obtain a local video flow field, the flow velocity fitting is performed to obtain a flow velocity distribution model, and the global flow field is calculated according to the symmetry of the channel.

[0072] For wide and shallow channel, the vertical line flow velocity distribution formula and the surface flow velocity fitting formula are the same, and the flow velocity distribution formula is as follows:

[0073] u=Alny+B

[0074] Wherein, A is Au * , B is B is the correlation coefficient; u * is the local friction velocity; v is the kinematic viscosity of water; y is the distance of each point on the vertical line of the channel from the side wall; K is the Karman coefficient.

[0075] According to a large number of experimental results fitting, the values of the correlation coefficients A and B can be determined.

[0076] Specifically, the step 300 of obtaining the cross-section average flow velocity based on the pre-trained surface cross-section flow velocity coupling model comprises the steps that: in step 310, a plurality of target normal points are determined based on the reference line position, and the channel is segmented by taking a plurality of vertical lines where the plurality of target normal points are located as the symmetry axes of the segmentation surface;

[0077] In step 320, the average flow velocity of each vertical line is calculated based on the surface cross-section flow velocity coupling model, and the average flow velocity of each vertical line is calculated by a weighted average method to obtain the cross-section average flow velocity.

[0078] It should be noted that, since the cross-section average flow velocity of the vertical line in the channel is equal to the average flow velocity of the center point to the side wall, the channel flow velocity distribution formula obtained by the reference line position is the same as the flow velocity distribution formula of the vertical line position in the cross-section, and therefore the surface cross-section flow velocity coupling model is established. Then, the normal points are selected according to the reference line position. The channel to be measured is a symmetrical channel, and the normal points are selected by calculating the global flow velocity by using the above flow velocity distribution formula, Figure 4 is a schematic diagram of the channel cross-section selection point normal and area segmentation provided by the embodiment, and five normal point positions are selected Figure 4 The flow velocity of the point is output.

[0079] Figure 5 is a schematic diagram of the channel optical flow value velocity and the measured velocity provided by the embodiment, Figure 6 is a schematic diagram of the cross-section vertical line flow velocity measured value and the calculated value provided by the embodiment, Figure 7 is a schematic diagram of the cross-section normal flow velocity distribution formula provided by the embodiment, Figure 8 is a schematic diagram of the optical flow method reference line predicted flow velocity and the real position flow velocity provided by the embodiment, Figure 9 is a schematic diagram of the global reference line flow velocity measured by the optical flow method and the real value flow velocity provided by the embodiment; and Figures 5-9This embodiment further explains the acquisition of the surface flow field velocity and the cross-sectional average velocity based on the optical flow method. Since the vertical lines of the channel cross-section may have the same slope but different intercepts, the velocity distribution formula on the selected normal point can be calculated based on the established surface cross-sectional velocity coupling model. Then, the channel cross-sectional area is divided using the selected normal point, and the channel flow information is obtained based on the velocity-area method. Specifically, the channel is divided into five zones using the vertical line containing the selected normal point as the axis of symmetry of the dividing surface, and the flow rate is calculated to obtain the channel flow information.

[0080] Furthermore, the weighted average method is used to calculate the cross-sectional average velocity, and the calculation formula is as follows:

[0081]

[0082] Where u is the average flow velocity of each vertical line, and n is the weight of the selected vertical line.

[0083] It should be noted that, before calculating the channel flow rate in step 300 based on the average flow velocity and water level at the cross-section, the method further includes:

[0084] Historical waterway shoreline videos are acquired, and the images extracted from the historical waterway shoreline videos are stitched together to obtain historical waterway shoreline images;

[0085] A biomimetic eagle-eye vision model is trained using the historical waterway shoreline images to obtain a water level recognition model. The water level recognition model includes a backbone network and a neck network. The backbone network is used to extract feature information from the waterway shoreline images and provide the feature information to the neck network. The neck network is used to perform feature fusion on the feature information.

[0086] In the specific implementation process, a video of the water area shoreline is acquired and input into the water level recognition model to obtain the cross-sectional water level value.

[0087] In one embodiment, the left front side field of view image and the right front side field of view image of the hawk eye are formed by using a binocular camera to perform image stitching integration, the central foveal structure of the hawk eye is simulated, and 2500 image data sets are obtained based thereon, wherein 2000 are randomly selected as training set images, and 500 are randomly selected as test set and verification set images. The images in the training set are converted into Pascal VOC format. The length of the training set images is adjusted to 500 pixels, and the width is adjusted accordingly to maintain the original aspect ratio when creating the training set. After numbering the images, manual labeling is performed using the image labeling tool Labelme software. The boundary box is drawn, and the categories are manually classified. Positive samples with insufficient or unclear pixel regions are not labeled to prevent overfitting in deep learning. According to the bionic hawk eye vision model, a YOLO water level recognition model is built, mainly including a Backbone backbone network and a Neck neck network. The Backbone backbone network is mainly used to extract information in the picture and provide it to the Neck neck network, which is composed of a Conv module, a C2f module and a SPPF module. The Neck neck network mainly performs feature fusion and processes the features extracted from the backbone network. The water area shoreline image dataset is input for training and verification, and a higher precision water area shoreline weight file is obtained by increasing the number of model training iterations. According to the PCN parallax principle, the distance c of the binocular camera from the water area shoreline is obtained. Figure 10 is a schematic diagram of the water level recognition model provided by the embodiment for recognizing the water level line, referring to Figure 10 , according to the cosine theorem, the vertical distance a of the binocular camera from the water area shoreline is c x cos a, and the water level value h of the channel section is m-a. Wherein a is the installation angle of the binocular camera, and m is the distance of the binocular camera from the channel bottom.

[0088] Further, the channel flow is calculated by the following formula:

[0089]

[0090] wherein, is the average flow velocity of each point on the first normal (i.e. normal 1 in is the average flow velocity of each point on the second normal (i.e. normal 2 in Figure 4 ); is the average flow velocity of each point on the third normal (i.e. normal 3 in Figure 4 ); is the average flow velocity of each point on the third normal (i.e. normal 4 in Figure 4 ); is the average flow velocity of each point on the third normal (i.e. normal 4 in Figure 4 ); is the average flow velocity of each point on the third normal (i.e. normal 4 in Figure 4The average flow velocity of each point on the normal line 5) in the channel; A1, A2, A3, A4, A5 are the areas of the plurality of partition polygons respectively.

[0091] Figure 11 is the channel water level calculation value and measured value comparison schematic diagram based on bionic eagle eye vision provided by the embodiment, see Figure 11 It can be known that, based on the channel flow measurement method based on bionic eagle eye vision provided by the embodiment, the measurement accuracy is higher.

[0092] Figure 12 is the complete flow chart of the channel flow measurement method based on bionic eagle eye vision provided by the embodiment, as shown in Figure 12 The complete flow of the channel flow measurement method based on bionic eagle eye vision provided by the embodiment is exemplarily illustrated.

[0093] According to the characteristic point flow velocity distribution law analysis, the logarithmic formula is selected to fit the trapezoidal channel flow velocity distribution formula, the flow velocity distribution formula suitable for the flow velocity distribution law from the center point to the wall is initially established, and the correlation coefficient is determined through the test data.

[0094] The embodiment of the application also provides an intelligent flow measurement model device, which comprises a solar panel, a monocular camera, an electric pole, an equipment mounting box, a display screen and a cable.

[0095] The specific implementation process comprises: firstly, collecting wide and shallow channel flow velocity video; then inputting the video into an optical flow model to obtain the flow velocity under a reference line; simultaneously, measuring the flow velocity under the reference line by using a flowmeter; and comparing the values measured by the optical flow model and the flowmeter to obtain a correlation coefficient of 95%, as shown in Figure 5 Then, the flow velocity distribution formula is obtained by fitting the collected surface flow velocity as follows:

[0096] u=0.4058ln(y)+3.7983

[0097] The flow velocity of 7 points on the median line (0.4, 0.37, 0.34, 0.31, 0.28, 0.25, 0.21) is measured by using a flowmeter; the measured data is fitted to obtain an expression which is the same as the above flow velocity distribution formula; a surface section flow velocity coupling model is established, and the fitting result is as shown in Figure 6

[0098] ​At the same time, the global flow velocity of the reference line is calculated by using the flow velocity distribution formula obtained by the local flow velocity, and the relative error calculation can obtain:

[0099]

[0100] Wherein, the real measured flow velocity refers to the flow velocity displayed by the flow meter in the test, and the optical flow predicted flow velocity is the result calculated by using the flow velocity distribution formula fitted by the optical flow method. Referring back to Figure 8 , the real measured data of the reference line and the data calculated by using the distribution formula are close, and the relative error is within 10%, indicating that the channel flow velocity distribution formula can accurately describe the global real flow velocity distribution.

[0101] In the specific implementation process, in order to verify the law between the section normal lines, 8 section normal lines are selected. The theoretical value (u) of the normal flow velocity representative point is taken as the longitudinal coordinate, and the distance value (lnx) of the representative point to the channel bottom is taken as the transverse coordinate. Referring to Figure 7 , the slopes of the flow velocity distribution formulas of the normal lines of the section are the same, and the intercepts are different.

[0102] The above is the step description of the channel flow measurement method based on bionic eagle eye vision provided in the embodiment. As can be seen from the description of the above steps, according to the channel flow measurement method based on bionic eagle eye vision provided in the embodiment, the channel flow video is acquired, and the target observation image is obtained by image frame interval extraction on the channel flow video; the optical flow value of the target observation image is calculated, and the surface flow velocity value of the target observation image in the world coordinate system is calculated based on the position change information between the image frames of the optical flow value of the target observation image and the spatial resolution of the target observation image; wherein, the position change information is used to indicate the change of the pixel points in the target observation image in the time domain; the average flow velocity of the section is obtained based on the pre-trained surface section flow velocity coupling model, and the channel flow is calculated based on the average flow velocity of the section and the section water level value; wherein, the surface section flow velocity coupling model is: fitting the surface flow velocity value in the world coordinate system based on the flow velocity distribution formula, constructing the surface flow velocity distribution law and the median line flow velocity distribution law, and training to obtain the surface flow velocity distribution law and the median line flow velocity distribution law; the section water level value is calculated based on the pre-acquired water area coastline image. Therefore, the local channel flow video is collected to infer the global region to obtain the average flow velocity of the section, and the channel flow is calculated based on the average flow velocity of the section, which is simple and has high measurement accuracy.

[0103] The channel flow measurement device based on bionic eagle eye vision provided in the present application is described below. The channel flow measurement device based on bionic eagle eye vision described below can be correspondingly referred to each other with the channel flow measurement method based on bionic eagle eye vision described above.

[0104] Figure 13 This is a schematic diagram of the channel flow measurement device based on biomimetic eagle eye vision provided in this embodiment, as shown below. Figure 13 As shown, the channel flow measurement device based on biomimetic eagle eye vision provided in this embodiment includes:

[0105] The extraction module 1301 is used to acquire channel flow video and extract image frame intervals from the channel flow video to obtain target observation images;

[0106] The first calculation module 1302 is used to calculate the optical flow value of the target observation image, and to calculate the surface velocity value of the target observation image in the world coordinate system based on the positional change information of the optical flow value of the target observation image between image frames and the spatial resolution of the target observation image; wherein, the positional change information is used to indicate the change of pixels in the target observation image in the time domain.

[0107] The second calculation module 1303 is used to obtain the average flow velocity of the cross section based on a pre-trained surface cross-sectional velocity coupling model, and to calculate the channel flow rate based on the average flow velocity and the cross-sectional water level. The surface cross-sectional velocity coupling model is constructed by fitting the surface velocity value in the world coordinate system based on a velocity distribution formula, thereby establishing a surface flow velocity distribution law and a perpendicular bisector velocity distribution law, and training the model based on these laws. The cross-sectional water level value is calculated based on a pre-acquired image of the water area's shoreline.

[0108] The channel flow measurement device based on biomimetic eagle eye vision provided in this embodiment acquires channel flow video and extracts the target observation image by image frame interval extraction; calculates the optical flow value of the target observation image; and calculates the surface velocity value of the target observation image in the world coordinate system based on the positional change information of the optical flow value between image frames and the spatial resolution of the target observation image. The positional change information indicates the change of pixels in the target observation image in the time domain. The average cross-sectional velocity is obtained based on a pre-trained surface cross-sectional velocity coupling model, and the channel flow is calculated based on the average cross-sectional velocity and the cross-sectional water level. The surface cross-sectional velocity coupling model is constructed by fitting the surface velocity value in the world coordinate system based on the velocity distribution formula, establishing the surface water flow velocity distribution law and the vertical velocity distribution law, and training the model based on these laws. The cross-sectional water level value is calculated based on a pre-acquired image of the water area shoreline. Therefore, this invention infers the average flow velocity of a cross section by collecting local channel flow video, and calculates the channel flow rate based on the average flow velocity of the cross section. The method is simple and has high measurement accuracy.

[0109] Based on the above embodiment, in this embodiment, the extraction module 1301 is specifically used for:

[0110] performing image frame interval extraction and image enhancement preprocessing on the channel flow video to obtain an initial observation image;

[0111] performing background segmentation processing on the initial observation image to obtain the target observation image.

[0112] Based on the above embodiment, in this embodiment, the first calculation module 1302 is specifically used for:

[0113] calculating the optical flow value of the target observation image, and calculating the surface flow velocity value of the target observation image in a pixel coordinate system based on the position change information of the optical flow value of the target observation image between image frames;

[0114] calculating the surface flow velocity value of the target observation image in a world coordinate system based on the spatial resolution of the target observation image and the surface flow velocity value of the target observation image in the pixel coordinate system.

[0115] Based on the above embodiment, in this embodiment, the device further comprises a fitting module, which is specifically used for:

[0116] determining an optimal frame image and a reference line position in the target observation image based on the surface flow velocity value in the world coordinate system before obtaining the cross-section average flow velocity based on the pre-trained surface cross-section flow velocity coupling model;

[0117] obtaining a local channel flow field based on the optimal frame image and the reference line position, performing flow velocity fitting on the local channel flow field, and obtaining a flow velocity distribution formula.

[0118] Based on the above embodiment, in this embodiment, the second calculation module 1303 is specifically used for:

[0119] determining a plurality of target normal points based on the reference line position, and segmenting the channel by taking a plurality of perpendicular lines where the plurality of target normal points are located as symmetry axes of a segmentation surface;

[0120] calculating the average flow velocity of each perpendicular line based on the surface cross-section flow velocity coupling model, and obtaining the cross-section average flow velocity by weighted average method.

[0121] Based on the above embodiment, in this embodiment, the device further comprises a training module, which is specifically used for:

[0122] Before calculating the channel flow based on the cross-section average flow rate and the cross-section water level value, a historical water area shoreline video is acquired, and images extracted from the historical water area shoreline video are spliced and integrated to obtain a historical water area shoreline image;

[0123] The historical water area shoreline image is used to train a biomimetic hawk-eye vision model to obtain a water level recognition model; the water level recognition model includes a backbone network and a neck network, the backbone network is used to extract feature information in the water area shoreline image and provide the feature information to the neck network, and the neck network is used to perform feature fusion on the feature information.

[0124] Based on the above embodiment, in this embodiment, the device further includes an acquisition module, specifically configured to:

[0125] Before calculating the channel flow based on the cross-section average flow rate and the cross-section water level value, a water area shoreline video is acquired, and the water area shoreline video is input into the water level recognition model to obtain the cross-section water level value.

[0126] Figure 14 An example of an entity structure diagram of an electronic device is shown in Figure 14 As shown, the electronic device can include a processor 1410, a communications interface 1420, a memory 1430, and a communications bus 1440, wherein the processor 1410, the communications interface 1420, and the memory 1430 complete mutual communication through the communications bus 1440. The processor 1410 can invoke a logical instruction in the memory 1430 to execute a channel flow determination method based on biomimetic hawk-eye vision, which includes:

[0127] A channel flow video is acquired, and image frame interval extraction is performed on the channel flow video to obtain a target observation image;

[0128] A light flow value of the target observation image is calculated, and based on position change information of the light flow value of the target observation image between image frames and a spatial resolution of the target observation image, a surface flow rate value of the target observation image in a world coordinate system is calculated; the position change information is used to indicate a change in a time domain of a pixel point in the target observation image;

[0129] The cross section average flow rate is obtained based on a pre-trained surface cross section flow rate coupling model, and the channel flow rate is calculated based on the cross section average flow rate and a cross section water level value; wherein the surface cross section flow rate coupling model is: the surface flow rate value in the world coordinate system is fitted based on a flow rate distribution formula, a surface water flow rate distribution law and a median line flow rate distribution law are constructed, and the surface water flow rate distribution law and the median line flow rate distribution law are trained to obtain; the cross section water level value is calculated based on a pre-obtained water area shoreline image.

[0130] In addition, the logic instructions in the memory 1430 described above can be implemented in the form of a software function unit and sold or used as an independent product, which can be stored in a computer readable storage medium. Based on such understanding, the technical solutions of the present application essentially or the parts that contribute to the prior art or parts of the technical solutions can be embodied in the form of a software product, which is stored in a storage medium, including a plurality of instructions to make a computer device (which can be a personal computer, a server, or a network device, etc.) execute all or part of the steps of the method described in various embodiments of the present application. The foregoing storage medium includes: a U disk, a mobile hard disk, a read-only memory (ROM, Read-Only Memory), a random access memory (RAM, Random Access Memory), a magnetic disk or an optical disk, and various media that can store program codes.

[0131] On the other hand, the present application also provides a computer program product, which comprises a computer program, the computer program can be stored on a non-transitory computer readable storage medium, and the computer program is executed by a processor, so that the computer can execute the channel flow rate determination method based on the bionic eagle eye vision provided by the above-mentioned method, which comprises: acquiring a channel flow video, and extracting image frame intervals of the channel flow video to obtain a target observation image;

[0132] The optical flow value of the target observation image is calculated, and the surface flow rate value of the target observation image in the world coordinate system is calculated based on the position change information between image frames of the optical flow value of the target observation image and the spatial resolution of the target observation image; wherein the position change information is used to indicate the change of the pixel points in the target observation image in the time domain;

[0133] The cross section average flow rate is obtained based on a pre-trained surface cross section flow rate coupling model, and the channel flow rate is calculated based on the cross section average flow rate and a cross section water level value.

[0134] In another aspect, the application further provides a non-transitory computer readable storage medium having a computer program stored thereon, which, when executed by a processor, implements a channel flow rate determination method based on bionic eagle eye vision, and the method comprises:

[0135] A channel flow video is obtained, and image frame interval extraction is performed on the channel flow video to obtain a target observation image.

[0136] A light flow value of the target observation image is calculated, and a surface flow rate value of the target observation image in a world coordinate system is calculated based on position change information between image frames of the light flow value of the target observation image and a spatial resolution of the target observation image, wherein the position change information is used to indicate a change in a time domain of a pixel point in the target observation image.

[0137] The cross section average flow rate is obtained based on a pre-trained surface cross section flow rate coupling model, and the channel flow rate is calculated based on the cross section average flow rate and a cross section water level value.

[0138] The device embodiments described above are only schematic, wherein the units shown as separated components can or can not be physically separated, and the components shown as units can or can not be physical units, i.e., can be located in one place, or can be distributed on multiple network units. Part or all of the modules can be selected according to actual needs to achieve the purpose of the present embodiment scheme. Those skilled in the art can understand and implement without creative labor.

[0139] Those skilled in the art can clearly understand the technical solutions of the various embodiments from the above description of the embodiments, and the various embodiments can be implemented by means of software with the necessary general hardware platforms, and of course, can also be implemented by hardware. Based on such understanding, the above technical solutions, essentially or in other words, the part of the prior art that makes a contribution, can be embodied in the form of a software product, which can be stored in a computer readable storage medium, such as a ROM / RAM, a magnetic disk, an optical disk, and the like, and includes a number of instructions for causing a computer device (which can be a personal computer, a server, or a network device, etc.) to execute the methods described in the various embodiments or some parts of the embodiments.

[0140] Finally, it should be noted that: the above embodiments are only used to illustrate the technical solutions of the present application, rather than limit them; although the present application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that: it can still modify the technical solutions recorded in the foregoing embodiments, or make equivalent replacement for some technical features therein; and these modifications or replacements do not make the essence of the corresponding technical solutions deviate from the spirit and scope of the technical solutions of the embodiments of the present application.

Claims

1. A method for measuring channel flow based on bionic eagle eye vision, characterized in that, The method comprises the following steps: acquire a channel flow video, and perform image frame interval extraction on the channel flow video to obtain a target observation image; wherein the bionic eagle eye vision acquires the channel flow video by simulating the lateral fovea structure of an eagle eye through a binocular camera; calculate the optical flow value of the target observation image, and based on the position change information between image frames of the optical flow value of the target observation image and the spatial resolution of the target observation image, calculate the surface flow velocity value of the target observation image in a world coordinate system; wherein the position change information is used to indicate the change of a pixel point in the target observation image in the time domain; acquire a cross-section average flow velocity based on a pre-trained surface cross-section flow velocity coupling model, and calculate the channel flow based on the cross-section average flow velocity and a cross-section water level value; wherein the surface cross-section flow velocity coupling model is obtained by fitting the surface flow velocity value in the world coordinate system based on a flow velocity distribution formula, constructing a surface flow velocity distribution law and a median line flow velocity distribution law, and training according to the surface flow velocity distribution law and the median line flow velocity distribution law; the cross-section water level value is calculated by a water level recognition model based on a pre-acquired water area shoreline image, the water level recognition model is trained based on a bionic eagle eye vision model and comprises a backbone network and a neck network, the backbone network is used to extract feature information in the water area shoreline image and provide the feature information to the neck network, and the neck network is used to perform feature fusion on the feature information; before acquiring the cross-section average flow velocity based on the pre-trained surface cross-section flow velocity coupling model, the method further comprises: determining an optimal frame image and a reference line position in the target observation image based on the surface flow velocity value in the world coordinate system; obtaining a local channel flow field based on the optimal frame image and the reference line position, performing flow velocity fitting on the local channel flow field, and obtaining a flow velocity distribution formula.

2. The channel flow measurement method based on biomimic hawk eye vision according to claim 1, characterized in that, The method of performing image frame interval extraction on the channel flow video to obtain a target observation image comprises the following steps: perform image frame interval extraction and image enhancement preprocessing on the channel flow video to obtain an initial observation image; perform background segmentation processing on the initial observation image to obtain the target observation image.

3. The channel flow measurement method based on biomimic hawk eye vision according to claim 1, characterized in that, The method of calculating the optical flow value of the target observation image, and based on the position change information between image frames of the optical flow value of the target observation image and the spatial resolution of the target observation image, calculating the surface flow velocity value of the target observation image in a world coordinate system comprises the following steps: calculate the surface flow velocity value of the target observation image in a pixel coordinate system based on the position change information between image frames of the optical flow value of the target observation image; based on the spatial resolution of the target observation image and the surface flow velocity value of the target observation image in the pixel coordinate system, calculate the surface flow velocity value of the target observation image in the world coordinate system.

4. The channel flow measurement method based on biomimic hawk eye vision according to claim 1, characterized in that, The method of acquiring a cross-section average flow velocity based on a pre-trained surface cross-section flow velocity coupling model comprises the following steps: Determine a plurality of target normal points based on the reference line position, and divide the channel based on a plurality of perpendicular lines where the plurality of target normal points are located as symmetry axes of a segmentation surface; Calculate the average flow velocity of each perpendicular line based on the surface section flow velocity coupling model, and obtain the section average flow velocity by calculating the average flow velocity of each perpendicular line by a weighted average method.

5. The channel flow measurement method based on biomimic hawk eye vision according to claim 1, characterized in that, Before calculating the channel flow based on the section average flow velocity and the section water level value, the method further comprises: Obtain a historical water area shoreline video, and integrate the images extracted from the historical water area shoreline video to obtain a historical water area shoreline image; Train the biomimetic hawk eye visual model based on the historical water area shoreline image to obtain a water level recognition model; wherein the water level recognition model comprises a backbone network and a neck network, the backbone network is used to extract feature information in the water area shoreline image and provide the feature information to the neck network, and the neck network is used to perform feature fusion on the feature information.

6. The channel flow measurement method based on biomimic hawk eye vision of claim 5, wherein, Before calculating the channel flow based on the section average flow velocity and the section water level value, the method further comprises: Obtain a water area shoreline video, and input the water area shoreline video into the water level recognition model to obtain the section water level value.

7. A channel flow measuring device based on biomimetic eagle eye vision, characterized in that, Comprise: The extraction module is used to obtain a channel flow video, and extract the channel flow video at an image frame interval to obtain a target observation image; wherein the biomimetic hawk eye vision simulates the lateral fovea structure of the hawk eye to collect the channel flow video through a binocular camera; The first calculation module is used to calculate the optical flow value of the target observation image, and calculate the surface flow value of the target observation image in the world coordinate system based on the position change information between image frames of the optical flow value of the target observation image and the spatial resolution of the target observation image; wherein the position change information is used to indicate the change of the pixel points in the target observation image in the time domain; The second calculation module is used to obtain the section average flow velocity based on the pre-trained surface section flow velocity coupling model, and calculate the channel flow based on the section average flow velocity and the section water level value; wherein the surface section flow velocity coupling model is obtained by fitting the surface flow value in the world coordinate system based on a flow velocity distribution formula, constructing a surface flow velocity distribution law and a perpendicular line flow velocity distribution law, and training based on the surface flow velocity distribution law and the perpendicular line flow velocity distribution law; the section water level value is calculated based on the pre-obtained water area shoreline image by the water level recognition model, the water level recognition model is trained based on the biomimetic hawk eye visual model, and comprises a backbone network and a neck network, the backbone network is used to extract feature information in the water area shoreline image and provide the feature information to the neck network, and the neck network is used to perform feature fusion on the feature information; The device further comprises a fitting module, which is specifically used for: Before obtaining the cross-section average flow velocity based on the pre-trained surface profile flow velocity coupling model, an optimal frame image and a reference line position are determined in the target observation image based on the surface flow velocity value in the world coordinate system; A local channel flow field is obtained based on the optimal frame image and the reference line position, and a flow velocity distribution formula is obtained by fitting the local channel flow field.

8. An electronic device comprising a memory, a processor, and a computer program stored on the memory and executable on the processor, characterized in that, The processor implements the channel flow measurement method based on the bionic eagle eye vision as claimed in any one of claims 1 to 6 when executing the program. 9.A non-transitory computer-readable storage medium having stored thereon a computer program, characterized in that, The computer program implements the channel flow measurement method based on the bionic eagle eye vision as claimed in any one of claims 1 to 6 when executed by the processor.

Images