Method, device and product for monitoring river flow in real time

Abstract

The invention relates to a method, device and product for monitoring river flow in real time, and the method comprises the steps: obtaining tower footing video data of a target river reach which comprises a target section and a pier buoy; inputting the tower footing video data into a water level and flow velocity identification model, and obtaining the water level elevation and the surface flow velocity of at least part of the cross section in the target section through the water level and flow velocity identification model; inputting the water surface elevation into a three-dimensional digital river channel model to obtain a section water passing area of a corresponding cross section; and obtaining the cross section flow of the target cross section according to the cross section water passing area and the surface flow velocity. According to the embodiment of the invention, automatic flow estimation based on the tower footing video can be realized, the manual measurement cost is reduced, the spatial-temporal resolution of flow monitoring is improved, and the method can be used for large-range river monitoring, especially in areas lacking hydrometric stations, and can provide data support for drainage basin water resource management and flood prevention early warning.

Description

Technical Field

[0001] The present application relates to the field of river water monitoring, and in particular to a method, device and product for real-time monitoring of river flow. Background Art

[0002] Traditional river flow monitoring technology mainly relies on the field measurement system of fixed hydrological stations. Specifically, its implementation method is to set up hydrological observation stations along the river, install water level meters, mechanical flow meters (such as rotor flow meters) or acoustic Doppler current profilers (ADCP) and other equipment, and combine manual inspections or semi-automatic data acquisition systems to periodically or continuously measure water levels, flow rates and cross-sectional areas, and calculate flow data based on hydraulic formulas (such as Manning's formula). This method can achieve high measurement accuracy at specific points (such as areas with straight river channels and stable flow patterns).

[0003] However, traditional technologies have significant defects in practical applications, which are mainly reflected in the following three aspects: First, the construction and operation and maintenance costs are high. Hydrological stations rely on permanent infrastructure (such as flow measurement cables and pile-based water level wells). The site selection, construction and equipment deployment require a lot of money and time, especially in remote mountainous areas or river sections with complex terrain. Geological surveys, material transportation and equipment anti-collision design further increase costs. In addition, the site needs regular maintenance (such as clearing sensor siltation and calibrating instrument accuracy), and professional personnel are deployed to perform data verification and abnormality processing, resulting in a heavy burden of long-term operation and maintenance.

[0004] Second, the spatial coverage capability is limited. Due to funding and geographical conditions, hydrological stations are usually sparsely distributed, making it difficult to fully cover tributaries, seasonal rivers or sudden flooding areas within the basin. For example, during the evolution of the flood peak, traditional stations are fixed in position and cannot dynamically track the sudden flow changes at the front of the flood, resulting in delayed flood warnings or blind spots in local area monitoring. In other words, the flow monitoring provided by related technologies relies on hydrological stations and manual testing, with limited coverage, and it is difficult to meet the needs of large-scale, continuous monitoring.

[0005] Third, dynamic monitoring and adaptability are insufficient. Traditional equipment has limited ability to capture complex flow patterns (such as turbulence and vortices), and relies on preset installation angles and flow measurement sections. When the river changes shape due to scouring or siltation, it is necessary to recalibrate parameters or even transform the site structure, which has poor flexibility. In addition, under extreme weather conditions (such as heavy rain and ice), the equipment is susceptible to physical damage or signal interference, and data continuity is difficult to guarantee.

[0006] The above technical bottlenecks seriously restrict the application efficiency of traditional methods in wide-area hydrological monitoring, real-time disaster warning and refined water resources management. Therefore, there is an urgent need for a river flow monitoring technology that can take into account low cost, high coverage and dynamic adaptability to break through the limitations of the existing system. Summary of the invention

[0007] In order to solve the above technical problems, the present invention provides a method, device and product for real-time monitoring of river flow. Through the embodiments of the present application, automatic flow estimation based on tower base video can be realized, the cost of manual measurement can be reduced, and the temporal and spatial resolution of flow monitoring can be improved. This method is suitable for large-scale river monitoring, especially in areas lacking hydrological stations, and can provide data support for river basin water resources management and flood prevention and early warning.

[0008] The present application provides a method for real-time monitoring of river flow, the method comprising: obtaining tower base video data of a target river section, wherein the target river section includes a target section and a bridge pier buoy; inputting the tower base video data into a water level and flow velocity identification model, and obtaining the water surface elevation and surface flow velocity of at least part of the cross section in the target section through the water level and flow velocity identification model; inputting the water surface elevation into a three-dimensional digital river channel model to obtain the cross-sectional water flow area of ​​the corresponding cross section, wherein the three-dimensional digital river channel model is used to couple the spatial terrain data of the target section with hydrodynamic parameters to simulate the water flow dynamics of the target section; and obtaining the cross-sectional flow of the target section according to the cross-sectional water flow area and the surface flow velocity.

[0009] The embodiment of the present application obtains the tower base video data of the target river section in real time, and then obtains the water surface elevations and surface flow velocities of multiple cross sections included in the target section based on the tower base video data. Thereafter, the cross-sectional water flow area of ​​the corresponding cross section is obtained through a pre-constructed three-dimensional digital river model of the target section. Finally, the cross-sectional flow of the target section is obtained based on the cross-sectional water flow area and the surface flow velocity. Automatic flow estimation based on the tower base video can be achieved, thereby reducing the cost of manual measurement and improving the spatiotemporal resolution of flow monitoring. This method is suitable for flow monitoring in rivers over a large range.

[0010] In some embodiments, the obtaining of tower base video data of the target river section includes: based on the baseline marker, collecting video data at the bridge piers in the target river section; the obtaining of the water surface elevation and surface velocity of at least part of the cross section in the target section through the water level and velocity identification model includes: segmenting the video data at the bridge piers to obtain a water level warning scale and the intersection of the bridge pier buoy and the water surface, and obtaining the water surface elevation with reference to the water level warning scale and according to the elevation of the intersection of the bridge pier buoy and the water surface.

[0011] Some embodiments of the present application use the water level warning scale as a reference to calculate the elevation of the intersection of the bridge pier buoy and the water surface, which can improve the accuracy of the calculated elevation of the intersection of the bridge pier buoy and the water surface.

[0012] In some embodiments, the video data at the pier is segmented to obtain a water level warning scale and a junction between the pier buoy and the water surface, and the water surface elevation is obtained based on the water level warning scale and the elevation of the junction between the pier buoy and the water surface, including: inputting the video data at the pier into a ship detection model to obtain a ship detection result of each image, and filtering the video data at the pier according to the ship detection result of each image to obtain a target video frame, wherein the target video frame is an image in which no ship is detected, or the target video frame is an image in which the ship is located in a non-sensitive area on the screen; inputting the target video frame into a pier segmentation model, obtaining pixel coordinates of the junction between the pier buoy and the water surface, and obtaining first elevation data of the junction between the pier buoy and the water surface according to the pixel coordinates; determining the pixel position of the lower edge of the water level warning scale according to the segmentation result output by the pier segmentation model, and obtaining second elevation data of the lower edge of the water level warning scale according to the pixel position; and obtaining the water surface elevation according to the first elevation data and the second elevation data.

[0013] Some embodiments of the present application can improve the accuracy of the calculated water surface elevation data by determining the water surface elevation based on the position of the lower edge of the water level warning ruler and the intersection of the pier buoy and the water surface.

[0014] In some embodiments, determining the pixel position of the lower edge of the water level warning ruler based on the segmentation result output by the pier segmentation model includes: extracting HSV color space features from the segmentation result output by the pier segmentation model; locating the water level warning ruler based on the HSV color space features and threshold segmentation; obtaining the mask boundary of the located water level warning ruler, and obtaining the pixel position of the lower edge of the water level warning ruler based on the mask boundary.

[0015] The embodiment of the present application extracts the HSV color space features based on the pier segmentation results, and locates the water level warning ruler through threshold segmentation (for example, segmenting the blue-green part) to obtain the precise mask boundary of the water level warning ruler and then determine the lower edge position of the water level warning ruler.

[0016] In some embodiments, the obtaining of tower base video data of the target river section includes: collecting video data of the river water surface corresponding to the target section; obtaining the water surface elevation and surface flow velocity of at least part of the cross section in the target section through the water level and flow velocity identification model includes: extracting flow velocity data of each section of the river water surface according to the video data of the river water surface to obtain segmented flow velocity results; applying a mathematical interpolation method to the segmented flow velocity results to generate continuous flow velocity distribution data as the surface flow velocity.

[0017] Some embodiments of the present application obtain the surface flow velocity of the target cross section through segmentation and interpolation strategies. Segmentation can improve the technical problem that the surface flow velocity is difficult to determine due to the shooting angle where the near is larger and the far is smaller, thereby improving the accuracy of the obtained surface flow velocity.

[0018] In some embodiments, the flow velocity data of each section of the river water surface is extracted according to the video data of the river water surface to obtain the segmented flow velocity result, including: dividing the target section into several sections according to the video scale relationship, and marking the actual spatial coordinate range of each section on the video screen included in the video data of the river water surface, and establishing a mapping relationship between pixel coordinates and actual distance, wherein the video scale relationship is used to characterize the scale relationship between the video data taken by the tower-based video monitoring system and the actual distance, and the unit of the video scale relationship is: pixel / meter; using tracer particles or ripple feature points as tracking targets, and calculating the pixel displacement of the tracking target between adjacent frames of the video data of the river water surface; converting the pixel displacement into actual displacement according to the mapping relationship, and calculating the segmented surface flow velocity according to the actual displacement and the frame rate.

[0019] The embodiments of the present application determine the segmented surface flow velocity corresponding to each segment through the pixel displacement of tracer particles or ripple feature points in adjacent frames and the frame rate, thereby improving the accuracy of the obtained segmented surface flow velocity.

[0020] In some embodiments, the method also includes: constructing the three-dimensional digital river model based on the water terrain obtained by the drone, the underwater terrain obtained by the unmanned boat, and the hydrodynamic parameters; when the unmanned boat navigates along the planned path, continuously collecting the marked three-dimensional coordinates through real-time dynamic differential positioning technology, wherein the three-dimensional coordinates include: longitude, latitude and elevation, and the planned path is used to cover the full river width of the target section; recording the sampling timestamp of each three-dimensional coordinate point, and aligning it with the time axis of the tower-based video monitoring system, and extracting the video frame corresponding to the sampling time of the real-time dynamic differential positioning technology from the video collected by the tower-based video monitoring system according to the synchronized timestamp; binding the pixel position in the corresponding video frame with the actual space coordinates collected by the real-time dynamic differential positioning technology, establishing the corresponding relationship between the video screen and the actual geographic space to obtain a coordinate mapping relationship; constructing an affine transformation model based on the coordinate mapping relationship to obtain the video scale relationship.

[0021] An embodiment of the present application provides a method for obtaining a video scale relationship. The video scale result obtained by the method is more accurate, thereby improving the accuracy of the obtained surface flow velocity.

[0022] In a second aspect, some embodiments of the present application provide a device for real-time monitoring of river flow, the device comprising: a tower base video data acquisition module, configured to acquire tower base video data of a target river section, wherein the target river section includes a target section and a bridge pier buoy; a water surface elevation and surface flow velocity acquisition module, configured to input the tower base video data into a water level and flow velocity identification model, and obtain the water surface elevation and surface flow velocity of at least part of the cross section in the target section through the water level and flow velocity identification model; a cross-sectional water flow area acquisition module, configured to input the water surface elevation into a three-dimensional digital river channel model to obtain the cross-sectional water flow area of ​​the corresponding cross section, wherein the three-dimensional digital river channel model is used to couple the spatial terrain data of the target section with hydrodynamic parameters to simulate the water flow dynamics of the target section; a cross-sectional flow rate calculation module, configured to obtain the cross-sectional flow of the target section based on the cross-sectional water flow area and the surface flow velocity.

[0023] In a third aspect, some embodiments of the present application provide a computer program product, including computer program instructions, which, when read and executed by a processor, can implement the method described in any one of the embodiments of the first aspect.

[0024] In a fourth aspect, some embodiments of the present application provide an electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor, when executing the computer program, can implement a method as described in any one of the embodiments included in the first aspect. BRIEF DESCRIPTION OF THE DRAWINGS

[0025] The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the present application and, together with the description, serve to explain the principles of the present application.

[0026] Figure 1 An architectural diagram of a system for real-time monitoring of river flow provided in an embodiment of the present application; Figure 2 A flow chart of a method for real-time monitoring of river flow provided in an embodiment of the present application; Figure 3 A schematic diagram of the composition of the water level and flow velocity identification model provided in the embodiment of the present application; Figure 4 A schematic diagram of segmenting the video data at the bridge pier to obtain the bridge pier buoy and the water level warning ruler provided in the embodiment of the present application; Figure 5 A schematic diagram of a three-dimensional digital river model provided in an embodiment of the present application; Figure 6 A block diagram of the composition of a device for real-time monitoring of river flow provided in an embodiment of the present application; Figure 7 A schematic diagram of the composition of an electronic device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0027] In order to more clearly understand the above-mentioned purposes, features and advantages of the present application, the embodiments of the present application will be further described below. It should be noted that the embodiments of the present application and the features in the embodiments can be combined with each other without conflict.

[0028] In the following description, many specific details are set forth to facilitate a full understanding of the present application, but the present application can also be implemented in other ways different from those described herein. Obviously, the embodiments in the specification are only part of the embodiments of the present application, not all of the embodiments.

[0029] Figure 1 A system for real-time monitoring of river flow is provided in some embodiments of the present application, and the system exemplarily includes: a tower-based video monitoring system 200 and a flow prediction server 100, wherein the tower-based video monitoring system 200 is configured to collect video data of a target river section in real time to obtain tower-based video data, and the flow prediction server 100 is configured to receive the tower-based video data collected by the tower-based video monitoring system 200 and predict the flow of a target section (a longitudinal section of the river) included in the target river section based on the tower-based video data.

[0030] For example, in some embodiments of the present application, a tower-based video monitoring system is a system obtained by installing tower-based camera equipment and data transmission equipment around a target river section (the river section includes a target section, bridge piers, and pier buoys). The tower-based video monitoring system uses the camera equipment to continuously shoot the target river section, obtain real-time status video data of river water movement and video data of the piers and pier buoys, and then calculates the cross-sectional flow of the target section based on these tower-based video data.

[0031] For example, in some embodiments of the present application, the flow prediction server 100 obtains the tower base video data of the target river section from the transmission equipment of the tower base video monitoring system, and then obtains the cross-sectional water flow area and surface flow velocity of the cross section included in the target section based on the tower base video data, and finally determines the cross-sectional flow of the target section based on these two quantities.

[0032] For example, some embodiments of the present application use tower-based video, combined with drone remote sensing, intelligent unmanned boats, and real-time dynamic carrier phase differential technology RTK (or real-time dynamic differential positioning technology) to build a three-dimensional digital river model of a typical section of a target river basin (i.e., a target section), and build a river flow estimation model (the model includes a water level and velocity identification model and a functional module for obtaining water surface flow). The embodiments of the present application use tower-based video monitoring to achieve real-time measurement of the flow of a typical section of a target river basin, providing a scientific basis and technical support for water resource management and water pollution prevention and control in the corresponding target river basin.

[0033] It should be noted that some embodiments of the present application are based on the constructed river flow model, using computer vision technology and deep learning algorithms to construct a conversion relationship model between river water surface width and flow, and realize flow estimation based on tower base video and multi-source remote sensing images. It should be noted that some embodiments of the present application also propose hydrological testing and ground verification. Specifically, the embodiments of the present application obtain the measured flow value of a typical section through equipment such as an on-site acoustic Doppler current profiler (ADCP) to verify the accuracy of the flow estimation model based on the tower base video.

[0034] Combine the following Figure 2 The method for real-time monitoring of river flow provided by some embodiments of the present application is exemplified. The method can be performed by Figure 1 The traffic prediction server 100 executes.

[0035] like Figure 2 As shown, some embodiments of the present application provide a method for real-time monitoring of river flow, the method comprising: S110, acquiring tower base video data of a target river section, wherein the target river section includes a target section, bridge piers, and bridge pier buoys.

[0036] For example, the tower base video data required for step S110 is obtained through the transmission equipment of the tower base video monitoring system. Since the embodiment of the present application requires the use of bridge piers or bridge pier buoys when calculating the flow of the target section, the tower base video data needs to collect images of these objects.

[0037] For example, in some embodiments of the present application, a tower-based video monitoring system consisting of a tower-based camera device and a data transmission device is set up around the target section, bridge piers, and bridge pier buoys. The camera device is then used to continuously shoot the target river section to obtain real-time video data of the movement of the river water and video data of the bridge piers and bridge pier buoys, thereby obtaining the tower-based video data required for S110.

[0038] S120, input the tower base video data into a water level and flow velocity identification model, and obtain the water surface elevation and surface flow velocity of at least part of the cross section of the target section (the target section is a longitudinal section, which includes many transverse sections) through the water level and flow velocity identification model.

[0039] It should be noted that the water level and flow velocity identification model of the embodiment of the present application is used to obtain the water surface elevation and surface flow velocity of each cross section included in the target cross section, for example, Figure 3 As shown, the water level and velocity identification model 310 of some embodiments of the present application exemplarily includes: a ship detection model 311, a pier segmentation model 312, a water level warning scale identification model 313 and a water surface elevation calculation module 314. Some embodiments of the present application select video frames that meet the requirements through the ship detection results of the ship detection model 311, and then use the video frames that meet the requirements as the input of the pier segmentation model to perform pier segmentation to obtain the position of the intersection of the pier buoy and the water surface, and then the water level warning scale identification model performs water level warning scale identification based on the data obtained by the pier segmentation result 474 to obtain the lower edge of the water level warning scale, and finally the water surface elevation calculation module 314 determines the water surface elevation according to the position of the lower edge of the water level warning scale and the position of the intersection of the pier buoy and the water surface. In some embodiments of the present application, the water level and velocity identification model determines the surface velocity of the target section by segmenting the target section to obtain the surface velocity of each section of the river and interpolating the surface velocity of each section of the river.

[0040] It is understandable that some embodiments of the present application require training of relevant models in the water level and flow velocity identification model 310 to obtain a ship detection model 311 and a pier segmentation model 312 that meet the requirements. An exemplary training process is provided later, so in order to avoid repetition, the training process will not be described here.

[0041] S130, inputting the water surface elevation into a three-dimensional digital river channel model to obtain a cross-sectional water flow area of ​​a corresponding cross section, wherein the three-dimensional digital river channel model is used to couple the spatial terrain data of the target section with the hydrodynamic parameters to simulate the water flow dynamics of the target section.

[0042] That is to say, some embodiments of the present application can determine the cross-sectional water flow area of ​​the corresponding cross-section by inputting the water surface elevation data of the corresponding cross-section in the determined target section into a pre-constructed three-dimensional digital river model.

[0043] For example, Figure 5 As shown in FIG. 1 , the left half of the figure is a pre-built three-dimensional digital river model of a target section, and the right half of the figure is a schematic diagram of determining the cross-sectional water area by the water surface position (i.e., the water surface elevation of a determined cross section). It should be noted that Figure 5 The area enclosed by the solid line curve in the right half of the figure and the dotted line corresponding to the water surface position is the cross-sectional water-passing area of ​​the cross section. For example, the size of the cross-sectional water-passing area can be determined by integration.

[0044] In some embodiments of the present application, the method for constructing the three-dimensional digital river channel model includes: using a drone or an unmanned boat to collect data around the target section according to the baseline mark; and constructing the three-dimensional digital river channel model according to the data collected by the drone and the unmanned boat. The following provides corresponding embodiments for this process, and the specific implementation process can be referred to below.

[0045] S140, obtaining the cross-sectional flow rate of the target cross-section according to the cross-sectional water flow area and the surface flow velocity.

[0046] For example, in some embodiments of the present application, the cross-sectional flow rate of the target cross-section may be determined by calculating the product of the cross-sectional water flow area and the surface flow velocity.

[0047] For example, in some embodiments of the present application, a three-dimensional digital river model, water level elevation and surface velocity data are integrated to accurately calculate the cross-sectional flow based on the flow calculation formula, thereby realizing tower base video flow measurement. The flow calculation formula is shown in (1): Q = AV (1) Where Q is the runoff, in m 3 / s; A is the cross-sectional water flow area, unit is m 2 ; V is the surface velocity, in m / s. Wherein, A is obtained through the water surface elevation and the three-dimensional digital river model (i.e., step S130). It can be understood that after the three-dimensional digital river model is constructed, the corresponding water level (water surface elevation) and cross-sectional water area can be obtained by inputting the river width, and the river width and cross-sectional water area can also be inferred from the water surface elevation. V represents the surface velocity.

[0048] The following is an example of Figure 2 Implementation of related steps. It should be noted that the tower base video data involved in S110 exemplarily includes video data at the bridge pier and video data on the river surface, wherein the video data at the bridge pier is used to calculate the water surface elevation, and the video data on the river surface is used to calculate the surface flow velocity.

[0049] In some embodiments of the present application, the obtaining of tower base video data of the target river section described in S110 includes: based on the baseline marker, collecting video data at the bridge piers in the target river section; the corresponding S120 exemplarily includes: segmenting the video data at the bridge piers to obtain a water level warning scale and the intersection of the bridge pier buoy and the water surface, and obtaining the water surface elevation with reference to the water level warning scale and according to the elevation of the intersection of the bridge pier buoy and the water surface.

[0050] For example, in some embodiments of the present application, the S120 exemplarily includes: In the first step, the video data at the bridge pier is input into the ship detection model to obtain the ship detection results of each image, and the video data at the bridge pier in the target river section is screened according to the ship detection results of each image to obtain the target video frame, wherein the target video frame is an image in which no ship is detected, or the target video frame is an image in which the ship is located in a non-sensitive area on the screen.

[0051] In the second step, the target video frame is input into the pier segmentation model to obtain the pixel coordinates of the intersection of the pier buoy and the water surface and obtain the first elevation data of the intersection of the pier buoy and the water surface according to the pixel coordinates.

[0052] In the third step, the pixel position of the lower edge of the water level warning ruler is determined according to the segmentation result output by the bridge pier segmentation model, and the second elevation data of the lower edge of the water level warning ruler is obtained according to the pixel position.

[0053] For example, the third step exemplarily includes: extracting HSV color space features from the segmentation results output by the pier segmentation model; locating the water level warning ruler based on the HSV color space features and threshold segmentation; obtaining the mask boundary of the located water level warning ruler, and obtaining the pixel position of the lower edge of the water level warning ruler based on the mask boundary.

[0054] The fourth step is to obtain the water surface elevation according to the first elevation data and the second elevation data.

[0055] That is to say, in some embodiments of the present application, S110 exemplarily includes: pier video acquisition, the acquisition process acquires video data of the pier position in the target river section according to the baseline mark, to ensure that the video clearly covers the status of the pier and the surrounding water area; the corresponding S120 exemplarily includes: video frame segmentation, specifically constructing a segmentation model suitable for the characteristics of the river water body, using the segmentation model to perform segmentation processing on the acquired video frames, and segmenting the piers, pier buoys and water level warning scales in the video frames; elevation calculation, using the segmented water level warning scale as the reference elevation, and calculating the elevation of the intersection point of the pier buoy and the water surface by analyzing the segmented video frames; water level elevation measurement, that is, based on the reference elevation and the elevation of the intersection point of the pier buoy and the water surface, the tower base video water level elevation is accurately measured, providing necessary elevation data support for subsequent flow measurement.

[0056] For example, in some embodiments of the present application, the video frame segmentation process included in S120 further includes: Step S120 - 1: Multi-environment video acquisition: Collect continuous video data of the target river section under different weather conditions (sunny, cloudy, rainy) and lighting conditions (morning, noon, dusk), covering a variety of environmental scenes to improve the generalization ability of the model.

[0057] Step S120-2: Video frame annotation and data set construction: randomly extract at least part of the video frames from the video, and manually annotate the ships, bridge pier buoys, and water level warning rulers in the extracted frames; wherein the ships are annotated as rectangular frames, and the bridge pier buoys and water level warning rulers are annotated as pixel-level masks.

[0058] Step S120 - 3: training a ship detection model: based on the manually labeled data, training a ship detection model (for example, the ship detection model may adopt a YOLOv7 network architecture) to ensure detection accuracy (mAP ≥ 0.90).

[0059] Step S120-4: training the bridge pier segmentation model: based on the manually labeled data, training the bridge pier segmentation model (for example, the model can adopt the nnUNet network architecture) to ensure the segmentation longitude (IoU ≥ 0.85); Figure 4 Based on the bridge pier segmentation result 474 shown, its HSV color space features are extracted, and the water level warning ruler is located through threshold segmentation (for example, blue-green); noise interference is eliminated based on morphological filtering to obtain the precise mask boundary of the water level warning ruler.

[0060] Step S120-5: Dynamic detection of ship interference: Input the measured video into the ship detection model to determine in real time whether there is a ship in the picture; if the ship is in the central area of ​​the picture (as an example of a sensitive area), mark the data of this period as "interfered" and suspend the water level calculation. The image where the ship is not in the central area is used as the target video frame.

[0061] Step S120-6: Key target positioning and water level calibration: For the video frame without ship interference (i.e., the target video frame), the pier segmentation model is input to obtain the pixel coordinates of the junction between the pier buoy and the water surface; based on the aforementioned color extraction results (i.e., HSV color space features), the pixel position of the lower edge of the water level warning ruler is determined.

[0062] Step S120-7: Calculate the water surface elevation: Based on the elevation data of the junction between the bridge pier buoy and the water surface and the lower edge of the water level warning ruler (i.e., the first elevation data and the second elevation data), take the weighted average (the weight is allocated by the confidence level) as the final water surface elevation. After that, the water surface elevation data is uploaded to the flow calculation module in real time to support the continuous measurement of the cross-sectional flow.

[0063] like Figure 4 As shown, the leftmost picture in the figure is the video data collected at the bridge pier, which includes the bridge 471, the bridge pier buoy 472 (the bridge pier buoy can float up and down along the bridge 471) and the water level warning ruler 473 (the water level warning ruler 473 is set on the bridge 471 and the elevation of the lower edge of the water level warning ruler is known); Figure 4 The middle image is a schematic diagram of the result of segmenting the pier buoy and the water surface (the intersection is the position of the bottom of the pier buoy in the figure) using the pier segmentation model (the figure is a schematic diagram of the pier segmentation result 474). Figure 4 The rightmost figure is used to show a schematic diagram of a water level warning ruler obtained by further segmentation based on the middle bridge pier segmentation result 474 (the blue line is the water level warning ruler obtained by segmentation). Figure 4 The pixel value of the intersection of the bridge pier buoy and the water level determined in the middle figure is consistent with the Figure 4 The difference in the pixel position of the lower edge of the water level warning ruler determined in the rightmost figure is then used to obtain the actual distance difference, and finally the water level elevation is calculated using the actual distance difference and the actual position of the lower edge of the water level warning ruler (i.e. the known elevation of the lower edge of the water level warning ruler).

[0064] As described above, some embodiments of the present application obtain the surface flow velocity based on the collected video data of the river water surface, and the process is exemplarily described below.

[0065] In some embodiments of the present application, the process of obtaining the tower base video data of the target river section described in S110 exemplarily includes: collecting video data of the river surface based on the baseline mark; the corresponding S120 exemplarily includes: S120-a, extracting the flow velocity data of each section of the river surface based on the video data of the river surface to obtain a segmented flow velocity result; S120-b, applying a mathematical interpolation method to the segmented flow velocity result to generate continuous flow velocity distribution data as the surface flow velocity.

[0066] In some embodiments of the present application, S120-a exemplarily includes: The first step is to divide the target section into several segments according to the video scale relationship, and mark the actual spatial coordinate range of each segment on the video screen included in the video data of the river surface, and establish a mapping relationship between pixel coordinates and actual distance, wherein the video scale relationship is used to characterize the scale relationship between the tower base video data and the actual distance, and the unit of the video scale relationship is: pixel / meter.

[0067] For example, in some embodiments of the present application, the target section length is 1000 meters, and the length is divided into units of 200 meters, and each section corresponds to an actual distance of about 200 meters. Specifically, in a real scene, the target section length is about 1000 meters, and what is presented in the tower base video is also 1000 meters. The embodiments of the present application search for the position corresponding to each 200 meters in the tower base video according to each 200 meters in the real scene, thereby segmenting 1000 meters. Executing this first step is to segment the length range actually covered by the target section, and correspond each section to the captured tower base video data.

[0068] In the second step, the tracer particles or ripple feature points are used as tracking targets, and the pixel displacement of the tracking targets between adjacent frames of the video data of the river water surface is calculated.

[0069] It can be understood that the tracking target exemplarily includes: tracer particles or ripple feature points, and the tracer particles are artificially added marks for tracking the flow, and the marks can be floating objects.

[0070] The third step is to convert the pixel displacement into actual displacement according to the mapping relationship, and calculate the segmented surface flow velocity according to the actual displacement and the frame rate.

[0071] For example, in some embodiments of the present application, the quotient of the actual displacement and the corresponding frame rate is solved to obtain the segmented surface flow velocity corresponding to the adjacent frame.

[0072] That is to say, in order to obtain the surface flow velocity, some embodiments of the present application include, for example, water surface video acquisition in S110: based on the baseline mark, high-definition video of the river water surface of the target section is acquired to ensure that the video covers the entire target section area and provides sufficient picture information for flow velocity measurement. The corresponding S120 includes, for example: video flow velocity processing, processing the acquired video images; extracting flow velocity data of the segmented river water surface to ensure the accuracy of the segmented measurement results, applying mathematical interpolation methods to the segmented flow velocity results, and generating continuous flow velocity distribution data as the complete flow velocity results of the target section. The video flow velocity measurement of some embodiments of the present application is realized: based on the interpolation results, a tower base video flow velocity measurement method is implemented to provide key flow velocity parameter support for flow calculation. For example, the video velocity processing process included in S120 exemplarily includes: the first step, video screen affine correction: according to the baseline identification data, the collected video screen is affine corrected to eliminate the image deformation caused by lens distortion or shooting angle, and a standard video image after geometric correction is generated; the second step, video segmentation and scale mapping: based on the video scale relationship (the video scale acquisition process can refer to the following example), the target section is divided into several segments (each segment corresponds to an actual distance of about 200 meters); the actual space coordinate range is marked for each video screen, and a mapping relationship between pixel coordinates and actual distance is established; the third step, segmented surface velocity calculation: for each video screen, the tracer particles (or ripple feature points) in the image are selected as the tracking target; through the large-scale particle image velocimetry method LSPIV (Large-Scale Particle Image Velocimetry) Velocimetry) algorithm is used to calculate the pixel displacement of tracer particles between adjacent frames; based on the mapping relationship recorded above, the pixel displacement is converted into actual displacement (unit: meter) to obtain the displacement Δd, combined with the shooting time interval Δt of adjacent frames determined by the frame rate (fps), the surface velocity is calculated using the following formula: v=Δd / Δt; the fourth step is three-dimensional velocity field interpolation reconstruction: the discrete surface velocity data in each segment is mapped to the corresponding position of the three-dimensional digital river model according to the actual spatial coordinates; the cubic spline interpolation method is used to generate a continuous velocity distribution field along the direction of the water flow; the interpolation results are smoothed and filtered to eliminate local outliers; the fifth step is the generation and output of a dynamic velocity map; the continuous velocity values ​​are plotted into a velocity map, which is the complete velocity result of the target section (that is, the surface velocity of each cross section included in the target section) for subsequent flow measurement.

[0073] The following is an example of the process of collecting tower base video data, building a three-dimensional digital river model, and obtaining the video scale.

[0074] Step S1: Baseline marking and video data acquisition: baseline marking is performed on the target river section to determine the target section to be measured; tower-based video equipment and data transmission equipment are set up around the target river section and target section, and the video equipment is used to continuously shoot the target river section to obtain real-time video data of the river water movement to obtain tower-based video data.

[0075] For example, the S1 exemplarily includes: Step S11: Baseline marker positioning: baseline marker is performed on the target river section, the latitude, longitude and elevation information of the baseline marker is obtained, and the marker point is accurately recorded.

[0076] Step S12: Camera equipment installation and debugging: Select the installation location of the tower base camera equipment, determine its latitude and longitude coordinates and elevation information; adjust the shooting angle of the camera equipment to ensure that the field of view completely covers the selected baseline identification area; at the same time, set up data transmission equipment to ensure that the captured video data can be completely stored and uploaded to the dedicated server in real time (for example, Figure 1 traffic prediction server).

[0077] Step S13: River status video acquisition: Start the camera equipment, continuously shoot the selected river section baseline identification area, obtain the real-time status video data of the target river section to obtain the tower base video data, and provide basic data support for subsequent analysis.

[0078] Step S2: Construction of a three-dimensional digital river channel model: Based on the baseline markers, use drones and unmanned boats to collect data around the target section; use the data collected by the drones and unmanned boats to construct a three-dimensional digital river channel model; at the same time, measure the water surface width information based on the three-dimensional digital river channel model.

[0079] For example, in some embodiments of the present application, S2 exemplarily includes: Step S21: UAV data collection: Based on the baseline identification data, the UAV is used to fly around the target section and around the upstream and downstream of the river, and the UAV images are stitched to obtain the digital orthophoto model (DOM) and digital surface model (DSM) above the river surface.

[0080] Step S22: Unmanned boat data collection: Based on the baseline identification data, the unmanned boat is equipped with an acoustic Doppler current profiler (ADCP) to travel back and forth along the section to collect riverbed data and construct an underwater terrain model of the section; at the same time, the operator allows the boat to be used for monitoring and auxiliary operations.

[0081] Step S23: Three-dimensional digital river channel modeling and water surface width measurement: The water terrain obtained by the UAV and the underwater terrain obtained by the unmanned boat are combined to construct a three-dimensional digital river channel model of the river channel. At the same time, the high-precision DOM results are used to measure the water surface width information of the river channel.

[0082] As an example of the method for constructing a three-dimensional digital river model of the present application, the method includes: ①Use a drone equipped with a real-time kinematic (RTK) module to conduct low-altitude measurements of typical sections to obtain digital orthophotos (DOM) and digital surface models (DSM) images.

[0083] ②Use unmanned boats equipped with single-beam echo sounders and ADCP and other equipment to conduct on-site measurements of typical sections to obtain high-precision underwater topography, flow velocity and flow data.

[0084] ③ Survey the terrain on site and use RTK to measure the hydraulic gradient. Combine the site conditions and the roughness empirical table to get the estimated value of the roughness (as an example of a hydrological variable).

[0085] ④ Use the ENVI extension module EcoHAT_UAV_RiverParameter to integrate the water terrain obtained by the drone, the underwater terrain obtained by the intelligent unmanned boat, and the hydrological variables obtained by ground measurement to form a Figure 5 The three-dimensional digital river model of the target section shown. The data input into the EcoHAT_UAV_RiverParameter module in the embodiment of the present application consists of three parts: the above-water topography of the UAV, the underwater topography of the unmanned boat, and the hydrological variables measured on the ground. Next, the fusion process involves data preprocessing, such as coordinate system one, data alignment, interpolation processing, etc. Then there are the specific technologies of multi-source data fusion, such as the ENVI module needs to perform spatial registration and three-dimensional reconstruction of different types of data, and finally generate a three-dimensional digital river model.

[0086] It can be understood that the application process of the three-dimensional digital river model constructed in some embodiments of the present application is: by inputting the river width into the digital river model, the water level, cross-sectional water area, wetted perimeter, water depth and other hydrological parameters can be obtained, and these parameters correspond one to one, and the river width, cross-sectional water area, etc. can also be obtained from the water level.

[0087] It should be noted that some embodiments of the present application obtain the video scale relationship through video scale calibration. The process is: shooting the round-trip navigation path of the unmanned boat; marking key positions in the video, and constructing the scale relationship between the captured image and the actual distance based on these marked points, providing an accurate distance reference for subsequent analysis.

[0088] As an example of the present application, the process of obtaining the video scale relationship through video scale calibration includes: The first step is to plan the navigation path of the unmanned boat and deploy the equipment for implementing the real-time dynamic differential positioning technology RTK (RTK equipment for short): based on the baseline identification data, plan the navigation path of the unmanned boat to and from the two banks to ensure that the full river width of the target section is covered; at the same time, deploy high-precision RTK equipment on the unmanned boat. It should be noted that the 3D coordinates of the markers are continuously collected by the RTK equipment.

[0089] The second step is RTK dynamic sampling and data synchronization: When the unmanned ship sails along the planned path, the RTK equipment collects three-dimensional coordinates (longitude, latitude, and elevation) in real time; the sampling timestamp is recorded synchronously and aligned with the time axis of the tower-based video monitoring system to ensure the temporal and spatial consistency of the data.

[0090] The third step is video frame matching and unmanned ship position calibration: based on the sampling timestamp, extract the video frame at the corresponding moment in the tower base video; identify the position of the unmanned ship in the video frame, and calibrate its actual spatial position in combination with the RTK coordinates.

[0091] The fourth step is dynamic calibration and verification of the video scale: Based on the mapping relationship between the pixel coordinates of the unmanned ship in the video and the actual space coordinates, an affine transformation model is established to calculate the scale coefficient of the video image and the actual scene; multiple groups of independent sampling points are selected to reduce the scale error.

[0092] That is to say, some embodiments of the present application obtain the video scale relationship through the following strategies: construct the three-dimensional digital river model according to the water terrain obtained by the drone, the underwater terrain obtained by the unmanned boat, and the hydrodynamic parameters; when the unmanned boat sails along the planned path, continuously collect the marked three-dimensional coordinates through the real-time dynamic positioning technology (that is, the RTK device executes this technology), wherein the three-dimensional coordinates include: longitude, latitude and elevation, the planned path is the navigation path of the unmanned boat between the two banks and the planned path is used to cover the full river width of the target section; record the sampling timestamp of each three-dimensional coordinate point, and align it with the time axis of the tower-based video monitoring system, and extract the video frame corresponding to the sampling moment of the real-time dynamic differential positioning technology (that is, the moment when the RTK device collects each three-dimensional coordinate) from the video collected by the tower-based video monitoring system according to the synchronized timestamp; bind the pixel position in the corresponding video frame with the actual space coordinate collected by the real-time dynamic differential positioning technology, establish the corresponding relationship between the video screen and the actual geographic space to obtain the coordinate mapping relationship; construct an affine transformation model based on the coordinate mapping relationship to obtain the video scale relationship. It should be noted that the tower base video is a video data collected in real time or periodically by a video surveillance system deployed at the tower base (i.e., the bridge pier location). In the embodiments of the present application, these data will be fused and analyzed with other sensors (such as RTK equipment) to support monitoring tasks such as water level measurement, structural deformation detection, etc.

[0093] Please refer to Figure 6 , Figure 6 The present invention provides a device for real-time monitoring of river flow. It should be understood that the device is similar to the above-mentioned Figure 2 The method embodiment corresponds to the method embodiment and can execute each step involved in the above method embodiment. The specific functions of the device can be referred to the description above. To avoid repetition, the detailed description is appropriately omitted here. The device includes at least one software function module that can be stored in the memory in the form of software or firmware or solidified in the operating system of the device. The device for real-time monitoring of river flow includes: a tower base video data acquisition module 410, a water surface elevation and surface flow velocity acquisition module 420, a cross-sectional water area acquisition module 430, and a cross-sectional flow calculation module 440.

[0094] The tower base video data acquisition module is configured to acquire tower base video data of a target river section, wherein the target river section includes a target section, a bridge pier, and a bridge pier buoy. As an example, the tower base video data acquisition module receives tower base video data provided by a camera device through a transmission module of a tower base video monitoring system.

[0095] The water surface elevation and surface flow velocity acquisition module is configured to input the tower base video data into a water level and flow velocity identification model, and obtain the water surface elevation and surface flow velocity of the target section through the water level and flow velocity identification model.

[0096] The cross-sectional water flow area acquisition module is configured to input the water surface elevation into a three-dimensional digital river channel model to obtain the cross-sectional water flow area of ​​the target section, wherein the three-dimensional digital river channel model is used to couple spatial terrain data with hydrodynamic parameters to simulate water flow dynamics.

[0097] The cross-sectional flow calculation module is configured to obtain the cross-sectional flow of the target cross-sectional area according to the cross-sectional water flow area and the surface flow velocity.

[0098] Those skilled in the art can clearly understand that, for the convenience and brevity of description, the specific working process of the device described above can refer to the corresponding process in the aforementioned method, and will not be described in detail here.

[0099] Some embodiments of the present application provide a computer program product, including computer program instructions. When the computer program instructions are read and executed by a processor, the method for real-time monitoring of river flow as described in any of the above embodiments can be implemented.

[0100] like Figure 7 As shown, some embodiments of the present application provide an electronic device 500, which includes a memory 510, a processor 520, and a computer program stored in the memory 510 and executable on the processor 520, wherein the processor 520 can implement a method for real-time monitoring of river flow as provided in the above embodiments when reading the program through a bus 530 and executing the computer program.

[0101] Processor 520 can process digital signals and can include various computing structures, such as complex instruction set computer structure, reduced instruction set computer structure, or a structure that implements a combination of multiple instruction sets. In some examples, processor 520 can be a microprocessor.

[0102] The memory 510 may be used to store instructions executed by the processor 520 or data related to the execution of instructions. These instructions and / or data may include code to implement some or all functions of one or more modules described in the embodiments of the present application. The processor 520 of the present disclosure embodiment may be used to execute the instructions in the memory 510 to implement Figure 2 The memory 510 includes a dynamic random access memory, a static random access memory, a flash memory, an optical memory or other memory known to those skilled in the art.

[0103] It should be noted that, in this article, relational terms such as "first" and "second" are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the terms "include", "comprise" or any other variants thereof are intended to cover non-exclusive inclusions, so that a process, method, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to the process, method, article or device. In the absence of further restrictions, the elements defined by the sentence "including a..." do not exclude the presence of other identical elements in the process, method, article or device including the elements.

[0104] The above is only an embodiment of the present application, and the embodiment enables those skilled in the art to understand and implement the present application. Various modifications to the embodiments will be apparent to those skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of the present application. Therefore, the present application will not be limited to the embodiments described herein, but will conform to the widest scope consistent with the principles and features disclosed herein.

Claims

1. A method for real-time monitoring of river flow, characterized in that: The method comprises: Acquire tower base video data of a target river section, wherein the target river section includes a target section and a bridge pier buoy; Inputting the tower base video data into a water level and flow velocity identification model, and obtaining the water surface elevation and surface flow velocity of at least a portion of the cross section in the target section through the water level and flow velocity identification model; Inputting the water surface elevation into a three-dimensional digital river channel model to obtain a cross-sectional water flow area of ​​a corresponding cross section, wherein the three-dimensional digital river channel model is used to couple the spatial topographic data of the target section with the hydrodynamic parameters to simulate the water flow dynamics of the target section; The cross-sectional flow rate of the target cross-sectional area is obtained according to the cross-sectional water flow area and the surface flow velocity.

2. The method according to claim 1, characterized in that The step of obtaining the tower base video data of the target river section includes: Based on the baseline mark, video data of the bridge pier in the target river section is collected; The step of obtaining the water surface elevation and surface velocity of at least a portion of the cross section in the target section by using the water level and velocity identification model includes: The video data at the bridge pier is segmented to obtain a water level warning scale and a junction between the bridge pier buoy and the water surface, and the water surface elevation is obtained based on the elevation of the junction between the bridge pier buoy and the water surface with the water level warning scale as a reference.

3. The method according to claim 2, characterized in that The video data at the bridge pier is segmented to obtain a water level warning scale and a junction between the bridge pier buoy and the water surface, and the water surface elevation is obtained based on the elevation of the junction between the bridge pier buoy and the water surface with the water level warning scale as a reference, including: Inputting the video data at the bridge pier into a ship detection model to obtain a ship detection result of each image, and filtering the video data at the bridge pier in the target river section according to the ship detection result of each image to obtain a target video frame, wherein the target video frame is an image in which no ship is detected, or the target video frame is an image in which the ship is located in a non-sensitive area on the screen; Input the target video frame into the pier segmentation model, obtain the pixel coordinates of the intersection of the pier buoy and the water surface, and obtain the first elevation data of the intersection of the pier buoy and the water surface according to the pixel coordinates; Determine the pixel position of the lower edge of the water level warning ruler according to the segmentation result output by the bridge pier segmentation model and obtain the second elevation data of the lower edge of the water level warning ruler according to the pixel position; The water surface elevation is obtained according to the first elevation data and the second elevation data.

4. The method according to claim 3, characterized in that The step of determining the pixel position of the lower edge of the water level warning ruler according to the segmentation result output by the bridge pier segmentation model comprises: Extracting HSV color space features from the segmentation results output by the bridge pier segmentation model; Positioning the water level warning ruler according to the HSV color space feature and threshold segmentation; The mask boundary of the located water level warning ruler is obtained, and the pixel position of the lower edge of the water level warning ruler is obtained according to the mask boundary.

5. The method according to any one of claims 1 to 4, characterized in that The step of obtaining the tower base video data of the target river section includes: Collecting video data of the river water surface corresponding to the target section; The step of obtaining the water surface elevation and surface flow velocity of at least a portion of the cross section in the target section through the water level and flow velocity identification model includes: Extracting flow velocity data of each section of the river water surface according to the video data of the river water surface to obtain segmented flow velocity results; A mathematical interpolation method is applied to the segmented flow velocity results to generate continuous flow velocity distribution data as the surface flow velocity.

6. The method according to claim 5, characterized in that The process of extracting the flow velocity data of each section of the river water surface according to the video data of the river water surface to obtain the segmented flow velocity results includes: The target section is divided into a plurality of sections according to the video scale relationship, and the actual spatial coordinate range of each section is marked on the video screen included in the video data of the river water surface, and a mapping relationship between pixel coordinates and actual distance is established, wherein the video scale relationship is used to characterize the scale relationship between the video data taken by the tower-based video monitoring system and the actual distance, and the unit of the video scale relationship is: pixel / meter; Using tracer particles or ripple feature points as tracking targets, and calculating pixel displacements of the tracking targets between adjacent frames of video data of the river surface; The pixel displacement is converted into an actual displacement according to the mapping relationship, and the segmented surface flow velocity is calculated according to the actual displacement and the frame rate.

7. The method according to claim 6, characterized in that The method further comprises: Constructing the three-dimensional digital river model according to the surface topography acquired by the drone, the underwater topography acquired by the unmanned boat, and the hydrodynamic parameters; When the unmanned boat navigates along the planned path, the three-dimensional coordinates of the marker are continuously collected by real-time dynamic differential positioning technology, wherein the three-dimensional coordinates include: longitude, latitude and elevation, and the planned path is used to cover the full river width of the target section; Recording the sampling timestamp of each three-dimensional coordinate point and aligning it with the time axis of the tower base video monitoring system, and extracting the video frame corresponding to the sampling time of the real-time dynamic differential positioning technology from the video collected by the tower base video monitoring system according to the synchronized timestamp; Binding the pixel position in the corresponding video frame with the actual space coordinates collected by the real-time dynamic differential positioning technology, establishing a corresponding relationship between the video screen and the actual geographic space to obtain a coordinate mapping relationship; An affine transformation model is constructed based on the coordinate mapping relationship to obtain the video scale relationship.

8. A device for real-time monitoring of river flow, characterized in that: The device comprises: A tower base video data acquisition module is configured to acquire tower base video data of a target river section, wherein the target river section includes a target section and a bridge pier buoy; A water surface elevation and surface velocity acquisition module is configured to input the tower base video data into a water level and velocity recognition model, and obtain the water surface elevation and surface velocity of at least a part of the cross section in the target section through the water level and velocity recognition model; A cross-sectional water flow area acquisition module is configured to input the water surface elevation into a three-dimensional digital river model to obtain a cross-sectional water flow area of ​​a corresponding cross section, wherein the three-dimensional digital river model is used to couple spatial terrain data with hydrodynamic parameters to simulate water flow dynamics; The cross-sectional flow calculation module is configured to obtain the cross-sectional flow of the target cross-sectional area according to the cross-sectional water flow area and the surface flow velocity.

9. A computer program product, characterized in that The method comprises computer program instructions, which can implement the method according to any one of claims 1 to 7 when read and executed by a processor.

10. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein: When the processor executes the computer program, the method according to any one of claims 1 to 7 can be implemented.

Images