A rain day river water level dynamic monitoring method and device based on a video monitoring network

Abstract

The application relates to a rain day river water level dynamic monitoring method and device based on a video monitoring network, which comprises the following steps: arranging a video monitoring network covering both banks of a river; based on panoramic video images, a mapping model for mutual conversion between river two-dimensional panoramic image pixel point coordinates and three-dimensional geographical space coordinates is established; a plurality of single frame images are extracted as key frames from the panoramic continuous video images of both banks of the river during rainfall by using the video monitoring network, rain streaks and raindrops are removed based on spectrum analysis and pixel recombination; binarization is carried out based on the gray mean value of the sampling points, and then the water level line image coordinates corresponding to the key frames are obtained by inputting the water level line recognition model; the water level line image coordinates are input into the mapping model to obtain the geographical space coordinates corresponding to the water level line; and the geographical space coordinates of the water level lines at different time points are combined to obtain the water level changes of both banks of the river. The application can realize continuous and complete monitoring of the river water level during rainfall at a low cost, and improve the monitoring precision.

Description

Technical Field

[0001] This application relates to the field of hydrological monitoring technology, and in particular to a method and device for dynamic monitoring of river water levels during rainy weather based on a video surveillance network. Background Technology

[0002] With the increasing frequency of extreme weather events, urban flood control places increasingly higher demands on hydrological monitoring technology. This is especially true given the large number of residential buildings located along riverbanks. Urban waterways must not only receive surface runoff but also rainwater from complex drainage systems at various levels, leading to unpredictable water level fluctuations. Furthermore, river overflows during heavy rainfall can easily cause economic and property damage to residents on both banks, and even pose serious threats to their lives and health. Therefore, monitoring river levels during rainfall, particularly in real-time during heavy rainfall, to understand dynamic changes in water levels and, in conjunction with rainfall forecasts, to take timely and appropriate preventative measures, is of significant emergency management importance. Currently, post-rainfall hydrological monitoring technology is relatively mature. However, during rainfall, the suddenness of rainfall, especially short-duration heavy rainfall, coupled with the complexity and danger of the environment, makes monitoring extremely challenging.

[0003] The main existing methods for monitoring river levels during rainfall include:

[0004] First, manual measurement is used, but conducting measurements near the river in rainy weather is dangerous and the data continuity is poor.

[0005] Second, while sensors can continuously acquire data, they can only be deployed at fixed locations, such as under bridges or at sluice gates, which limits the coverage of the data and makes it impossible to obtain complete river water level information.

[0006] Third, remote sensing can be used to measure river channels. Although this technology can obtain complete information about the river, it is costly and affected by rainfall, resulting in low image clarity and insufficient monitoring accuracy. Summary of the Invention

[0007] In order to achieve continuous and complete monitoring of river water levels during rainfall at low cost and improve monitoring accuracy, this application provides a method and device for dynamic monitoring of river water levels during rainy weather based on a video surveillance network.

[0008] Firstly, this application provides a method for dynamic monitoring of river water levels during rainy weather based on a video surveillance network.

[0009] This application is achieved through the following technical solution:

[0010] A method for dynamic monitoring of river water levels during rainy weather based on a video surveillance network includes the following steps:

[0011] Deploy a video surveillance network covering both banks of the river;

[0012] Use the video surveillance network to acquire panoramic video images of the two banks of the river at low water level on a sunny day;

[0013] Based on the panoramic video images, combined with the geospatial coordinate system, establish a mapping model for mutual conversion between the pixel point coordinates of the panoramic two-dimensional images of the two banks of the river and the three-dimensional geospatial coordinates;

[0014] Use the video surveillance network to extract several single-frame images at a preset first time interval from the panoramic continuous video images of the two banks of the river during rainfall when the rainfall amount is less than or equal to a preset rainfall threshold, so as to obtain a first key frame sequence;

[0015] Use the video surveillance network to extract several single-frame images at a preset second time interval from the panoramic continuous video images of the two banks of the river during rainfall when the rainfall amount is greater than the preset rainfall threshold, so as to obtain a second key frame sequence, wherein the second time interval is smaller than the first time interval;

[0016] Input the first key frame sequence or the second key frame sequence into a preset water level line recognition model, so as to obtain the image coordinates of the water level line corresponding to the key frames;

[0017] Input the image coordinates of the water level line into the mapping model, so as to obtain the geospatial coordinates of the water level line;

[0018] Combine the geospatial coordinates of the water level line at different moments to obtain the continuous time-series water level lines of the two banks of the river.

[0019] This application may be further configured in a preferred example as follows: the step of using the video surveillance network to extract several single-frame images at a preset second time interval from the panoramic continuous video images of the two banks of the river during rainfall when the rainfall amount is greater than the preset rainfall threshold to obtain the second key frame sequence comprises,

[0020] Acquire several single-frame images extracted at the second time interval from the panoramic continuous video images of the two banks of the river during rainfall when the rainfall amount is greater than the rainfall threshold, wherein the second time interval is smaller than the first time interval, divide the several single-frame images into several image groups in chronological order, each of the image groups comprises n single-frame images, satisfying int(n / 2)*t<T, where T is the first time interval and t is the second time interval;

[0021] Calculate the brightness and saturation of each pixel point on each of the single-frame images;

[0022] The pixels at the same image coordinate in each single frame image in the image group are sorted in descending order of brightness, and the first m pixels are taken as the target keyframe pixel sequence set, where m <n;

[0023] Based on the target keyframe pixel sequence set, the pixel point corresponding to the minimum saturation is selected as the pixel point of the image group keyframe, and the RGB values ​​corresponding to the pixel points of the image group keyframe are associated.

[0024] The RGB values ​​of pixels at all image coordinates of the image group keyframes are combined to obtain the image group keyframes. The image group keyframes of all image groups are combined to obtain the second keyframe sequence.

[0025] In a preferred embodiment, this application can be further configured such that the step of calculating the brightness and saturation of each pixel in each single-frame image includes,

[0026] Obtain the RGB value of each pixel in the single frame image, and obtain the RGB value (R) of the pixel with image coordinates (x, y) in the i-th single frame image. ixy G ixy B ixy );

[0027] Calculate the brightness B of the pixel based on its RGB values. i ,include,

[0028] B i =((R) ixy *299)+(G ixy *587)+(B ixy *114)) / 1000;

[0029] Calculate the saturation S of the pixel based on its RGB values. i ,include,

[0030] S i =(V max -V min ) / V max ;

[0031] V max =max(R) ixy G ixy B ixy );

[0032] V min =min(R) ixy G ixy B ixy ).

[0033] In a preferred embodiment, this application can be further configured such that the step of deploying a video surveillance network covering both banks of the river includes:

[0034] Based on the width and length of the riverbank, high-definition cameras that rotate up, down, left, and right are deployed.

[0035] A resolution threshold is preset for the high-definition camera. Monitoring images captured by the high-definition camera with a resolution exceeding the resolution threshold are taken as effective shooting areas. The effective shooting areas are connected to form a complete shooting area until the complete shooting area covers both sides of the river.

[0036] In a preferred embodiment, this application can be further configured to include the following steps:

[0037] If the complete camera area fails to cover both banks of the river, it is determined whether the width of the river exceeds a preset width threshold.

[0038] When the width of the river channel exceeds the width threshold, the high-definition cameras are adjusted until they are deployed facing each other on both banks.

[0039] In a preferred embodiment, this application can be further configured to include the following steps:

[0040] When the width of the river channel is less than or equal to the width threshold, the high-definition cameras are adjusted until they are arranged in an alternating pattern on both banks.

[0041] In a preferred embodiment, this application may be further configured such that, after the step of obtaining the first keyframe sequence, or the step of obtaining the second keyframe sequence, the following additional steps are included:

[0042] Mark the area of ​​water in the panoramic video image;

[0043] Based on the defined water area, record the image coordinates of several sampling points located at the edge of the water area to obtain a set of sampling point coordinates;

[0044] The first keyframe sequence or the second keyframe sequence is binarized using binarization rules, wherein the binarization rules include:

[0045]

[0046] In the formula, g(x,y) is the gray value of a single frame image at coordinates (x,y), and g b (x,y) is the gray value of a single frame image at coordinates (x,y) after binarization, and L is the gray threshold. The gray threshold is obtained by solving the average of the gray values ​​of each image pixel in the corresponding sampling point coordinate set of the first keyframe sequence or the second keyframe sequence.

[0047] Secondly, this application provides a dynamic monitoring device for river water levels during rainy weather based on a video surveillance network.

[0048] This application is achieved through the following technical solution:

[0049] A dynamic monitoring device for river water levels during rainy weather based on a video surveillance network includes,

[0050] The video surveillance network module is used to deploy a video surveillance network covering both sides of the river.

[0051] The basic data module is used to acquire panoramic video images of both banks of the river at low tide on a clear day using the video surveillance network.

[0052] The mapping model module is used to establish a mapping model for converting between pixel coordinates of panoramic two-dimensional images and three-dimensional geospatial coordinates on both sides of the river based on the panoramic video images and combined with the geospatial coordinate system.

[0053] The light rainfall keyframe module is used to obtain a number of single-frame images extracted from the panoramic continuous video images of both sides of the river when the rainfall is less than or equal to a preset rainfall threshold during rainfall using the video surveillance network, and to obtain a first keyframe sequence.

[0054] The heavy rainfall keyframe module is used to extract several single-frame images from the panoramic continuous video images of both sides of the river when the rainfall exceeds a preset rainfall threshold during rainfall using the video surveillance network, at a preset second time interval, to obtain a second keyframe sequence, wherein the second time interval is less than the first time interval.

[0055] The water level image recognition module is used to input the first key frame sequence or the second key frame sequence into a preset water level recognition model to obtain the water level image coordinates corresponding to the key frames.

[0056] The water level spatial coordinate module is used to input the water level image coordinates into the mapping model to obtain the geospatial coordinates of the water level.

[0057] The river channel full-time water level information module is used to combine the geospatial coordinates of the water level lines at different times to obtain the continuous time-series water level lines on both sides of the river channel.

[0058] Thirdly, this application provides a computer device.

[0059] This application is achieved through the following technical solution:

[0060] A computer device includes a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement the steps of any of the above-described methods for dynamic monitoring of river water levels during rainy weather based on a video surveillance network.

[0061] Fourthly, this application provides a computer-readable storage medium.

[0062] This application is achieved through the following technical solution:

[0063] A computer-readable storage medium storing a computer program, which, when executed by a processor, implements the steps of any of the above-described methods for dynamic monitoring of river water levels during rainy weather based on a video surveillance network.

[0064] In summary, compared with the prior art, the beneficial effects of the technical solution provided in this application include at least the following:

[0065] Deploying a video surveillance network covering both banks of the river, instead of traditional sensors and manual measurements, provides more comprehensive and sustainable river water level data, resulting in richer data collection. A mapping model for converting between 2D image pixel coordinates and 3D geospatial coordinates is established, allowing the acquisition of geospatial coordinates corresponding to 2D coordinates in captured water level images. This eliminates the need for remote sensing to collect 3D water level information, reducing implementation costs. The video surveillance network is used to extract several single-frame images from continuous panoramic video images of both banks during periods of light and heavy rainfall. This allows for the extraction of more local features to mitigate the impact of unclear water level images during heavy rainfall. These extracted single-frame images are then input into a pre-defined water level recognition model to obtain the water level image coordinates corresponding to keyframes. Inputting these coordinates into the mapping model yields the geospatial coordinates of the water level. Combining the geospatial coordinates of the water level at different times allows for the acquisition of continuous, temporal water level data for both banks of the river. This enables low-cost, continuous, and complete monitoring of river water levels during rainfall, improving the accuracy of water level monitoring during rainfall events. Attached Figure Description

[0066] Figure 1 A is a schematic diagram of the deployment of opposing cameras on both banks of a method for dynamic monitoring of river water levels in rainy weather based on a video surveillance network, which is another exemplary embodiment of this application.

[0067] Figure 1 B is a schematic diagram of the deployment of spaced cameras on both banks of a method for dynamic monitoring of river water levels in rainy weather based on a video surveillance network, provided as another exemplary embodiment of this application.

[0068] Figure 2This is a schematic diagram illustrating the setting of sampling points at the water surface in a method for dynamic monitoring of river water levels in rainy weather based on a video surveillance network, provided as an exemplary embodiment of this application.

[0069] Figure 3 A is a key frame sequence grayscale image of a method for dynamic monitoring of river water levels in rainy weather based on a video surveillance network, provided as an exemplary embodiment of this application.

[0070] Figure 3 B is a keyframe sequence binarized image of a method for dynamic monitoring of river water levels in rainy weather based on a video surveillance network, provided as an exemplary embodiment of this application.

[0071] Figure 4 This is a schematic diagram of the main process of a method for dynamic monitoring of river water levels in rainy weather based on a video surveillance network, which is an exemplary embodiment of this application. Detailed Implementation

[0072] This specific embodiment is merely an explanation of this application and is not intended to limit it. After reading this specification, those skilled in the art can make modifications to this embodiment without contributing any inventive step, but such modifications are protected by patent law as long as they fall within the scope of the claims of this application.

[0073] To make the objectives, technical solutions, and advantages of the embodiments of this application clearer, the technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0074] Furthermore, the term "and / or" in this article is merely a description of the relationship between related objects, indicating that three relationships can exist. For example, A and / or B can represent: A existing alone, A and B existing simultaneously, or B existing alone. Additionally, the character " / " in this article, unless otherwise specified, generally indicates that the preceding and following related objects have an "or" relationship.

[0075] This application proposes a method for dynamic monitoring of river water levels during rainy weather based on a video surveillance network. By deploying a high-definition video surveillance network and establishing a one-to-one correspondence between monitoring images and spatial locations based on geospatial mapping, the method acquires images at continuous fixed time intervals during rainy days. Keyframes are constructed through pixel reconstruction, binarized, and a neural network model is used to extract water level lines. The spatial information of the water level lines is obtained, and all monitoring times are combined to obtain full-time series water level information for the entire river channel. Here, full-time series water level information refers to continuous time-series river water level information. The water level line refers to the boundary between the water body and the riverbank. Compared to traditional methods, this approach reduces the risks associated with artificial rainmaking operations and provides high-precision, full-river, full-time series water level information during rainfall at a low cost, providing technical support for urban flood control.

[0076] This application provides a method for dynamic monitoring of river water levels during rainy weather based on a video surveillance network. The main steps of the method are described below.

[0077] Deploy a video surveillance network covering both banks of the river;

[0078] The video surveillance network was used to acquire panoramic video images of both banks of the river at low tide on a clear day.

[0079] Based on the panoramic video images, and combined with the geospatial coordinate system, a mapping model is established for the mutual conversion between the pixel coordinates of the panoramic two-dimensional images on both sides of the river and the three-dimensional geospatial coordinates.

[0080] Using the video surveillance network, several single-frame images are extracted from the panoramic continuous video images of both sides of the river when the rainfall is less than or equal to a preset rainfall threshold, at a preset first time interval, to obtain a first keyframe sequence.

[0081] Using the video surveillance network, several single-frame images are extracted from the panoramic continuous video images of both sides of the river when the rainfall exceeds a preset rainfall threshold, at a preset second time interval, to obtain a second keyframe sequence, wherein the second time interval is less than the first time interval;

[0082] Input the first keyframe sequence or the second keyframe sequence into a preset water level recognition model to obtain the water level image coordinates corresponding to the keyframes;

[0083] The coordinates of the water level image are input into the mapping model to obtain the geospatial coordinates of the water level.

[0084] By combining the geospatial coordinates of the water level at different times, a continuous temporal sequence of water level lines on both sides of the river channel can be obtained.

[0085] The embodiments of this application will now be described in further detail with reference to the accompanying drawings.

[0086] Specifically, high-definition cameras are evenly deployed along both banks of the river, with each camera assigned a unique number to comprehensively monitor real-time water level information on both sides of the river, establishing a video surveillance network covering both banks. In clear weather, this application uses the adjustment and rotation of the cameras to record the optimal focal length for camera initialization, and then uses the initialized cameras to capture high-definition images of the riverbank at low tide from different angles. These high-definition images of the riverbank at low tide can be obtained by managers who have pre-fixed the camera's shooting angle, and the images from the fixed-angle cameras are then stitched together to obtain a panoramic video image.

[0087] Based on panoramic video images, a one-to-one mapping between the two-dimensional image pixel coordinates (x, y) of the panoramic video images and the geospatial coordinates (x', y', z') of the actual sampling points in the river channel is determined through a unified geospatial coordinate system. A mapping model between the two-dimensional image pixel coordinates and geospatial coordinate information of the panoramic video images is established for mutual conversion between the two-dimensional image pixel coordinates and the three-dimensional geospatial coordinates on both sides of the river channel.

[0088] Specifically, a coordinate system is established with the center of the panoramic video image as the origin and the pixel size as the unit length. Oblique photogrammetry is performed using a drone to obtain the three-dimensional coordinates of the panoramic video image on the ground, thereby establishing a real-world three-dimensional model of the target area. Control point ground measurements are also performed using a drone to correct the results of the oblique photogrammetry and improve the accuracy of the photogrammetry. Then, by rotating the real-world three-dimensional model, a bird's-eye view with coordinate data and the same perspective as the panoramic video image is obtained. Based on the coordinate system of the bird's-eye view and the coordinate system of the panoramic video image, a coordinate transformation equation is established to realize the mapping between two-dimensional image coordinates and three-dimensional geographic space coordinates.

[0089] Once the model is established, by inputting any image coordinate of the panoramic 2D image into the mapping model, the mutual conversion between the 2D image pixel points and the 3D geospatial coordinates of the actual sampling points in the river channel can be realized.

[0090] Next, using a video surveillance network, several single-frame images are extracted from panoramic continuous video images of both banks of the river during rainfall when the rainfall is less than or equal to a preset rainfall threshold, at preset first time intervals, to obtain the first keyframe sequence. The rainfall threshold is the maximum water volume during light rainfall, statistically determined based on historical rainfall data of both banks of the river. The panoramic continuous video images are continuous video images of both banks of the river at preset shooting angles within the same period during rainfall. The period setting can be the time required for the camera to traverse all preset shooting angles. Several single-frame images extracted from each camera at preset shooting angles within the same period during rainfall are obtained to obtain the first keyframe sequence.

[0091] When the rainfall on both sides of the river is less than or equal to the rainfall threshold, that is, the current rainfall is relatively small, single-frame images with a fixed time interval T are directly extracted from continuous video images as keyframes, and the extracted n single-frame images are set in chronological order to obtain the first keyframe sequence.

[0092] When rainfall is heavy, a second keyframe sequence is obtained by extracting several single-frame images from panoramic continuous video images of both banks of the river at a second time interval when the rainfall exceeds a preset rainfall threshold, using a video surveillance network. The rainfall threshold can be the minimum water volume during heavy rainfall, statistically determined based on historical rainfall data of both banks of the river. The minimum water volume is allowed to fluctuate within a preset numerical range. The panoramic continuous video images are continuous video images of both banks of the river at a preset shooting angle during the same period of rainfall.

[0093] When the rainfall on both sides of the river exceeds the rainfall threshold, i.e. the current rainfall is relatively large, single-frame images with a fixed time interval t are extracted from continuous video images as keyframes, and the extracted n single-frame images are set in chronological order to obtain the second keyframe sequence.

[0094] To identify and extract water level lines from images, this application uses extracted single-frame images as samples for water level line recognition and employs a pre-trained neural network model for water level line recognition. For example, based on the YOLOv5 model, a large number of water level images of both banks of rivers are collected and the water level lines are calibrated as a training set. The data is input into the YOLOv5 model for iterative training, and the continuous two-dimensional water level line coordinates of the output image are output until the output accuracy meets the preset accuracy threshold. The currently trained YOLOv5 model is then used as the trained neural network model to obtain the water level line recognition model.

[0095] Input the first keyframe sequence or the second keyframe sequence into the water level recognition model above, and output the water level image coordinates corresponding to the keyframes.

[0096] Input the coordinates of the water level image into the mapping model to obtain the geospatial coordinates of the water level.

[0097] By combining the geospatial coordinates of water levels at different times, complete full-time water level information is established within a continuous monitoring period to obtain continuous time-series water level lines on both sides of the river.

[0098] Ultimately, continuous water level information is available on both banks of the river. By monitoring data from different periods, the changes in the target river's water level over time can be reflected, achieving the goal of comprehensive, complete, and high-precision monitoring of river water levels during rainfall.

[0099] In one embodiment, in order for the video surveillance network to capture high-definition video of water levels on both banks of the river, the step of deploying a video surveillance network covering both banks of the river includes,

[0100] Based on the width and length of the riverbank, high-definition cameras that rotate up, down, left, and right are deployed.

[0101] A resolution threshold is preset for the high-definition camera. Monitoring images captured by the high-definition camera with a resolution exceeding the resolution threshold are taken as effective shooting areas. The effective shooting areas are connected to form a complete shooting area until the complete shooting area covers both sides of the river.

[0102] Specifically, the number of high-definition cameras can be evenly distributed according to the width and length of the riverbank. After the cameras are fixedly installed, the monitoring images captured by the high-definition cameras with a resolution exceeding a preset resolution threshold are selected as the effective monitoring area. That is, the area in the monitoring image of the camera at a certain angle with a resolution exceeding a certain threshold is called the effective monitoring area. The cameras are combined in the order of shooting time to obtain a complete monitoring area covering both sides of the river. That is, the largest area formed by connecting the effective monitoring areas of each camera when it rotates, so as to obtain high-definition video of water level monitoring on both sides of the river.

[0103] If the combined effective camera areas captured from the high-definition cameras cannot cover both banks of the river, the resolution threshold of the high-definition cameras is manually adjusted based on the missing images until the combined complete camera area can cover both banks of the river. In this embodiment, the resolution threshold can be 0.01m.

[0104] Reference Figure 1 A. In one embodiment, to address the issue that when a river channel is wide, the effective capture area of ​​a camera on one side of the riverbank cannot cover the opposite bank, the step of deploying a video surveillance network covering both banks of the river further includes the following steps.

[0105] If the complete camera area fails to cover both banks of the river, it is determined whether the width of the river exceeds a preset width threshold.

[0106] When the width of the river channel exceeds the width threshold, the high-definition cameras are adjusted until they are deployed facing each other on both banks.

[0107] By determining whether the width of the river channel exceeds a preset width threshold, it's possible to confirm if the river channel is too wide, causing the effective capture area of ​​cameras on one side of the riverbank to not cover the opposite bank, resulting in unclear water level monitoring videos from both banks. This width threshold is designed by professionals based on the shooting range of high-definition cameras. When the river channel width exceeds the threshold, high-definition cameras are positioned at preset intervals along both banks to ensure that the effective capture area of ​​cameras on one side of the riverbank covers the opposite bank.

[0108] In one embodiment, on the premise of acquiring high-definition water level monitoring videos of both banks of the river channel, to reduce hardware expenditure cost, the step of arranging the video monitoring network covering both banks of the river channel further comprises the following steps,

[0109] When the width of the river channel is less than or equal to the width threshold, the high-definition cameras are adjusted until they are arranged in a spaced manner on both banks.

[0110] As shown in Figure 1 B, when the width of the river channel is less than or equal to the width threshold, that is, the river channel is in a relatively narrow case at this time, the camera on only one side of the river bank can also clearly capture the image of the opposite bank, so the high-definition cameras are arranged in a crossed and spaced manner along both banks at a preset interval distance.

[0111] In one embodiment, when the rainfall is heavy, the lens field of view of the camera will be blurred by rain. To acquire clearer monitoring images in the heavy rainfall scenario, this solution proposes a method for extracting key frames through spectral analysis and pixel recombination to perform rain removal processing on the monitoring images, comprising,

[0112] When collecting a plurality of single-frame images extracted at a preset second time interval from the panoramic continuous video images of both banks of the river channel during rainfall when the rainfall is greater than the preset rainfall threshold, the second time interval is set to be smaller than the first time interval adopted when extracting single-frame images at a low rainfall, that is, t<T, so as to collect more local water level image information. The plurality of single-frame images are divided into a plurality of image groups in chronological order, each image group includes n single-frame images, and the time interval between the shooting moment of int(n / 2)+1 single-frame images and the previous single-frame image is equal to the first time interval T, so as to ensure that the water level information of the extracted single-frame images has continuity, that is, int(n / 2)*t<T, where T is the first time interval and t is the second time interval.

[0113] The luminance and saturation of each pixel point on each single-frame image are calculated. Specifically, since raindrops or rain streaks in the image have the characteristics of high luminance and low saturation, the luminance and saturation of each pixel are calculated by reading the R, G, and B three-channel values of each pixel point on the single-frame image. Assuming that the RGB values at the image coordinate (x,y) of the pixel point in the i-th image are respectively (R ixy , G ixy , B ixy ), the luminance B i is calculated as follows:

[0114] B i =((R ixy *299)+(G ixy *587)+(B ixy *114)) / 1000;

[0115] Saturation S at the corresponding position i The calculation formula is as follows:

[0116] S i =(V max -V min ) / V max ;

[0117] V max =max(R) ixy G ixy B ixy );

[0118] V min =min(R) ixy G ixy B ixy );

[0119] Repeat the above steps until the brightness and saturation (B) of each pixel in each single frame image are obtained. i S i ).

[0120] The pixels at the same image coordinate in each single frame image in the image group are sorted in descending order of brightness, and the first m pixels are taken as the target keyframe pixel sequence set, where m <n;

[0121] Based on the target keyframe pixel sequence set, the pixel point corresponding to the minimum saturation is selected as the pixel point of the image group keyframe, and the RGB values ​​corresponding to the pixel points of the image group keyframe are associated.

[0122] Similarly, obtain the RGB values ​​corresponding to the pixels of the keyframes in the image group;

[0123] The RGB values ​​of pixels at all image coordinates of the keyframes of the image group are combined to obtain the keyframes of the image group. The keyframes of all image groups are combined to obtain the second keyframe sequence, wherein the shooting time of the second keyframe sequence is the average shooting time of the corresponding image group.

[0124] By using spectral analysis and pixel reconstruction to process the collected water level images during periods of heavy rainfall, the acquired water level images become clearer, which helps improve the accuracy of subsequent water level line extraction.

[0125] Furthermore, since the geographical scenes captured by the cameras are basically fixed, the main variable in the monitoring process is dynamic water level information. Water level changes along the riverbanks are difficult to monitor during rainy weather. Water level markers serve as a reference for monitoring these changes. To avoid the inconvenience and safety hazards of manually marking water levels during rain, automated pixel optimization can be achieved by pre-collecting sampling points for reference. Figure 2 The sampling points are k evenly distributed sampling points near the edge of the water area, which are manually selected, and k ≥ 6. In this embodiment, the number of sampling points can be 6.

[0126] Specifically, in one embodiment, after obtaining the first keyframe sequence or the second keyframe sequence, the method further includes:

[0127] Mark the area of ​​water in the panoramic video image;

[0128] Based on the defined water area, record the image coordinates of several sampling points located at the edge of the water area to obtain a set of sampling point coordinates;

[0129] The first keyframe sequence or the second keyframe sequence is binarized using binarization rules, wherein the binarization rules include:

[0130]

[0131] In the formula, g(x,y) is the gray value of a single frame image at coordinates (x,y), and g b (x,y) is the gray value of a single frame image at coordinates (x,y) after binarization, and L is the gray threshold. The gray threshold is obtained by solving the average of the gray values ​​of each image pixel in the corresponding sampling point coordinate set of the first keyframe sequence or the second keyframe sequence.

[0132] When monitoring water levels during rainfall, the same set of sampling point coordinates is used to binarize the pixels in the first keyframe sequence or the second keyframe sequence. This reduces the problem of insufficient recognition accuracy of the neural network model when extracting water level lines, which is caused by the color difference of the water body and the brightness of the light during rainfall. This further improves the recognition accuracy of the model.

[0133] This scheme uses prior information from sample images for analysis. By selecting sample images with the same camera number and rotation angle corresponding to keyframes, and combining them with k pre-defined fixed sampling points on the sample images, the image coordinates corresponding to the k sampling points are matched on the keyframe sequence, and the pixel grayscale values ​​corresponding to the k image coordinates are recorded. Finally, the average of the k grayscale values ​​is calculated as the grayscale threshold of the keyframe sequence, and the keyframe sequence images are converted into keyframe sequence grayscale images using binarization rules (e.g., ...). Figure 3 After (as shown in A), binarization is performed using a grayscale threshold to obtain the binarized keyframe sequence (as shown in A). Figure 3 (as shown in B).

[0134] In this embodiment, the solution does not directly use the original keyframes as input data for the neural network model. Instead, it uses the binarized images of the keyframes as input data for the neural network model and outputs the coordinates of the water level image in the keyframe sequence. This separates the water body of the river from the riverbank or other fixed scenes, which can better highlight the detailed features of the water level image. This eliminates the influence of dynamic environmental factors on water level image recognition, and thus enables better recognition of the water level line.

[0135] In one embodiment, pixel clustering and smoothing algorithms can also be used to identify and extract water level lines, and the method further includes the following steps:

[0136] Pixel clustering is performed on the binarized keyframe image, grouping pixels with a gray level of 0 into a pixel cluster;

[0137] Determine the riverbank orientation in the keyframe image, and use the pixels at the boundary of the pixel cluster of the riverbank orientation as the initial water level line pixels;

[0138] Based on the image coordinates of the initial water level line pixels, a moving window least squares polynomial smoothing algorithm is used to smooth the image and obtain the water level line image coordinates.

[0139] Reference Figure 4 In one embodiment, a method for dynamic monitoring of river water levels during rainy weather based on a video surveillance network includes:

[0140] Construct a high-definition video surveillance network covering both banks of the river;

[0141] Perform mutual mapping between multi-view fixed scene water level images and geospatial coordinate information;

[0142] Keyframe extraction is performed through spectral analysis and pixel reconstruction.

[0143] Binarization of keyframe sequences is performed based on the gray-scale mean value of preset sampling points as the gray-scale threshold.

[0144] A pre-defined convolutional neural network model is used to detect the binarized keyframe sequence and extract the water level line;

[0145] By combining water level lines at multiple time points and converting them into spatial coordinates, a full-time sequence of river water level changes can be constructed.

[0146] In summary, a video surveillance network-based method for dynamic monitoring of river water levels during rainy weather replaces traditional sensors and manual measurements by deploying a video surveillance network covering both banks of the river. This provides more comprehensive and sustainable river water level data, resulting in richer data collection. Furthermore, a mapping model is established for the conversion between two-dimensional image pixel coordinates and three-dimensional geographic spatial coordinates, thereby obtaining the geographic spatial coordinates corresponding to the two-dimensional coordinates in the captured water level image. This eliminates the need for remote sensing to collect the three-dimensional location information of the water level, reducing implementation costs. The video surveillance network can also be used to obtain data on river water levels during periods of low and high rainfall. Several single-frame images are extracted from panoramic continuous video images of both banks to extract more local features for heavy rainfall scenarios and reduce the impact of unclear water level images. The extracted single-frame images are then input into a preset water level recognition model to obtain the water level image coordinates corresponding to the keyframes. The water level image coordinates are then input into a mapping model to obtain the geospatial coordinates of the water level. By combining the geospatial coordinates of the water level at different times, continuous temporal water level lines on both banks of the river are obtained. This method enables continuous and complete monitoring of river water levels during rainfall at low cost, improving the accuracy of water level monitoring during rainfall.

[0147] Furthermore, by proposing a method to extract keyframes through spectral analysis and pixel reconstruction, rain removal processing is performed on water level images during heavy rainfall, which effectively improves the clarity of water level images caused by raindrops and rain streaks under rainstorm conditions, so as to obtain clearer and more realistic water level images of both sides of the river during rainstorms.

[0148] Furthermore, after extracting several single-frame images from panoramic continuous video images of both sides of the river during periods of low and high rainfall, the keyframe images are first binarized based on the coordinate set of presampled points on the panoramic video images during sunny days before being input into the neural network model for water level extraction. This makes the water features in the keyframes more prominent, which is beneficial for the model to better identify and extract water levels and improve the model's generalization ability.

[0149] It should be understood that the sequence number of each step in the above embodiments does not imply the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of this application.

[0150] This application also provides a video surveillance network-based dynamic monitoring device for river water levels during rainy weather. This video surveillance network-based dynamic monitoring device for river water levels during rainy weather corresponds one-to-one with the video surveillance network-based dynamic monitoring method for river water levels during rainy weather described in the above embodiments. This video surveillance network-based dynamic monitoring device for river water levels during rainy weather includes...

[0151] The video surveillance network module is used to deploy a video surveillance network covering both sides of the river.

[0152] The basic data module is used to acquire panoramic video images of both banks of the river at low tide on a clear day using the video surveillance network.

[0153] The mapping model module is used to establish a mapping model for converting between pixel coordinates of panoramic two-dimensional images and three-dimensional geospatial coordinates on both sides of the river based on the panoramic video images and combined with the geospatial coordinate system.

[0154] The light rainfall keyframe module is used to obtain a number of single-frame images extracted from the panoramic continuous video images of both sides of the river when the rainfall is less than or equal to a preset rainfall threshold during rainfall using the video surveillance network, and to obtain a first keyframe sequence.

[0155] The heavy rainfall keyframe module is used to extract several single-frame images from the panoramic continuous video images of both sides of the river when the rainfall exceeds a preset rainfall threshold during rainfall using the video surveillance network, at a preset second time interval, to obtain a second keyframe sequence, wherein the second time interval is less than the first time interval.

[0156] The water level image recognition module is used to input the first key frame sequence or the second key frame sequence into a preset water level recognition model to obtain the water level image coordinates corresponding to the key frames.

[0157] The water level spatial coordinate module is used to input the water level image coordinates into the mapping model to obtain the geospatial coordinates of the water level.

[0158] The river channel full-time water level information module is used to combine the geospatial coordinates of the water level lines at different times to obtain the continuous time-series water level lines on both sides of the river channel.

[0159] A video surveillance network-based device for dynamic monitoring of river water levels during rainy weather also includes,

[0160] The sampling point module is used to mark the water area in the panoramic video image. Based on the water area, it records the image coordinates of several sampling points located at the edge of the water area to obtain a set of sampling point coordinates.

[0161] A binarization module is used to perform binarization processing on the first keyframe sequence or the second keyframe sequence using binarization rules, wherein the binarization rules include,

[0162]

[0163] In the formula, g(x,y) is the gray value of a single frame image at coordinates (x,y), and g b(x,y) is the gray value of a single frame image at coordinates (x,y) after binarization, and L is the gray threshold. The gray threshold is obtained by solving the average of the gray values ​​of each image pixel in the corresponding sampling point coordinate set of the first keyframe sequence or the second keyframe sequence.

[0164] For specific limitations regarding a video surveillance network-based dynamic monitoring device for river water levels during rainy weather, please refer to the limitations of a video surveillance network-based dynamic monitoring method for river water levels during rainy weather mentioned above, which will not be repeated here.

[0165] The various modules in the aforementioned video surveillance network-based dynamic monitoring device for river water levels during rainy weather can be implemented entirely or partially through software, hardware, or a combination thereof. These modules can be embedded in or independent of the processor in a computer device, or stored in the computer device's memory as software, so that the processor can call and execute the corresponding operations of each module.

[0166] In one embodiment, a computer device is provided, which may be a server. The computer device includes a processor, memory, a network interface, and a database connected via a system bus. The processor provides computing and control capabilities. The memory includes a non-volatile storage medium and internal memory. The non-volatile storage medium stores an operating system, computer programs, and a database. The internal memory provides an environment for the operation of the operating system and computer programs in the non-volatile storage medium. The network interface is used to communicate with external terminals via a network connection. When the computer program is executed by the processor, it implements any of the above-described methods for dynamic monitoring of river water levels during rainy weather based on a video surveillance network.

[0167] In one embodiment, a computer-readable storage medium is provided, including a memory, a processor, and a computer program stored on the memory and executable on the processor, wherein the processor executes the computer program to perform the following steps:

[0168] Deploy a video surveillance network covering both banks of the river;

[0169] The video surveillance network was used to acquire panoramic video images of both banks of the river at low tide on a clear day.

[0170] Based on the panoramic video images, and combined with the geospatial coordinate system, a mapping model is established for the mutual conversion between the pixel coordinates of the panoramic two-dimensional images on both sides of the river and the three-dimensional geospatial coordinates.

[0171] Using the video surveillance network, several single-frame images are extracted from the panoramic continuous video images of both sides of the river when the rainfall is less than or equal to a preset rainfall threshold, at a preset first time interval, to obtain a first keyframe sequence.

[0172] Using the video surveillance network, several single-frame images are extracted from the panoramic continuous video images of both sides of the river when the rainfall exceeds a preset rainfall threshold, at a preset second time interval, to obtain a second keyframe sequence, wherein the second time interval is less than the first time interval;

[0173] Input the first keyframe sequence or the second keyframe sequence into a preset water level recognition model to obtain the water level image coordinates corresponding to the keyframes;

[0174] Input the coordinates of the water level image into the mapping model to obtain the geospatial coordinates of the water level;

[0175] By combining the geospatial coordinates of the water level at different times, a continuous temporal sequence of water level lines on both sides of the river channel can be obtained.

[0176] Those skilled in the art will understand that all or part of the processes in the methods of the above embodiments can be implemented by a computer program instructing related hardware. The computer program can be stored in a non-volatile computer-readable storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of this application. When executed, the computer program may include the processes of the embodiments of the methods described above. Any references to memory, storage, databases, or other media used in the embodiments provided in this application may include non-volatile and / or volatile memory. Non-volatile memory may include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM), or flash memory. Volatile memory may include random access memory (RAM) or external cache memory. By way of illustration and not limitation, RAM is available in a variety of forms, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), dual data rate SDRAM (DDRSDRAM), enhanced SDRAM (ESDRAM), synchronous link DRAM (SLDRAM), RAMbus direct RAM (RDRAM), direct memory bus dynamic RAM (DRDRAM), and memory bus dynamic RAM (RDRAM).

[0177] Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the above-described division of functional units and modules is used as an example. In practical applications, the above functions can be assigned to different functional units and modules as needed, that is, the internal structure of the system can be divided into different functional units or modules to complete all or part of the functions described above.

Claims

1. A method for dynamic monitoring of river water levels during rainy weather based on a video surveillance network, characterized in that, comprising the following steps, deploying a video surveillance network covering both banks of the river; acquiring, via the video surveillance network, panoramic video images of the two banks of the river at a low water level on a sunny day; establishing, based on the panoramic video images and in combination with a geospatial coordinate system, a mapping model for mutual conversion between the pixel point coordinates of the panoramic two-dimensional images of the two banks of the river and the three-dimensional geospatial coordinates; acquiring, via the video surveillance network, a plurality of single-frame images extracted at a preset first time interval from panoramic continuous video images of the two banks of the river during rainfall when the rainfall amount is less than or equal to a preset rainfall threshold, to obtain a first key frame sequence; acquiring, via the video surveillance network, a plurality of single-frame images extracted at a preset second time interval from panoramic continuous video images of the two banks of the river during rainfall when the rainfall amount is greater than the preset rainfall threshold, to obtain a second key frame sequence, wherein the second time interval is smaller than the first time interval; specifically, when extracting the plurality of single-frame images at the preset second time interval from the panoramic continuous video images of the two banks of the river during rainfall when the rainfall amount is greater than the preset rainfall threshold, the second time interval is set to be smaller than the first time interval adopted when extracting single-frame images under a smaller rainfall amount, that is, t<T, so as to collect more local water level image information; the plurality of single-frame images are divided into a plurality of image groups in chronological order, each image group comprises n single-frame images, and the time interval between the shooting moment of int(n / 2)+1 single-frame images and the previous single-frame image is equal to the first time interval T, so as to ensure that the water level information of the extracted single-frame images has continuity, that is, int(n / 2)*t<T, where T is the first time interval and t is the second time interval; inputting the first key frame sequence or the second key frame sequence into a preset waterline recognition model, to obtain the image coordinates of the waterline corresponding to the key frames; inputting the image coordinates of the waterline into the mapping model, to obtain the geospatial coordinates of the waterline; combining the geospatial coordinates of the waterline at different moments, to obtain continuous time-series waterlines of the two banks of the river.

2. The method for dynamic monitoring of river water levels during rainy weather based on a video surveillance network according to claim 1, characterized in that, the step of acquiring, via the video surveillance network, a plurality of single-frame images extracted at a preset second time interval from panoramic continuous video images of the two banks of the river during rainfall when the rainfall amount is greater than the preset rainfall threshold to obtain a second key frame sequence comprises, collecting a plurality of single-frame images extracted at the second time interval from the panoramic continuous video images of the two banks of the river during rainfall when the rainfall amount is greater than the rainfall threshold, wherein the second time interval is smaller than the first time interval, dividing the plurality of single-frame images into a plurality of image groups in chronological order, each of the image groups comprises n single-frame images, satisfying int(n / 2)*t<T, where T is the first time interval and t is the second time interval; calculating the brightness and saturation of each pixel point on each of the single-frame images; sorting the pixel points at the same image coordinate of each of the single-frame images in the image group in descending order of the brightness, and taking the first m pixel points as a target key frame pixel sequence set, wherein m<n; Based on the target keyframe pixel sequence set, the pixel point corresponding to the minimum saturation is selected as the pixel point of the image group keyframe, and the RGB values ​​corresponding to the pixel points of the image group keyframe are associated. The RGB values ​​of pixels at all image coordinates of the image group keyframes are combined to obtain the image group keyframes. The image group keyframes of all image groups are combined to obtain the second keyframe sequence.

3. The method for dynamic monitoring of river water levels during rainy weather based on a video surveillance network according to claim 2, characterized in that, The step of calculating the brightness and saturation of each pixel in each single frame image includes... Obtain the RGB value of each pixel in the single frame image, and obtain the RGB value (R) of the pixel with image coordinates (x, y) in the i-th single frame image. ixy G ixy B ixy ); Calculate the brightness of the pixel based on its RGB values. ,include, =((R ixy *299)+(G ixy *587)+(B ixy *114)) / 1000; Calculate the saturation of the pixel based on its RGB values. ,include, =(V max -V min ) / V max ; V max =max(R ixy ,G ixy ,B ixy ) ; V min =min(R ixy ,G ixy B ixy ) 。 4. The method for dynamic monitoring of river water levels during rainy weather based on a video surveillance network according to claim 1, characterized in that, The steps for deploying a video surveillance network covering both banks of the river include: Based on the width and length of the riverbank, high-definition cameras that rotate up, down, left, and right are deployed. A resolution threshold is preset for the high-definition camera. Monitoring images captured by the high-definition camera with a resolution exceeding the resolution threshold are taken as effective shooting areas. The effective shooting areas are connected to form a complete shooting area until the complete shooting area covers both sides of the river.

5. The method for dynamic monitoring of river water levels during rainy weather based on a video surveillance network according to claim 4, characterized in that, It also includes the following steps, If the complete camera area fails to cover both banks of the river, it is determined whether the width of the river exceeds a preset width threshold. When the width of the river exceeds the width threshold, the high-definition cameras are adjusted until they are deployed facing each other on both banks.

6. The method for dynamic monitoring of river water levels during rainy weather based on a video surveillance network according to claim 5, characterized in that, It also includes the following steps, When the width of the river channel is less than or equal to the width threshold, the high-definition cameras are adjusted until they are deployed at intervals along both banks.

7. The method for dynamic monitoring of river water levels during rainy weather based on a video surveillance network according to claim 1, characterized in that, After the step of obtaining the first keyframe sequence, or the step of obtaining the second keyframe sequence, the method further includes: Mark the area of ​​water in the panoramic video image; Based on the defined water area, record the image coordinates of several sampling points located at the edge of the water area to obtain a set of sampling point coordinates; The first keyframe sequence or the second keyframe sequence is binarized using binarization rules, wherein the binarization rules include: In the formula, (x,y) represents the grayscale value of a single frame image at coordinates (x,y), g b (x,y) represents the grayscale value of a single frame image at coordinates (x,y) after binarization. L The grayscale threshold is obtained by averaging the grayscale values ​​of each image pixel in the corresponding sampling point coordinate set of the first keyframe sequence or the second keyframe sequence.

8. A dynamic monitoring device for river water levels during rainy weather based on a video surveillance network, characterized in that, include, The video surveillance network module is used to deploy a video surveillance network covering both sides of the river. The basic data module is used to acquire panoramic video images of both banks of the river at low tide on a clear day using the video surveillance network. The mapping model module is used to establish a mapping model for converting between pixel coordinates of panoramic two-dimensional images and three-dimensional geospatial coordinates on both sides of the river based on the panoramic video images and combined with the geospatial coordinate system. The light rainfall keyframe module is used to obtain a number of single-frame images extracted from the panoramic continuous video images of both sides of the river when the rainfall is less than or equal to a preset rainfall threshold during rainfall using the video surveillance network, and to obtain a first keyframe sequence. Heavy rainfall key frame module, configured to extract a number of single-frame images at a preset second time interval from panoramic continuous video images of both banks of the river channel obtained by the video surveillance network when the rainfall amount is greater than a preset rainfall threshold during rainfall, so as to obtain a second key frame sequence, wherein the second time interval is smaller than the first time interval; specifically, when extracting a number of single-frame images at a preset second time interval from the panoramic continuous video images of both banks of the river channel when the rainfall amount is greater than the preset rainfall threshold during rainfall, the second time interval is set to be smaller than the first time interval adopted when extracting single-frame images under a relatively small rainfall amount, that is, t<T, so as to collect more local water level image information, the plurality of single-frame images are divided into a plurality of image groups in chronological order, each image group includes n single-frame images, and the time interval between the shooting moment of int(n / 2)+1 pieces of the single-frame images and the previous single-frame image is equal to the first time interval T, which ensures that the water level information of the extracted single-frame images has continuity, that is, int(n / 2)*t<T, where T is the first time interval and t is the second time interval; Water level line image recognition module, configured to input the first key frame sequence or the second key frame sequence into a preset water level line recognition model, so as to obtain the image coordinates of the water level line corresponding to the key frame; Water level line spatial coordinate module, configured to input the image coordinates of the water level line into the mapping model, so as to obtain the geospatial coordinates of the water level line; River channel full-time-series water level information module, configured to combine the geospatial coordinates of the water level line at different moments to obtain continuous time-series water level lines of both banks of the river channel.

9. A computer device, characterized in that, Comprising a memory, a processor and a computer program stored in the memory, wherein the processor executes the computer program to implement the steps of the method according to any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a computer program, wherein the computer program is executed by a processor to implement the steps of the method according to any one of claims 1 to 7.

Images