A waterlogging depth monitoring method based on instance segmentation and target detection

Abstract

The application discloses a kind of based on instance segmentation and target detection's waterlogging depth monitoring method, fixed object perpendicular to ground and surface with known size marker is selected as reference object;Reference object-inversion-water surface instance segmentation model and indicator target detection model are constructed;The pixel equivalent of the center position of each indicator is calculated, and a pixel equivalent estimation model based on Gaussian process regression is constructed;The instance segmentation model is used to identify the reference object mask bottom boundary line under waterlogging state, to determine the water surface line height and its uncertainty;The indicator above water surface is identified using the target detection model, to determine the nearest indicator center above water surface line;Based on the pixel equivalent output by the pixel equivalent estimation model, the distance and uncertainty of the indicator center to water surface line are calculated;The waterlogging depth is obtained by subtracting the distance of the indicator center to water surface line from the distance of the indicator center to ground, and the waterlogging depth uncertainty is obtained based on the water surface line height uncertainty and distance uncertainty.

Description

Technical Field

[0001] This invention belongs to the field of hydrological monitoring and computer vision technology, and relates to water depth monitoring based on video images and deep learning technology, specifically to a method for monitoring water depth based on instance segmentation and target detection. Background Technology

[0002] Due to the impact of global climate change and urbanization, urban flooding disasters occur frequently. As a core indicator of the severity of urban flooding, water depth is an important basis for decision-making in response measures such as drainage and flood control, emergency dispatch, and rescue.

[0003] Traditional methods for water depth monitoring rely on water gauges and manual readings. This approach has limitations: water gauges have limited accuracy, and manual readings are subject to subjective bias and fail to capture the complete process of water depth changes. Introducing deep learning-based image recognition methods presents challenges. Conventional water gauges have sparse markings, making accurate monitoring difficult. Conversely, using densely marked water gauges with tiny markings as small targets in the image places high demands on camera resolution and the small target detection capabilities of the target detection algorithm, making it unsuitable for large-scale urban deployments.

[0004] Therefore, existing methods for monitoring water depth based on video images typically do not directly read water level gauge readings, but instead use known dimensions of a reference object for estimation. Specifically, the method first identifies the complete reference object and its portion above the water surface from the image. Then, it subtracts the number of pixels occupied by the portion above the water surface from the number of pixels occupied by the complete reference object, obtaining the pixel difference. Finally, it multiplies this difference by the overall pixel equivalent to obtain the water depth value. This type of method is easily affected by the reflection of the reference object in the water, impacting the accuracy of identifying the portion above the water surface and consequently affecting the accuracy of the water depth. Furthermore, in uncalibrated scenes, due to perspective projection and lens distortion, the pixel equivalent at different locations of the reference object in the image may not be entirely consistent, potentially exhibiting significant differences. Directly using the overall pixel equivalent for conversion can affect the accuracy of the water depth calculation. In addition, existing methods typically only output the water depth value and cannot measure the uncertainty of the result.

[0005] In summary, existing water depth monitoring technologies based on reference objects still suffer from limitations in accuracy and lack of measurement uncertainty, which restricts their widespread application. Therefore, there is an urgent need for a monitoring method that can accurately monitor water depth and measure water depth uncertainty. Summary of the Invention

[0006] To address the shortcomings of existing technologies, this invention proposes a method for monitoring water depth based on instance segmentation and object detection. A fixed object with several known-sized indicators on its surface is selected as a reference from a video image. An instance segmentation model is trained to segment the reference object, its reflection, and the water surface simultaneously, overcoming the interference of the reflection on reference object recognition. Based on the pixel equivalent of the indicator center on the reference object's surface at its image location, a pixel equivalent prediction model based on Gaussian process regression is constructed, outputting the pixel equivalent and its uncertainty, and using this model to estimate the water depth and its uncertainty.

[0007] A method for monitoring water depth based on instance segmentation and target detection, the specific steps of which are as follows:

[0008] Step 1: Selection of Monitoring Devices and Video Acquisition

[0009] s1.1 Select an object that is vertically fixed to the ground in the urban environment and has known-sized indicators distributed on its surface as a reference object.

[0010] s1.2. Pre-measure the actual physical distance from the center of each indicator on the reference surface to the ground, and adjust the camera field of view to cover the entire reference object and its surrounding area.

[0011] Step 2: Construct a reference object-reflection-water surface instance segmentation model

[0012] s2.1 Collect a series of image data in various scenes including reference objects, reference object reflections, and water surfaces. Introduce generative artificial intelligence to expand the image data and perform data augmentation processing on the collected and generated images.

[0013] s2.2. Using annotation tools, pixel-level annotations are performed on the outlines of reference objects, reference object reflections, and water surfaces in the image to obtain an instance segmentation annotation dataset containing three categories: reference objects, reference object reflections, and water surfaces.

[0014] s2.3. Using the above instance segmentation annotation dataset, train the deep learning-based instance segmentation pre-training model to obtain the reference object-reflection-water surface instance segmentation model.

[0015] Step 3: Construct a target detection model for indicators

[0016] s3.1. Collect a series of image data from various scenarios containing indicators, and perform data augmentation processing.

[0017] s3.2. Use annotation tools to annotate the bounding boxes of the indicators in the image to obtain the target detection annotation dataset of the indicators.

[0018] s3.3. Use the above-mentioned object detection labeled dataset to train the deep learning-based object detection pre-training model to obtain the indicator object detection model.

[0019] Step 4: Determine the pixel equivalent of the center position of the indicator.

[0020] s4.1. Use a camera to capture an image of the reference object in a water-free state, and input the image into the reference object-reflection-water surface instance segmentation model to obtain the reference object mask.

[0021] s4.2 Input the image within the reference mask area into the indicator target detection model to obtain the bounding boxes of each indicator.

[0022] s4.3 For each indicator, obtain its actual vertical dimension, and use the ratio of this dimension to the number of pixels between the upper and lower boundaries of the indicator's bounding box as the pixel equivalent of the indicator's center position.

[0023] Step 5: Construct a pixel equivalent inference model

[0024] s5.1 For each pointer, calculate the number of pixels from the center of its bounding box to the bottom boundary of the reference object, i.e., the ground, and use this number as the pixel height of its center position.

[0025] s5.2 For each indicator, construct a pixel height-pixel equivalent dataset by using the pixel height at its center position as the input feature and the pixel equivalent at its center position as the output label.

[0026] s5.3. Use the pixel height-pixel equivalent dataset to train a Gaussian process regression model to obtain the pixel equivalent inference model.

[0027] Step 6: Calculate the pixel height of the water surface line and its uncertainty.

[0028] s6.1. In the state of water accumulation, the image captured by the camera is input into the reference object-reflection-water surface instance segmentation model to obtain the reference object mask.

[0029] s6.2 Obtain the pixel height of all pixels on the bottom boundary line of the reference mask, and use the average pixel height as the pixel height of the water surface line; use the standard error of the average pixel height as the uncertainty of the pixel height of the water surface line. .

[0030] Step 7: Determine the distance from the center of the nearest marker above the water surface to the waterline and its uncertainty.

[0031] s7.1. In the state of water accumulation, the image within the reference mask area is input into the indicator target detection model to identify each indicator above the water surface and determine the center A of the indicator that is above the water surface line and closest to the water surface line.

[0032] s7.2. Draw a perpendicular line from A to the water surface line, and calculate the number of pixels on the perpendicular line segment that are at the bottom boundary of the reference object, i.e., the pixel height.

[0033] s7.3 Input the pixel height of each pixel on the vertical line segment into the pixel equivalent prediction model to obtain the pixel equivalent of each pixel on the vertical line segment and the covariance matrix between each pixel equivalent.

[0034] s7.4 Calculate the sum of the pixel equivalents of all pixels on the vertical segment, which is the actual distance L from A to the water surface line. W .

[0035] s7.5 Calculate the square root of the sum of the elements of the covariance matrix, which represents the uncertainty introduced by the pixel equivalent. , As L W Uncertainty.

[0036] Step 8: Calculate the water depth and its uncertainties.

[0037] s8.1. Based on the actual distance L from A to the ground measured in step 1. G Using L G Subtract L W The depth of the accumulated water is obtained.

[0038] s8.2 Input the pixel height of the water surface line into the pixel equivalent prediction model to obtain the corresponding pixel equivalent K. L .

[0039] s8.2 Uncertainty in calculating water depth :

[0040]

[0041] The present invention has the following beneficial effects:

[0042] 1. The training instance segmentation model simultaneously segments reference objects, their reflections, and the water surface in the image, effectively avoiding the impact of reference object reflections on the accurate identification of reference objects and improving the reliability of water depth monitoring.

[0043] 2. The pixel equivalent of each pixel height is inferred based on Gaussian process regression, and the water depth is calculated based on different pixel equivalents. This reduces the problem of inconsistent pixel equivalents caused by perspective projection and lens distortion. Compared with using a single pixel equivalent, the water depth monitoring accuracy is higher.

[0044] 3. The uncertainties introduced by pixel equivalent and water surface height were measured separately, and the uncertainty of water depth was obtained by combining these two types of uncertainties, thus characterizing the reliability range and error level of water depth monitoring.

[0045] 4. It can make full use of existing objects in the city as reference points. Compared with the scale lines of the water gauge, the size of the indicator is larger, which reduces the resolution requirements of the camera. Compared with small targets such as scale lines, the indicator is easier to identify and the requirements for the target detection model are also lower. Attached Figure Description

[0046] Figure 1 This is a flowchart of a water depth monitoring method based on instance segmentation and target detection;

[0047] Figure 2 This is a schematic diagram of the indicator and reference object in the embodiments of this method.

[0048] Figure 3 This is a schematic diagram of water surface line extraction in an embodiment of this method. Detailed Implementation

[0049] The present invention will be further explained below with reference to the accompanying drawings;

[0050] like Figure 1 As shown, a method for monitoring water depth based on instance segmentation and target detection specifically includes the following steps:

[0051] Step 1: Deployment of monitoring devices and video acquisition

[0052] s1.1 Select objects vertically fixed to the ground in the urban environment with known-sized markers distributed on their surfaces as reference objects, such as common roadside bicycle service shed posts, lane divider posts, crash barriers, crash cones, and height restriction gantries. These reference objects typically have alternating colored areas on their surfaces, which can serve as markers. Furthermore, this method is also applicable to scenarios where known-sized markers are painted on common objects such as lampposts, utility poles, and even walls.

[0053] For ease of explanation, this embodiment uses the uprights of the height-limiting gantry as a reference, such as... Figure 2 As shown. The reference point is divided into multiple regions, with adjacent regions having different colors. The vertical dimensions of each region are known, and each region serves as an indicator.

[0054] s1.2 The reference object is vertically fixed on the ground. The actual physical distance from the center position of each indicator on the reference object to the ground is measured in advance. By adjusting the camera, the center of the video image covers the complete reference object, all indicators and their surrounding area.

[0055] Step 2: Construct a reference object-reflection-water surface instance segmentation model

[0056] s2.1. Collect image data containing reference objects, their reflections, and water surfaces under different lighting conditions, wave textures, and perspective angles; using generative artificial intelligence as a benchmark based on real-world images of reference objects in a water-free state, generate image data of water surfaces, reference objects, and their reflections with varying degrees of turbidity, wave intensity, and light and shadow fragmentation; perform data enhancement on both the collected and generated images, including flipping, rotating, scaling, affine transformation, brightness, contrast, saturation adjustment, noise addition, color space transformation, blurring, sharpening, and grayscale conversion, to expand the image scale and diversity.

[0057] s2.2 For each image, the fine contours of the reference object, its reflection, and the water surface are marked in polygonal form using the annotation tool X-AnyLabeling. The contour annotation data is saved in YOLO format to obtain an instance segmentation annotation dataset containing three categories: reference object, reference object reflection, and water surface.

[0058] s2.3. Using the above instance segmentation annotation dataset, train the deep learning-based instance segmentation pre-trained model YOLO26-Seg to obtain the reference object-reflection-water surface instance segmentation model.

[0059] Step 3: Construct a target detection model for indicators

[0060] s3.1. Collect image data of the indicator under different lighting conditions, wave textures and perspective angles, and perform data enhancement processing on the image, including flipping, rotating, scaling, affine transformation, brightness, contrast, saturation adjustment, adding noise, color space transformation, blurring, sharpening and grayscale conversion, in order to expand the image scale and diversity.

[0061] s3.2. Use the annotation tool X-AnyLabeling to accurately annotate the bounding boxes of the indicators in the image, save the bounding box annotation data in YOLO format, and obtain the target detection annotation dataset of the indicators.

[0062] s3.3. Using the above-mentioned object detection labeled dataset, train the deep learning-based object detection pre-trained model YOLO26 to obtain the indicator object detection model.

[0063] Step 4: Determine the pixel equivalent of the center position of the indicator.

[0064] s4.1. Use a camera to capture an image of the reference object in a water-free state, and input the image into the reference object-reflection-water surface instance segmentation model to obtain the reference object mask.

[0065] s4.2. Crop out the image within the reference mask area and input it into the indicator target detection model to obtain the bounding boxes of each indicator.

[0066] s4.3 For each indicator, the ratio of its known vertical dimension to the number of pixels between the top and bottom boundaries of its bounding box is used as the pixel equivalent of the indicator's center position.

[0067] Step 5: Construct a pixel equivalent inference model

[0068] s5.1 For each pointer, count the number of pixels from the center of its bounding box to the bottom boundary of the reference object, i.e., the ground, and use this as the pixel height of its center position.

[0069] s5.2 For each indicator, construct a pixel height-pixel equivalent dataset by using the pixel height at its center position as the input feature and the pixel equivalent at its center position as the output label.

[0070] s5.3. Use the pixel height-pixel equivalent dataset to train a Gaussian process regression model. The kernel function of this model adopts the radial basis function kernel to obtain the pixel equivalent prediction model.

[0071] Step 6: Calculate the pixel height of the water surface line and its uncertainty.

[0072] s6.1. In the state of water accumulation, the image captured by the camera is input into the reference object-reflection-water surface instance segmentation model to obtain the reference object mask.

[0073] s6.2. Use the outline between the lowest points of the left and right edges of the reference mask as the bottom boundary line of the reference mask.

[0074] s6.3 Obtain the pixel height of each pixel on the bottom boundary line of the reference mask. The average pixel height As the pixel height of the water surface line, such as Figure 3 As shown; calculate the standard error SE of the average pixel height, and use SE as the uncertainty of the waterline pixel height. :

[0075]

[0076]

[0077] Where n is the number of pixels on the bottom boundary line of the mask, and s is the sample standard deviation.

[0078] Step 7: Determine the distance from the center of the nearest marker above the water surface to the waterline and its uncertainty.

[0079] s7.1. In the state of water accumulation, the image within the reference mask area is input into the indicator target detection model to identify each indicator above the water surface and select the indicator center A that is above the water surface line and closest to the water surface line.

[0080] s7.2. Draw a perpendicular line from A to the water surface line, and calculate the number of pixels on the perpendicular line segment that are at the bottom boundary of the reference object, i.e., the pixel height.

[0081] s7.3. Input the pixel heights of each pixel on the vertical segment into the pixel equivalent prediction model to obtain the pixel equivalent of each pixel on the vertical segment and the covariance matrix C between the pixel equivalents. m×m , where m is the number of pixels on the vertical line segment.

[0082] s7.4 Calculate the sum of the pixel equivalents of all pixels on the vertical segment, which is the actual distance L from A to the water surface line. W :

[0083]

[0084] in, Let be the pixel equivalent of the j-th pixel on the vertical segment.

[0085] s7.5 Calculate the square root of the sum of the elements of the covariance matrix, which represents the uncertainty introduced by the pixel equivalent. , As L W Uncertainty:

[0086]

[0087] Step 8: Calculate the water depth and its uncertainties.

[0088] s8.1 Obtain the true distance L from A to the ground. G L G Subtract L W This refers to the depth of the accumulated water.

[0089] s8.2 Input the pixel height of the water surface line into the pixel equivalent prediction model to obtain the corresponding pixel equivalent K. L .

[0090] s8.2 Uncertainty in calculating water depth :

[0091]

Claims

1. A method for monitoring water depth based on instance segmentation and target detection, characterized in that: Specifically, the following steps are included: Step 1: Select an object that is vertically fixed to the ground and has several indicators distributed on its surface from the urban environment as a reference object. Measure the size of each indicator and the actual physical distance from the center of each indicator to the ground in advance, and adjust the camera field of view to cover the entire reference object and its surrounding area. Step 2: Collect a series of image data in various scenes containing reference objects, reference object reflections, and water surfaces. Perform pixel-level annotation on the contours of reference objects, reference object reflections, and water surfaces in the images to train a reference object-reflection-water surface instance segmentation model. Step 3: Collect a series of image data containing indicators in various scenarios, and annotate the bounding boxes of the indicators in the images to train an indicator target detection model; Step 4: Use a camera to capture an image of the reference object in a water-free state, and input it into the reference object-reflection-water surface instance segmentation model. Input the reference object image within the reference object mask range into the indicator target detection model to obtain the bounding box of each indicator on the reference object surface. For each indicator, calculate the pixel equivalent of its center position using its size and bounding box information. Step 5: For each indicator, obtain the number of pixels from its center position to the bottom boundary of the reference object, i.e., the pixel height. Use the pixel height as the input feature and the pixel equivalent of the center position as the output label to train a Gaussian process regression model as the pixel equivalent inference model. Step 6: Under the condition of water accumulation, input the image captured by the camera into the reference object-reflection-water surface instance segmentation model to obtain the reference object mask. Extract the water surface line from the bottom boundary line of the reference object mask and calculate the uncertainty introduced by the water surface line. ; Step 7: Input the image within the reference mask area under the water accumulation state into the indicator target detection model to identify each indicator above the water surface; Determine the center A of the indicator that is above and closest to the waterline; Draw a perpendicular line from point A to the waterline. Calculate the number of pixels on the perpendicular segment that are at least as far as the bottom boundary of the reference object. Input this number into the pixel equivalent estimation model to obtain the pixel equivalent of each pixel on the perpendicular segment and the covariance matrix between these pixel equivalents. Calculate the sum of the pixel equivalents of all pixels on the perpendicular segment, which is the actual distance L from point A to the waterline. W The square root of the sum of the elements of the covariance matrix represents the uncertainty introduced by the pixel equivalent. , As L W Uncertainty; Step 8: Obtain the actual distance L from A to the ground. G Using L G Subtract L W Obtain the water depth; combine and Uncertainty in calculating water depth.

2. The method for monitoring water depth based on instance segmentation and target detection as described in claim 1, characterized in that: The acquired image data is augmented and then used to train the model.

3. The method for monitoring water depth based on instance segmentation and target detection as described in claim 2, characterized in that: The image data augmentation involves using generative artificial intelligence to generate images of water surfaces with different turbidity, different degrees of undulation, and different degrees of light and shadow fragmentation, as well as reference objects and their reflections.

4. The method for monitoring water depth based on instance segmentation and target detection as described in claim 2, characterized in that: The data augmentation processing includes one or more of the following: flipping, rotating, scaling, affine transformation, brightness, contrast, saturation adjustment, adding noise, color space transformation, blurring, sharpening, and grayscale conversion.

5. The method for monitoring water depth based on instance segmentation and target detection as described in claim 1, characterized in that: The reference-reflection-water surface instance segmentation model is obtained after training the pre-trained YOLO26-Seg.

6. The method for monitoring water depth based on instance segmentation and target detection as described in claim 1, characterized in that: The target detection model is obtained by training the pre-trained YOLO26.

7. The method for monitoring water depth based on instance segmentation and target detection as described in claim 1, characterized in that: The outline between the lowest points of the left and right edges of the reference mask is used as the bottom boundary line of the reference mask.

8. A method for monitoring water depth based on instance segmentation and target detection as described in claim 1 or 7, characterized in that: Obtain the pixel height of each pixel on the bottom boundary line of the reference mask. The average pixel height The standard error SE of the average pixel height is calculated as the pixel height of the waterline, and SE is used as the uncertainty of the pixel height of the waterline. .

9. The method for monitoring water depth based on instance segmentation and target detection as described in claim 8, characterized in that: The uncertainty of the water depth for: Among them, K L This is the pixel equivalent corresponding to the pixel height of the water surface line.

Images