A method and system for detecting, morphologically analyzing and measuring distance in waterlogged areas

Abstract

The present invention relates to the field of image processing, and more specifically, to a method and system for detecting, morphologically analyzing, and measuring distances in waterlogged areas. The method comprises the following steps: acquiring a road image; using a pre-trained waterlogging detection model to detect the acquired road image and obtain an image containing waterlogged areas; inputting the image containing the waterlogged areas into a pre-trained UNet3+ network for segmentation to obtain a mask image of the waterlogged areas; performing morphological analysis on the mask image of the waterlogged areas to obtain characteristic parameters of the waterlogged areas; and performing distance measurement on the image containing the waterlogged areas based on a parallax method to obtain the distance between the waterlogged areas and vehicles. The present invention combines a waterlogging detection model with a UNet3+ network and parallax ranging, significantly improving the accuracy, segmentation efficiency, and ranging reliability of road waterlogging detection, and addressing the problems of limited detection range, insufficient feature extraction, and lack of spatial information in the prior art.

Description

Technical Field

[0001] The present invention relates to the field of image processing, and in particular to a method and system for detecting, morphologically analyzing and measuring distance in waterlogged areas. Background Art

[0002] Waterlogging on roads is an adverse factor affecting traffic safety, because there are more and more vehicle accidents caused by waterlogging on roads. In addition, waterlogging on roads can lead to traffic interruptions, and in severe cases, it can cause traffic paralysis and serious traffic accidents.

[0003] At present, the existing methods for detecting road water accumulation mainly include: water depth monitoring based on roadside sensors, water texture discrimination based on machine learning, and road water accumulation detection and instance segmentation based on deep learning.

[0004] However, sensor-based road water depth monitoring requires the installation of electronic water gauge sensors, power supply systems, and other supporting facilities. This leads to high installation and maintenance costs, a limited detection range, and is only applicable to urban flooding after heavy rain. Traditional machine learning-based methods for detecting water textures extract texture features from single video frames and use support vector machines to determine whether the texture represents water. This method requires manual extraction of texture features from images, a laborious task with limited features. Relying solely on texture features makes it difficult to effectively detect road water.

[0005] Deep learning-based road surface water detection and instance segmentation have strong self-learning capabilities and are applicable to various complex scenarios. Currently, commonly used algorithms include the YOLO (You Only Look Once) series of algorithms and U-Net (U-shaped network). However, existing methods generally rely solely on the YOLO series of algorithms for detection or models such as the UNet3+ (U-shaped Network 3+) for segmentation. This single approach has certain limitations. If only the YOLO series of algorithms are used for detection, detailed information about the shape and area of ​​the waterlogged area will be lacking. Using only models such as UNet3+ will result in slow segmentation efficiency and excessive computational complexity, making it difficult to meet real-time requirements.

[0006] In addition, the current detection method has not yet combined road surface water identification with the distance measurement system, and the vehicle cannot accurately obtain distance information, which limits the effectiveness and reliability of waterlogged road surface identification in practical applications. Summary of the Invention

[0007] In response to the problems mentioned in the prior art, the present invention proposes a method and system for detecting, morphologically analyzing, and measuring distance in waterlogged areas. This method combines an improved YOLOv8 network with a UNet3+ network to further improve the detection accuracy and segmentation accuracy of waterlogged areas. At the same time, ranging technology can be used to obtain distance information between vehicles and waterlogged areas, effectively avoiding accidents.

[0008] In order to achieve the above object, the present invention adopts the following technical solutions:

[0009] In a first aspect, the present invention provides a method for detecting, analyzing, and ranging waterlogged areas, which is applied to edge devices and includes the following steps:

[0010] Acquire road images;

[0011] Use the pre-trained waterlogging detection model to detect the acquired road image and obtain an image containing waterlogged areas;

[0012] Input the image containing the waterlogged area into the pre-trained UNet3+ network for segmentation to obtain the mask map of the waterlogged area;

[0013] Perform morphological analysis on the mask image of the waterlogged area to obtain the characteristic parameters of the waterlogged area;

[0014] The image containing the waterlogged area is processed based on the parallax method to obtain the distance between the waterlogged area and the vehicle;

[0015] The water accumulation detection model is a water accumulation detection model based on the YOLOv8 network. The construction process specifically includes: replacing the Conv module in the backbone network of the original YOLOv8 network with the SPDConv module; replacing the Upsample module in the neck network of the original YOLOv8 network with the DySample module; and replacing the original detection head of the head network of the original YOLOv8 network with the LSCD detection head.

[0016] As a further improvement of the present invention, a pre-trained waterlogging detection model is used to detect the acquired road image to obtain an image containing waterlogged areas, including:

[0017] The road image is resized and pixel normalized in sequence to obtain a standardized feature map;

[0018] The standardized feature map is input into the backbone network for multi-stage feature extraction to obtain a multi-channel feature map;

[0019] Input the multi-channel feature map into the neck network, dynamically adjust the multi-channel feature map through the DySample module to obtain a high-resolution feature map, and splice the high-resolution feature map with the multi-channel feature map to obtain a multi-scale fusion feature map;

[0020] The multi-scale fusion feature map is input into the head network, and deep features are extracted from the multi-scale fusion feature map to obtain deep features. The deep features are located and classified to obtain a detection frame containing target positioning and classification information. The detection frame containing target positioning and classification information is integrated through the LSCD detection head to output an image containing the water accumulation area.

[0021] As a further improvement of the present invention, an image containing a waterlogged area is input into a pre-trained UNet3+ network for segmentation to obtain a mask map of the waterlogged area, including:

[0022] The UNet3+ network includes an encoder and a decoder. The image containing the water area is input into the encoder for downsampling. Each downsampling includes convolution, batch normalization, ReLU activation and maximum pooling operations to obtain a deep feature map.

[0023] The decoder performs bilinear upsampling on the deep feature map to obtain a multi-scale feature map, performs fusion alignment on the multi-scale feature map to obtain a fused feature map, and performs segmentation prediction on the fused feature map to obtain an initial segmentation map;

[0024] The decoder also includes a classification guidance module, which is used to perform global classification on the deep feature map to obtain a classification probability value, and fuse the classification probability value with the initial segmentation map to obtain a fused segmentation map;

[0025] Construct a hybrid loss function, input the fused segmentation map into the hybrid loss function for update, and obtain the final segmentation map;

[0026] The final segmentation map is morphologically processed to obtain the mask map of the water accumulation area.

[0027] As a further improvement of the present invention, a morphological analysis is performed on the mask image of the waterlogged area to obtain characteristic parameters of the waterlogged area, including:

[0028] Binarize the mask image of the waterlogged area to generate a binary image;

[0029] Perform contour extraction on the binary image to obtain the characteristic parameters of the water accumulation area;

[0030] The characteristic parameters of the water accumulation area include contour area, contour perimeter, single contour area ratio, contour total area ratio, circularity and total area ratio of the water accumulation area.

[0031] As a further improvement of the present invention, the image containing the waterlogged area is processed for distance measurement based on the parallax method to obtain the distance between the waterlogged area and the vehicle, including:

[0032] Performing distortion correction on the image containing the water accumulation area to obtain a corrected image;

[0033] Convert the rectified image into a grayscale image and perform histogram equalization on the grayscale image;

[0034] The image after histogram equalization is processed based on the SGM algorithm to obtain the initial disparity map;

[0035] Filter the initial disparity map to obtain a smoothed disparity map;

[0036] The smoothed disparity map is reconstructed in three dimensions to obtain a visual point cloud map, and the distance between the water accumulation area and the vehicle is obtained based on the visual point cloud map.

[0037] As a further improvement of the present invention, it also includes a TensorRT inference engine, which is used to accelerate the water accumulation detection model and the UNet3+ network.

[0038] In a second aspect, the present invention provides a system for detecting, analyzing, and ranging waterlogged areas, comprising:

[0039] An acquisition module, used for acquiring road images;

[0040] A detection module is used to detect acquired road images using a pre-trained waterlogging detection model to obtain images containing waterlogged areas. The waterlogging detection model is based on the YOLOv8 network. The construction process specifically includes: replacing the Conv module in the backbone network of the original YOLOv8 network with the SPDConv module; replacing the Upsample module in the neck network of the original YOLOv8 network with the DySample module; and replacing the original detection head in the head network of the original YOLOv8 network with the LSCD detection head.

[0041] The segmentation module is used to input the image containing the waterlogged area into the pre-trained UNet3+ network for segmentation to obtain the mask map of the waterlogged area;

[0042] An analysis module is used to perform morphological analysis on the mask image of the waterlogged area to obtain characteristic parameters of the waterlogged area;

[0043] The ranging module is used to perform ranging processing on the image containing the water area based on the parallax method to obtain the distance between the water area and the vehicle.

[0044] In a third aspect, the present invention proposes a device for detecting, morphologically analyzing and ranging waterlogged areas, comprising a processor and a memory, wherein the processor implements the above-mentioned method for detecting, morphologically analyzing and ranging waterlogged areas when executing a computer program stored in the memory.

[0045] In a fourth aspect, the present invention proposes a computer-readable storage medium for storing a computer program, wherein the computer program, when executed by a processor, implements the above-mentioned method for detecting, analyzing the morphology, and measuring the distance of waterlogged areas.

[0046] Compared with the prior art, the present invention has achieved the following technical effects:

[0047] The present invention significantly improves the accuracy, segmentation efficiency and ranging reliability of road water accumulation detection by combining the improved YOLOv8 network with the UNet3+ network and ranging technology, and solves the problems of limited detection range, insufficient feature extraction and lack of spatial information in the existing technology. First, the improvement of the YOLOv8 network optimizes the feature extraction and multi-scale fusion capabilities by introducing the SPDConv module, DySample module and LSCD detection head. The SPDConv module enhances the sensitivity to small-scale water accumulation areas through cross-layer connection and dynamic convolution kernel adjustment, effectively improving the detection accuracy in complex scenes. The DySample module reduces the feature loss in the upsampling process through dynamic offset adjustment, making the high-resolution feature map generated by the neck network closer to the real scene and improving the positioning accuracy of the detection frame. The LSCD detection head adopts a lightweight structure and realizes multi-target parallel detection while reducing the amount of calculation through shared convolution modules and multi-branch classification regression mechanism, ensuring the real-time requirements of edge devices.

[0048] The UNet3+ network of the present invention solves the problem of loss of detail information in traditional segmentation models by integrating multi-level features of the encoder through full-scale skip connections and deep supervision mechanism, and significantly improves the segmentation accuracy of the mask map of the waterlogged area, especially the edge segmentation effect of irregularly shaped waterlogged areas. In addition, morphological analysis realizes the quantification of key parameters such as the area, perimeter, and shape of the waterlogged area through contour extraction and geometric parameter calculation, providing data support for road risk assessment. Finally, by combining binocular vision and parallax method, the defect of insufficient monocular ranging accuracy is overcome, and the three-dimensional coordinates of the waterlogged area and the real-time distance to the vehicle can be obtained, enhancing the practicality of the system in actual traffic scenarios. BRIEF DESCRIPTION OF THE DRAWINGS

[0049] Figure 1 It is a schematic diagram of the overall process of the present invention;

[0050] Figure 2 This is a schematic diagram of the LabelMe processing of the present invention, where a represents the original unlabeled image, b represents the target detection labeling schematic diagram, and c represents the image segmentation labeling schematic diagram;

[0051] Figure 3 This is a schematic diagram of the data set enhancement processing of the present invention;

[0052] Figure 4 This is a schematic diagram of the flow of the water accumulation detection model of the present invention;

[0053] Figure 5 This is an example diagram of the visualization of the detection results of the present invention;

[0054] Figure 6 This is a schematic diagram of the UNet3+ network segmentation process of the present invention;

[0055] Figure 7 This is an example diagram of the visualization of the segmentation results of the present invention;

[0056] Figure 8 This is an example diagram of a binary mask of a segmentation result of the present invention;

[0057] Figure 9 This is a flow chart of the morphological analysis of the present invention;

[0058] Figure 10 This is an example diagram of the morphological analysis results of the present invention;

[0059] Figure 11 Schematic diagram of the ranging process of the present invention. DETAILED DESCRIPTION

[0060] The present invention will be further described in detail below with reference to the accompanying drawings and examples. It will be understood that the specific embodiments described herein are intended only to illustrate the present invention and are not intended to limit the present invention. It should also be noted that, for ease of description, the accompanying drawings only illustrate portions relevant to the present invention, not all structures.

[0061] like Figure 1 As shown, the present invention provides a method for detecting, analyzing, and ranging waterlogged areas, which is applied to edge devices and includes the following steps:

[0062] Acquire road images;

[0063] Use the pre-trained waterlogging detection model to detect the acquired road image and obtain an image containing waterlogged areas;

[0064] Input the image containing the waterlogged area into the pre-trained UNet3+ network for segmentation to obtain the mask map of the waterlogged area;

[0065] Perform morphological analysis on the mask image of the waterlogged area to obtain the characteristic parameters of the waterlogged area;

[0066] The image containing the waterlogged area is processed based on the parallax method to obtain the distance between the waterlogged area and the vehicle;

[0067] The water accumulation detection model is a water accumulation detection model based on the YOLOv8 network. The construction process specifically includes: replacing the Conv module in the backbone network of the original YOLOv8 network with the SPDConv module; replacing the Upsample module in the neck network of the original YOLOv8 network with the DySample module; and replacing the original detection head of the head network of the original YOLOv8 network with the LSCD detection head.

[0068] The present invention will be further explained below with reference to the accompanying drawings and specific embodiments:

[0069] Step 1: Figure 2 As shown in the figure, labelme (image annotation tool) is used to annotate the road images containing waterlogged areas captured by the camera to form a waterlogged area image dataset. Figure 2 a represents the original unlabeled image, Figure 2 In the figure, b represents the target detection annotation diagram. The annotation box in the figure selects the overall position of the water accumulation area. The target detection annotation diagram is segmented by any segmentation model to form an image segmentation annotation diagram, which marks the boundary contour of the water accumulation area, such as Figure 2 As shown in c.

[0070] like Figure 3 As shown in the figure, the waterlogged area image dataset is enhanced by horizontal flipping, random brightness and contrast adjustment, rotation and Gaussian noise addition, and the enhanced waterlogged area image dataset is divided into training set, validation set and test set in the ratio of 8:1:1.

[0071] Step 2. Build a water accumulation detection model based on the YOLOv8 (You Only Look Once end-to-end single-stage object detection version 8) network. Replace the Conv (convolution) module in the backbone network of the original YOLOv8 network with the SPDConv (spatial depth conversion convolution) module; replace the Upsample (upsampling) module in the neck network of the original YOLOv8 network with the DySample (dynamic sampling) module; replace the original detection head of the head network of the original YOLOv8 network with the LSCD (Localization-Sensitive Contextual Detection) lightweight shared convolution detection head; input the training set in step 1 into the water accumulation detection model for training to obtain a trained water accumulation detection model.

[0072] Step 3: Use the binocular camera installed on the vehicle to obtain the road image, and use the water accumulation detection model to detect the water accumulation area in the image, such as Figure 4 As shown, the detection process is as follows:

[0073] The road image is input into the trained waterlogging detection model. The preprocessed road image is first adjusted to a size suitable for the waterlogging detection model and normalized. The processed image is then input into the backbone network. In this embodiment, the backbone network is divided into four stages for feature extraction. The first three stages consist of an SPDConv module and a C2f (feature fusion) module, and the fourth stage consists of an SPDConv module, a C2f module, and an SPPF (spatial pyramid pooling) module.

[0074] First, the input image L×W×C (Length×Width×Channels) is divided into four sub-feature maps through the SPDConv module, and reorganized in the channel dimension to achieve space-to-depth conversion, in which the spatial size is halved and the number of channels is increased fourfold, and the image size becomes X2∈( , ,4C), convert X2 into X3 (0.5L×0.5W×C2) through a non-strided convolutional layer and C2 (the number of convolution kernels used in the convolutional layer) filters, and generate and output the feature map X3 when the number of channels may be reduced.

[0075] The output feature map enters the C2f module, and the feature map depth is further increased using the C2f module configured with add=False (non-element-by-element addition) and n=3×d (n represents the number of repeated convolution units, d represents the depth multiplier) and the C2f module configured with add=False and n=6×d. The output feature map undergoes 3×d or 6×d convolution operations and is concatenated layer by layer, which increases the feature diversity in the channel dimension, retains more independent feature information, and obtains a multi-channel feature map.

[0076] In this embodiment, the fourth stage introduces the SPPF module to add multi-scale feature representation to the multi-channel feature map, that is, the number of channels is increased while keeping the image size unchanged, and the multi-channel feature maps of the second, third and fourth stages are input into the neck network. It should be noted that the multi-channel feature map of the first stage in the embodiment is not input into the neck network because the first C2f module extracts shallow, low-level features, and the neck network requires subsequent more semantic mid- and high-level features.

[0077] In the neck network, the multi-channel feature maps output by the backbone network are integrated, and the offset is calculated and generated through the DySample module. At the same time, the dynamic range factor is introduced to adjust the offset, and the input multi-channel feature maps are dynamically adjusted. The offset after dynamic adjustment is shown in the following formula:

[0078]

[0079] Where: O is the offset; X is the input multi-channel feature map; sigmoid represents the Sigmoid activation function, which is used to map the linear transformation result to between 0 and 1 to achieve nonlinear scaling; linear1() and linear2() represent the first and second linear transformation functions, respectively.

[0080] The pixel size is generated by pixel reorganization (sH: sampled Height, indicating the height of the adjusted multi-channel feature map; sW: sampled Width, indicating the width of the adjusted multi-channel feature map; g: grid, indicating the number of sampled groups) offset, which is superimposed on the initial sampling grid G ​​to form a sampling set S:

[0081]

[0082] Use the grid sampling function to process the sampling set, realize the dynamic adjustment of the sampling rate, and generate a high-resolution feature map , the processing process is as follows:

[0083]

[0084] Where: is a high-resolution feature map; S is a sampling set; grid_sample() is a grid sampling function.

[0085] The high-resolution feature map is spliced ​​with the multi-channel feature map output by the backbone network to obtain a richer multi-scale fusion feature map to improve the detection effect. At the same time, the multi-scale feature fusion map is input again into the Conv module and C2f module in the neck network to enhance and compress the spatial features. The processed multi-scale feature fusion map is input into the head network.

[0086] In the embodiment, the use of 1×1 convolution layer and GN (Group Normalization) in the head network can unify the number of channels of the multi-scale feature fusion map and converge the features into a shared convolution module, and extract deep features through a 3×3 convolution kernel; after feature extraction, the task is decomposed into two independent branches, namely the regression branch and the classification branch. In the regression branch, a 1×1 convolution layer is used to predict the coordinate offset of the bounding box, and a Scale layer is introduced to achieve dynamic adjustment of the feature scale to ensure accurate positioning of targets of different sizes; the classification branch uses a 1×1 convolution layer to predict the probability of each target category. The convolution layer weights of the two branches remain independent to ensure accurate processing of target positioning and classification tasks, and output a detection box containing target positioning and classification information, which is used to mark the location and category information of the water accumulation area in the image. The visualization result is shown in the following example. Figure 5 As shown, for example Figure 5 The category information of the image containing the water area on the left is water, the confidence level is 1.0, and dis (distance) 0.4 (unit: meter) represents the distance between the location of the water area and the edge device that obtained the image of the water area.

[0087] Step 3: Figure 6 As shown in the figure, the image containing the water area is input into the UNet3+ network. First, the pixel values ​​of the image containing the water area are normalized to reduce the difference in feature distribution and improve training stability.

[0088] In the embodiment, the UNet3+ network includes an encoder and a decoder. The image containing the water accumulation area first enters the encoder. In the embodiment, the encoder includes 5 layers. The downsampling of each layer includes two 3×3 convolutions, batch normalization and ReLU (rectified linear unit) activation functions.

[0089] After the image containing the water accumulation area is input, two 3×3 convolutions are first performed to extract local features and introduce nonlinearity. Batch normalization is applied after each convolution. After normalization, feature extraction is completed by connecting the ReLU activation function. After feature extraction, a 2×2 maximum pooling operation is used for downsampling to reduce the spatial resolution of the feature map and extract more abstract deep semantic information to obtain a deep feature map.

[0090] The entire encoder performs four downsampling operations, and no downsampling operation is performed on the fifth layer to maintain a low spatial size and rich semantic features. In this process, the output of each layer is a subset of the previous layer, containing more abstract feature information. The width and height of the feature map are halved layer by layer, and the number of channels is doubled.

[0091] The deep feature map is fed into the decoder for feature recovery and segmentation prediction: The decoder also contains five layers. Each layer doubles the width and height of the deep feature map through bilinear upsampling, gradually recovering to the spatial resolution of the input image, and obtaining a multi-scale feature map. UNet3+ uses a full-scale skip connection mechanism in the decoding process, which not only passes the feature map of the same scale as the current decoding layer in the encoder to the decoder, but also passes the feature map of other scales to the decoder. Figure 1 and merge.

[0092] Specifically, if Figure 6As shown in the figure, the feature maps from the lower layers (large scale) of the encoder are downsampled by maximum pooling to match the resolution of the current decoding layer; while the feature maps from the higher layers (small scale) of the encoder or the deeper layers of the decoder are upsampled by bilinear upsampling to align with the current decoding layer. After the feature maps of all scales are made consistent in spatial size, they are fused by feature splicing, and feature aggregation is performed using 3×3 convolution and ReLU activation function to generate a fused feature map containing rich spatial details and deep semantic information.

[0093] In this example, UNet3+ introduces a Deep Supervision mechanism in the decoder to perform multi-scale segmentation output on fused feature maps at different levels. This is achieved by setting side branch outputs at each decoding layer. Each decoding layer performs a 3×3 convolution on the fused feature map and restores it to the same spatial size as the input image through bilinear upsampling to obtain the initial segmentation map.

[0094] To further improve segmentation accuracy, the embodiment introduces a classification guidance module to classify and predict whether the input depth feature map contains water accumulation areas. The prediction results output by the classification guidance module are normalized using a sigmoid function and then element-wise multiplied with the initial segmentation map obtained after upsampling at each decoding layer. This suppresses background areas and enhances target areas, ultimately generating a fused segmentation map from each layer of deep supervision. Finally, this fused segmentation map output by all deep supervision is fed into a loss function for joint optimization to obtain the final segmentation map.

[0095] In order to further improve the model's ability to recognize fuzzy boundaries and complex areas, this embodiment uses a hybrid loss function based on the multi-scale structural similarity index (MS-SSIM) in the UNet3+ network to give fuzzy boundaries a higher weight.

[0096] MS-SSIM extracts corresponding local area blocks from the segmentation map and label map using a sliding window at multiple scales to calculate the structural similarity between the predicted segmentation result and the true label, effectively characterizing the distribution differences in regional structures. Specifically, the greater the regional distribution difference, the higher the corresponding MS-SSIM value, and the model will impose stronger optimization constraints on the boundary features of the region. It is calculated as follows: at the same spatial position in the image, two corresponding N×N area blocks are cropped from the segmentation result and the true value label, that is, each pair of area blocks is completely aligned in spatial position to ensure that the structural information can be accurately compared pixel by pixel:

[0097] p={p j :j=1,…,N 2} and g={g j :j=1,…,N2}

[0098] Where: p represents the prediction region block vector; g represents the label region block vector; j represents the position index in the expansion vector; p j represents the j-th predicted pixel value; g j represents the jth label pixel value; N represents the side length of the area block; N 2 Indicates the total number of pixels in the region block.

[0099] The MS-SSIM loss function is defined as:

[0100]

[0101] Where: M represents the total number of scales; μ p is the mean of p; σ p is the variance of p; μ g is the mean value of g; σ g is the variance of g; σ pg represents the covariance of p and g; β m 、 γ m Define the relative importance of each scale; C1 and C2 are constants, C1 is taken as 0.012 and C2 is taken as 0.032.

[0102] In addition, the loss function adopts a combination of multiple loss functions to construct a multi-dimensional joint optimization mechanism at the pixel level, block level, and image level. Specifically, it includes: Focal loss for optimizing pixel category prediction, MS-SSIM loss for optimizing fuzzy boundary areas, and IoU loss for improving overall segmentation overlap. The three loss functions together constitute a hybrid loss function, which is defined as:

[0103]

[0104] Where: for Focal loss; is the MS-SSIM loss; is the IoU loss; is a mixed loss function.

[0105] By using the hybrid loss function, we can effectively improve the clarity of the boundary of the waterlogged area and the recognition ability of complex structures while ensuring the overall segmentation accuracy, further optimize the segmentation results, and generate the final segmentation map. The visualization result of the final segmentation map is shown in the following example. Figure 7As shown in the figure, the specific boundary and shape of the water accumulation can be identified; the final segmentation map is morphologically processed to obtain a mask map of the water accumulation area. The mask map result example is as follows Figure 8 shown.

[0106] Step 4: Figure 9 As shown, the input waterlogged area mask image is first binarized to separate the waterlogged area from the non-waterlogged area to form a binary image; the total number of pixels in the binary image is obtained, that is, the total area, which is used for the subsequent calculation of the area ratio.

[0107] A contour extraction algorithm is used to extract the contours of waterlogged areas in binary images, obtaining all independent waterlogged contours. Key morphological features are then calculated for these contours, including area, perimeter, individual contour area ratio, total contour area ratio, and circularity. Area represents the number of pixels within a contour, reflecting the actual size of the waterlogged area; perimeter represents the length of the contour boundary, reflecting its complexity; individual contour area ratio measures the proportion of a single contour area within the entire image, reflecting the extent of waterlogged areas; total contour area ratio calculates the ratio of the total area of ​​all contours to the entire image; and circularity, used to assess the regularity of contour shapes, is typically calculated using the relationship between area and perimeter. A circularity closer to 1 indicates a contour shape that is closer to a perfect circle.

[0108] The specific method for calculating the above morphological features is as follows:

[0109] Contour area: obtained by the number of pixels in the water accumulation area. The calculation formula is as follows:

[0110]

[0111] Where: Area represents the area of ​​the water accumulation area; Δx represents the physical length of the pixel in the image in the horizontal direction; Δy represents the physical length of the pixel in the image in the vertical direction; contour represents the contour area.

[0112] Contour perimeter: obtained by accumulating the pixel distance on the boundary after boundary extraction. The calculation formula is as follows:

[0113]

[0114] Where: Perimeter represents the perimeter of the contour; Indicates the contour i +1 boundary pixel coordinates; Indicates the contour i The coordinates of the boundary pixels.

[0115] Single contour area ratio: obtained by dividing the single contour area by the total image area. The calculation formula is as follows:

[0116]

[0117] Where: Image Area represents the total area of ​​the image; Area Ratio represents the area ratio of a single contour;

[0118] Total contour area ratio: obtained by dividing the total contour area by the total image area. The calculation formula is as follows:

[0119]

[0120] Where: ∑Contour represents the total contour area; Total Area Ratio represents the ratio of the total contour area;

[0121] Circularity: Circularity is used to describe the roundness of the shape of the waterlogged area. The calculation formula is as follows:

[0122]

[0123] Where: Circularity represents the circularity; Area represents the area of ​​the water accumulation area; Permeter represents the perimeter of the contour.

[0124] The total area ratio of the waterlogged area is obtained based on the contour area, contour perimeter, single contour area ratio, total contour area ratio and circularity.

[0125] After the calculation is completed, it is preferred to use the OpenCV (open source computer vision library) drawing function to visualize all contours, draw each contour on the color image, and use rectangular bounding boxes and fill lines to enhance the visual effect of the waterlogged area. Through this morphological analysis, information such as contour area, contour perimeter, proportion of a single contour, proportion of the total contour area, circularity, and proportion of the total waterlogged area can be displayed to help analyze and evaluate the morphological characteristics of the waterlogged area, and finally generate the characteristic parameters of the waterlogged area. The results are shown in the following example. Figure 10 shown.

[0126] Step 5: Figure 11 As shown in the figure, after the binocular camera installed on the vehicle acquires the road image, the road image may have radial and tangential distortion, which affects the accuracy of distance measurement. Therefore, calibration and distortion correction are first required. The Zhang Zhengyou calibration method is adopted. Multiple sets of left and right images are taken from different angles within the field of view of the binocular camera through a checkerboard calibration plate. The checkerboard corner information is extracted using the corner detection function of OpenCV. The intrinsic and extrinsic parameters of the binocular camera are calculated based on the corner information, and distortion correction is performed.

[0127] The rectified image is converted to a grayscale image and histogram equalized. The SGM (Semi-Global Matching) algorithm is used to match the same points in the left and right views, calculate the current point disparity, and generate an initial disparity map.

[0128] Since disparity holes may appear in the road area in the initial disparity map due to occlusion or unsuccessful feature point matching, in order to smooth the hole area, the present invention uses a weighted least squares filter to fill the hole, ensuring that the hole area contains disparity information and obtaining a smoothed disparity map.

[0129] Based on the parameters of the road image obtained by the binocular camera, the smoothed disparity map is converted into three-dimensional coordinates in the world coordinate system. Open3D (a three-dimensional processing library) is used to generate a point cloud map from the three-dimensional coordinates and visualize it. The distance between the flooded area on the road and the binocular camera is then calculated, and the distance between the vehicle and the flooded area is obtained. Based on the location and distance information of the flooded area, the driver can respond to potential risks in a timely manner.

[0130] To detect and analyze waterlogged areas, the present invention uses the NVIDIA Jetson series as the preferred edge device platform. During the hardware and software platform construction process, the Ubuntu operating system is first installed, and a deep learning framework is configured to ensure that the waterlogging detection model and the U-Net3+ network can run on this platform. In this embodiment, the TensorRT inference engine is preferably used to accelerate the waterlogging detection model and the U-Net3+ network, increasing inference speed and ensuring real-time system performance. This effectively reduces the computational burden and ensures that the system can detect, segment, and measure waterlogged areas in real time in complex environments.

[0131] Based on the same inventive concept, an embodiment of the present invention also provides a detection, morphological analysis and ranging system for water accumulation areas. Since the principle of solving the problem by the detection, morphological analysis and ranging system for water accumulation areas is similar to the aforementioned detection, morphological analysis and ranging method for water accumulation areas, the implementation of the detection, morphological analysis and ranging system for water accumulation areas can refer to the implementation of the detection, morphological analysis and ranging method for water accumulation areas, and the repeated parts will not be repeated.

[0132] In a specific implementation, an embodiment of the present invention provides a system for detecting, analyzing, and ranging waterlogged areas, including:

[0133] An acquisition module, used for acquiring road images;

[0134] A detection module is used to detect acquired road images using a pre-trained waterlogging detection model to obtain images containing waterlogged areas. The waterlogging detection model is based on the YOLOv8 network. The construction process specifically includes: replacing the Conv module in the backbone network of the original YOLOv8 network with the SPDConv module; replacing the Upsample module in the neck network of the original YOLOv8 network with the DySample module; and replacing the original detection head in the head network of the original YOLOv8 network with the LSCD detection head.

[0135] The segmentation module is used to input the image containing the waterlogged area into the pre-trained UNet3+ network for segmentation to obtain the mask map of the waterlogged area;

[0136] An analysis module is used to perform morphological analysis on the mask image of the waterlogged area to obtain characteristic parameters of the waterlogged area;

[0137] The ranging module is used to perform ranging processing on the image containing the water area based on the parallax method to obtain the distance between the water area and the vehicle.

[0138] Correspondingly, an embodiment of the present invention also provides a device for detecting, morphologically analyzing and ranging waterlogged areas, comprising a processor and a memory, wherein the processor implements a method for detecting, morphologically analyzing and ranging waterlogged areas as provided in an embodiment of the present invention when executing a computer program stored in the memory.

[0139] For more specific details of the above method, please refer to the corresponding contents disclosed in the aforementioned embodiments, which will not be repeated here.

[0140] Accordingly, an embodiment of the present invention further provides a computer-readable storage medium for storing a computer program, wherein when the computer program is executed by a processor, the above-mentioned method for detecting, analyzing the morphology, and measuring the distance of waterlogged areas provided in an embodiment of the present invention is implemented.

[0141] The various embodiments in this specification are described in a progressive manner, with each embodiment focusing on its differences from the other embodiments. References to the same or similar portions of the various embodiments will be sufficient. The systems, devices, and storage media disclosed in the embodiments are described briefly because they correspond to the methods disclosed in the embodiments. For relevant details, refer to the method descriptions.

[0142] Professionals may further appreciate that the units and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, computer software, or a combination of the two. In order to clearly illustrate the interchangeability of hardware and software, the above description has generally described the components and steps of each example according to their functions. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professionals and technicians may use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this application.

[0143] The steps of the methods or algorithms described in conjunction with the embodiments disclosed herein may be implemented directly using hardware, a software module executed by a processor, or a combination of the two. The software module may be placed in random access memory (RAM), internal memory, read-only memory (ROM), electrically programmable ROM, electrically erasable programmable ROM, registers, a hard disk, a removable disk, a CD-ROM, or any other form of storage medium known in the art.

[0144] Finally, it should be noted that, in this document, relational terms such as first and second, etc., are used only to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any actual relationship or order between these entities or operations. Moreover, the terms "comprises," "comprising," or any other variations thereof are intended to encompass non-exclusive inclusion, such that a process, method, article, or device comprising a series of elements includes not only those elements, but also other elements not explicitly listed, or elements inherent to such process, method, article, or device. In the absence of further limitations, an element defined by the phrase "comprising a ..." does not preclude the presence of additional identical elements in the process, method, article, or device comprising the element.

[0145] The above is a detailed introduction to the methods, systems, equipment and storage media for detecting, morphologically analyzing and measuring waterlogged areas provided by the present invention. Specific examples are used herein to illustrate the principles and implementation methods of the present invention. The description of the above embodiments is only used to help understand the method of the present invention and its core idea. At the same time, for those skilled in the art, according to the ideas of the present invention, there will be changes in the specific implementation methods and application scope. In summary, the content of this specification should not be understood as a limitation on the present invention.

Claims

1. A method for detecting, analyzing, and measuring the distance between waterlogged areas, characterized in that: Applied to edge devices, it includes the following steps: Acquire road images; Use the pre-trained waterlogging detection model to detect the acquired road image and obtain an image containing waterlogged areas; Input the image containing the waterlogged area into the pre-trained UNet3+ network for segmentation to obtain the mask map of the waterlogged area; Perform morphological analysis on the mask image of the waterlogged area to obtain the characteristic parameters of the waterlogged area; The image containing the waterlogged area is processed based on the parallax method to obtain the distance between the waterlogged area and the vehicle; The water accumulation detection model is a water accumulation detection model based on the YOLOv8 network. The construction process specifically includes: replacing the Conv module in the backbone network of the original YOLOv8 network with the SPDConv module; replacing the Upsample module in the neck network of the original YOLOv8 network with the DySample module; and replacing the original detection head of the head network of the original YOLOv8 network with the LSCD detection head.

2. A method for detecting, analyzing, and measuring distance in a waterlogged area according to claim 1, characterized in that: Use the pre-trained waterlogging detection model to detect the acquired road image and obtain images containing waterlogged areas, including: The road image is resized and pixel normalized in sequence to obtain a standardized feature map; The standardized feature map is input into the backbone network for multi-stage feature extraction to obtain a multi-channel feature map; Input the multi-channel feature map into the neck network, dynamically adjust the multi-channel feature map through the DySample module to obtain a high-resolution feature map, and splice the high-resolution feature map with the multi-channel feature map to obtain a multi-scale fusion feature map; The multi-scale fusion feature map is input into the head network, and deep features are extracted from the multi-scale fusion feature map to obtain deep features. The deep features are located and classified to obtain a detection frame containing target positioning and classification information. The detection frame containing target positioning and classification information is integrated through the LSCD detection head to output an image containing the water accumulation area.

3. A method for detecting, analyzing and ranging waterlogged areas according to claim 1, characterized in that: The image containing the waterlogged area is input into the pre-trained UNet3+ network for segmentation to obtain the waterlogged area mask map, including: The UNet3+ network includes an encoder and a decoder. The image containing the water area is input into the encoder for downsampling. Each downsampling includes convolution, batch normalization, ReLU activation and maximum pooling operations to obtain a deep feature map. The decoder performs bilinear upsampling on the deep feature map to obtain a multi-scale feature map, performs fusion alignment on the multi-scale feature map to obtain a fused feature map, and performs segmentation prediction on the fused feature map to obtain an initial segmentation map; The decoder also includes a classification guidance module, which is used to perform global classification on the deep feature map to obtain a classification probability value, and fuse the classification probability value with the initial segmentation map to obtain a fused segmentation map; Construct a hybrid loss function, input the fused segmentation map into the hybrid loss function for update, and obtain the final segmentation map; The final segmentation map is morphologically processed to obtain the mask map of the water accumulation area.

4. A method for detecting, analyzing, and measuring distance in a waterlogged area according to claim 1, characterized in that: Perform morphological analysis on the mask image of the waterlogged area to obtain the characteristic parameters of the waterlogged area, including: Binarize the mask image of the waterlogged area to generate a binary image; Perform contour extraction on the binary image to obtain the characteristic parameters of the water accumulation area; The characteristic parameters of the water accumulation area include contour area, contour perimeter, single contour area ratio, contour total area ratio, circularity and total area ratio of the water accumulation area.

5. The method for detecting, analyzing and ranging waterlogged areas according to claim 1, wherein: The image containing the waterlogged area is processed based on the parallax method to obtain the distance between the waterlogged area and the vehicle, including: Performing distortion correction on the image containing the water accumulation area to obtain a corrected image; Convert the rectified image into a grayscale image and perform histogram equalization on the grayscale image; The image after histogram equalization is processed based on the SGM algorithm to obtain the initial disparity map; Filter the initial disparity map to obtain a smoothed disparity map; The smoothed disparity map is reconstructed in three dimensions to obtain a visual point cloud map, and the distance between the water accumulation area and the vehicle is obtained based on the visual point cloud map.

6. A method for detecting, analyzing, and measuring distance in a waterlogged area according to claim 1, characterized in that: It also includes the TensorRT inference engine, which is used to accelerate the water detection model and UNet3+ network.

7. A system for detecting, analyzing and ranging waterlogged areas, characterized in that: include: An acquisition module, used for acquiring road images; A detection module is used to detect acquired road images using a pre-trained waterlogging detection model to obtain images containing waterlogged areas. The waterlogging detection model is based on the YOLOv8 network. The construction process specifically includes: replacing the Conv module in the backbone network of the original YOLOv8 network with the SPDConv module; replacing the Upsample module in the neck network of the original YOLOv8 network with the DySample module; and replacing the original detection head in the head network of the original YOLOv8 network with the LSCD detection head. The segmentation module is used to input the image containing the waterlogged area into the pre-trained UNet3+ network for segmentation to obtain the mask map of the waterlogged area; An analysis module is used to perform morphological analysis on the mask image of the waterlogged area to obtain characteristic parameters of the waterlogged area; The ranging module is used to perform ranging processing on the image containing the water area based on the parallax method to obtain the distance between the water area and the vehicle.

8. A device for detecting, analyzing and measuring the distance of waterlogged areas, characterized in that: The method comprises a processor and a memory, wherein when the processor executes the computer program stored in the memory, the method for detecting, analyzing the morphology and measuring the distance of the water accumulation area as described in any one of claims 1 to 6 is implemented.

9. A computer-readable storage medium, characterized in that Used to store a computer program, wherein when the computer program is executed by a processor, it implements the method for detecting, morphologically analyzing and ranging waterlogged areas according to any one of claims 1 to 6.

Images