An urban waterlogging detection method based on a wheel segmentation model, a storage medium, an equipment and a computer program product

Abstract

The application discloses a kind of urban waterlogging detection methods based on wheel segmentation model, storage medium, equipment and computer program product, comprising: using city road video monitoring camera to collect historical car waterlogging image, each car waterlogging image is labeled vehicle type, vehicle type detection model is trained, the car waterlogging image of labeled vehicle type is segmented out car tire image, and wheel segmentation model is trained;Real-time car waterlogging image is collected, and input into the trained vehicle type detection model, the vehicle class is identified, and wheel diameter is obtained;Car waterlogging image is input into the trained wheel segmentation model and is segmented into wheel image, and wheel mask image is output, and the tire center coordinates of wheel mask image are extracted by elliptical fitting algorithm;The minimum circumscribed rectangle frame detection is carried out to wheel mask image using opencv, and finally the waterlogging depth is calculated.

Description

Technical Field

[0001] This invention relates to the field of urban waterlogging detection technology, specifically to an urban waterlogging detection method, storage medium, equipment, and computer program product based on a wheel segmentation model. Background Technology

[0002] With the intensification of global climate change, the frequency and intensity of extreme rainfall events have increased significantly in urban areas, leading to traffic disruptions in residential and public buildings and flooding, severely impacting urban safety and residents' daily lives. Simultaneously, with the accelerating pace of urbanization, inadequate urban drainage systems are causing an increase in impermeable areas, further exacerbating the risk of urban flooding. To effectively manage urban flooding emergencies and reduce losses, it is crucial to conduct urban road traffic flooding monitoring to mitigate the risk of flooding disasters and reduce potential safety hazards to urban traffic and residents.

[0003] Currently, the main methods for detecting urban flooding are manual detection and sensor detection. Manual detection involves sending personnel to measure water levels in flood-prone areas. While intuitive and flexible, this method requires significant manpower and resources, is inefficient, and poses safety risks to personnel. Sensor detection involves deploying water level sensors to collect and transmit water level data in real time. This method is convenient, fast, and highly accurate, but the instruments are expensive, require regular maintenance, are susceptible to external interference, and cannot meet the needs of urban road traffic maintenance, making it difficult to comprehensively detect urban road flooding. Therefore, a safe, efficient, convenient, and low-cost method for detecting urban flooding is particularly important. Summary of the Invention

[0004] To address the problems existing in the prior art, this invention provides a method, storage medium, device, and computer program product for detecting urban waterlogging based on a wheel segmentation model. Using urban road surveillance cameras as a medium, it enables efficient and low-cost detection of urban waterlogging.

[0005] To achieve the above technical objectives, the present invention adopts the following technical solution: a method for detecting the depth of urban floodwater accumulation based on a wheel segmentation model, specifically including the following steps:

[0006] Step 1: Collect historical images of vehicles flooded with water using urban road video surveillance cameras, and preprocess these images to expand the historical image set.

[0007] Step 2: Label each car flood image in the expanded historical car flood image set with the vehicle type, and segment the car flood images labeled with the vehicle type to extract the car tire images;

[0008] Step 3: Use images of cars with water accumulation labeled with vehicle types to train the vehicle type detection model YOLOv5-s until the maximum number of training rounds is reached, thus completing the training of the vehicle type detection model.

[0009] Step 4: Use car tire images to train the wheel segmentation model until the maximum number of training rounds is reached, thus completing the training of the wheel segmentation model.

[0010] Step 5: Real-time acquisition of images of water accumulation on vehicles, scale normalization processing, input into the trained vehicle type detection model YOLOv5-s, identify vehicle categories, and obtain wheel diameters based on vehicle categories.

[0011] Step 6: Input the scale-normalized car water accumulation image into the trained wheel segmentation model for wheel image segmentation, and output the wheel mask image.

[0012] Step 7: Extract the tire center coordinates of the wheel mask image using an ellipse fitting algorithm;

[0013] Step 8: Use OpenCV to perform minimum bounding box detection on the wheel mask image and obtain the height and coordinates of the two ends of the top edge of the minimum bounding box.

[0014] Step 9: Calculate the wheel pixel radius based on the tire center coordinates extracted in Step 7 and the coordinates of the two ends of the top edge of the minimum bounding rectangle obtained in Step 8. Combine the height of the minimum bounding rectangle and the wheel diameter obtained in Step 5 to calculate the water depth.

[0015] Furthermore, step 1 includes the following sub-steps:

[0016] Step 1.1: Perform frame segmentation processing on the historical urban road surveillance videos acquired by urban road video surveillance cameras, select historical vehicle flood images and perform scale normalization processing to obtain a set of historical vehicle flood images;

[0017] Step 1.2: Select a portion of the historical images of vehicles flooded with water and rotate them from 0° to 90°. Randomly add salt and pepper noise to a portion of the selected images of vehicles flooded with water to expand the historical image set of vehicles flooded with water.

[0018] Further, the specific process of randomly adding salt and pepper noise in step 1.2 is as follows: set the noise ratio rate to 0.01, for any selected historical car flood image, traverse each pixel in the historical car flood image, generate a random number rdn between 0 and 1, if rdn < rate, set the pixel to 0; if rdn > 1 - rate, set the prime pixel to 255; if rate ≤ rdn ≤ 1 - rate, the pixel remains unchanged.

[0019] Furthermore, the wheel segmentation model used in step 4 is based on the existing YOLOv5-seg network. The CBS module in the backbone layer of the YOLOv5-seg network is replaced with the GhostConv module, which is composed of CBS modules and Ghost Modules connected in sequence. The residual module of the C3 module in the backbone layer of the YOLOv5-seg network is replaced with the first GhostBottleneck and the second GhostBottleneck, and the C3 module is changed into the C3Ghost module.

[0020] Furthermore, the process of generating feature maps by the GhostConv module is as follows: convolution is performed through the CBS module to obtain a small number of feature maps, then identity mapping is performed through Ghost Modules, and the obtained small number of feature maps are convolved layer by layer to obtain more feature maps, and the small number of feature maps are fused with the more feature maps for output.

[0021] Furthermore, the output of the first GhostBottleneck is connected to the input of the second GhostBottleneck. The first GhostBottleneck serves as an extension layer to increase the number of channels, and its input and output are connected via a shortcat layer. The first GhostBottleneck is composed of two stacked Ghostmodules connected together. The second GhostBottleneck is used to reduce the number of channels, and its input and output are connected via a shortcat layer. The second GhostBottleneck is composed of two stacked Ghostmodules connected together, and a depthwise convolutional layer with a stride of 2 is inserted between the two stacked Ghostmodules.

[0022] Furthermore, in step 9, the wheel pixel radius H r The calculation process is as follows: Where k represents the slope of the line, c and d represent the x and y coordinates of one endpoint of the top edge, respectively; e and f represent the x and y coordinates of the other endpoint of the top edge, respectively; b represents the intercept of the line; b = d - kc.

[0023] The water depth H d The calculation process is as follows: Where H represents the wheel diameter and h represents the height of the minimum circumscribed rectangle.

[0024] Furthermore, the present invention also provides a computer-readable storage medium storing a computer program that enables a computer to execute the urban flooding depth detection method based on a wheel segmentation model.

[0025] Furthermore, the present invention also provides an electronic device, including: a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the computer program, it implements the aforementioned method for detecting urban flooding depth based on a wheel segmentation model.

[0026] Furthermore, the present invention also provides a computer program product, including a computer program that, when executed by a processor, implements the urban flooding depth detection method based on the wheel segmentation model.

[0027] Compared with existing technologies, this invention has the following advantages: The urban flooding depth detection method based on a wheel segmentation model combines CBS and GhostConv, reducing the convolutional computation required for feature extraction. Furthermore, GhostBottleneck replaces the Bottleneck residual module in the C3 module of the existing YOLOv5-seg network, first reducing the number of channels, then increasing them at the end of feature extraction. This allows the wheel segmentation model to maintain high accuracy while reducing the number of parameters and computational load, thus reducing computational load and accelerating inference speed on resource-constrained hardware. Simultaneously, this invention uses the most common urban road vehicle—the wheel—as a medium to calculate urban flooding depth. Compared to the high cost and limited deployment of sensors, this invention can be widely and cost-effectively deployed using urban road surveillance cameras, significantly saving hardware resources and equipment deployment costs. Attached Figure Description

[0028] Figure 1 This is a flowchart of the urban flooding depth detection method based on a wheel segmentation model according to the present invention;

[0029] Figure 2 This is a schematic diagram of the network connection of the wheel segmentation model in this invention;

[0030] Figure 3This is a schematic diagram of the structure of the first GhostBottleneck and the second GhostBottleneck in this invention;

[0031] Figure 4 This is a schematic diagram illustrating the calculation of water depth in this invention. Detailed Implementation

[0032] The technical solution of the present invention will be further explained and described below with reference to the accompanying drawings.

[0033] like Figure 1 This is a flowchart of the urban flooding depth detection method based on a wheel segmentation model according to the present invention. The urban flooding depth detection method specifically includes the following steps:

[0034] Step 1: Collect historical images of vehicles flooded with water using urban road video surveillance cameras, and preprocess these images to expand the historical image set; this includes the following sub-steps:

[0035] Step 1.1: Perform frame segmentation processing on the historical urban road surveillance videos acquired by urban road video surveillance cameras, select historical vehicle flood images and perform scale normalization processing to obtain a set of historical vehicle flood images;

[0036] Step 1.2: Select 10% of the historical car flood images from the historical car flood image set and rotate them from 0° to 90°. Randomly add salt and pepper noise to another 10% of the car flood images from the historical car flood image set to expand the historical car flood image set. This will enrich the car flood image set and improve the generalization ability of the following vehicle type detection model YOLOv5-s and wheel segmentation model.

[0037] The specific process of randomly adding salt and pepper noise in this invention is as follows: Set the noise ratio rate to 0.01. For any selected historical image of a car flooded with water, iterate through each pixel in the historical image of a car flooded with water and generate a random number rdn between 0 and 1. If rdn < rate, set the pixel to 0; if rdn > 1 - rate, set the prime number pixel to 255; if rate ≤ rdn ≤ 1 - rate, the pixel remains unchanged.

[0038] Step 2: Label each car flood image in the expanded historical car flood image set with the vehicle type, which is divided into car, SUV and bus, and then segment the car flood images labeled with the vehicle type to extract the car tire images.

[0039] Step 3: Train the YOLOv5-s vehicle type detection model using images of vehicles with labeled vehicle types, until the maximum number of training epochs is reached, completing the training of the vehicle type detection model. The relevant training parameters for the YOLOv5-s vehicle type detection model are set as follows: momentum is 0.937, weight_decay is 0.0005, lr0 is 0.01, lrf is 0.01, batch-size is 16, and epochs are 100.

[0040] Step 4: Train the wheel segmentation model using car tire images until the maximum number of training epochs is reached, thus completing the training of the wheel segmentation model. The training parameters for the wheel segmentation model are set as follows: momentum is set to 0.937, weight_decay is set to 0.0005, lr0 is set to 0.01, lrf is set to 0.01, batch-size is set to 16, and epochs is set to 100.

[0041] like Figure 2 The wheel segmentation model used in this invention is based on the existing YOLOv5-seg network. The CBS module in the backbone layer of the YOLOv5-seg network is replaced with a GhostConv module, which consists of a CBS module and Ghost Modules connected sequentially. GhostConv not only retains the function of extracting image features but also reduces convolutional computation operations for feature extraction, lowering the number of parameters and computational cost, thus reducing the network size. The GhostConv module generates feature maps as follows: convolution is performed through the CBS module to obtain a small number of feature maps; then, identity mapping is performed through Ghost Modules, and the resulting small number of feature maps are convolved layer by layer to obtain more feature maps; finally, the small number of feature maps are fused with the more feature maps for output. The residual module of the C3 module in the backbone layer of the YOLOv5-seg network is replaced with the first GhostBottleneck and the second GhostBottleneck, with strides of 1 and 2 respectively, transforming the C3 module into a C3Ghost module. This not only captures feature information but also further reduces the number of parameters and computational cost, thereby reducing memory usage and accelerating inference speed. Specifically, the output of the first GhostBottleneck is connected to the input of the second GhostBottleneck. The first GhostBottleneck serves as an extension layer to increase the number of channels, and its input and output are connected through a shortcat layer. Figure 3The first GhostBottleneck consists of two stacked Ghost modules connected together. The second GhostBottleneck reduces the number of channels to match the path of the shortcut layer, and its input and output are connected through the shortcut layer. The second GhostBottleneck also consists of two stacked Ghost modules connected together, with a depthwise convolutional layer with a stride of 2 inserted between them. The first Ghost module uses batch normalization and ReLU non-linear activation, which effectively alleviates the gradient vanishing problem in the C3Ghost module and increases its sparsity, reducing the interdependence between parameters and avoiding overfitting. Since the ReLU non-linear activation function stops activating neurons when the input is negative, the second Ghost module does not use ReLU non-linear activation, effectively mitigating the neuron failure problem.

[0042] Step 5: Real-time acquisition of images of water accumulation on vehicles, scale normalization processing, and input into the trained vehicle type detection model YOLOv5-s to identify vehicle categories and obtain wheel diameters based on vehicle categories; for vehicle types "car, SUV, bus", the tire specifications used are 205 / 55R16, 225 / 60R18 and 275 / 70R22.5 respectively, with corresponding diameters of 63.19cm, 72.72cm and 95.65cm respectively.

[0043] Step 6: Input the scale-normalized car water accumulation image into the trained wheel segmentation model for wheel image segmentation, and output the wheel mask image.

[0044] Step 7: Extract the tire center coordinates (x0, y0) of the wheel mask image using an ellipse fitting algorithm;

[0045] Step 8: Use OpenCV to perform minimum bounding box detection on the wheel mask image, and obtain the height and coordinates of the two endpoints of the top edge of the minimum bounding box; specifically,

[0046] Step 8.1: Use the Canny edge detector to detect edges in the wheel mask image;

[0047] Step 8.2: Use the findContours function in OpenCV to find the contours in the wheel mask image;

[0048] Step 8.3: Use the minAreaRect function in OpenCV to calculate the minimum bounding rectangle, obtain the length and height of the minimum bounding rectangle, and output the coordinates of the two points on the top edge of the minimum bounding rectangle, J(c, d) and Q(e, f).

[0049] Step 9: Based on the tire center coordinates extracted in Step 7 and the coordinates of the two ends of the top edge of the minimum bounding rectangle obtained in Step 8, calculate the wheel pixel radius. Combined with the height of the minimum bounding rectangle and the wheel diameter obtained in Step 5, calculate the water depth. By using machine vision with the wheel as a reference to construct a mathematical model for water depth measurement, the height of the minimum bounding rectangle and the wheel pixel diameter can be quickly extracted as water depth measurement parameters, which is convenient for subsequent calculations. It is also low-cost, fast-paced, and has high detection accuracy, which can effectively alleviate the disadvantages of high sensor measurement costs that are not conducive to large-scale deployment.

[0050] In this invention, the wheel pixel radius H r The calculation process is as follows: Where k represents the slope of the line, c and d represent the x and y coordinates of one endpoint of the top edge, respectively; e and f represent the x and y coordinates of the other endpoint of the top edge, respectively; b represents the intercept of the line; b = d - kc.

[0051] like Figure 4 Water depth H d The calculation process is as follows: Where H represents the wheel diameter and h represents the height of the minimum circumscribed rectangle.

[0052] The urban flooding detection method of this invention was tested in four flooding depth environments: 5cm, 8cm, 15cm, and 32cm. For each depth, 25, 30, 27, and 24 images were collected, totaling 106 test data points. The experimental data are shown in Table 1. At a depth of 5cm, the average detection error of this invention was 2.70cm, with a standard deviation of 2.43cm; at a depth of 8cm, the average detection error was 2.38cm, with a standard deviation of 2.66cm; at a depth of 15cm, the average detection error was 2.35cm, with a standard deviation of 2.84cm; and at a depth of 32cm, the average detection error was 2.38cm, with a standard deviation of 2.85cm. The test results show that the detection error of this invention is less than 5cm in 92.5% of the results, providing relatively accurate detection of urban flooding depth. Meanwhile, the wheel segmentation model based on the improved YOLOv5-seg of this invention can significantly reduce the computational load and the number of parameters. Compared with the original YOLOv5-seg, GFLOPs are reduced from 25.7 to 20.5, a reduction of 20.2%, and the number of parameters is reduced from 7.4 × 10⁻⁶.6 Reduced to 5.4 × 10 6 This represents a 27% reduction.

[0053] Table 1 Test Results

[0054]

[0055]

[0056] In one technical solution of the present invention, a computer-readable storage medium is also provided, storing a computer program that enables a computer to execute the urban flooding depth detection method based on a wheel segmentation model.

[0057] In one technical solution of the present invention, an electronic device is also provided, including: a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the computer program, it implements the urban flooding depth detection method based on the wheel segmentation model.

[0058] In one technical solution of the present invention, a computer program product is also provided, including a computer program, which, when executed by a processor, implements the urban waterlogging depth detection method based on the wheel segmentation model.

[0059] In the embodiments disclosed in this application, a computer storage medium may be a tangible medium that may contain or store programs for use by or in conjunction with an instruction execution system, apparatus, or device. The computer storage medium may include, but is not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, apparatus, or devices, or any suitable combination of the foregoing. More specific examples of computer storage media include electrical connections based on one or more wires, portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fibers, portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination of the foregoing.

[0060] Those skilled in the art will recognize that the units and algorithm steps of the various examples described in conjunction with the embodiments disclosed in this application can be implemented in electronic hardware or a combination of computer software and electronic hardware. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this application.

[0061] The above are merely preferred embodiments of the present invention. The scope of protection of the present invention is not limited to the above embodiments. All technical solutions falling within the scope of the present invention's concept are within the scope of protection of the present invention. It should be noted that for those skilled in the art, any improvements and modifications made without departing from the principles of the present invention should be considered within the scope of protection of the present invention.

Claims

1. A method for detecting the depth of urban floodwater accumulation based on a wheel segmentation model, characterized in that, Specifically, the steps include the following: Step 1: Collect historical images of vehicles flooded with water using urban road video surveillance cameras, and preprocess these images to expand the historical image set. Step 2: Label each car flood image in the expanded historical car flood image set with the vehicle type, and segment the car flood images labeled with the vehicle type to extract the car tire images; Step 3: Use images of cars with water accumulation labeled with vehicle types to train the vehicle type detection model YOLOv5-s until the maximum number of training rounds is reached, thus completing the training of the vehicle type detection model. Step 4: Use car tire images to train the wheel segmentation model until the maximum number of training rounds is reached, thus completing the training of the wheel segmentation model. The wheel segmentation model used is based on the existing YOLOv5-seg network. The CBS module in the backbone layer of the YOLOv5-seg network is replaced with the GhostConv module, which is composed of CBS module and Ghost Modules connected in sequence. The residual module of C3 module in the backbone layer of the YOLOv5-seg network is replaced with the first GhostBottleneck and the second GhostBottleneck, and the C3 module is changed into the C3Ghost module. The GhostConv module generates feature maps in the following ways: it performs convolution through the CBS module to obtain a small number of feature maps, then performs identity mapping through Ghost Modules, performs layer-by-layer convolution on the small number of feature maps to obtain more feature maps, and finally fuses the small number of feature maps with the more feature maps for output. The output of the first GhostBottleneck is connected to the input of the second GhostBottleneck. The first GhostBottleneck serves as an extension layer to increase the number of channels. The input and output of the first GhostBottleneck are connected through a shortcat layer. The first GhostBottleneck consists of two stacked Ghost modules connected together. The second GhostBottleneck is used to reduce the number of channels. The input and output of the second GhostBottleneck are connected through a shortcat layer. The second GhostBottleneck consists of two stacked Ghost modules connected together, and a depthwise convolutional layer with a stride of 2 is inserted between the two stacked Ghost modules. Step 5: Real-time acquisition of images of water accumulation on vehicles, scale normalization processing, input into the trained vehicle type detection model YOLOv5-s, identify vehicle categories, and obtain wheel diameters based on vehicle categories. Step 6: Input the scale-normalized car water accumulation image into the trained wheel segmentation model for wheel image segmentation, and output the wheel mask image. Step 7: Extract the tire center coordinates of the wheel mask image using an ellipse fitting algorithm; Step 8: Use OpenCV to perform minimum bounding box detection on the wheel mask image and obtain the height and coordinates of the two ends of the top edge of the minimum bounding box. Step 9: Calculate the wheel pixel radius based on the tire center coordinates extracted in Step 7 and the coordinates of the two ends of the top edge of the minimum bounding rectangle obtained in Step 8. Combine the height of the minimum bounding rectangle and the wheel diameter obtained in Step 5 to calculate the water depth. The wheel pixel radius H r The calculation process is as follows: ,in, k Represents the slope of a straight line. , c , d These represent the x and y coordinates of one endpoint of the top edge, respectively. e , f These represent the x and y coordinates of the other endpoint of the top edge, respectively. b Represents the intercept of the line. ; The depth of the accumulated water H d The calculation process is as follows: ,in, H Indicates the diameter of the wheel. h This represents the height of the smallest bounding rectangle.

2. The method for detecting urban floodwater depth based on a wheel segmentation model according to claim 1, characterized in that, Step 1 includes the following sub-steps: Step 1.1: Perform frame segmentation processing on the historical urban road surveillance videos acquired by urban road video surveillance cameras, select historical vehicle flood images and perform scale normalization processing to obtain a set of historical vehicle flood images; Step 1.2: Select a portion of the historical car flood images from the historical car flood image set and rotate them from 0° to 90°. Randomly add salt and pepper noise to a portion of the historical car flood images from the historical car flood image set to expand the historical car flood image set.

3. The method for detecting urban floodwater depth based on a wheel segmentation model according to claim 2, characterized in that, The specific process of randomly adding salt and pepper noise in step 1.2 is as follows: Set the noise ratio rate to 0.

01. For any selected historical image of a car flooded with water, iterate through each pixel in the historical image of a car flooded with water and generate a random number rdn between 0 and 1. If rdn < rate, set the pixel to 0; if rdn > 1 - rate, set the pixel to 255; if rate ≤ rdn ≤ 1 - rate, the pixel remains unchanged.

4. A computer-readable storage medium storing a computer program, characterized in that, The computer program causes the computer to execute the urban flooding depth detection method based on the wheel segmentation model as described in any one of claims 1-3.

5. An electronic device, characterized in that, include: The device includes a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the computer program, it implements the urban flooding depth detection method based on a wheel segmentation model as described in any one of claims 1-3.

6. A computer program product, comprising a computer program, characterized in that, When the computer program is executed by the processor, it implements the urban flooding depth detection method based on the wheel segmentation model as described in any one of claims 1-3.

Images