A method and system for identifying flooded urban roads and tunnel entrances and affected individuals

Abstract

The present invention relates to a method and system for identifying flooded urban roads and tunnel entrances and affected individuals. The method comprises: Step S1: collecting videos of urban road flooding conditions and normal conditions and preprocessing them to obtain Sample Set 1, as well as preprocessing videos of vehicles and pedestrians in both conditions to obtain Sample Set 2; Step S2: inputting Sample Set 1 into an urban road flooding monitoring model for training to detect whether road flooding has occurred; Step S3: inputting Sample Set 2 into an affected individual monitoring model for training to identify whether there are affected vehicles or pedestrians; Step S4: cascading the trained Urban Road Flooding Monitoring Model and the trained Impacted Individual Monitoring Model. When the Urban Road Flooding Monitoring Model predicts that road flooding has occurred, the Impacted Individual Monitoring Model is activated to detect whether there are affected vehicles or pedestrians. The method provided by the present invention can monitor urban flooding and affected individuals in real time and issue early warnings.

Description

Technical Field

[0001] The present invention relates to the fields of information technology and natural disaster monitoring technology, and in particular to a method and system for identifying flooded urban roads and tunnel entrances and affected individuals. Background Art

[0002] In recent years, urban waterlogging, caused by heavy or continuous rainfall, poorly designed urban drainage systems, and urban topography, has become increasingly prevalent. In particular, flooding struck Henan Province again in 2021, resulting in 302 deaths and 50 missing persons. Over 200 vehicles were trapped in the Zhengzhou Jingguang Tunnel, leaving several people missing. Although the tunnel was equipped with specialized drainage equipment, a large amount of silt blocked the drainage pipes, resulting in poor drainage. Silt accumulation at the tunnel bottom, coupled with the accumulation of vehicles wading through the water, complicated rescue efforts. Separately, in the same year, two drivers were trapped under the Hanhe Road railway bridge in Beijing and tragically died after efforts to rescue them proved ineffective. Urban waterlogging is widespread in my country, occurring not only in coastal cities but now also in inland cities. It poses a significant threat to the lives and property of citizens. Urban waterlogging places enormous pressure on urban flood control and drainage efforts. The resulting traffic disruptions have a significant impact on economic development and people's livelihoods. Waterlogging in tunnels, in particular, poses a significant threat to the safety of citizens.

[0003] Several technologies are currently being used to monitor urban waterlogging. Traditional visual inspection methods, however, are insufficient due to significant limitations in cost, accuracy, and timeliness. Another common approach involves using equipment to set safety lines to monitor water levels within cities or tunnels. However, this approach can lead to potential safety hazards, such as alarms not sounding if water levels fail to reach the safety line. Furthermore, if a traffic jam occurs in a tunnel, vehicles outside the tunnel may not be notified of the situation and may continue to enter the tunnel, failing to mitigate safety hazards and congestion, as the water level does not reach the safety line, alarms may not sound, and gates may not close. Summary of the Invention

[0004] In order to solve the above technical problems, the present invention provides a method and system for identifying flooded urban roads and tunnel entrances and affected individuals.

[0005] The technical solution of the present invention is: a method for identifying flooded urban roads and tunnel entrances and affected individuals, comprising:

[0006] Step S1: collecting surveillance images or videos of urban roads under flooded conditions and normal conditions, as well as surveillance images or videos of urban roads under flooded conditions and normal conditions containing vehicles and pedestrians; preprocessing the surveillance images or videos of urban roads under flooded conditions and normal conditions to obtain sample set 1, and preprocessing the images of urban roads under flooded conditions and normal conditions containing vehicles and pedestrians to obtain sample set 2;

[0007] Step S2: inputting the sample set 1 into the urban road flooding monitoring model for training, and obtaining a judgment result of whether road flooding occurs by performing target detection on the input video by the urban road flooding monitoring model;

[0008] Step S3: inputting the sample set 2 into the disaster-affected individual monitoring model for training, and the disaster-affected individual monitoring model performs disaster-affected individual target recognition on the input video to obtain a judgment result on whether the disaster-affected vehicle or pedestrian is recognized;

[0009] Step S4: The trained urban road flooding monitoring model and the trained disaster-affected individual monitoring model are cascaded to form a complete urban road flooding and disaster-affected individual identification model. When the urban road flooding monitoring model predicts that road flooding has occurred, the disaster-affected individual monitoring model is activated to monitor whether there are vehicles or pedestrians.

[0010] Compared with the prior art, the present invention has the following advantages:

[0011] The present invention discloses a method for identifying flooded urban roads and tunnel entrances and affected individuals. By acquiring real-time monitoring images and fusing features using an improved MobileNet model and an LBP algorithm, an urban road flooding monitoring model is constructed to monitor roads. LBP (local binary pattern) is a relatively simple and effective texture feature extraction method that is insensitive to lighting and has good robustness for road monitoring results in the evening (such as in low-light scenarios). In addition, the light in tunnels is generally weak, and the use of the improved model can achieve a higher accuracy rate. The activation function of the original Mobilenet convolutional layer was changed to the RReLU activation function, which overcame the neuron death phenomenon of the original activation function, and the calculation speed was relatively fast, which avoided information loss to a certain extent and made the training features more complete. In the monitoring process of affected individuals, the YOLO algorithm with GIoU replacing the original IoU activation function was used, so that the feature image size remained unchanged after passing through the activation function, and the problem of target overlap that could not be optimized was overcome. In the monitoring process, the two models were cascaded to make the overall application logic scenario more complete and practical. If there was no flooding, the monitoring would continue until flooding was detected. Then the affected individual monitoring model would be activated to monitor whether there were pedestrians or vehicles entering in time. Once pedestrians or vehicles were found to enter, an early warning would be issued in time to remind pedestrians not to enter, and the relevant departments would be informed in time so that the regulatory authorities could persuade pedestrians or vehicles to return in time. BRIEF DESCRIPTION OF THE DRAWINGS

[0012] Figure 1 This is a flow chart of a method for identifying flooded urban roads and tunnel entrances and affected individuals in an embodiment of the present invention;

[0013] Figure 2 Schematic diagram of the structure of an urban road flooding monitoring model in an embodiment of the present invention;

[0014] Figure 3 The figure is a structural block diagram of a method system for identifying flooded urban roads and tunnel entrances and affected individuals according to an embodiment of the present invention. DETAILED DESCRIPTION

[0015] The present invention provides a method for identifying flooded urban roads and tunnel entrances and affected individuals. Once urban flooding is detected, the method begins to monitor the conditions of pedestrians and vehicles on roads and tunnel entrances in real time, and issues early warnings in a timely manner to reduce the threat posed by urban flooding disasters to the personal safety of citizens and minimize the disaster.

[0016] In order to make the purpose, technical solutions and advantages of the present invention more clear, the present invention is further described in detail below through specific implementation and in conjunction with the accompanying drawings.

[0017] Example 1

[0018] like Figure 1 As shown, an embodiment of the present invention provides a method for identifying flooded urban roads and tunnel entrances and affected individuals, including the following steps:

[0019] Step S1: collecting surveillance images or videos of urban roads under flooded conditions and normal conditions, as well as surveillance images or videos of urban roads under flooded conditions and normal conditions containing vehicles and pedestrians; preprocessing the surveillance images or videos of urban roads under flooded conditions and normal conditions to obtain sample set 1, and preprocessing the images of urban roads under flooded conditions and normal conditions containing vehicles and pedestrians to obtain sample set 2;

[0020] Step S2: Inputting sample set 1 into the urban road flooding monitoring model for training, obtaining a result of determining whether road flooding occurs by performing target detection on the input video;

[0021] Step S3: Input sample set 2 into the disaster-affected individual monitoring model for training, and obtain the disaster-affected individual monitoring model to identify the disaster-affected individual target in the input video to obtain a judgment result on whether the disaster-affected vehicle or pedestrian is identified;

[0022] Step S4: The trained urban road flooding monitoring model and the trained disaster-affected individual monitoring model are cascaded to form a complete urban road flooding and disaster-affected individual identification model. When the urban road flooding monitoring model predicts that road flooding has occurred, the disaster-affected individual monitoring model is activated to monitor whether there are vehicles and pedestrians.

[0023] In one embodiment, step S1 above: collecting surveillance images or videos of urban roads under flooded conditions and normal conditions, and surveillance images or videos of urban roads under flooded conditions and normal conditions containing vehicles and pedestrians; preprocessing the surveillance images or videos of urban roads under flooded conditions and normal conditions to obtain sample set 1, and preprocessing the images of urban roads under flooded conditions and normal conditions containing vehicles and pedestrians to obtain sample set 2, specifically includes:

[0024] Step S11: acquiring surveillance images or videos of urban road waterlogging and normal conditions respectively, performing frame acquisition operations at a preset interval 1, and obtaining images of urban road waterlogging and images of urban road normal conditions respectively as sample set 1;

[0025] Surveillance images can be obtained through various platforms such as the Internet or public security systems; surveillance images or videos of urban road waterlogging to be identified and under normal conditions are framed at regular intervals to obtain images that can be used for urban waterlogging monitoring to form a sample set 1. In the embodiment of the present invention, interval 1 is set to save one frame of image every 1 second; the LBP value image of each frame is calculated and saved in subsequent steps, and an adjacent frame compensation strategy is adopted to stabilize the image and reduce the impact of light on the grayscale value.

[0026] Sample set 1 is divided into two categories of images, specifically flooding and normal. The data set contains a total of 12,500 color images, all in jpg format. Among them, there are 10,000 training images, of which flooding contains 2,000 images and normal contains 8,000 images. There are 2,000 evaluation images, of which the flooding category contains 400 images and the normal category contains 1,600 images. Sample set 1 is divided into 100 training batches and 1 test batch. The test batch selects images in specific scenes for testing, which can not only test the test accuracy, but also lay the foundation for monitoring. The sample set 1 constructed by the embodiment of the present invention is generally small and has a fast training speed. It is suitable for testing new improved algorithms and testing new image classification technologies. At the same time, the monitoring accuracy in specific scenes can also be significantly improved;

[0027] Step S12: The surveillance images or videos of urban road waterlogging and vehicles and pedestrians under normal conditions are framed at a preset interval 2. In the embodiment of the present invention, interval 2 is set to save a frame every 3 seconds; this is used to increase the difference between the images and make the features between the images more obvious; the categories and bounding boxes of the vehicle and pedestrian images are marked as sample set 2.

[0028] At the same time, images of urban roads with water accumulation and vehicles and pedestrians under normal conditions are preprocessed to obtain sample set 2; vehicle and pedestrian images are specifically classified and labeled using labelimg; an xml file in VOC format is created to annotate the true categories and bounding boxes of vehicles and pedestrians in the image, and the xml file required by the YOLO neural network is automatically generated; the xml file in VOC format is converted into a txt file required by the YOLO neural network; the images in the training set are preprocessed and resized.

[0029] like Figure 2 As shown, in one embodiment, the above step S2: inputting sample set 1 into the urban road flooding monitoring model for training, performing target detection on the input video to obtain a judgment result on whether road flooding occurs, specifically includes:

[0030] This embodiment of the present invention crops the input image into a target image of 224×224 size and then feeds it into the urban road flooding monitoring model. This embodiment of the present invention uses an improved MobileNet model and LBP algorithm to fuse features and perform classification.

[0031] The network structure of the urban road flooding monitoring model is divided into three parts:

[0032] Step S21: The image in sample set 1 is used as the input image and input into the urban road flooding monitoring model constructed by fusing the improved MobileNet model and the LBP algorithm. First, the input image passes through a 3x3 convolutional layer to extract the input image feature x. At the same time, the LBP algorithm is used to extract the input image texture feature y. Feature x and feature y are fused through the Add module. The Add module uses a parallel strategy during feature fusion to combine the two feature vectors into a complex vector. For input features x and y, z = x + iy, where i is an imaginary unit, to obtain feature z.

[0033] The specific steps of using the LBP algorithm to extract the texture feature y of the input image are as follows:

[0034] (1) First, the detection window is divided into small area cells of 16×16;

[0035] (2) For a pixel in each cell, the grayscale values ​​of the eight adjacent pixels are compared with it. If the surrounding pixel values ​​are greater than the central pixel value, the position of the pixel is marked as 1, otherwise it is 0. The points in the neighborhood are compared to generate a binary number, that is, the LBP value of the central pixel of the window is obtained.

[0036] (3) Then calculate the histogram of each cell, that is, the frequency of each number (decimal LBP value); then normalize the histogram;

[0037] (4) Finally, the obtained histograms are connected to form a feature vector, that is, the LBP texture feature vector of the input image;

[0038] The LBP value is calculated as follows:

[0039]

[0040]

[0041] In formula (1), let the position coordinates of the central pixel be (x, y), the number of pixels in the area with a radius of c be n, and these pixels are the neighborhood pixels of the central pixel, g n is the gray value of the neighborhood pixel, g b is the gray value of the center pixel;

[0042] Formula (2) performs binarization processing, compares the neighborhood pixels with the central pixel, and finally encodes the binary string into a decimal number;

[0043] S22: Use multiple convolutional layers including depthwise convolution and point-to-point convolution to extract feature z;

[0044] In this step, n convolutional layers are stacked. Each convolutional layer includes Depthwise Convolution and Pointwise Convolution. Depthwise Convolution divides the feature z into three groups, performs 3×3 convolution on each group respectively, and collects the spatial features of each channel. Pointwise Convolution performs 1×1 convolution on the input image and collects the features of each point.

[0045] In the embodiment of the present invention, both Depthwise Convolution and Pointwise Convolution use the RReLU activation function, as shown in formula (3):

[0046]

[0047] Where x is the value of the RReLU function on the x-axis, a~U(l,u) means that a follows a uniform distribution, l < u, and u ∈ [0,1);

[0048] Using the RReLU activation function will not cause the phenomenon of neuron death, and the calculation speed is relatively fast, which can avoid the loss of information to a certain extent and make the training features more complete;

[0049] S23: Finally, after two 1x1 convolutional layers, instead of the fully connected layer, the category is output, that is, it is judged whether the input image has a road flood.

[0050] The activation function used in the output layer of the urban road flood monitoring model is the softmax activation function, as shown in formula (4):

[0051]

[0052] Where a k is the input signal, y k is the output y of calculating the kth neuron k , and n represents that there are n neurons in the output layer.

[0053] In the embodiment of the present invention, LBP is used to improve the existing MobileNet model to construct an urban road flood monitoring model to monitor roads. LBP (Local Binary Pattern) is a relatively simple and effective texture feature extraction method, which is insensitive to light, especially has good robustness to the road monitoring results in the evening (scenes with weak light, etc.); in addition, the light in general tunnels is weak, and using the improved model can obtain a higher accuracy.

[0054] In one embodiment, the above step S3: inputting the sample set 2 into the disaster-affected individual monitoring model for training, performing disaster-affected individual target recognition on the input video to obtain a judgment result on whether the disaster-affected vehicle or pedestrian is recognized, specifically includes:

[0055] Based on the YOLOv5 model, a disaster-affected individual monitoring model is constructed. Sample set 2 is input into the disaster-affected individual monitoring model for training to obtain the pedestrian or vehicle prediction box in the input image. At the same time, the loss function GIOU is constructed. Loss , as shown in formula (5):

[0056]

[0057] Among them, A is the predicted box, B is the real box, C represents the minimum convex closed box containing A and B, and IOU is the overlap rate between the predicted box and the real box.

[0058] The image feature extraction module in the disaster-affected individual monitoring model uses the darknet-53 network to extract features from vehicle and pedestrian images; and uses GIoU as the loss function for bounding box regression. The GIoU method overcomes the shortcomings of IoU while fully utilizing its advantages. Loss Curve to determine whether the YOLO v5 neural network has converged; if it has converged, stop training; if not, continue training.

[0059] In one embodiment, step S4 above: cascading the trained urban road flooding monitoring model and the trained disaster-affected individual monitoring model to form a complete urban road flooding and disaster-affected individual identification model. When the urban road flooding monitoring model predicts that road flooding has occurred, the disaster-affected individual monitoring model is activated to monitor whether there are affected vehicles or pedestrians, specifically including:

[0060] The trained urban road flooding monitoring model and the trained disaster-affected individual monitoring model are cascaded to form a complete urban road flooding and disaster-affected individual identification model.

[0061] First, a trained urban road flooding monitoring model is used to monitor urban roads and tunnel entrances in real time. If waterlogging is detected, an alert is issued to the monitoring display terminal, allowing flood control personnel to quickly take appropriate measures. Simultaneously, a trained individual disaster monitoring model is triggered to enable real-time monitoring of pedestrians and vehicles. If waterlogging is detected and pedestrians or vehicles are passing through, an alarm is issued, warning pedestrians to stay away and notifying relevant departments, allowing regulatory authorities to promptly persuade pedestrians or vehicles to return.

[0062] The present invention discloses a method for identifying flooding and affected individuals on urban roads and tunnel entrances. The method obtains real-time monitoring images and uses an urban road flooding monitoring model to monitor the road. If no flooding occurs, the monitoring continues until flooding is detected. Then, the affected individual monitoring model is activated to timely monitor whether there are pedestrians or vehicles entering. Once pedestrians or vehicles are found entering, an early warning is issued in time to remind pedestrians not to enter, and the relevant departments are informed in time so that the regulatory authorities can persuade the pedestrians or vehicles to return in time.

[0063] Example 2

[0064] like Figure 3 As shown, an embodiment of the present invention provides a method and system for identifying flooded urban roads and tunnel entrances and affected individuals, including the following modules:

[0065] Sample collection and preprocessing module 1 is used to collect surveillance images or videos of urban roads under water accumulation conditions and normal conditions, as well as surveillance images or videos of urban roads under water accumulation conditions and normal conditions containing vehicles and pedestrians; preprocess the surveillance images or videos of urban roads under water accumulation conditions and normal conditions to obtain sample set 1, and preprocess the images of urban roads under water accumulation conditions and normal conditions containing vehicles and pedestrians to obtain sample set 2;

[0066] Urban road flooding monitoring model training module 2 is used to input sample set 1 into the urban road flooding monitoring model for training, perform target detection on the input video, and obtain a judgment result on whether road flooding occurs;

[0067] The disaster-affected individual monitoring model training module 3 is used to input the sample set 2 into the disaster-affected individual monitoring model for training, and perform disaster-affected individual target recognition on the input video to obtain a judgment result on whether the disaster-affected vehicle or pedestrian is recognized;

[0068] The road flooding and affected individual identification module 4 is used to cascade the trained urban road flooding monitoring model and the trained affected individual monitoring model to form a complete urban road flooding and affected individual identification model. When the urban road flooding monitoring model predicts road flooding, the affected individual monitoring model is activated to monitor for vehicles and pedestrians. If a pedestrian or vehicle is detected entering the area, an early warning is issued, prompting pedestrians to be kept away, and the relevant authorities are promptly notified, allowing regulatory authorities to promptly persuade the pedestrian or vehicle to return.

[0069] The above embodiments are provided for the purpose of describing the present invention only and are not intended to limit the scope of the present invention. The scope of the present invention is defined by the appended claims. Various equivalent substitutions and modifications made without departing from the spirit and principles of the present invention are intended to be within the scope of the present invention.

Claims

1. A method for identifying flooded and affected individuals on urban roads and tunnel entrances, characterized by: include: Step S1: collecting surveillance images or videos of urban roads under flooded conditions and normal conditions, as well as surveillance images or videos of urban roads under flooded conditions and normal conditions containing vehicles and pedestrians; preprocessing the surveillance images or videos of urban roads under flooded conditions and normal conditions to obtain sample set 1, and preprocessing the images of urban roads under flooded conditions and normal conditions containing vehicles and pedestrians to obtain sample set 2; Step S2: inputting the sample set 1 into the urban road flooding monitoring model for training, and obtaining a judgment result of whether road flooding occurs by performing target detection on the input video by the urban road flooding monitoring model; Step S3: inputting the sample set 2 into the disaster-affected individual monitoring model for training, and obtaining a result of whether the disaster-affected individual monitoring model recognizes the disaster-affected individual target on the input video and recognizes the disaster-affected vehicle or pedestrian; Step S4: The trained urban road flooding monitoring model and the trained disaster-affected individual monitoring model are cascaded to form a complete urban road flooding and disaster-affected individual identification model. When the urban road flooding monitoring model predicts that road flooding has occurred, the disaster-affected individual monitoring model is activated to monitor whether there are vehicles or pedestrians.

2. The method for identifying flooded urban roads and tunnel entrances and affected individuals according to claim 1, characterized in that: The step S1 includes collecting surveillance images or videos of urban roads under water accumulation and normal conditions, and surveillance images or videos of urban roads under water accumulation and normal conditions containing vehicles and pedestrians; pre-processing the surveillance images or videos of urban roads under water accumulation and normal conditions to obtain sample set 1, and pre-processing the images of urban roads under water accumulation and normal conditions containing vehicles and pedestrians to obtain sample set 2, which specifically includes: Step S11: acquiring surveillance images or videos of urban road waterlogging and normal conditions respectively, performing frame acquisition operations at a preset interval 1, and obtaining images of urban road waterlogging and images of urban road normal conditions respectively as sample set 1; Step S12: The surveillance images or videos of urban road waterlogging and vehicles and pedestrians under normal conditions are framed at a preset interval of 2, and the image categories and bounding boxes of the vehicle and pedestrian targets are annotated as sample set 2.

3. The method for identifying flooded urban roads and tunnel entrances and affected individuals according to claim 2, characterized in that: Step S2: inputting the sample set 1 into the urban road flooding monitoring model for training, performing target detection on the input video to obtain a judgment result on whether road flooding occurs, specifically including: Step S21: Using the image in sample set 1 as the input image, input the urban road flooding monitoring model constructed based on the fusion of the improved MobileNet model and the LBP algorithm; first, the input image is subjected to a 3x3 convolutional layer to extract feature x, and the LBP algorithm is used to extract the texture feature y of the input image. Feature z is obtained by fusing x and y; Step S22: extracting the feature z using multiple convolutional layers including depthwise convolution and point-to-point convolution; Step S23: Finally, two 1x1 convolutional layers are used to replace the fully connected layer to output the category, that is, to determine whether the input image has road flooding.

4. The method for identifying flooded urban roads and tunnel entrances and affected individuals according to claim 3, characterized in that: Step S3: inputting the sample set 2 into the disaster-affected individual monitoring model for training, performing disaster-affected individual target recognition on the input video to obtain a judgment result on whether the disaster-affected vehicles and pedestrians are recognized, specifically including: The affected individual monitoring model is constructed based on the YOLOv5 model, and the sample set 2 is input into the affected individual monitoring model for training to obtain the pedestrian or vehicle prediction frame in the input image. At the same time, the loss function GIOU is constructed. Loss , as shown in formula (5): Among them, A is the predicted box, B is the real box, C represents the minimum convex closed box containing A and B, and IOU is the overlap rate between the predicted box and the real box.

5. A method and system for identifying flooded urban roads and tunnel entrances and individuals affected by the disaster, characterized in that: Includes the following modules: The sample collection and preprocessing module is used to collect surveillance images or videos of urban roads under water accumulation conditions and normal conditions, as well as surveillance images or videos of urban roads under water accumulation conditions and normal conditions containing vehicles and pedestrians; preprocess the surveillance images or videos of urban roads under water accumulation conditions and normal conditions to obtain sample set 1, and preprocess the images of urban roads under water accumulation conditions and normal conditions containing vehicles and pedestrians to obtain sample set 2; An urban road flooding monitoring model training module is used to input the sample set 1 into the urban road flooding monitoring model for training, and perform target detection on the input video to obtain a judgment result on whether road flooding occurs; The disaster-affected individual monitoring model training module is used to input the sample set 2 into the disaster-affected individual monitoring model for training, and perform disaster-affected individual target recognition on the input video to obtain a judgment result on whether the disaster-affected vehicle or pedestrian is recognized; The road flooding and affected individual identification module is used to cascade the trained urban road flooding monitoring model and the trained affected individual monitoring model to form a complete urban road flooding and affected individual identification model. When the prediction result of the urban road flooding monitoring model is that road flooding occurs, the affected individual monitoring model is activated to monitor whether there are vehicles and pedestrians.

Images