Urban inland inundation ponding depth identification method based on image intelligent classification

Through the method based on intelligent image classification, deep learning models are used to identify and classify the depth of road water accumulation, the problem of traditional monitoring methods consume manpower and relying on fixed reference objects is solved, and the rapid and comprehensive real-time monitoring of urban waterlogging is achieved, and flood warning and Emergency response.

CN119007104AActive Publication Date: 2024-11-22SOUTH CHINA UNIV OF TECH

Patent Information

Application Number
CN202411023606.7
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2024-07-29
Publication Date
2024-11-22
Estimated Expiration
2044-07-29

AI Technical Summary

Technical Problem

Traditional urban flooding monitoring methods consume a lot of manpower and material resources, have poor real-time performance, and existing deep learning technologies rely on fixed reference materials, which are generally low in applicability and cannot meet the needs of urban flooding monitoring.

Method used

Using an intelligent image classification method, the deep learning model is trained by monitoring video and synchronous water accumulation depth data to achieve rapid identification of road water accumulation depth. The image segmentation and classification network model are used to identify water accumulation areas and classify them, which breaks through the fixed reference The dependence of matter improves the universality and efficiency of monitoring.

Benefits of technology

It has achieved rapid and comprehensive real-time monitoring of the depth of urban waterlogging, reduced manpower and material resources costs, improved the real-time and universality of monitoring, and supported urban flood warning and emergency response.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN119007104A_ABST
    Figure CN119007104A_ABST
Patent Text Reader

Abstract

The invention discloses an urban inland inundation ponding depth identification method based on image intelligent classification. According to the method, urban waterlogging monitoring video images are intercepted, waterlogging feature information data tags are made, a data set is expanded, and then a Fast-SCNN model is trained. Urban inland inundation monitoring images are reasoned in batches by using a Fast-SCNN training result to obtain images of segmented ponding areas, ponding depth level data labels are made by combining actually measured ponding depth data of urban roads, ResNet model parameters are adjusted, and learning training is performed on ponding depth levels in the images. When an input image is predicted, the road waterlogging depth grade is finally output through prediction of the image segmentation model and the image classification model in sequence. According to the method, the accumulated water depth can be identified by directly using the image shot by the road monitoring camera, the dependence of an existing deep learning road accumulated water detection technology on an accumulated water marker is broken through, and the adaptability of intelligently identifying the road accumulated water depth by the image is improved.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present invention relates to an application field of urban waterlogging monitoring, and in particular to an urban waterlogging water depth recognition method based on intelligent image classification. Background Art

[0002] In recent years, due to global climate change, the frequency and intensity of extreme precipitation events in urban areas have increased significantly. At the same time, with the continuous acceleration of urbanization, the impervious area in urban areas has increased, and the insufficient capacity of the drainage system has further increased the risk of urban flood disasters, threatening the safety of residents' lives and property. In order to efficiently carry out emergency response work for urban waterlogging disasters, it is urgent to monitor the waterlogging on urban roads in real time and provide technical support for rapid response to urban waterlogging. Traditional manual monitoring methods consume a lot of manpower and material resources, cannot guarantee the timeliness of waterlogging monitoring, and pose a hidden danger to the personal safety of monitoring personnel; the sensor monitoring method has high installation and maintenance costs, and it is difficult to meet the needs of comprehensive urban waterlogging monitoring.

[0003] With the continuous improvement of urbanization, a large number of traffic monitoring cameras have been widely deployed in urban road networks, which can fully record the process of urban flooding and provide a data source for real-time monitoring of waterlogging information. The continuous development of deep learning technology provides an effective technical approach for the development of low-cost image processors, and has shown excellent performance in the task of urban waterlogging water depth monitoring. Chaudhary et al. (Chaudhary P, D'aronco S, Moy de Vitry M, et al. Flood-water level estimation from social media images [J]. ISPRS Annals of the Photogrammetry, Remote Sensing and Spatial Information Sciences, 2019. DOI: 10.5194 / isprs-annals-IV-2-W5-5-2019.) used human bodies, cars, buses, bicycles and houses as markers, and used Mask RCNN to classify the depth of waterlogging into 11 water depth levels. Huang Jing et al. (Huang Jing, Wang Huimin, Kang Jinle, et al. A method for monitoring the depth of urban waterlogging based on deep learning [P]. Jiangsu Province: CN202010004348.3, 2020-06-05.) proposed a method for monitoring the depth of urban waterlogging based on MaskRCNN, using automobile tires as markers, and calculating the depth of waterlogging in the monitoring image using the height difference method and the Pythagorean theorem. However, the current related technical methods all rely on fixed reference objects, which are only effective when the image contains ideal reference objects, and their general applicability is low. Therefore, the present invention proposes for the first time a technical framework for grading the depth of waterlogging after segmenting the waterlogging area, breaking through the dependence of the existing deep learning technology for detecting road waterlogging on waterlogging markers, and improving the universality of image intelligent recognition of road waterlogging depth. The present invention can quickly and comprehensively realize remote real-time monitoring of road waterlogging depth, and provide data support for urban flood warning and emergency response. Summary of the invention

[0004] In order to solve the problems that traditional urban waterlogging monitoring methods consume a lot of manpower, material and financial resources, have poor real-time performance and insufficient stability, and that existing deep learning waterlogging detection technologies rely on fixed reference objects and have low general applicability, the present invention provides a method for identifying the depth of urban waterlogging based on intelligent image classification. By training a deep learning image recognition model using surveillance video and synchronized waterlogging depth data, the results can be used to quickly identify the level of waterlogging depth and realize real-time monitoring of urban waterlogging.

[0005] The purpose of the present invention is achieved by at least one of the following technical solutions.

[0006] A method for identifying the depth of urban waterlogging based on intelligent image classification comprises the following steps:

[0007] S1, obtaining a video shot by a city waterlogging monitoring camera, and capturing a single frame image containing water areas recognizable by human eyes from the video as initial image data;

[0008] S2, delineating the boundaries and labeling the water accumulation range visible in a single frame image in the initial image dataset;

[0009] S3, using data enhancement technology to expand the initial image data set, generate image data, and divide it into a first training set, a first validation set and a first test set, and convert it into a data set format that can be read by the image segmentation network model;

[0010] S4, adjusting the parameters of the image segmentation network model, and performing learning and training on the visible waterlogging range of a single frame image in the initial image data set after the data expansion, to obtain a trained image segmentation network model; using the trained image segmentation network model to batch infer the urban waterlogging monitoring images, to obtain images of segmented waterlogging areas, and using the images of segmented waterlogging areas as the data source for the next step of training the image classification network model;

[0011] S5, according to the measured water depth data of urban roads, preparing water depth level data labels of the images of the segmented waterlogged areas in step S4, and dividing the images of the segmented waterlogged areas and their labels into a second training set, a second verification set and a second test set;

[0012] S6, adjusting the parameters of the image classification network model, and performing learning and training on the water depth levels of the images of the segmented waterlogged areas in the second training set and the second verification set to obtain a trained image classification network model;

[0013] S7. When predicting the input image, the trained image segmentation model and the image classification model are used to predict the road water depth level.

[0014] Furthermore, in step S2, Labelme data annotation software is used to create boundary labels for the waterlogged area. Specifically, the image is imported into the Labelme data annotation software, and polygon creation is selected to start annotation. Points and lines are drawn on the boundaries of the waterlogged areas visible in the image in turn, and finally a closed polygon that just encompasses the waterlogged area is generated by connecting the ends. The generated closed polygon is labeled 'water', and a corresponding Labelme format json label file is created for each image.

[0015] Furthermore, step S3 specifically includes the following contents:

[0016] S3.1. Expand the initial image data set using data enhancement techniques, including changing brightness, adding noise, adding random points, translating images, flipping images, and reducing image resolution to generate more water accumulation images. Meanwhile, perform corresponding transformation enhancement processing on the image labels generated in step S2 to achieve the expansion of the initial image data set, so as to improve the diversity and generalization ability of the data set.

[0017] S3.2. All images in the initial image dataset that have undergone data augmentation are randomly divided into a first training set, a first validation set, and a first test set in a ratio of 8:1:1, and converted into a dataset format that can be read by the image segmentation network model.

[0018] Furthermore, in step S3, the image segmentation network model used for waterlogged area segmentation is Fast-SCNN.

[0019] Furthermore, step S4 specifically includes the following contents:

[0020] S4.1. Register the paths and categories of the first training set, the first validation set, and the first test set in the image segmentation network model, customize the main training parameters according to computer performance and project requirements, including the basic learning rate, learning rate decay, the number of samples per batch, the number of working processes, the number of iterations, the training rounds, the training log recording interval, and the weight file saving interval, and start training the image segmentation network model;

[0021] S4.2. After the training is completed, the waterlogging detection effect of the image segmentation network model is evaluated according to the intersection over union (IoU), precision and recall, and the network model parameters are further adjusted to repeatedly train the model until the model training effect reaches the optimal one, that is, the training is completed, and the final model weight file is generated to obtain the trained image segmentation network model; the trained image segmentation network model is used to batch infer the urban waterlogging monitoring images to obtain the images of the segmented waterlogged areas. The images of the segmented waterlogged areas are binary images, and the pixel grayscale values ​​of the waterlogged areas are assigned to 255, that is, white, and the pixel grayscale values ​​of the background areas are assigned to 0, that is, black.

[0022] Furthermore, in step S5, the measured water depth data of urban roads is the water depth at the road water monitoring point recorded synchronously with the monitoring video in time. Based on the data, water depth level data labels of the images of the segmented waterlogged areas are produced and randomly divided into a second training set, a second verification set and a second test set in a ratio of 8:1:1.

[0023] Furthermore, water depth level data labels are produced through sub-folder method. This method places images into self-named sub-folders according to the different water depth levels to which each image of the segmented waterlogged area belongs. The name of the sub-folder is the water depth level to which the image belongs, and the sub-folder name will be identified as the name of the category to which the image belongs.

[0024] Furthermore, the image classification network model adopts a ResNet model.

[0025] Furthermore, in step S6, the paths and categories of the second training set, the second validation set and the second test set are registered in the image classification network model, and the main training parameters are customized according to computer performance and project requirements, including basic learning rate, learning rate decay, the size of the number of samples in each batch, the number of working processes, the number of iterations, the training rounds, the training log recording interval, and the weight file saving interval, and the training of the image classification network model is started; after the training is completed, the water depth classification effect of the image classification network model is evaluated according to the multi-category average precision (mAP) and Recall, and the network model parameters are further adjusted to repeatedly train until the model training effect reaches the optimal value, that is, the training is completed, and the final model weight file is generated to obtain the trained image classification network model.

[0026] Furthermore, in step S7, based on the weight file obtained by training the Fast-SCNN and ResNet models, when predicting the input image, the trained Fast-SCNN model and ResNet model are used to predict the road water depth level.

[0027] The beneficial effects of the present invention are:

[0028] 1. It breaks through the reliance of existing deep learning technology for detecting road waterlogging on waterlogging markers, and improves the universality of image intelligent recognition of road waterlogging depth.

[0029] 2. The reasoning speed of the water segmentation model and the water depth classification model is relatively high, and in theory it is possible to directly call the camera for reasoning.

[0030] 3. By adopting deep learning image recognition technology, there is no need to spend a lot of manpower, material and financial resources on manual monitoring. The waterlogging points can be monitored in real time through remote terminal operation, thereby automatically extracting the water depth level at the waterlogging points at the monitoring site. BRIEF DESCRIPTION OF THE DRAWINGS

[0031] Figure 1 It is a flow chart of a method for identifying the depth of urban waterlogging based on intelligent image classification;

[0032] Figure 2is a Fast-SCNN network architecture diagram for waterlogged area segmentation in an embodiment of the present invention;

[0033] Figure 3 is a ResNet network architecture diagram for water depth classification in an embodiment of the present invention;

[0034] Figure 4 Schematic diagram of the classification standard of road water depth in an embodiment of the present invention;

[0035] Figure 5 It is a schematic diagram of the classification effect of road water depth levels in one embodiment of the present invention.

[0036] Figure 6 This is a schematic diagram of a recognition error case during testing in another embodiment of the present invention. DETAILED DESCRIPTION

[0037] The specific implementation scheme of the present invention is further described below in conjunction with the specific implementation scheme and the accompanying drawings. The accompanying drawings referred to below are only exemplary in nature, and the implementation scheme of the present invention is not limited thereto.

[0038] Example:

[0039] A method for identifying the depth of urban waterlogging based on intelligent image classification. Figure 1 As shown, the following steps are included:

[0040] S1, obtaining a video shot by a city waterlogging monitoring camera, and capturing a single frame image containing water areas recognizable by human eyes from the video as initial image data;

[0041] In one embodiment, videos captured by surveillance cameras at the scene of urban flooding during a typhoon were collected. The videos recorded the rise and fall of the water level at the waterlogging point due to rainfall and drainage for several hours. 9790 single-frame images containing water areas that can be identified by the human eye were intercepted from the videos. The intercepted images were in JPEG format with an image resolution of 1920×1080. Images in which the waterlogged area was blocked by pedestrians or vehicles for more than 1 / 3 were removed, and the remaining 7317 images were used as the initial data set.

[0042] S2, delineating the boundaries and labeling the water accumulation range visible in a single frame image in the initial image dataset;

[0043] In one embodiment, in order to improve the efficiency of data annotation, the initial image dataset is screened according to the principle of "selecting as few images as possible while ensuring that all image scenes are covered", and finally 2924 images are selected, and the boundaries of the visible waterlogging range in the images are delineated and labels are produced to construct a waterlogging image dataset;

[0044] In one embodiment, Labelme data annotation software is used to create a boundary label for the waterlogging range. In the Labelme data annotation software, select "Open Directory" to open the folder storing the 2924 waterlogging image data sets, select one of the images, and choose to create a polygon to start annotation. The boundaries of the visible waterlogging areas in the image are drawn and connected in turn, and finally a closed polygon that just covers the waterlogging area is generated by connecting the beginning and the end. The generated closed polygon is assigned the classification label information "water", and the changes are saved to generate the corresponding json label file of the image. The corresponding Labelme format json label files are created for all 2924 waterlogging images in turn.

[0045] S3. Use data enhancement technology to expand the initial image data set to generate image data, and divide it into a first training set, a first validation set, and a first test set, and convert it into a data set format that can be read by the image segmentation network model, specifically including the following contents:

[0046] S3.1. Expand the initial image data set using data enhancement techniques, including changing brightness, adding noise, adding random points, translating images, flipping images, and reducing image resolution to generate more water accumulation images. Meanwhile, perform corresponding transformation enhancement processing on the image labels generated in step S2 to achieve the expansion of the initial image data set, so as to improve the diversity and generalization ability of the data set.

[0047] S3.2. All images in the initial image dataset that have undergone data augmentation are randomly divided into a first training set, a first validation set, and a first test set in a ratio of 8:1:1, and converted into a dataset format that can be read by the image segmentation network model.

[0048] In one embodiment, the command "python DataAugmentforLabelMe.py" is entered in the terminal, and the data augmentation technology of changing brightness, adding random points, and flipping images is applied to expand the water accumulation image data, generating twice the original image data, that is, 5848 images. The command "python labelme_create_mask.py" is entered in the terminal to divide all the images after data augmentation into the first training set, verification set, and test set in a ratio of 8:1:1, and convert them into a data set format readable by the image segmentation network model Fast-SCNN. In this format, the image files divided into the training set, verification set, and test set are placed in the "img_dir" folder, and the PNG format annotation files divided into the training set, verification set, and test set are placed in the "ann_dir" folder.

[0049] S4, adjusting the parameters of the image segmentation network model, learning and training the visible waterlogging range of the single frame image in the initial image data set after the data expansion, and obtaining the trained image segmentation network model; using the trained image segmentation network model to batch infer the urban waterlogging monitoring images, i.e., the initial data set described in step S1, to obtain 7317 images of segmented waterlogging areas, and using the images of segmented waterlogging areas as the data source for the next step of training the image classification network model, specifically including the following contents:

[0050] S4.1. Register the paths and categories of the first training set, the first validation set, and the first test set in the image segmentation network model, customize the main training parameters according to computer performance and project requirements, including the basic learning rate, learning rate decay, the number of samples per batch, the number of working processes, the number of iterations, the training rounds, the training log recording interval, and the weight file saving interval, and start training the image segmentation network model;

[0051] S4.2. After the training is completed, the waterlogging detection effect of the image segmentation network model is evaluated according to the intersection over union (IoU), precision and recall, and the network model parameters are further adjusted to repeatedly train the model until the model training effect reaches the optimal one, that is, the training is completed, and the final model weight file is generated to obtain the trained image segmentation network model; the trained image segmentation network model is used to batch infer the urban waterlogging monitoring images to obtain the images of the segmented waterlogged areas. The images of the segmented waterlogged areas are binary images, and the pixel grayscale values ​​of the waterlogged areas are assigned to 255, that is, white, and the pixel grayscale values ​​of the background areas are assigned to 0, that is, black.

[0052] In one embodiment, a Fast-SCNN model under the MMSegmentation framework is used, and the paths and categories of the first training set, the first validation set, and the first test set are registered in the Fast-SCNN model. The built-in "IoUMetric" with IoU, Precision, and Recall as evaluation indicators is selected as the evaluator, and a round of verification is set every 16,000 iterations. Other parameter configurations are shown in Table 1, and the configured development environment is activated, and the model training program "train.py" is run; after the training is completed, the model weight file is obtained, and the test program "test.py" is run. According to IoU, Precision, and Recall, the model weight file is obtained. n and Recall were used to evaluate the water accumulation detection effect of the image segmentation model on the test set. The model was tested on 585 test set images, and the results showed that IoU was 0.970, Precision was 0.984, and Recall was 0.985. The recognition speed (FPS) was tested by running "benchmark.py", and the result was 189.33 (img / s). The initial data set described in S1 was run by running "infer.py" to batch infer, and 7317 images of segmented water accumulation areas were obtained. The image is a binary image, and the pixel grayscale value of the water accumulation area is assigned to 255, that is, white, and the pixel grayscale value of the background area is assigned to 0, that is, black.

[0053] Table 1 Fast-SCNN neural network parameter configuration

[0054]

[0055]

[0056] S5, according to the measured water depth data of urban roads, preparing water depth level data labels of the images of the segmented waterlogged areas in step S4, and dividing the images of the segmented waterlogged areas and their labels into a second training set, a second verification set and a second test set;

[0057] The measured water depth data of urban roads is the water depth at the road water monitoring point recorded synchronously with the monitoring video in time. Based on the data, water depth level data labels of images of segmented waterlogged areas are produced and randomly divided into a second training set, a second verification set and a second test set in a ratio of 8:1:1.

[0058] The water depth level data labels are produced by sub-folder method. This method places the images into self-named sub-folders according to the different water depth levels of each image of the segmented waterlogged area. The name of the sub-folder is the water depth level to which the image belongs. The sub-folder name will be identified as the name of the category to which the image belongs.

[0059] In one embodiment, the image classification network model adopts a ResNet model.

[0060] S6, adjusting the parameters of the image classification network model, and performing learning and training on the water depth levels of the images of the segmented waterlogged areas in the second training set and the second verification set to obtain a trained image classification network model;

[0061] Register the paths and categories of the second training set, the second validation set and the second test set in the image classification network model, customize the main training parameters according to computer performance and project requirements, including basic learning rate, learning rate decay, sample size per batch, number of working processes, number of iterations, training rounds, training log recording interval, and weight file saving interval, and start training the image classification network model; after the training is completed, evaluate the water depth classification effect of the image classification network model according to the multi-category average precision (mAP) and Recall, and further adjust the network model parameters and repeatedly train until the model training effect reaches the optimal value, that is, the training is completed, and the final model weight file is generated to obtain the trained image classification network model.

[0062] S7. Based on the weight file obtained by training the Fast-SCNN and ResNet models, when predicting the input image, the trained Fast-SCNN model and ResNet model are used to predict the road water depth level.

[0063] In one embodiment, according to the water depth data at the road water monitoring point recorded synchronously with the monitoring video, the 7317 image dataset subfolders of the segmented waterlogged areas obtained in step S4.2 are reorganized, such as Figure 4 As shown in Figure a, the interval between each water level is 4.5 cm. The 7317 images are correctly classified into subfolders named "level_1" to "level_8", and all samples are placed in one folder. The subfolder names will be retrieved and identified as image category names to complete the production of water depth level data labels; run "data_split_class.py" to divide the second training set, the second validation set, and the second test set in a ratio of 8:1:1.

[0064] The image classification network model adopts the ResNet model under the MMpretrain framework. The backbone network selects an 18-layer ResNet residual network. The paths and 8 categories of the second training set, validation set and test set are registered in the model. The built-in multi-label classification standard "MultiLabelMetric" with mAP and Recall as evaluation indicators is selected as the evaluator. It is set to perform a verification once in each round. Other parameter configurations are shown in Table 2. The configured development environment is activated and the model training program "train.py" is run. After the training is completed, the model weight file is obtained, and the test program "test.py" is run. The water depth recognition and grading effect of the image classification model is evaluated according to the multi-category average precision (mAP) and Recall. In this embodiment, the test is carried out on 736 test set pictures. The results show that the mAP reaches 0.987, the Recall is 0.955, and the recognition speed (FPS) is 120.50 (img / s).

[0065] Table 2 ResNet neural network parameter configuration

[0066]

[0067]

[0068] In this embodiment, by entering a command in the terminal to load the image to be predicted, load the Fast-SCNN model configuration file and weight file, identify and segment the waterlogged area, output the image file of the RGB channel after segmentation and drawing, and then load the ResNet model configuration file and weight file, perform image inference after water segmentation and return the category name, which is the waterlogged depth level; the classification effect diagram is shown in Figure 5 As shown, the field "Prediction:3,0.94(level_4)" in the upper left corner represents the prediction result of the sample, where "0.94" represents the confidence score (pred_score) of the prediction result, and "level_4" represents that the predicted water depth level is level 4.

[0069] In another embodiment, based on the above embodiment, only the standard for water depth classification is changed. According to the water depth data at the road water monitoring point recorded synchronously with the monitoring video, such as Figure 4 As shown in Figure b, the interval between each water level is 3 cm. The 7317 images are correctly classified into subfolders named "level_1" to "level_12". The second training set, the second validation set, and the second test set are divided into the second training set, the second validation set, and the second test set in the ratio of 8:1:1. The other operations are the same as above.

[0070] The image classification network model adopts the ResNet model under the MMpretrain framework, and the backbone network selects an 18-layer ResNet residual network. The paths and 12 categories of the second training set, the second validation set, and the second test set are registered in the model, and other operations are the same as above. In this embodiment, tests were carried out on 736 test set images, and the results showed that the mAP reached 0.974, the Recall was 0.914, and the recognition speed (FPS) was 103.57 (img / s).

[0071] In this embodiment, the image to be predicted is also loaded, and the Fast-SCNN model weight file and the ResNet model weight file trained according to the 12-level division method are loaded in sequence, and then reasoning is performed and the category name is returned; Figure 6 The figure shows a schematic diagram of an incorrect recognition case during model testing. The field in the upper left corner indicates the test result of the sample, where "Ground truth: 9 (level_3)" means that the true label of the sample is "level_3", that is, the water depth level is level 3. "Prediction: 10, 0.51 (level_4)" means that the confidence score is 0.51, and the model incorrectly predicts the sample as "level_4", that is, it incorrectly predicts the water depth level to be level 4.

[0072] The present invention is a method for identifying the depth of urban waterlogging based on intelligent image classification. According to the examples and related steps provided in the specification, it can be used to monitor the process of waterlogging in urban areas. The present invention uses monitoring videos and synchronized waterlogging depth data to train a deep learning image recognition model, which can quickly and accurately segment the scope of waterlogging and further identify the level of waterlogging depth. The results of the present invention can promote the rapid, efficient and comprehensive monitoring of urban waterlogging, ensure the real-time, effectiveness and comprehensiveness of information acquisition during urban waterlogging monitoring, and provide an effective idea and approach for orderly carrying out urban waterlogging emergency management and improving the ability of relevant departments and the public to respond to sudden urban waterlogging disasters.

[0073] The same or similar symbols and notations mentioned in the description of this specification represent the same or similar physical meanings or have the same or similar functions, and the legends used in this specification are only for better explanation of the present invention, and the applicability of the present invention is not limited thereto. Any equivalent changes, modifications or evolutions made by those skilled in the art to the above embodiments using the technical solution of the present invention still fall within the scope of the technical solution of the present invention.

Claims

1. A method for identifying the depth of urban waterlogging based on intelligent image classification, characterized in that: The steps include: S1. Obtain a video shot by a city waterlogging monitoring camera, and extract a single frame image containing water areas recognizable by human eyes from the video as initial image data; S2, delineating the boundaries and labeling the water accumulation range visible in a single frame image in the initial image dataset; S3, using data enhancement technology to expand the initial image data set, generate image data, and divide it into a first training set, a first validation set and a first test set, and convert it into a data set format that can be read by the image segmentation network model; S4, adjusting the parameters of the image segmentation network model, and performing learning and training on the visible waterlogging range of a single frame image in the initial image data set after the data expansion, to obtain a trained image segmentation network model; using the trained image segmentation network model to batch infer the urban waterlogging monitoring images, to obtain images of segmented waterlogging areas, and using the images of segmented waterlogging areas as the data source for the next step of training the image classification network model; S5, according to the measured water depth data of urban roads, preparing water depth level data labels of the images of the segmented waterlogged areas in step S4, and dividing the images of the segmented waterlogged areas and their labels into a second training set, a second verification set and a second test set; S6, adjusting the parameters of the image classification network model, and performing learning and training on the water depth levels of the images of the segmented waterlogged areas in the second training set and the second verification set to obtain a trained image classification network model; S7. When predicting the input image, the trained image segmentation model and the image classification model are used to predict the road water depth level.

2. According to the method of claim 1, the depth recognition method of urban waterlogging based on intelligent image classification is characterized in that: In step S2, use Labelme data annotation software to create boundary labels for the waterlogged area. The specific method is to import the image into Labelme data annotation software, select Create Polygon to start annotation, and then draw points and connect lines on the boundaries of the visible waterlogged areas in the image. Finally, connect the ends to generate a closed polygon that just covers the waterlogged area. Give the generated closed polygon a label of 'water', and create a corresponding Labelme format json label file for each image.

3. According to the method of claim 1, the depth recognition method of urban waterlogging based on intelligent image classification is characterized in that: Step S3 specifically includes the following contents: S3.

1. Expand the initial image data set using data enhancement techniques, including changing brightness, adding noise, adding random points, translating images, flipping images, and reducing image resolution to generate more water accumulation images. Meanwhile, perform corresponding transformation enhancement processing on the image labels generated in step S2 to achieve the expansion of the initial image data set, so as to improve the diversity and generalization ability of the data set. S3.

2. All images in the initial image dataset that have undergone data augmentation are randomly divided into a first training set, a first validation set, and a first test set in a ratio of 8:1:1, and converted into a dataset format that can be read by the image segmentation network model.

4. According to the method of claim 1, the depth recognition method of urban waterlogging based on intelligent image classification is characterized in that: In step S3, the image segmentation network model used for waterlogged area segmentation is Fast-SCNN.

5. The method for identifying the depth of urban waterlogging based on intelligent image classification according to claim 1 is characterized in that: Step S4 specifically includes the following contents: S4.

1. Register the paths and categories of the first training set, the first validation set, and the first test set in the image segmentation network model, customize the main training parameters according to computer performance and project requirements, including the basic learning rate, learning rate decay, the number of samples per batch, the number of working processes, the number of iterations, the training rounds, the training log recording interval, and the weight file saving interval, and start training the image segmentation network model; S4.

2. After the training is completed, the waterlogging detection effect of the image segmentation network model is evaluated according to the intersection over union (IoU), precision and recall, and the network model parameters are further adjusted to repeatedly train the model until the model training effect reaches the optimal one, that is, the training is completed, and the final model weight file is generated to obtain the trained image segmentation network model; the trained image segmentation network model is used to batch infer the urban waterlogging monitoring images to obtain the images of the segmented waterlogged areas. The images of the segmented waterlogged areas are binary images, and the pixel grayscale values ​​of the waterlogged areas are assigned to 255, that is, white, and the pixel grayscale values ​​of the background areas are assigned to 0, that is, black.

6. The method for identifying the depth of urban waterlogging based on intelligent image classification according to claim 1 is characterized in that: In step S5, the measured water depth data of urban roads is the water depth at the road water monitoring point recorded synchronously with the monitoring video in time. Based on the data, water depth level data labels of the images of the segmented waterlogged areas are produced and randomly divided into a second training set, a second verification set and a second test set in a ratio of 8:1:

1.

7. The method for identifying the depth of urban waterlogging based on intelligent image classification according to claim 6 is characterized in that: The water depth level data labels are produced by sub-folder method. This method places the images into self-named sub-folders according to the different water depth levels of each image of the segmented waterlogged area. The name of the sub-folder is the water depth level to which the image belongs. The sub-folder name will be identified as the name of the category to which the image belongs.

8. The method for identifying the depth of urban waterlogging based on intelligent image classification according to claim 1 is characterized in that: In step S6, the image classification network model adopts the ResNet model.

9. The method for identifying the depth of urban waterlogging based on intelligent image classification according to claim 1 is characterized in that: In step S6, the paths and categories of the second training set, the second validation set and the second test set are registered in the image classification network model, and the main training parameters are customized according to computer performance and project requirements, including basic learning rate, learning rate decay, sample size per batch, number of working processes, number of iterations, training rounds, training log recording interval, and weight file saving interval, and the training of the image classification network model is started; after the training is completed, the water depth classification effect of the image classification network model is evaluated according to the multi-category average precision (mAP) and Recall, and the network model parameters are further adjusted to repeatedly train until the model training effect reaches the optimal value, that is, the training is completed, and the final model weight file is generated to obtain the trained image classification network model.

10. The method for identifying the depth of urban waterlogging based on intelligent image classification according to any one of claims 1 to 9, characterized in that: In step S7, based on the weight file obtained by training the Fast-SCNN and ResNet models, when predicting the input image, the trained Fast-SCNN model and ResNet model are used to predict the road water depth level.

Citation Information

Patent Citations

  • Urban inland inundation ponding area monitoring method based on deep learning technology

    CN111462218A

  • Urban waterlogging depth identification method based on social media and deep learning

    CN115170800A

  • Road waterlogging area identification method based on data mining and deep learning

    CN116452850A

  • Training a deep learning system for maritime applications

    US20200050893A1

Cited By

  • Low-lying road section water depth identification method and electronic device

    CN122714994A