A road waterlogging area identification method based on data mining and deep learning

Abstract

The application discloses a kind of road waterlogging area identification method based on data mining and deep learning.The method comprises the following steps: obtaining city road waterlogging image from internet big data platform by web crawler and data mining technology;Image is preprocessed, and waterlogging feature information is data labeled;Image processing algorithm and data enhancement technology are used to expand data set, generate a large number of image data, and divide training set and test set;The training set data label frame is clustered using K-means clustering algorithm, and the model configuration parameter is adjusted;Based on deep learning Mask RCNN target detection model, the visible waterlogging range in data set is learned and trained to obtain the final model training weight file;Through the extraction and visualization prediction of image waterlogging feature, the detection and identification of waterlogging range are realized.The application widens the channel of image data acquisition, and greatly reduces the difficulty of image data set construction.

Description

TECHNICAL FIELD

[0001] The application relates to the application field of urban waterlogging monitoring, and in particular to a road waterlogging area identification method based on data mining and deep learning technology. BACKGROUND

[0002] In recent years, under the background of climate change, the urbanization process is accelerating, and urban rainstorm waterlogging disasters occur frequently under the influence of urban construction and human life, which not only seriously affects the daily life and work of residents, but also causes serious damage to social public property. In order to facilitate the relevant departments to carry out urban waterlogging emergency rescue and disaster emergency management work, and to reduce the harm to life safety and property safety caused by urban rainstorm waterlogging as much as possible, it is urgent to monitor the urban waterlogging in real time, improve the emergency management ability to cope with urban rainstorm flood disasters, and reduce the loss. However, the traditional manual or instrument monitoring method not only needs a large amount of manpower, material resources and financial resources, but also has the problems of difficult real-time and rapid monitoring, high price of instrument equipment, easy influence by external environment, insufficient stability and the like. Therefore, a new type of waterlogging real-time monitoring technical method needs to be sought.

[0003] Deep learning, as an emerging research field, has attracted much attention in recent years. The emergence of R-CNN and YOLO models has demonstrated the unique advantages of deep learning technology in feature extraction and simulation optimization in the field of computer vision. In 2017, He et al. (He K, Gkioxari G, Dollár P, et al. Mask r-cnn[C] / / Proceedings of the IEEE international conference on computer vision.2017:2961-2969.) proposed the Mask R-CNN object detection network model, which has provided a new technical approach for monitoring urban flooding. The application of deep learning in urban flood monitoring has been studied by scholars. For example, Bai Ganggang et al. (Bai Ganggang, Hou Jingming, Han Hao et al. Intelligent monitoring method for road waterlogging based on deep learning [J]. Water Resources Protection, 2021, 37(05):75-80.) explored the use of YOLOv2 network model for automatic identification of waterlogged areas and extraction of waterlogged area; Jafari et al. (Jafari NH, Li X, Chen Q, et al. Real-time water level monitoring using live cameras and computer vision techniques [J]. Computers & Geosciences, 2021, 147:104642.) used urban hydrological images obtained by traffic cameras to train a deep learning-based image segmentation model, successfully distinguishing image areas of rainstorm waterlogging and local floods. However, the image sources of the above studies rely on fixed monitoring cameras, which has limitations such as single data source and poor diversity of waterlogging scenes. In addition, the deployment of multiple cameras also increases the difficulty of image acquisition and high equipment costs.

[0004] The rapid development of the Internet makes the extraction and mining of network big data become a new trend of data acquisition. At present, domestic scholars have carried out research on waterlogging disasters related to micro-blog text big data, such as Wu Xianhua et al. (Wu Xianhua, Xiao Yang, Wang Guofu, etc. City waterlogging disaster disaster and public sentiment research based on micro-blog big data: Taking Nanjing as an example[J]. Disaster science, 2018, 33(03): 117-122.) proposed a technical method for waterlogging disaster grade discrimination and public opinion research based on micro-blog big data. However, the application of network images is still relatively less, and the research on city waterlogging monitoring technology combined with network big data and deep learning technology is still one of the directions worthy of further exploration. Based on this, a road water accumulation area recognition method based on data mining and deep learning is proposed, which is of great significance to the orderly development of waterlogging disaster emergency management, improving the ability of relevant departments and the public to respond to sudden urban waterlogging disasters, and reducing property losses and casualties caused by urban waterlogging. SUMMARY

[0005] In order to solve the problems of traditional city waterlogging monitoring methods, such as large consumption of manpower, material resources and financial resources, poor real-time performance, insufficient stability, and single image data source and difficult data acquisition of existing deep learning waterlogging detection technology, the purpose of the present application is to provide a road water accumulation area recognition method based on data mining and deep learning, which fully utilizes big data and mass media platforms as data acquisition channels, and uses deep learning image recognition technology to solve the real-time and rapid monitoring problems of city waterlogging.

[0006] The purpose of the present application is achieved at least by one of the following technical solutions.

[0007] A road water accumulation area recognition method based on data mining and deep learning, comprising the following steps:

[0008] S1, acquiring city waterlogging accumulation image from Internet big data platform through network crawler and data mining technology, constructing water accumulation image database, and establishing initial water accumulation image data set through screening;

[0009] S2, preprocessing the images in the initial water accumulation image data set, and performing boundary drawing and label making on the visible water accumulation range in the data set;

[0010] S3, using image processing algorithm to simultaneously perform data enhancement on the images and labels, expand the data set, divide the labeled data set into training set and test set, and convert it into data set format readable by neural network model;

[0011] S4, using K-means clustering algorithm, clustering the target frame of the training set, automatically generating a set of more suitable anchor frame for the custom data set, adjusting the size and aspect ratio of the anchor in the model configuration file;

[0012] S5, input the labeled training set and validation set into the Mask RCNN instance segmentation model for learning, and evaluate the waterlogging detection performance of the neural network model after training, repeatedly train by adjusting the neural network model training parameters until the model training effect reaches the optimal, that is, complete the model training, and get the final model training weight file;

[0013] S6, based on the weight file trained in S5, input the test image or video into the detection project, extract the image waterlogging feature, and get the waterlogging range detection and recognition result.

[0014] Further, in step S1, the Internet big data platform includes network social media and crowdsourcing data platform;

[0015] Formulate information retrieval keywords, such as selecting relatively broad "waterlogging" "road waterlogging" "heavy rain" and other keywords, or selecting specific rainfall events as retrieval range, then using network crawler and data mining technology, through keyword retrieval, obtaining city road waterlogging image and related location and time information from Internet big data platform, downloading to local database for storage and arrangement. From the image database, waterlogging image is screened and extracted to form an initial city road waterlogging image dataset.

[0016] Further, in step S2, the following steps are included:

[0017] S2.1, using Opencv computer vision image processing library, image operation is performed on the initial waterlogging image, including cropping, size expansion or scaling, adjusting it to a uniform resolution to facilitate subsequent image processing and data label making;

[0018] S2.2, using Labelme data labeling software to extract waterlogging area features, importing the image into Labelme, selecting to create a polygon to start labeling, drawing the boundary contour and connecting the lines of the visible waterlogging target in the image by sequentially drawing points, finally connecting the head and tail to generate a closed polygon that just covers the waterlogging area, and assigning classification label information to the generated closed polygon, thereby creating a corresponding Labelme format json label file for each image.

[0019] Further, in step S3, new waterlogging images are generated by using image processing algorithms in the Opencv library and Augmentor semantic segmentation data enhancement methods, data augmentation of the image dataset is realized to improve the diversity and generalization ability of the dataset;

[0020] The image processing method includes a color transformation method and a geometric transformation method. The color transformation method includes adjusting contrast, changing brightness, modifying RGB values, adding noise, and blackening or replacing the background area. The geometric transformation method includes enlarging or reducing the image, scale transformation, image translation, flipping, and rotation.

[0021] At the same time, the image label generated in step S2 is subjected to corresponding transformation and enhancement processing: if only color transformation data enhancement is performed without involving geometric position transformation, the label position information remains unchanged; if geometric transformation data enhancement is performed, the label also needs to change the position corresponding to the image transformation, which is as follows:

[0022] The position of the marker point in the label file is transformed according to the coordinate position of the image transformation method, or the mask image transformed from the label file is subjected to corresponding transformation processing by using the data enhancement method, thereby generating a label corresponding to the new image, and realizing the expansion of the number of image datasets by several times.

[0023] After generating more image data through data enhancement, all images and corresponding labels with information annotation are divided into a training set and a test set according to a proportion, and all labels corresponding to the training set and the test set images are batch-converted by using a data set format conversion tool, so as to be read by a target detection framework and further used.

[0024] Further, in step S4, the basic environment of deep learning image recognition is configured, and a target detection framework is installed. Then, the K-means clustering algorithm is used to cluster the labeled target frame in the training set. By adjusting the K value, the frame samples are clustered into K clusters, so as to find the best number and size of anchor frames, and the aspect ratio corresponding to the anchor is input into the target detection model configuration file.

[0025] Further, in step S4, the algorithm process for calculating the anchor frame by K-means clustering is as follows:

[0026] (1) Select the number K of clustering;

[0027] (2) Randomly select K clustering boxes as initial anchor boxes;

[0028] (3) Using IOU value as a measure, calculate the IOU value of each anchor box and each target bounding box; the IOU value is the intersection over union of the anchor box and the target bounding box, the value range is [0, 1]; when calculating IOU, it is assumed that the top left corner of all boxes is at the origin, and it is assumed that the size of the anchor is (w a ,h a ), and the size of the bounding box is (w b ,h b ), then

[0029]

[0030] Since the larger the IOU value is in image recognition, in order to make the smaller the metric value, the larger the similarity, here a distance parameter d is defined, so that each target bounding box is assigned to the anchor with the smallest distance parameter d error, the distance parameter d is taken as:

[0031] d = 1-IOU

[0032] In the formula, w a and h a are the bottom and height of the anchor box, w b and h b are the bottom and height of the target bounding box, and IOU is the intersection over union of the anchor box and the target bounding box.

[0033] (4) Calculate the median or mean size of the bottom and height of all target bounding boxes in each cluster, recalculate the new cluster center, and take it as the size of the new anchor box to update the anchor.

[0034] (5) Repeat the above steps until the anchor no longer changes, meets the convergence requirement, or reaches the maximum number of iterations.

[0035] Further, in step S5, a neural network model and its configuration file are selected according to the training requirements and configuration conditions of the self, and the model pre-training weight file needed to be used is downloaded from the network;

[0036] Register the path and type of the training set and test set in the training of the neural network model, define the main training parameter information, including the basic learning rate, learning rate decay, the size of each batch of samples, the number of iterations, the number of training periods, and start the neural network training;

[0037] After training, evaluate the effect of the neural network model on the waterlogging detection, and further adjust various neural network model parameters and repeatedly train multiple times until the model training effect reaches the optimal, that is, the training is completed, and the final model training weight file is generated.

[0038] Further, in step S6, a detection item file is created, the category information of the detection target and the model inference confidence threshold are written, and the model weight file is loaded with the weight file trained in step S5. Through inputting a command in the terminal, the waterlogging area detection and recognition of multiple input data sources are performed, the model will perform inference and draw the prediction box and the mask corresponding to the area with the inference score higher than the confidence threshold, so as to output the image or video with the waterlogging range mask, and the prediction result is visualized.

[0039] Further, the multiple input data sources include images, videos or network monitoring cameras.

[0040] The beneficial effects of the present application are:

[0041] 1. By obtaining the waterlogging images from the network platform big data such as network social media, the difficulty of obtaining image and video data is effectively reduced, the massive image data is obtained at low cost, the channel of image data acquisition is widened, and the generalization ability of the data set is improved.

[0042] 2. The data set is expanded by using data enhancement, the image and the label are augmented at the same time, the massive data set does not need to be labeled one by one, so that the workload of image data labeling is greatly reduced, and the work efficiency of data set production is significantly improved.

[0043] 3. By using the deep learning image recognition technology, the artificial monitoring which consumes a large amount of manpower, material resources and financial resources is not needed, the waterlogging points can be monitored in real time through remote terminal operation, so that the feature and boundary information of the waterlogging in the monitoring site are automatically extracted, and the waterlogging boundary and waterlogging area are quickly identified. BRIEF DESCRIPTION OF DRAWINGS

[0044] Figure 1 It is a road waterlogging area recognition method based on data mining and deep learning;

[0045] Figure 2 It is a waterlogging range extraction process schematic diagram based on the deep learning Mask RCNN algorithm in the embodiment of the present application;

[0046] Figure 3 It is an algorithm flowchart for K-means clustering calculation of anchor box in the embodiment of the present application. DETAILED DESCRIPTION

[0047] The specific embodiments of the present application are further described below in combination with the specific embodiments and the drawings. The drawings referred to below are only exemplary in nature, and the embodiments of the present application are not limited thereto.

[0048] Embodiment 1:

[0049] A road waterlogging area identification method based on data mining and deep learning, as shown in Figure 1 includes the following steps:

[0050] S1, through network crawler and data mining technology, obtain city waterlogging image from internet big data platform, construct waterlogging image database, and establish initial waterlogging image dataset through screening;

[0051] The internet big data platform includes network social media and crowdsourcing data platform;

[0052] In this embodiment, information retrieval keywords are formulated, such as "road waterlogging", "waterlogging", "waterlogging", and similar words used to describe urban waterlogging events. It can also be specific to a certain rainfall event such as "Guangzhou 5·22 heavy rain". Using network crawler and data mining technology, the city road waterlogging image and related location and time information are obtained from the internet big data platform through keyword retrieval, downloaded to the local database for storage and arrangement. The waterlogging image is filtered and extracted from the image database to form the initial city road waterlogging image dataset.

[0053] S2, pre-process the images in the initial waterlogging image dataset, and draw boundaries and make labels for the visible waterlogging range in the dataset, including the following steps:

[0054] S2.1, using Opencv computer vision image processing library, image operation is performed on the initial waterlogging image, including cropping, size expansion or scaling, adjusting it to a uniform resolution to facilitate subsequent image processing and data label making;

[0055] S2.2, using Labelme data labeling software to extract waterlogging area features, importing the image into Labelme, selecting to create a polygon to start labeling, drawing the boundary contour of the visible waterlogging target in the image by sequentially drawing points, and connecting the lines, finally connecting the head and tail to generate a closed polygon that just covers the waterlogging area. By assigning classification label information to the generated closed polygon, a corresponding Labelme format json label file is created for each image.

[0056] In this embodiment, the size specification of the dataset image is not particularly required in the Detectron2 environment. On the one hand, it is convenient for subsequent data enhancement and label processing during the dataset production process, and on the other hand, it is also to avoid the program running error caused by the large memory space occupied during training. Therefore, the size of the large resolution image is first reduced, and the image resolution is unified. The initial water accumulation image can be cropped and scaled in size by using the Opencv computer vision image processing library, and the resolution is adjusted to 512*512 to facilitate subsequent image processing and data label production.

[0057] Subsequently, in the use of the Labelme data labeling software, the visible water boundary in the image is depicted by creating a polygon, and the generated closed polygon is assigned with “water” label information to represent water accumulation. A corresponding json file in the Labelme format is generated for each image. After labeling the image, the image with the label mask can be output by inputting the command labelme_json_to_dataset<file name>.json in the terminal, and the json file is converted into label information data.

[0058] S3, using image processing algorithms, simultaneously performing data enhancement on images and labels, expanding the dataset, and dividing the labeled dataset into a training set and a test set, and converting the dataset into a format readable by a neural network model;

[0059] For deep learning, in order to achieve better model recognition effect, sufficient training data is needed for training. Generally speaking, the larger the amount of data in the dataset, the better the deep learning training effect. However, in many practical projects, it is often difficult to find a sufficient amount of high-quality data to complete a deep learning task. One way to solve this problem is data augmentation, which is a technique for generating new training samples from existing training samples. For limited water accumulation image data, data augmentation can be used to generate more new images that are different and unique in the computer's perspective, thereby increasing the number of training samples and their diversity, reducing the overfitting of the neural network, obtaining a network with stronger generalization ability, and better adapting to the application scenario.

[0060] In this embodiment, new water accumulation images are generated by using image processing algorithms in the Opencv library and the Augmentor semantic segmentation data enhancement method, and data enhancement of the image dataset is realized to improve the diversity and generalization ability of the dataset.

[0061] The image processing method includes a color transformation method and a geometric transformation method. The color transformation method includes adjusting contrast, changing brightness, modifying RGB values, adding noise, and blackening or replacing a background area. The geometric transformation method includes enlarging or reducing an image, scale transformation, translating an image, flipping, and rotating.

[0062] Meanwhile, the image label generated in step S2 is subjected to corresponding transformation enhancement processing. If only color transformation data enhancement is performed without involving geometric position transformation, the label position information remains unchanged. If geometric transformation data enhancement is performed, the label also needs to change position corresponding to the image transformation, as follows:

[0063] The label corresponding to the new image is generated by transforming the coordinate positions of the marked points in the label file corresponding to the image transformation method or by using data enhancement methods to transform the mask image converted from the label file, thereby achieving a fold expansion of the number of images in the data set.

[0064] After generating more image data through data enhancement, all images and corresponding labels with information annotation are divided into a training set and a test set according to a proportion, and all labels corresponding to the training set and the test set images are converted in batches by using a data set format conversion tool, so as to be read and further used by a target detection framework.

[0065] In this embodiment, various image processing methods such as flipping, pixel RGB value transformation, adding noise, reducing picture brightness, 45° rotation, 90° rotation, and image equal-proportion scaling are used to generate a number of pictures several times that of the original image, effectively expand the data set, and increase the diversity of data. Since the framework used in this embodiment is Detectron2, the data set format supported by the framework is COCO format, and therefore data set format conversion is needed. The images with information annotation are divided into a training set and a test set according to a 7:3 proportion, and the Labelme format label is converted into a COCO data set format by using a data set format conversion tool, so as to be read and used by the open-source target detection framework Detectron2.

[0066] S4. The K-means clustering algorithm is used to cluster the target bounding boxes of the training set to automatically generate a set of anchor boxes more suitable for the custom data set, and the size and aspect ratio of the anchor boxes are adjusted in the model configuration file.

[0067] The basic environment of deep learning image recognition is configured and the target detection framework is installed, and then the K-means clustering algorithm is used to cluster the labeled target frame in the training set; by adjusting the K value, the frame sample is clustered into K clusters, so as to find the best number and size of anchor frame, and the corresponding aspect ratio of anchor is input into the target detection model configuration file.

[0068] The related parameters such as anchor size and aspect ratio in the Detectron2 framework are artificially designed, which are defined in configs / Base-RCNN-FPN.yaml, and the anchor aspect ratio value is fixed as [0.5, 1.0, 2.0]. The preset anchor parameters are suitable for commonly used public data sets, but not necessarily suitable for self-made urban road water data sets, so in this embodiment, the K-means algorithm is used to cluster the box of the water image training set.

[0069] As shown in Figure 3 , the algorithm flow of K-means clustering calculation anchor frame is:

[0070] (1) Select the number K of clustering;

[0071] (2) Randomly select K clustering boxes as initial anchor boxes;

[0072] (3) Use the IOU value as the metric to calculate the IOU value of each anchor box and each target frame; the IOU value is the intersection over union of the anchor box and the target frame, and the value range is [0, 1]; when calculating the IOU, it is assumed that the top left corner of all boxes is at the origin, and the anchor size is (w a ,h a ), and the frame size is (w b ,h b ), then

[0073]

[0074] Since the larger the IOU value is in image recognition, in order to make the metric value smaller and the similarity larger, a distance parameter d is defined, so that each target frame is assigned to the anchor with the smallest distance parameter d error, and the distance parameter d is taken as:

[0075] d = 1-IOU

[0076] In the formula, w a and h a are the bottom and height of the anchor box, w b and h bThe bottom and height of the target bounding box, respectively, and the IOU is the intersection over union of the anchor box and the target bounding box.

[0077] (4) Calculate the median or mean size of the bottom and height of all target bounding boxes in each cluster, recalculate the new cluster center as the new anchor box size, and update the anchor;

[0078] (5) Repeat the above steps until the anchor no longer changes, meets the convergence requirement, or reaches the maximum number of iterations.

[0079] In this embodiment, the number of clusters K is selected as 9, and 9 kinds of anchor aspect ratios suitable for the self-made urban road waterlogging data set are automatically generated by K-means clustering algorithm, which are [0.6, 0.7, 0.9, 1.0, 1.2, 1.3, 1.5, 1.8, 2.4], then according to the clustering result of K-means algorithm, the anchor aspect ratio parameters in the configuration file are adjusted accordingly, and the size is not changed.

[0080] S5, input the labeled training set and validation set into the Mask RCNN neural network model for learning, and evaluate the waterlogging detection performance of the neural network model after training, repeatedly train by adjusting the neural network model training parameters until the model training effect reaches the optimal, that is, complete the model training, and obtain the final model training weight file;

[0081] According to the training requirements and configuration conditions, select the neural network model and its configuration file, and download the model pre-training weight file needed for use from the network;

[0082] Register the path and category of the training set and test set in the training of the neural network model, define the main training parameter information, including the basic learning rate, learning rate decay, batch size, iteration number, training period number, and start neural network training;

[0083] After training, evaluate the waterlogging detection effect of the neural network model, and further adjust the parameters of the neural network model, and repeatedly train several times until the model training effect reaches the optimal, that is, complete the training, and generate the final model training weight file.

[0084] In this embodiment, the Mask RCNN neural network model is used for waterlogging range extraction, and its running structure is as shown in Figure 2 The main steps are as follows:

[0085] (1) Input the image data set, the network model will first send the input picture to a pre-trained feature extraction backbone network to generate the corresponding feature map;

[0086] (2) The feature map output after feature extraction is sent to the region candidate network. The network traverses each pixel position of the feature map and sets a fixed number of anchors. After a 3*3 convolution, it enters different branches corresponding to different 1*1 convolutions. The first convolution is the positioning layer, which outputs the 4 coordinate offsets of the anchor. The second convolution is the classification layer, which performs object classification and outputs the foreground and background probabilities of the anchor. By performing the NMS method, the IOU value of the overlapping anchor and the target bounding box is judged to obtain the refined candidate ROI region;

[0087] (3) The ROI Align operation is performed on the candidate ROI region. The coordinate value of the unit sampling point is calculated using the bilinear interpolation method, and the corresponding region is maximally pooled into a fixed size feature map for subsequent classification and candidate box regression operations;

[0088] (4) The candidate region is operated respectively. The classification of the target object and the regression of the candidate box are realized through a fully connected layer, and the FCN full convolution operation is performed on each candidate region in a separate Mask branch to generate a Mask, thereby completing the segmentation and recognition task.

[0089] According to the training requirements and configuration conditions, the model and its configuration file are selected. In embodiment 1, the 50-layer Resnet residual network is selected as the backbone network, and the FPN feature pyramid network is selected as the feature extraction network. Therefore, the “configs / COCO-InstanceSegmentation / mask_rcnn_R_50_FPN_3x.yaml” configuration file is selected, and the model pre-training weight file required for use is downloaded from the official model_zoo. The training project program train.py is created, and the path and category of the training set and test set are registered in the program. The number of training set and test set images in the data set input during network training is 5040 and 2160 respectively. The configuration information of several main parameters in the Mask RCNN neural network is shown in Table 1. The learning rate decay is set to 0.1 times of the original value, i.e. 0.001, when the iteration step reaches 21000. According to the computer configuration condition, the number of samples per batch is defined as 8, and the complete training process needs to be learned and trained for 63000 batches. Through training, the positioning accuracy AP bbox of embodiment 1 is 83.662%, and the segmentation accuracy AP seg is 78.624%.

[0090] Table 1 Mask RCNN neural network parameter configuration

[0091]

[0092] In order to better illustrate the method mentioned in the specification, three additional embodiments are supplemented here to further illustrate the specific implementation of the present application by changing the network structure configuration of the neural network model parameters and the model.

[0093] In example 2, the learning rate decay step is changed based on example 1. When the training iteration step reaches 31500 and 50400, i.e., the training period reaches 50 and 80 times, the learning rate is decayed once, and the learning rate decay coefficient is 0.1. Then, the learning rate is changed to 0.001 and 0.0001 when the step reaches 31500 and 50400, respectively.

[0094] In example 3, the training backbone network is changed to a 101-layer Resnet residual network, and the feature extraction network remains the FPN feature pyramid network. The configuration file is selected as “configs / COCOInstanceSegmentation / mask_rcnn_R_101_FPN_3x.yaml”, and the remaining training parameters are consistent with example 1.

[0095] In example 4, the number of clusters K=5 in the K-means clustering algorithm is changed to automatically generate five anchor aspect ratios, which are [0.7, 1.0, 1.2, 1.5, 2.0], respectively. The Mask RCNN neural network structure and the configuration of the remaining parameters are the same as example 1.

[0096] According to the configuration of the four examples, the training is performed, and after the training is completed, the performance of the neural network model for waterlogging detection is evaluated. The model performance comparison of the four examples is shown in Table 2.

[0097] Table 2 Comparison of Mask RCNN neural network training results of four examples

[0098]

[0099] By comparing the four examples, it can be seen that by changing the model parameters and configuration, the model training effect will be different. In example 3, only by changing the backbone network configuration, the best training result is obtained among the four examples, but the time used in the entire training process will also be significantly lengthened. In order to obtain better model recognition effect, subsequent further adjustment of training parameters is needed, and repeated training is performed until the model training effect reaches the optimal value, i.e., the training is completed, and the final model training weight file model_final.pth is generated.

[0100] S6, based on the trained weight file in S5, input the test image or video into the detection project to extract the image waterlogging features and obtain the waterlogging range detection and recognition result;

[0101] A detection project file is created to write the category information of the detection target and the model inference confidence threshold, and the model weight file is loaded with the trained weight file in S5. By inputting commands in the terminal, the waterlogging area detection and recognition of multiple input data sources is performed, the model will perform inference and draw the prediction box and mask corresponding to the area with an inference score higher than the confidence threshold, thereby outputting the image or video with the waterlogging range mask, and visualizing the prediction result.

[0102] The multiple input data sources include images, videos, or network monitoring cameras.

[0103] The four embodiments in the specification refer to the demo / demo.py file in the Detectron2 source code to create the prediction result visualization program predict.py, write the dataset category information defined in the train.py during the model training process and the model threshold parameters, and load the model_final.pth weight file pre-trained in S5. The Detectron2 framework allows detection and recognition of target objects through multiple input sources such as images, videos, or network monitoring cameras, visualizes and saves the prediction results. Using “--input” followed by the image or file path, the image or file can be detected and recognized; using “--video-input”, the video can be waterlogging predicted; using “--webcam”, the network camera can be run.

[0104] In the embodiments, the images to be detected are placed in the img / test folder created under the Detectron2 root directory, and the command “python predict.py--input img / test--output result” is input in the terminal, that is, the images in the test folder are batch detected and recognized, and the recognized image results are output to the result folder. The recognition speed of each image is determined by the computer configuration, but is generally less than 1 second or within a few seconds, with very fast detection speed, which can realize the rapid recognition of waterlogging range and can further develop and develop related software and application programs according to this method.

[0105] The same or similar symbols and notations mentioned in the description of the present application represent the same or similar physical meanings or have the same or similar functions, and the legends used in the present application are only for better explanation of the present application, and the applicability of the present application is not limited thereto. Any equivalent changes, modifications or evolutions of the above-mentioned embodiments made by those skilled in the art using the technical solutions of the present application are still within the scope of the technical solutions of the present application.

[0106] The application is a road waterlogging area identification method based on data mining and deep learning. According to the examples and related steps provided in the specification, it can be used for monitoring the waterlogging process in urban areas. The application uses network crawler and data mining technology to obtain network images from network social media and crowdsourcing data platforms as data sources, effectively utilizes social public data resources, reduces the difficulty of obtaining waterlogging image data, and realizes low-cost acquisition of massive image data based on big data. Through deep learning target detection algorithm, the characteristic information of urban waterlogging can be learned, and the waterlogging range can be quickly and accurately detected and identified, which promotes the rapidity, efficiency and safety of the urban waterlogging monitoring process, ensures the real-time, effectiveness and accuracy of information acquisition in the urban waterlogging monitoring process, and provides an effective way and idea for the orderly development of waterlogging disaster emergency management and the improvement of the ability of relevant departments and the public to respond to sudden urban waterlogging disaster events.

Claims

1.A method for identifying a road waterlogging area based on data mining and deep learning, characterized by, It comprises the following steps: S1, through network crawler and data mining technology, obtain urban waterlogging image from internet big data platform, construct waterlogging image database, and establish initial waterlogging image dataset through screening; S2, pre-process the images in the initial waterlogging image dataset, and draw the boundaries and labels of the visible waterlogging range in the dataset; S3, use image processing algorithm to simultaneously enhance the data of images and labels, expand the dataset, and divide the labeled dataset into training set and test set, and convert it into a dataset format readable by neural network model; S4, use K-means clustering algorithm to cluster the target bounding box of the training set, automatically generate a set of anchor boxes for the custom dataset, adjust the size and aspect ratio of the anchor boxes in the model configuration file, configure the basic environment of deep learning image recognition and install the target detection framework, then use K-means clustering algorithm to cluster the labeled target bounding box in the training set; by adjusting the K value, the bounding box samples are clustered into K clusters, so as to find the best number and size of anchor boxes, and the aspect ratio of the corresponding anchor is input into the target detection model configuration file; S5, input the labeled training set and validation set into the Mask RCNN instance segmentation model for learning, and evaluate the waterlogging detection performance of the neural network model after training, and repeatedly train by adjusting the neural network model training parameters until the model training effect reaches the optimal, that is, the model training is completed, and the final model training weight file is obtained; S6, based on the weight file trained in S5, input the test image or video into the detection project to extract the image waterlogging features and obtain the waterlogging range detection and recognition result. 2.The road waterlogging area recognition method based on data mining and deep learning according to claim 1, characterized in that, In step S1, the internet big data platform includes network social media and crowdsourcing data platform; Formulate information retrieval keywords, or select specific rainfall events as retrieval range, then use network crawler and data mining technology to retrieve urban road waterlogging images and related location and time information from internet big data platform, and download them to local database for storage and arrangement; screen and extract waterlogging images from the image database to form the initial urban road waterlogging image dataset. 3.The road waterlogging area recognition method based on data mining and deep learning according to claim 1, characterized in that, In step S2, it specifically includes the following contents: S2.1, use Opencv computer vision image processing library to perform image operation on the initial waterlogging image, including cropping, size expansion or scaling, and adjust it to a uniform resolution to facilitate subsequent image processing and data label making; S2.2, use Labelme data labeling software to extract waterlogging area features, import the image into Labelme, select create polygon to start labeling, draw the boundary contour and connect the lines of the visible waterlogging target in the image by drawing points one by one, and finally connect the beginning and end to generate a closed polygon that just covers the waterlogging area, and assign classification label information to the generated closed polygon, thereby creating a json label file in Labelme format corresponding to each image. 4.The road waterlogging area recognition method based on data mining and deep learning according to claim 1, characterized in that, In step S3, new waterlogging images are generated by using image processing algorithms in the Opencv library and Augmentor semantic segmentation data augmentation methods to realize data augmentation of the image dataset and improve the diversity and generalization ability of the dataset. The image processing method includes a color transformation method and a geometric transformation method. The color transformation method includes adjusting contrast, changing brightness, modifying RGB values, adding noise, and blackening or replacing the background area. The geometric transformation method includes enlarging or reducing the image, scale transformation, image translation, flipping, and rotation. Meanwhile, the image labels generated in step S2 are subjected to corresponding transformation and enhancement processing: if only color transformation data augmentation is performed without involving geometric position transformation, the label position information remains unchanged; if geometric transformation data augmentation is performed, the label also needs to change its position corresponding to the image transformation, as follows: By transforming the coordinates of the marked points in the label file corresponding to the image transformation method, or by using data augmentation methods to transform the mask image converted from the label file, the labels corresponding to the new images are generated, and the number of images in the dataset is doubled. 5.The road waterlogging area recognition method based on data mining and deep learning according to claim 1, characterized in that, After generating more image data through data augmentation, all images with information labels and corresponding labels are divided into training and test sets according to the proportion, and all labels corresponding to the training and test set images are batch-converted using a data set format conversion tool, so that they can be read and further used by the target detection framework. 6.The road waterlogging area identification method based on data mining and deep learning according to claim 5, characterized in that, In step S4, the algorithm flow for K-means clustering calculation of anchor boxes is as follows: (1) Select the number of clusters K; (2) Randomly select K cluster boxes as initial anchor boxes; (3) Using the IOU value as a metric, the IOU value of each anchor box and each target bounding box is calculated; the IOU value is the intersection over union of the anchor box and the target bounding box, and the value range is [0, 1]; when calculating the IOU, it is assumed that the top left corners of all boxes are at the origin, and the size of the anchor is , and the size of the bounding box is , then Since a larger IOU value is better in image recognition, in order to make the metric value smaller and the similarity larger, a distance parameter d is defined, so that each target bounding box is assigned to the anchor with the smallest distance parameter d error. The distance parameter d is taken as: wherein, and are the bottom and height of the anchor box, respectively, and are the bottom and height of the target bounding box, respectively, is the intersection over union of the anchor box and the target bounding box. (4) Calculate the median or mean size of the height and width of all target bounding boxes in each cluster, and recalculate the new cluster center as the new anchor box size to update the anchor; (5) Repeat the above steps until the anchor no longer changes, meets the convergence requirement, or reaches the maximum number of iterations. 7.The road waterlogging area recognition method based on data mining and deep learning according to claim 1, characterized in that, In step S5, select a neural network model and its configuration file according to the training requirements and configuration conditions, and download the model pre-training weight file needed for use from the network; Register the paths and categories of the training set and test set in the training of the neural network model, define the training parameter information, including the basic learning rate, learning rate decay, batch size, iteration number, training period number, and start the neural network training; After training, evaluate the waterlogging detection effect of the neural network model, and further adjust various neural network model parameters and repeat the training multiple times until the model training effect reaches the optimal value, i.e., the training is completed, and the final model training weight file is generated. 8.The road waterlogging area recognition method based on data mining and deep learning according to claim 1, characterized in that, In step S6, a detection item file is created, the category information of the detection target and the model inference confidence threshold are written, and the model weight file is loaded with the weight file trained in step S5; through inputting a command in the terminal, the waterlogged area detection and recognition of multiple input data sources are performed, the model will perform inference and draw the prediction box and the mask corresponding to the area with the inference score higher than the confidence threshold, thereby outputting the image or video with the waterlogged area mask, and visualizing the prediction result. 9.The road waterlogging area identification method based on data mining and deep learning according to claim 8, characterized in that, The multiple input data sources include images or videos.

Images