Intelligent detection method and system of underground voids based on multi-dimensional ground penetrating radar images

Abstract

The present invention discloses an intelligent detection method and system for underground cavities based on multi-dimensional ground penetrating radar images, which selectively selects different deep learning networks to detect and segment B‑scan and C‑scan images respectively, fully integrates the different disease information reflected by the two, and performs information matching based on the proposed disease screening rules to exclude pipes and cracks from all diseases, thereby improving the automatic detection accuracy of cavities. At the same time, compared with existing technologies, this method can output three-dimensional information of cavity diseases during implementation, provide guidance for road maintenance and repair work, and make it more convenient for road maintenance personnel to analyze and plan maintenance from a global perspective. It aims to combine B‑scan ground penetrating radar images and C‑scan ground penetrating radar images, exclude other abnormal situations with the same characteristics as cavity radar images, reduce the false detection rate, and improve the detection accuracy of underground cavities. In addition, based on B‑scan ground penetrating radar images and C‑scan ground penetrating radar images, the spatial information of cavity diseases is output to provide guidance for road maintenance and repair work.

Description

Technical Field

[0001] The present invention belongs to the technical field of underground disease detection, and in particular relates to an intelligent underground cavity detection method and system based on multi-dimensional ground penetrating radar images. Background Art

[0002] As the years of operation increase, the huge scale of transportation infrastructure has brought huge challenges to maintenance and repair. Road underground diseases are highly hidden and have diverse causes. If they are not treated in time and allowed to develop, they will cause serious road surface diseases, greatly increase maintenance costs, and even affect people's travel safety and property safety in serious cases.

[0003] Traditional underground disease detection methods rely heavily on core sampling. Although this method has high accuracy, it is poorly representative, destructive to road structures, and will affect the normal operation of traffic flow. The most popular detection equipment at present is ground penetrating radar, which can detect the internal structure of roads over a large area. Professionals can determine the type and location of the disease by analyzing the abnormal reflection wave characteristics on the radar image. Compared with the traditional destructive detection ground penetrating radar method, it can detect the underground structure of roads over a large area. Deep learning technology has made great progress in the field of target recognition in recent years. It can realize the automatic positioning and classification of objects and output relevant information of objects.

[0004] Cavity is one of the main underground diseases. It is caused by factors such as uneven settlement of roadbed, insufficient construction compaction, and traffic load. If it is not discovered and repaired in time, it will cause road collapse, seriously affecting driving safety. The dielectric constant of underground diseases is significantly different from that of pavement materials or roadbed materials. It will show abnormal features on the ground penetrating radar B-scan. Currently, some scholars use deep learning technology to identify B-scan images. Based on a data set with labels, the model trained using a deep learning algorithm can realize the automatic identification of underground diseases. Commonly used target recognition algorithms include one-stage recognition algorithms such as YOLOv1~v5 and SSD, and two-stage algorithms such as RCNN and Faster-RCNN. Although deep learning can detect abnormal features on B-scan images, it also has the following shortcomings for underground cavity recognition:

[0005] 1. Due to the material, size and other properties of the underground material itself, other foreign objects or diseases will also produce the same characteristics. For example, underground pipelines and cracks under high-frequency antennas will form hyperbolic features on the B-scan image, which will cause misjudgment of the deep learning model;

[0006] 2. Currently, B-scan ground penetrating radar images are commonly used to train deep learning models. Since B-scan images can only represent a single profile, after the B-scan images are detected by the model, only the depth information of the cavity disease can be obtained, and its horizontal distribution information cannot be obtained, making it difficult to provide specific cavity spatial distribution information for road maintenance and repair work.

[0007] 3. Currently, the abnormal feature recognition of ground penetrating radar images based on computer vision technology often only uses one algorithm, target detection or semantic segmentation, without combining the two to obtain more information, so the advantages of the two algorithms are not integrated. Summary of the invention

[0008] The purpose of the present invention is to overcome the above-mentioned shortcomings of the prior art and provide an intelligent underground cavity detection method and system based on multi-dimensional ground penetrating radar images to solve the problem that it is difficult to obtain real underground cavity diseases in the prior art.

[0009] In order to achieve the above object, the present invention adopts the following technical solutions:

[0010] The intelligent detection method of underground cavities based on multi-dimensional ground penetrating radar images includes the following steps:

[0011] Step 1, acquiring multiple multi-channel B-scan images and C-scan images of the road surface;

[0012] Step 2: Select B-scan images with disease features from the B-scan images to form a B-scan abnormal feature data set, and mark all diseases in the B-scan abnormal feature data set as holes; select C-scan images with disease features from the C-scan images to form a C-scan abnormal feature data set, and mark all diseases in the C-scan abnormal feature data set as holes; divide the B-scan abnormal feature data set and the C-scan abnormal feature data set into a training set, a validation set, and a test set;

[0013] Step 3, based on the training set in the B-scan abnormal feature data set, a B-scan hole target recognition model is constructed, and the B-scan hole target recognition model is a YOLOv5 network; based on the training set in the C-scan abnormal feature data set, a C-scan hole segmentation model is constructed, and the C-scan hole segmentation model is a U-Net network;

[0014] Step 4, detecting the validation set in the B-scan abnormal feature data set through the B-scan void target recognition model, and outputting B disease information, wherein the B disease information includes the B disease type, the B disease center point pile number, and the B depth distribution range; segmenting the validation set in the C-scan abnormal feature data set through the C-scan void segmentation model, and outputting C disease information, wherein the C disease information includes the C disease type, the C disease area center pile number, and the C width-to-length ratio;

[0015] Step 5, match the center pile number of the disease B and the centroid pile number of the disease area C. If they match, it is determined that there is a disease. The disease is determined to be a cavity through the depth distribution range of B and the width-to-length ratio of C, and the cavity is detected.

[0016] A further improvement of the present invention is:

[0017] Preferably, in step 2, the B-scan image is processed by direct wave removal, background removal, vertical band filtering and gain operation in sequence before selecting the B-scan image with disease characteristics.

[0018] Preferably, in step 2, the B-scan image is annotated by Labelimg, and the disease is annotated with a rectangular frame containing the entire hyperbolic abnormal feature; the C-scan image is annotated by Labelme, and the disease is annotated with a polygonal frame containing the entire abnormal color area.

[0019] Preferably, in step 3, the process of constructing the B-scan hole target recognition model includes: image input, feature extraction, feature fusion and target prediction and automatic weight adjustment; the CIOU loss in the YOLOv5 network is replaced by the EIOU loss;

[0020] The process of building a C-scan cavity segmentation model includes: image input, feature extraction, feature fusion, target segmentation and automatic adjustment of model weights.

[0021] Preferably, in step 4, the data of the verification set in the B-scan abnormal feature data set is input into the B-scan void target recognition model, and a ground penetrating radar image and a text file with a marked frame are output, wherein the text file contains the disease type, the coordinates of the disease center point, and the length and width of the detection frame; the actual length of the image is calculated according to the image size and the corresponding pile number; the B disease center point pile number, B actual length and B depth distribution range are calculated according to the coordinates of the disease center point, the length and width of the detection frame, the actual pile number of the monitored section and the actual length of the image.

[0022] Preferably, in step 4, the data of the validation set in the C-scan abnormal feature data set is input into the C-scan void segmentation model to obtain a binary image, in which the background grayscale value is 0 and the diseased area grayscale value is 1; the centroid pile number of the diseased area in the binary image is calculated, and the C width-to-length ratio is calculated by MATLAB.

[0023] Preferably, the matching process is: traverse the centroid pile number of the C disease area, if the centroid pile number of the B disease area is the same as that of the C disease area or the difference is ≤2cm, it is determined that the C disease area and the B disease area recognize each other and determine that there is a disease.

[0024] Preferably, in step 5, a new column is obtained based on the B depth distribution range column and the C width ratio column, each of which is the C average width-to-length ratio; and the disease type is determined through the new column.

[0025] Preferably, the specific process of determining the type of disease through the new column is: C average width-to-length ratio less than 1.5 is considered as a void, and C average width-to-length ratio greater than 5 is considered as a pipe or crack.

[0026] An intelligent underground cavity detection system based on multi-dimensional ground penetrating radar images, comprising:

[0027] An image acquisition unit, used for acquiring a plurality of multi-channel B-scan images and C-scan images of a road surface;

[0028] A data acquisition unit is used to select B-scan images with disease features from B-scan images to form a B-scan abnormal feature data set, and mark all diseases in the B-scan abnormal feature data set as holes; select C-scan images with disease features from C-scan images to form a C-scan abnormal feature data set, and mark all diseases in the C-scan abnormal feature data set as holes; divide the B-scan abnormal feature data set and the C-scan abnormal feature data set into a training set, a validation set, and a test set;

[0029] A model building unit is used to build a B-scan hole target recognition model based on a training set in a B-scan abnormal feature data set, wherein the B-scan hole target recognition model is a YOLOv5 network; and to build a C-scan hole segmentation model based on a training set in a C-scan abnormal feature data set, wherein the C-scan hole segmentation model is a U-Net network;

[0030] The training unit is used to detect the validation set in the B-scan abnormal feature data set through the B-scan void target recognition model, and output B disease information, wherein the B disease information includes the B disease type, the B disease center point pile number, and the B depth distribution range; segment the validation set in the C-scan abnormal feature data set through the C-scan void segmentation model, and output C disease information, wherein the C disease information includes the C disease type, the C disease area center pile number, and the C width-to-length ratio;

[0031] The detection unit is used to match the center pile number of the disease B with the centroid pile number of the disease area C. If they match, it is determined that the disease exists. The disease is determined to be a cavity through the depth distribution range B and the width-to-length ratio C, and the cavity is detected.

[0032] Compared with the prior art, the present invention has the following beneficial effects:

[0033] The present invention discloses an intelligent detection method and system for underground cavities based on multi-dimensional ground penetrating radar images. Different deep learning networks are selected to detect and segment B-scan and C-scan images respectively, and the different disease information reflected by the two images is fully integrated. Information matching based on the proposed disease screening rules can exclude pipes and cracks from all diseases, thereby improving the automatic detection accuracy of cavities. At the same time, compared with existing technologies, this method can output three-dimensional information of cavity diseases during implementation, provide guidance for road maintenance and repair work, and make it more convenient for road maintenance personnel to analyze and plan maintenance from a global perspective. It aims to combine B-scan ground penetrating radar images and C-scan ground penetrating radar images, exclude other abnormal situations with the same characteristics as cavity radar images, reduce the false detection rate, and improve the detection accuracy of underground cavities. In addition, based on B-scan ground penetrating radar images and C-scan ground penetrating radar images, the spatial information of cavity diseases is output to provide guidance for road maintenance and repair work. BRIEF DESCRIPTION OF THE DRAWINGS

[0034] Figure 1 It is a schematic diagram of the working process of an embodiment of the present invention;

[0035] Figure 2 It is a three-dimensional ground penetrating radar map;

[0036] Figure 3 It is the B-scan image annotation map;

[0037] Figure 4 It is the C-scan image annotation map;

[0038] Figure 5 It is the disease detection XML file;

[0039] Figure 6 This is the YOLOv5s network structure diagram;

[0040] Figure 7 It is the UNet network structure diagram;

[0041] Figure 8 Disease detection results chart. DETAILED DESCRIPTION

[0042] The present invention is further described in detail below with reference to the accompanying drawings and specific embodiments:

[0043] In the description of the present invention, it should be noted that the orientations or positional relationships indicated by the terms "center", "upper", "lower", "left", "right", "vertical", "horizontal", "inside", "outside", etc. are based on the orientations or positional relationships shown in the drawings, and are only for the convenience of describing the present invention and simplifying the description, rather than indicating or implying that the device or element referred to must have a specific orientation, be constructed and operated in a specific orientation, and therefore cannot be understood as limiting the present invention; the terms "first", "second", and "third" are only used for descriptive purposes and cannot be understood as indicating or implying relative importance; in addition, unless otherwise clearly specified and limited, the terms "installed", "connected", and "connected" should be understood in a broad sense, for example, it can be a fixed connection or a detachable connection; it can be directly connected or indirectly connected through an intermediate medium, and it can be the internal communication of two elements. For those of ordinary skill in the art, the specific meanings of the above terms in the present invention can be understood according to specific circumstances.

[0044] The present invention discloses an underground cavity detection method based on multi-dimensional ground penetrating radar images, comprising the following steps:

[0045] Step S1, using a vehicle-mounted three-dimensional ground penetrating radar to collect data and obtain a multi-channel B-scan image and its corresponding C-scan image;

[0046] Step S2, selecting B-scan images with typical disease features to form a B-scan abnormal feature data set, and marking the abnormal features in the image; selecting C-scan images with typical disease features to form a C-scan abnormal feature data set, and marking the abnormal areas in the image;

[0047] Step S201, the B-scan image is sequentially subjected to direct wave removal, background removal, vertical band filtering, and gain operation to make the waveform abnormal area visible, and then the image contrast is increased to make the abnormal features more obvious. The C-scan image is displayed in Jet color system, and the abnormal area is more obvious in this display state.

[0048] Step S202, use Labelimg software to annotate the B-scan image, and use a rectangular frame containing the entire hyperbolic abnormal feature to annotate the defect. The defect may be a cavity, a pipeline, or a crack, but they are all uniformly annotated as a cavity.

[0049] Step S203, use Labelme software to annotate the C-scan image, and use a polygonal box containing the entire abnormal color area to annotate the defect. The defect may be a cavity, a pipeline, or other defects, but they are all uniformly annotated as a cavity, and a Json file is output.

[0050] Step S3, based on the B-scan abnormal feature data set, a B-scan void target recognition model is constructed, and based on the C-scan abnormal feature data set, a C-scan void segmentation model is constructed;

[0051] In step S301, since the amount of data is sufficient (B-scan radar images come from multiple channels, and C-scan images come from different depths), the data set is divided into a training set, a validation set, and a test set according to the ratio of 8:1:1 using the holdout method.

[0052] Step S302, use the YOLOv5 network to train the target detection model to detect the disease in the B-scan, wherein the YOLOv5 network performs Mosaic data enhancement, adaptive anchor box calculation, and adaptive image scaling on the image at the input end, the YOLOv5 skeleton uses the C3 structure, the neck part uses FPN+PAN for feature fusion, and the initial GIOU is replaced by EIOU to calculate the loss and update the weight.

[0053] Step S303, use the U-Net network to train the target segmentation model to segment the defects in the C-scan. U-Net is a typical encoder-decoder structure, whose encoder includes 4 modules, each module includes two 3×3 convolution modules and a maximum pooling layer, which is used to extract deep features; its decoder also includes 4 modules (the entire network is symmetrical), each module includes two 3×3 convolution modules and an upsampling module, which enlarges the feature map containing deep features to the original size and fuses it with the feature map with low-level features obtained by the decoder part in the channel dimension, fusing high-level semantic information with low-level visual information.

[0054] Step S4, load B-scan data, use the disease detection model to detect diseases and output disease information; apply disease screening rules based on the disease information to exclude the same disease on different images, and output the final disease information. Load C-scan data, use the disease segmentation model to segment the disease area on the C-scan image and output disease information.

[0055] Step S401, import the B-scan data set into the disease detection model trained in S3 for detection. After the detection is completed, the ground penetrating radar image with a marked frame and a text file containing the disease type, the coordinates of the disease center point, and the length and width of the detection frame are automatically output. According to the image size and the corresponding stake number, the actual length corresponding to the image is calculated; according to the coordinates of the disease center point, the length and width of the detection frame, the actual stake number of the detection section, and the actual length corresponding to a single image, the B disease center point stake number, B actual length, and B depth distribution range of the disease are calculated. At the same time, the channel number, disease type, disease center point stake number, and depth distribution range information are automatically output to the B-scan image information table, and the four types of information each occupy a separate column in the table.

[0056] Step S402, import the C-scan data set into the disease segmentation model for segmentation, and obtain a binary image with only the disease area and the background, where the background gray value is 0 and the disease area gray value is 1. Since some small areas will be misjudged and segmented during disease segmentation, it is necessary to exclude these misjudged areas and use MATLAB to solve the maximum connected area in the binary image. Use MATLAB to solve the maximum connected area in the binary image and solve its minimum circumscribed rectangle size and the minimum circumscribed rectangle width-to-length ratio, which is the C width ratio.

[0057] Step S403, according to the original size of the image and the actual stake number corresponding to the starting point and the end point of the image, determine the actual stake number of the center of the diseased area. Since the diseased area is mostly an irregular polygon and does not have a geometric center, the center of gravity of the diseased area is selected as the solution point, and the OpenCV library in Python is used to automatically calculate the center of gravity of the diseased area in the binary image, and the actual stake number of the center of gravity is solved according to the actual stake number corresponding to the image. After solving various information of the diseased area, output C disease type, C disease area center of gravity stake number, C width-to-length ratio to the C-scan image information table, and the three types of information each occupy a separate column in the table.

[0058] Step S5, integrate the output information of B-scan and C-scan, and use the disease screening rules to output the cavity disease detection results. The automatic screening information is realized based on the macro function of the Excel table, where the macro is a collection of VBA codes that implement a single function or complex functions. The code you want to implement can be used to implement a set of commands with one click.

[0059] Step S501: In this step, the hyperbolic features generated by the same disease in different channels are eliminated with the help of the Excel macro function, and only the information of one channel is retained to represent the original suspected disease.

[0060] Step S502, write a program to add the contents of each column of the B-scan image information table and the C-scan image information table to a new table. At this time, the new table has six columns, namely, B disease type, B disease center point pile number, B disease depth distribution range, C disease type, C disease area center of gravity pile number, and C width-to-length ratio.

[0061] Match the pile number of the center point of the B disease with the pile number of the center of gravity of the C disease area to determine whether the pile numbers are mutually recognized (the pile numbers are the same or the difference between the front and back does not exceed 0.2m). Traverse the column of the center of gravity pile number of the C disease area. If the B-scan suspected disease and the C-scan suspected disease pile number are not mutually recognized, it is determined to be clutter or a small disease. Delete the three columns of information in the table: B disease type, B disease center point pile number, and B disease depth distribution range of the B-scan suspected disease that has not found mutual recognition information; if the column of the center of gravity pile number of the C disease area is found to have a C-scan suspected disease that is mutually recognized with the B-scan suspected disease, it is determined that there is a disease, which may be a cavity, pipeline, or crack, and save the corresponding six columns of information.

[0062] Since multiple images are exported from the same location when C-scan images are exported, the disease is three-dimensional, and C-scan images are exported at a certain distance. Therefore, after the initial judgment, one B-scan suspected disease corresponds to multiple C-scan suspected diseases at different depths. Each C-scan image will correspond to an aspect ratio. The average of the aspect ratios of multiple C-scan images represents the aspect ratio of the disease. Therefore, according to the information of the B depth distribution range column and the C aspect ratio column, the average aspect ratio of the minimum circumscribed rectangle of the C-scan suspected disease within the distribution range is calculated in a new column to represent the overall shape of a C-scan. The new column is the new C average aspect ratio column. After the calculation is completed, the C aspect ratio column and the C disease type column are deleted. At this time, the list shows that one B-scan suspected disease corresponds to one C-scan suspected disease. At this time, the table includes five columns: B disease type, B disease center point pile number, B disease depth distribution range, C disease area center pile number, and C average aspect ratio.

[0063] The C average width-to-length ratio column was screened, and the average width-to-length ratio less than 1.5 was identified as a void, and the average width-to-length ratio greater than 5 was considered a pipe or crack, and the cells with the B disease type as "void" were changed to "pipe or crack". The information finally retained in the table includes the disease type, the pile number of the center point of the disease, the distribution range of the disease depth, the pile number of the center of gravity of the disease area, and the average width-to-length ratio, which distinguishes the void disease from the pipeline or crack, and improves the accuracy of void detection.

[0064] Embodiment 1

[0065] like Figure 1As shown, the present invention provides an underground cavity intelligent detection method based on multi-dimensional ground penetrating radar images, comprising the following steps:

[0066] Step S1, using a vehicle-mounted three-dimensional ground penetrating radar to collect data ( Figure 1 ) to obtain a multi-channel B-scan image and its corresponding C-scan image.

[0067] Step S101, determine the detection range according to the road material type, road structure layer combination and layer thickness, and the type of disease to be detected, select a three-dimensional ground penetrating radar with a suitable frequency antenna, and install the three-dimensional ground penetrating radar on the rear connection frame of the detection vehicle, such as Figure 2 . Adjust the ground penetrating radar acquisition parameters, including setting time window, sampling mode, number of sampling points, sampling interval and other parameters. The inspection vehicle drives on the road at normal speed to complete the internal structure condition inspection.

[0068] Step S2, selecting B-scan images with typical disease features to form a B-scan abnormal feature data set, and marking the abnormal features in the image; selecting C-scan images with typical disease features to form a C-scan abnormal feature data set, and marking the abnormal areas in the image.

[0069] Step S201, using computer-side ground penetrating radar data processing software to process the data as follows:

[0070] (1) Vertical bandpass filtering: suppresses radar low-frequency drift and high-frequency noise, retains the main frequency part of the radar signal to improve the signal-to-noise ratio, highlights the energy of the layer reflection signal, and improves the resolution of the radar profile signal.

[0071] (2) Zero line correction: remove the first arrival waves received by the receiving antenna that do not enter the ground and are transmitted through the air.

[0072] (3) Background removal: Select an area with obvious horizontal interference signals in the radar profile, calculate the average of all the data channels in this section as the background noise, and calculate the difference between all the channels in the radar profile and the background noise to achieve the purpose of removing the background noise. The background noise calculation formula is as follows;

[0073]

[0074] Where: N 1 is the starting channel number of the profile background noise; N 2 is the termination channel number of the profile background noise.

[0075] (4) Gain: Balance the energy of effective waves in each time period on the radar profile to facilitate the tracking of effective waves and the comparison of weak signals. The gain process is implemented by the following formula:

[0076]

[0077] Where: y(t)—radar record before automatic gain;

[0078] P(t)—automatic gain weight function;

[0079] y(t)—Radar record after automatic gain.

[0080] Step S202: A typical three-dimensional ground penetrating radar device uses an array antenna with more than 10 channels, and uses the ground penetrating radar data processing software on the computer to improve the contrast of the B-scan ground penetrating radar image to make the hyperbolic features more obvious; the B-scan ground penetrating radar image is exported from all channels at an interval of 5m. The ground penetrating radar data processing software on the computer is used to set the C-scan image color system to Jet format, and automatically export the C-scan ground penetrating radar image at a horizontal interval of 5m and a depth interval of 1cm.

[0081] Step S203, selecting images containing hyperbolic abnormal features from all exported B-scan data to form a B-scan abnormal feature data set, which is a disease feature; selecting images containing color abnormal areas from all exported C-scan data to form a C-scan abnormal feature data set.

[0082] Step S204, use Labelimg software to annotate the selected ground penetrating radar B-scan image, use a rectangular frame that completely contains the hyperbolic abnormal features to calibrate the defects, and uniformly classify the defects as cavities, such as Figure 3 The saving format is PascalVOC, which means that an XML file containing the annotation information is automatically generated each time annotation is completed.

[0083] Step S205, use Labelme software to annotate the ground penetrating radar C-scan image, use a polygonal box that completely contains the abnormal color area to mark the disease, and uniformly classify the disease as a cavity, such as Figure 4 . Each time annotation is completed, a JSON file is generated, which contains the annotation information.

[0084] Step S3, using the YOLOv5s network and the B-scan abnormal feature data set to train a disease detection model, and using the UNet network and the C-scan abnormal feature data set to train a disease segmentation model;

[0085] Step S301: Since the data scale is large enough, the holdout method can be used to divide the data set. Each data set is sufficient to represent the distribution of all data. Therefore, the labeled ground penetrating radar B-scan data obtained in step S2 is divided into a training set, a validation set, and a test set in a ratio of 8:1:1.

[0086] Step S302, select YOLOv5s deep learning network as the training framework, with a network depth of 0.33 and a network width of 0.5. The trained detection model has both speed and accuracy. The YOLOv5 network structure is as follows: Figure 6 As shown. The network implements model training through the following process:

[0087] (1) Image input: The algorithm uses the Mosaic data enhancement method to take 4 pictures, and splices them in a random scaling, random cropping and random arrangement manner, combining several pictures into one picture, which can enrich the data set, improve the network training speed and reduce the model memory. At the same time, the algorithm uses the adaptive image scaling method to unify the size of the input image, while adding the least black border, minimizing the redundancy caused by padding, and improving the network training speed.

[0088] (2) Feature extraction: YOLOv5s uses the C3 module as the main module of the skeleton. The C3 module divides the feature map generated by the CBL layer into two branches for feature learning. One branch directly performs a 1×1 convolution to halve the number of channels in the feature map. The other branch first performs a 1×1 convolution on the feature map to halve the number of channels, and then uses the Bottleneck residual block to extract more features. Finally, the feature maps of the two branches are spliced ​​to obtain a feature map with the same number of channels as the previous layer. The existence of the Bottleneck greatly reduces the number of parameters and calculations in the whole process. The SPPF module in the skeleton uses three maximum pooling layers with convolution kernel sizes of 5×5, 9×9, and 13×13 to extract features from the feature map, and finally splices it with the input feature map of the pooling layer to achieve the fusion of local features and global features.

[0089] (3) Feature fusion: YOLOv5s adopts the FPN and PAN feature pyramid structures, and fuses high-level features with more semantic information and less target information with low-level features with less semantic information and more target information through horizontal connections. It finally outputs three feature maps of different sizes to achieve multi-scale target detection.

[0090] (4) Target prediction and automatic adjustment of model weights: Calculate based on the original weights and feature maps, generate multiple prediction boxes on the original image, and use NMS (non-maximum suppression) to delete duplicate boxes. YOLOv5s uses EIOU loss to calculate the loss value between the prediction box and the real box. After each round of training, the YOLOv5s algorithm uses the BP back propagation algorithm to calculate the loss value and adjust the weight value until the loss value no longer decreases. The model obtained at this time is the final detection model.

[0091] (5) During the training process, select the YOLOv5s.pt file as the initial weight, and select the hyperparameters obtained by training based on the coco dataset as the initial hyperparameters. YOLOv5 uses a genetic algorithm to evolve hyperparameters, that is, define a fitness (weighted by precision, recall, map@0.5, and map@0.5:0.95) as the initial value for hyperparameter mutation. Each mutation selects the hyperparameter combination with the best previous mutation effect as the basis for the next mutation. The hyperparameter combination with the best fitness is obtained through repeated mutation of hyperparameters, and the number of mutations is set to 300. In addition, according to the different performance of computers, the batch size is set to 32 as a moderate value, the number of iterations epoch is set to 500, and the image size defaults to 640.

[0092] (6) Compared with the original YOLOv5s network, the YOLOv5 algorithm adopted in the present invention replaces the CIOU loss with the EIOU loss. Although the CIOU loss takes into account the overlapping area, center point distance, and aspect ratio of the bounding box regression. However, the difference in aspect ratio reflected by v in its formula is not the actual difference between the width and height and their confidence, so it sometimes hinders the effective optimization similarity of the model. The EIOU penalty term is based on the CIOU penalty term, and the influencing factor of the aspect ratio is separated to calculate the length and width of the target box and the anchor box respectively. The loss function includes three parts: overlap loss, center distance loss, and width and height loss. The first two parts continue the method in CIOU, but the width and height loss directly minimizes the difference in width and height between the target box and the anchor box, making the convergence faster. The calculation formula of the EIOU loss is as follows:

[0093] L EIOU =L IOU +L dis +L asp

[0094]

[0095] Among them, L IOU is the overlap area loss, L dis is the center point distance loss, L asp is the aspect ratio loss.

[0096] Step S303, select U-Net network to train the disease segmentation model. Compared with other segmentation networks, the biggest feature of U-Net is that it uses a U-shaped network. While using fewer training images, the segmentation accuracy is not bad. The structure diagram of the U-Net network is shown in the figure, and its training implementation process is as follows:

[0097] (1) The overall structure of the U-Net network is a U-shape. The left part is an encoder, also known as the contraction path. The encoder follows the basic structure of the convolutional neural network and repeatedly applies two 3×3 convolution units. Each convolution unit is followed by a ReLU activation function and a 2*2 maximum pooling layer. The pooling layer has a stride of 2 for downsampling. The main function of the encoder is to parse image information, extract image features, and obtain higher-order semantic information. The right part is a decoder, also known as the expansion path. Each step includes an upsampling of the feature map, followed by a 2×2 convolution unit, which halves the number of feature channels and is connected to the corresponding cropped feature map from the contraction path, followed by two 3*3 convolution units and a ReLU activation function. Since boundary pixels are lost in each convolution, cropping is necessary. In the last layer, a 1×1 convolution unit is used to map each 64-component feature vector to the required number of classes. The network has a total of 23 convolution layers.

[0098] (2) Image input: The U-Net network input is a 572×572 image with mirrored edges.

[0099] (3) Feature extraction: The feature extraction module is called the contraction path, which consists of four blocks. The image undergoes three effective convolutions and one downsampling after passing through each block. After each downsampling, the size of the feature map is reduced by half. After multiple downsampling, a feature map of size 32×32 is finally obtained.

[0100] (4) Feature fusion: The expansion path also consists of 4 blocks. Each block will reduce the number of channels of the input feature map by half through deconvolution (the last layer is slightly different), and at the same time, expand the size of the feature map to twice the original size through the upsampling layer, and then merge it with the feature map of the symmetrical contraction path on the left. The convolution operation of the expansion path still uses the effective convolution operation, and the size of the final feature map is 338×338.

[0101] (5) The decoder of U-Net fuses the feature maps of the downsampling process during the upsampling process, which can fuse the previous underlying features and improve the richness of the features. The fusion method used by U-Net is called Concat, which directly superimposes the feature maps of the same size at the channel level. Since the sizes of the feature maps of the left contraction path and the right expansion path are different, U-Net normalizes the feature maps of the contraction path by cropping them to the same size as the expansion path.

[0102] (6) Target segmentation: The segmentation problem is equivalent to a binary classification problem, that is, segmenting the target and the background. Therefore, after the final 1×1 convolution operation of the expansion path, the output is a two-channel feature map.

[0103] (7) Automatic adjustment of model weights: Loss calculation is performed based on the output segmentation map and label map. U-Net uses Dice loss to calculate the loss value of the output segmentation map and label map. Dice loss is suitable for binary segmentation of images and can alleviate the problem of imbalance in the number of positive and negative samples to a certain extent. The calculation formula of Dice loss is as follows:

[0104]

[0105] Among them, y i and are the label value and predicted value of pixel i respectively, and N is the total number of pixels.

[0106] The U-Net algorithm calculates the Dice loss value after each round of training and adjusts the weight value through back propagation until the loss value no longer decreases. The model obtained at this time is the final disease segmentation model.

[0107] (8) Training parameter settings: Set the rotation range to 0.2, the width transformation range to 0.05, the height transformation range to 0.05, the cropping range to 0.05, the zoom range to 0.05, the horizontal flip to “on”, the filling mode to “nearest neighbor filling”, the batch size to 32, the number of epochs to 200, and the early stopping mechanism to “on”.

[0108] Step S4, load B-scan data, use the disease detection model to detect diseases and output disease information; apply disease screening rules based on the disease information to exclude the same disease on different images, and output the final disease information. Load C-scan data, use the disease segmentation model to segment the disease area on the C-scan image and output disease information (the diseases mentioned above and below are only suspected diseases, which may actually be cavities, pipes, cracks or clutter).

[0109] Step S401, import the B-scan data set into the disease detection model trained in S3 for detection, and automatically output the ground penetrating radar image with a marked box after the detection is completed (such as Figure 8 ) and a text file containing the disease type, disease center coordinates, and detection frame length and width (such as Figure 5 ). According to the image size and the corresponding pile number, the actual length corresponding to the image is calculated; according to the coordinates of the center point of the disease, the length and width of the detection frame, the actual pile number of the detection section, and the actual length corresponding to a single image, the center point pile number, actual length, and depth distribution range of the disease are calculated. At the same time, the B channel number, B disease type, B disease center point pile number, and B depth distribution range information are automatically output to the B-scan image information table (hereinafter referred to as "B table"), and the four types of information each occupy a separate column in the table.

[0110] Step S402, import the C-scan data set into the disease segmentation model for segmentation, and obtain a binary image with only the disease area and the background, where the background gray value is 0 and the disease area gray value is 1. Since some small areas will be misjudged and segmented during disease segmentation, these misjudged areas need to be excluded. Use MATLAB to solve the maximum connected area in the binary image and solve its minimum circumscribed rectangle size. The implementation process is as follows:

[0111] (1) Add a property of type Bool to indicate whether the binary image has been visited (to avoid an infinite loop);

[0112] (2) Find the first non-zero pixel, push it into the stack and set its visited attribute to true;

[0113] (3) Taking whether the stack size is 0 as the termination condition, find the eight neighboring non-zero pixels adjacent to the top element of the stack and push them into the stack. After the end, delete the top element of the stack;

[0114] (4) When the stack is empty, it indicates that a connected area has been traversed, and it is necessary to continue to find the next non-empty and unvisited pixel as the starting point and repeat the previous step until all non-zero pixels have been visited;

[0115] (5) After all connected regions are solved, the connected region with the largest number of pixels is marked.

[0116] (6) In MATLAB, find the minimum enclosing rectangle of the largest connected area and calculate its aspect ratio based on the size.

[0117] Step S403, according to the original size of the image and the actual stake numbers corresponding to the starting point and the end point of the image, determine the actual stake number of the center of the diseased area. Since the diseased area is mostly an irregular polygon and does not have a geometric center, the center of gravity of the diseased area is selected as the solution point to solve the actual stake number of the center of gravity. The solution process is as follows:

[0118] (1) Traverse the folders that store all binary images, use the OpenCV library in Python to automatically calculate the center of gravity of the diseased area in the binary image, and automatically mark and display it on the original image;

[0119] (2) Calculate the relative position of the center of gravity of the diseased area in the whole world, and determine the actual pile number of the center of gravity based on the existing actual pile number information.

[0120] After solving various information of the diseased area, output C disease type, C disease area center of gravity pile number, C width-to-length ratio to the C-scan image information table (hereinafter referred to as "C table"), and the three types of information each occupy a separate column in the table.

[0121] Step S5, integrate the output information of B-scan and C-scan, and use the disease screening rules (see Table 1 and Table 2) to output the cavity disease detection results. The automatic screening information is realized based on the macro function of the Excel table, where the macro is a collection of VBA codes that implement a single function or complex functions, and the code you want to implement can be used to implement a set of commands with one click (the diseases mentioned above and below are only suspected diseases, and may actually be cavities, pipes, cracks or clutter).

[0122] Step S501, process the B table to eliminate redundant information and retain core information. Because the distribution range of cavity diseases is large, generally spanning multiple channels, the same hyperbolic features will appear on the B-scan radar images of adjacent channels. When the disease is detected, the multi-channel B-scan radar image is input, and the hyperbolic features of each channel will be detected. Therefore, the hyperbolic features generated by the same disease in different channels should be eliminated before the joint analysis with the C-scan information, and only the information of one channel should be retained to represent the original suspected disease. This step uses the Excel macro function to implement B table processing according to the following steps:

[0123] (1) Sort the table information by pile number from small to large, and the suspected disease information of the same or similar pile numbers will be gathered together;

[0124] (2) The sorted information is filtered according to the column of the center point of the disease and the column of the channel number. If a hyperbolic feature appears at the same position or at a position with a horizontal distance difference of no more than 0.1m on multiple adjacent channel images, it is determined to be the same suspected disease, which may be a cavity, pipeline, or crack. Only the middle channel is retained as the representative of the disease at this location, and the channel number, disease type, center point of the disease, and depth distribution range information of the middle channel of the suspected disease are retained accordingly. If only a single image has a hyperbolic feature and its adjacent channels do not have a hyperbolic feature, this situation is determined to be clutter or a small disease, and its information is deleted. Table B only retains the B disease type, B disease center point number, and B disease depth distribution range, which is called Table B1.

[0125] Table 1B-scan image information table information screening rules

[0126]

[0127] Step S502, write a program to add the contents of each column of the B-scan image information table and the C-scan image information table to a new table. At this time, the new table has six columns, namely, B disease type, B disease center point pile number, B disease depth distribution range, C disease type, C disease area center of gravity pile number, and C width-to-length ratio.

[0128] Match the pile number of the center point of the B disease with the pile number of the center of gravity of the C disease area to determine whether the pile numbers are mutually recognized (the pile numbers are the same or the difference between the front and back does not exceed 0.2m). Traverse the column of the center of gravity pile number of the C disease area. If the B-scan suspected disease and the C-scan suspected disease pile number are not mutually recognized, it is determined to be clutter or a small disease. Delete the three columns of information in the table: B disease type, B disease center point pile number, and B disease depth distribution range of the B-scan suspected disease that has not found mutual recognition information; if the column of the center of gravity pile number of the C disease area is found to have a C-scan suspected disease that is mutually recognized with the B-scan suspected disease, it is determined that there is a disease, which may be a cavity, pipeline, or crack, and save the corresponding six columns of information.

[0129] Since multiple images are exported from the same location when C-scan images are exported, after the initial judgment, one B-scan suspected disease corresponds to multiple C-scan suspected diseases of different depths. Therefore, according to the information of the B depth distribution range column and the C width-to-length ratio column, the average width-to-length ratio of the minimum circumscribed rectangle of the C-scan suspected disease within the distribution range is calculated in a new column to represent the overall shape of a C-scan. The new column is the C average width-to-length ratio column. After the calculation is completed, the C width-to-length ratio column and the C disease type column are deleted. At this time, the list shows that one B-scan suspected disease corresponds to one C-scan suspected disease. At this time, the table includes five columns: B disease type, B disease center point pile number, B disease depth distribution range, C disease area center pile number, and C average width-to-length ratio.

[0130] The C average width-to-length ratio column was screened, and the average width-to-length ratio less than 1.5 was identified as a void, and the average width-to-length ratio greater than 5 was considered a pipe or crack, and the cells with the B disease type as "void" were changed to "pipe or crack". The information finally retained in the table includes the disease type, the pile number of the center point of the disease, the distribution range of the disease depth, the pile number of the center of gravity of the disease area, and the average width-to-length ratio, which distinguishes the void disease from the pipeline or crack, and improves the accuracy of void detection.

[0131] Table 2 B-scan image and C-scan image matching rules

[0132]

[0133] Embodiment 2

[0134] This embodiment discloses an intelligent underground cavity detection method and system based on multi-dimensional ground penetrating radar images, which includes a data acquisition module, a data preprocessing module, a model training module, a disease detection and segmentation module, and an information matching module. The main equipment of the system is a vehicle-mounted three-dimensional ground penetrating radar and a computer. It includes the following steps:

[0135] Step S1, the acquisition module mainly completes the survey line layout and two-dimensional and three-dimensional ground penetrating radar data acquisition.

[0136] Step S2, the data preprocessing module mainly completes tasks such as image filtering, contrast adjustment, data export, data labeling, and disease image selection, and constructs B-scan disease image datasets and C-scan disease image datasets for the training module.

[0137] Step S3: The model training module mainly completes the training of the disease detection model and the disease segmentation model.

[0138] Step S4, the disease detection and segmentation module mainly completes the detection and segmentation of the multi-channel B-scan image set and the multi-depth C-scan image set and derives the corresponding disease location and size information.

[0139] Step S5, the information matching module mainly excludes non-cavity anomalies according to the suspected disease information matching rules to achieve the refinement of cavity disease information.

[0140] For the specific implementation of the above steps, please refer to the corresponding steps in Example 1.

[0141] Embodiment 3

[0142] This embodiment discloses a computer-readable storage medium. The computer-readable storage medium stores a computer program, which, when executed by a processor, performs the following steps:

[0143] Step S1, importing B-scan and C-scan ground penetrating radar image data sets and corresponding label sets, and automatically training disease detection models and disease segmentation models based on deep learning algorithms.

[0144] Step S2, importing the B-scan radar image and C-scan radar image to be detected and segmented, using the model trained in step S1 to perform detection and segmentation, and outputting a table containing the location and size information of suspected diseases.

[0145] Step S3, exclude non-cavity anomalies according to the suspected disease information matching rules, refine the cavity disease information, and improve the detection accuracy of the disease.

[0146] Embodiment 4

[0147] This embodiment discloses an electronic device. An electronic device includes a memory, a processor, a touch screen, a device housing, and an executable program. The electronic device runs the following steps when executing the program:

[0148] Step S1, importing B-scan and C-scan ground penetrating radar image data sets and corresponding label sets, and automatically training disease detection models and disease segmentation models based on deep learning algorithms.

[0149] Step S2, import the B-scan radar image and C-scan radar image to be detected and segmented, use the model trained in step S1 to detect and segment, and output a table containing the location and size information of suspected diseases. Images and text files can be opened and displayed through the display button.

[0150] Step S3: exclude non-cavity anomalies according to the suspected disease information matching rules, refine the cavity disease information, and improve the detection accuracy of the disease. The final cavity disease information table can be displayed on the display screen.

[0151] For the specific implementation methods and principles of each step in the above-mentioned embodiments 3 and 4, please refer to the specific description part of embodiment 1.

[0152] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principle of the present invention should be included in the protection scope of the present invention.

Claims

1. Intelligent detection method of underground voids based on multi-dimensional ground penetrating radar images, It is characterized in that The following steps are involved: Step 1, acquiring multiple multi-channel B-scan images and C-scan images of the road surface; Step 2: Select B-scan images with disease features from the B-scan images to form a B-scan abnormal feature data set, and mark all diseases in the B-scan abnormal feature data set as holes; select C-scan images with disease features from the C-scan images to form a C-scan abnormal feature data set, and mark all diseases in the C-scan abnormal feature data set as holes; divide the B-scan abnormal feature data set and the C-scan abnormal feature data set into a training set, a validation set, and a test set; Step 3, based on the training set in the B-scan abnormal feature data set, a B-scan hole target recognition model is constructed, and the B-scan hole target recognition model is a YOLOv5 network; based on the training set in the C-scan abnormal feature data set, a C-scan hole segmentation model is constructed, and the C-scan hole segmentation model is a U-Net network; Step 4, detecting the validation set in the B-scan abnormal feature data set through the B-scan void target recognition model, and outputting B disease information, wherein the B disease information includes the B disease type, the B disease center point pile number, and the B depth distribution range; segmenting the validation set in the C-scan abnormal feature data set through the C-scan void segmentation model, and outputting C disease information, wherein the C disease information includes the C disease type, the C disease area center pile number, and the C width-to-length ratio; Step 5, match the center pile number of the disease B and the centroid pile number of the disease area C. If they match, it is determined that there is a disease. The disease is determined to be a cavity through the depth distribution range of B and the width-to-length ratio of C, and the cavity is detected.

2. The intelligent underground cavity detection method based on multi-dimensional ground penetrating radar images according to claim 1, It is characterized in that In step 2, the B-scan image is processed by direct wave removal, background removal, vertical band filtering and gain operation in sequence before selecting the B-scan image with disease characteristics.

3. The intelligent underground cavity detection method based on multi-dimensional ground penetrating radar images according to claim 1, It is characterized in that In step 2, the B-scan image is annotated by Labelimg, and the disease is annotated with a rectangular box containing the entire hyperbolic abnormal feature; the C-scan image is annotated by Labelme, and the disease is annotated with a polygonal box containing the entire abnormal color area.

4. The intelligent underground cavity detection method based on multi-dimensional ground penetrating radar images according to claim 1, It is characterized in that In step 3, the process of constructing the B-scan hole target recognition model includes: image input, feature extraction, feature fusion, target prediction and automatic weight adjustment; the CIOU loss in the YOLOv5 network is replaced by the EIOU loss; The process of building a C-scan cavity segmentation model includes: image input, feature extraction, feature fusion, target segmentation and automatic adjustment of model weights.

5. The intelligent underground cavity detection method based on multi-dimensional ground penetrating radar images according to claim 1, It is characterized in that In step 4, the data of the validation set in the B-scan abnormal feature data set is input into the B-scan void target recognition model, and a ground penetrating radar image with a marked frame and a text file are output, wherein the text file contains the type of disease, the coordinates of the center point of the disease, and the length and width of the detection frame; the actual length of the image is calculated according to the image size and the corresponding pile number; the pile number of the center point of the B disease, the actual length of B, and the distribution range of the depth of B are calculated according to the coordinates of the center point of the disease, the length and width of the detection frame, the actual pile number of the monitored section, and the actual length of the image.

6. The intelligent underground cavity detection method based on multi-dimensional ground penetrating radar images according to claim 1, It is characterized in that In step 4, the data of the validation set in the C-scan abnormal feature data set is input into the C-scan cavity segmentation model to obtain a binary image, in which the background gray value is 0 and the diseased area gray value is 1; The centroid number of the defective area in the binary image is calculated, and the C width-to-length ratio is calculated by MATLAB.

7. The intelligent underground cavity detection method based on multi-dimensional ground penetrating radar images according to claim 1, It is characterized in that The matching process is as follows: traverse the centroid pile number of the C disease area. If the pile number of the center point of the B disease area is the same as that of the C disease area or the difference is ≤2cm, it is determined that the C disease area and the B disease area recognize each other and determine that there is a disease.

8. The intelligent underground cavity detection method based on multi-dimensional ground penetrating radar images according to claim 1, It is characterized in that In step 5, a new column is obtained based on the depth distribution range column B and the width ratio column C, each of which is the average width-to-length ratio of C; and the disease type is determined through the new column.

9. The intelligent underground cavity detection method based on multi-dimensional ground penetrating radar images according to claim 8, It is characterized in that The specific process of determining the type of disease through the new column is as follows: C average width-to-length ratio less than 1.5 is considered a cavity, and C average width-to-length ratio greater than 5 is considered a pipe or crack.

10. An intelligent underground cavity detection system based on multi-dimensional ground penetrating radar images. It is characterized in that include: An image acquisition unit, used for acquiring a plurality of multi-channel B-scan images and C-scan images of a road surface; A data acquisition unit is used to select B-scan images with disease features from B-scan images to form a B-scan abnormal feature data set, and mark all diseases in the B-scan abnormal feature data set as holes; select C-scan images with disease features from C-scan images to form a C-scan abnormal feature data set, and mark all diseases in the C-scan abnormal feature data set as holes; divide the B-scan abnormal feature data set and the C-scan abnormal feature data set into a training set, a validation set, and a test set; A model building unit is used to build a B-scan hole target recognition model based on a training set in a B-scan abnormal feature data set, wherein the B-scan hole target recognition model is a YOLOv5 network; and to build a C-scan hole segmentation model based on a training set in a C-scan abnormal feature data set, wherein the C-scan hole segmentation model is a U-Net network; The training unit is used to detect the validation set in the B-scan abnormal feature data set through the B-scan void target recognition model, and output B disease information, wherein the B disease information includes the B disease type, the B disease center point pile number, and the B depth distribution range; segment the validation set in the C-scan abnormal feature data set through the C-scan void segmentation model, and output C disease information, wherein the C disease information includes the C disease type, the C disease area center pile number, and the C width-to-length ratio; The detection unit is used to match the center pile number of the disease B with the centroid pile number of the disease area C. If they match, it is determined that the disease exists. The disease is determined to be a cavity through the depth distribution range B and the width-to-length ratio C, and the cavity is detected.

Images