Road collapse hidden danger detection method and electronic equipment based on three-dimensional ground penetrating radar

Abstract

The application provides a road collapse hidden danger detection method and electronic equipment based on three-dimensional ground penetrating radar, and relates to the technical field of road detection. The method comprises the following steps: collecting initial radar images corresponding to each collection point on the road to be detected; for any one collection point, the initial radar image corresponding to the collection point is divided into at least one group, and each group is combined to obtain a radar combined image; inputting the radar combined image corresponding to any one collection point into a pre-trained detection model to obtain the detection result corresponding to the collection point, and on this basis, obtaining the collapse hidden danger detection result of the road to be detected; wherein the loss function of the pre-trained detection model is determined based on an attenuation factor; the attenuation factor is determined according to the distance between any one radar combined image and the radar combined image closest to the collapse hidden danger center. The application can significantly shorten the whole process time from data collection to hidden danger identification, and adapt to the urban large-scale road hidden danger investigation demand.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of road detection, and in particular to a road collapse hidden danger detection method based on three-dimensional ground penetrating radar and an electronic device. BACKGROUND

[0002] Road collapse hidden danger identification is a core link to ensure urban public safety. With the acceleration of urbanization, factors such as aging of underground pipelines and construction disturbance lead to frequent collapse incidents, threatening the safety of people's lives and property. Traditional manual inspection has low detection rate, while early identification can effectively avoid risks.

[0003] Three-dimensional ground penetrating radar has the characteristics of high precision, high efficiency and non-destructiveness, and has become one of the most effective means for detecting road collapse hidden dangers.

[0004] However, despite the introduction of three-dimensional ground penetrating radar, the current road collapse hidden danger identification work still faces significant bottlenecks. On the one hand, it is extremely difficult to manually interpret three-dimensional ground penetrating radar maps: the detection scene covers complex environments such as old pipe network areas and construction disturbance areas, and the interpreter needs to combine knowledge from multiple fields such as geology and geophysical exploration, requiring high professional ability and experience of the interpreter, making it difficult to manually interpret ground penetrating radar images and difficult to meet the rapid expansion of detection needs. On the other hand, intelligent detection technology has limitations: existing methods mostly analyze single radar images, and cannot adaptively associate spatial features of multiple adjacent images. And although AI can improve detection accuracy, the single-image analysis mode still restricts detection efficiency in complex scenarios, making it difficult to adapt to large-scale, high-precision hidden danger investigation needs. SUMMARY

[0005] The embodiments of the present application provide a road collapse hidden danger detection method based on three-dimensional ground penetrating radar and an electronic device to solve the problem that it is difficult to accurately detect road collapse hidden dangers based on large-scale radar images at the present stage.

[0006] In a first aspect, the embodiments of the present application provide a road collapse hidden danger detection method based on three-dimensional ground penetrating radar, comprising:

[0007] Collecting initial radar images corresponding to each collection point on the road to be detected; wherein the initial radar image corresponding to each collection point is a multi-channel radar image;

[0008] For any one collection point, the initial radar image corresponding to the collection point is divided into at least one group, wherein each group includes a first predetermined number of adjacent channel radar images, and each group is combined to obtain at least one radar combined image corresponding to the collection point;

[0009] inputting at least one radar combination image corresponding to any one of the collection points into the pre-trained detection model to obtain a detection result corresponding to the collection point; wherein, a loss function of the pre-trained detection model is determined based on an attenuation factor; the attenuation factor is determined according to a distance between any one of the radar combination images and a radar combination image closest to the center of the distance collapse hidden danger;

[0010] determining the collapse hidden danger detection result of the to-be-detected road according to the detection results corresponding to the respective collection points.

[0011] In a possible implementation, the pre-trained detection model includes a plurality of sub-models; the pre-trained detection model is obtained by the following manner:

[0012] In the to-be-trained sub-model, feature extraction is performed on the input sample radar combination image to obtain a second preset number of sample feature maps corresponding to each sample radar combination image;

[0013] candidate boxes of the respective second preset number of sample feature maps corresponding to the input sample radar combination image are determined in the to-be-trained sub-model; and the respective second preset number of sample feature maps after the candidate boxes are determined are fused to obtain a first sample feature fusion image; wherein, the to-be-trained sub-model is used to determine a detection result of the sample radar combination image input thereto based on the first sample feature fusion image;

[0014] receive an instruction signal, and perform candidate box labeling on the second preset number of sample feature maps corresponding to the sample radar combination image corresponding to the instruction signal, and fuse the second preset number of sample feature maps corresponding to the labeled sample radar combination image to obtain a second sample feature fusion image corresponding to each sample radar combination image;

[0015] calculate a target intersection over union according to the first sample feature fusion image and the second sample feature fusion image corresponding to each sample radar combination image to obtain a target positive sample and a target negative sample;

[0016] calculate a loss value according to the target positive sample, the target negative sample and the loss function, and update parameters of the to-be-trained sub-model based on the loss value to obtain the pre-trained detection model.

[0017] In a possible implementation, calculating a target intersection over union according to the first sample feature fusion image and the second sample feature fusion image corresponding to each sample radar combination image to obtain a target positive sample and a target negative sample, comprises:

[0018] calculate a target intersection over union according to the first sample feature fusion image and the second sample feature fusion image corresponding to each sample radar combination image to obtain a target positive sample and a target negative sample, comprises:

[0019] The first sample feature fusion map with a target intersection-union ratio greater than a preset value is used as the target positive sample, and the first sample feature fusion map with a target intersection-union ratio not greater than a preset value is used as the initial negative sample.

[0020] The initial negative samples are sorted according to the target intersection-union ratio;

[0021] Then, from the sorted initial negative samples, a preset number of initial negative samples are selected as target negative samples; where the preset number of samples is determined based on the number of target positive samples.

[0022] In one possible implementation, the formula for calculating the intersection-union ratio is:

[0023]

[0024] In the formula, Indicates the intersection-union ratio of the target; Indicates the first Zhang's first sample feature fusion map; Represents the attenuation factor term; Indicates the baseline value of the attenuation factor; Indicates the closest point to the center of the potential collapse. Zhang's first sample feature fusion map; Indicates the first Candidate boxes in the feature fusion graph of the first sample; Indicates the relationship with the first Candidate boxes in the second sample feature fusion map corresponding to the first sample feature fusion map.

[0025] In one possible implementation, the loss function is:

[0026]

[0027] In the formula, This represents the number of positive target samples; positive target samples indicate radar composite images with potential collapse risks. For classification loss; For regression loss; where:

[0028]

[0029]

[0030] In the formula, Represents the target negative sample dataset The Middle Zhang's target is a positive sample; In the radar combination diagram, the first An indicator of whether a machine-recognized candidate bounding box matches a manually labeled candidate bounding box in category y; This represents the predicted confidence score for the target positive sample. Represents the target negative sample dataset The a-th target negative sample in the middle; This represents the predicted confidence score for the target negative sample. This represents the regression loss corresponding to the target positive sample; The number of radar channels is represented; x and y represent the center coordinates of the candidate boxes; w and h represent the width and height of the candidate boxes, respectively. Represents the attenuation factor term; Indicates the baseline value of the attenuation factor; Indicates the first One radar channel; This indicates the passageway closest to the center of the potential collapse; Indicates the first Zhang Radar's combined diagram; Indicates the first The radar image corresponding to the radar channel corresponds to the first... The regression bounding box prediction value of a preset bounding box; Indicates the first The radar image corresponding to the radar channel corresponds to the first... The regression of the true value of a preset outline.

[0031] In one possible implementation, the parameters of the sub-model to be trained are updated based on the loss value to obtain a pre-trained sub-model, including:

[0032] If the loss value is not less than the preset loss threshold, backpropagation is performed based on the loss function to update the parameters of the sub-model to be trained until the loss value is less than the preset loss threshold, thus obtaining the pre-trained detection model.

[0033] In one possible implementation, each sub-model includes multiple feature extraction channels, each with a different extraction depth; at least one radar composite image corresponding to any given acquisition point is input into a pre-trained detection model to obtain the detection result corresponding to that acquisition point, including:

[0034] For any combined radar image from any acquisition point, perform the following steps:

[0035] Input the combined radar image into any one of the sub-models;

[0036] In this sub-model, each feature extraction channel performs feature extraction on the radar composite image to obtain multiple feature images corresponding to the radar composite image; the number of feature images is determined based on the feature extraction channels in this sub-model.

[0037] According to the multiple feature maps corresponding to each radar combination map, a detection result corresponding to the collection point is obtained.

[0038] In a possible implementation, according to the multiple feature maps corresponding to each radar combination map, the detection result corresponding to the collection point is obtained, including:

[0039] For the multiple feature maps corresponding to any one radar combination map in the collection point, the following steps are performed:

[0040] Based on the identified target in the multiple feature maps corresponding to the radar combination map, multiple candidate boxes are determined in each feature map; wherein the proportions of each candidate box are different;

[0041] The multiple feature maps are fused to obtain a feature fusion map corresponding to the radar combination map;

[0042] The feature fusion map is identified to obtain a detection result of the radar combination map;

[0043] The detection results of each radar combination map corresponding to the collection point are taken as the detection result of the collection point.

[0044] In a possible implementation, the detection result of the radar combination map includes a category and a position coordinate of the radar combination map; wherein the category is used to represent whether there is a collapse hidden danger in the radar combination map; and the position coordinate represents the position coordinate of the collapse hidden danger when the collapse hidden danger exists.

[0045] In a second aspect, an embodiment of the present application provides an electronic device, including a memory and a processor, the memory stores a computer program, and the processor implements the method in the first aspect or any possible implementation of the first aspect when executing the computer program.

[0046] The method provided in this invention does not rely on the professional knowledge and experience of interpreters in multiple fields such as geology and geophysics, thus avoiding the drawbacks of high difficulty and time-consuming manual interpretation in complex scenarios. It also avoids fluctuations in interpretation efficiency due to differences in personnel's professional abilities, meeting the large-scale and rapidly expanding detection needs of urban roads. Furthermore, considering that existing intelligent detection methods mostly analyze single radar images and cannot correlate the spatial features of adjacent images across multiple channels, the method provided in this invention combines multiple radar images acquired at each acquisition point, fully utilizing the correlation features between images from different channels to capture subtle hazard signals that are difficult to detect in a single image, solving the problem of low detection accuracy in traditional methods. In addition, inputting the combined radar images from each acquisition point into the detection model allows for early risk prediction. Moreover, during the identification process, there is no need for manual analysis of each individual image, significantly shortening the entire process from data acquisition to hazard identification, and adapting to the needs of large-scale urban road hazard investigation. Finally, in traditional detection methods, the weights set for images at all locations are uniform. This may lead to inaccurate identification results for images close to the hazard area and images not close to the hazard area. To solve this problem, this embodiment adds an attenuation factor to the loss function of the detection model. Based on the distance between the obtained radar composite image and the radar composite image closest to the center of the collapse hazard, different weights are assigned to images at different locations, thereby improving the accuracy of detection. Attached Figure Description

[0047] Figure 1 This is a flowchart illustrating the implementation of road collapse hazard detection based on three-dimensional ground-penetrating radar according to an embodiment of the present invention;

[0048] Figure 2 These are the initial radar images collected from collection points with potential collapse hazards, as provided in this embodiment of the invention.

[0049] Figure 3 This is a flowchart of the detection model training process provided in an embodiment of the present invention;

[0050] Figure 4 This is a flowchart of the detection process for road collapse hazard detection based on three-dimensional ground-penetrating radar provided in an embodiment of the present invention. Detailed Implementation

[0051] The embodiments of the present invention will now be described in detail with reference to the accompanying drawings.

[0052] Figure 1 This is a flowchart illustrating the implementation of road collapse hazard detection based on three-dimensional ground-penetrating radar according to an embodiment of the present invention, as shown below. Figure 1 As shown, the method includes:

[0053] Step 110: Collecting initial radar images corresponding to each collection point on the road to be detected; wherein the initial radar image corresponding to each collection point is a multi-channel radar image.

[0054] In this embodiment, the collection points can be evenly arranged on the road to be detected according to a preset interval. The initial radar image of each collection point can be obtained by data collection through a three-dimensional ground penetrating radar. The obtained initial radar image is a vertical slice image. The radar used can be a multi-channel radar, and accordingly, the obtained initial radar image is a multi-channel radar image. That is, one collection point can collect multiple radar images, and the number of radar images is determined according to the number of radar channels used, and each channel corresponds to one radar image. In this embodiment, the number of channels of the radar provided should be a positive integer greater than or equal to 2.

[0055] For example, one collection point is arranged every two hundred meters on the road to be detected. The radar used includes six channels, which are first channel, second channel, third channel, fourth channel, fifth channel and sixth channel in turn. Then, each collection point corresponds to six radar images. The arrangement order of the six images is the same as the arrangement order of the corresponding radar channels.

[0056] Optionally, a collection point can also be arranged at a preset interval on the road to be detected, and a vehicle-mounted three-dimensional ground penetrating radar is installed on a target vehicle. During the driving of the vehicle, data collection is performed through the three-dimensional ground penetrating radar every time the vehicle reaches a collection point, so as to obtain the initial radar image of each collection point.

[0057] Step 120: For any collection point, the initial radar image corresponding to the collection point is divided into at least one group, wherein each group includes a first preset number of radar images of adjacent channels, and each group is combined to obtain at least one radar combined image corresponding to the collection point.

[0058] In this embodiment, the detection result of the entire road to be detected is determined by point-by-point detection.

[0059] Correspondingly, the initial radar image corresponding to each collection point is detected during detection. Considering that the traditional method only detects a single radar image, under this detection method, the collected radar image may not exist collapse hazard information due to the deviation of the collection position.

[0060] In this embodiment, the first preset number is a positive integer greater than 2. When the radar used has only two channels, the two collected radar images are combined to obtain one radar combined image.

[0061] The present application explains this step through the following examples:

[0062] For example, for any one collection point, the radar used includes 6 channels, which are the first channel, the second channel, the third channel, the fourth channel, the fifth channel and the sixth channel in turn. Then each collection point corresponds to 6 radar images. In the case of the first preset number being 2, the radar images corresponding to the collection point are divided into the following groups:

[0063] The radar images corresponding to radar channels 1 and 2 are taken as a group; the radar images corresponding to radar channels 2 and 3 are taken as a group; the radar images corresponding to radar channels 3 and 4 are taken as a group; the radar images corresponding to radar channels 4 and 5 are taken as a group; and the radar images corresponding to radar channels 5 and 6 are taken as a group.

[0064] The radar images in the above obtained groups are combined to obtain at least one radar combined image corresponding to the collection point (for example, 6 channels in this embodiment, and 4 radar combined images are obtained).

[0065] Figure 2 is the initial radar image collected by the collection point with a collapse hidden danger provided by the embodiment of the present application; wherein 1-6 represents the radar image collected by the corresponding radar channel, and the oval in it represents the collapse hidden danger. If only one radar image is identified, it is far from accurately determining the position of the collapse hidden danger. The present embodiment considers this situation and combines the radar images of adjacent channels, so as to ensure that the radar combined image obtained as far as possible includes all the information of the collapse hidden danger.

[0066] In order to unify the image size, each radar combined image can be cropped to obtain a radar combined image with a preset size.

[0067] Step 130: input at least one radar combined image corresponding to any one collection point into a pre-trained detection model to obtain a detection result corresponding to the collection point; wherein the loss function of the pre-trained detection model is determined based on an attenuation factor; and the attenuation factor is determined according to the distance between any one radar combined image and the radar combined image closest to the center of the collapse hidden danger.

[0068] In this embodiment, the pre-trained detection model may include multiple sub-models. To ensure processing speed and further enhance the correlation between adjacent radar images, each radar feature map of any acquisition point is processed by a sub-model. Simultaneous processing of multiple sub-models can greatly improve processing speed. Furthermore, the final detection results corresponding to each acquisition point are also fused results obtained from various radar combination maps. By fusing the detection results of various radar combination maps, the correlation between adjacent radar images can be further enhanced, improving the accuracy of the detection results.

[0069] Optionally, traditional methods use the same coefficients for all images during processing, but... Figure 2 As shown, the amount of information contained in channel 2 and channel 3 is different, so the information in the obtained radar composite images is also different. In order to enhance feature detection, this embodiment introduces a loss function based on the attenuation factor during the training of the detection model. The attenuation factor is determined according to the distance between any radar composite image and the radar composite image closest to the center of the collapse hazard.

[0070] The radar composite image closest to the center of the collapse hazard is the one with the most obvious features. The closer any radar composite image is to the radar composite image closest to the center of the collapse hazard, the larger the attenuation factor and the more obvious the feature enhancement effect. The distance between these two images is determined based on the distance between the radar channels.

[0071] Step 140: Based on the detection results corresponding to each collection point, determine the detection results of the potential collapse of the road to be inspected.

[0072] Optionally, there may be partial collapse hazard information in the radar image of the last passage collected by collection point 1, and partial collapse hazard information in the radar image of the first passage collected by collection point 2. In this case, the collapse may exist between collection point 1 and collection point 2. Therefore, it is necessary to merge the detection results of each collection point to obtain an accurate collapse hazard detection result for the road to be detected.

[0073] In summary, the method provided by the embodiment of the present application does not need to rely on the geological, geophysical and other multi-field professional knowledge and experience of interpreters, avoids the defects of high difficulty and long time consumption of manual interpretation in complex scenarios, avoids the fluctuation of interpretation efficiency caused by the difference in professional ability of personnel, and can meet the detection needs of large-scale and rapid expansion of urban roads. In addition, considering that the existing intelligent detection method is mainly for analyzing a single radar image and cannot associate the spatial features of adjacent images of multiple channels, the method provided by the embodiment of the present application combines multiple radar images collected at each collection point, fully utilizes the associated features between different channel images, captures subtle hidden danger signals that cannot be reflected by a single image, and solves the problem of low detection accuracy in the traditional method. In addition, the radar combination image of each collection point obtained is input into the detection model to predict the risk in advance. Moreover, in the identification process, there is no need for manual analysis of a single image one by one, which significantly shortens the time consumption of the whole process from data collection to hidden danger identification, and adapts to the demand of city-level large-scale road hidden danger investigation. Finally, in the traditional detection method, the weight set for the images at each position is a uniform value, which may lead to inaccurate identification results of the images close to the hidden danger area and the images of the principle hidden danger area. To solve this problem, the embodiment adds an attenuation factor to the loss function of the detection model, assigns different weights to the images at different positions according to the distance between the obtained radar combination image and the radar combination image closest to the collapse hidden danger center, and improves the detection accuracy.

[0074] To implement the above method, the embodiment of the present application uses an improved single multi-box detection model (Single Shot MultiBox Detector, SDD) as the detection model, wherein the parameters of multiple sub-models included in the model are the same, and the model is essentially an extensible large model. In the training process, training of a sub-model can obtain the entire detection model. Figure 3 The detection model training flowchart provided by the embodiment of the present application is shown in FIG. 1. Figure 3 The training process of the model is as follows.

[0075] The input data of the model in the training process is the sample radar combination image included in the training set. The sample radar combination image included in the training set is obtained based on the initial radar images collected at each collection point in the historical stage. The acquisition method of the sample radar combination image can refer to the related embodiments in step 120, that is, the sample radar combination image used in the embodiment is also obtained by first collecting the initial radar images at each historical collection point.

[0076] In the training process, the embodiment needs to provide an auxiliary model to complete the training of the sub-model, wherein the auxiliary model can be a neural network model.

[0077] The following are the execution steps of the sub-model during training:

[0078] In the sub-model to be trained, feature extraction is performed on the input sample radar combination image to obtain a second preset number of sample feature maps corresponding to each sample radar combination image.

[0079] In the sub-model to be trained, the candidate boxes of the second preset number of sample feature maps corresponding to the input sample radar combination image are determined; and the second preset number of sample feature maps after the candidate boxes are determined are fused to obtain a first sample feature fusion image; wherein the sub-model to be trained is used to determine the detection result of the sample radar combination image input thereto based on the first sample feature fusion image.

[0080] In this embodiment, each sub-model to be trained includes a plurality of detection channels; each channel is composed of a Visual Geometry Group (VGG) 16-layer convolutional neural network and a convolutional layer. Each VGG16 network has different extraction depths. For a sample radar combination image, the corresponding sub-model extracts different features through different VGG16 networks and inputs them into different convolutional layers, further strengthens each feature, and obtains a plurality of sample feature maps corresponding to the sample radar combination image. The number of sample feature maps is consistent with the number of detection channels.

[0081] For each sample feature map obtained, candidate box recognition is performed, wherein the candidate box recognition can be performed through the PriorBox algorithm. Through this algorithm, a plurality of candidate boxes with different scales are generated from the center of each pixel point. The number of generated candidate boxes can be 4 or 6, etc. These candidate boxes not only provide a range for recognition, but also reduce the complexity of recognizing each sample feature fusion image. The sample feature maps after candidate box recognition are fused to obtain a first sample feature fusion image corresponding to the sample radar combination image, i.e., one sample radar combination image corresponds to one first sample feature fusion image.

[0082] The following is the content executed by the auxiliary model:

[0083] The instruction signal is received, the second preset number of sample feature maps corresponding to the sample radar combination image corresponding thereto are labeled with candidate boxes, and the second preset number of sample feature maps corresponding to the sample radar combination image after labeling are fused to obtain a second sample feature fusion image corresponding to each sample radar combination image.

[0084] According to the first sample feature fusion image and the second sample feature fusion image corresponding to each sample radar combination image, the target intersection over union is calculated to obtain target positive samples and target negative samples.

[0085] According to the target positive sample, the target negative sample and the loss function, a loss value is calculated, and parameters of the to-be-trained sub-model are updated based on the loss value to obtain the pre-trained detection model.

[0086] In this embodiment, the instruction signal can be triggered by the staff. Since the feature extraction algorithm at the present stage has been able to realize high-precision feature extraction, when performing manual labeling of the candidate box, the plurality of sample feature maps corresponding to the sample radar combination image obtained by the to-be-trained sub-model can be directly used. In this step, the steps of obtaining the second sample feature fusion image after labeling the candidate box are the same as the steps of obtaining the first sample feature fusion image, and therefore will not be described here.

[0087] It should be noted here that the selected picture in each group of sample feature maps obtained in this step is the same as the selected picture in each group of sample feature maps obtained in the previous step, and correspondingly, the first sample feature fusion image and the second sample feature fusion image obtained in this step are Figure 1 corresponding, and the difference between the two is only the candidate box in them.

[0088] This embodiment obtains the target intersection-over-union by comparing the difference between the candidate box in each first sample feature fusion image and the corresponding second sample feature fusion image. The target intersection-over-union is used to judge whether each sample radar combination image is a target positive sample or a target negative sample; wherein the target positive sample indicates that there is a collapse hazard in the sample radar combination image; and the target negative sample indicates that there is no collapse hazard in the sample radar combination image.

[0089] When adjusting the model parameters of the to-be-trained sub-model, the loss value is calculated by the loss function, if the obtained loss value is not less than the preset loss threshold, the loss function is used for back propagation to update the parameters of the to-be-trained sub-model, until the loss value is less than the preset loss threshold, and the pre-trained detection model is obtained based on the to-be-trained sub-model which has completed the training.

[0090] The loss function is:

[0091]

[0092] In the formula, is the number of target positive samples; the target positive sample indicates a radar combination image with a collapse hazard; is the classification loss; is the regression loss; wherein:

[0093]

[0094]

[0095] In the formula, Represents the target negative sample dataset The Middle Zhang's target is a positive sample; Indicating the radar combination diagram, the first... An indicator of whether a machine-recognized candidate bounding box matches a manually labeled candidate bounding box in category y; This represents the predicted confidence score for the target positive sample. Represents the target negative sample dataset The a-th target negative sample in the middle; This represents the predicted confidence score for the target negative sample. This represents the regression loss corresponding to the target positive sample; The number of radar channels is represented; x and y represent the center coordinates of the candidate boxes; w and h represent the width and height of the candidate boxes, respectively. Represents the attenuation factor term; Indicates the baseline value of the attenuation factor; Indicates the first One radar channel; This indicates the passageway closest to the center of the potential collapse; Indicates the first Zhang Radar's combined diagram; Indicates the first The radar image corresponding to the radar channel corresponds to the first... The regression bounding box prediction value of a preset bounding box; Indicates the first The radar image corresponding to the radar channel corresponds to the first... The regression of the true value of a preset outline.

[0096] In the formula, ; Similarly;

[0097] , , ,

[0098] The parameter 'a' represents the candidate box 'a', whose center is represented as (x, y).

[0099] In an optional embodiment, the target crossover ratio (CVR) is calculated based on the first sample feature fusion map and the second sample feature fusion map corresponding to each sample radar composite map, to obtain target positive samples and target negative samples, including:

[0100] The target crossover ratio is calculated based on the first sample feature fusion map, the second sample feature fusion map, and the crossover ratio calculation formula corresponding to each sample radar combination map; wherein, the crossover ratio calculation formula includes an attenuation factor.

[0101] The first sample feature fusion map with a target intersection-union ratio greater than a preset value is used as the target positive sample, and the first sample feature fusion map with a target intersection-union ratio not greater than a preset value is used as the initial negative sample.

[0102] The initial negative samples are sorted according to the target intersection-union ratio.

[0103] Then, from the sorted initial negative samples, a preset number of initial negative samples are selected as target negative samples; where the preset number of samples is determined based on the number of target positive samples.

[0104] In many cases, the number of negative samples is much greater than the number of positive samples. To ensure sample balance, the number of negative samples needs to be determined based on the number of positive samples. Typically, the number of negative samples is three times the number of positive samples.

[0105] Optionally, the formula for calculating the intersection-union ratio is:

[0106]

[0107] In the formula, Indicates the intersection-union ratio of the target; Indicates the first Zhang's first sample feature fusion map; Represents the attenuation factor term; Indicates the baseline value of the attenuation factor; Indicates the closest point to the center of the potential collapse. Zhang's first sample feature fusion map; Indicates the first Candidate boxes in the feature fusion graph of the first sample; Indicates the relationship with the first Candidate boxes in the second sample feature fusion map corresponding to the first sample feature fusion map.

[0108] Similar to the loss function, the crossover ratio (CR) calculation formula also includes an attenuation factor term. The attenuation factors involved in these two formulas have essentially the same meaning.

[0109] The loss function and crossover ratio (CUP) calculation formula mentioned above are only used during the model training process. In practical applications, the detection model can output the detection results based on the feature map fusion map.

[0110] Based on the model trained above, accurate detection results can be obtained by detecting the obtained radar images. Figure 4 This is a flowchart of the road collapse hazard detection based on three-dimensional ground-penetrating radar provided in an embodiment of the present invention. It shows the detection process of a sub-model. The following uses a sub-model as an example to illustrate the detection process of the model:

[0111] In an optional embodiment, each sub-model includes multiple feature extraction channels, and the extraction depth of each feature extraction channel is different; the step of inputting at least one radar combination image corresponding to any one collection point into the pre-trained detection model to obtain the detection result corresponding to the collection point in step 130 can include:

[0112] The following steps are performed on any one radar combination image of any one collection point:

[0113] The radar combination image is input into any one sub-model.

[0114] In the sub-model, each feature extraction channel extracts features from the radar combination image to obtain multiple feature maps corresponding to the radar combination image; the number of feature maps is determined based on the feature extraction channels in the sub-model.

[0115] According to the multiple feature maps corresponding to each radar combination image, the detection result corresponding to the collection point is obtained.

[0116] To further strengthen the correlation between adjacent radar images, each sub-model includes multiple feature extraction channels, and the extraction depth of each feature extraction channel is different to capture different fine-grained features. The number of feature maps in this embodiment and the concept represented by the second preset number are the same, and different expression methods are used to distinguish the training process and the use process of the model.

[0117] In actual application, a radar combination image is input into a sub-model. Each sub-model has multiple detection channels, and each channel is composed of a Visual Geometry Group (VGG) 16-layer convolutional neural network and a convolutional layer. The second preset number represents the number of channels in the sub-model.

[0118] In each sub-model, the extraction depth of each VGG16 network is different. For a radar combination image, its corresponding sub-model extracts different features through different VGG16 networks and inputs them into different convolutional layers to further strengthen each feature to obtain the second preset number of feature maps corresponding to the radar combination image. The size of the convolutional layer in each channel is determined according to the extraction depth of the VGG16 network corresponding to it to match it.

[0119] This embodiment converts the detection result recognition process of each collection point into the recognition of each radar combination image corresponding to each collection point, and converts the recognition of each collection point radar combination image into the recognition of each feature map thereof. Through this step-by-step detailed recognition method, the detection result of each collection point is gradually obtained.

[0120] In an optional embodiment, the detection result corresponding to the collection point is obtained according to a plurality of feature maps corresponding to each radar combination map, including:

[0121] For the plurality of feature maps corresponding to any one radar combination map in the collection point, the following steps are performed:

[0122] Based on the identified target in the plurality of feature maps corresponding to the radar combination map, a plurality of candidate boxes are determined in each feature map; wherein each candidate box has a different scale.

[0123] The plurality of feature maps are fused to obtain a feature fusion map corresponding to the radar combination map.

[0124] The feature fusion map is identified to obtain the detection result of the radar combination map.

[0125] The detection results of each radar combination map corresponding to the collection point are taken as the detection result of the collection point.

[0126] The reason why the feature maps are further fused in this embodiment is that before fusion, the sub-models respectively perform candidate box identification on the plurality of feature maps corresponding to the radar combination map input thereto. The candidate box identification can be performed by the PriorBox algorithm, which generates a plurality of candidate boxes with different scales based on the center of each pixel point. The number of generated candidate boxes can be 4 or 6, etc. These candidate boxes not only provide a range for identification, but also reduce the complexity of identifying each feature fusion map.

[0127] In this way, the candidate boxes included in each feature map in each group are different, the identification range and focus of each feature fusion map are also different, i.e., the identified target is slightly different, and the identification results of these feature fusion maps can improve the accuracy of the detection result.

[0128] In an optional embodiment, the detection result of the radar combination map includes the category and position coordinates of the radar combination map; wherein the category represents whether there is a collapse hazard in the radar combination map; and the position coordinates represent the position coordinates of the collapse hazard when there is a collapse hazard.

[0129] In this embodiment, the detection result can be obtained by the identification (prediction) network in each sub-model based on the input feature fusion map. Since the radar image when there is a collapse hazard has obvious feature differences compared with the normal radar image, during identification, whether each image has a collapse hazard can be determined based on these obvious features. If there is a collapse hazard, the position coordinates of the collapse hazard can be determined by the following method:

[0130] The acquisition point corresponding to the radar combination image is determined to locate the approximate coordinates, the corresponding radar image in the radar combination image is determined, the radar channel of the radar image is determined based on the radar image, and thus the position coordinates of the collapse hidden danger can be determined.

[0131] To sum up, the embodiment of the application introduces an attenuation factor in the calculation formula of the intersection-over-union ratio, which can accurately distinguish positive samples and negative samples. Based on this, the attenuation factor is also introduced in the loss function, and the model obtained by training can strengthen the detection of the slice graph of the hidden danger center area while taking into account the information contained in the slice graph of other areas. The model obtained by the above method can effectively and efficiently detect the hidden danger information in the radar images generated by different channels. It should be understood that the size of the serial number of each step in the above embodiments does not mean the order of execution, and the execution order of each process should be determined according to its function and internal logic, and should not constitute any limitation on the implementation process of the embodiment of the application.

[0132] The embodiment of the application also provides an electronic device, including a memory and a processor, the memory stores a computer program, and the processor implements the method in the above method embodiment when executing the computer program.

[0133] In the above embodiments, the description of each embodiment has its own emphasis, and the parts not described or recorded in a certain embodiment can be referred to the relevant description of other embodiments. If there is no special description and logical conflict, the terms and / or descriptions of different embodiments are consistent and can be mutually referenced. The technical features in different embodiments can be combined to form new embodiments according to their inherent logical relationship.

[0134] The above embodiments are only used to illustrate the technical solutions of the application, and not to limit them; although the application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that they can still modify the technical solutions recorded in the foregoing embodiments, or make equivalent replacement for part of the technical features; and these modifications or replacements do not make the essence of the corresponding technical solutions deviate from the spirit and scope of the technical solutions of the embodiments of the application, and should be included in the protection scope of the application.

Claims

1. A method for detecting road collapse hazards based on three-dimensional ground-penetrating radar, characterized in that, include: Initial radar images are acquired at each acquisition point on the road to be detected; the initial radar image for each acquisition point is a multi-channel radar image. For any acquisition point, the initial radar image corresponding to the acquisition point is divided into at least one group, wherein each group includes a first preset number of radar images of adjacent channels, and each group is combined to obtain at least one radar combined image corresponding to the acquisition point. At least one radar composite image corresponding to any acquisition point is input into a pre-trained detection model to obtain the detection result corresponding to that acquisition point; wherein, the loss function of the pre-trained detection model is determined based on an attenuation factor; the attenuation factor is determined based on the distance between any radar composite image and the radar composite image closest to the center of the collapse hazard; the distance between any radar composite image and the radar composite image closest to the center of the collapse hazard is determined based on the distance of its corresponding radar channel; Based on the detection results corresponding to each collection point, the detection results of the potential collapse of the road to be inspected are determined.

2. The method for detecting road collapse hazards based on three-dimensional ground-penetrating radar according to claim 1, characterized in that, The pre-trained detection model includes multiple sub-models; the pre-trained detection model is obtained through the following methods: In the sub-model to be trained, features are extracted from the input sample radar combination map to obtain the second preset number of sample feature maps corresponding to each sample radar combination map. In the sub-model to be trained, candidate boxes of each second preset number of sample feature maps corresponding to the input sample radar combination map are determined; and the second preset number of sample feature maps after the candidate boxes are determined are fused to obtain a first sample feature fusion map; wherein, the sub-model to be trained is used to determine the detection result of its input sample radar combination map based on the first sample feature fusion map; Upon receiving the instruction signal, candidate boxes are marked on the second preset number of sample feature maps corresponding to the corresponding sample radar combination map. The second preset number of sample feature maps corresponding to the marked sample radar combination map are then fused to obtain the second sample feature fusion map corresponding to each sample radar combination map. Based on the first sample feature fusion map and the second sample feature fusion map corresponding to each sample radar combination map, the target crossover ratio is calculated to obtain the target positive sample and the target negative sample. Based on the target positive sample, the target negative sample, and the loss function, a loss value is calculated, and the parameters of the sub-model to be trained are updated based on the loss value to obtain a pre-trained detection model.

3. The method for detecting road collapse hazards based on three-dimensional ground-penetrating radar according to claim 2, characterized in that, The step of calculating the target crossover ratio (CBR) based on the first sample feature fusion map and the second sample feature fusion map corresponding to each sample radar combination map, to obtain target positive samples and target negative samples, includes: The target cross-union ratio is calculated based on the first sample feature fusion map, the second sample feature fusion map, and the cross-union ratio calculation formula corresponding to each sample radar combination map; wherein, the cross-union ratio calculation formula includes an attenuation factor; The first sample feature fusion map with the target intersection-union ratio greater than a preset value is used as the target positive sample, and the first sample feature fusion map with the target intersection-union ratio not greater than a preset value is used as the initial negative sample. The initial negative samples are sorted according to the target intersection-union ratio; Then, from the sorted initial negative samples, a preset number of initial negative samples are selected as target negative samples; wherein, the preset number of samples is determined based on the number of target positive samples.

4. The method for detecting road collapse hazards based on three-dimensional ground-penetrating radar according to claim 3, characterized in that, The formula for calculating the crossover-union ratio is: In the formula, Indicates the intersection and union ratio of the objectives; Indicates the first Zhang's first sample feature fusion map; Represents the attenuation factor term; Indicates the baseline value of the attenuation factor; Indicates the closest point to the center of the potential collapse. Zhang's first sample feature fusion map; Indicates the first Candidate boxes in the feature fusion graph of the first sample; Indicates the relationship with the first Candidate boxes in the second sample feature fusion map corresponding to the first sample feature fusion map.

5. The method for detecting road collapse hazards based on three-dimensional ground-penetrating radar according to claim 1, characterized in that, The loss function is: In the formula, The number of positive target samples; the positive target samples represent radar composite images with potential collapse risks; For classification loss; For regression loss; where: In the formula, Represents the target negative sample dataset The Middle Zhang's target is a positive sample; In the radar combination diagram, the first An indicator of whether a machine-recognized candidate bounding box matches a manually labeled candidate bounding box in category y; This represents the predicted confidence score for the target positive sample. Represents the target negative sample dataset The a-th target negative sample in the middle; This represents the predicted confidence score for the target negative sample. This represents the regression loss corresponding to the target positive sample; The number of radar channels is represented; x and y represent the center coordinates of the candidate boxes; w and h represent the width and height of the candidate boxes, respectively. Represents the attenuation factor term; Indicates the baseline value of the attenuation factor; Indicates the first One radar channel; This indicates the passageway closest to the center of the potential collapse; Indicates the first Zhang Radar's combined diagram; Indicates the first The radar image corresponding to the radar channel corresponds to the first... The regression bounding box prediction value of a preset bounding box; Indicates the first The radar image corresponding to the radar channel corresponds to the first... The regression of the true value of a preset outline.

6. The method for detecting road collapse hazards based on three-dimensional ground-penetrating radar according to claim 2, characterized in that, The step of updating the parameters of the sub-model to be trained based on the loss value to obtain a pre-trained sub-model includes: If the loss value is not less than the preset loss threshold, backpropagation is performed based on the loss function to update the parameters of the sub-model to be trained until the loss value is less than the preset loss threshold, thus obtaining the pre-trained detection model.

7. The method for detecting road collapse hazards based on three-dimensional ground-penetrating radar according to claim 2, characterized in that, Each sub-model includes multiple feature extraction channels, and each feature extraction channel has a different extraction depth; The step of inputting at least one combined radar image corresponding to any given acquisition point into a pre-trained detection model to obtain the detection result corresponding to that acquisition point includes: For any combined radar image from any acquisition point, perform the following steps: Input the combined radar image into any one of the sub-models; In this sub-model, each feature extraction channel performs feature extraction on the radar composite image to obtain multiple feature images corresponding to the radar composite image; wherein, the number of feature images is determined based on the feature extraction channels in this sub-model; Based on the multiple feature maps corresponding to each radar composite image, the detection result corresponding to that acquisition point is obtained.

8. The method for detecting road collapse hazards based on three-dimensional ground-penetrating radar according to claim 7, characterized in that, The step of obtaining the detection result corresponding to the acquisition point based on multiple feature maps corresponding to each radar combination image includes: For any radar composite image at this acquisition point, corresponding to multiple feature images, perform the following steps: Based on the target identified in the multiple feature maps corresponding to the radar composite image, multiple candidate boxes are determined in each feature map; wherein, the proportion of each candidate box is different; Multiple feature maps are fused to obtain the feature fusion map corresponding to the radar composite image; The feature fusion map is identified to obtain the detection result of the radar composite map; The detection results of each radar combination map corresponding to the acquisition point are taken as the detection results of that acquisition point.

9. The method for detecting road collapse hazards based on three-dimensional ground-penetrating radar according to claim 8, characterized in that, The detection results of the radar composite image include the category of the radar composite image and its location coordinates; wherein, the category is used to characterize whether there is a potential collapse hazard in the radar composite image; and the location coordinates characterize the location coordinates of the potential collapse hazard when it exists.

10. An electronic device, characterized in that, It includes a memory and a processor, the memory storing a computer program, and the processor executing the computer program to implement the method as described in any one of claims 1 to 9.

Images