Road collapse hidden danger detection method and electronic equipment

Abstract

The application provides a road collapse hidden danger detection method and electronic equipment, and relates to the technical field of road detection. The method comprises the following steps: detecting a to-be-detected road based on a preset collection interval to obtain initial radar images of each collection point; for any one collection point, dividing the initial radar image corresponding to the collection point into at least one group, and combining each group to obtain at least one radar combined image corresponding to the collection point; inputting the at least one radar combined image corresponding to any one collection point into a pre-trained detection model to obtain a detection result corresponding to the collection point, and determining a collapse hidden danger detection result of the to-be-detected road; wherein the detection model comprises a plurality of sub-models, and each sub-model is used for detecting one radar combined image. The application does not need to analyze a single image manually, significantly shortens the time consumption of the whole process from data collection to hidden danger identification, and adapts to the urban-level large-scale road hidden danger investigation demand.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of road detection, in particular to a road collapse hidden danger detection method and an electronic device. BACKGROUND

[0002] Road collapse hidden danger identification is a core link to ensure urban public safety. With the acceleration of urbanization, factors such as aging of underground pipelines and construction disturbance lead to frequent collapse incidents, threatening the safety of people's lives and property. Traditional manual inspection has low detection rate, and early identification can effectively avoid risks.

[0003] The current identification work still faces significant bottlenecks. On the one hand, it is extremely difficult for manual interpretation of three-dimensional ground penetrating radar maps: the detection scene covers complex environments such as old pipe network areas and construction disturbance areas, and the interpreter needs to combine geological, geophysical and other multi-disciplinary knowledge, which requires high professional ability and experience of the interpreter, making it difficult for manual interpretation of ground penetrating radar images to meet the detection needs of rapid expansion. On the other hand, intelligent detection technology has limitations: existing methods mostly analyze single radar images, and cannot adaptively associate spatial features of adjacent images of multiple channels. And although AI can improve detection accuracy, the single-image analysis mode still restricts detection efficiency in complex scenarios, making it difficult to adapt to large-scale, high-precision hidden danger investigation needs. SUMMARY

[0004] The embodiments of the present application provide a road collapse hidden danger detection method and an electronic device to solve the problem that it is difficult to accurately detect road collapse hidden dangers based on large-scale radar images at the present stage.

[0005] In a first aspect, the embodiments of the present application provide a road collapse hidden danger detection method, comprising:

[0006] Detecting the road to be detected based on a preset collection interval to obtain initial radar images of each collection point; wherein the initial radar image corresponding to each collection point is a multi-channel radar image;

[0007] For any one collection point, the initial radar image corresponding to the collection point is divided into at least one group, wherein each group includes a first preset number of adjacent channel radar images, and each group is combined to obtain at least one radar combined image corresponding to the collection point;

[0008] Inputting the at least one radar combined image corresponding to any one collection point into a pre-trained detection model to obtain a detection result corresponding to the collection point; wherein the detection model includes a plurality of sub-models, each sub-model is used to detect a radar combined image; and the detection result corresponding to the collection point is determined based on the detection results of the radar combined images;

[0009] According to the detection results corresponding to each collection point, a collapse hidden danger detection result of the to-be-detected road is determined.

[0010] In a possible implementation, at least one radar combination image corresponding to any one collection point is input into a pre-trained detection model to obtain a detection result corresponding to the collection point, including:

[0011] At least one radar combination image corresponding to any one collection point is input into a pre-trained detection model.

[0012] The following steps are performed in any one sub-model of the pre-trained detection model:

[0013] Features of the input radar combination image are extracted, and each extracted feature is input into a second preset number of convolution layers to obtain a second preset number of feature maps corresponding to the input radar combination image.

[0014] According to the second preset number of feature maps corresponding to the input radar combination image, a detection result of the input radar combination image is obtained.

[0015] According to the detection results of the radar combination images corresponding to the collection point, a detection result corresponding to the collection point is obtained.

[0016] In a possible implementation, according to the second preset number of feature maps corresponding to the input radar combination image, a detection result of the input radar combination image is obtained, including:

[0017] The second preset number of feature maps corresponding to the input radar combination image are randomly divided into multiple groups, and each group includes a third preset number of feature maps; each group of feature maps is fused to obtain at least one feature fusion image.

[0018] Each feature fusion image is identified to obtain a detection result of the input radar combination image.

[0019] In a possible implementation, before randomly selecting a third preset number of feature maps as a group and fusing to obtain at least one feature fusion image, the method further includes:

[0020] A plurality of candidate boxes with different proportions are determined in the second preset number of feature maps corresponding to the input radar combination image; wherein the candidate boxes are used to reduce the complexity of identifying each feature fusion image.

[0021] In a possible implementation, the detection result of the radar combination image includes a category and a position coordinate of the radar combination image; wherein the category is used to represent whether a collapse hidden danger exists in the radar combination image; and the position coordinate represents the position coordinate of the collapse hidden danger when the collapse hidden danger exists.

[0022] In a possible implementation, the pre-trained detection model is obtained by the following manner:

[0023] inputting a sample radar combination image included in the training set into the to-be-trained sub-model as an input;

[0024] performing feature extraction on the input sample radar combination image in the to-be-trained sub-model, and obtaining a second preset number of sample feature maps corresponding to each sample radar combination image by passing the extracted features through a second preset number of convolution layers;

[0025] determining candidate boxes of the second preset number of sample feature maps corresponding to the input sample radar combination image in the to-be-trained sub-model; and randomly dividing the second preset number of sample feature maps after the candidate boxes are determined into multiple groups; performing fusion on each group of sample feature maps to obtain at least one first sample feature fusion image; each group includes a third preset number of sample feature maps; and each to-be-trained sub-model is configured to determine a detection result of the input sample radar combination image based on the first sample feature fusion image;

[0026] receiving an instruction signal, performing candidate box labeling on the second preset number of sample feature maps corresponding to the sample radar combination image corresponding to the instruction signal, and randomly dividing the second preset number of sample feature maps after the candidate boxes are determined into multiple groups; performing fusion on each group of sample feature maps to obtain at least one second sample feature fusion image; each first sample feature fusion image corresponds to a second sample feature fusion image; and the selected sample feature map in each second sample feature fusion image is the same as the selected sample feature map in the first sample feature fusion image corresponding thereto;

[0027] calculating an intersection over union value according to each first sample feature fusion image and the second sample feature fusion image corresponding thereto to obtain a target positive sample and a target negative sample;

[0028] calculating a loss value according to the target positive sample, the target negative sample, and a loss function, and updating parameters of the to-be-trained sub-model based on the loss value to obtain the pre-trained detection model.

[0029] In a possible implementation, calculating an intersection over union value according to each first sample feature fusion image and the second sample feature fusion image corresponding thereto to obtain a target positive sample and a target negative sample includes:

[0030] calculating an intersection over union value according to each first sample feature fusion image, the second sample feature fusion image corresponding thereto, and an intersection over union calculation formula;

[0031] selecting each first sample feature fusion image according to the intersection over union value to obtain the target positive sample and the target negative sample.

[0032] In a possible implementation, the first sample feature fusion graphs are screened according to the intersection-over-union values to obtain the target positive samples and the target negative samples, including:

[0033] The first sample feature fusion graph with the intersection-over-union value greater than the preset value is taken as the target positive sample, and the first sample feature fusion graph with the intersection-over-union value not greater than the preset value is taken as the initial negative sample.

[0034] The obtained initial negative samples are sorted according to the intersection-over-union values.

[0035] The preset sample quantity of the initial negative samples is screened from the sorted initial negative samples as the target negative samples, where the preset sample quantity is determined based on the quantity of the target positive samples.

[0036] In a possible implementation, the loss value is calculated according to the target positive samples, the target negative samples and the loss function, and the parameters of the to-be-trained sub-model are updated based on the loss value to obtain the pre-trained detection model, including:

[0037] The quantity of the positive samples corresponding to each sample radar combination graph included in the training set is determined.

[0038] The loss value corresponding to each sample radar combination graph is calculated by the loss function according to the quantity of the positive samples, the value of the target positive samples and the value of the target negative samples.

[0039] The sum of the loss values corresponding to each sample radar combination graph is taken as the loss value of the training set.

[0040] If the loss value is not less than the preset loss threshold, the back propagation is performed based on the loss function to update the parameters of the to-be-trained sub-model, until the loss value is less than the preset loss value, and the pre-trained detection model is obtained.

[0041] In a second aspect, an electronic device is provided, including a memory and a processor, the memory stores a computer program, and the processor implements the method in the first aspect or any possible implementation manner of the first aspect when executing the computer program.

[0042] The method provided in this invention does not rely on the professional knowledge and experience of interpreters in multiple fields such as geology and geophysics, avoiding the drawbacks of high difficulty and time-consuming manual interpretation in complex scenarios. It also avoids fluctuations in interpretation efficiency due to differences in personnel's professional abilities, meeting the large-scale and rapidly expanding detection needs of urban roads. Furthermore, considering that existing intelligent detection methods mostly analyze single radar images and cannot correlate the spatial features of adjacent images across multiple channels, the method provided in this invention, at each acquisition point, randomly combines multiple acquired radar images, fully utilizing the correlation features between images from different channels to capture subtle hazard signals that are difficult to detect in a single image. During the detection process, the detection results of multiple sub-models are fused to determine the detection results at each acquisition point, solving the problem of low detection accuracy in traditional methods. Finally, the obtained combined radar images of each acquisition point are input into the detection model, enabling early risk prediction. Moreover, during the identification process, there is no need for manual analysis of each individual image, significantly shortening the entire process from data acquisition to hazard identification, and adapting to the needs of large-scale urban road hazard investigation. Attached Figure Description

[0043] Figure 1 This is a flowchart illustrating the implementation of the road collapse hazard detection method provided in this embodiment of the invention;

[0044] Figure 2 These are the initial radar images collected from collection points with potential collapse hazards, as provided in this embodiment of the invention.

[0045] Figure 3 This is a flowchart of the road collapse hazard detection method provided in this embodiment of the invention;

[0046] Figure 4 This is a flowchart of the detection model training process provided in an embodiment of the present invention. Detailed Implementation

[0047] The embodiments of the present invention will now be described in detail with reference to the accompanying drawings.

[0048] Figure 1 This is a flowchart illustrating the implementation of the road collapse hazard detection method provided in this embodiment of the invention. Figure 1 As shown, the method includes:

[0049] Step 110: Detect the road to be detected based on the preset acquisition interval to obtain the initial radar image of each acquisition point; wherein, the initial radar image corresponding to each acquisition point is a multi-channel radar image.

[0050] In the embodiment, one collection point can be set at every preset interval on the road to be detected, and the three-dimensional ground penetrating radar is used to detect and collect data at each collection point to obtain an initial radar image of each collection point. The obtained initial radar image is a vertical slice image. The radar used can be a multi-channel radar, and accordingly, the obtained initial radar image is a multi-channel radar image, that is, one collection point can collect multiple radar images, and the number of radar images is determined according to the number of radar channels, and each channel corresponds to one radar image. The number of channels of the radar provided in the embodiment should be a positive integer greater than or equal to 3.

[0051] For example, one collection point is set at every 200 meters on the road to be detected. The radar used includes six channels, which are a first channel, a second channel, a third channel, a fourth channel, a fifth channel and a sixth channel in turn. Then, each collection point corresponds to six radar images. The arrangement order of the six images is the same as that of the corresponding radar channels.

[0052] Optionally, one collection point can also be set at every preset interval on the road to be detected, and a vehicle-mounted three-dimensional ground penetrating radar is installed on the target vehicle. During the driving of the vehicle, data is collected by the three-dimensional ground penetrating radar once the vehicle reaches each collection point to obtain an initial radar image of each collection point.

[0053] Step 120: For any one collection point, the initial radar image corresponding to the collection point is divided into at least one group, wherein each group includes a first preset number of radar images of adjacent channels, and each group is combined to obtain at least one radar combined image corresponding to the collection point.

[0054] The embodiment determines the detection result of the entire road to be detected by point-by-point detection.

[0055] Correspondingly, the initial radar image corresponding to each collection point is detected during detection. Considering that the traditional method only detects a single radar image, under this detection method, the collected radar image can deviate from the collection position, and the collected radar image can not include the collapse hazard information.

[0056] The embodiment explains and describes the step as follows:

[0057] For example, for any one collection point, the radar used includes six channels, which are a first channel, a second channel, a third channel, a fourth channel, a fifth channel and a sixth channel in turn. Then, each collection point corresponds to six radar images. In the case where the first preset number is 3, the radar images corresponding to the collection point are divided into the following groups:

[0058] The radar images corresponding to the radar channels 1, 2, and 3 are taken as a group; the radar images corresponding to the radar channels 2, 3, and 4 are taken as a group; the radar images corresponding to the radar channels 3, 4, and 5 are taken as a group; and the radar images corresponding to the radar channels 4, 5, and 6 are taken as a group.

[0059] The radar images in each of the groups obtained above are combined to obtain at least one radar combined image corresponding to the collection point (in this embodiment, 6 channels are taken as an example, and 4 radar combined images are obtained).

[0060] Figure 2 The initial radar images collected by the collection point with a collapse hidden danger provided in this embodiment; wherein 1-6 represent the radar images collected by the corresponding radar channels, and the oval shape in the figure represents a collapse hidden danger. If only one radar image is identified, the position of the collapse hidden danger cannot be accurately determined, and this embodiment considers this situation and combines the radar images of adjacent channels, so that the radar combined image obtained can include as much information of the collapse hidden danger as possible.

[0061] In order to unify the image size, each radar combined image can be cropped to obtain a radar combined image with a preset size.

[0062] Step 130: inputting at least one radar combined image corresponding to any one collection point into a pre-trained detection model to obtain a detection result corresponding to the collection point; wherein the detection model includes a plurality of sub-models, each sub-model is used to detect one radar combined image, and the detection result corresponding to the collection point is determined based on the detection results of the radar combined images.

[0063] Figure 3 is a detection flowchart of the road collapse hidden danger detection method provided in this embodiment, and the following will be described in combination with Figure 3 The present embodiment will be described below:

[0064] In order to ensure the processing speed and further enhance the correlation between adjacent radar images, each radar feature map of any one collection point in this embodiment is processed by one sub-model, and the multiple sub-models are processed synchronously, which can greatly improve the processing speed. Moreover, the detection results corresponding to each collection point obtained finally are also fusion results obtained from each radar combined image. Through fusing the detection results of each radar combined image, the correlation between adjacent radar images can be further enhanced, and the accuracy of the detection result can be improved.

[0065] Step 140: determining a collapse hidden danger detection result of the road to be detected according to the detection results corresponding to each collection point.

[0066] Optionally, there may be partial collapse hidden danger information in the radar image of the last channel collected by the collection point 1 and there may be partial collapse hidden danger information in the radar image of the first channel collected by the collection point 2, in which case the collapse may exist between the collection point 1 and the collection point 2, and therefore, it is necessary to fuse the detection results of each collection point to obtain the accurate collapse hidden danger detection result of the to-be-detected road.

[0067] As can be seen from the above, the method provided by the embodiment of the present application does not need to rely on the geological, geophysical and other multi-field professional knowledge and experience of interpreters, avoids the defects of high difficulty and long time consumption of manual interpretation in complex scenes, avoids the fluctuation of interpretation efficiency caused by the difference in professional ability of personnel, and can meet the detection needs of large-scale and rapid expansion of urban roads. In addition, considering that the existing intelligent detection method is mainly for single radar image analysis and cannot associate the spatial features of multiple adjacent images, the method provided by the embodiment of the present application collects multiple radar images at each collection point, randomly combines them, fully utilizes the correlation features between different channel images, captures subtle hidden danger signals that cannot be reflected by a single image, fuses the detection results of multiple sub-models in the detection process to determine the detection result of each collection point, and solves the problem of low detection accuracy in the traditional method. Finally, the obtained radar combination image of each collection point is input into the detection model to predict the risk in advance. Moreover, in the identification process, there is no need for manual analysis of single images one by one, which significantly shortens the time consumption of the whole process from data collection to hidden danger identification, and adapts to the demand of city-level large-scale road hidden danger investigation.

[0068] In an optional embodiment, the step of inputting the at least one radar combination image corresponding to any one collection point into the pre-trained detection model to obtain the detection result corresponding to the collection point in step 130 can include:

[0069] Inputting the at least one radar combination image corresponding to any one collection point into the pre-trained detection model.

[0070] In any one sub-model of the pre-trained detection model, the following steps are performed:

[0071] Extracting the features of the input radar combination image, and obtaining the second preset number of feature maps corresponding to the input radar combination image by the second preset number of convolution layers for each extracted feature.

[0072] Obtaining the detection result of the input radar combination image according to the second preset number of feature maps corresponding to the input radar combination image.

[0073] Obtaining the detection result corresponding to the collection point according to the detection results of the radar combination images corresponding to the collection point.

[0074] In actual application, one radar composition image corresponds to input into one sub-model. Each sub-model has multiple detection channels, each channel being composed of a Visual Geometry Group (VGG) 16-layer convolutional neural network and a convolutional layer. The second preset number represents the number of channels in the sub-model.

[0075] In each sub-model, the extraction depth of each VGG16 network is different. For one radar composition image, the corresponding sub-model extracts different features through different VGG16 networks and inputs the features into different convolutional layers for further strengthening to obtain the second preset number of feature maps corresponding to the radar composition image. The size of the convolutional layer in each channel is determined according to the extraction depth of the corresponding VGG16 network to match it.

[0076] The embodiment converts the detection result recognition process of each collection point into the recognition of each radar composition image corresponding to each collection point, and converts the recognition of each radar composition image into the recognition of each feature map thereof. Through this step-by-step refined recognition, the detection result of each collection point is gradually obtained.

[0077] In an optional embodiment, the step of obtaining the detection result of the input radar composition image according to the second preset number of feature maps corresponding to the input radar composition image can include:

[0078] randomly dividing the second preset number of feature maps corresponding to the input radar composition image into multiple groups, wherein each group includes a third preset number of feature maps; and fusing each group of feature maps to obtain at least one feature fusion image.

[0079] recognizing each feature fusion image to obtain the detection result of the input radar composition image.

[0080] To further enhance the connection between adjacent radar images, in the process of obtaining the detection result corresponding to each radar composition image through the second preset number of feature maps corresponding to each radar composition image, each sub-model performs the following steps:

[0081] randomly dividing the second preset number of feature maps corresponding to the obtained radar composition image into multiple groups; in particular, each group can include three feature maps. This is because the model used in this embodiment uses a three-dimensional intersection-over-union method for corresponding calculation during training. In this case, the obtained feature fusion image is a three-dimensional image. Of course, in an optional embodiment, the third preset number can be selected to be a positive integer greater than 2.

[0082] Here, when selecting the feature maps, it is not necessary to select the radar images of adjacent channels as when selecting the radar combination image, and random selection can be performed.

[0083] The feature maps are further grouped and fused in this embodiment because, before grouping, the sub-models perform candidate box recognition on the second preset number of feature maps corresponding to the radar combination image input thereto. The candidate box recognition can be performed by the PriorBox algorithm, which generates a plurality of candidate boxes of different scales with the center of each pixel point. The number of generated candidate boxes can be 4 or 6, etc. These candidate boxes not only provide a range for recognition, but also reduce the complexity of recognizing the feature fusion images.

[0084] In this way, the candidate boxes included in each feature map in each group are different, the recognition range and recognition focus of each feature fusion image are different, that is, the recognition target is slightly different, and the accuracy of the detection result can be improved by combining the recognition results of these feature fusion images.

[0085] In an optional embodiment, the detection result of the radar combination image includes a category and a position coordinate of the radar combination image; the category is used to represent whether there is a collapse hazard in the radar combination image; and the position coordinate is used to represent the position coordinate of the collapse hazard when the collapse hazard exists.

[0086] The detection result can be obtained by the recognition (prediction) network in each sub-model based on the input feature fusion image. Since the radar image when the collapse hazard exists has obvious feature differences compared with the normal radar image, during recognition, whether each image has a collapse hazard can be determined based on these obvious features. If a collapse hazard exists, the position coordinate of the collapse hazard can be determined by the following method:

[0087] The position coordinate of the collapse hazard can be determined by determining the acquisition point corresponding to the radar combination image to locate the approximate coordinate, determining the radar image corresponding to the radar combination image, and determining the radar channel of the radar image, thereby determining the position coordinate of the collapse hazard.

[0088] Figure 4 is a detection model training flowchart provided by an embodiment of the present application. The training process of the detection model will be described below. Figure 4

[0089] The sample radar combination image included in the training set is input into the sub-model to be trained.

[0090] In the sub-model to be trained, the input sample radar combination image is subjected to feature extraction, and the extracted features are subjected to the second preset number of convolution layers to obtain the second preset number of sample feature maps corresponding to each sample radar combination image.​

[0091] In the sub-model to be trained, candidate boxes of each second preset number of sample feature maps corresponding to the input sample radar combination map are determined; and each second preset number of sample feature maps after the candidate boxes are determined are randomly divided into multiple groups; each group of sample feature maps is fused to obtain at least one first sample feature fusion map; wherein, each group includes a third preset number of sample feature maps; each sub-model to be trained is used to determine the detection result of its input sample radar combination map based on the first sample feature fusion map.

[0092] Upon receiving the command signal, candidate boxes are marked on the second preset number of sample feature maps corresponding to the sample radar composite image, and the second preset number of sample feature maps after the candidate boxes are determined are randomly divided into multiple groups; each group of sample feature maps is fused to obtain at least one second sample feature fusion map; wherein, each first sample feature fusion map corresponds to a second sample feature fusion map; the sample feature map selected in each second sample feature fusion map is the same as the sample feature map selected in its corresponding first sample feature fusion map.

[0093] Based on the feature fusion map of each first sample and its corresponding feature fusion map of the second sample, the crossover ratio is calculated to obtain the target positive sample and the target negative sample.

[0094] Based on the target positive sample, target negative sample, and loss function, the loss value is calculated, and the parameters of the sub-model to be trained are updated based on the loss value to obtain the pre-trained detection model.

[0095] In this embodiment, the pre-trained detection model can be a Single Shot MultiBox Detector (SDD), in which the parameters of the multiple sub-models are the same. Essentially, it is a scalable large model. During the training process, training one sub-model is sufficient to obtain the entire detection model.

[0096] like Figure 4 As shown, the sub-model is trained in the following way:

[0097] Similar to the process of using the detection model, the input data for model training is the sample radar combination map included in the training set. The sample radar combination map included in the training set is obtained based on the initial radar images of each acquisition point collected in historical phases. The method for obtaining the sample radar combination map can refer to the relevant embodiment in step 120; that is, the sample radar combination map used in this embodiment is also obtained by first acquiring initial radar images from each historical acquisition point, and then using those initial radar images.

[0098] In the training process, the embodiment needs to provide an auxiliary model to complete the training of the sub-model, wherein the auxiliary model can be a neural network model.

[0099] The following are the execution steps of the sub-model in the training process:

[0100] In the to-be-trained sub-model, feature extraction is performed on the input sample radar combination image, and the extracted features are passed through a second preset number of convolution layers to obtain a second preset number of sample feature maps corresponding to the sample radar combination image.

[0101] In the to-be-trained sub-model, the bounding boxes of each second preset number of sample feature maps corresponding to the input sample radar combination image are determined; and each second preset number of sample feature maps after determining the bounding boxes is randomly divided into multiple groups; each group of sample feature maps is fused to obtain at least one first sample feature fusion image; wherein each group includes a third preset number of sample feature maps; each to-be-trained sub-model is used to determine the detection result of the input sample radar combination image based on the first sample feature fusion image.

[0102] In the to-be-trained sub-model, feature extraction is performed on the input sample radar combination image, and the extracted features are passed through a second preset number of convolution layers to obtain a second preset number of sample feature maps corresponding to the sample radar combination image.

[0103] The following is the content executed by the auxiliary model:

[0104] The instruction signal is received, the bounding boxes of the second preset number of sample feature maps corresponding to the sample radar combination image are labeled, and each second preset number of sample feature maps after determining the bounding boxes is randomly divided into multiple groups; each group of sample feature maps is fused to obtain at least one second sample feature fusion image; wherein each first sample feature fusion image corresponds to a second sample feature fusion image; the selected sample feature map in each second sample feature fusion image is the same as the selected sample feature map in the corresponding first sample feature fusion image.

[0105] In the embodiment, the instruction signal can be triggered by the staff. Since the feature extraction algorithm at the present stage can achieve high-precision feature extraction, and since subsequent calculations need to ensure that the selected pictures are consistent when compared, when manually labeling the bounding boxes, the second preset number of sample feature maps corresponding to the sample radar combination image obtained by the trained sub-model can be directly used. In this step, the step of obtaining the second sample feature fusion image after labeling the bounding boxes is the same as the step of obtaining the first sample feature fusion image, so it will not be repeated here.

[0106] It should be noted that each set of sample feature maps obtained in this step is the same as the selected picture in each set of sample feature maps obtained in the previous step, and accordingly, the first sample feature fusion map and the second sample feature fusion map obtained are the same. Figure 1 One corresponds to the other, and the only difference between the two is the candidate box.

[0107] According to the first sample feature fusion map and the corresponding second sample feature fusion map, the intersection over union value is calculated to obtain the target positive sample and the target negative sample.

[0108] The embodiment obtains the intersection over union value by comparing the difference between the candidate box in each first sample feature fusion map and the corresponding second sample feature fusion map. The intersection over union value is used to screen and judge each sample radar combination map to determine whether it is a target positive sample or a target negative sample; wherein the target positive sample indicates that there is a collapse hazard in the sample radar combination map; and the target negative sample indicates that there is no collapse hazard in the sample radar combination map.

[0109] According to the target positive sample, the target negative sample and the loss function, the loss value is calculated, and the parameters of the to-be-trained sub-model are updated based on the loss value to obtain the pre-trained detection model.

[0110] In the embodiment, the loss value can be determined in groups, that is, each sample radar combination map corresponds to a group of loss values, and the sum of the loss values corresponding to each sample radar combination map is taken as the total loss value to adjust the model parameters through the total loss value.

[0111] The loss function can be determined by the classification loss and the regression loss.

[0112] In calculating the loss value, take each group as an example:

[0113] First, determine the number of positive samples corresponding to the group, that is, the number of first sample feature fusion maps with collapse hazards corresponding to the sample radar combination map.

[0114] According to the number of positive samples, the value of the target positive sample and the value of the target negative sample, the loss value corresponding to each sample radar combination map is calculated by the loss function.

[0115] The sum of the loss values corresponding to each sample radar combination map is taken as the loss value of the training set.

[0116] If the obtained loss value of the training set is not less than the preset loss threshold, the loss function is used for back propagation to update the parameters of the to-be-trained sub-model, and until the loss value is less than the preset loss value, the pre-trained detection model is obtained based on the to-be-trained sub-model which has completed the training.

[0117] The loss function can be represented as:

[0118]

[0119] In the formula, This represents the loss value corresponding to the radar composite image of the l-th sample. This indicates the number of positive samples corresponding to the radar composite image of this sample; For classification loss; For regression loss; where:

[0120]

[0121] In the formula, The softmax loss is adopted using multi-class confidence. Represents the target negative sample dataset The Middle Zhang's target is a positive sample; In the radar combination diagram, the first Machine-recognized candidate boxes in category y The following and the first j An indicator to determine whether manually annotated candidate boxes match; This represents the predicted confidence score for the target positive sample. Represents the target negative sample dataset The Middle a Zhang's target negative sample; This represents the predicted confidence score for the target negative sample.

[0122] in, ; Similarly.

[0123]

[0124] In the formula, Indicates the total number of radar channels; x y represents the center coordinates of the candidate box; w and h represent the width and height of the candidate box, respectively; This represents the regression loss corresponding to the target positive sample; Indicates the first The radar image corresponding to the radar channel corresponds to the first... The regression bounding box prediction values ​​of each candidate box; Indicates the first The radar image corresponding to the radar channel corresponds to the first... The regression delineation of the true values ​​of the candidate boxes.

[0125] in, , , ,

[0126] Parameter a represents a candidate box a, the center of which is represented as (x, y).

[0127] Each radar combination image corresponds to a historical acquisition point. Based on the sum of loss values of the radar combination images corresponding to each historical acquisition point, the loss value of the historical acquisition point is calculated, and the formula is as follows:

[0128] .

[0129] That is, finally, the model parameters are adjusted according to the loss value L .

[0130] In an optional embodiment, the intersection over union value is calculated according to each first sample feature fusion image and the corresponding second sample feature fusion image to obtain the target positive sample and the target negative sample, including:

[0131] The intersection over union value is calculated according to each first sample feature fusion image, the corresponding second sample feature fusion image and the intersection over union calculation formula.

[0132] Each first sample feature fusion image is screened according to the intersection over union value to obtain the target positive sample and the target negative sample.

[0133] In this embodiment, the intersection over union calculation formula is as follows:

[0134]

[0135] In the formula, the first sample feature fusion image is represented as I, the second sample feature fusion image corresponding to the first sample feature fusion image is represented as J, and the intersection over union value is represented as Iou. represents a cube composed of any one first sample feature fusion image. represents a cube composed of the second sample feature fusion image corresponding to the first sample feature fusion image.

[0136] This is because in this embodiment, when the feature map is screened, every three images form a group, so here the first sample feature fusion image is essentially a cube. According to the obtained intersection over union, the first sample feature fusion image with an intersection over union value greater than a preset value is taken as the target positive sample, and the first sample feature fusion image with an intersection over union value not greater than the preset value is taken as the initial negative sample.

[0137] The obtained initial negative sample is sorted according to the size of the intersection over union value.

[0138] And the initial negative sample with a preset sample quantity is screened from the sorted initial negative sample as the target negative sample; wherein the preset sample quantity is determined based on the quantity of the target positive sample.

[0139] In the above manner, since the number of negative samples is usually much larger than the number of positive samples, in order to ensure sample balance, the number of negative samples needs to be determined according to the number of positive samples, and in general, the number of negative samples is three times the number of positive samples.

[0140] As can be seen from the above, the method provided by the embodiment of the application can fully utilize the features between adjacent images by combining the radar images corresponding to adjacent channels respectively and detecting the radar images of adjacent channels combined together. Then, the combined images are recognized, and the detection results of different combined images are fused, so that potential road collapse hidden dangers can be effectively detected.

[0141] It should be understood that the size of the serial number of each step in the above embodiment does not mean the order of execution, and the execution order of each process should be determined according to its function and internal logic, and should not constitute any limitation on the implementation process of the embodiment of the application.

[0142] The embodiment of the application also provides an electronic device, including a memory and a processor, the memory stores a computer program, and the processor realizes the method in the method embodiment when executing the computer program.

[0143] In the above embodiments, the description of each embodiment has its own emphasis, and the parts not described or recorded in a certain embodiment can be referred to the related description of other embodiments. If there is no special description and no logical conflict, the terms and / or descriptions of different embodiments are consistent and can be mutually referenced, and the technical features in different embodiments can be combined to form a new embodiment according to their inherent logical relationship.

[0144] The above-described embodiments are only used to illustrate the technical solutions of the present application, rather than limit them; although the present application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that they can still modify the technical solutions recorded in the foregoing embodiments, or make equivalent replacements for part of the technical features; and these modifications or replacements do not make the essence of the corresponding technical solutions deviate from the spirit and scope of the technical solutions of the embodiments of the present application, and should be included in the protection scope of the present application.

Claims

1. A method for detecting potential road collapse hazards, characterized in that, include: The road to be detected is detected based on a preset acquisition interval to obtain the initial radar image of each acquisition point; wherein, the initial radar image corresponding to each acquisition point is a multi-channel radar image; each channel corresponds to one initial radar image; For any acquisition point, the initial radar image corresponding to the acquisition point is divided into at least one group according to the channel order, wherein each group includes a first preset number of consecutive adjacent channel radar images, and each group is combined to obtain at least one radar combination image corresponding to the acquisition point. At least one radar composite image corresponding to any acquisition point is input into a pre-trained detection model to obtain the detection result corresponding to that acquisition point; wherein, the detection model includes multiple sub-models, each sub-model is used to detect one radar composite image; the detection result corresponding to that acquisition point is determined based on the detection results of each radar composite image; Based on the detection results corresponding to each collection point, the detection results of the potential collapse of the road to be inspected are determined.

2. The method for detecting road collapse hazards according to claim 1, characterized in that, The step of inputting at least one combined radar image corresponding to any given acquisition point into a pre-trained detection model to obtain the detection result corresponding to that acquisition point includes: Input at least one radar composite image corresponding to any collection point into the pre-trained detection model; Perform the following steps in any sub-model of the pre-trained detection model: Extract the features of the input radar composite image, and pass each extracted feature through a second preset number of convolutional layers to obtain the second preset number of feature maps corresponding to the input radar composite image. Based on the second preset number of feature maps corresponding to the input radar combination map, the detection result of the input radar combination map is obtained. Based on the detection results of each radar combination map corresponding to the acquisition point, the detection result corresponding to the acquisition point is obtained.

3. The method for detecting road collapse hazards according to claim 2, characterized in that, The step of obtaining the detection result of the input radar composite image based on the second preset number of feature images corresponding to the input radar composite image includes: The input radar composite image is randomly divided into multiple groups of a second preset number of feature images; each group includes a third preset number of feature images; each group of feature images is fused to obtain at least one feature fusion image; The feature fusion map is identified to obtain the detection result of the input radar combination map.

4. The method for detecting road collapse hazards according to claim 3, characterized in that, Before fusing each set of feature maps separately to obtain at least one fused feature map, the method further includes: Multiple candidate boxes with different proportions are determined in the second preset number of feature maps corresponding to the input radar composite map; wherein, the candidate boxes are used to reduce the complexity of recognizing each feature fusion map.

5. The method for detecting road collapse hazards according to claim 3, characterized in that, The detection result of the radar composite image includes the category and location coordinates of the radar composite image; wherein, the category is used to characterize whether there is a collapse hazard in the radar composite image; the location coordinates characterize the location coordinates of the collapse hazard when there is a collapse hazard.

6. The method for detecting road collapse hazards according to claim 1, characterized in that, The pre-trained detection model is obtained in the following way: The sample radar combination map included in the training set is used as input and fed into the sub-model to be trained. In the sub-model to be trained, features are extracted from the input sample radar combination map, and the extracted features are passed through a second preset number of convolutional layers to obtain a second preset number of sample feature maps corresponding to each sample radar combination map. In the sub-model to be trained, candidate boxes of each second preset number of sample feature maps corresponding to the input sample radar combination map are determined; and each second preset number of sample feature maps after the candidate boxes are determined are randomly divided into multiple groups; each group of sample feature maps is fused to obtain at least one first sample feature fusion map; wherein, each group includes a third preset number of sample feature maps; each sub-model to be trained is used to determine the detection result of its input sample radar combination map based on the first sample feature fusion map; Upon receiving the command signal, candidate boxes are marked on the second preset number of sample feature maps corresponding to the sample radar combination map. The second preset number of sample feature maps after the candidate boxes are determined are randomly divided into multiple groups. Each group of sample feature maps is fused to obtain at least one second sample feature fusion map. Each first sample feature fusion map corresponds to a second sample feature fusion map. The sample feature map selected in each second sample feature fusion map is the same as the sample feature map selected in its corresponding first sample feature fusion map. Based on the feature fusion map of each first sample and its corresponding feature fusion map of the second sample, the crossover ratio is calculated to obtain the target positive sample and the target negative sample. Based on the target positive sample, the target negative sample, and the loss function, a loss value is calculated, and the parameters of the sub-model to be trained are updated based on the loss value to obtain a pre-trained detection model.

7. The method for detecting road collapse hazards according to claim 6, characterized in that, The step of calculating the intersection-union ratio (IUGR) based on each of the first sample feature fusion maps and its corresponding second sample feature fusion maps to obtain the target positive sample and the target negative sample includes: Calculate the crossover ratio (CUP) value based on the feature fusion map of each first sample, the feature fusion map of its corresponding second sample, and the crossover ratio calculation formula; Based on the intersection-union ratio, the feature fusion maps of each first sample are filtered to obtain target positive samples and target negative samples.

8. The method for detecting road collapse hazards according to claim 7, characterized in that, The step of filtering the feature fusion maps of each first sample according to the intersection-union ratio to obtain target positive samples and target negative samples includes: The first sample feature fusion map with an intersection-union ratio greater than a preset value is used as the target positive sample, and the first sample feature fusion map with a ratio not greater than the preset value is used as the initial negative sample. The initial negative samples are sorted according to their intersection-union ratio (IU). And from the sorted initial negative samples, a preset number of initial negative samples are selected as target negative samples; wherein, the preset number of samples is determined based on the number of target positive samples.

9. The method for detecting road collapse hazards according to claim 6, characterized in that, The step of calculating a loss value based on the target positive sample, the target negative sample, and the loss function, and updating the parameters of the sub-model to be trained based on the loss value to obtain a pre-trained detection model includes: Determine the number of positive samples corresponding to each sample radar combination map included in the training set; Based on the number of positive samples, the value of the target positive sample, and the value of the target negative sample, the loss value corresponding to each sample radar composite image is calculated using the loss function; The sum of the loss values ​​corresponding to the combined radar images of each sample is used as the loss value of the training set. If the loss value is not less than the preset loss threshold, backpropagation is performed based on the loss function to update the parameters of the sub-model to be trained until the loss value is less than the preset loss value, thus obtaining the pre-trained detection model.

10. An electronic device, characterized in that, It includes a memory and a processor, the memory storing a computer program, and the processor executing the computer program to implement the method as described in any one of claims 1 to 9.

Images