A method for detecting urban road damage based on radar images

Abstract

The present invention discloses a method for detecting urban road defects based on radar maps, including establishing a Gaussian blurred image layer based on images before and after Gaussian blur, performing differential processing on the image layer, selecting a differential image of a suitable level to be superimposed on the original image to enhance the feature performance of the image; adding a deconvolution layer to the shallow network part of the single-step multi-frame detection model, integrating the semantic information of the high-level image into the feature information of the low-level network, and adding a convolution layer to the low-level network to extract deep features, performing selective feature fusion according to the different levels of the feature map, adopting a step-by-step joint training strategy, first training a disease data set with a low detection accuracy to obtain a preliminary training weight, and then selecting the weight with the best performance in detecting the disease type as the pre-training weight to train a neural network to detect all disease types. The present invention is helpful to improve the recognition accuracy of target diseases in road maps.

Description

Technical Field

[0001] The present invention relates to the technical field of disease detection, and in particular to a method for detecting urban road diseases based on radar spectra. Background Art

[0002] As an effective technology for detecting underground conditions, ground penetrating radar has the advantages of being non-destructive, rapid and continuous. The amount of road data collected by ground penetrating radar is huge. With the continuous expansion of urban roads, it is very time-consuming to manually detect the defects in radar data, and there are certain subjective factors.

[0003] In recent years, with the rapid development of computer technology, deep learning, as the mainstream technology in the field of artificial intelligence recognition, has been widely used in various scenarios of target detection. Due to the open source of the classic deep learning algorithm detection model and the good portability of the model, in order to solve the inconvenience brought by the traditional road disease recognition method, modern road researchers have turned their attention to the automatic, efficient and low-manpower-cost ground penetrating radar map detection based on deep learning. The characteristics of different types of road underground diseases are relatively similar, which makes detection difficult. How to further improve the accuracy of road disease detection has become the focus of research by scholars from all over the world. The work carried out on ground penetrating radar detection is mainly divided into the following two directions: ground penetrating radar target detection based on improved algorithms and ground penetrating radar target detection based on multi-type radar data. Ground penetrating radar target detection based on improved algorithms can be subdivided into radar target detection based on traditional machine learning and radar target detection based on deep learning.

[0004] Traditional machine learning algorithms provide an effective model for ground penetrating radar target detection problems. The positive results achieved in radar road map recognition also prove the feasibility of automated road disease detection. Compared with target detection based on traditional machine learning algorithms, target detection based on deep learning algorithms is far superior to traditional algorithms in terms of accuracy and speed. With the development of deep learning, the quality of the algorithm will affect the accuracy of map recognition, and the number of data sets will also have a great impact on the detection accuracy. Generally speaking, the larger the data set, the better the recognition effect, but the corresponding training time will increase and the training cost will be higher. Summary of the invention

[0005] The purpose of the present invention is to provide a method for detecting urban road damage based on radar images to solve the problems raised in the above background technology.

[0006] To achieve the above object, the present invention provides the following technical solutions:

[0007] A method for detecting urban road damage based on radar images comprises the following steps:

[0008] S1. Obtaining original image data, which is a ground penetrating radar map, Gaussian blurring the ground penetrating radar map, establishing a Gaussian blurred image layer based on the images before and after Gaussian blurring, and then performing differential processing on the image layer, selecting a differential image of a suitable level and superimposing it on the original image to enhance the feature expression of the image;

[0009] S2. Use the single-step multi-frame detection model as the backbone network to construct a multi-layer semantic fusion network. Add a deconvolution layer to the shallow network part of the multi-layer semantic fusion network to integrate the high-level semantic information of the image into the feature information of the low-level network. Add a convolution layer to the low-level network to extract deep features. Selectively fuse features according to the different levels of the feature map.

[0010] S3. Based on a multi-layer semantic fusion network, a step-by-step joint training strategy is adopted. The disease data set with low detection accuracy is first trained to obtain preliminary training weights. Then, the weights with the best performance in detecting the disease type are selected as pre-training weights to train a neural network model for detecting all disease types, and the neural network model is used to identify disease features.

[0011] Furthermore, step S1 specifically includes: performing differential processing on the image layer to enhance features by using a Gaussian difference method, converting the original image into a grayscale image during Gaussian processing, and then establishing a Gaussian blurred image layer based on a Gaussian function.

[0012] Furthermore, the Gaussian blurred image is generated by convolution of the Gaussian function and the original image. The formula of the two-dimensional Gaussian function and the function of the Gaussian blurred image are:

[0013] ;

[0014] ;

[0015] in, is a Gaussian function, is the coordinate of the image pixel in two-dimensional space, is the standard deviation of the Gaussian distribution, is the grayscale image converted from the original image, is a Gaussian blurred image, is the convolution operator.

[0016] Furthermore, according to the reference value of the SIFT algorithm, four layers of Gaussian blur image layers are established for each input image. The image size of each layer is equal to the size of the input image. For different layers, The value calculation formula is as follows:

[0017] ;

[0018] is the Gaussian convolution kernel parameter of the first layer, which is a constant ,coefficient , The number of layers.

[0019] Furthermore, when the Gaussian blurred image has four layers, the Gaussian difference image has three layers. By differentiating the four layers of Gaussian blurred image layers, a three-layer difference image can be obtained. The process of obtaining the three-layer difference image is calculated using the following formula:

[0020] ;

[0021] ;

[0022] For the The Gaussian function of the layer, For the Layer Gaussian difference function, is the Gaussian difference image.

[0023] Furthermore, step S2 specifically includes: first defining the depth of each layer of the multi-layer semantic fusion network, namely shallow layer, middle layer, middle-deep layer, deep layer and deeper layer; superimposing deconvolution feature maps on the shallow layer, superimposing shallow convolution feature maps and deep deconvolution feature maps on the middle-deep layer; superimposing shallow convolution feature maps on the deep layer; and performing scale fusion on the feature maps of the deeper layer that have not been convolutionally output.

[0024] Furthermore, some convolutional layers and pooling layers of the VGG16 network are used. The convolutional layers are conv3-2, conv4-3, conv5-2, conv6, conv7, conv8-2, conv9-2, conv10-2 and conv11-2, and the pooling layers are pool4 and pool5; conv3-2, conv4-3, pool4, conv5-2, and pool5 are in the shallow layer of the network structure, conv6 and conv7 are in the middle layer, conv8-2 and conv9-2 are in the middle and deep layers, the convolutional layer conv10-2 is a deep layer, and the convolutional layer conv11-2 is a deeper feature map.

[0025] Furthermore, step S1 specifically includes:

[0026] S11. First, based on the preprocessed full-category data set, initialize the network weights and conduct training. Select the target class with poor detection effect according to the indicators on the test set and train it separately, which is single-category detection.

[0027] S12. After the training is completed, repeat the above S11 test process, select the network weight with the best single-category detection performance, and use it as the pre-training weight to perform network training on all categories of targets again to obtain the detection network and the neural network model.

[0028] Furthermore, the quantitative relationship between the types of training selected during the training process and the total types of detection targets satisfies the following formula:

[0029] ;

[0030] in, is the number of training types selected in step-by-step training, The total number of detected target types.

[0031] To achieve the above object, the present invention also provides the following technical solutions:

[0032] A computer device comprises a memory and a processor, wherein the memory stores a computer program, and the processor implements the steps of any one of the above methods when executing the computer program.

[0033] To achieve the above object, the present invention also provides the following technical solutions:

[0034] A computer-readable storage medium stores a computer program, which, when executed by a processor, implements the steps of any one of the methods described above.

[0035] Compared with the prior art, the present invention has the following beneficial effects:

[0036] The present invention, by improving the single-step multi-frame detection model, proposes a novel method based on a multi-layer semantic fusion network and Gaussian difference map enhancement features, which helps to improve the recognition accuracy of target diseases in road maps.

[0037] The present invention, through a step-by-step joint training strategy, has a strong recognition of multiple types of diseases that are similar in appearance and difficult to identify. This strategy does not increase the model burden and can be easily adapted to various detection models. BRIEF DESCRIPTION OF THE DRAWINGS

[0038] Figure 1 The present invention is a flow chart of the urban road disease detection method based on radar spectrum.

[0039] Figure 2 This is a diagram of the Gaussian difference feature enhancement process of the present invention.

[0040] Figure 3 This is a diagram showing the recognition effect of the graph before feature enhancement after the Gaussian difference enhancement feature is introduced in the present invention.

[0041] Figure 4 This is a diagram showing the recognition effect of the spectrum after the Gaussian difference enhancement feature is introduced in the present invention.

[0042] Figure 5 A multi-layer semantic fusion network structure diagram constructed by the method of the present invention.

[0043] Figure 6 The figure is a structural block diagram of the urban road disease detection system based on radar spectrum of the present invention.

[0044] Figure 7 It is a diagram of the internal structure of the computer device of the present invention. DETAILED DESCRIPTION

[0045] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0046] Target detection algorithms are mainly divided into two categories: one-stage detection algorithms and two-stage detection algorithms. In view of the problems existing in algorithms and data types, existing research usually improves the algorithm network to improve detection accuracy, while image data with enhanced features and appropriate training methods can also help with disease detection. The present invention improves the accuracy of disease detection by constructing a new network, data enhancement and proposing a new training method.

[0047] At present, in the field of road damage detection, research on algorithms has always been the focus of this field. The present invention proposes an urban road damage detection model based on radar maps. On the one hand, it focuses on the detection algorithm, and on the other hand, it also attempts to study at the data level and training method level. The present invention has made improvements in three aspects: data, network structure and training strategy, aiming to provide an urban road damage detection model for radar maps based on feature enhancement and step-by-step joint training strategy to improve the accuracy of algorithm recognition.

[0048] To solve the above problems, please refer to Figures 1 to 7 , the present invention provides a technical solution:

[0049] (1) Enhance the feature expression of the image based on the Gaussian difference map. In order to solve the shortcomings of the ground penetrating radar map, which has low resolution and not rich grayscale levels, the present invention establishes a Gaussian blurred image layer based on the image before and after Gaussian blur, and then performs differential processing on the image layer. The differential image of the appropriate level is selected and superimposed on the original image to enhance the feature expression of the image.

[0050] (2) The present invention constructs a multi-layer semantic fusion network. The present invention defines the depth of each layer of the network, adds a deconvolution layer to the shallow network part of the single-step multi-frame detection model, integrates the semantic information of the high-level image into the feature information of the low-level network, and adds a convolution layer to the low-level network to extract deep features, and selectively fuses features according to the different levels of the feature map.

[0051] (3) A step-by-step joint training strategy is adopted. The disease data set with low detection accuracy is first trained to obtain preliminary training weights. Then, the weights with the best performance in detecting the disease type are selected as pre-training weights to train the neural network to detect all disease types.

[0052] The present invention mainly includes three aspects: firstly, the ground penetrating radar image is processed by using Gaussian difference to enhance the image features; secondly, a multi-layer semantic fusion network is proposed, that is, the scale feature fusion of the feature map output by the single-step multi-frame detection model is performed through convolution, deconvolution and superposition processing to enhance the useful information of the feature map; finally, the initial test result indicators are analyzed, and a step-by-step joint training strategy is proposed.

[0053] 1. Gaussian difference feature enhancement

[0054] Feature enhancement using Gaussian difference is an operation based on the image before and after Gaussian blur. Blurring can be understood as the process of taking the weighted average of image pixels, and Gaussian blur is the process of recalculating pixel values ​​by weighting them according to the Gaussian function. When performing Gaussian processing, this paper needs to convert the original image into a grayscale image and then establish a Gaussian blurred image layer based on the Gaussian function. The Gaussian blurred image is generated by the convolution of the Gaussian function and the original image. The formula of the two-dimensional Gaussian function and the function of the Gaussian blurred image are:

[0055] ;

[0056] ;

[0057] in, is a Gaussian function, is the coordinate of the image pixel in two-dimensional space, is the standard deviation of the Gaussian distribution, is the grayscale image converted from the original image, is a Gaussian blurred image, is the convolution operator.

[0058] like Figure 2 As shown in Figure 2, this paper establishes 4 layers of Gaussian blur image layers for each input image based on the reference value of the SIFT algorithm. The image size of each layer is equal to the size of the input image. For different layers, The value calculation formula is as follows:

[0059] ;

[0060] in, is the Gaussian convolution kernel parameter of the first layer, which is a constant ,coefficient , is the number of layers. In the Gaussian blur process, as the number of layers increases, The larger the value of is, the higher the blur. By differentiating the 4-layer Gaussian blurred image layer, we can get 3 layers of differential images. The calculation process is as follows:

[0061] ;

[0062] ;

[0063] in, For the The Gaussian function of the layer, For the Layer Gaussian difference function, is the Gaussian difference image. When the Gaussian blurred image has 4 layers, the Gaussian difference image has 3 layers in total. Figure 2 In the figure, the red dotted box area is the place where the stronger features are expressed in the original image. It can be seen that the third layer difference image obtained by the difference between the third and fourth layers contains stronger features of the disease and very few background features. Therefore, the last layer, that is, the third layer difference image, is selected as the enhanced feature map and superimposed on the original image to generate a Gaussian feature enhanced radar image.

[0064] In the present invention, 3 difference images can be obtained by establishing 4 layers. Figure 2 It can be seen from the red framed areas of the three difference images during the difference process that the third difference image (the fourth layer) contains almost only the target area features, which is equivalent to superimposing it on the original image to only enhance the target features without enhancing the background (enhancing the background is equivalent to adding noise, which is not conducive to work). The second and first difference images contain some background information and are not suitable as enhanced information. If there is one more layer, the image will be more blurred, and the target information will be weakened after the difference image comes out. This operation will also increase the amount of calculation for the early processing. Therefore, it is more appropriate to choose the third difference image from the four blurred image layers.

[0065] The SIFT algorithm was originally used in image registration and other aspects to propose scale-invariant features. The present invention draws on the principle part of it and proposes a feature enhancement method suitable for road image detection tasks by extracting the difference map and superimposing it on the original image. The key to the present invention is to expand the application of the principle. In the detection scene task, no other solution adopts a feature enhancement method that crosses research and tasks.

[0066] 2. Multi-layer semantic fusion network

[0067] Most of the road disease targets are small targets. The feature graphs extracted by the shallow network of the single-step multi-frame detection model have strong geometric detail information representation capabilities, but weak semantic information representation capabilities, which is not conducive to the detection of small targets. Therefore, the present invention constructs a multi-layer semantic fusion network with the single-step multi-frame detection model as the backbone network. According to the characteristics of different levels of the network structure, the present invention adopts different fusion strategies for feature graphs of different levels, so that the fusion operation is more in line with the characteristics of the network structure and feature graphs, and improves the pertinence and effectiveness of feature fusion. A deconvolution layer is added to the shallow network part of the multi-layer semantic fusion network to integrate the semantic information of the high-level image into the feature information of the low-level network, and a convolution layer is also added to the low-level network for deep feature extraction. Through feature fusion strategies at different levels, it can better adapt to the detection needs of targets of different sizes and improve the comprehensiveness and accuracy of detection.

[0068] The multi-layer semantic fusion network structure is as follows Figure 5 As shown in Figure 2, due to the relatively complex network structure, Figure 5 The main modules are listed in , while the padding module and the normalization module are not included. In the multi-layer semantic fusion network, this paper uses some convolutional layers and pooling layers of the VGG16 network, namely convolutional layer conv3-2, convolutional layer conv5-2, pooling layer pool4 and pooling layer pool5. Figure 5 middle represents the deconvolution operation, Represents a convolution operation (the size of the convolution kernel is 3×3), which can effectively eliminate the aliasing effect caused by scale fusion. All feature maps that have undergone scale feature fusion have undergone convolution to eliminate the aliasing effect. It also represents the convolution operation. Its structure is the same as the anti-aliasing convolution operator, but its function is to extract deeper features. Represents the superposition operation. First, the depth of each layer of the multi-layer semantic fusion network is defined, which are shallow, middle, middle-deep, deep and deeper. The shallow network is Figure 3The conv3-2, conv4-3, pool4, conv5-2, and pool5 in the network are conv6 (fc6) and conv7 (fc7). The middle layers are conv8-2 and conv9-2, which are middle-to-deep layers. Conv10-2 is defined as a deep network, and the last conv11-2 is a deeper layer. For the convolutional layers conv4-3 and conv7 (fc7), it is considered that these two layers are in the shallow and middle layers of the network structure, and the deconvolution feature map is superimposed on them. In order to ensure the validity of the superimposed features and take into account the scale transition problem, the deconvolution layers superimposed on conv4-3 and conv7 (fc7) are also transformed from adjacent layers. Conv8-2 and conv9-2 are divided into middle-to-deep layers, so shallow convolution feature maps and deep deconvolution feature maps are superimposed on conv8-2 and conv9-2 to enhance the detail information representation ability and semantic information representation ability of conv8-2 and conv9-2. The convolution layer conv10-2 is defined as the deep network structure, and a shallow convolution feature map is superimposed on it to make it contain more geometric detail information. The convolution layer conv11-2 is a deeper feature map. For this layer, if a shallow convolution feature map is superimposed, the amount of calculation will increase, the time cost will increase, and there is uncertainty as to whether the detection accuracy of large targets will decrease and whether the detection accuracy of small targets will increase. Therefore, the feature map output by the last convolution layer is not scale-fused.

[0069] 3. Step-by-step joint training

[0070] The performance of target detection is related to the data set and algorithm, and also has a great relationship with the training strategy. The step-by-step joint training strategy is a multi-process network training method. First, (Process 1) based on the preprocessed full-category data set, initialize the network weights and conduct training. Select the target class with poor detection effect according to the indicators on the test set and train it separately. (Process 2) After the training is completed, repeat the above test process, select the network weight with the best single-class detection performance, and use it as the pre-training weight to train the network again for all categories of targets to obtain the detection network. It is worth mentioning that the network obtained in the second training process does not bring an improvement in accuracy for all target classes in the detection task. Therefore, the above process can be repeated according to the detection results to finally obtain the optimal detection network. When performing repeated training, it can be not only a single-class step-by-step training, but also two or three categories of all types. The quantitative relationship between the training types selected during the training process and the total types of detection targets should satisfy:

[0071] ;

[0072] in, is the number of training types selected in step-by-step training, The total number of detected target types.

[0073] In the present invention, in a multi-class detection task scenario, if there are several classes with very low recognition accuracy, these classes can be trained first (process 1), and the optimal weights can be taken as the pre-training model to train the general detector (process 2), rather than just selecting the lowest class in process 1.

[0074] Comparative Example 1

[0075] The method of the present invention is used to perform Gaussian difference feature enhancement processing on the road map, such as Figure 3 to Figure 4 After the features shown are enhanced, the target that was originally not recognized can be accurately recognized.

[0076] Comparative Example 2

[0077] The YOLO v5 algorithm, Faster R-CNN algorithm, Deformable DETR algorithm, single-step multi-frame detection model and the present invention are respectively used to identify disease features for the data enhanced graph in Example 1, and the results are shown in Table 1. It can be seen that the improved SSD algorithm has the highest mean average detection accuracy (mAP) among several algorithms, and the algorithm of the present invention is the best in single-class target detection and overall performance compared to the four test algorithms, and has achieved the highest average detection accuracy.

[0078] Table 1 Existing algorithm tests based on radar images after data enhancement

[0079]

[0080] The superiority of the present invention is further demonstrated through experiments below, where +Guass_Strong represents the introduction of Gaussian difference enhancement, and the experimental results are shown in Tables 2 and 3 respectively. As can be seen from Table 2, the enhancement of the ground penetrating radar image by Gaussian difference has improved the average detection accuracy of the algorithm to a certain extent. As can be seen from Table 3, the multi-layer semantic fusion network constructed in this paper has achieved good detection results in the detection of various types of diseases, which fully demonstrates the effectiveness of the constructed network model. In addition, compared with the original network, the improved multi-layer semantic fusion network only loses 2.45FPS in the detection frame number (Frames per second, FPS) indicator. In summary, the multi-layer semantic fusion network ensures the detection speed and accuracy of the image. FPS is used to measure the image processing speed or real-time performance.

[0081] Table 2 Comparison of detection results of algorithms before and after Gaussian difference enhancement feature

[0082]

[0083] Table 3 Comparison of detection results of multi-layer semantic fusion network

[0084]

[0085] The following experiments are used to illustrate the effectiveness of the step-by-step joint training method, and the experimental results are shown in Table 4. As can be seen from Table 4, the multi-layer semantic fusion network constructed in this paper combined with the step-by-step joint training method has achieved better detection results in the detection of various types of diseases than the multi-layer semantic fusion network alone, which fully demonstrates the effectiveness of the step-by-step joint training method.

[0086] Table 4 Comparison of algorithm detection results before and after the introduction of step-by-step joint training method

[0087]

[0088] like Figure 6 The present invention also provides a device for constructing an urban road disease detection model based on radar maps, including: a memory for storing computer programs; a processor for executing the urban road disease detection (model construction) method based on radar maps during actual operation.

[0089] When the processor executes the computer program, the steps of the above-mentioned method for detecting urban road damage (building a model) based on radar maps are implemented. Alternatively, when the processor executes the computer program, the functions of each module in the above-mentioned system are implemented, for example, Figure 7 As shown:

[0090] An urban road disease detection (model construction) system based on radar maps, comprising:

[0091] The difference module is used to obtain the original image, which is a ground penetrating radar map. The ground penetrating radar map is Gaussian blurred, and a Gaussian blurred image layer is established based on the images before and after Gaussian blurring. Then, the image layer is differentially processed, and the differential image of the appropriate level is selected and superimposed on the original image to enhance the feature expression of the image.

[0092] The construction module is used to construct a multi-layer semantic fusion network with a single-step multi-frame detection model as the backbone network, add a deconvolution layer to the shallow network part of the multi-layer semantic fusion network, integrate the high-level semantic information of the image into the feature information of the low-level network, and add a convolution layer to the low-level network to extract deep features, and perform selective feature fusion according to the different levels of the feature map;

[0093] The training module is used to train the disease data set with low detection accuracy based on a multi-layer semantic fusion network and adopt a step-by-step joint training strategy. It first trains the disease data set with low detection accuracy to obtain preliminary training weights, and then selects the weights with the best performance in detecting the disease type as pre-training weights to train the neural network model for detecting all disease types.

[0094] The equipment for constructing the urban road disease detection model based on radar maps can adopt various types of computing devices such as desktop computers, notebooks, PDAs and cloud servers. This equipment is composed of multiple components such as processors and memories, but is not limited to these and may include other components or different combinations. The equipment examples provided here are for illustrative purposes only and are not restrictive.

[0095] The processor can be various types of devices, such as CPU, DSP, ASIC, FPGA, etc., and can also include programmable logic devices, discrete gates, transistor logic devices and discrete hardware components, etc. The memory is used to store computer programs and modules. The processor runs or executes the computer program modules stored in the memory and calls the data stored in the memory, thereby realizing various functions of the microgrid day-ahead economic dispatch model.

[0096] The memory may mainly include a program storage area and a data storage area, wherein the program storage area may store an operating system, an application required for at least one function (such as a sound playback function, an image playback function, etc.), etc.; the data storage area may store data created according to the use of the mobile phone (such as audio data, a phone book, etc.), etc. In addition, the memory may include a high-speed random access memory, and may also include a non-volatile memory, such as a hard disk, a memory, a plug-in hard disk, a smart memory card (SmartMediaCard, SMC), a secure digital (SecureDigital, SD) card, a flash card (FlashCard), at least one disk storage device, a flash memory device, or other volatile solid-state storage devices.

[0097] The present invention also provides a computer-readable storage medium, which stores a computer program. When the computer program is executed by a processor, the steps of the urban road disease detection (model construction) method based on radar maps are implemented.

[0098] If the urban road disease detection (model construction) method based on radar atlas is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage device.

[0099] Based on such understanding, the present invention implements all or part of the processes in the above-mentioned method for detecting urban road defects (building a model) based on radar maps, and can also be completed by instructing related hardware through a computer program, and the computer program can be stored in a computer-readable storage medium, and when the computer program is executed by a processor, the steps of the above-mentioned method for detecting urban road defects (building a model) based on radar maps can be implemented. Among them, the computer program includes computer program code, and the computer program code can be in source code form, object code form, executable file or preset intermediate form, etc.

[0100] Computer-readable storage media may refer to any entity or device that can carry computer program code, including recording media, USB flash drives, mobile hard drives, disks, optical disks, computer memory, read-only memory (ROM), random access memory (RAM), software distribution media, etc.

[0101] It should be noted that the content contained in the computer-readable storage medium can be appropriately increased or decreased according to the requirements of legislation and patent practice in the jurisdiction. For example, in some jurisdictions, according to legislation and patent practice, computer-readable storage media do not include electrical carrier signals and telecommunication signals.

[0102] Although embodiments of the present invention have been shown and described, it will be appreciated by those skilled in the art that various changes, modifications, substitutions and variations may be made to the embodiments without departing from the principles and spirit of the present invention, and that the scope of the present invention is defined by the appended claims and their equivalents.

Claims

1. A method for detecting urban road damage based on radar images, characterized in that: The steps include: S1. Obtaining original image data, which is a ground penetrating radar map, Gaussian blurring the ground penetrating radar map, establishing a Gaussian blurred image layer based on the images before and after Gaussian blurring, and then performing differential processing on the image layer, selecting a differential image of a suitable level and superimposing it on the original image to enhance the feature expression of the image; S2. Use the single-step multi-frame detection model as the backbone network to construct a multi-layer semantic fusion network. Add a deconvolution layer to the shallow network part of the multi-layer semantic fusion network to integrate the high-level semantic information of the image into the feature information of the low-level network. Add a convolution layer to the low-level network to extract deep features. Selectively fuse features according to the different levels of the feature map. S3. Based on a multi-layer semantic fusion network, a step-by-step joint training strategy is adopted to first train the disease data set with low detection accuracy to obtain preliminary training weights, and then the weights with the best performance in detecting the disease type are selected as pre-training weights to train a neural network model for detecting all disease types, and the neural network model is used for disease feature recognition; Step S1 specifically includes: S11. First, based on the preprocessed full-category dataset, initialize the network weights and conduct training. Select the target class with poor detection effect according to the indicators on the test set and train it separately. S12, after the training is completed, repeat the above S11 test process, select the network weight with the best single-category detection performance, and use it as the pre-training weight to perform network training on all categories of targets again to obtain the detection network and the neural network model; The quantitative relationship between the types of training selected during the training process and the total types of detection targets satisfies the following formula: Among them, n is the number of training types selected in the step-by-step training, and N is the total number of detection target types.

2. The method for detecting urban road damage based on radar spectrum according to claim 1, characterized in that: Step S1 specifically includes: performing differential processing on the image layer to enhance features by using a Gaussian difference method, converting the original image into a grayscale image during Gaussian processing, and then establishing a Gaussian blurred image layer based on a Gaussian function.

3. The urban road disease detection method based on radar spectrum as claimed in claim 2, characterized in that: The Gaussian blurred image is generated by convolution of the Gaussian function and the original image. The formula of the two-dimensional Gaussian function and the function of the Gaussian blurred image are: g(x,y)=G(x,y)*f(x,y); Where G is the Gaussian function, (x, y) is the coordinate of the image pixel in the two-dimensional space, σ is the standard deviation of the Gaussian distribution, f(x, y) is the grayscale image converted from the original image, g(x, y) is the Gaussian blurred image, and * is the convolution operator.

4. The method for detecting urban road damage based on radar spectrum as claimed in claim 3, characterized in that: According to the reference value of the SIFT algorithm, four layers of Gaussian blur image layers are established for each input image. The image size of each layer is equal to the size of the input image. For different layers, the calculation formula for the value of σ is as follows: σ=k n-1 σ0; σ0 is the Gaussian convolution kernel parameter of the first layer, which is a constant coefficient n is the number of layers.

5. The method for detecting urban road damage based on radar spectrum as claimed in claim 4, characterized in that: When the Gaussian blurred image has four layers, the Gaussian difference image has three layers. By differentiating the four layers of Gaussian blurred image layers, a three-layer difference image can be obtained. The process of obtaining the three-layer difference image is calculated using the following formula: DOG n (x,y)=G n (x,y)-G n+1 (x,y); g DOG (x,y)=DOG(x,y)*f(x,y); G n (x,y) is the Gaussian function of the nth layer, DOG n (x, y) is the Gaussian difference function of the nth layer, g DOG (x,y) is the Gaussian difference image.

6. The method for detecting urban road damage based on radar spectrum according to claim 1, characterized in that: Step S2 specifically includes: first defining the depth of each layer of the multi-layer semantic fusion network, namely shallow layer, middle layer, middle-deep layer, deep layer and deeper layer; superimposing deconvolution feature maps on the shallow layer, superimposing shallow convolution feature maps and deep deconvolution feature maps on the middle-deep layer; superimposing shallow convolution feature maps on the deep layer; and performing scale fusion on the feature maps of the deeper layer that have not been convolutionally output.

7. The method for detecting urban road damage based on radar spectrum as claimed in claim 6, characterized in that: Some convolutional layers and pooling layers of the VGG16 network are used. The convolutional layers are conv3-2, conv4-3, conv5-2, conv6, conv7, conv8-2, conv9-2, conv10-2 and conv11-2, and the pooling layers are pool4 and pool5 respectively; conv3-2, conv4-3, pool4, conv5-2, and pool5 are in the shallow layer of the network structure, conv6 and conv7 are in the middle layer, conv8-2 and conv9-2 are in the middle and deep layers, convolutional layer conv10-2 is a deep layer, and convolutional layer conv11-2 is a deeper feature map.

Images