Pavement defect detection method, pavement defect detection equipment and storage medium

Abstract

The invention relates to the technical field of pavement detection, in particular to a pavement defect detection method, pavement defect detection equipment and a storage medium. The pavement defect detection method comprises the following steps: firstly, obtaining a to-be-detected pavement image; inputting the to-be-detected pavement image into the trained pavement defect detection model to obtain a defect detection result of the to-be-detected pavement; the defect detection result comprises defect-free and existing defect types and defect areas, and the defect types comprise crack, net crack and asphalt strip crack. Wherein the pavement defect detection model comprises a teacher network, a reconstructed student network and a segmentation network, and meanwhile, a multi-scale feature fusion module is introduced in front of a decoder of the student network, so that feature information of different scales can be better restored. It can be seen that the adaptive pavement defect detection model is designed for pavement defect detection, the pavement defect detection efficiency is improved, and the method can be suitable for detection of various defect types.

Description

Technical Field

[0001] The present invention relates to the technical field of road surface detection, and in particular to a road surface defect detection method, road surface defect detection equipment and a storage medium. Background Art

[0002] As the road surface ages and becomes damaged, it will have defects of varying degrees. Pavement cracks are a common defect on the road surface. Pavement cracks mainly include cracks, web cracks, asphalt cracks, etc. These cracks are caused by factors such as pavement aging, water penetration, and climate change. The presence of these cracks will aggravate the damage to the road and pose a potential threat to driving safety. Therefore, it is necessary to detect this potential danger in a timely manner and repair it.

[0003] In related technologies, pavement crack detection is generally based on digital image processing technology, which combines image enhancement, image denoising and other pre-processing algorithms with image segmentation algorithms such as threshold segmentation and edge detection to improve the efficiency of crack feature extraction. However, due to the different characteristics of pavement images, the pavement crack detection method based on image processing cannot adapt to different defect types, resulting in low efficiency in pavement defect detection. Summary of the invention

[0004] The main purpose of the present invention is to provide a road surface defect detection method, a road surface defect detection device and a storage medium, aiming to solve the technical problem of low efficiency of road surface defect detection in related technologies.

[0005] In one aspect, the present application provides a road surface defect detection method, comprising:

[0006] Acquire an image of the road surface to be tested;

[0007] Inputting the road surface image to be tested into the trained road surface defect detection model to obtain the defect detection result of the road surface to be tested; the defect detection result includes the defect type and defect area of ​​no defect and existing defect, and the defect type includes cracks, web cracks and asphalt strip cracks;

[0008] The road surface defect detection model includes a teacher network, a reconstruction student network and a segmentation network;

[0009] Among them, the teacher network is used to extract features of the input road surface image to be tested, and obtain first feature information, second feature information and third feature information of different sizes; the reconstruction student network is used to extract and reconstruct the input road surface image to be tested, and obtain fourth feature information, fifth feature information and sixth feature information of different sizes after reconstruction; the segmentation network is used to perform feature fusion and feature sampling on the first feature information, the second feature information, the third feature information, the fourth feature information, the fifth feature information and the sixth feature information to obtain the defect detection result.

[0010] In some possible embodiments, the teacher network is a residual network structure, and the residual network structure includes three feature layers of different dimensions;

[0011] The feature extraction is performed on the input road surface image to obtain first feature information, second feature information and third feature information of different sizes, including:

[0012] The three feature layers of different dimensions of the residual network structure respectively extract features from the road surface image to be tested to obtain the first feature information, the second feature information and the third feature information respectively.

[0013] In some possible embodiments, the residual network structure includes a 64×64×64 feature layer, a 128×32×32 feature layer, and a 256×16×16 feature layer.

[0014] In some possible embodiments, the reconstructed student network includes an encoder, a decoder and a multi-scale feature fusion module; the encoder and the decoder each include feature layers of four different dimensions;

[0015] The step of characterizing and reconstructing the input road surface image to obtain fourth feature information, fifth feature information, and sixth feature information of different sizes after reconstruction includes:

[0016] The feature layer of the encoder extracts features from the road surface image to be tested to obtain feature information of different sizes; the multi-scale feature fusion module performs feature fusion on the feature information of different sizes output by the encoder to obtain fused feature information; the feature layer of the decoder reconstructs features on the fused feature information to obtain the fourth feature information, the fifth feature information and the sixth feature information.

[0017] In some possible embodiments, the encoder includes: a 64×64×64 feature layer, a 128×32×32 feature layer, a 256×16×16 feature layer, and a 512×8×8 feature layer;

[0018] The decoder includes: a 64×64×64 feature layer, a 128×32×32 feature layer, a 256×16×16 feature layer, and a 512×8×8 feature layer.

[0019] In some possible embodiments, the segmentation network includes a spatial channel reconvolution module and a spatial pyramid pool;

[0020] Among them, the dilated convolution of the spatial pyramid pool is:

[0021] k'=k+(k-1)×(r-1)

[0022] Among them, k is the size of the standard convolution kernel, r is the dilation rate of the dilated convolution, and k′ is the size of the dilated convolution kernel.

[0023] In some possible embodiments, before inputting the road surface image to be tested into the trained road surface defect detection model, the method further includes:

[0024] Acquire multiple road surface defect image samples, and defect types and defect areas corresponding to each road surface defect image sample;

[0025] Inputting each of the pavement defect image samples into an initial pavement defect detection model to obtain a predicted defect type and a predicted defect area corresponding to each of the pavement defect image samples;

[0026] The model parameters of the initial pavement defect detection model are updated according to the defect type, defect area, predicted defect type and predicted defect area corresponding to each pavement defect image sample to obtain a trained pavement defect detection model.

[0027] In some possible embodiments, the road surface defect detection method further includes: acquiring a road surface defect-free image sample;

[0028] The step of inputting each of the pavement defect image samples into an initial pavement defect detection model to obtain a predicted defect type and a predicted defect area corresponding to each of the pavement defect image samples includes:

[0029] Inputting the road surface defect-free image sample into the teacher network in the initial road surface defect detection model, inputting the multiple road surface defect image samples into the student network, the teacher network uses pre-trained weights, and only trains the student network;

[0030] When the training stop condition is met, the weights of the student network and the teacher network are frozen, the multiple road defect image samples are input into the teacher network and the student network again, and then the segmentation network is trained according to the outputs of the teacher network and the student network.

[0031] On the other hand, the present application also provides a road surface defect detection device, which includes: a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the computer program is configured to implement the steps of the road surface defect detection method as described in any one of the above items.

[0032] On the other hand, the present application also provides a storage medium having a computer program stored thereon, and when the computer program is executed by a processor, the steps of the road surface defect detection method as described in any one of the above items are implemented.

[0033] According to the pavement defect detection method provided by the present application, firstly, an image of the pavement to be tested is obtained; then, the pavement image to be tested is input into the trained pavement defect detection model to obtain the defect detection result of the pavement to be tested; the defect detection result includes the defect type and defect area of ​​no defect and existence, and the defect type includes cracking, web cracking and asphalt cracking. Among them, the pavement defect detection model includes a teacher network, a reconstructed student network and a segmentation network; the teacher network is used to extract features of the input pavement image to be tested, and obtain the first feature information, the second feature information and the third feature information of different sizes; the reconstructed student network is used to extract features and reconstruct the input pavement image to be tested, and obtain the fourth feature information, the fifth feature information and the sixth feature information of different sizes after reconstruction; the segmentation network is used to perform feature fusion and feature sampling on the first feature information, the second feature information, the third feature information, the fourth feature information, the fifth feature information and the sixth feature information, and obtain the defect detection result. It can be seen that for pavement defect detection, the present application designs an adaptive pavement defect detection model, improves the efficiency of pavement defect detection, and can be applied to the detection of various defect types. BRIEF DESCRIPTION OF THE DRAWINGS

[0034] Figure 1 It is a structural schematic diagram of a road surface defect detection device in a hardware operating environment involved in an embodiment of the present invention;

[0035] Figure 2 A schematic diagram of a process flow of a road surface defect detection method provided by an embodiment of the present invention;

[0036] Figure 3 A schematic diagram of the structure of a road surface defect detection model provided by an embodiment of the present invention;

[0037] Figure 4 A schematic diagram of a teacher network structure provided by an embodiment of the present invention;

[0038] Figure 5 A schematic diagram of a reconstructed student network structure provided by an embodiment of the present invention;

[0039] Figure 6 A schematic diagram of a spatial pyramid pool structure provided by an embodiment of the present invention;

[0040] Figure 7 A flow chart of a road surface defect detection model training method provided in an embodiment of the present invention.

[0041] The realization of the purpose, functional features and advantages of the present invention will be further explained in conjunction with embodiments and with reference to the accompanying drawings. DETAILED DESCRIPTION

[0042] It should be understood that the specific embodiments described herein are only used to explain the present invention, and are not used to limit the present invention.

[0043] Reference Figure 1 , Figure 1 It is a schematic diagram of the structure of the road surface defect detection equipment in the hardware operating environment involved in the embodiment of the present invention.

[0044] like Figure 1 As shown, the road surface defect detection device may include: a processor 1001, such as a central processing unit (CPU), a communication bus 1002, a user interface 1003, a network interface 1004, and a memory 1005. Among them, the communication bus 1002 is used to realize the connection and communication between these components. The user interface 1003 may include a display screen (Display), an input unit such as a keyboard (Keyboard), and the optional user interface 1003 may also include a standard wired interface and a wireless interface. The network interface 1004 may optionally include a standard wired interface and a wireless interface (such as a wireless fidelity (WIreless-FIdelity, WI-FI) interface). The memory 1005 may be a high-speed random access memory (Random Access Memory, RAM) memory, or a stable non-volatile memory (Non-Volatile Memory, NVM), such as a disk memory. The memory 1005 may also be a storage device independent of the aforementioned processor 1001.

[0045] Those skilled in the art will understand that Figure 1 The structure shown in the figure does not constitute a limitation on the road surface defect detection device, and may include more or less components than shown in the figure, or combine certain components, or arrange the components differently.

[0046] like Figure 1 As shown, the memory 1005 as a storage medium may include an operating system, a data storage module, a network communication module, a user interface module and a road surface defect detection program.

[0047] exist Figure 1In the pavement defect detection device shown, the network interface 1004 is mainly used for data communication with other devices; the user interface 1003 is mainly used for data interaction with the user; the processor 1001 and the memory 1005 in the pavement defect detection device of the present invention can be set in the pavement defect detection device, and the pavement defect detection device calls the pavement defect detection program stored in the memory 1005 through the processor 1001, and executes the pavement defect detection method provided by the embodiment of the present invention.

[0048] In related technologies, pavement crack detection is generally based on digital image processing technology, which combines image enhancement, image denoising and other pre-processing algorithms with image segmentation algorithms such as threshold segmentation and edge detection to improve the efficiency of crack feature extraction. However, due to the different characteristics of pavement images, the pavement crack detection method based on image processing cannot adapt to different defect types, and the detection efficiency and accuracy are not high.

[0049] In order to overcome the deficiencies in the related art, the present application provides a road surface defect detection method. Based on the idea of ​​deep learning, a suitable initial road surface defect detection model is first constructed, the initial road surface defect detection model is trained, and then the road surface image to be tested is input into the trained road surface defect detection model to obtain the defect detection result of the road surface to be tested. Compared with the image processing method in the related art, the road surface crack detection method based on deep learning provided by the present application has the advantages of strong learning ability and high robustness, and can learn high-level semantic features. At the same time, the neural network semantic segmentation method can well detect road cracks.

[0050] The present invention provides a road surface defect detection method, referring to Figure 2 , Figure 2 A schematic flow chart of a road surface defect detection method provided in an embodiment of the present invention.

[0051] In this embodiment, the road surface defect detection method includes:

[0052] Step S201: Acquire an image of a road surface to be tested.

[0053] For example, the road surface image of the road section to be tested can be collected by a roadside camera or an unmanned aerial vehicle device, and then the collected road surface image to be tested can be transmitted to the backend detection device.

[0054] Step S202: input the road surface image to be tested into the trained road surface defect detection model to obtain the defect detection result of the road surface to be tested; the defect detection result includes the defect type and defect area without defects, and the defect type includes cracks, web cracks and asphalt strip cracks.

[0055] Figure 3 For a schematic diagram of the road surface defect detection model structure provided by an embodiment of the present invention, please refer to Figure 3 As shown, the pavement defect detection model of the present application includes a teacher network 301 , a reconstruction student network 302 and a segmentation network 303 .

[0056] Among them, the teacher network 301 is used to extract features of the input road surface image to be tested, and obtain first feature information, second feature information and third feature information of different sizes; the reconstruction student network 302 is used to feature and reconstruct the input road surface image to be tested, and obtain fourth feature information, fifth feature information and sixth feature information of different sizes after reconstruction; the segmentation network 303 is used to perform feature fusion and feature sampling on the first feature information, the second feature information, the third feature information, the fourth feature information, the fifth feature information and the sixth feature information to obtain the defect detection result.

[0057] In some embodiments, Figure 4 For a diagram of the teacher network structure provided by an embodiment of the present invention, see Figure 4 As shown in Figure 1, the teacher network is a residual network structure, which includes three feature layers of different dimensions. Specifically, the residual network structure includes a 64×64×64 feature layer, a 128×32×32 feature layer, and a 256×16×16 feature layer. The three feature layers are Figure 3 T 1 、T 2 、T 3 .

[0058] The feature extraction is performed on the input road surface image to obtain first feature information, second feature information and third feature information of different sizes, which specifically includes:

[0059] The three feature layers of different dimensions of the residual network structure respectively extract features from the road image to be tested, and obtain the first feature information, the second feature information and the third feature information respectively.

[0060] In some embodiments, Figure 5 For a schematic diagram of the reconstructed student network structure provided by an embodiment of the present invention, see Figure 5 As shown, the reconstructed student network includes an encoder, a decoder and a multi-scale feature fusion module. The multi-scale feature fusion module is arranged between the encoder and the decoder. The multi-scale feature fusion module can fuse feature information of multiple different scales output by the encoder to make the collected road feature information richer.

[0061] The road surface defect detection model of this application introduces a teacher network and a student network, making the model more lightweight, reducing the number of parameters of the inference network during model training, and improving the model training speed. At the same time, a multi-scale feature fusion module is introduced before the decoder of the student network, which can better restore feature information of different scales and make the detection results more accurate.

[0062] The encoder and decoder in this embodiment both include four feature layers of different dimensions. Figure 5 As shown, the encoder includes: a 64×64×64 feature layer, a 128×32×32 feature layer, a 256×16×16 feature layer, and a 512×8×8 feature layer. The four feature layers are Figure 5 In The decoder includes: a 64×64×64 feature layer, a 128×32×32 feature layer, a 256×16×16 feature layer, and a 512×8×8 feature layer. The four feature layers are respectively Figure 5 In

[0063] The feature extraction and reconstruction of the input road surface image to be tested are performed to obtain the fourth feature information, the fifth feature information and the sixth feature information of different sizes after reconstruction, which specifically include:

[0064] The feature layer of the encoder performs feature extraction on the road surface image to be tested to obtain feature information of different sizes; the multi-scale feature fusion module performs feature fusion on the feature information of different sizes output by the encoder to obtain fused feature information; the feature layer of the decoder performs feature extraction and reconstruction on the fused feature information to obtain reconstructed fourth feature information, fifth feature information and sixth feature information.

[0065] The teacher network and the reconstructed student network of this embodiment constitute a knowledge distillation network model.

[0066] The segmentation network includes a spatial channel reconvolution module and a spatial pyramid pool. The spatial channel reconvolution module uses the channel attention mechanism to enhance the response to important features, thereby improving the performance of the model. The spatial pyramid pool extracts discriminative feature information from feature maps of different scales through multi-level dilated convolution operations to further improve the accuracy and stability of the road defect detection model.

[0067] Figure 6 For a schematic diagram of the spatial pyramid pool structure provided by an embodiment of the present invention, see Figure 6 As shown in the figure, spatial pyramid pooling can output accurate road surface detection results through multi-level hole convolution operations, upsampling and feature fusion operations.

[0068] In this embodiment, the teacher network and the student network in the knowledge distillation network model take paired road surface defect-free image samples and road surface defect image samples as input respectively, and pass the corresponding feature layer T of the teacher network. 1 , T 2 , T 3 Feature layer for decoder Provide supervision to achieve better restoration of the same features as the teacher network.

[0069] Among them, the segmentation network takes the feature information of the different scale features output by the teacher network and the student network as input. For example, the feature information of the different scale features output by the teacher network and the student network is Figure 3 x1, x2 and x3 in , and fuse them into the segmentation network.

[0070] The spatial channel reconvolution module and the spatial pyramid pool perform parallel sampling with dilated convolutions at different sampling rates. The dilated convolution of the spatial pyramid pool is:

[0071] k'=k+(k-1)×(r-1)

[0072] Among them, k is the size of the standard convolution kernel, r is the dilation rate of the dilated convolution, and k′ is the size of the dilated convolution kernel.

[0073] In this embodiment, before inputting the road surface image to be tested into the trained road surface defect detection model, it is first necessary to train the road surface defect detection model. Figure 7 For a flow chart of the road surface defect detection model training method provided by an embodiment of the present invention, please refer to Figure 7 As shown, the specific training methods include:

[0074] S701: Acquire multiple road surface defect image samples, and the defect type and defect area corresponding to each road surface defect image sample.

[0075] S702: Input each road surface defect image sample into an initial road surface defect detection model to obtain a predicted defect type and a predicted defect area corresponding to each road surface defect image sample.

[0076] S703. Update the model parameters of the initial pavement defect detection model according to the defect type, defect area, predicted defect type and predicted defect area corresponding to each pavement defect image sample to obtain a trained pavement defect detection model.

[0077] Specifically, the defect detection model training method in this embodiment can be divided into two major steps, and the knowledge distillation network model and the segmentation network model are trained in sections.

[0078] Specifically, first, a road surface defect-free image sample is input into the teacher network in the initial road surface defect detection model, and a plurality of road surface defect image samples are input into the student network. The teacher network uses pre-trained weights and only trains the student network. When the training stop condition is met, the weights of the student network and the teacher network are frozen, and a plurality of road surface defect image samples are input into the teacher network and the student network again, and then the segmentation network is trained according to the outputs of the teacher network and the student network.

[0079] Among them, the loss function of the knowledge distillation network model composed of the teacher network and the student network in this embodiment includes a cosine loss function and a structural similarity loss function.

[0080] Exemplarily, in the first stage, paired road surface defect image samples and road surface non-defect image samples are used as input, and cosine loss and structural similarity loss functions are used as the loss functions of the first stage in the training process for the knowledge distillation network model. The SGD (Stochastic Gradient Descent) optimizer is used for training 1000 times, the learning rate is 0.4, and the training batch size is 32.

[0081] Among them, the cosine loss function is:

[0082]

[0083] In the above formula, i represents the i-th feature layer. Ω neg Indicates the normal area, N neg Represents the total number of pixels in the normal area, V t i and V s i denote the feature vectors generated by the teacher and the student, respectively.

[0084] Among them, the structural similarity can be expressed as:

[0085]

[0086] In the above formula, p and q represent two feature layers, μ and σ represent mean and variance, λ1 and λ2 represent two numerical stability methods, and SSIM is in the range of [-1,1]. In particular, when p is the same as q, SSIM(p,q) is equal to 1, indicating that the two feature layers have the same structure.

[0087] Among them, the formula of the structural similarity loss function is as follows:

[0088]

[0089] Among them, f t i and fs i Represent the i-th layer feature map of the teacher and the student respectively. Finally, combining the above two losses, the training objective of the knowledge distillation network framework is formed:

[0090] L kd =L cos +L SS

[0091] Among them, L kd is the loss function of the knowledge distillation network model. The training functions of the segmentation network model of this embodiment are the focal loss and the L1 loss function. The L1 loss function is also called the mean absolute error, which is the average value of the absolute difference between the predicted value of the calculation model and the true value.

[0092] For example, in the second stage of training, the weights of the student network and the teacher network are frozen to train the segmentation network. The road surface defect image samples are used as inputs of the teacher network and the student network, and the focal loss and L1 loss are used as the loss functions in the training process for the segmentation network. The SGD optimizer is used for training 4000 times, the learning rate is 0.001, and the size of the training batch is 32.

[0093] The focal loss is:

[0094]

[0095] In the above formula, represents the predicted mask map, M represents the real mask map, γ is the focusing parameter, H1 and W1 represent the length and width of the image.

[0096] The L1 loss function is:

[0097]

[0098] in, represents the predicted mask map, M represents the real mask map, H1 and W1 represent the length and width of the image. Finally, combining the above two losses, the training objective of the segmentation network framework is formed:

[0099] L seg =L focal +L l1

[0100] In this embodiment, after the pavement defect detection model is completed, a test set is also collected to test the performance of the trained pavement defect detection model, such as testing the detection speed and detection accuracy of the trained pavement defect detection model. If the preset requirements are not met, the pavement defect detection model continues to be trained.

[0101] In one embodiment, the overall pixel accuracy and pixel-level AUC may be used to evaluate the performance of the road surface defect detection model.

[0102] The overall pixel accuracy is:

[0103]

[0104] Among them, Acc is the total pixel accuracy, which refers to the proportion of all predicted correct pixels to all pixels; TP is a true positive example, which means that the model correctly predicts positive samples as positive; TN is a true negative example, which means that the model correctly predicts negative samples as negative; FP is a false positive example, which means that negative samples are incorrectly predicted as positive; FN is a true negative example, which means that positive samples are incorrectly predicted as negative.

[0105] Pixel-level AUC: The area under the Roc curve, between 0.1 and 1. Auc as a numerical value can intuitively evaluate the quality of the classifier, and the larger the Auc value, the better.

[0106] It can be seen that the road surface defect detection model provided by this application first introduces a teacher-student network to make the model more lightweight, reduce the number of parameters of the inference network during model training, and improve the model training speed. At the same time, a multi-scale feature fusion module is introduced before the decoder of the student network, which can better restore feature information of different scales and improve the accuracy of road surface defect detection. In addition, a segmentation network is used to adaptively fuse multi-level feature similarities, which can accurately outline the abnormal shape, that is, the defect area can be accurately detected.

[0107] Furthermore, the segmentation network of the present application includes a spatial channel reconvolution module and a spatial pyramid pool (i.e., a dilated spatial convolution pooling pyramid). The spatial channel reconvolution module uses a channel attention mechanism to enhance the response to important features, thereby improving the performance of the model. The spatial pyramid pool extracts discriminative feature information from feature maps of different scales through multi-level dilated convolution operations to further improve the accuracy and stability of the model, so that the detected results are more accurate.

[0108] The present application also provides a road surface defect detection device, the device comprising: a memory, a processor, and a computer program stored in the memory and executable on the processor, the computer program being configured to implement the steps of the road surface defect detection method as described above. The road surface defect detection method comprises: obtaining a road surface image to be tested; inputting the road surface image to be tested into a trained road surface defect detection model to obtain a defect detection result of the road surface to be tested; the defect detection result comprises defect types and defect areas of no defect and existence, and the defect types comprise cracks, web cracks, and asphalt cracks; the road surface defect detection model comprises a teacher network, a reconstructed student network, and a segmentation network; the teacher network is used to extract features from the input road surface image to be tested, and obtain first feature information, second feature information, and third feature information of different sizes; the reconstructed student network is used to extract features and reconstruct the input road surface image to be tested, and obtain fourth feature information, fifth feature information, and sixth feature information of different sizes after reconstruction; the segmentation network is used to perform feature fusion and feature sampling on the first feature information, the second feature information, the third feature information, the fourth feature information, the fifth feature information, and the sixth feature information, and obtain the defect detection result.

[0109] The present application also provides a storage medium having a computer program stored thereon, and when the computer program is executed by a processor, the steps of the road surface defect detection method as described above are implemented.

[0110] It should be noted that, in this article, the terms "include", "comprises" or any other variations thereof are intended to cover non-exclusive inclusion, so that a process, method, article or system including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or system. In the absence of further restrictions, an element defined by the sentence "comprises a ..." does not exclude the existence of other identical elements in the process, method, article or system including the element.

[0111] The serial numbers of the above embodiments of the present invention are only for description and do not represent the advantages or disadvantages of the embodiments.

[0112] Through the description of the above implementation methods, those skilled in the art can clearly understand that the above-mentioned embodiment methods can be implemented by means of software plus a necessary general hardware platform, and of course by hardware, but in many cases the former is a better implementation method. Based on such an understanding, the technical solution of the present invention is essentially or the part that contributes to the prior art can be embodied in the form of a software product, which is stored in a storage medium (such as ROM / RAM, magnetic disk, optical disk) as described above, and includes a number of instructions for a terminal device (which can be a mobile phone, a computer, a server, or a network device, etc.) to execute the methods described in each embodiment of the present invention.

[0113] The above are only preferred embodiments of the present invention, and are not intended to limit the patent scope of the present invention. Any equivalent structure or equivalent process transformation made using the contents of the present invention specification and drawings, or directly or indirectly applied in other related technical fields, are also included in the patent protection scope of the present invention.

Claims

1. A road surface defect detection method, characterized in that: include: Acquire an image of the road surface to be tested; Inputting the road surface image to be tested into the trained road surface defect detection model to obtain a defect detection result of the road surface to be tested; The defect detection results include the defect types and defect areas of no defect and existence, and the defect types include cracks, web cracks and asphalt strip cracks; The road surface defect detection model includes a teacher network, a reconstruction student network and a segmentation network; The teacher network is used to extract features from the input road surface image to be tested, and obtain first feature information, second feature information, and third feature information of different sizes; the reconstructed student network is used to extract features from and reconstruct the input road surface image to be tested, and obtain fourth feature information, fifth feature information, and sixth feature information of different sizes after reconstruction; the segmentation network is used to perform feature fusion and feature sampling on the first feature information, the second feature information, the third feature information, the fourth feature information, the fifth feature information, and the sixth feature information, and obtain the defect detection result; The reconstructed student network includes an encoder, a decoder and a multi-scale feature fusion module; the encoder and the decoder each include feature layers of four different dimensions; The feature extraction and reconstruction of the input road surface image to be tested are performed to obtain the fourth feature information, the fifth feature information and the sixth feature information of different sizes after reconstruction, including: The feature layer of the encoder extracts features from the road surface image to be tested to obtain feature information of different sizes; the multi-scale feature fusion module performs feature fusion on the feature information of different sizes output by the encoder to obtain fused feature information; the feature layer of the decoder reconstructs features on the fused feature information to obtain the fourth feature information, the fifth feature information and the sixth feature information; The encoder includes: a 64×64×64 feature layer, a 128×32×32 feature layer, a 256×16×16 feature layer, and a 512×8×8 feature layer; The decoder includes: a 64×64×64 feature layer, a 128×32×32 feature layer, a 256×16×16 feature layer, and a 512×8×8 feature layer.

2. The road surface defect detection method according to claim 1, characterized in that: The teacher network is a residual network structure, and the residual network structure includes three feature layers of different dimensions; The feature extraction is performed on the input road surface image to obtain first feature information, second feature information and third feature information of different sizes, including: The three feature layers of different dimensions of the residual network structure respectively extract features from the road surface image to be tested to obtain the first feature information, the second feature information and the third feature information respectively.

3. The road surface defect detection method according to claim 2, characterized in that: The residual network structure includes a 64×64×64 feature layer, a 128×32×32 feature layer, and a 256×16×16 feature layer.

4. The road surface defect detection method according to any one of claims 1 to 3, characterized in that: The segmentation network includes a spatial channel reconvolution module and a spatial pyramid pool; Among them, the dilated convolution of the spatial pyramid pool is: k'=k+(k-1)×(r-1) Among them, k is the size of the standard convolution kernel, r is the dilation rate of the dilated convolution, and k′ is the size of the dilated convolution kernel.

5. The road surface defect detection method according to any one of claims 1 to 3, characterized in that: Before inputting the road surface image to be tested into the trained road surface defect detection model, the method further includes: Acquire multiple road surface defect image samples, and defect types and defect areas corresponding to each road surface defect image sample; Inputting each of the pavement defect image samples into an initial pavement defect detection model to obtain a predicted defect type and a predicted defect area corresponding to each of the pavement defect image samples; The model parameters of the initial pavement defect detection model are updated according to the defect type, defect area, predicted defect type and predicted defect area corresponding to each pavement defect image sample to obtain a trained pavement defect detection model.

6. The road surface defect detection method according to claim 5, characterized in that: The road surface defect detection method further includes: obtaining a road surface defect-free image sample; The step of inputting each of the pavement defect image samples into an initial pavement defect detection model to obtain a predicted defect type and a predicted defect area corresponding to each of the pavement defect image samples includes: Inputting the road surface defect-free image sample into the teacher network in the initial road surface defect detection model, inputting the multiple road surface defect image samples into the student network, and the teacher network uses pre-trained weights to train the student network; When the training stop condition is met, the weights of the student network and the teacher network are frozen, the multiple road defect image samples are input into the teacher network and the student network again, and then the segmentation network is trained according to the outputs of the teacher network and the student network.

7. A road surface defect detection device, characterized in that: The device comprises: a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the computer program is configured to implement the steps of the road surface defect detection method according to any one of claims 1 to 6.

8. A storage medium, characterized in that: The storage medium stores a computer program, which, when executed by a processor, implements the steps of the road surface defect detection method according to any one of claims 1 to 6.

Images