Road disease lightweight detection method, device and system, and storage medium

Abstract

The application discloses a road disease lightweight detection method and device, system and storage medium, and comprises the following steps: S1, training a YOLO-RDD model according to historical road disease data to obtain a road disease detection model; S2, pruning the road disease detection model; S3, taking the road disease detection model before pruning as a teacher network, taking the road disease detection model after pruning as a student network, and learning and training the road disease detection model by using a knowledge distillation algorithm to obtain an optimal road disease detection model; and S4, inputting to-be-processed road disease data into the optimal road disease detection model to perform road disease lightweight detection.

Description

Technical Field

[0001] This invention belongs to the field of image processing technology, and in particular relates to a lightweight detection method, device, system, and storage medium for road defects. Background Technology

[0002] When detecting a large number of target types, larger datasets are typically required for training, and higher-resolution images are needed to provide richer information. The trained models are often also larger, requiring significant space for deployment and incurring high caching and time costs during invocation and inference. Therefore, achieving high accuracy and high efficiency in multi-target detection is difficult to achieve simultaneously. Larger models and deployment requirements also impact the embedded hardware and software environment, affecting stability. Summary of the Invention

[0003] The technical problem to be solved by the present invention is to provide a method, device, system, and storage medium for lightweight detection of road defects.

[0004] To achieve the above objectives, the present invention adopts the following technical solution:

[0005] A method for lightweight detection of road defects, comprising:

[0006] Step S1: Train the YOLO-RDD model based on historical road defect data to obtain a road defect detection model;

[0007] Step S2: Prune the road defect detection model;

[0008] Step S3: Use the road disease detection model before pruning as the teacher network and the road disease detection model after pruning as the student network. Use the knowledge distillation algorithm to learn and train the road disease detection model to obtain the optimal road disease detection model.

[0009] Step S4: Input the road defect data to be processed into the optimal road defect detection model to perform lightweight road defect detection.

[0010] As a preferred option, in step S2, a performance-aware global channel pruning algorithm is used to prune the road defect detection model.

[0011] Preferably, in step S3, a channel knowledge distillation algorithm based on dense prediction is used to train the road defect detection model to obtain the optimal road defect detection model.

[0012] The present invention also provides a lightweight road defect detection device, comprising:

[0013] The preprocessing module is used to train a YOLO-RDD model based on historical road defect data to obtain a road defect detection model;

[0014] The pruning module is used to prune the road defect detection model;

[0015] The training module uses the road disease detection model before pruning as the teacher network and the road disease detection model after pruning as the student network. The knowledge distillation algorithm is used to train the road disease detection model to obtain the optimal road disease detection model.

[0016] The detection module is used to input the road defect data to be processed into the optimal road defect detection model for lightweight road defect detection.

[0017] As a preferred option, the pruning module uses a performance-aware global channel pruning algorithm to prune the road defect detection model.

[0018] As a preferred approach, the training module employs a channel knowledge distillation algorithm based on dense prediction to train the road defect detection model, thereby obtaining the optimal road defect detection model.

[0019] The present invention also provides a lightweight road defect detection system, comprising: a memory and a processor, wherein the memory stores a computer program executed by the processor, and the computer program performs a lightweight road defect detection method when executed by the processor.

[0020] The present invention also provides a storage medium storing a computer program, which executes a method for lightweight detection of road defects when running.

[0021] This invention trains a YOLO-RDD model based on historical road defect data to obtain a road defect detection model. The road defect detection model is then pruned. The unpruned model is used as the teacher network, and the pruned model as the student network. A knowledge distillation algorithm is employed to train the road defect detection model, resulting in an optimal model. The road defect data to be processed is then input into this optimal model for lightweight road defect detection. This invention achieves high-accuracy, high-efficiency multi-target detection. Attached Figure Description

[0022] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on the provided drawings without creative effort.

[0023] Figure 1This is a flowchart of a method for lightweight detection of road defects according to an embodiment of the present invention;

[0024] Figure 2 Flowchart for pruning road defect detection models;

[0025] Figure 3 The distillation process flow chart for the road defect detection model. Detailed Implementation

[0026] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0027] To make the above-mentioned objects, features and advantages of the present invention more apparent and understandable, the present invention will be further described in detail below with reference to the accompanying drawings and specific embodiments.

[0028] Example 1:

[0029] like Figure 1 As shown, this embodiment of the invention provides a method for lightweight detection of road defects, including:

[0030] Step S1: Train the YOLO-RDD model based on historical road defect data to obtain a road defect detection model;

[0031] Step S2: Prune the road defect detection model;

[0032] Step S3: Use the road disease detection model before pruning as the teacher network and the road disease detection model after pruning as the student network. Use the knowledge distillation algorithm to learn and train the road disease detection model to obtain the optimal road disease detection model.

[0033] Step S4: Input the road defect data to be processed into the optimal road defect detection model to perform lightweight road defect detection.

[0034] As one embodiment of the present invention, in step S2, a performance-aware global channel pruning algorithm is used to prune the road disease detection model.

[0035] Furthermore, the road defect detection model has a total of 76 layers. Layers 1-73 consist of convolutional layers, pooling layers, upsampling layers, and spatial pyramid pooling structures. Layers 74-76 form the detection head structure, consisting of classification and regression branches. This embodiment of the invention prunes the structure of layers 1-73. For example... Figure 2As shown, firstly, all target layers in the new sequence are sorted according to their contribution to the reduction of total FLOPs, calculated by subtracting the FLOPs of the pruned model from the FLOPs of the original model. In the new sequence, all target layers are sorted, with each layer corresponding to a step operation. Pruning operations from step 1 to step 73 are performed based on the 73-layer network structure. During pruning within each layer in each step, the structure after the previous step and the pruned loss state vector are introduced, and channel pruning is performed on this basis. This process is repeated until the 73-layer structure is pruned. After this round of pruning, a loss value is generated to estimate the pruning rate. If the pruning rate does not reach the set threshold, the above pruning process is repeated until the reduction in FLOPs or parameters meets the requirements. Secondly, the pruned road defect detection model is fine-tuned through retraining. Because the detection accuracy of the pruned model decreases significantly due to changes in neuron weights and channels, it cannot be directly used for target detection tasks and requires retraining to quickly restore detection accuracy.

[0036] As one embodiment of the present invention, in step S3, a channel knowledge distillation algorithm based on dense prediction is used to learn and train the road defect detection model to obtain the optimal road defect detection model.

[0037] Furthermore, knowledge distillation is a technique used for model compression and transfer learning, aiming to maintain good performance in resource-constrained environments by transferring the knowledge of a teacher network to a student network. The student network's loss function is defined during distillation, consisting of two parts: a loss based on the ground truth labels, which measures the student network's prediction error on real data; and a knowledge distillation loss, which measures the student network's prediction error on the teacher network's output.

[0038] Existing knowledge distillation methods typically employ point-by-point alignment or alignment of structured information between spatial locations. These methods all use the N channel activation values ​​at each spatial location as feature vectors for computation. To better utilize the knowledge in each channel, this invention employs a channel-based distillation method based on dense prediction, performing soft alignment of the activations of corresponding channels between the teacher and student networks. First, the channel activations are converted into probability distributions, and a probabilistic distance metric is used to measure the differences. Then, a new channel distillation paradigm guides the student network to learn knowledge from the teacher network. The knowledge distillation loss is calculated as follows:

[0039]

[0040] Where T represents the teacher network, S represents the student network, and y T and y SThe activation images represent teachers and students respectively.

[0041] The probability distribution is calculated as follows:

[0042]

[0043] Where c represents the channel index, i represents the spatial location of a certain channel, and T is a hyperparameter.

[0044] Therefore, it can be seen that when the value of T increases, the probability becomes lower, which means that a wider spatial region can be considered during the distillation process.

[0045] The formula for calculating the channel distribution difference (commonly known as KL divergence) between the teacher network and the student network is shown in Figure 5-3. When it is large, They must also become the same size; conversely, when When it is small, the KL divergence will not cause As it continues to decrease, it can be seen that this asymmetry of KL is more conducive to knowledge distillation learning for dense prediction.

[0046]

[0047] The road defect detection model has been pruned, but as the number of parameters and computational cost decreases, the accuracy always declines somewhat regardless of the pruning strategy used. Therefore, in this embodiment of the invention, the unpruned model is used as the teacher network, and the pruned model is used as the student network, employing a knowledge distillation algorithm based on dense prediction for learning and training.

[0048] The knowledge distillation process does not involve the student network learning every parameter of the teacher network. Instead, it requires selecting the appropriate network layers between the student and teacher networks where knowledge transfer will occur. It is particularly important to ensure that the number of channels between these network layers remains consistent. The road defect detection model employs a bidirectional, multi-scale feature fusion structure; therefore, the backbone network uses downsampling, while the head network uses upsampling, with feature maps decreasing in size and then increasing again.

[0049] The distillation strategy of this invention embodiment is as follows: Figure 3As shown, knowledge distillation is performed at layers 47, 57, 65, and 73. This is deliberate; the feature distribution in the head network layers has already undergone multiple convolutions and pooling operations in the backbone network layers and spatial pyramid pooling layers, possessing good feature information representation capabilities. When the student network then learns knowledge from the teacher network, it can achieve even better feature representation capabilities. Specifically: First, for the input image, the optimal model is trained based on the RDD model to obtain the teacher network, and hierarchical knowledge distillation is performed on the teacher network. Second, the pruned student network performs feature learning and training on the input image to obtain the student model. Third, the feature information from the knowledge distillation is integrated into the student network. Finally, an effective feature mapping is formed for the output result.

[0050] Example 2:

[0051] This invention also provides a lightweight road defect detection device, comprising:

[0052] The preprocessing module is used to train a YOLO-RDD model based on historical road defect data to obtain a road defect detection model;

[0053] The pruning module is used to prune the road defect detection model;

[0054] The training module uses the road disease detection model before pruning as the teacher network and the road disease detection model after pruning as the student network. The knowledge distillation algorithm is used to train the road disease detection model to obtain the optimal road disease detection model.

[0055] The detection module is used to input the road defect data to be processed into the optimal road defect detection model for lightweight road defect detection.

[0056] As one embodiment of the present invention, the pruning module uses a performance-aware global channel pruning algorithm to prune the road disease detection model.

[0057] As one embodiment of the present invention, the training module uses a channel knowledge distillation algorithm based on dense prediction to learn and train the road defect detection model to obtain the optimal road defect detection model.

[0058] Example 3:

[0059] This invention also provides a lightweight road defect detection system, comprising: a memory and a processor, wherein the memory stores a computer program executed by the processor, and the computer program executes a lightweight road defect detection method when run by the processor.

[0060] Example 4:

[0061] This invention also provides a storage medium storing a computer program, which executes a road defect lightweight detection method during runtime.

[0062] The embodiments described above are merely preferred embodiments of the present invention and are not intended to limit the scope of the present invention. Various modifications and improvements made to the technical solutions of the present invention by those skilled in the art without departing from the spirit of the present invention should fall within the protection scope defined by the claims of the present invention.

Claims

1. A method for lightweight detection of road defects, characterized in that, include: Step S1: Train the YOLO-RDD model based on historical road defect data to obtain a road defect detection model; Step S2: Prune the road defect detection model; Step S3: Use the road disease detection model before pruning as the teacher network and the road disease detection model after pruning as the student network. Use the knowledge distillation algorithm to learn and train the road disease detection model to obtain the optimal road disease detection model. Step S4: Input the road defect data to be processed into the optimal road defect detection model to perform lightweight road defect detection.

2. The method for lightweight detection of road defects as described in claim 1, characterized in that, In step S2, a performance-aware global channel pruning algorithm is used to prune the road defect detection model.

3. The method for lightweight detection of road defects as described in claim 2, characterized in that, In step S3, a channel knowledge distillation algorithm based on dense prediction is used to train the road defect detection model to obtain the optimal road defect detection model.

4. A lightweight detection device for road defects, characterized in that, include: The preprocessing module is used to train a YOLO-RDD model based on historical road defect data to obtain a road defect detection model; The pruning module is used to prune the road defect detection model; The training module uses the road disease detection model before pruning as the teacher network and the road disease detection model after pruning as the student network. The knowledge distillation algorithm is used to train the road disease detection model to obtain the optimal road disease detection model. The detection module is used to input the road defect data to be processed into the optimal road defect detection model for lightweight road defect detection.

5. The lightweight road defect detection device as described in claim 4, characterized in that, The pruning module uses a performance-aware global channel pruning algorithm to prune the road defect detection model.

6. The lightweight road defect detection device as described in claim 5, characterized in that, The training module uses a channel knowledge distillation algorithm based on dense prediction to learn and train the road defect detection model, thereby obtaining the optimal road defect detection model.

7. A lightweight road defect detection system, characterized in that, include: A memory and a processor, wherein the memory stores a computer program executed by the processor, the computer program performing the road defect lightweight detection method as described in any one of claims 1-3 when executed by the processor.

8. A storage medium, characterized in that, The storage medium stores a computer program that, when executed, performs the road defect lightweight detection method as described in any one of claims 1-3.

Images