Road surface crack detection method, system and device based on knowledge distillation

Abstract

The application provides a pavement crack detection method, system and equipment based on knowledge distillation, and the method comprises the following steps: constructing an image data set, wherein the image data set comprises an image data set of a source domain and an image data set of a target domain; based on the image data set, performing data preprocessing and data augmentation; specifically, after the image data on the target domain is subjected to adaptive block enhancement and global balance preprocessing, the image data is combined with the image data set of the source domain, and then data augmentation is performed to obtain an augmented image data set; based on the augmented image data set, a pre-trained teacher model is used to perform knowledge distillation training on a preset student model, and the student model after knowledge distillation is used as a pavement crack detection model; and the pavement crack detection model is used for crack detection on a to-be-detected pavement image. The pavement crack detection method based on knowledge distillation is adopted, so that the problem of low accuracy of the existing pavement crack detection method is solved from the root.

Description

TECHNICAL FIELD

[0001] The present application belongs to the technical field of pattern recognition, and particularly relates to a road crack detection method, system and device based on knowledge distillation. BACKGROUND

[0002] With the development of economy, the infrastructure construction in China is constantly improved, and the highway basically covers all regions, and the traffic is increasingly convenient. However, with the passage of time, the road will appear cracks and other diseases due to oxidation, long-term repeated stress, rain erosion and other factors. When the road surface appears slight cracks, if the maintenance is not timely, the cracks may cause serious traffic accidents. Therefore, road crack detection has important research significance.

[0003] The existing road crack detection methods include: a lightweight detection network based on quantization, a lightweight detection network based on network pruning, and a lightweight detection network based on network architecture technology. Specifically, (1) the lightweight road crack detection network based on quantization, the quantization process involves converting model parameters and activation functions from floating-point numbers to integer or low-bit-width data representations. This conversion can cause information loss, especially when quantizing to a lower bit width, which can affect model performance, especially in the case of a large degree of quantization. Moreover, quantization technology introduces additional complexity, including the quantization and dequantization processes of the model, and special operations during inference, which can require more effort to integrate and maintain. (2) The lightweight road crack detection network based on network pruning, the network pruning process introduces additional complexity, including the selection of channel pruning strategies, the adjustment of pruning hyperparameters, and the retraining of the model. This can require more engineering and computing resources to implement. Incorrect network pruning decisions can lead to a decrease in model performance, especially when pruning is too aggressive or inappropriate, pruning can harm the model's performance, requiring careful adjustment and verification to ensure that performance is not affected. In addition, some network pruning techniques may require special hardware support, such as accelerators that support sparse matrix multiplication. This can increase the complexity and cost of the system. (3) The lightweight road crack detection network based on network architecture technology, in order to reduce the size and computational complexity of the model, lightweight networks usually need to reduce the depth, width and number of parameters. This can lead to a decrease in performance, especially when compared to large, complex models. At the same time, lightweight networks are usually limited by available hardware. In some cases, the hardware may not support certain lightweight model architectures or operations. In summary, the existing road crack detection methods mainly have two problems: first, different hardware or software has a greater impact on the road crack detection model; second, useful information is easily lost when lightening. The above problems lead to inaccurate road crack detection results.

[0004] Therefore, how to improve the accuracy of road crack detection is a problem to be solved in the field. SUMMARY

[0005] The present application aims at the defects of the prior art, and provides a road crack detection method, system and equipment based on knowledge distillation. The road crack detection method based on knowledge distillation calculates the mask of each crack image in the expanded image data set according to the real bounding box of the object and the existing anchor box, calculates the imitation loss based on the output features of the imitation teacher model on the local feature area close to the object and the mask, and calculates the distillation loss to balance the high-level feature layer of the teacher model and the guide layer of the student model, so that the knowledge is better transferred from the teacher model to the student model, thereby fundamentally solving the problem of low precision of existing road crack detection.

[0006] To achieve the above object, the present application adopts the following technical scheme:

[0007] A road crack detection method based on knowledge distillation, characterized in that it comprises the following steps:

[0008] S1, constructing an image data set, wherein the image data set comprises a source domain image data set and a target domain image data set;

[0009] S2, based on the image data set, performing data preprocessing and data expansion; specifically, after the image data on the target domain is preprocessed through adaptive block enhancement and global equalization, the image data is combined with the source domain image data set and then expanded to obtain an expanded image data set;

[0010] S3, based on the expanded image data set, using a pre-trained teacher model to perform knowledge distillation training on a preset student model, and taking the knowledge distilled student model as a road crack detection model; comprising the following steps:

[0011] S31, calculating the mask I of each crack image in the expanded image data set according to the real bounding box of the object and the existing anchor box;

[0012] S32, calculating the imitation loss based on the output features of the imitation teacher model on the local feature area close to the object and the mask; the specific calculation formula of the imitation loss is as follows:

[0013]

[0014] wherein s is defined as the feature map of the guide feature layer of the student model, t is defined as the corresponding teacher model high-level feature response map, for each anchor box position (i, j) on the feature map with width W and height H close to the object, I ij represents the mask value on the anchor box position (i, j), C represents the number of channels of the feature map, and f(s)ijc represents the feature response of the c-th channel at the position (i, j) on the feature map;

[0015] S33, calculate a distillation loss to balance the high-level feature layer of the teacher model and the guided layer of the student model, and migrate the knowledge of the teacher model; the distillation loss is calculated as follows:

[0016]

[0017] wherein the variable p represents the weight proportion of the distillation loss, and the variables h box , h cls and h obj represent their weights in L kd ; the variable l obj represents whether the batch of images contains objects, and if there are objects, l obj = 1, otherwise l obj = 0; are the bounding box loss, the classification loss, the confidence loss and the domain loss in the previous step, respectively;

[0018] S34, calculate the total loss of the student model based on the imitation loss and the distillation loss, and in combination with the loss of the teacher model;

[0019] L stu = L + L kd + L imit ;

[0020] wherein L is the total loss of the teacher model;

[0021] S35, based on the total loss of the student model, perform knowledge distillation training on the pre-set student model, and take the student model after knowledge distillation as a road crack detection model;

[0022] S4, use the road crack detection model to detect cracks in the road image to be detected.

[0023] Specifically, in step S2, the image data on the target domain is preprocessed by adaptive block enhancement and global equalization, and the specific steps are as follows:

[0024] S21, block the image according to the resolution size, and the block size increases from 50x50 to 500x500, with an increment step of 50 each time;

[0025] S22, perform Gamma transformation on each block to correct dark parts, wherein the Gamma value is set from 0.5 to 2, with an increment step of 0.1;

[0026] S23, when the block size is fixed, calculate the variance sigma for all blocks, set the Gamma value when the block after enhancement gets the maximum sigma as the final local enhancement parameter of the region;

[0027] S24, the pixel points between adjacent blocks are processed by smoothing;

[0028] S25, the CLAHE algorithm is used for global enhancement to balance the overall image.

[0029] Specifically, in step S3, YOLO v5s is taken as the student model weight to be trained.

[0030] Specifically, in step S31, the specific steps include:

[0031] S311, for each real boundary box of the crack image, calculate the IOU value between it and all anchor boxes, obtain an IOU graph of WxHxK, denoted as m IOU , wherein K represents the number of preset anchor boxes of YOLO v5, W represents the width of the image, and H represents the height of the image;

[0032] S312, find the IOU graph corresponding to the maximum IOU value in m IOU , denoted as M=max(m IOU ), and construct a filter F therefrom, , wherein represents a threshold factor;

[0033] S313, filter m IOU using the filter F, retain the IOU graph greater than F, and combine them using the or operator to obtain a mask of WxH;

[0034] S314, through this operation, loop all real boxes of the crack image, and finally obtain a fine-grained mask of the imitated teacher model, denoted as I.

[0035] The application also provides a road crack detection system based on knowledge distillation, characterized in that the road crack detection system executes the road crack detection method based on knowledge distillation, and includes an image data set construction module, a data preprocessing and expansion module, a model training module and an image crack detection module.

[0036] The image data set construction module includes a source domain image data set and a target domain image data set.

[0037] The data preprocessing and expansion module combines the image data on the target domain with the image data set of the source domain after adaptive block enhancement and global balance preprocessing, and then performs data expansion to obtain an expanded image data set.

[0038] a model training module, based on the expanded image dataset, performing knowledge distillation training on a preset student model by using a pre-trained teacher model, and taking the student model after knowledge distillation as a road crack detection model;

[0039] an image crack detection module, which adopts the road crack detection model to perform crack detection on a road image to be detected.

[0040] The application further provides a computer device, which comprises a memory and a processor, the memory stores a computer program, and the processor implements the method described above when executing the computer program.

[0041] Compared with the prior art, the application has the following beneficial effects:

[0042] The road crack detection method based on knowledge distillation has the advantages that according to the real boundary frame of an object and the existing anchor frame, the mask of each crack image in the expanded image dataset is calculated, the mimic loss is calculated based on the mask and the output features of the mimic teacher model on the local feature area close to the object, and the distillation loss is calculated to balance the high-level feature layer of the teacher model and the guide layer of the student model, so that the knowledge is better transferred from the teacher model to the student model, that is, the detection knowledge of the large-scale excellent teacher model is transferred to the small network model by using knowledge distillation, the precise detection of the small model is realized, and the accuracy of road crack detection is improved. BRIEF DESCRIPTION OF DRAWINGS

[0043] In order to more clearly illustrate the technical solutions of the embodiments of the application, the drawings needed in the embodiment description will be briefly introduced. Obviously, the drawings in the following description are some embodiments of the application, and other drawings can be obtained by those skilled in the art without creative labor.

[0044] Figure 1 The road crack detection method based on knowledge distillation provided by the embodiments of the application is shown in the figure.

[0045] Figure 2 The network architecture diagram of the road crack detection method based on knowledge distillation provided by the embodiments of the application is shown in the figure.

[0046] Figure 3 The local feature area schematic diagram and the process diagram of the teacher model guiding the student model provided by the embodiments of the application are shown in the figure.

[0047] Figure 4 The method schematic diagram of the local feature estimation area module provided by the embodiments of the application is shown in the figure.

[0048] Figure 5A schematic diagram of a road crack detection system based on knowledge distillation is provided for an embodiment of the present application. DETAILED DESCRIPTION

[0049] The technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the drawings in the embodiments of the present application. Obviously, the described embodiments are some of the embodiments of the present application, but not all the embodiments of the present application. Based on the embodiments in the present application, all other embodiments obtained by those of ordinary skill in the art without creative efforts fall within the scope of the present application.

[0050] It should be understood that, when used in the specification and the appended claims, the terms “comprise” and “include” indicate the presence of the described features, integers, steps, operations, elements, and / or components, but do not exclude one or more other features, integers, steps, operations, elements, components, and / or groups thereof.

[0051] The present application will be further described below with reference to the drawings and specific embodiments, but is not limited by the embodiments.

[0052] It should also be understood that the terms used in the specification of the present application are only for the purpose of describing specific embodiments and are not intended to limit the present application. As used in the specification and the appended claims of the present application, the singular forms “a”, “an” and “the” are intended to include the plural forms unless the context clearly indicates otherwise.

[0053] It should be further understood that the term “and / or” used in the specification and the appended claims of the present application refers to any combination of one or more of the associated listed items and all possible combinations, and includes these combinations.

[0054] The present application proposes a road crack detection method based on knowledge distillation. As shown in Figure 1 The road crack detection method based on knowledge distillation includes the following steps S1 to S4. At the same time, the network architecture of the present application is as shown in Figure 2 .

[0055] S1, an image data set is constructed, the image data set including an image data set of a source domain and an image data set of a target domain.

[0056] The CQU-BPDD dataset contains 60,059 asphalt pavement images, a total of eight categories, including seven types of pavement cracks, namely transverse cracks, large cracks, cracks, crack pouring, longitudinal cracks, potholes, and repairs. These images are mainly automatically taken by professional vehicle-mounted high-definition cameras on high-speed highways in southern China, taken at different times and different locations, and then annotated by professional personnel. The CQU-BPDD training set has 49,919 images, and the test set has 10,140 images. Each pavement image corresponds to a 2x3 meter highway pavement patch, with a resolution of 1200x900, and only image-level labels.

[0057] RDD2020 is a crack image dataset with position annotation for automatic detection of road cracks, including 26,620 pavement images collected from Japan, India, and the Czech Republic, including a training set of 21,041, with four categories of pavement disease, namely transverse cracks, longitudinal cracks, potholes, and cracks. The resolution of each image in the training set is 600x600, and the XML crack position information annotation file in PASCALVOC format is included.

[0058] CFD was originally composed of 118 images with a resolution of 480x320. Since the CFD dataset is for the so-called road crack detection task, i.e., the road crack segmentation task from the perspective of computer vision, all samples are actually disease images.

[0059] CQU-BPMDD is a large-scale asphalt pavement multi-label disease dataset published by the present application, which can be used for detection and classification tasks. It contains 9,851 crack images and 29,143 normal images from different highways in southwestern China, with a resolution of 3,692x2,147 for each image. The crack images have seven categories, namely transverse cracks, longitudinal cracks, large cracks, repairs, loose, wavy congestion, and potholes. Currently, only three categories have a large amount of data, namely longitudinal cracks, transverse cracks, and repairs. Unlike CQU-BPDD, CQU-BPMDD is a crack disease dataset where all images are imaged using the exposure compensation method, with most being small crack images, which are more consistent with actual scenarios and difficult to detect.

[0060] CQU-BPDD and RDD2020 have more training data and similar categories, so these two datasets are selected as the target domain and source domain in domain adaptation training, respectively. The CFD dataset contains less data and does not include normal images, so the present application uses image restoration methods to generate normal crack images and serves as a validation set for cross-data generalization capability. In addition, cross-dataset verification is also performed on the CQU-BPMDD dataset, which can better reflect the cross-dataset generalization capability of the designed model.

[0061] S2, based on the image data set, data preprocessing and data augmentation are performed; specifically including: after the image data on the target domain is preprocessed by adaptive patch enhancement and global equalization, the image data is combined with the image data of the source domain and then data augmentation is performed to obtain an augmented image data set.

[0062] Since the road surface images of CQU-BPDD are captured in different road conditions and at different times, the cracks in the road surface images are usually affected by light and exhibit serious uneven gray scale distribution. In order to enhance the road surface images, a simple and effective preprocessing method, namely adaptive patch enhancement and global equalization, is proposed. Specifically, first, in order to improve the local contrast, the image is divided into patches for enhancement.

[0063] In step S2, the image data on the target domain is preprocessed by adaptive patch enhancement and global equalization, and the specific steps are as follows:

[0064] S21, the image is divided into patches according to the resolution size, and the patch size increases from 50x50 to 500x500 with an increment of 50 each time;

[0065] S22, Gamma transformation is performed on each patch to correct the dark part, wherein the Gamma value is set from 0.5 to 2 with an increment of 0.1;

[0066] S23, when the patch size is fixed, the variance σ of all patches is calculated, and the Gamma value at which the patch is enhanced to obtain the maximum σ is set as the final local enhancement parameter of the region; wherein the contrast of the image is improved and the crack is easy to distinguish when σ is larger;

[0067] S24, the pixels between adjacent blocks are smoothed;

[0068] When all patches are converted, the gray value conversion between adjacent patches may be discontinuous. At this time, the pixels between adjacent blocks are smoothed;

[0069] S25, CLAHE algorithm is used for global enhancement to balance the overall image.

[0070] In order to further suppress the negative effects of discontinuous gray value conversion, CLAHE algorithm is used to balance the overall image.

[0071] After the target domain crack image is processed by steps S21-S25, an adaptively enhanced image is obtained. Then, the adaptively enhanced image is combined with the image data of the source domain and then data augmentation is performed to obtain an augmented image data set.

[0072] After the data on the target domain is preprocessed by the proposed adaptive patch enhancement and global equalization, the data is augmented with the data of the source domain by common data augmentation methods and input into the designed student model for training. The augmentation methods mainly use Gaussian processing, image sharpening, flipping, translation, channel scaling, and image contrast transformation. Among them, the target domain CQU-BPDD is augmented by six methods, and the source domain RDD2020 is randomly augmented by one method.

[0073] S3, based on the augmented image data set, using a pre-trained teacher model to perform knowledge distillation training on a pre-set student model, and taking the knowledge distilled student model as a pavement crack detection model;

[0074] First, when distilling, a lightweight model structure is usually selected as a student model. The target detection network usually has different sizes of weights that can be selected for training. In order to obtain good detection results, a weight with a large number of parameters and a complex model structure is often selected for training. YOLO v5 is selected as the baseline, and the main weight types of YOLO v5 include the following categories: YOLO v5s, YOLO v5m, YOLO v5n, YOLO v5l, and YOLO v5x. Among them, the parameter number of YOLO v5x weight is the largest, and the FLOPs is also the largest. Compared with YOLO v5n and YOLO v5s, the parameter number and FLOPs are more than ten times larger. The larger the number of network parameters, the higher the detection accuracy, and small parameter networks are more likely to fall into local optima than large parameter networks. Therefore, in order to realize a lightweight crack detection network, the five weights need to be studied respectively, that is, to be verified on the CQU-BPDD dataset. The results prove that YOLO v5n has a large gap in detection effect due to the small number of parameters. The reason mainly lies in that in the case of small cracks, not only low-level spatial features are needed, but also abstract and deep semantic information. On the YOLO v5n with a small number of parameters and shallow layers, it is difficult to learn the semantic information of small cracks, which easily leads to false detection. Considering the parameter size and actual detection results of different weights, YOLO v5s is selected as the student model weight to be trained, and it is used together with the pre-trained YOLO v5x of the deep domain adaptive crack detection network DDACDN for lightweight model research. In subsequent work, the detection knowledge of the pre-trained YOLO v5x weight model is transferred to the model based on YOLO v5s by using the knowledge distillation method, realizing accurate detection of small models.

[0075] That is, in step S3, YOLO v5s is taken as the student model weight to be trained, and pre-trained YOLOv5x is taken as the teacher model.

[0076] The specific steps of step S3 are as follows:

[0077] S31, calculating the mask I of each crack image in the expanded image data set according to the real boundary box of the object and the existing anchor box;

[0078] Generally speaking, the local position feature information near the object to be detected is very important in the detection process, and this area contains rich knowledge. Because in the distillation process, this area usually reveals how a trained complex teacher model separates the ordinary image background from the object to be detected. Therefore, the method proposed in this part is based on these areas rich in detection knowledge. In order to formalize the definition of this area for calculation, the present application uses the real boundary box and the existing anchor box to design the mask of this area, which participates in the subsequent imitation loss calculation, so that the student model imitates the teacher model's response to their high-level features. As shown in Figure 3 、 Figure 4 As shown, the local feature area of each object is formed near the anchor box position. Among them, the red and green boundary boxes on the left image are anchor boxes in the corresponding position. The red anchor box has the largest overlap with the ground real boundary box, and the green anchor box is closer to the object. Figure 3 The present application reveals how a learned teacher model generalizes in order to guide the learning of the student model. It reflects that the high-level feature response of the pre-trained teacher model contains the positioning information of the area near the object anchor box, so the present application performs fine-grained feature imitation around this area. Since the anchor box mechanism of the YOLO v5 detection model will be recalculated and fitted before the first training, the anchor box has a large overlap with the object real annotation box, and this feature can well formalize the local feature area.

[0079] For the student model, in order to study the local feature area of the object, the present application calculates the mask for each crack image.

[0080] Step S31, the specific steps of calculating the mask I of each crack image include:

[0081] S311, for each real boundary box of the crack image, calculate the IOU value between it and all anchor boxes to obtain an IOU graph of WxHxK, denoted as m IOU , where K represents the number of anchor boxes preset by YOLO v5, W represents the width of the image, and H represents the height of the image;

[0082] S312, finding the IOU graph corresponding to the maximum IOU value in m IOU , denoted as M=max(mIOU ), and a filter F is constructed therefrom, wherein represents a threshold factor;

[0083] S313, using the filter F to filter m IOU , retaining the IOU map greater than F, and combining them using an or operator to obtain a WxH mask;

[0084] S314, with this operation, all crack image real boxes are cycled to finally obtain a fine-grained mask of the imitated teacher model, denoted as I.

[0085] S32, calculating the imitation loss based on the mask and the output features of the imitated teacher model on the local feature region close to the object;

[0086] The final mask of the crack image obtained by step S31 represents the local feature region close to the object, and the imitation loss needs to be calculated. The present application directly calculates the feature response of the teacher model and the student model on the last hidden layer of the student model. The following two points are considered: 1. The architecture of the student model and the teacher model designed by the present application belongs to the simplified structure type, and shares a model framework YOLO v5. In training, the number of channels of the teacher model and the student model is compatible, and the feature imitation loss can be directly calculated. 2. The teacher model is a pre-trained model with excellent performance. The simulation of the teacher model on the high-level feature response of the local region close to the object anchor box position on the student model with a smaller network structure can well perform knowledge distillation, so that the student model fully learns the detection knowledge of the teacher model for the local region close to the object.

[0087] In calculating the imitation loss, the high-level feature map of the student model and the teacher model is needed. Specifically, s is defined as the feature map of the guided feature layer of the student model, and t is defined as the corresponding teacher model high-level feature response map. For each anchor box position (i,j) close to the object on the feature map with width W and height H, the student model is trained to minimize the following loss in this section.

[0088]

[0089] wherein f(*) represents the adaptive function of the last hidden layer of the student model. The present application also needs to imitate the local feature region, calculate the loss therefrom, and update the network in reverse. Therefore, in order to further learn the knowledge of the teacher detection model teacher model, the local region mask obtained in the previous step is used for further distillation, and the imitation loss is calculated as follows:

[0090]

[0091] where s is defined as the feature map of the guided feature layer of the student model, t is defined as the corresponding teacher model high-level feature response map; for each anchor box position (i, j) close to the object on the feature map with width W and height H, I ij represents the mask value on the anchor box position (i, j), C represents the number of channels of the feature map, f(s) ijc represents the feature response of the cth channel at position (i, j) on the feature map.

[0092] Here f(s) is the feature map processed from the last hidden layer of the student model through the adaptation function f, f(s) ijc represents the feature response of the cth channel at position (i, j) on the feature map. By minimizing this formula, the student model is guided to mimic the teacher model to respond to the feature of the local area close to the object, and the student model is guided to approach the teacher model more and more, so as to achieve the purpose of guiding the student model.

[0093] S33, calculate the distillation loss to balance the teacher model high-level feature layer and the guided layer of the student model, and transfer the knowledge of the teacher model;

[0094] In the training process of the student model, in order to balance the teacher model high-level feature layer and the guided layer of the student model, the present application further designs a distillation loss. The soft label predicted by the teacher model is calculated with the prediction of the student model. Specifically, in the distillation process, a variable p is set, which represents the weight proportion of the distillation loss. Usually, the softmax layer is used to output the probability of each category in the classification task. However, the YOLO v5 model is used in the L-DDACDN, and the binary cross entropy is used to calculate each label when calculating the classification loss, which reduces the complexity of loss calculation. The distillation loss calculation method is as follows:

[0095]

[0096] where variable p represents the weight proportion of the distillation loss, p represents the weight proportion of the distillation loss, and η box , η cls and η obj represent their weights in L kd ; λ obj represents whether the batch of images contains objects, if there are objects, λ obj = 1, otherwise λ obj = 0; are the bounding box loss, classification loss, confidence loss and domain loss in the previous step, respectively; the difference is that the input of the four types of loss functions is adjusted from the true value and the predicted value to the predicted value of the student model and the teacher model.

[0097] In order to improve the problem of unbalanced number of positive and negative samples, Continue to use Focalloss to calculate. It is not difficult to see that the purpose of distillation loss is to minimize the difference between student model and teacher model in classification, confidence, domain adaptation and boundary box regression, so as to better transfer knowledge from teacher model to student model.

[0098] S34, based on the imitation loss and the distillation loss, and combined with the loss of the teacher model, the total loss of the student model is calculated;

[0099] L stu =L+L kd +L imit ;

[0100] Wherein, L is the total loss of the teacher model;

[0101] S35, based on the total loss of the student model, the knowledge distillation training is carried out on the preset student model, and the student model after knowledge distillation is used as a road crack detection model.

[0102] S4, the road crack detection model is used for crack detection of the road image to be detected.

[0103] As a method of learning end-to-end compact model, knowledge distillation neither needs hardware nor custom software to obtain acceleration, nor can it train small network model by distilling knowledge of large-scale excellent model. Therefore, the present application proposes a lightweight network architecture based on knowledge distillation.

[0104] In the present application, firstly, the problem that large deep learning network may not be deployed and inferred under the condition of limited storage space and computing power is analyzed, and a lightweight network architecture L-DDACDN based on knowledge distillation is proposed. The weight selection of the lightweight crack detection model is analyzed by experiment, and YOLO v5s with small number of parameters is selected as the target of knowledge distillation. Secondly, in order to imitate the prediction of the teacher model in the local position close to the object anchor frame, i.e. to learn the high-level feature response of the region, a mask of the near-object region is designed by using the real boundary frame, and the imitation loss is calculated combined with the feature response of the teacher and student models, so as to guide the prediction of the student model. Finally, the distillation loss is designed, in order to balance the high-level feature layer of the teacher model and the guide layer of the student model, so as to guide the student model by the teacher model.

[0105] In the present application, the mask is designed by using the real boundary frame label of the crack image and the existing anchor frame, and the imitation loss is calculated by using the high-level features of the teacher and student networks. In order to be compatible with the guide feature layer of the student network and the teacher network, the corresponding distillation loss is designed, and the prediction of the student model is calculated by using the soft label of the teacher model, so as to better transfer knowledge from the teacher model to the student model.

[0106] Figure 5 The application provides a road crack detection system based on knowledge distillation. Figure 5 The road crack detection system based on knowledge distillation comprises an image data set construction module, a data preprocessing and expansion module, a model training module and an image crack detection module.

[0107] The image data set construction module comprises a source domain image data set and a target domain image data set.

[0108] The data preprocessing and expansion module combines the image data on the target domain with the image data set of the source domain after adaptive block enhancement and global balance preprocessing, and then performs data expansion to obtain an expanded image data set.

[0109] The model training module performs knowledge distillation training on a preset student model based on the expanded image data set by using a pre-trained teacher model, and takes the student model after knowledge distillation as a road crack detection model.

[0110] The image crack detection module adopts the road crack detection model to perform crack detection on a road image to be detected.

[0111] The road crack detection system based on knowledge distillation can be realized in the form of a computer program, which can run on a computer device.

[0112] The computer device can be a server, which can be a stand-alone server or a server cluster composed of multiple servers.

[0113] The computer device comprises a processor, a memory and a network interface connected through a system bus, wherein the memory can comprise a non-volatile storage medium and an internal memory.

[0114] The non-volatile storage medium can store an operating system and a computer program. The computer program comprises program instructions, which, when executed, can enable the processor to perform a road crack detection method based on knowledge distillation.

[0115] The processor is used to provide computing and control capabilities to support the operation of the entire computer device.

[0116] The internal memory provides an environment for the running of the computer program in the non-volatile storage medium, and the computer program, when executed by the processor, can enable the processor to perform a road crack detection method based on knowledge distillation.

[0117] The network interface is used for network communication with other devices. Those skilled in the art can understand that the above computer device structure is only part of the structure related to the scheme of the present application, and does not constitute a limitation on the computer device to which the scheme of the present application is applied. A specific computer device can include more or fewer components than those shown in the figure, or combine certain components, or have a different arrangement of components.

[0118] The processor is configured to execute a computer program stored in the memory, and the computer program implements the knowledge distillation-based road crack detection method in Embodiment 1.

[0119] It should be understood that, in the embodiments of the present application, the processor can be a central processing unit (CPU), and the processor can also be other general-purpose processors, digital signal processors (DSPs), application specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs) or other programmable logic devices, discrete gates or transistor logic devices, discrete hardware components, etc. The general-purpose processor can be a microprocessor or the processor can also be any conventional processor.

[0120] It can be understood by those skilled in the art that all or part of the processes in the above-mentioned embodiments can be completed by a computer program instructing related hardware. The computer program includes program instructions, and the computer program can be stored in a storage medium, which is a computer-readable storage medium. The program instructions are executed by at least one processor in the computer system to implement the process steps of the above-mentioned embodiments.

[0121] The present application also provides a storage medium. The storage medium can be a computer-readable storage medium. The storage medium stores a computer program, wherein the computer program is executed by a processor to make the processor execute the knowledge distillation-based road crack detection method in Embodiment 1.

[0122] The storage medium can be a U disk, a mobile hard disk, a read-only memory (ROM), a magnetic disk or an optical disk, and various computer-readable storage media that can store program codes.

[0123] Those skilled in the art can understand that the units and algorithm steps of each example described in combination with the embodiments disclosed herein can be realized in electronic hardware, computer software or a combination of both. In order to clearly illustrate the interchangeability of hardware and software, the components and steps of each example have been described in the above description in a general manner. Whether the functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of the present application.

[0124] In several embodiments provided by the present application, it should be understood that the disclosed devices and methods can be implemented in other ways. For example, the device embodiments described above are merely illustrative. For example, the division of each unit is only a logical function division, and actual implementation can have another division manner. For example, multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed.

[0125] The steps in the method embodiments of the present application can be sequentially adjusted, combined and deleted according to actual needs. The units in the device embodiments of the present application can be combined, divided and deleted according to actual needs. In addition, each functional unit in each embodiment of the present application can be integrated in one processing unit, or each unit can exist physically, or two or more units can be integrated in one unit.

[0126] The integrated unit, if realized in the form of a software functional unit and sold or used as an independent product, can be stored in a storage medium. Based on such understanding, the technical solutions of the present application essentially or say the parts that make contributions to the prior art, or the whole or part of the technical solutions can be embodied in the form of a software product. The computer software product is stored in a storage medium and includes a plurality of instructions for causing a computer device (which can be a personal computer, a terminal or a network device, etc.) to execute all or part of the steps of the methods described in each embodiment of the present application.

[0127] Note that the above is only the preferred embodiment of the present application and the technical principle applied. Those skilled in the art will understand that the present application is not limited to the specific embodiments described herein, and those skilled in the art can make various obvious changes, readjustments and substitutions without departing from the scope of the present application. Therefore, although the present application has been described in more detail through the above embodiments, the present application is not limited to the above embodiments, and can include more other equivalent embodiments without departing from the concept of the present application, and the scope of the present application is determined by the scope of the appended claims.

Claims

1. A pavement crack detection method based on knowledge distillation, characterized in that: Including steps: S1. Construct an image dataset, where the image dataset includes an image dataset of a source domain and an image dataset of a target domain; S2. performing data preprocessing and data expansion based on the image dataset; Specifically, the method includes: performing adaptive block enhancement and global equalization preprocessing on the image data in the target domain, merging the image data in the source domain with the image data set, and then performing data expansion to obtain an expanded image data set; S3, based on the expanded image dataset, using the pre-trained teacher model to perform knowledge distillation training on the preset student model, and using the student model after knowledge distillation as the pavement crack detection model; comprising the following steps: S31, calculating a mask I of each crack image in the expanded image dataset according to the real bounding box of the object and the existing anchor box; S32, calculating the imitation loss based on the mask and the output features of the imitation teacher model in the local feature area close to the object; the specific calculation formula of the imitation loss is as follows: Where s is defined as the feature map of the guided feature layer of the student model, and t is defined as the corresponding high-level feature response map of the teacher model; for each anchor box position (i, j) close to the object on the feature map of width W and height H, I ij Represents the mask value at the anchor box position (i, j), C represents the number of channels of the feature map, f(s) ijc Represents the feature response of the cth channel at position (i, j) on the feature map; S33. Calculate the distillation loss to balance the high-level feature layer of the teacher model and the guidance layer of the student model, and transfer the knowledge of the teacher model. The distillation loss is calculated as follows: Among them, the variable ρ represents the weight ratio of distillation loss, η box ,η cls and η obj Represents them in L kd The weight in ;λ obj Indicates whether this batch of images contains objects. If so, it is λ obj =1, otherwise λ obj =0; They are the bounding box loss, classification loss, confidence loss, and domain loss in the previous step; S34. Calculate the total loss of the student model based on the imitation loss and the distillation loss, and in combination with the loss of the teacher model; L stu =L+L kd +L imit ; Where L is the total loss of the teacher model; S35. Based on the total loss of the student model, perform knowledge distillation training on the preset student model, and use the student model after knowledge distillation as a pavement crack detection model; S4. Using the pavement crack detection model, perform crack detection on the pavement image to be detected.

2. The method according to claim 1, characterized in that In step S2, the image data in the target domain is subjected to adaptive block enhancement and global equalization preprocessing. The specific steps are as follows: S21, dividing the image into blocks according to the resolution size, and increasing the block size from 50×50 to 500×500, with a step size of 50 each time; S22, performing a gamma transformation on each block to correct the gray part, wherein the gamma value is set from 0.5 to 2, and the increment step is 0.1; S23, when the block size is fixed, calculate the variance σ for all blocks, and set the Gamma value when the maximum σ is obtained after the block is enhanced as the final local enhancement parameter of the region; S24, performing smoothing processing on the pixel points between adjacent blocks; S25. Use the CLAHE algorithm for global enhancement to balance the overall image.

3. The method according to claim 1, characterized in that In step S3, YOLO v5s is used as the student model weight to be trained.

4. The method according to claim 1, wherein Step S31, specifically the steps include: S311. For each true bounding box of the crack image, calculate the IOU value between it and all anchor boxes to obtain a W×H×K IOU map, denoted as m IOU , where K represents the number of anchor boxes preset by YOLO v5, W represents the width of the image, and H represents the height of the image; S312, in m IOU Find the IOU graph corresponding to the maximum IOU value, recorded as M=max(m IOU ), and use this to construct a filter F, in represents a threshold factor; S313, use filter F to filter m IOU , retain the IOU graphs larger than F and combine them using the or operator to obtain a W×H mask; S314. In this way, all the real frames of the crack images are looped, and finally a fine-grained mask imitating the teacher model is obtained, which is recorded as I.

5. A pavement crack detection system based on knowledge distillation, characterized in that: The pavement crack detection system implements the pavement crack detection method based on knowledge distillation as claimed in claim 1, comprising: an image data set construction module, a data preprocessing and expansion module, a model training module, and an image crack detection module; An image dataset construction module, wherein the image dataset includes an image dataset of a source domain and an image dataset of a target domain; A data preprocessing and expansion module performs data expansion on the target domain image data after adaptive block enhancement and global equalization preprocessing, and then merges the image data set in the source domain to obtain an expanded image data set; A model training module, based on the expanded image dataset, uses a pre-trained teacher model to perform knowledge distillation training on a preset student model, and uses the student model after knowledge distillation as a pavement crack detection model; The image crack detection module uses the pavement crack detection model to perform crack detection on the pavement image to be detected.

6. A computer device, characterized in that: The device includes a memory and a processor, wherein a computer program is stored in the memory, and the processor implements the method according to any one of claims 1 to 4 when executing the computer program.

Images