Real-time rail crack detection method and system based on knowledge distillation

Abstract

The invention discloses a rail crack real-time detection method and system based on knowledge distillation. The method specifically comprises the following steps: acquiring a rail crack image data set for knowledge distillation training; training a teacher model based on the image set, and obtaining network parameters of the teacher model part; network parameters of the teacher model part are substituted into a distillation network, a plurality of temperature parameters are set, and knowledge distillation training is carried out for many times to obtain student models; the trained student model is extracted from the distillation network, detection precision comparison is carried out, and a model which most meets actual requirements is selected; and deploying the student model in an embedded device, and carrying out real-time detection. Aiming at the problems of low precision and large manpower resource consumption of traditional rail crack detection, the method effectively improves the detection precision while keeping the advantage of light weight of a model through a knowledge distillation algorithm, and supports real-time rail crack detection on embedded equipment (such as Raspberry Pi and the like).

Description

Technical Field

[0001] This invention relates to the fields of computer vision and railway material quality monitoring, and in particular to a real-time detection method and system for rail cracks based on knowledge distillation. Background Technology

[0002] With the continuous development of artificial intelligence technology, computer vision technology has been widely applied in many fields. The YOLOv8 model is a rapidly developing object detection algorithm in recent years, widely used due to its real-time performance and high accuracy. However, it still has the following shortcomings: 1. Insufficient accuracy in detecting small objects in complex scenes; 2. Difficulty in balancing high accuracy and real-time requirements in inference speed. Therefore, an improved method based on the YOLOv8 architecture is urgently needed to solve this problem.

[0003] With the rapid development of the railway transportation industry, rail safety inspection technology plays a crucial role in ensuring railway transportation safety. Traditional rail crack detection methods mainly rely on manual inspection, which is not only inefficient and labor-intensive but also prone to missed or false detections due to human factors. In recent years, with the continuous advancement of computer vision technology, image recognition-based rail crack detection methods have gradually gained attention. However, current detection technologies still have the following shortcomings: 1. In complex railway environments (such as changes in lighting, rail corrosion, and stains), the accuracy of crack identification is low, and misjudgments are prone to occur; 2. Existing detection systems struggle to meet the real-time requirements of railway operation while maintaining high accuracy, resulting in insufficient timeliness of detection results. Therefore, a real-time rail crack detection technology is urgently needed to efficiently and accurately identify rail cracks, thereby effectively improving the safety and reliability of railway transportation. Summary of the Invention

[0004] This invention addresses the problems of low detection accuracy and inability to meet real-time requirements for rail crack detection. It implements a knowledge distillation algorithm on the YOLOv8 network, which effectively improves detection accuracy while maintaining the model's lightweight advantage, thus supporting real-time rail crack detection on embedded devices (such as Raspberry Pi).

[0005] To achieve the above objectives, the technical solution of the present invention is: a real-time detection method for rail cracks based on knowledge distillation, comprising: S1, Obtain the rail crack dataset for knowledge distillation training; S2, based on the railway crack dataset, train the teacher model and obtain some network parameters of the teacher model; S3 involves incorporating some network parameters from the teacher model into the distillation network, setting multiple temperature parameters, and performing knowledge distillation training multiple times. S4. Extract the trained model from the distillation network, compare the detection accuracy, and select the model that best meets the actual needs as the student model. S5 deploys student models onto embedded devices for real-time detection.

[0006] Preferably, step S4 specifically includes: designing an improved knowledge distillation loss function based on the characteristics of the YOLOv8 loss function, as follows: in It is the loss of the classification part. It is the loss of the predicted box regression. This is the loss of the DFL portion, in addition , , These are the preset hyperparameters.

[0007] Preferably, the loss calculation formula for the classification portion is as follows: set up , and These are the predicted values ​​for the categorical part of the data for students and teachers, respectively: right and Calculate the distillation prediction probabilities for students and teachers using the following formula. and : Where T represents the hyperparameter temperature, the degree of knowledge distillation can be adjusted by regulating T, and the distillation prediction probability of students and teachers can be predicted. and Calculate the binary cross-entropy to obtain the components of the distillation portion: Where log() represents logarithmic operation, with base 10; Next, it is necessary to calculate the non-distilled component. First, calculate the student's non-distilled prediction probability. : Distillation prediction probability for students Compared with the true value Calculate the binary cross-entropy to obtain the components of the non-distilled portion: The total classification loss is obtained by weighting the distilled and non-distilled components. : in, In practical engineering, it is recommended to adopt =0.99, It only applies to the positive sample region, while the original calculation formula remains unchanged for the negative sample region.

[0008] Preferably, the loss calculation formula for the predicted box regression is as follows: set up , and These are the predicted values ​​of students, teachers, and the actual values ​​regressed with respect to the prediction box: Before calculating the loss for the predicted bounding box, it is also necessary to calculate the weights of the predicted bounding box. : in , and All of these are related to the calculation of the classification part. In addition, the formula for calculating the Intersection over Union (IOU) is as follows: Predicted probabilities for students and teachers and Calculate the crossover ratio to obtain the fraction of the distillate: Similarly, predictions of students versus actual values and Calculate the crossover ratio to obtain the fraction of the distillate: The total classification loss is obtained by weighting the distilled and non-distilled components. : in, In practical engineering, it is recommended to adopt =0.99, It only applies to positive sample regions; it does not calculate negative sample regions.

[0009] Preferably, the loss calculation formula for the DFL portion is as follows: set up , and These are the values ​​for students, teachers, and the actual value with respect to the DFL portion: Before calculating the loss for the predicted bounding box portion, the weights of the DFL also need to be calculated. : Predicted probabilities for students and teachers and Calculate the cross-entropy to obtain the components of the distillation fraction: Similarly, predictions of students versus actual values and Calculate the crossover ratio to obtain the fraction of the distillate: The total classification loss is obtained by weighting the distilled and non-distilled components. : in, In practical engineering, it is recommended to adopt =0.99, It only applies to positive sample regions; it does not calculate negative sample regions.

[0010] Preferably, in S3 and S4, a flexible and convenient parallel knowledge distillation network is designed. First, the same set of crack images are simultaneously input into a common transition layer, then output to the student part network and the teacher part network respectively, and finally output through a unified connection layer. Through the parallel network architecture, the network features of the student and teacher are extracted and fused, thereby providing stable data input for subsequent loss function calculation. This invention also provides a real-time rail crack detection system based on knowledge distillation, the system being used to implement the aforementioned real-time rail crack detection method based on knowledge distillation, comprising: An image acquisition and preprocessing module is used to acquire the image to be processed and to preprocess the image; The knowledge distillation network loading module is used to load pre-trained teacher networks; The knowledge distillation training module is the core function, used to conduct knowledge distillation training. The knowledge distillation network extraction module is used to extract the student portion of the network for separate use. The system environment setup module is used to set up an embedded system environment; Install OpenCV related library modules to set up the model's runtime environment; The model loading module is used to select the required model and fine-tune the relevant parameters; The model data post-processing module is used to post-process the image after image recognition to obtain the final image recognition result; the post-processing includes at least non-maximum suppression, filtering and optimization. The image display module is used to display the final image recognition results through a display device and provide real-time feedback.

[0011] Compared to existing technologies, this invention offers the following advantages: It constructs a parallel training framework based on knowledge distillation on top of YOLOv8; it proposes corresponding knowledge distillation algorithms for classification, bounding box prediction, and DFL (Discretionary Flipping) in its loss function design; and it selects and deploys the improved model in an embedded system. The parallel training framework effectively improves the training efficiency of knowledge distillation, providing a suitable training platform for freely selecting student and teacher models (not limited to YOLOv8). The knowledge distillation loss function design fully considers the computational characteristics of classification, bounding box prediction, and DFL, providing an improved algorithm for object detection in the field of knowledge distillation. The flexible network architecture and loss function design together provide great transferability, allowing for flexible deployment on other algorithms in the field of object detection. Finally, corresponding algorithm optimizations and adjustments are made for embedded systems, and the algorithms are deployed in the system, providing a reliable solution for real-time railway track crack detection. Attached Figure Description

[0012] Figure 1 This is a flowchart of a real-time detection method for rail cracks based on knowledge distillation, according to the present invention.

[0013] Figure 2 This is a flowchart illustrating a real-time detection method for rail cracks based on knowledge distillation, as described in this invention. Figure 1 The details will be explained in more detail.

[0014] Figure 3 This is a schematic diagram of the parallel training network for knowledge distillation in this invention.

[0015] Figure 4 This is a schematic diagram of the knowledge distillation loss function design in this invention.

[0016] Figure 5 This is a monitoring diagram of the training process of the real-time rail crack detection model based on knowledge distillation of the present invention, which explains the structural details of the parallel training network.

[0017] Figure 6This is a monitoring diagram of the training process of the real-time rail crack detection model based on knowledge distillation of the present invention, which illustrates the changes of each component of the loss function during training.

[0018] Figure 7 This is a schematic diagram illustrating the deployment and application of the knowledge distillation-based real-time detection method for rail cracks according to the present invention. Detailed Implementation

[0019] The present invention will be further described below with reference to the accompanying drawings and specific embodiments, so that those skilled in the art can better understand and implement the present invention. The embodiments are not intended to limit the present invention.

[0020] Example 1 like Figure 1 and Figure 2 As shown, this invention discloses a real-time detection method for rail cracks based on knowledge distillation, comprising the following steps: S1: Obtain the rail crack image dataset for knowledge distillation training.

[0021] S1-1: Acquiring images from the rail crack image dataset: In practical applications of rail crack detection, various devices deployed along the railway line can be used for image acquisition. High-definition cameras and sensors installed near the railway tracks, as well as handheld mobile devices carried by staff, can all be used to acquire images of the rails. These devices not only help build the dataset in the early stages but also play a role in subsequent routine inspections.

[0022] During the inspection, intelligent devices capture images of the railway tracks from multiple angles and distances, thereby obtaining a large number of on-site images covering crack details and features. These image samples include railway tracks under different lighting conditions, with different degrees of wear, and in different crack states, ensuring the diversity and representativeness of the sample library. After collecting the images, the inspection equipment transmits the collected images of the power grid equipment to a remote NAS server via the network for storage and processing.

[0023] S1-2: Image preprocessing of the rail crack image dataset: S1-2-1: To optimize image quality and improve the accuracy of subsequent target detection algorithms, Gaussian denoising is performed on all images to eliminate noise. This method uses a Gaussian filter to smooth the image, removing some noise components, thereby improving image clarity and discernibility. This preprocessing effectively improves the performance and reliability of the target detection algorithm in subsequent rail crack inspection. Even in complex backgrounds and low-quality images, rail cracks can still be accurately detected.

[0024] S1-2-2: Adding labels to the denoised images: Accurate bounding boxes are added to the crack targets in the images using manual operation and dedicated image annotation software such as Labelme. Each image is manually labeled with a crack category label and its precise location within the image. The annotation results for each image are then converted into a TXT file, forming a complete target dataset. Labelme and similar tools can be used to meet the requirements for target annotation in railway crack detection. Manual operation ensures the accuracy and reliability of the annotation, providing high-quality labeled data for subsequent target detection algorithms.

[0025] S1-2-3: Perform multi-angle and multi-level rotation, flipping, and reconstruction on the samples in the dataset to augment the data. This includes using sample augmentation methods, sample expansion methods based on inner loop rotation, expansion methods based on sample reconstruction and inner loop rotation, Mosaic data augmentation techniques, and Mixup data augmentation techniques to expand the samples in the original dataset. By expanding the samples, the overfitting problem in deep learning models caused by a small number and variety of training samples can be minimized.

[0026] The following is combined with Figures 3-7 The following further illustrates the model training and detection process: S2: Based on the image set, train the teacher model and obtain the network parameters of the teacher model. S2-1: Select a suitable teacher model to ensure consistency in feature dimensions between the student and teacher models. The quality of the teacher model determines the upper limit of the learning model. The teacher model must maintain consistency in feature dimensions with the student model. In this example, among the five available sizes in YOLOv8: n, s, m, l, and x, size n or s is selected as the student model for actual deployment, and m, l, or x is selected as the teacher model for guidance. This selection ensures consistency in feature dimensions between the student and teacher models, which is key to improving detection accuracy while maintaining low computational power consumption for the student model.

[0027] S2-2: Train the teacher model using an image set containing smaller targets and more image-enhanced images. Teacher training does not need to consider computational limitations during deployment; it can be conducted using rented big data cloud computing servers. To maximize the quality of the teacher model, it is advisable to use image sets containing smaller target images for training. The learning process from the teacher model to the student model is essentially a mathematical approximation of a function with fewer parameters to a function with more parameters. To enable the learning model to learn as much detail as possible about small targets in the images, it is necessary to maximize the accuracy of the teacher model in judging small targets. Furthermore, it is advisable to employ more data augmentation techniques, including scaling, flipping, and appropriate occlusion.

[0028] S2-3: Specify the loading parameters for the teacher model to enable automatic parameter addition.

[0029] S2-3-1: Specifies the parameters of the teacher model. The teacher network consists of the following 23 layers: [1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 19, 20, 21, 22, 23]. In the trained teacher model, the parameters of each layer need to be extracted and loaded into the distillation network. These layers correspond to the following 23 layers in the distillation network: [23, 24, 25, 26, 27, 28, 29, 30, 31, 32, 33, 34, 35, 36, 37, 38, 39, 40, 41, 42, 43, 44, 46]. Using PyTorch's built-in functions, each layer of the teacher network is modified to its corresponding layer number, and the modified parameters are stored in the dictionary `save_state{}`. At the same time, the number of classes in the model is specified (nc, e.g., 80) to ensure consistency between teacher and student classifications during loss function calculation.

[0030] S2-3-2: Automatic parameter addition. Modify the YAML file to implement the knowledge distillation parallel training network: Layer 0 is set as a transition layer, layers [1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 19, 20, 21, 22, 45] are set as layers of the student network, layers [23, 24, 25, 26, 27, 28, 29, 30, 31, 32, 33, 34, 35, 36, 37, 38, 39, 40, 41, 42, 43, 44, 46] are set as layers of the teacher network, and finally, layer 47 is set as a connection layer. A distillation network without parameters is constructed using the YOLOv8 model building function YOLO() and a YAML file. The parameters from the dictionary save_state{} is added to the distillation network using PyTorch's load_state_dict function. This entire process automatically adds network parameters, effectively avoiding the repetitive operation of repeatedly extracting parameters.

[0031] S3: Input the network parameters of the teacher model into the distillation network, set multiple temperature parameters, and perform knowledge distillation training multiple times.

[0032] S3-1: Knowledge distillation training hyperparameter settings. In this example, the hyperparameters that need to be modified for knowledge distillation training include: the YAML file, the dataset `dataset=railway crack dataset`, the number of training epochs=800, whether to save during training `bool_save = True`, how many epochs to save if saving `save_period = 50`, whether to train on CPU or GPU `device = '0'` (on GPU), setting `batch_size = 16`, whether to distill `bool_distill = True`, the distillation model size `scale_student = 's'`, `scale_teacher = 'l'`, and the distillation hyperparameter settings `hyp_T = 10.0`, `hyp_box_distill = 0.70`, `hyp_cls_distill = 1.10`, `hyp_dfl_distill = 1.00`, `hyp_w_t_cls = 0.99`, `hyp_w_t_box = 0.99`, and `hyp_w_t_dfl = 0.99`. In addition to the hyperparameters mentioned above, the gradient descent algorithm and learning rate can also be modified. The core hyperparameter for knowledge distillation training is temperature (hyp_T): different detection accuracies can be achieved by adjusting the temperature. Therefore, it is advisable to set up multiple control groups ranging from 1 / 50 to 50, and select the required learning model for deployment based on the detection accuracy.

[0033] S3-2: Modify the knowledge distillation loss function. In this example, the loss function components for classification, predicted bounding boxes, and DFL are calculated according to the following formulas: Finally, the knowledge distillation loss is obtained by weighted summation of the loss function components: Where C1, C2, and C3 are the products of hyp_box_distill, hyp_cls_distill, and hyp_dfl_distill with the initial weights of the classification, predicted bounding box, and DFL, respectively.

[0034] S3-3: Input the network parameters of the teacher model into the distillation network and perform knowledge distillation training multiple times. After completing the hyperparameter settings and loss function modification, import the network parameters of the teacher model into the distillation network, set different T values, and train repeatedly. Select the model with the best detection accuracy for subsequent extraction and deployment. Figure 5As shown, after importing the network parameters of the teacher model into the distillation network, the network is trained. The distillation network has 47 layers. The teacher model layers must be frozen to prevent the backpropagation process from destroying the trained parameters. Figure 6 As shown, each round of training is monitored in real time to ensure that each component of the loss function decreases normally as training progresses, until training stops automatically ahead of schedule or 800 rounds of training are completed.

[0035] Among them, a knowledge distillation parallel training network is designed: The parallel training network consists of four main parts: an EmptyLayer for storing input image data, a Student's network, a Teacher's network, and a Unite Layer for summarizing the output features of the student and teacher.

[0036] The Empty Layer transition layer is relatively simple to construct. In the standard PyTorch module construction function, only the most basic backpropagation function `def forward(self, x): return x` is retained. An Empty Layer definition is added to the function initialization definition `_init_`, and the number of input channels is specified as 144 (including initial data for classification, predicted bounding boxes, and DFL) when building the network. The design of the transition layer ensures that students and teachers have consistent image input, allowing teachers to provide accurate guidance on the same image data during backpropagation.

[0037] The structure of the Student's network and the Teacher's network is consistent, and their construction is as follows: The backbone of the Student's network and the Teacher's network are retained, and the header DetectLayer of both is modified. In the constructor of the YOLOv8 DetectLayer module, the number of output channels (144) is modified to match the input format of the subsequent Unite Layer. The subsequent processing is canceled in the function, and the result of the operation is output only for the three feature dimensions: for i in range(self.nl): x[i] = torch.cat((self.cv2[i](x[i]), self.cv3[i](x[i])), 1) return x. The output x is the integrated feature of the student and teacher across three dimensions, which is then processed by the connection layer.

[0038] The construction of the connection layer Unite Layer is as follows: In the standard PyTorch module construction function, select the data function to be returned as needed. In the training mode, if self.training: return x directly returns the merged feature data of the teacher and the student; in the detection model, if bool_val_output: x_ = x[1] else: x_ = x[0] select x[1], which is the output data of the teacher, for fault detection and monitoring, and select x[0], which is the output data of the student, and perform post-processing on the student output in the subsequent process, including: formatting the data tensor to obtain classification and prediction box prediction, calculating the activation function of the classification prediction data to obtain the prediction probability, and performing non-maximum suppression (NMS) on the prediction of each anchor box to filter the final output.

[0039] The described construction method provides a network for parallel training of knowledge distillation. The data stream first passes through the Empty Layer, then is input to the student network, pausing at the Student's Detect Layer at the head of the student network. Next, buffered data is input from the teacher network, and the data stream continues until the Teacher's Detect Layer at the head of the teacher network. The outputs of both the Student's Detect Layer and the Teacher's Detect Layer are then combined and aggregated into the Unite Layer for subsequent training or detection processing. The entire process strictly adheres to PyTorch's unidirectional data transmission requirement, meaning only one module of data can be processed at a time, physically ensuring the reliability of output transmission. Logically, the data stream first passes through the Empty Layer, then simultaneously through both the student and teacher networks, and finally aggregates at the Unite Layer, thus achieving logical parallel transmission.

[0040] Next, we design a knowledge distillation loss function algorithm: Before calculating the knowledge distillation loss function, the features are first integrated. The feature tensor obtained from the parallel network is in the format [(8400, 144), (8400, 144)]. In this tensor format, 8400 represents the sum of the three feature dimensions (80, 80), (40, 40), and (20, 20), and 144 indicates that there are 144 channels in the prediction. The two numbers represent the features of the student and the teacher, respectively. The data integration method is as follows: The data integration method first splits the 144 channels. The first 80 channels correspond to 80 categories (set by the hyperparameter nc), and the last 64 channels correspond to the DFL prediction. The DFL prediction is divided into 4 groups, with the tensor format [16, 16, 16, 16] representing the 4 directions, used to locate the specific position of the prediction box. The dot product is calculated between the tensor and the tensor vector (0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15) to obtain the specific position information of the 4 directions as the prediction box value. The final split tensor structure is [(8400, 80, 4, 64), (8400, 80, 4, 64)].

[0041] The data integration method then calculates positive samples. Based on YOLOv8's TaskAlignedAssigner strategy, it calculates the positive sample region for students and the corresponding true value size, selecting a small number of samples as positive student samples. The positive sample region for teachers is consistent with that for students. The selected positive samples are then used to reconstruct a tensor [(x, 80, 4, 64), (x, 80, 4, 64)], where x is the number of positive samples.

[0042] The algorithm for calculating the knowledge distillation loss function is as follows: First, calculate the classification components: Take the student component (x, 80) and the teacher component (x, 80) from tensor[(x, 80, 4, 64), (x, 80, 4, 64)] as... and The true value tensor (x, 80) obtained during the calculation of positive samples is taken as... .right and Calculate the distillation prediction probabilities for students and teachers using the following formula. and The tensor format is [(x, 80), (x, 80)]: The distillation prediction probability is introduced by incorporating temperature T into the original activation function. The distillation prediction probabilities for students and teachers are then calculated. and Calculate the binary cross-entropy to obtain the components of the distillation fraction. The tensor format is (x, 80): Here, log() represents the logarithmic operation, usually base 10. Next, the non-distilled component is calculated, starting with the student's non-distilled prediction probability. The tensor format is (x, 80): Distillation prediction probability for students Compared with the true value Calculate the binary cross-entropy to obtain the components of the non-distilled portion. The tensor format is (x, 80): The total classification loss is obtained by weighting the distilled and non-distilled components. The tensor format is (x, 80): in, In practical engineering, it is recommended to adopt =0.99. Calculate the weights of the component parts. The tensor format is (x): The calculation of the knowledge distillation loss function, for The tensor format is (x, 80), which is then weighted and integrated to obtain the final result. The format is a single floating-point number: in This represents the loss sum outside the positive sample region, consistent with the initial loss function of YOLOv8.

[0043] Next, the predicted bounding box components are calculated: the student component (x, 4) and the teacher component (x, 4) from tensor[(x, 80, 4, 64), (x, 80, 4, 64)] are taken as... and The true value tensor (x, 80) obtained during the calculation of positive samples is taken as... Before calculating the loss for the predicted bounding box, calculate the weights of the predicted bounding box. Tensor format (x): in , and All of these are related to the calculation of the classification part. The formula for calculating the Intersection over Union (IOU) is as follows: Predicted probabilities for students and teachers and Calculate the crossover ratio to obtain the fraction of the distillate. Tensor format (x, 4): Similarly, predictions of students versus actual values and Calculate the crossover and union ratio to obtain the fraction of the non-distilled portion. Tensor format (x, 4): The total classification loss is obtained by weighting the distilled and non-distilled components. Tensor format (x, 4): in, In practical engineering, it is recommended to adopt =0.99.

[0044] The calculation of the knowledge distillation loss function, for The tensor format is (x, 4), which is then weighted and integrated to obtain the final result. The format is a single floating-point number: Any samples within the non-positive sample region are not counted.

[0045] Next, the DFL components are calculated: the student component (x, 64) and the teacher component (x, 64) in tensor[(x, 80, 4, 64), (x, 80, 4, 64)] are taken as... and The true value tensor (x, 64) obtained during the calculation of positive samples is taken as... Before calculating the loss for the predicted bounding box portion, calculate the weights of the DFL. Tensor format (x): Predicted probabilities for students and teachers and Calculate the cross-entropy to obtain the components of the distillation portion. The tensor format is (x, 64): Similarly, predictions of students versus actual values and Calculate the crossover ratio to obtain the fraction of the distillate. The tensor format is (x, 64): The total classification loss is obtained by weighting the distilled and non-distilled components. The tensor format is (x, 64): in, In practical engineering, it is recommended to adopt =0.99.

[0046] The calculation of the knowledge distillation loss function, for The tensor format is (x, 64), which is then weighted and integrated to obtain the final result. The format is a single floating-point number: Any samples within the non-positive sample region are not counted.

[0047] The knowledge distillation loss function is ultimately calculated using the following formula: in , , These are the pre-defined hyperparameters.

[0048] S4: Extract the trained student model from the distillation network, compare the detection accuracy, and select the model that best meets the actual needs.

[0049] S4-1: Specifies the extracted parameters for the student model. The student network also consists of the following 23 layers: [1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 19, 20, 21, 22, 23]. In the trained student model, the parameters of each layer need to be extracted and loaded into a separate student model for subsequent processing. These layers correspond to the following 23 layers in the distillation network: [1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 19, 20, 21, 22, 45]. Similarly, using PyTorch's built-in functions, each layer in the distillation network is modified to correspond to the layer number of the student model, and the modified parameters are stored in the dictionary `save_state{}`. At the same time, the number of classes in the model is specified (e.g., 80) to ensure consistency between teacher and student classifications during loss function calculation.

[0050] S4-2: Compare the detection accuracy of the extracted student models. The training parameters of the models trained at each temperature are saved once during each training round, and these parameters can be used as student models for deployment. In this example, the detection accuracy of these student models first increases and then gradually decreases as training progresses, eventually stabilizing. Five to ten models with the best detection accuracy are selected from these models and compared horizontally with the models with the best detection accuracy at other temperatures. Finally, the model with the best detection accuracy is selected as the student model for subsequent deployment.

[0051] S5: Deploy the student model to an embedded device for real-time detection.

[0052] S5-1: Setting up the Embedded System Environment. It is advisable to choose an embedded system that meets real-time detection requirements and is reasonably priced as the target system. In this example, a Raspberry Pi 4B is chosen as the deployed embedded system. First, burn the Ubuntu system onto the Raspberry Pi. Obtain the Ubuntu system image from the Raspberry Pi website or a tutorial, and burn the system onto the Raspberry Pi's memory card using a USB flash drive or similar method to complete the system environment setup. Configure Image Input and Output: The Raspberry Pi 4B comes with a small camera and a wired screen, supporting both real-time image transmission and remote image monitoring modes. It is recommended to use a wired screen for real-time detection, ensuring a stable connection between the camera and screen and the embedded board, guaranteeing both are available in real-time.

[0053] S5-2: Install OpenCV related library modules and set up the model's runtime environment.

[0054] S5-2-1: Install OpenCV dependencies. OpenCV is currently the mainstream dependency library for deploying object recognition models. Obtain the most stable version from the OpenCV official website and load it into the embedded system. Note that you should choose the OpenCV version compatible with the Ubuntu system. Compile the OpenCV files. In this example, after compiling the OpenCV files using CMake, two dependency library files, lib and build, can be used on the Ubuntu system. In the detection program, specify the generated lib and build as the runtime dependencies for the C++ library.

[0055] S5-2-1: Install QT dependency libraries and import the detection program. QT dependency libraries are responsible for UI construction and are essential for visualizing real-time detection. It is recommended to use QT dependency library version 5 or higher for deployment; otherwise, QT dependency libraries may conflict with OpenCV versions. The installation steps for QT dependency libraries are similar to those for OpenCV and will not be repeated here. In this example, the real-time detection program is written in C++ and its functions include: capturing images from the real-time data stream from the camera, performing forward propagation of the model to output the predicted bounding box positions and classification information in the images, and visualizing the detection results. Each time an image is captured, image processing is performed, and the results are simultaneously generated on the display screen, thus achieving real-time monitoring.

[0056] S5-3: Model Selection and Loading. Select the best model obtained in step S4 and modify its Pt format (PyTorch format) to a suitable ncnn format for deployment. First, use PyTorch's built-in ONNX format conversion function to convert the best model to ONNX format. Then, further convert the ONNX format to ONNXSIM format. Finally, use a commercially available ncnn conversion tool to complete the ncnn format conversion. The final best model will be in .bin and .param formats. Set specific parameters such as image resolution and image refresh rate. The initial image resolution depends on the camera resolution. In actual detection, it is advisable to set the input model's image resolution to a multiple of 320. In this example, the image resolution is set to 640x640. The image refresh rate depends on the real-time processing speed of the embedded system. In actual detection, it is advisable to set it to match the processing rate of one frame of the model. In this example, the image refresh rate is set to 10 frames per second.

[0057] S5-4: Post-process the image after image recognition and visualize the results. The recognized image will generate numerous predicted bounding boxes, making the selection of these boxes crucial. Non-maximum suppression (NMS) is recommended. Currently, mainstream NMS strategies include Soft NMS, GIoU NMS, DIoU NMS, and CIoU NMS. In this example, SoftNMS is used for non-maximum suppression. The processed image is input into the UI module and displayed on a device to provide real-time feedback. Finally, the configured embedded device is handed over to railway monitoring personnel for real-time detection of rail cracks.

[0058] Example 2 This invention also provides a real-time rail crack detection system based on knowledge distillation, comprising: The image acquisition and preprocessing module is used to acquire the image to be processed and to preprocess the crack image.

[0059] The knowledge distillation network loading module is used to load pre-trained teacher networks, including: selecting a suitable teacher model to ensure that the feature dimensions of students and teachers are consistent; training the teacher model using an image set containing smaller targets and more image-enhanced images; and specifying the loading parameters of the teacher model to achieve automatic parameter addition.

[0060] The knowledge distillation training module is the core function used to conduct knowledge distillation training, including: setting hyperparameters related to knowledge distillation; and conducting multiple rounds of distillation training at various temperatures.

[0061] The knowledge distillation network extraction module is used to extract the student component from the network for separate use, including: specifying the extraction parameters of the student model and testing the extracted model.

[0062] The system environment setup module is used to build the embedded system environment, including: burning the embedded device system and configuring the visualization system.

[0063] This module installs OpenCV-related libraries to set up the model's runtime environment, including: installing OpenCV dependencies, installing QT dependencies, and importing the detection program.

[0064] The model loading module is used to select the required model, including selecting a suitable training model for loading and setting specific parameters such as image resolution and image refresh rate.

[0065] The model data post-processing module is used to post-process the images after image recognition to obtain the final image recognition results, including nonmaximum suppression, filtering, and optimization.

[0066] The image display module is used to display the final image recognition results through a display device and provide real-time feedback information.

Claims

1. A real-time detection method for rail cracks based on knowledge distillation, characterized in that, include: S1, Obtain the rail crack dataset for knowledge distillation training; S2, based on the railway crack dataset, train the teacher model and obtain some network parameters of the teacher model; S3 involves incorporating some network parameters from the teacher model into the distillation network, setting multiple temperature parameters, and performing knowledge distillation training multiple times. S4. Extract the trained model from the distillation network, compare the detection accuracy, and select the model that best meets the actual needs as the student model. S5 deploys student models onto embedded devices for real-time detection.

2. The real-time detection method for rail cracks based on knowledge distillation as described in claim 1, characterized in that, Specifically, S4 includes: designing an improved knowledge distillation loss function based on the characteristics of the YOLOv8 loss function, as follows: in It is the loss of the classification part. It is the loss of the predicted box regression. This is the loss of the DFL portion, in addition , , These are the preset hyperparameters.

3. The real-time detection method for rail cracks based on knowledge distillation as described in claim 2, characterized in that: The formula for calculating the loss in the classification part is as follows: set up , and These are the predicted values ​​for the categorical part of the data for students and teachers, respectively: right and Calculate the distillation prediction probabilities for students and teachers using the following formula. and : Where T represents the hyperparameter temperature, the degree of knowledge distillation can be adjusted by regulating T, and the distillation prediction probability of students and teachers can be predicted. and Calculate the binary cross-entropy to obtain the components of the distillation portion: Where log() represents logarithmic operation, with base 10; Next, it is necessary to calculate the non-distilled component. First, calculate the student's non-distilled prediction probability. : Distillation prediction probability for students Compared with the true value Calculate the binary cross-entropy to obtain the components of the non-distilled portion: The total classification loss is obtained by weighting the distilled and non-distilled components. : in, In practical engineering =0.99, It only applies to the positive sample region, while the original calculation formula remains unchanged for the negative sample region.

4. The real-time detection method for rail cracks based on knowledge distillation as described in claim 2, characterized in that: The loss calculation formula for bounding box regression is as follows: set up , and These are the predicted values ​​of students, teachers, and the actual values ​​regressed with respect to the prediction box: Before calculating the loss for the predicted bounding box, it is also necessary to calculate the weights of the predicted bounding box. : in , and All of these are related to the calculation of the classification part. In addition, the formula for calculating the Intersection over Union (IOU) is as follows: Predicted probabilities for students and teachers and Calculate the crossover ratio to obtain the fraction of the distillate: Similarly, predictions of students versus actual values and Calculate the crossover ratio to obtain the fraction of the distillate: The total classification loss is obtained by weighting the distilled and non-distilled components. : in, In practical engineering, it is recommended to adopt =0.99, It only applies to positive sample regions; it does not calculate negative sample regions.

5. The real-time detection method for rail cracks based on knowledge distillation as described in claim 2, characterized in that: The formula for calculating the loss of the DFL portion is as follows: set up , and These are the values ​​for students, teachers, and the actual value with respect to the DFL portion: Before calculating the loss for the predicted bounding box portion, the weights of the DFL also need to be calculated. : Predicted probabilities for students and teachers and Calculate the cross-entropy to obtain the components of the distillation fraction: Similarly, predictions of students versus actual values and Calculate the crossover ratio to obtain the fraction of the distillate: The total classification loss is obtained by weighting the distilled and non-distilled components. : in, In practical engineering, it is recommended to adopt =0.99, It only applies to positive sample regions; it does not calculate negative sample regions.

6. The real-time detection method for rail cracks based on knowledge distillation as described in claim 1, characterized in that: In S3 and S4, a flexible and convenient parallel knowledge distillation network is designed. First, the same set of crack images are simultaneously input into a common transition layer, and then output to the student part network and the teacher part network respectively. Finally, the output is output through a unified connection layer. Through the parallel network architecture, the network features of the student and teacher are extracted and fused, thereby providing stable data input for subsequent loss function calculation.

7. A real-time rail crack detection system based on knowledge distillation, the system being used to implement the real-time rail crack detection method based on knowledge distillation as described in any one of claims 1-6, characterized in that, include: An image acquisition and preprocessing module is used to acquire the image to be processed and to preprocess the image; The knowledge distillation network loading module is used to load pre-trained teacher networks; The knowledge distillation training module is the core function, used to conduct knowledge distillation training. The knowledge distillation network extraction module is used to extract the student portion of the network for separate use. The system environment setup module is used to set up an embedded system environment; Install OpenCV related library modules to set up the model's runtime environment; The model loading module is used to select the required model and fine-tune the relevant parameters; The model data post-processing module is used to post-process the image after image recognition to obtain the final image recognition result; the post-processing includes at least non-maximum suppression, filtering and optimization. The image display module is used to display the final image recognition results through a display device and provide real-time feedback.

Images