Steel rail damage identification method, device and equipment based on multi-teacher guidance and medium

Abstract

The invention relates to a steel rail damage identification method and device based on multi-teacher guidance, equipment and a medium. The method comprises the steps that a first data set is acquired; a plurality of target teacher network models are obtained through training according to the first data set, and at least one target teacher network model for processing the first task and one target teacher network model for processing the second task are obtained through training; according to the first data set and the target teacher network model, training to obtain a first student network model for processing the first task and a second student network model for processing the second task; obtaining a second data set; and processing the second data set according to the target detection model to obtain a steel rail damage identification result. The training of the student network model is realized through the multi-teacher network model corresponding to different tasks, and the steel rail flaw target detection is realized by using the student network model, so that the deployment flexibility of the target detection model on the edge equipment can be improved, and the target detection accuracy of the steel rail flaw detection equipment can be effectively improved.

Description

Technical Field

[0001] The present application relates to the fields of signal processing and computer vision technology, and in particular to a rail damage identification method, device, equipment and medium based on multi-teacher guidance. Background Art

[0002] In ultrasonic rail damage detection, large rail flaw detection vehicles use ultrasonic detection technology to identify damage inside the rails. Currently, it is mainly based on the ultrasonic B-display image of the rails and uses target detection and other related technologies to locate and detect rail damage.

[0003] Target detection algorithms can automatically locate and detect damage, but their target recognition performance is often not as accurate as that of specialized classification models. This is especially true for ultrasonic B-scan image data, which has been artificially abstracted to express specific physical meanings. Target detection models often have certain defects.

[0004] In addition, larger network models not only consume a lot of computer hardware resources for their own training, inference, and calculation, but also are difficult to deploy in edge devices to provide lightweight and fast detection due to the huge number of parameters, which limits the use scenarios of intelligent identification methods for rail flaw detection. Summary of the Invention

[0005] Based on this, it is necessary to provide a rail damage identification method, device, equipment and medium based on multi-teacher guidance, which can effectively and accurately complete rail damage target detection and facilitate the deployment of target detection models in edge devices to address the above technical problems.

[0006] In a first aspect, the present application provides a rail damage identification method based on multi-teacher guidance, comprising:

[0007] Acquire a first data set, wherein the first data set includes historical B-display image data during a rail ultrasonic flaw detection process;

[0008] Training multiple target teacher network models based on the first data set, wherein at least one target teacher network model for processing the first task is trained, and at least one target teacher network model for processing the second task is trained;

[0009] Training a target detection model based on the first data set and the target teacher network model, wherein the target detection model includes a first student network model for processing the first task and a second student network model for processing the second task;

[0010] Acquiring a second data set, wherein the second data set includes real-time B-display image data during the rail ultrasonic flaw detection process;

[0011] The second data set is processed according to the target detection model to obtain a rail damage identification result.

[0012] In one embodiment, the first task includes a classification task, the second task includes a target box regression task, the target teacher network model for processing the first task includes a ResNet-101 network model, and the target teacher network model for processing the second task includes a YOLOv5X network model; the model scale of the target teacher network model for processing the first task is larger than the scale of the first student network model, and the model scale of the target teacher network model for processing the second task is larger than the scale of the second student network model;

[0013] The training of a target detection model based on the first data set and the target teacher network model includes:

[0014] respectively determining a basic model structure of the first student network model and a basic model structure of the second student network model;

[0015] Performing model training on the first student network model and the second student network model based on the first data set, wherein, during the model training process, the student network model is guided to perform model training according to multiple target teacher network models and target loss functions;

[0016] When the target loss function of the first student network model meets the preset optimal conditions and the target loss function of the second student network model meets the preset optimal conditions, the target detection model is obtained by combining the first student network model and the second student network model.

[0017] In one embodiment, when the target teacher network model for processing the second task is a YOLOv5X network model, the basic model structure of the second student network model adopts the model structure of the YOLOv5l network model; wherein the backbone part of the YOLOv5l network model adopts the ResNet18 network structure.

[0018] In one embodiment, obtaining the first data set includes:

[0019] Acquire a rail ultrasonic signal collected by a preset channel during a historical detection period, wherein the rail ultrasonic signal includes a pulse position of an echo point, depth information, and a channel name;

[0020] Obtaining the historical B-display image data based on the rail ultrasonic signal;

[0021] A preprocessing operation is performed on the historical B-display image data to obtain the first data set, wherein the preprocessing operation includes image feature optimization processing and signal noise removal processing.

[0022] In one embodiment, obtaining the second data set includes:

[0023] Acquire the rail ultrasonic signal collected by the preset channel in the current detection cycle, wherein the rail ultrasonic signal includes the pulse position, depth information and channel name of the echo point;

[0024] Obtaining the real-time B-display image data based on the rail ultrasonic signal;

[0025] A preprocessing operation is performed on the real-time B-display image data to obtain the second data set, wherein the preprocessing operation includes image feature optimization processing and signal noise removal processing.

[0026] In one embodiment, the image feature optimization processing includes:

[0027] Determining each target to be identified in the B-display image data, wherein the target to be identified includes a rail head area, a rail waist area, and a rail bottom area;

[0028] Adjusting the background color of the target to be identified to a uniform color according to preset color information, wherein different targets to be identified have different background colors;

[0029] The signal noise removal process includes:

[0030] Converting the B-display image data into a binary image, wherein positions where echo point signals exist in the B-display image data correspond to pixel values ​​1 in the binary image, and other positions in the B-display image data correspond to pixel values ​​0 in the binary image;

[0031] Performing dilation processing on the binary image based on a preset rectangular dilation kernel to obtain a dilated image;

[0032] The dilated image is corroded based on a preset corrosion kernel to obtain denoised B-display image data.

[0033] In a second aspect, the present application further provides a rail damage identification device based on multi-teacher guidance, comprising:

[0034] A first acquisition module is configured to acquire a first data set, wherein the first data set includes historical B-display image data during a rail ultrasonic flaw detection process;

[0035] A first training module is configured to train multiple target teacher network models based on the first data set, wherein at least one target teacher network model for processing the first task is trained, and at least one target teacher network model for processing the second task is trained;

[0036] a second training module, configured to train a target detection model based on the first data set and the target teacher network model, wherein the target detection model includes a first student network model for processing the first task and a second student network model for processing the second task;

[0037] A second acquisition module is configured to acquire a second data set, wherein the second data set includes real-time B-display image data during the rail ultrasonic flaw detection process;

[0038] A data processing module is used to process the second data set according to the target detection model to obtain a rail damage identification result.

[0039] In a third aspect, the present application further provides a computer device comprising a memory and a processor, wherein the memory stores a computer program, and when the processor executes the computer program, the steps of the rail damage identification method based on multi-teacher guidance described in the first aspect are implemented.

[0040] In a fourth aspect, the present application further provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the steps of the rail damage identification method based on multi-teacher guidance described in the first aspect.

[0041] In a fifth aspect, the present application further provides a computer program product, which includes a computer program, and when the computer program is executed by a processor, it implements the steps of the rail damage identification method based on multi-teacher guidance described in the first aspect.

[0042] In summary, the present application proposes a rail damage identification method, apparatus, device and medium based on multi-teacher guidance, including: obtaining a first data set; training multiple target teacher network models based on the first data set, wherein at least one target teacher network model for processing the first task and one target teacher network model for processing the second task are trained; training a first student network model for processing the first task and a second student network model for processing the second task based on the first data set and the target teacher network model; obtaining a second data set; processing the second data set according to the target detection model to obtain rail damage identification results. The present application implements the training of student network models through multi-teacher network models corresponding to different tasks, and uses student network models to implement rail damage target detection. This not only improves the deployment flexibility of target detection models on edge devices, but also effectively improves the accuracy of target detection by rail flaw detection equipment. BRIEF DESCRIPTION OF THE DRAWINGS

[0043] Figure 1 FIG2 is an application environment diagram of a rail damage identification method based on multi-teacher guidance in one embodiment;

[0044] Figure 2 1 is a flow chart of a rail damage identification method based on multi-teacher guidance in one embodiment;

[0045] Figure 3 Schematic diagram of the model structure of the ResNet series network model in one embodiment;

[0046] Figure 4 Schematic diagram of a YOLOv5 series network model in another embodiment;

[0047] Figure 5 A schematic diagram of the steps for training an object detection model in one embodiment;

[0048] Figure 6 Schematic diagram of the model structure of the YOLOv51 network model in one embodiment;

[0049] Figure 7 A schematic diagram of a flow chart of image feature optimization processing steps in one embodiment;

[0050] Figure 8 A schematic diagram of a process flow for signal noise removal in one embodiment;

[0051] Figure 9 Schematic diagram of an application of image denoising based on puffing and erosion in one embodiment;

[0052] Figure 10 A schematic diagram of signal trends of a rail damage identification method based on multi-teacher guidance in one embodiment;

[0053] Figure 11 1 is a structural block diagram of a rail damage identification device based on multi-teacher guidance in one embodiment;

[0054] Figure 12 FIG. 1 is a diagram showing the internal structure of a computer device in one embodiment. DETAILED DESCRIPTION

[0055] In order to make the purpose, technical solutions and advantages of this application more clear, the following further describes this application in detail with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain this application and are not intended to limit this application.

[0056] In related technologies, rails inevitably develop internal damage during daily operation due to the influence of environmental conditions such as pressure, humidity, and temperature. As a nondestructive testing method, rail ultrasonic inspection systems utilize the principle that ultrasonic waves propagating through a uniform medium are reflected upon encountering damage (inhomogeneous media) to detect damage. Mapping ultrasonic signals into B-display image data for frame-by-frame playback is a common method in current operations. Because B-display image data is redrawn from the original ultrasonic echo signal, it is, to some extent, a processed data form, resulting in a loss of some physical information. This makes rail flaw detection more difficult and requires extensive prior knowledge.

[0057] In practical applications, directly applying target detection algorithms and models for damage detection, such as Faster-RCNN, the YOLO series of models, SSD, and even the DETR target detection model, can achieve end-to-end damage detection. However, due to the unclear classification boundaries between detection targets and the lack of prior knowledge to guide them, it is necessary to set numerous constraints on the target detection model and use post-processing methods to achieve optimal damage detection. Furthermore, target detection models are relatively weak in terms of simple classification results and target bounding box regression compared to simple classification models and large detection networks, making ultimate damage identification difficult.

[0058] At the same time, due to the large number of parameters in the deep model, its network is difficult to migrate to a small edge computing host, making it difficult to truly use it on board.

[0059] In order to solve the above problems, the present invention proposes a rail damage recognition method based on multi-teacher guidance knowledge distillation, which trains multiple teacher models with excellent performance for different recognition tasks. For example, a classification teacher model is trained for the classification task, and a larger-sized target detection teacher model is trained for the target detection task. Then, a target detection model with a smaller size and fewer parameters is used as the student network model. Under the guidance of multiple teacher models, relevant guided training is completed for the classification task and the target detection task respectively, and finally the smaller-sized target detection model is used to achieve end-to-end detection of ultrasonic rail damage. The rail damage recognition method based on multi-teacher guidance provided by this embodiment can not only enable the small model to have stronger recognition ability, solve the lack of category judgment ability of a single target detection network, but also reduce the model size, which is of great significance for edge deployment.

[0060] The following is a detailed introduction to the rail damage identification method based on multi-teacher guidance proposed in this implementation:

[0061] The rail damage identification method based on multi-teacher guidance provided in the embodiment of the present application can be applied to Figure 1 In the application environment shown, the terminal 102 communicates with the server 104 via a network. The data storage system can store data that the server 104 needs to process. The data storage system can be integrated with the server 104 or placed in the cloud or other network servers. The terminal 102 can be a rail flaw detection vehicle. The rail ultrasonic flaw detection vehicle in this embodiment can be used to detect damage defects in areas such as the rail head, rail waist, and rail bottom. The server 104 can be implemented as a standalone server or a server cluster consisting of multiple servers.

[0062] In one embodiment, Figure 2 As shown in the figure, a rail damage identification method based on multi-teacher guidance is provided. Figure 1 The rail flaw detection vehicle in the example is used as an example to illustrate, including the following steps:

[0063] S201: Acquire a first data set, wherein the first data set includes historical B-display image data during a rail ultrasonic flaw detection process.

[0064] In this embodiment, the first data set is used to implement the training and verification of the teacher network model and the student network model. In actual application, the first data set can be generated by collecting ultrasonic signals collected from each edge end of the rail flaw detection system within a preset historical detection cycle, and then plotting them according to different legends to represent different channels to generate a B-display image for playback. Among them, the B-display image is an image form that intuitively displays the internal defect characteristics of the inspected object through two-dimensional cross-sectional imaging technology. In this embodiment, the B-display image can directly display the shape, location, and depth of the defect, such as the rail head core flaw and weld seam weld bar echo in rail flaw detection. The playback function of the B-display image allows for secondary analysis of the ultrasonic echo data to assist in determining the nature of the rail defect (such as the relationship between the core flaw and the incident direction of the sound beam).

[0065] It should be noted that the historical B-display image data can be obtained from the memory, or obtained by communicating with other terminals, or obtained through a server. This embodiment does not limit the specific method of obtaining the historical B-display image data, and the appropriate acquisition method can be selected according to the needs of the actual application scenario.

[0066] In this embodiment, the preset historical detection cycle can be a detection cycle prior to the current detection cycle, or multiple detection cycles prior to the current detection cycle. The number of historical detection cycles and the specific duration of each cycle can be configured according to the needs of the actual application scenario. It should be noted that the duration of the preset historical detection cycle must ensure that sufficient historical B-display image data is collected to complete the training and verification of the teacher network model and the student network model based on the first data set.

[0067] S202, multiple target teacher network models are trained based on the first data set, wherein at least one target teacher network model is trained to process the first task, and at least one target teacher network model is trained to process the second task.

[0068] In this embodiment, after obtaining a first dataset, the first dataset is divided into a training dataset for model training and a validation dataset for model validation. Multiple target teacher network models are trained based on the training dataset, and the performance of each target teacher network model is verified using the validation dataset. After verification, the multiple target teacher networks and loss functions are used to guide the learning and training of the student network model.

[0069] In this embodiment, the target teacher network model is used to implement training guidance for the student network model. Furthermore, the target teacher network model does not provide parameter guidance for the student network model. Therefore, the network structure and parameters of the student network model do not need to be consistent with those of the teacher model. This allows for a larger design space for the student network model.

[0070] In this embodiment, based on the various recognition tasks that the target detection model needs to complete, the target teacher network model corresponding to the single recognition task can be trained separately. In actual application, the number of recognition tasks is greater than or equal to 2. In this embodiment, the recognition task includes at least a first task and a second task. In one embodiment, the first task includes a classification task, and the second task includes a target box regression task. It should be noted that the number and specific types of recognition tasks can be configured according to the needs of the actual application scenario.

[0071] In this embodiment, there can be one or more target teacher network models trained based on the same recognition task. By training multiple different target teacher network models for the same recognition task, the most suitable teacher network model can be selected for guidance in the subsequent training of the student network model. Alternatively, multiple teacher network models can be used to simultaneously guide the training of the student network, effectively improving the processing accuracy of the student network model in the corresponding recognition task.

[0072] In one embodiment, the teacher network model selected in this embodiment for processing classification tasks can be a ResNet-101 network model. Among them, the ResNet-101 network model uses 101 convolutional layers for feature extraction and final classification. In actual scenarios, the ResNet series of networks adopts a residual structure to avoid the problem of gradient explosion caused by the continuous extraction of higher-level semantic information during the convolution process as the depth of the convolutional depth network deepens. It combines high-level semantic information with low-level basic features through cross-bridging.

[0073] When training the teacher network model ResNet-101, this embodiment cuts each target out of the image according to the annotation content in the B-display image data and scales it to 224. The image size of 224 is used as the image input of the classification network. The ResNet-101 network structure is shown in the figure below. Figure 3 In this embodiment, the classification network model loss is recorded as , the damage category is classified through the softmax function.

[0074] In one embodiment, the teacher network model used in this embodiment to process the target frame regression task can be a YOLOv5X model, and its target frame loss is the calculated mean square error loss, which uses the regression loss method to calculate the difference between the predicted target frame position and the ground_truth, and finally iterates to determine the target position. Its target frame regression loss is recorded as Among them, ground_truth refers to the exact location and category of the object in the image. The YOLOv5X model is the largest in the YOLO series of models. The specific sizes of the series are as follows: Figure 4 The target regression loss function is calculated using CIOU-Loss. This embodiment uses the YOLOv5X network model to effectively improve the target frame regression capability and the accuracy of determining the target frame.

[0075] Based on the above steps, this embodiment can obtain multiple target teacher network models specifically for processing corresponding recognition tasks based on the first data set training. The teacher-student network model training method used in this embodiment does not use the parameters of the teacher model to guide the parameters of the student network model, but rather associates according to the recognition task. The student network model is guided by multiple teacher networks corresponding to the recognition task, and the loss function is combined to jointly guide the student network model for learning and training. This can greatly expand the design flexibility of the student network model, and can train a student network model with higher recognition accuracy, thereby obtaining a high-precision, low-size target detection model suitable for edge deployment.

[0076] In actual applications, a single target detection model mainly realizes end-to-end automatic detection and identification of damage. Taking YOLO as an example, the final loss function combines the prediction box regression loss, target category and confidence loss, and foreground and background judgment loss. Its judgment of target category is not as high as that of a simple classification network. The larger the target box regression, the more accurate the regression loss calculated by the model.

[0077] S203: Train a target detection model based on the first data set and the target teacher network model, wherein the target detection model includes a first student network model for processing the first task and a second student network model for processing the second task.

[0078] In this embodiment, for a student network model that is smaller in size and scale than the teacher network model, multiple targeted task classifications and target teacher network models are used to provide specific task guidance, and multiple student network models that can cope with different recognition tasks can be trained.

[0079] In this embodiment, the first student network model and the second student network model can be a separate target detection model, or can be multiple sub-models that can be subsequently combined into the same target detection model.

[0080] In actual applications, by guiding a certain recognition task of the target detection model through a specific target teacher network model, the accuracy of the target detection model on the recognition task can be effectively improved. For example, by using a target teacher network model for processing the target box regression task to guide the target detection model to train the target box regression task, the damage recognition effect of the target teacher network model with a larger scale, larger size and more parameters can be adapted to the target detection model with a smaller scale, smaller size and fewer parameters, thereby facilitating the deployment of the target detection model at the edge while greatly improving the accuracy of the target detection model in processing the target box regression task. Similarly, the target teacher network model used to process the classification task guides the target detection model to train the classification process, which can effectively improve the accuracy of the target detection model in processing the classification task.

[0081] In one embodiment, the scale of the student network model corresponding to the target detection model may also be greater than or equal to the scale and size of the teacher network model. In this case, the target detection model may be configured on the non-edge side, or other devices or systems that can adapt to the deployment of large-scale target detection models, according to the needs of actual applications. It should be noted that in this embodiment, the teacher network model's recognition effect on rail damage is better than the damage recognition effect of the target detection model. As the target detection model is learned and trained, the recognition effect of the target detection model will continue to approach the rail damage recognition effect of the teacher network model. Among them, the recognition effect on rail damage includes the positioning accuracy and classification accuracy of the damaged target position. The recognition speed of the target detection model and the teacher network model is determined according to the specific structure and model scale of the model in the actual application process. The recognition speed of the target detection model and the teacher network model may be the same or different, and is not limited here.

[0082] S204: Acquire a second data set, wherein the second data set includes real-time B-display image data during the rail ultrasonic flaw detection process.

[0083] In this embodiment, the second dataset is acquired in a similar manner to the first dataset. Ultrasonic signals collected from each edge of the rail flaw detection system during a real-time detection cycle are collected and then plotted according to different legends representing different channels to generate a B-display image for playback. The specific method for acquiring the second dataset can be referenced to the method for acquiring the first dataset in the aforementioned embodiment and will not be further elaborated here.

[0084] In this embodiment, the real-time detection period is the detection period of the ultrasonic detection vehicle at the current moment.

[0085] S205: Process the second data set according to the target detection model to obtain a rail damage recognition result.

[0086] In this embodiment, after obtaining the verified target detection model, the B-display image data in the newly acquired second data set can be used as input and output to the target detection model to finally obtain the rail damage recognition result.

[0087] Based on the above steps, this embodiment provides a rail damage identification method based on multi-teacher guidance. This method can guide the training and verification of a student network model using a multi-teacher network model and loss function. The teacher network model and target detection model are trained using the same dataset, improving the design flexibility of the target detection model and effectively reducing the model size and parameters of the target detection model, thereby facilitating its deployment at the edge, such as on a detection vehicle. Furthermore, this embodiment, based on the multi-teacher network model guiding the training of the student network model, can significantly improve the flaw detection and identification performance of the target detection model deployed at the edge, achieving precise classification and target frame determination, thereby obtaining more accurate rail damage location and classification results.

[0088] In one embodiment, the model scale of the target teacher network model for processing the first task is larger than the scale of the first student network model, and the model scale of the target teacher network model for processing the second task is larger than the scale of the second student network model. In this embodiment, the target teacher network model for processing the first task includes a ResNet-101 network model. The model structure of the ResNet network model is as follows: Figure 3 In this embodiment, the target teacher network model for processing the second task includes a YOLOv5X network model. The model structure of the YOLOv5X network model is as follows: Figure 4 shown.

[0089] like Figure 5 As shown, according to the first data set and the target teacher network model, the target detection model is trained, including:

[0090] S501 , respectively determining a basic model structure of a first student network model and a basic model structure of a second student network model.

[0091] In this embodiment, a basic model structure smaller than the target teacher network model can be selected as the basic model structure of the first student network model and the basic model structure of the second student network model. It should be noted that the basic model structure of the first student network model and the basic model structure of the second student network model can be the same or different.

[0092] In this embodiment, the model structure of the YOLOv5l network model can be selected as the basic model structure of the first student network model and the second student network model.

[0093] S502, based on the first data set, model training is performed on the first student network model and the second student network model respectively, wherein, during the model training process, the student network model is guided to perform model training according to multiple target teacher network models and target loss functions.

[0094] In this embodiment, the training data set and the validation data set divided from the first data set that is the same as the target teacher network are used to complete the model training of the first student network model and the second student network model.

[0095] When multiple teachers guide the training of a student network model, the primary goal is to guide the student network's target loss to as close as possible to the detection limit of the multiple teacher networks for different tasks, without completely negating the student network model's loss contribution during training. In this embodiment, the student network model, as an independent model, is required to continuously align with the recognition check results of the teacher models, without completely deviating from its actual network architecture to train a loss hyperplane out of thin air.

[0096] For example, the YOLOv5L model structure is used as the basic structure of the first student network model. In actual application, the YOLOv5L model uses the cross entropy loss function to calculate the target classification loss. Its model structure is completely different from the teacher network model used to process the first task. The loss function for correcting the target classification in the classification task is .in, and Represent the loss function of the teacher model and the loss function of the student network model respectively, 、 is a hyperparameter, which indicates the influence of the teacher model and the student network model's own loss on the final loss. The final training result is 、 Achieve balance In practice, in order to improve the classification results of the student network model to the classification effect of the teacher model, this embodiment can set the hyperparameter to , .when When the global optimum is reached, the losses of the object detection model and the teacher network model are more similar.

[0097] S503, when the target loss function of the first student network model meets the preset optimal conditions and the target loss function of the second student network model meets the preset optimal conditions, the target detection model is obtained by integrating the first student network model and the second student network model.

[0098] In this embodiment, when the target loss function of the first student network model meets the preset optimal conditions, the recognition result of the first student network model is closest to the recognition result of the target teacher network model for processing the first task. When the target loss function of the second student network model meets the preset optimal conditions, the recognition result of the second student network model is closest to the recognition result of the target teacher network model for processing the second task.

[0099] For example, when the target teacher network model for processing the second task is the YOLOv5X network model, the basic model structure of the second student network model adopts the model structure of the YOLOv5l network model; wherein, the backbone part of the YOLOv5l network model adopts the ResNet18 network structure. The network structure of the YOLOv5l network model is as follows: Figure 6 shown.

[0100] In this embodiment, the YOLOv5l model structure can be used as the foundation for the student network model. Based on the teacher network model, model training is performed on the same training and validation datasets. The YOLOv5l model consists of backbone, neck, and head components. In this embodiment, the backbone component uses the ResNet18 network model structure, which enables faster model analysis and is small enough to be easily embedded in edge computing hosts.

[0101] In this embodiment, the YOLOv5X model structure is used as the target teacher network model for target box regression to guide the loss of the target box regression task of the second student network model for training. The final target box regression loss is recorded as ,in, 、 is a hyperparameter that represents the impact of the teacher model and the student network model's own loss on the final target box regression loss. and Respectively represent the loss function of the teacher model and the loss function of the target box regression of the student network model. In this embodiment, similar to the training classification network model in the previous embodiment, it is possible to set , , in order to improve the training effect of the second student network model, so that the recognition result of the second student network model is closer to the recognition result of the teacher network model processing the second task.

[0102] In one embodiment, obtaining a first data set includes:

[0103] Acquire rail ultrasonic signals collected by a preset channel during a historical detection cycle. The rail ultrasonic signals include the pulse position and depth information of the echo point, as well as the channel name. Historical B-display image data is plotted based on the rail ultrasonic signals. Preprocessing operations are performed on the historical B-display image data to obtain a first data set. The preprocessing operations include image feature optimization and signal noise removal.

[0104] In one embodiment, obtaining the second data set includes:

[0105] Acquire the rail ultrasonic signal collected by the preset channel during the current detection cycle. The rail ultrasonic signal includes the pulse position of the echo point, depth information, and channel name. Render real-time B-display image data based on the rail ultrasonic signal. Perform preprocessing on the real-time B-display image data to obtain a second data set. The preprocessing includes image feature optimization and signal noise removal.

[0106] In this embodiment, the rail ultrasonic signal is ,in, , , Respectively represent the pulse position, depth information and channel name of the current echo point. Among them, a complete rail ultrasonic signal This represents a combination of multiple ultrasound echo data points that change over time. By plotting different channels using different legends, a B-display image for playback can be generated. To conserve computing resources and avoid excessive image length, this embodiment sets the pulse length covered by each B-display image data point to a preset pulse length, for example, 960 pulses.

[0107] In this embodiment, after acquiring B-mode image data, it needs to be preprocessed. This preprocessing operation not only includes increasing the image quantity but also its quality, ensuring that the image features that meet the task requirements are fully expressed. In this embodiment, rail ultrasonic B-mode image preprocessing is performed from two aspects: image feature optimization and signal noise removal.

[0108] In one embodiment, Figure 7 As shown, the image feature optimization processing includes:

[0109] S701, determining each target to be identified in the B-display image data, wherein the target to be identified includes a rail head area, a rail waist area, and a rail bottom area;

[0110] S702: Adjust the background color of the target to be identified to a uniform color according to preset color information, wherein different targets to be identified correspond to different background colors.

[0111] In this embodiment, the B-display image data has a playback function, which can effectively identify each target to be identified in the B-display image data. The background color of the target to be identified is adjusted to a uniform color using preset color information corresponding to the target to be identified, or other information that effectively highlights the location characteristics of the target to be identified, thereby significantly highlighting the location characteristics of different targets to be identified.

[0112] In practical applications, the targets to be identified within B-display images are highly consistent. For example, the echo from a normal weld typically appears in the shape of an inverted "eight," while a normal screw hole appears in the shape of an "A." While object detection models and deep learning networks like classification have significant combined recognition capabilities based on features such as shape, texture, and color, they are insensitive to the target's position and size. Therefore, it is crucial to include positional information reflecting the target in the B-display image.

[0113] In this embodiment, the rail head, rail waist, and rail bottom areas are distinguished by modifying the background color of each target to be identified in the B-display image data, keeping the background information of each image data consistent, thereby making the position information features of the target to be identified more prominent, which can effectively improve the recognition accuracy of the small-scale target detection model deployed on the edge.

[0114] In this embodiment, the RGB values ​​of the rail head area are set to [201, 174, 255], the RGB values ​​of the rail waist area are set to [90, 120, 50], and the RGB values ​​of the rail bottom area are set to [150, 150, 150]. By limiting the position information, the problem of insufficient position information in the B-display image can be compensated.

[0115] like Figure 8 As shown, the signal noise removal process includes:

[0116] S801, converting the B-display image data into a binary image, wherein positions where echo point signals exist in the B-display image data correspond to pixel values ​​1 in the binary image, and other positions in the B-display image data correspond to pixel values ​​0 in the binary image.

[0117] S802 , performing dilation processing on the binary image based on a preset rectangular dilation kernel to obtain a dilated image.

[0118] S803 , performing corrosion processing on the dilated image based on a preset corrosion kernel to obtain denoised B-display image data.

[0119] In practical applications, image processing is used to remove noise from ultrasonic echo signals. Because B-display image data is rendered based on ultrasonic signals, image quality issues such as uneven brightness and blurring, often seen in natural environments, are eliminated. The only factor that affects the data presentation is the generation of useless information from ultrasonic signals during data acquisition, due to noise and other factors.

[0120] In this embodiment, if Figure 9 As shown in Figure 1, scattered noise is removed by dilation-erosion method. Erosion operation is to expand the points with pixel value 0 in the image, and dilation is to expand the points with pixel value 1 in the image.

[0121] In this embodiment, the B-display echo image is first converted into a binary image, that is, the position with the echo point signal is set to 1, and the other positions are set to 0. Since the useful target in the B-display image generally has a tendency to extend obliquely downward, 2 The rectangular expansion kernel of 3 is used to expand the binary image B first. Then the image is expanded with a size of 2 The erosion kernel of 4 performs an erosion operation on the expanded image. Erosion after expansion can remove the holes in the image. The specific operation steps are as follows: Figure 9 As shown in Figure 2, the dilation-erosion denoising results of the image with only single-point noise and the image with the target to be identified are shown. Figure 9 The left side shows the dilation-erosion denoising result including only single point noise. Figure 9 The right side shows the dilation-erosion denoising result of the image including the target to be identified.

[0122] It should be noted that the specific sizes of the rectangular expansion kernel and the erosion kernel can be configured according to the needs of the actual application scenario.

[0123] In summary, the rail damage identification method based on multi-teacher guidance provided in this embodiment can achieve high accuracy and recognition rate by using a model that is as small and simple as possible. The detection upper limit of the target detection model can be confirmed and improved by the teacher network model. The teacher network models of multiple different tasks are used to guide the training of different tasks of the student network model respectively. The method of using the results of the loss function for guidance does not require the student network model and the teacher network model to maintain the same structure, leaving more room for selection and design for the student network model. The trained student network model is small enough that it can be easily and quickly converted into a model and embedded in the relevant host of edge computing to achieve the adaptation of lightweight edge devices.

[0124] In a more detailed embodiment, Figure 10 As shown, the specific steps of the rail damage identification method based on multi-teacher guidance knowledge distillation provided by this embodiment are as follows:

[0125] First, construct a dataset of B-display image data and preprocess the dataset.

[0126] Second, use the compiled dataset to train target teacher network models that excel in each task. This includes a teacher network model focused on object classification and a teacher network model focused on bounding box regression. During the training of the target teacher network models, the performance of multiple teacher network models can be verified and tested, and the optimal target teacher network model or multiple optimal target teacher network models can be selected for subsequent training.

[0127] Third, the student network model is trained using the dataset. During training, the loss function of the teacher network model is used to iteratively guide the student network's loss. For different tasks, different high-performing teacher network models are used to guide the student network's recognition and target box regression based on the loss function, allowing the student network to reach the global optimal solution. The final recognition results continuously approach the teacher network's recognition results for each task, ultimately completing the training of the student network model. The recognition and target regression results of the student network model are then verified.

[0128] Fourth, in the actual data, after detecting a new sample, the rail ultrasonic B-display image is preprocessed in the same way as in the first step and then sent to the student network model (target detection model) for positioning and identification, thereby realizing rail ultrasonic damage detection.

[0129] It should be understood that, although the various steps in the flowcharts involved in the various embodiments described above are displayed in sequence according to the instructions of the arrows, these steps are not necessarily executed in sequence in the order indicated by the arrows. Unless otherwise specified herein, there is no strict order restriction on the execution of these steps, and these steps can be executed in other orders. Moreover, at least a portion of the steps in the flowcharts involved in the various embodiments described above can include multiple steps or multiple stages, and these steps or stages are not necessarily executed and completed at the same time, but can be executed at different times, and the execution order of these steps or stages is not necessarily to be carried out in sequence, but can be executed in turn or alternately with other steps or at least a portion of steps or stages in other steps.

[0130] Based on the same inventive concept, the embodiments of the present application also provide a rail damage identification device based on multi-teacher guidance for implementing the aforementioned rail damage identification method based on multi-teacher guidance. The implementation solution provided by this device is similar to the implementation solution described in the aforementioned method. Therefore, the specific limitations of one or more embodiments of the rail damage identification device based on multi-teacher guidance provided below can be found in the limitations of the rail damage identification method based on multi-teacher guidance above, and will not be repeated here.

[0131] In one embodiment, Figure 11 As shown, a rail damage identification device 1100 based on multi-teacher guidance is provided, comprising: a first acquisition module 1110, a first training module 1120, a second training module 1130, a second acquisition module 1140 and a data processing module 1150, wherein:

[0132] A first acquisition module 1110 is configured to acquire a first data set, wherein the first data set includes historical B-display image data during the rail ultrasonic flaw detection process;

[0133] A first training module 1120 is configured to train multiple target teacher network models based on the first data set, wherein at least one target teacher network model for processing the first task is trained, and at least one target teacher network model for processing the second task is trained;

[0134] A second training module 1130 is configured to train a target detection model based on the first data set and the target teacher network model, wherein the target detection model includes a first student network model for processing the first task and a second student network model for processing the second task;

[0135] A second acquisition module 1140 is configured to acquire a second data set, wherein the second data set includes real-time B-display image data during the rail ultrasonic flaw detection process;

[0136] The data processing module 1150 is used to process the second data set according to the target detection model to obtain a rail damage identification result.

[0137] In one embodiment, the first training module 1120 is specifically used to determine the basic model structure of the first student network model and the basic model structure of the second student network model, respectively; based on the first data set, the first student network model and the second student network model are respectively trained, wherein, during the model training process, the student network model is guided to perform model training according to multiple target teacher network models and target loss functions; when the target loss function of the first student network model meets the preset optimal conditions and the target loss function of the second student network model meets the preset optimal conditions, the target detection model is obtained by combining the first student network model and the second student network model.

[0138] In one embodiment, the first acquisition module 1110 is specifically used to obtain the rail ultrasonic signal collected by a preset channel in a historical detection period, wherein the rail ultrasonic signal includes the pulse position, depth information and channel name of the echo point; historical B-display image data is obtained based on the rail ultrasonic signal; and the historical B-display image data is preprocessed to obtain a first data set, wherein the preprocessing operation includes image feature optimization processing and signal noise removal processing.

[0139] In one embodiment, the second acquisition module 1140 is specifically used to obtain the rail ultrasonic signal collected by the preset channel in the current detection cycle, wherein the rail ultrasonic signal includes the pulse position, depth information and channel name of the echo point; obtain real-time B-display image data based on the rail ultrasonic signal; perform preprocessing operations on the real-time B-display image data to obtain a second data set, wherein the preprocessing operations include image feature optimization processing and signal noise removal processing.

[0140] Each module in the multi-teacher-guided rail damage identification device can be implemented in whole or in part through software, hardware, or a combination thereof. Each module can be embedded in or independent of a processor in a computer device in hardware form, or stored in a computer device memory in software form, so that the processor can call and execute the corresponding operations of each module.

[0141] As shown in one embodiment, a computer device is provided. The computer device may be a terminal, and its internal structure may be as shown in FIG. Figure 12As shown. The computer device includes a processor, memory, an input / output interface, a communication interface, a display unit, and an input device. The processor, memory, and input / output interface are connected via a system bus, and the communication interface, display unit, and input device are connected to the system bus via the input / output interface. The processor of the computer device provides computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and internal memory. The non-volatile storage medium stores an operating system and computer programs. The internal memory provides an environment for the operation of the operating system and computer programs in the non-volatile storage medium. The input / output interface of the computer device is used to exchange information between the processor and external devices. The communication interface of the computer device is used to communicate with external terminals via wired or wireless means, and the wireless means can be implemented via Wi-Fi, mobile cellular networks, NFC (near-field communication), or other technologies. When executed by the processor, the computer program implements a rail damage identification method based on multi-teacher guidance. The display unit of the computer device is used to produce a visually visible image and can be a display screen, a projection device, or a virtual reality imaging device. The display screen can be a liquid crystal display screen or an electronic ink display screen, and the input device of the computer device can be a touch layer covering the display screen, or a button, trackball or touchpad set on the computer device casing, or an external keyboard, touchpad or mouse.

[0142] Those skilled in the art will understand that Figure 12 The structure shown in the figure is only a block diagram of a part of the structure related to the solution of the present application, and does not constitute a limitation on the computer device to which the solution of the present application is applied. The specific computer device may include more or fewer components than shown in the figure, or combine certain components, or have a different component arrangement.

[0143] In one embodiment, a computer device is provided, including a memory and a processor, wherein a computer program is stored in the memory, and when the processor executes the computer program, the following steps are implemented:

[0144] Acquire a first data set, wherein the first data set includes historical B-display image data during a rail ultrasonic flaw detection process;

[0145] A plurality of target teacher network models are trained according to the first data set, wherein at least one target teacher network model for processing the first task is trained, and at least one target teacher network model for processing the second task is trained;

[0146] Training a target detection model based on the first data set and the target teacher network model, wherein the target detection model includes a first student network model for processing the first task and a second student network model for processing the second task;

[0147] Acquiring a second data set, wherein the second data set includes real-time B-display image data during the rail ultrasonic flaw detection process;

[0148] The second data set is processed according to the target detection model to obtain the rail damage identification result.

[0149] In one embodiment, a computer-readable storage medium is provided, on which a computer program is stored. When the computer program is executed by a processor, the following steps are implemented:

[0150] Acquire a first data set, wherein the first data set includes historical B-display image data during a rail ultrasonic flaw detection process;

[0151] A plurality of target teacher network models are trained according to the first data set, wherein at least one target teacher network model for processing the first task is trained, and at least one target teacher network model for processing the second task is trained;

[0152] Training a target detection model based on the first data set and the target teacher network model, wherein the target detection model includes a first student network model for processing the first task and a second student network model for processing the second task;

[0153] Acquiring a second data set, wherein the second data set includes real-time B-display image data during the rail ultrasonic flaw detection process;

[0154] The second data set is processed according to the target detection model to obtain the rail damage identification result.

[0155] In one embodiment, a computer program product is provided, comprising a computer program, which, when executed by a processor, implements the following steps:

[0156] Acquire a first data set, wherein the first data set includes historical B-display image data during a rail ultrasonic flaw detection process;

[0157] A plurality of target teacher network models are trained according to the first data set, wherein at least one target teacher network model for processing the first task is trained, and at least one target teacher network model for processing the second task is trained;

[0158] Training a target detection model based on the first data set and the target teacher network model, wherein the target detection model includes a first student network model for processing the first task and a second student network model for processing the second task;

[0159] Acquiring a second data set, wherein the second data set includes real-time B-display image data during the rail ultrasonic flaw detection process;

[0160] The second data set is processed according to the target detection model to obtain the rail damage identification result.

[0161] Those skilled in the art will appreciate that all or part of the processes in the above-mentioned embodiments can be implemented by instructing the relevant hardware through a computer program. The computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the above-mentioned embodiments. In particular, any reference to memory, database, or other media used in the embodiments provided in this application can include at least one of non-volatile and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetic random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory can include random access memory (RAM) or external cache memory, etc. By way of illustration and not limitation, RAM can take various forms, such as static random access memory (SRAM) or dynamic random access memory (DRAM). The databases involved in the various embodiments provided herein may include at least one of a relational database and a non-relational database. Non-relational databases may include, but are not limited to, distributed databases based on blockchains. The processors involved in the various embodiments provided herein may be, but are not limited to, general-purpose processors, central processing units (CPUs), graphics processing units (GPUs), digital signal processors (DSPs), programmable logic devices (PLDs), data processing logic devices based on quantum computing, and the like.

[0162] The technical features of the above embodiments can be combined arbitrarily. To make the description concise, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0163] The above-described embodiments merely represent several implementation methods of the present application. While the descriptions are relatively specific and detailed, they should not be construed as limiting the scope of the present application. It should be noted that a person of ordinary skill in the art may make various modifications and improvements without departing from the spirit of the present application, and these modifications and improvements fall within the scope of protection of the present application. Therefore, the scope of protection of the present application shall be determined by the appended claims.

Claims

1. A rail damage identification method based on multi-teacher guidance, characterized in that: The method comprises: Acquire a first data set, wherein the first data set includes historical B-display image data during a rail ultrasonic flaw detection process; Training multiple target teacher network models based on the first data set, wherein at least one target teacher network model for processing the first task is trained, and at least one target teacher network model for processing the second task is trained; Training a target detection model based on the first data set and the target teacher network model, wherein the target detection model includes a first student network model for processing the first task and a second student network model for processing the second task; Acquiring a second data set, wherein the second data set includes real-time B-display image data during the rail ultrasonic flaw detection process; The second data set is processed according to the target detection model to obtain a rail damage identification result.

2. The method according to claim 1, characterized in that The first task includes a classification task, the second task includes a target box regression task, the target teacher network model for processing the first task includes a ResNet-101 network model, and the target teacher network model for processing the second task includes a YOLOv5X network model; the model scale of the target teacher network model for processing the first task is larger than the scale of the first student network model, and the model scale of the target teacher network model for processing the second task is larger than the scale of the second student network model; The training of a target detection model based on the first data set and the target teacher network model includes: respectively determining a basic model structure of the first student network model and a basic model structure of the second student network model; Performing model training on the first student network model and the second student network model based on the first data set, wherein, during the model training process, the student network model is guided to perform model training according to multiple target teacher network models and target loss functions; When the target loss function of the first student network model meets the preset optimal conditions and the target loss function of the second student network model meets the preset optimal conditions, the target detection model is obtained by combining the first student network model and the second student network model.

3. The method according to claim 2, characterized in that When the target teacher network model for processing the second task is the YOLOv5X network model, the basic model structure of the second student network model adopts the model structure of the YOLOv5l network model; wherein the backbone part of the YOLOv5l network model adopts the ResNet18 network structure.

4. The method according to claim 1, wherein The obtaining of the first data set includes: Acquire a rail ultrasonic signal collected by a preset channel during a historical detection period, wherein the rail ultrasonic signal includes a pulse position of an echo point, depth information, and a channel name; Obtaining the historical B-display image data based on the rail ultrasonic signal; A preprocessing operation is performed on the historical B-display image data to obtain the first data set, wherein the preprocessing operation includes image feature optimization processing and signal noise removal processing.

5. The method according to claim 1, wherein The obtaining of the second data set includes: Acquire the rail ultrasonic signal collected by the preset channel in the current detection cycle, wherein the rail ultrasonic signal includes the pulse position, depth information and channel name of the echo point; Obtaining the real-time B-display image data based on the rail ultrasonic signal; A preprocessing operation is performed on the real-time B-display image data to obtain the second data set, wherein the preprocessing operation includes image feature optimization processing and signal noise removal processing.

6. The method according to any one of claims 4 or 5, characterized in that The image feature optimization processing includes: Determining each target to be identified in the B-display image data, wherein the target to be identified includes a rail head area, a rail waist area, and a rail bottom area; Adjusting the background color of the target to be identified to a uniform color according to preset color information, wherein different targets to be identified have different background colors; The signal noise removal process includes: Converting the B-display image data into a binary image, wherein positions where echo point signals exist in the B-display image data correspond to pixel values ​​1 in the binary image, and other positions in the B-display image data correspond to pixel values ​​0 in the binary image; Performing dilation processing on the binary image based on a preset rectangular dilation kernel to obtain a dilated image; The dilated image is corroded based on a preset corrosion kernel to obtain denoised B-display image data.

7. A rail damage identification device based on multi-teacher guidance, characterized in that: The device comprises: A first acquisition module is configured to acquire a first data set, wherein the first data set includes historical B-display image data during a rail ultrasonic flaw detection process; A first training module is configured to train multiple target teacher network models based on the first data set, wherein at least one target teacher network model for processing the first task is trained, and at least one target teacher network model for processing the second task is trained; a second training module, configured to train a target detection model based on the first data set and the target teacher network model, wherein the target detection model includes a first student network model for processing the first task and a second student network model for processing the second task; A second acquisition module is configured to acquire a second data set, wherein the second data set includes real-time B-display image data during the rail ultrasonic flaw detection process; A data processing module is used to process the second data set according to the target detection model to obtain a rail damage identification result.

8. A computer device comprising a memory and a processor, wherein the memory stores a computer program, wherein: When the processor executes the computer program, the steps of the rail damage identification method based on multi-teacher guidance according to any one of claims 1 to 6 are implemented.

9. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the rail damage identification method based on multi-teacher guidance according to any one of claims 1 to 6 are implemented.

10. A computer program product comprising a computer program, characterized in that When the computer program is executed by a processor, the steps of the rail damage identification method based on multi-teacher guidance according to any one of claims 1 to 6 are implemented.

Images