Intelligent railway track crack defect detection method based on LMSA
By introducing the LMSA module and CBAM attention mechanism in the YOLOv10 model, the problems of slow detection rate and insufficient accuracy in the prior art are solved, and efficient and intelligent detection of railway track crack defects are achieved.
Patent Information
- Application Number
- CN202510211656.6
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2025-02-25
- Publication Date
- 2025-06-10
AI Technical Summary
The existing railway track fault detection methods have a slow detection rate, making it difficult to achieve accurate detection of fault areas.
The improved YOLOv10 model based on LMSA is adopted, and the C2F module is replaced as the LMSA module in the YOLOv10 backbone network and the CBAM attention mechanism is introduced in Neck to improve the model's capture ability and detection efficiency of crack features.
It significantly improves detection speed and accurate detection capabilities of faulty areas, reduces attention to non-crack areas, and improves the generalization ability and real-time detection performance of the model.
Smart Images
Figure CN120126090A_ABST
Abstract
Description
Technical Field
[0001] The present invention belongs to the technical direction of object detection in the field of computer vision, and particularly relates to an intelligent detection method for railway track crack defects based on LMSA. Background Technique
[0002] As an important means of transportation, the safety performance of railways is directly related to the safety of passengers and goods. Railway track failures, such as deformation, dislocation, missing fasteners, etc., may cause serious accidents such as train derailment and overturning. Therefore, timely detection and repair of these failures can greatly reduce the likelihood of accidents and ensure the safety of railway transportation. During the long-term use of railway track equipment, various failures will occur due to wear, aging, etc. Timely detection and repair of these failures can prevent further damage to the equipment, extend the service life of the equipment, and reduce maintenance costs. Compared with other means of transportation, such as cars and airplanes, railway transportation has lower carbon emissions. However, railway track failures may cause unnecessary noise and vibration during train operation, polluting the surrounding environment. Through fault detection, this unnecessary pollution can be reduced, which is beneficial to environmental protection. Summary of the Invention
[0003] Aiming at the problems pointed out in the background technique, the present invention provides an intelligent detection method for railway track crack defects based on LMSA, which solves problems such as slow current detection rate. The present invention can ensure the improvement of detection speed and the accurate detection of the fault area.
[0004] Technical Solution: The present invention discloses an intelligent detection method for railway track crack defects based on LMSA, including the following steps:
[0005] Step 1: Construct a railway track fault dataset, preprocess these data, and increase data diversity by using image rotation, brightness enhancement, and Gaussian blur methods. Perform manual annotation to clarify the fault type or status of the samples. Finally, divide the entire dataset into a training set, a validation set, and a test set for subsequent model training and evaluation;
[0006] Step 2: Build an improved YOLOv10 model framework. Replace the C2F modules in the second, fourth, sixth, and eighth layers of the YOLOv10 backbone network with the LMSA module, and introduce the CBAM attention mechanism in the thirteenth, sixteenth, and nineteenth layers. Selecting these specific layers for replacement (i.e., the 2nd, 4th, 6th, and 8th layers) can achieve an effect of gradually enhancing feature extraction. These layers are in the middle and early stages of the backbone network and are responsible for extracting basic underlying information such as edges, shapes, and textures, which are exactly the key features of cracks. Incorporating the LMSA module into these layers can gradually enhance the model's capture of crack features, ensuring that important details are not lost in subsequent layers. After adding spatial attention in the Neck (i.e., the 13th, 16th, and 19th layers), the model can focus on the area where the cracks are located and reduce the attention to other non-crack areas, which is particularly important for tasks such as crack detection that require precise positioning. Spatial attention enables the model to better exclude interference information, such as background information like stones and debris around the tracks;
[0007] Step 3: Train the improved YOLOv10 model using the training set of the railway track fault dataset.
[0008] Furthermore, the LMSA module described in step 2 has characteristics such as reduced parameters and computational complexity, enhanced feature representation ability, and improved flexibility and scalability. These characteristics enable the fused model to further improve its performance in various tasks while maintaining computational efficiency; the LMSA module is an efficient lightweight network that can reduce redundant computations while maintaining a high feature extraction ability. It has significant advantages in terms of computational complexity and the number of parameters, making it suitable for embedded or resource-constrained environments. The train track crack detection task is usually deployed in actual monitoring devices and needs to process a large amount of real-time data. The lightweight characteristics of the LMSA module make it more suitable for such tasks than C2F. Train track cracks can be very subtle and easily interfered with by other background information. In this case, the LMSA module can help the model focus on the crack area and texture information on the feature map, enabling the model to focus more on the key features of the crack area. In contrast, the C2F structure does not include such an attention module and cannot guide the model to selectively capture the details of the crack area, making it difficult to detect crack details. Train track crack detection usually requires high real-time performance, and the LMSA module has more advantages in terms of inference speed. The fully connected layer of the C2F structure requires more computational resources, while the LMSA module can help the model automatically focus on the crack area at a relatively low computational cost during the feature extraction stage, thereby improving the detection efficiency and making it more suitable for applications that require high inference speed. Train track crack data is usually difficult to obtain, and limited data volume may lead to overfitting of the model. The LMSA module can effectively improve the generalization ability of the model. Moreover, the features generated by the LMSA module have redundancy and rich expressive ability, can adaptively focus on important features, and reduce the sensitivity of the model to noise. In contrast, the C2F structure is more prone to overfitting when trained on limited data and is not suitable for obtaining robust detection results on small samples;
[0009] Furthermore, inside the LMSA (Light Multi-Scale Attention) module, the processing of the image aims to efficiently extract and enhance key features for subsequent train track crack detection tasks.
[0010] The input feature map first undergoes a 1×1 dynamic depthwise separable convolution (Dynamic DS Conv) operation. The core purpose of this step is to extract key features and reduce the number of channels. The dynamic depthwise separable convolution can integrate information from different channels without changing the spatial dimensions (i.e., width and height) of the feature map and effectively reduce the number of channels, thereby reducing the subsequent computational burden and improving the overall processing efficiency.
[0011] The original feature map processed by the dynamic depthwise separable convolution then enters a lightweight deepening convolution stage, which transforms each channel to generate redundant residual feature maps, i.e., preliminary feature maps. This step can calculate more features in a short time, capture deeper feature information through deepening convolution, and form a redundant but rich feature representation. This redundancy helps the model better adapt and generalize in subsequent processing.
[0012] Next, the original feature map is concatenated with the preliminary feature map after convolution processing. The purpose of concatenation is to combine the core features extracted initially with the information of the original feature map to retain as much useful information as possible. In this way, the complete output feature map after concatenation not only contains the spatial information of the original feature map but also integrates the deep feature information after preliminary processing. These comprehensive features will be used as the input of the subsequent network layers and continue to be transmitted and processed in the network.
[0013] Furthermore, a CBAM attention module is set after the feature concatenation in the LMSA module, which can not only retain its advantage in computational efficiency but also use the attention mechanism of CBAM to enhance the model's ability to capture important information, thus showing better performance in various tasks.
[0014] Furthermore, the image input first passes through a basic convolution module Conv of 3×3 in the Backbone, then continues to deeply extract image features through two groups of basic convolutions and LMSA modules in sequence, then passes through a sequence composed of two groups of SCDown and LMSA modules to effectively compress feature information and strengthen key features, then passes through an SPPF module, and finally passes through a PSA partial self-attention module to provide a basis for subsequent tasks.
[0015] Furthermore, the process of SCDown convolution includes adjusting the dimension on the channel through a 1×1 point convolution to adjust the number of channels of the input image, allowing the model to perform information interaction and feature fusion between supplementary channels at the same time, and then using a 3×3 depth convolution for spatial downsampling. Its convolution process independently applies a convolution kernel to each input channel, reducing its height and width while keeping the channels unchanged. After the output is processed by SPPF for feature extraction and enhancement, and after the PSA processing, it is output to the Head part of the model.
[0016] Furthermore, it includes a data preprocessing part, an improved YOLOv10 model module, and a training and detection module;
[0017] The data preprocessing module constructs a railway track fault dataset, and uses image enhancement techniques (such as image rotation, brightness enhancement, and Gaussian blur, etc.) to increase the diversity of samples. At the same time, a data filtering mechanism is applied to eliminate noise or abnormal data. After manual annotation, the training set and the test set are divided to train the model.
[0018] For the improved YOLOv10 model, some C2F modules in the Backbone backbone network are replaced with LMSA modules. Among them, LMSA is a Ghost module fused with the CBAM attention mechanism to improve the ability and efficiency of object detection in the backbone network and make the model lighter. At the same time, the CBAM attention mechanism is incorporated into the Neck. After adding spatial attention to the Neck, the model can focus on the area where the crack is located and reduce the attention to other non-crack areas, which is particularly important for tasks such as crack detection that require precise positioning. Spatial attention enables the model to better exclude interference information, such as background information like stones and debris around the track.
[0019] The training and detection module uses the training set to train the improved YOLOv10 model and validates its effect with the test set.
[0020] Beneficial effects:
[0021] Based on the YOLOv10 as the main model, this invention is improved. Compared with other object detection models, it has a more advanced basic network architecture and stronger feature extraction and fusion capabilities, and is more suitable for the needs of railway track fault object detection in practical applications. Based on YOLOv10, first, the Ghost module and the CBAM module are fused to construct the LMSA module, and then the LMSA module is used to replace the C2F module in the backbone network of YOLOv10, enabling it to have characteristics such as reduced parameters and computational complexity, improved feature representation ability, and enhanced flexibility and scalability. These characteristics enable the fused model to further improve its performance in various tasks while maintaining computational efficiency. Incorporating the CBAM attention mechanism into some C2F modules in the Neck increases the self-adaptability of the model, which can adaptively adjust the feature weights of different channels and spatial positions, making this improved model more flexible and robust when dealing with different scenarios and complex objects, and significantly improving the representation ability of object detection. Description of the Drawings
[0022] Figure 1 is the overall flowchart provided by the present invention;
[0023] Figure 2 is the structural schematic diagram of the improved YOLOv10 provided by the present invention;
[0024] Figure 3It is a schematic diagram of the LMSA module provided by the present invention;
[0025] Figure 4 It is a comparison of the effect diagrams of the YOLOv10 and the improved YOLOv10 algorithm models;
[0026] Figure 5 It is the actual detection effect diagram of the present invention. Specific implementation manners
[0027] The present invention will be further clarified below in conjunction with the accompanying drawings and specific embodiments.
[0028] The present invention provides an intelligent detection method for railway track crack defects based on LMSA. The specific process is as Figures 1-5 shown, including the following steps:
[0029] Step 1: Construct a railway track fault data set, perform data preprocessing and data augmentation on it, and divide it into a training set, a validation set and a test set after manual annotation. The specific method is as follows:
[0030] Step 1.1: Construct a railway track fault data set;
[0031] Step 1.2: Collect fault pictures such as track geometric deformation, turnout faults, and sleeper damage through web crawler technology;
[0032] Step 1.3: Use methods such as hash value comparison, global feature extraction, and feature vector comparison to clean the data and remove duplicate and damaged data pictures;
[0033] Step 1.4: Denoise the images using methods such as median filtering and Gaussian filtering;
[0034] Step 1.5: For missing areas or occluded parts in the pictures, image inpainting algorithms (such as content-based image inpainting, deep learning image inpainting, etc.) can be used for filling;
[0035] Step 1.6: Perform data augmentation on the sample data, which can increase the diversity and generalization ability of the data set, thereby improving the robustness of the model. Operations such as rotating, translating, scaling, and flipping the pictures are adopted; color attributes such as brightness, contrast, and saturation of the pictures are adjusted; random noise is added to the pictures to simulate noise interference in the real scene. To generate more training samples;
[0036] Step 1.7: Manually annotate the data set using Labelme and divide the training set and the test set according to 8:2.
[0037] Step 2: Build an improved YOLOv10 model framework and construct a Ghost module.
[0038] Step 3: Incorporate the CBAM attention mechanism, combine it with the Ghost module described in Step 2, and construct a new module, the LMSA module. The LMSA module is shown as follows. The specific method is as follows:
[0039] Step 3.1: Embed the CBAM module into the output part of the LMSA module. That is, first perform dynamic depthwise separable convolution on the input feature map to generate the original feature map, then perform depthwise convolution to generate the redundant feature map, i.e., the preliminary feature map. After feature concatenation, add the CBAM attention mechanism to enhance the feature expression ability of the feature map using the attention mechanism. This can make the generated redundant features more focused on the key features. Among them, CBAM is an attention mechanism that combines channel attention and spatial attention, and its core formula is as follows:
[0040] Channel attention: Calculate the importance of each channel through global pooling. Use two operations, global average pooling and global max pooling, and then generate an attention map through a fully connected layer. The formula is as follows:
[0041] M c (F) = σ(MLP(AvgPool(F)) + MLP(MaxPool(F)))
[0042] F is the feature map input by the user.
[0043] AvgPool(F) and MaxPool(F) are the operations of global average pooling and max pooling on the input feature map respectively.
[0044] MLP is a two-layer fully connected neural network.
[0045] σ is the Sigmoid function, which is used to generate attention weights.
[0046] M c is the channel attention map, which will be used to weight each channel of the input feature map.
[0047] Spatial attention: Fuse features in the channel dimension through average pooling and max pooling, and then generate a spatial attention map through a convolution operation. The formula is as follows:
[0048] M s (F) = σ(f 7×7 ([AvgPool(F); MaxPool(F)]))
[0049] [AvgPool(F); MaxPool(F)] is the concatenation of the results of global average pooling and max pooling in the channel dimension.
[0050] f 7×7It is a 7x7 convolutional operation used to generate features in the spatial dimension.
[0051] σ is still the Sigmoid function, which generates a spatial attention map.
[0052] M s is the spatial attention map, which is used to adjust the spatial dimension of the feature map.
[0053] The final feature map is weighted by channel and spatial attention to obtain an enhanced feature map:
[0054] F′ = M S (M C (F) ⊙ F) ⊙ F
[0055] M S and M C are the channel attention map and the spatial attention map respectively.
[0056] ⊙ represents element-wise multiplication.
[0057] F′ is the output feature map enhanced by channel and spatial attention.
[0058] CBAM adjusts the weights of different channels and spatial positions in the feature map by calculating attention weights, rather than downsampling or upsampling the feature map. Therefore, after being processed by CBAM, the width, height, and number of channels of the input feature map remain the same as those of the output feature map.
[0059] Step 3.2: The input feature map in the LMSA module first passes through a 1x1 dynamic depthwise separable convolution to extract core features and reduce the number of channels, thereby reducing the computational load. Secondly, these final feature maps pass through depthwise convolution to generate redundant residual feature maps, and more features are calculated in a short time. Finally, the original feature map and the preliminary feature map are concatenated to obtain the complete output features, and the output feature map will be used as the input of the subsequent layer and continue to be transmitted and processed in the network.
[0060] Step 4: Replace the C2F module in the YOLOv10 backbone network with the LMSA module:
[0061] Step 4.1: After the image data is input, the image first passes through a basic convolution module Conv of 3×3 in the Backbone, and then continues to deeply extract image features through two groups of basic convolutions and the LMSA module in sequence. Then, it passes through a sequence composed of two groups of SCDown and the LMSA module to effectively compress the feature information and strengthen the key features. Then, it passes through an SPPF module, and finally, through a PSA partial self-attention module to provide a basis for subsequent tasks.
[0062] Step 4.2: The process of SCDown convolution includes adjusting the dimension in the channel. First, a 1×1 point convolution is used to adjust the number of channels of the input image, allowing the model to perform information interaction and feature fusion among different channels. Then, a 3×3 depth convolution is used for spatial downsampling. The convolution process applies a convolution kernel independently to each input channel, reducing its height and width while keeping the number of channels unchanged.
[0063] Step 5: Introduce the CBAM attention mechanism in the C2F module in the Neck. The input feature map first passes through a standard 3x3 convolutional layer to extract preliminary features. The C2F module divides the input feature map into two parts through cross-stage feature reuse. One part extracts deep features through multiple convolutions, and the other part retains the original features and performs shallow convolutions. This partial reuse strategy helps reduce redundant calculations and improve the efficiency of the network. The CBAM attention mechanism processes the output feature map of the C2F module. That is, after the cross-stage processing of the C2F module, the feature map is passed to the CBAM module for attention weighting in the channel and spatial dimensions. Finally, the feature map processed by the C2F module and the CBAM attention mechanism is passed to the next layer of the network for subsequent tasks.
[0064] Step 6: Use the improved YOLOv10 model for training to obtain the detection results. The specific method is as follows:
[0065] Step 6.1: Put the enhanced dataset into the improved model for training to obtain the optimal weight file.
[0066] Step 6.2: Use the optimal weights for validation and adjust the specific parameters.
[0067] Step 6.3: Use the optimal weights for data detection to obtain the detection results.
[0068] In the implementation of the present invention, the evaluation indicators used are accuracy (Accuracy), precision (Precision), and recall (Recall). The formulas for each evaluation indicator are as follows:
[0069] Accuracy (A):
[0070]
[0071] Precision (P):
[0072]
[0073] Recall (R):
[0074]
[0075] Among them, TP refers to the number of samples that are positive and predicted as positive, FP refers to the number of samples that are negative and predicted as positive, FN refers to the number of samples that are positive and predicted as negative, and TN refers to the number of samples that are negative and predicted as positive.
[0076] The experimental environment parameters are shown in Table 1 below.
[0077] Table 1:
[0078] Environmental parameters Operating system Win11 CPU i5-12400f GPU NVIDIA GeForce RTX 4060 GPU RAM 16G ROM 1T Python 3.9 Pythorch 2.3.0
[0079] The experimental results are shown in the figure. Figure 4 It is a comparison diagram of the YOLOv10 and the improved YOLOv10 algorithm models. Among them, P is the precision and R is the recall rate. Figure 4 (a) is the PR graph obtained by training the improved YOLOv10. Figure 4 (b) is the PR graph obtained by training the improved YOLOv10, where P is the precision and R is the recall rate.
[0080] As mentioned above, it is not a restriction on the present invention in any form. According to the technical essence of the present invention, within the spirit and principle of the present invention, any simple modification, equivalent replacement, and improvement of the above embodiments are still within the protection scope of the technical solution of the present invention. Specific examples are used in this article to elaborate on the principle and implementation manner of the present invention. The description of the above embodiments is only used to help understand the method and its core idea of the present invention; at the same time, for those of ordinary skill in the art, there will be changes in the specific implementation manner and application scope according to the idea of the present invention. In summary, the content of this specification should not be construed as a limitation on the present invention.
Claims
1. A railway track crack defect intelligent detection method based on LMSA, characterized in that: The steps include: Step 1: Construct a railway track fault dataset and divide the entire dataset into a training set, a validation set, and a test set; Step 2: Build an improved YOLOv10 model framework, use the LMSA module to replace the C2F modules in the second, fourth, sixth, and eighth layers of the YOLOv10 backbone network, and introduce the CBAM attention mechanism in the thirteenth, sixteenth, and nineteenth layers. The LMSA module is in the Ghost model. The standard convolution of the main convolution is replaced by a dynamic depth-separable convolution, and a CBAM attention module is set after the image splicing output. Among them, the convolution kernels of the three convolutions after the dynamic depth-separable convolution are inserted with zero values; Step 3: Use the training set of the railway track fault dataset to train the improved YOLOv10 model to obtain a trained model.
2. According to claim 1, a railway track crack defect intelligent detection method based on LMSA is characterized in that: Inside the LMSA module, the input feature map is first transformed through a 1×1 dynamic depthwise separable convolution to obtain the original feature map. The original feature map is transformed through each channel in the lightweight depthwise convolution to generate more redundant feature maps, namely the preliminary feature maps. The original feature map and the generated preliminary feature map are concatenated in the channel dimension to obtain a new feature map output.
3. According to claim 2, a railway track crack defect intelligent detection method based on LMSA is characterized in that: The LMSA module is also provided with a CBAM attention module after the feature concatenation is output.
4. The method for intelligent detection of railway track crack defects based on LMSA according to claim 1 is characterized in that: When the image is input into Backbone, it first passes through a 3×3 basic convolution module Conv, and then passes through two groups of basic convolution and LMSA modules to further extract image features. It then passes through a sequence composed of two groups of SCDown and LMSA modules to effectively compress feature information and strengthen key features, and then passes through an SPPF module. Finally, it passes through a PSA partial self-attention module to provide a basis for subsequent tasks.
5. The method for intelligent detection of railway track crack defects based on LMSA according to claim 4 is characterized in that: The SCDown convolution process includes adjusting the dimension on the channel. The number of channels of the input image is adjusted through 1×1 point convolution, while allowing the model to interact and fuse features between subsidized channels. Then, a 3×3 depth convolution is used for spatial downsampling. The convolution process applies a convolution kernel to each input channel independently, reducing its height and width while keeping the channel unchanged. The output is output to the Head part of the model after feature extraction and enhancement by SPPF and processing by PSA.
6. The method for intelligent detection of railway track crack defects based on LMSA according to claim 1, characterized in that: Including data preprocessing, improving YOLOv10 model module, training detection module; The data preprocessing module constructs a railway track fault data set, and uses image enhancement technology to increase the diversity of samples, while applying a data filtering mechanism to remove noise or abnormal data, and divides the data into training sets and test sets after manual annotation for model training; The improved YOLOv10 model replaces some C2F modules in the Backbone network with LMSA modules, where LMSA is a Ghost module fused with the CBAM attention mechanism to improve the ability and efficiency of target detection in the backbone network and make the model lighter. At the same time, the CBAM attention mechanism is integrated into the Neck. The spatial attention mechanism of CBAM weights the feature map in the spatial dimension to help the model locate the target area. After adding spatial attention to the Neck, the model can focus on the area where the crack is located and reduce attention to other non-crack areas. The training detection module uses the training set to train the improved YOLOv10 model and uses the test set to verify its effect.
7. The method for intelligent detection of railway track crack defects based on LMSA according to claim 1 is characterized in that: The evaluation indicators use accuracy, precision and recall as evaluation indicators. The formulas of each evaluation indicator are as follows: Accuracy: Accuracy: Recall: Among them, TP refers to the number of samples that are positive and predicted to be positive, FP refers to the number of samples that are negative and predicted to be positive, FN refers to the number of samples that are positive and predicted to be negative, and TN refers to the number of samples that are negative and predicted to be positive.