Deep learning-based railway wagon foreground and background side frame axle box spring loss fault identification method

Abstract

The invention discloses a deep learning-based railway wagon foreground and background side frame axle box spring loss fault identification method, and relates to the technical field of image processing. The invention aims to solve the problem that the detection precision of a deep learning network is reduced due to the fact that missing detection or false detection is easy to occur when the loss of the foreground and background side frame axle box spring is detected by adopting the existing image processing and deep learning technology. The invention discloses a deep learning-based railway wagon foreground and background side frame axle box spring loss fault identification method, which comprises the following steps of: inputting a detected image into a YOLOv5s network to obtain an identification result; according to the YOLOv5s network, a SimAM parameter-free attention mechanism is embedded at the tail end of each CSP1 unit of a Backbone part and after Focus slicing operation, an AFF attention feature fusion module is adopted in a Neck part to replace a Concat module, and the positioning loss function of the YOLOv5s network is SIoU.

Description

Technical Field

[0001] The invention belongs to the technical field of image processing, and in particular relates to the fault identification of axle box spring loss on front and rear side frames of railway freight cars. Background Art

[0002] The front and rear side frame axle box springs are very important components of the train. They play an important role in the suspension system, mainly in the following aspects:

[0003] There are all kinds of unevenness and bumps on the track, and springs can play a shock-absorbing and buffering role, reducing the impact and vibration caused by uneven tracks during the vehicle's driving process, and improving the ride comfort. Springs also bear the role of supporting the vehicle's own weight. They help to disperse and bear the weight of the vehicle to ensure that the vehicle remains stable and balanced during driving. Springs also help maintain appropriate contact pressure between wheels and rails, which helps to ensure that the wheels fit the track closely during driving and improve traction and braking performance. By designing different types of springs, the stiffness of the train's suspension system can be adjusted to adapt to different operating conditions and loads. In general, the front and rear side frame axle box springs play an important role in the train's suspension system, and have an important impact on ensuring the smooth driving of the train, ride comfort and traction performance. However, during long-term operation, some unexpected situations may cause the axle box spring to be lost, which will make the vehicle run unsteadily and affect the safe operation of railway freight cars. Therefore, it is necessary to pay attention to whether the axle box spring is lost during operation. The current manual inspection method of checking drawings one by one is not only time-consuming and labor-intensive, but also requires a lot of labor costs and is very inefficient. In addition, errors and omissions in inspection often occur, making it difficult to ensure the quality of the inspection.

[0004] By combining image processing and deep learning technology, automatic fault identification and alarm can be achieved, which can effectively improve the quality and efficiency of vehicle inspection operations. The advantages of the YOLOv5s network are fast detection speed and high detection accuracy. It can show better performance under the same hardware equipment. It has high scalability and can be applied to different hardware platforms. YOLOv5s is widely used in industry, but YOLOv5s also has some defects. Combined with the actual detection process of the missing axle box springs of the front and rear side frames, when the axle box springs are blocked or overlapped, and the axle box springs have large size differences, the detection effect is not good; and when encountering complex detection environments, missed detection or false detection is prone to occur. These factors will cause the detection accuracy of deep learning networks to decrease. Summary of the invention

[0005] The present invention aims to solve the problem that when the existing image processing and deep learning technologies are used to detect the loss of the front and rear side frame axle box springs, missed detection or false detection is prone to occur, resulting in a decrease in the detection accuracy of the deep learning network. The original YOLOv5s network is optimized and improved to provide a railway freight car front and rear side frame axle box spring loss fault identification method based on deep learning. The method not only ensures real-time detection speed, but also improves the accuracy of fault detection, greatly reduces the possibility of missed detection or false detection, and does not increase too many parameters.

[0006] The deep learning-based railway freight car front and rear side frame axle box spring loss fault identification method includes:

[0007] Input the image to be tested into the YOLOv5s network to obtain the recognition result;

[0008] The YOLOv5s network embeds the SimAM parameter-free attention mechanism at the end of each CSP1 unit in the Backbone part and after the Focus slicing operation, and adopts the AFF attention feature fusion module to replace the Concat module in the Neck part. The positioning loss function of the YOLOv5s network is SIoU.

[0009] Furthermore, the calculation expression of the above AFF attention feature fusion module is:

[0010]

[0011] Among them, Z represents output, X and Y represent initial input features, M() represents MS-CAM module, + represents initial feature integration, Indicates the fusion operation of two features.

[0012] Furthermore, the above SioU loss L box The expression is:

[0013]

[0014] Among them, IoU is IoU loss, Δ is distance loss, and Ω is shape loss.

[0015] Furthermore, the expression of the above distance loss Δ is:

[0016]

[0017] Where γ = 2-Λ,

[0018] Λ is the angle loss, c w is the width difference between the center point of the real box and the predicted box, is the center coordinate of the real frame, is the center coordinate of the prediction box.

[0019] Furthermore, the expression of the above angle loss Λ is:

[0020]

[0021] in, c h is the height difference between the center point of the real box and the predicted box, and σ is the distance between the center point of the real box and the predicted box.

[0022] Furthermore, the expression of the above shape loss Ω is:

[0023]

[0024] Where Ω is the shape loss, (w gt ,h gt ) is the width and height of the real box, (w,h) is the width and height of the predicted box, and the value range of parameter θ is [2,6].

[0025] Furthermore, the expression of the above IoU loss IoU is:

[0026]

[0027] Among them, B and B GT They are the predicted box and the true box respectively.

[0028] Furthermore, the method for acquiring the image under test includes:

[0029] The image of the side of the truck is collected by line scanning, and the positions of the front and rear side frame axle boxes are roughly located according to the basic information of the truck and the leading information, and the local area image containing the components is obtained as the measured image.

[0030] A computer-readable storage medium stores a computer program, which, when executed, implements the above-mentioned method for identifying spring loss faults of front and rear side frames of railway freight cars based on deep learning.

[0031] 10. An electronic device, comprising a storage medium, a processor, and a computer program stored in the storage medium and executable on the processor, wherein the processor executes the computer program to implement the above-mentioned method for identifying the spring loss fault of the front and rear side frames of railway freight cars based on deep learning.

[0032] The beneficial effects of the present invention are as follows:

[0033] 1. Apply deep learning algorithms to the detection of spring loss faults in the front and rear side frames to achieve automatic fault identification and alarm, effectively save labor costs, and improve detection efficiency and quality.

[0034] 2. Compared with traditional machine vision detection methods, deep learning methods have higher accuracy, stability, and robustness, and have faster reasoning speed and higher efficiency.

[0035] 3. For the axle box spring missing fault, due to the problem of feature loss in the Backbone feature extraction process of YOLOv5s, the detection effect is poor when the axle box spring is blocked or overlapped, and the axle box spring size difference is large, which seriously affects the accurate detection of the axle box spring missing fault. In order to solve this problem, the SimAM parameter-free attention mechanism is introduced at the end of each CSP1 module in Backbone to enhance the learning ability of the feature network. By adopting the SimAM attention mechanism, the loss of some features in the feature extraction process is successfully reduced, and at the same time, too many parameters are not introduced, thereby improving the detection accuracy and reliability of the axle box spring missing fault, enabling the network to better capture the characteristics of the axle box spring missing fault, and also improving the overall detection performance and robustness.

[0036] 4. In order to deal with the problem of information loss and reduce missed detection and false detection in the axle box spring loss fault, a new AFF feature fusion module is introduced in the feature fusion process of the Neck structure to replace the original Concat operation. Compared with the traditional Concat operation, the AFF module can more effectively reduce information loss, improve the effect of feature fusion, and help improve the accuracy and robustness of target detection. Moreover, while improving the feature fusion effect, the AFF module does not introduce too much computational complexity, and can improve network performance while maintaining high computational efficiency.

[0037] 5. Improve the loss function. The methods proposed and used so far do not consider the direction of mismatch between the expected ground box and the predicted "experimental" box. This deficiency leads to slower convergence and lower efficiency. Therefore, SIoU is used instead of the original CIoU to improve the training speed and reasoning accuracy. BRIEF DESCRIPTION OF THE DRAWINGS

[0038] Figure 1 Training flow chart for target detection model;

[0039] Figure 2 This is the network structure diagram of the SimAM module;

[0040] Figure 3 This is the network structure diagram of the AFF module;

[0041] Figure 4 Schematic diagram of angle loss;

[0042] Figure 5 It is a schematic diagram of shape loss;

[0043] Figure 6 This is the improved YOLOv5s network structure diagram;

[0044] Figure 7 This is a fault identification flow chart. DETAILED DESCRIPTION

[0045] The following will be combined with the accompanying drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, rather than all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work belong to the scope of protection of the present invention. It should be noted that the embodiments of the present invention and the features in the embodiments can be combined with each other without conflict.

[0046] Specific implementation method 1, refer to Figures 1 to 7 Specifically describing this embodiment, the method for identifying the spring loss fault of the front and rear side frame axle box of a railway freight car based on deep learning in this embodiment includes:

[0047] 1. Linear Image Acquisition

[0048] High-definition equipment is installed on the side of the truck track to take real-time photos of the truck passing at high speed and obtain images of the side of the truck. The image is collected by line scanning, which can achieve seamless image splicing and generate a two-dimensional image with a large field of view and high clarity.

[0049] 2. Rough positioning of components

[0050] The front and rear side frame axle boxes are roughly located based on the basic information and leading information of the truck, and the local area image containing the components is captured from the large side image, which can effectively reduce the reasoning time required for fault identification and improve identification accuracy and efficiency.

[0051] 3. Establishing the original sample data set

[0052] Since truck parts may be affected by natural conditions or man-made conditions such as rain, mud, oil stains, and black paint, there are many uncontrollable situations in the pictures taken. In addition, due to differences in lighting, shooting equipment, and other factors at different sites, the images taken will also be different. Therefore, in the process of collecting image datasets, it is necessary to ensure the diversity of the dataset images and try to collect images under various conditions and at different sites.

[0053] The dataset contains two parts: images and their corresponding marking information xml files. The images are taken by the equipment and include the front and back side frame parts. The fault location target frame is obtained through manual marking. The fault location information is stored in the marking information xml file. The image and the xml file correspond to each other. Both parts are grayscale images, that is, each image corresponds to a marking data.

[0054] 4. Data Augmentation

[0055] Data augmentation refers to the process of generating new data samples by performing a series of transformations and processing on the original data set. Data augmentation plays an important role in machine learning and deep learning. The main reasons for data augmentation include increasing the number of samples in the training set, alleviating model overfitting, and improving the generalization ability of the model. By increasing the number of samples in the training set, it can effectively alleviate the overfitting of the model and improve the generalization ability of the model. This is because deep learning models usually have a large number of parameters, and the number of samples in the training set is often difficult to match. Through data augmentation, the sample space can be expanded so that the model can learn more features and patterns, thereby avoiding excessive dependence on training data and improving the performance of the model on new data. The augmentation forms include operations such as image translation, scaling, and brightness adjustment. Each operation is performed under random conditions. Horizontal flipping and vertical flipping are used to enhance the diversity of the image, adjust the image brightness and contrast, reduce the impact of uneven lighting, highlight the edge features of defects, use histogram equalization to enhance image contrast, add Gaussian blur and random noise, and improve the generalization ability of the model.

[0056] 5. Model Training

[0057] After the training data set is established, the deep learning target detection algorithm is used to detect the position of the front and rear side frame axle boxes. The model training process is as follows: Figure 1 shown.

[0058] 1. Improve the Backbone feature extraction network of YOLOv5s

[0059] YOLOv5 is divided into four versions of models. The structures of these models are basically the same, but the performance varies greatly. Among them, YOLOv5s has the smallest network depth and width among the four models, but the inference speed reaches 6.4ms. Although the YOLOv5s algorithm has achieved certain results in target detection, there are still some challenges and limitations. During detection, it will be affected by a variety of complex external factors, such as lighting, environment, target color and shape, and occlusion overlap. The feature extraction ability greatly affects the detection effect, and the detection effect is not very good when detecting small targets. The Backbone framework of the YOLOv5s network uses CSPDarknet. The entire backbone uses a residual structure. Feature loss will occur during the feature extraction process, resulting in room for improvement in the detection effect. Therefore, in order to improve the feature extraction capability and reduce feature loss, the SimAM parameter-free attention mechanism is embedded at the end of each CSP1 unit in the backbone and after the Focus slice operation. The SimAM structure is as follows Figure 2 As shown in the figure, most existing attention modules generate one-dimensional or two-dimensional weights from feature x, and then generalize the generated weights to 1-D channel attention and 2-D spatial attention. The SimAM module directly estimates the three-dimensional weights. In each sub-image, the areas represented by each box are one-to-one corresponding, which means that the features of each box are independent. The SimAM parameter-free attention mechanism innovatively proposes a way to directly derive three-dimensional attention weights, and no additional parameters need to be added in this process. It is flexible and effective in improving convolutional networks, which can help the model better understand and utilize the information of the input data, significantly improve the performance of various visual tasks, and does not require too many complex operations to implement.

[0060] 2. Improve the Neck feature fusion network of YOLOv5s

[0061] In the feature fusion process of the Neck structure, the AFF attention feature fusion module is used to replace the original Concat. The structure is as follows Figure 3 As shown. The calculation method of AFF attention feature fusion module is as follows:

[0062]

[0063] Among them, Z represents the output, X and Y represent the initial input features, M() represents the MS-CAM module (multi-scale channel attention module), + represents the initial feature integration, Indicates the fusion operation of two features.

[0064] The two features X and Y input to the AFF attention feature fusion module are first fused with initial features, and then the initial features are passed through the MS-CAM module and the sigmoid activation function. The output value is between 0 and 1. The weighted average of X and Y is taken. Then, 1 is subtracted from this set of fusion weights M(X+Y) to obtain soft selection. The network can determine their respective weights through training.

[0065] The AFF attention feature fusion module proposes a unified and universal scheme that is applicable to most common scenarios for better fusing features with inconsistent semantics and scales. By aggregating multi-scale contextual information in the channel dimension, it can simultaneously emphasize large objects with wider distribution and small objects with more local distribution, which helps the network to recognize and detect objects under extreme scale changes. The initial fusion quality is improved by adding another attention module to the input features, and the performance is improved by gradually improving the initial fusion. The performance of various networks can be improved by simply replacing the existing feature fusion operator with the proposed iterative attention feature fusion module. As a feature fusion method, the AFF module has the advantages of strong adaptability, rich information, effective reduction of information loss, flexible design and high efficiency, and is suitable for various target detection and feature fusion scenarios.

[0066] 3. Improve the loss function

[0067] CIoU is used as the positioning loss function in YOLOv5s. It is a loss function for target detection tasks that aims to measure the difference between the predicted bounding box and the true bounding box. Compared with IoU, CIoU makes the model more accurate in learning target positioning, especially when there are large changes in the shape and position of the bounding box, it can provide better training signals, which helps to improve the accuracy of target detection. However, in some cases, CIoU may not perform as well as other loss functions for positioning small targets, and additional tuning and experiments may be required. Therefore, this implementation uses SIoU to replace the original CIoU in YOLOv5s. It introduces a smoothing term when calculating IoU, which aims to improve the stability and robustness of the model when learning target positioning. The advantages of SIoU are mainly reflected in the following aspects:

[0068] (1) Stability: A smoothness term is introduced to make the position and shape changes of the bounding box more stable when calculating IoU.

[0069] (2) Robustness: The introduction of the smoothness term makes SIoU more robust to some extreme cases and can better handle various boundary conditions, thereby improving the robustness of the model.

[0070] (3) Convergence speed: SIoU may accelerate the convergence speed of the model during training and help improve training efficiency.

[0071] (4) Reduced sensitivity to hyperparameters: The smooth nature of SIoU makes it more robust to the choice of hyperparameters. Compared with some other loss functions, the performance of SIoU is less sensitive to the choice of hyperparameters.

[0072] The SIoU loss function consists of 4 Cost functions:

[0073] Angle cost: such as Figure 4 Describes the minimum angle between the center point connection and the xy axis. When the center point is aligned on the x-axis or y-axis, the angle loss Λ = 0. When the center point is connected to the x-axis 45°, the angle loss Λ = 1. This penalty can guide the anchor box to move to the nearest axis of the target box, reducing the total number of degrees of freedom of BBR. The formula is as follows:

[0074]

[0075] in, c h is the height difference between the center point of the real box and the predicted box, and σ is the distance between the center point of the real box and the predicted box.

[0076] Distance cost: describes the distance between the center points. Its penalty cost is positively correlated with the angle cost. When α→0, the contribution of distance loss is greatly reduced. On the contrary, when α→π / 4, the contribution of distance loss is greater. The distance loss formula is as follows:

[0077]

[0078] Where γ = 2-Λ, c w is the width difference between the center point of the real box and the predicted box, is the center coordinate of the real frame, is the center coordinate of the prediction box.

[0079] Shape cost: such as Figure 5 The aspect ratio between the two boxes considered here is defined by calculating the difference in width between the two boxes and the ratio of the maximum width between the two boxes (the same applies to the length). The general idea is similar to CIoU, except that CIoU can consider the convergence of the overall shape of the two boxes, while SIoU achieves the effect of overall shape convergence by converging the length and width.

[0080]

[0081] Where Ω is the shape loss, (w gt ,h gt) is the width and height of the real box, (w,h) is the width and height of the predicted box, and the value range of parameter θ is [2,6].

[0082] IoU cost:

[0083]

[0084] Among them, IoU is the IoU loss, B and B GT They are the predicted box and the true box respectively.

[0085] SIoU loss L box The expression is:

[0086]

[0087] The overall network structure designed in this implementation is as follows Figure 6 As shown. First, this implementation method embeds the SimAM parameter-free attention mechanism at the end of each CSP1 unit in the Backbone part and after the Focus slice operation. In the process of downsampling the backbone network, shallow and high-level semantic information is fully utilized, and the detection accuracy of targets of three different scales is improved. Most importantly, the SimAM parameter-free attention mechanism is lightweight. After adding it to the Backbone, it will hardly increase the parameters and also improve the training efficiency of the network. By introducing the SimAM parameter-free attention mechanism, we can better pay attention to the feature information transmission in the backbone network, reduce feature loss, and thus improve the accuracy of target detection. Secondly, in the Neck part, the AFF attention feature fusion module is used to replace the original Concat. The AFF attention feature fusion module is a unified and universally applicable feature fusion solution that can better fuse features of different scales and improve the problem of information loss in the Neck part of the YOLOv5s network. Finally, SIoU is used instead of the original CIoU as the positioning loss function, which not only speeds up the convergence of the model, helps to improve the training efficiency, but also improves the detection accuracy.

[0088] 6. Fault Identification

[0089] 1. Rough positioning of components

[0090] Based on the vehicle model information and reasoning through the coarse positioning network, the sub-region image containing the parts is obtained.

[0091] 2. Load the model

[0092] Load the precise positioning target detection model.

[0093] 3. Generate detection results

[0094] The sub-image of the area to be detected is input into the target detection model to obtain the detection results of each category for subsequent detection.

[0095] 4. Fault determination

[0096] Based on the detection results, determine whether there is a detection frame for the front and rear side frame axle box spring loss. If the detection frame threshold is greater than 0.6, it is determined that the front and rear side frame axle box spring is lost, and the front and rear side frame axle box spring loss is marked through a marking frame.

[0097] 5. Upload alarm

[0098] According to the fault information, the fault location and fault category are generated into a message and uploaded to the alarm platform.

[0099] like Figure 7 As shown, this embodiment first uses high-definition imaging equipment on the side of the truck track to shoot the high-speed moving truck, obtain high-definition images of the front and rear side frames of the truck, collect and organize the images, and perform data amplification to obtain a training image sample set. Secondly, the rough positioning of each component is determined based on the prior information of each vehicle, and the position of each component is preliminarily determined to facilitate subsequent fault identification of specific parts. Then, a suitable deep neural network is built according to the type of truck fault, and multiple trainings are performed until the model converges and the corresponding parameters are obtained. Finally, in the recognition stage, the network model is loaded, and the real image taken is input into the network to obtain the prediction result, and the prediction result is logically judged to confirm whether there is a fault. If there is a fault, an alarm is issued for the fault area.

[0100] Specific implementation method 2: This implementation method describes a computer-readable storage medium, wherein the storage medium stores a computer program, and is characterized in that when the computer program is executed, the method described in specific implementation method 1 is implemented.

[0101] Specific implementation method three: The electronic device described in this implementation method includes a storage medium, a processor, and a computer program stored in the storage medium and executable on the processor, and is characterized in that the processor executes the computer program to implement the method described in specific implementation method one.

[0102] Although the present invention is described herein with reference to specific embodiments, it should be understood that these embodiments are merely examples of the principles and applications of the present invention. It should therefore be understood that many modifications may be made to the exemplary embodiments and that other arrangements may be devised without departing from the spirit and scope of the present invention as defined by the appended claims. It should be understood that the various dependent claims and features described herein may be combined in a manner different from that described in the original claims. It should also be understood that the features described in conjunction with a single embodiment may be used in other described embodiments.

Claims

1. A method for identifying the spring loss fault of the front and rear side frame axle box of a railway freight car based on deep learning, including: Input the image to be tested into the YOLOv5s network to obtain the recognition result; It is characterized in that The YOLOv5s network embeds the SimAM parameter-free attention mechanism at the end of each CSP1 unit in the Backbone part and after the Focus slicing operation, and adopts the AFF attention feature fusion module to replace the Concat module in the Neck part. The positioning loss function of the YOLOv5s network is SIoU.

2. The method for identifying the spring loss fault of the front and rear side frame axle box of a railway freight car based on deep learning according to claim 1 is characterized in that: The calculation expression of the AFF attention feature fusion module is: Among them, Z represents output, X and Y represent initial input features, M() represents MS-CAM module, + represents initial feature integration, Indicates the fusion operation of two features.

3. The method for identifying spring loss fault of front and rear side frame axle box of railway freight car based on deep learning according to claim 1 is characterized in that: The SioU loss L box The expression is: Among them, IoU is IoU loss, Δ is distance loss, and Ω is shape loss.

4. The method for identifying the spring loss fault of the front and rear side frame axle box of a railway freight car based on deep learning according to claim 3 is characterized in that: The expression of the distance loss Δ is: Where, γ=2-Λ, Λ is the angle loss, c w is the width difference between the center point of the real box and the predicted box, is the center coordinate of the real frame, is the center coordinate of the prediction box.

5. The method for identifying the spring loss fault of the front and rear side frame axle box of a railway freight car based on deep learning according to claim 4 is characterized in that: The expression of the angle loss Λ is: in, c h is the height difference between the center point of the real box and the predicted box, and σ is the distance between the center point of the real box and the predicted box.

6. The method for identifying the spring loss fault of the front and rear side frames of railway freight cars based on deep learning according to claim 3 is characterized in that: The expression of the shape loss Ω is: Where Ω is the shape loss, (w gt ,h gt ) is the width and height of the real box, (w,h) is the width and height of the predicted box, and the value range of parameter θ is [2,6].

7. The method for identifying the spring loss fault of the front and rear side frames of railway freight cars based on deep learning according to claim 3 is characterized in that: The expression of the IoU loss IoU is: Among them, B and B GT They are the predicted box and the true box respectively.

8. The method for identifying the spring loss fault of the front and rear side frames of railway freight cars based on deep learning according to claim 1 is characterized in that: The method for acquiring the measured image comprises: The image of the side of the truck is collected by line scanning, and the positions of the front and rear side frame axle boxes are roughly located according to the basic information of the truck and the leading information, and the local area image containing the components is obtained as the measured image.

9. A computer-readable storage medium storing a computer program, characterized in that: When the computer program is executed, the method according to any one of claims 1 to 8 is implemented.

10. An electronic device comprising a storage medium, a processor, and a computer program stored in the storage medium and executable on the processor, characterized in that: The processor executes the computer program to implement the method according to any one of claims 1 to 8.

Images