Method and device for identifying faults of key components of a train

Abstract

The present invention discloses a method and a device for identifying faults in key components of a train. The method includes: obtaining an image of the faulty part of the train as the image to be detected, and obtaining a train fault sample data set; inputting the train fault sample data set into a deep learning model for model training to obtain a target fault identification model; and inputting the image to be detected into the target fault identification model for fault identification. The present invention uses an improved yolov5 deep learning network model to identify faults in key components of a train. By improving the overall performance of the model, the accuracy of fault detection of key components of the train is improved, which can meet the accuracy requirements of on-site maintenance operations of the train, and the model is well deployed to hardware devices, and the model has strong universality.

Description

Technical Field

[0001] The present invention relates to the technical field of railway train fault detection, and particularly relates to a method and device for identifying faults of key components of a train. Background Art

[0002] When a train is running at a high speed, especially for key components on the train, such as important components related to train safety with loose fixed skirt panels, any small and subtle faults may cause major accidents. Therefore, it is crucial to improve the accuracy of fault identification of key components of the train.

[0003] Currently, the detection of key components of a train mainly relies on manual inspection for fault detection. With the continuous increase in the number of train formations, the workload of on-site operators is large and requires high concentration, making them prone to fatigue. Only relying on manual inspection, the detection efficiency and accuracy are relatively low. In some fault identification methods, automatic identification technology is considered to assist manual inspection. By automatically identifying the images collected of key components of the train, suspicious images are screened out and then manually screened again for fault verification. However, this technology is highly dependent on manual experience, and the detection and identification rate of faults is not very ideal, and it cannot meet the requirements of on-site work in terms of detection accuracy. Summary of the Invention

[0004] The purpose of the present invention is to provide a method and device for identifying faults of key components of a train to solve the problem of low accuracy of current train fault detection.

[0005] To achieve the above purpose, on the one hand, the present invention provides a method for identifying faults of key components of a train, including:

[0006] Obtaining an image of the train fault location as the image to be detected, and obtaining a train fault sample data set;

[0007] Inputting the train fault sample data set into a deep learning model for model training to obtain a target fault identification model;

[0008] Inputting the image to be detected into the target fault identification model for fault identification.

[0009] Optionally, before inputting the train fault sample data set into the deep learning model for model training, it further includes:

[0010] Classifying the train fault sample data set according to the location and fault characteristics of the train fault, as well as the fault categories existing in the key parts of the train, and dividing it into multiple train fault sample sub-data sets;

[0011] Identifying the fault information in each train fault sample sub-data set;

[0012] Preprocess each type of the train fault sample sub - datasets, and screen out the train fault sample sub - datasets with a small amount of fault information data for image data augmentation;

[0013] Divide the preprocessed train fault sample dataset into a train fault sample training set, a train fault sample test set, and a train fault sample validation set.

[0014] Optionally, inputting the train fault sample dataset into a deep learning model for model training to obtain a target fault recognition model, including:

[0015] The deep learning model adopts an improved yolov5 deep learning network model, including an input end, a backbone network module, a neck network module, and an output end;

[0016] After inputting the train fault sample dataset into the input end, through data augmentation processing, the train fault sample dataset is trained after being spliced by multiple fault images; for the data - augmented train fault sample dataset, adjust the initial anchor boxes according to different ratios of the key components to be detected, and adaptively scale the sizes of the fault images;

[0017] Further input the train fault sample dataset output from the input end into the backbone network module. Slice the fault images of the train fault sample dataset through the Focus unit to form an original fault feature map; extract the feature information in the sliced original fault feature map through the CBL unit and fuse the feature information through the CSP unit, and perform pooling operations on the original fault feature map after feature information fusion through the SPP unit using different - sized pooling windows to output a first predicted fault feature map;

[0018] Input the train fault sample dataset output from the backbone network module into the neck network module. Aggregate features for different detection layers according to different layers of the backbone network. Convey strong semantic features from top to bottom through the FPN unit, and transfer and fuse the high - level feature information through downsampling; and convey strong localization features from bottom to top through the PAN unit, and transfer and fuse the low - level feature information through upsampling to output a second predicted fault feature map;

[0019] Mark the defect information in the image for the second predicted fault feature map output from the neck network module, and determine the target confidence according to the defect information;

[0020] Take the trained deep learning model as the target fault recognition model.

[0021] Optionally, the Mosaic-6 data augmentation method is used to perform data augmentation on the train fault sample dataset input to the input end, and the train fault sample dataset is randomly cropped, randomly scaled, and randomly arranged and combined into one picture for training.

[0022] Optionally, the backbone network module further includes an SE unit, which learns the feature weights according to the size of the residual value of the first predicted fault feature map through the network, and trains the model in a way that the effective fault feature map has a large weight and the invalid fault feature map has a small weight.

[0023] Optionally, the neck network module further includes a CRNet unit, which fuses and adds the features obtained by the PAN unit again to optimize the feature fine-grainedness.

[0024] Optionally, the improved yolov5 deep learning network model uses the CIOU loss function.

[0025] Optionally, the improved yolov5 deep learning network model uses the K-means mean algorithm to recalculate the candidate box size, and replaces the original candidate box size with the calculated optimal candidate box size.

[0026] Optionally, the obtained train fault part image is subjected to light source compensation processing to obtain the image to be detected.

[0027] On the other hand, the present invention also provides a train key component fault identification device, which adopts the above-mentioned train key component fault identification method, including:

[0028] An image acquisition module, configured to obtain a train fault part image as an image to be detected, and obtain a train fault sample dataset;

[0029] A model training module, configured to input the train fault sample dataset into a deep learning model for model training to obtain a target fault identification model; and

[0030] Input the image to be detected into the target fault identification model for fault identification.

[0031] The method of the present invention has the following advantages:

[0032] The method for identifying faults in key components of a train according to the present invention uses an improved YOLOv5 deep learning network model to train a model on a train fault sample data set, and then obtains the trained model as the target fault identification model; then the image to be detected is input into the target fault identification model for fault identification. The present invention uses an improved YOLOv5 deep learning network model for fault identification of key components of a train. By improving the overall performance of the model, the accuracy of fault detection of key components of a train is improved, which can meet the accuracy requirements of on-site maintenance operations of trains, and the model can be well deployed to hardware devices, and the model has strong universality. BRIEF DESCRIPTION OF THE DRAWINGS

[0033] Figure 1 is a schematic flow chart of the method for identifying faults in key components of a train according to this embodiment;

[0034] Figure 2 is a schematic diagram of the basic components of an improved YOLOv5 deep learning network model;

[0035] Figure 3 is a schematic diagram of the SE unit structure;

[0036] Figure 4 is a schematic diagram of the CRNet unit structure;

[0037] Figure 5 is a framework diagram of the device for identifying faults in key components of a train. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0038] The technical solutions of the present invention will be clearly and completely described below in conjunction with specific implementation solutions. However, those skilled in the art should understand that the implementation solutions described below are only used to illustrate the present invention and should not be regarded as limiting the scope of the present invention. Based on the implementation solutions in the present invention, all other implementation solutions obtained by those of ordinary skill in the art without creative work fall within the scope of protection of the present invention.

[0039] At the same time, it should be understood that the following embodiments are given only for the purpose of illustration and are not used to limit the scope of protection of the present invention. Those skilled in the art can make various modifications and substitutions to the present invention without departing from the purpose and spirit of the present invention, and all such modifications and substitutions fall within the scope of protection claimed in the claims of the present invention.

[0040] A method for identifying faults in key components of a train, as Figure 1 shown, Figure 1 shows a schematic flow chart of the method for identifying faults in key components of a train according to this embodiment, including:

[0041] S1. Obtain an image of the fault location of the train as the image to be detected, and obtain a train fault sample data set;

[0042] In a specific implementation, an infrared camera device pre-built around the track is used to obtain an image of a passing train, and an image of the required train fault part is selected from the image of the passing train as the image to be detected. At the same time, a fill light module can be set for the image acquisition module to control fill light during the operation of the image acquisition module. The light source compensation has the function of resisting sunlight interference, a larger irradiation range and depth, a more concentrated irradiation area, and adjustable brightness. By performing light source compensation processing on the obtained image of the train fault part, the image to be detected is obtained, enhancing the shooting and screening accuracy of the image to be detected.

[0043] At the same time, according to the location and fault characteristics of the train fault, as well as the fault categories existing in the key parts of the train, the train fault sample data set is classified into multiple train fault sample sub-data sets; then, the fault information in each train fault sample sub-data set is identified.

[0044] In this embodiment, specifically, the fault images of various faults in the train fault sample data set are converted from RGB color images to grayscale images, and the fault information in each train fault sample sub-data set is identified by manual marking. For example, annotation software such as Sprite Annotation Assistant and Labelimage is used to save the fault information in the picture, including the name of the grayscale image, the coordinates of the upper left and lower right points of the marked box of the fault area, and the fault categories existing in the key parts of the train, in the form of an xml file as described above. The fault categories existing in the key parts of the train include: the opening of the main door, foreign objects, hitting, scratching and deformation, oil leakage and splashing, component detachment and loss, noise reduction layer detachment, anti-loosening wire disconnection, cable damage, anti-off-chain disconnection of the axle box rubber cover, and the orientation of the safety lock cylinder.

[0045] Then, preprocessing is performed on each train fault sample sub-data set, and the train fault sample sub-data set with a small amount of fault information data is selected for image data augmentation. For example, methods such as adding Gaussian noise, salt and pepper noise, mirroring, rotation, and brightness are used to perform image data augmentation on the images of the train fault sample sub-data set with a small amount of fault information data. These methods can imitate different situations of fault occurrence to balance each fault type and avoid data asymmetry and overfitting.

[0046] Finally, the preprocessed train fault sample data set is divided into a train fault sample training set, a train fault sample test set, and a train fault sample validation set. In this embodiment, the specific ratio is set to 7:1:2. Then, the train fault sample data set can be input into the deep learning model for model training according to the set ratio, and hyperparameters such as the learning rate, training batch, and image size are set.

[0047] S2. Input the train fault sample data set into a deep learning model for model training to obtain a target fault recognition model.

[0048] In specific implementation, the deep learning model in this embodiment specifically adopts an improved yolov5 deep learning network model, as Figure 2 shown, Figure 2 which shows a schematic diagram of the basic components of the improved yolov5 deep learning network model;

[0049] The schematic diagram of the basic components of the improved yolov5 deep learning network model includes an input end, a backbone network module, a neck network module, and an output end;

[0050] After inputting the train fault sample data set into the input end, through data augmentation processing, the train fault sample data set is trained after being stitched together according to multiple fault images; for the train fault sample data set after data augmentation, the initial anchor box is adjusted according to different ratios of the key components to be detected, and the size of the fault image is adaptively scaled. In this embodiment, the original Mosaic-4 data augmentation method is specifically improved to the Mosaic-6 data augmentation method, and the train fault sample data set input into the input end is subjected to data augmentation processing, and the train fault sample data set is randomly cropped, randomly scaled, and randomly arranged and combined into one picture according to 6 fault images for training.

[0051] The train fault sample data set output from the input end is further input into the backbone network module. The fault images in the train fault sample data set are sliced through the Focus unit to form an original fault feature map, so as to ensure that when the image undergoes downsampling, information will not be lost; the feature information in the sliced original fault feature map is extracted through the CBL unit and the feature information is fused through the CSP unit to retain richer feature information; and the original fault feature map after feature information fusion is subjected to pooling operation through the SPP unit using different size pooling windows, and a first predicted fault feature map is output. In this embodiment, 4 different size pooling windows are specifically used for the pooling operation.

[0052] At the same time, in this embodiment, by improving the yolov5 deep learning network model, an SE unit is also added to the backbone network module. The structure of the SE unit is as Figure 3 shown. The network learns the feature weights according to the magnitude of the residual value of the first predicted fault feature map. The effectiveness of the fault feature map is determined according to the residual value. The model is trained in a way that the effective fault feature map has a large weight and the ineffective fault feature map has a small weight. The SE unit is lightweight and can bring performance improvement only by increasing a small amount of computation.

[0053] The neck network module is set after the backbone network module. The train fault sample data set after the output of the backbone network module is input into the neck network module. Feature aggregation is performed on different detection layers according to different layers of the backbone network. The FPN combined with the PAN architecture is adopted. Through the FPN unit, strong semantic features are conveyed from top to bottom, and the high-level feature information is transmitted and fused by means of downsampling; and through the PAN unit, strong localization features are conveyed from bottom to top, and the low-level feature information is transmitted and fused by means of upsampling, and the second predicted fault feature map is output.

[0054] At the same time, in this embodiment, by improving the yolov5 deep learning network model, a CRNet unit is also added to the neck network module. The structure diagram of adding CRNet to the neck network is as Figure 4 shown. The features obtained by the PAN unit are fused and added again to optimize the feature fine-grainedness. In this embodiment, specifically, the features obtained by the Conv and Relu methods and the PAN unit can be fused and added again to collect richer and finer-grained feature information.

[0055] The output end is connected to each output feature layer in the PAN unit of the neck network module. The second predicted fault feature map output after passing through the neck network module is marked with the defect information in the image through the output end, and the target confidence level is determined according to the defect information.

[0056] In addition, in this embodiment, by improving the yolov5 deep learning network model, the improved yolov5 deep learning network model uses the CIOU loss function to repair the defect that the original GIOU does not converge and cannot be recognized when the predicted box and the true box have an inclusion relationship or the length and width coincide, and can effectively improve the recognition probability of occluded objects. At the same time, the improved yolov5 deep learning network model uses the K-means mean algorithm to recalculate the candidate box size, and replaces the original candidate box size with the calculated optimal candidate box size. The improved yolov5 deep learning network model is much stronger than the previous yolo algorithm in terms of flexibility and speed, and can also support the conversion of the pt model file format to the torchscript, onnx, and coreml file formats, and has a great advantage in the rapid deployment of the model.

[0057] S3. Input the to-be-detected image into the target fault recognition model for fault recognition.

[0058] In this embodiment, through the above model training process, the training process of inputting the train fault sample data set into the deep learning model for model training is realized. The trained deep learning model is used as the target fault recognition model to train the to-be-detected images of the key components of the EMU, and the output is the fault detection result.

[0059] Therefore, for the method for identifying faults in key components of a train according to the present invention, an improved YOLOv5 deep learning network model is used to train a model on a train fault sample data set, and then the trained model is obtained as the target fault identification model; then the image to be detected is input into the target fault identification model for fault identification. The present invention uses an improved YOLOv5 deep learning network model for fault identification of key components of a train. By improving the overall performance of the model, the accuracy of fault detection of key components of a train is improved, which can meet the accuracy requirements of on-site maintenance operations of trains, and the model can be well deployed to hardware devices, and the model has strong universality.

[0060] Specifically, in the present invention, an SE unit is added to the backbone network module of the conventional YOLOv5 deep learning network model, and a CRNet unit is also added to the neck network module, so that the overall performance of the improved YOLOv5 deep learning network model is improved with little increase in the amount of calculation, and the accuracy of fault detection of key components of a train is improved; and by performing noise, flipping, and rotation changes on the images of the original input training set, the augmentation comparison of the training set is realized, avoiding data asymmetry and overfitting during the training of the YOLOv5 network model, and effectively improving the resolution accuracy of the network model.

[0061] On the other hand, the present invention also provides a device for identifying faults in key components of a train, which adopts the above-mentioned method for identifying faults in key components of a train, as Figure 4 shown Figure 4 FIG. shows a schematic flowchart of the device 400 for identifying faults in key components of a train according to this embodiment, including:

[0062] An image acquisition module 401, configured to obtain an image of a train fault part as an image to be detected, and obtain a train fault sample data set;

[0063] A model training module 402, configured to input the train fault sample data set into a deep learning model for model training to obtain a target fault identification model; and

[0064] Input the image to be detected into the target fault identification model for fault identification.

[0065] Optionally, before inputting the train fault sample data set into the deep learning model for model training, it further includes:

[0066] Classify the train fault sample data set according to the location and fault characteristics of the train fault, as well as the fault categories existing in the key parts of the train, and divide it into multiple train fault sample sub-data sets;

[0067] Identify the fault information in each type of the train fault sample sub-datasets;

[0068] Preprocess each type of the train fault sample sub-datasets, and screen out the train fault sample sub-datasets with a small amount of fault information data for image data augmentation;

[0069] Divide the preprocessed train fault sample dataset into a train fault sample training set, a train fault sample test set, and a train fault sample validation set.

[0070] Optionally, inputting the train fault sample dataset into a deep learning model for model training to obtain a target fault recognition model includes:

[0071] The deep learning model uses an improved yolov5 deep learning network model, including an input end, a backbone network module, a neck network module, and an output end;

[0072] After inputting the train fault sample dataset into the input end, perform data augmentation processing, and train the train fault sample dataset after splicing multiple fault images; adjust the initial anchor box according to different ratios of the key components to be detected in the data-augmented train fault sample dataset, and adaptively scale the size of the fault images;

[0073] Further input the train fault sample dataset output by the input end into the backbone network module, perform slicing operations on the fault images of the train fault sample dataset through the Focus unit to form an original fault feature map; extract the feature information in the sliced original fault feature map through the CBL unit and fuse the feature information through the CSP unit, and perform pooling operations on the original fault feature map after feature information fusion through the SPP unit using different-sized pooling windows to output a first predicted fault feature map;

[0074] Input the train fault sample dataset output by the backbone network module into the neck network module, perform feature aggregation on different detection layers according to different layers of the backbone network, convey strong semantic features from top to bottom through the FPN unit, and transfer and fuse the high-level feature information through downsampling; and convey strong localization features from bottom to top through the PAN unit, and transfer and fuse the low-level feature information through upsampling to output a second predicted fault feature map;

[0075] Mark the defect information in the image for the second predicted fault feature map output by the neck network module, and determine the target confidence according to the defect information;

[0076] Use the trained deep learning model as the target fault recognition model.

[0077] Optionally, use the Mosaic-6 data augmentation method to perform data augmentation on the train fault sample dataset input to the input end, and randomly crop, randomly scale, and randomly arrange and splice 6 fault images in the train fault sample dataset into one image for training.

[0078] Optionally, the backbone network module further includes an SE unit, which learns feature weights according to the magnitude of the residual value of the first predicted fault feature map through the network, and trains the model in a way that the weight of the effective fault feature map is large and the weight of the ineffective fault feature map is small.

[0079] Optionally, the neck network module further includes a CRNet unit, which fuses and adds the features obtained by the PAN unit again to optimize the feature fine-grainedness.

[0080] Optionally, the improved yolov5 deep learning network model uses the CIOU loss function.

[0081] Optionally, the improved yolov5 deep learning network model uses the K-means mean algorithm to recalculate the candidate box size, and replaces the original candidate box size with the calculated optimal candidate box size.

[0082] Optionally, the image acquisition module 401 further includes a light supplement module 4011, which is used to perform light source compensation processing on the obtained train fault part image to obtain the image to be detected.

[0083] The train critical component fault recognition device of the present invention uses an improved yolov5 deep learning network model to recognize train critical component faults. By improving the overall performance of the model, the accuracy of train critical component fault detection is improved, which can meet the accuracy requirements of train on-site maintenance operations, and the model is well deployed to hardware devices with strong model universality.

[0084] Although the present invention has been described in detail with general descriptions and specific embodiments above, based on the present invention, some modifications or improvements can be made, which are obvious to those skilled in the art. Therefore, these modifications or improvements made without departing from the spirit of the present invention all fall within the scope of protection required by the present invention.

Claims

1. A method for identifying faults in key components of a train, characterized in that, Including: Obtaining an image of the train fault location as the image to be detected, and obtaining a train fault sample data set; Classifying the train fault sample data set according to the location and fault characteristics of the train fault, as well as the fault categories existing in the key parts of the train, and dividing it into multiple train fault sample sub-data sets; Identifying the fault information in each train fault sample sub-data set; Preprocessing each train fault sample sub-data set, screening out the train fault sample sub-data sets with less fault information data volume, and performing image data augmentation by adding Gaussian noise, salt-and-pepper noise, mirroring, rotation, and brightness; Dividing the preprocessed train fault sample data set into a train fault sample training set, a train fault sample test set, and a train fault sample validation set; Inputting the preprocessed and divided train fault sample data set into a deep learning model for model training to obtain a target fault recognition model; The deep learning model adopts an improved yolov5 deep learning network model, including an input end, a backbone network module, a neck network module, and an output end; The improvement points of the improved yolov5 deep learning network model include: The backbone network module also includes an SE unit, which learns the feature weights according to the magnitude of the residual value of the first predicted fault feature map output by the main network module, and trains the model in a way that the effective fault feature map has a large weight and the ineffective fault feature map has a small weight; The neck network module also includes a CRNet unit, which fuses and adds the features obtained by the PAN unit of the neck network module again to optimize the feature fine-grainedness; Inputting the image to be detected into the target fault recognition model for fault recognition.

2. The method according to claim 1, wherein Inputting the preprocessed and divided train fault sample data set into a deep learning model for model training to obtain a target fault recognition model, including: After inputting the train fault sample data set into the input end, performing data augmentation processing, and training the train fault sample data set after splicing multiple fault images; adjusting the initial anchor box according to the different proportions of the key components to be detected for the data-augmented train fault sample data set, and adaptively scaling the size of the fault images; Further inputting the train fault sample data set output by the input end into the backbone network module, performing slicing operations on the fault images of the train fault sample data set through the Focus unit to form an original fault feature map; extracting the feature information in the sliced original fault feature map through the CBL unit and fusing the feature information through the CSP unit, and performing pooling operations on the original fault feature map after feature information fusion through the SPP unit using different size pooling windows to output the first predicted fault feature map; Input the train fault sample data set after the output of the backbone network module into the neck network module, aggregate features for different detection layers according to different layers of the backbone network, convey strong semantic features from top to bottom through the FPN unit, and transfer and fuse the feature information of the high layer by downsampling; and convey strong localization features from bottom to top through the PAN unit, and transfer and fuse the feature information of the low layer by upsampling to output the second predicted fault feature map. Mark the defect information in the image through the output end of the second predicted fault feature map output by the neck network module, and determine the target confidence according to the defect information. Use the trained deep learning model as the target fault recognition model.

3. The method according to claim 2, wherein Use the Mosaic-6 data augmentation method to perform data augmentation on the train fault sample data set input to the input end, randomly crop, randomly scale, and randomly arrange and splice 6 fault images of the train fault sample data set into one picture and then perform training.

4. The method according to claim 2, wherein The improved yolov5 deep learning network model uses the CIOU loss function.

5. The method according to claim 2, wherein The improved yolov5 deep learning network model uses the K-means mean algorithm to recalculate the candidate box size, and replaces the original candidate box size with the calculated optimal candidate box size.

6. The method according to claim 2, wherein It further includes: Perform light source compensation processing on the obtained train fault part image to obtain the image to be detected.

7. A device for identifying faults in key components of a train, characterized in that, Adopt the train key component fault recognition method according to any one of claims 1-6, including: An image acquisition module, configured to acquire a train fault part image as an image to be detected, and acquire a train fault sample data set; A model training module, configured to input the train fault sample data set into a deep learning model for model training to obtain a target fault recognition model; and Input the image to be detected into the target fault recognition model for fault recognition.

Images