Bullet train bogie component winding foreign matter detection method based on image recognition

Abstract

The invention discloses a bullet train bogie component winding foreign matter detection method based on image recognition, and belongs to the technical field of railway bullet train operation state fault detection. The method comprises the steps that a feature extractor is trained by applying a SimCLR model, so that training of the feature extractor can be achieved by applying an unmarked data set; and training a target detector by using a small amount of labeled data sets to complete the training process of the whole model, and inputting a to-be-detected image into the model based on the trained model so as to realize the winding foreign matter detection of the bullet train bogie part. Most of the training sets used by the method are unlabeled training sets, so that the manpower and material resource cost generated by a large amount of fault sample data and labeled files is greatly reduced.

Description

Technical Field

[0001] The present invention relates to the technical field of railway motor vehicle running state fault detection, and in particular to a method for detecting foreign objects wrapped around motor vehicle bogie components based on image recognition. Background Art

[0002] To ensure the safe operation of EMUs, a Train Fault Detection System (TEDS) has been installed on high-speed rail operating lines and EMU depot entry and exit lines. This intelligent system integrates high-speed digital image acquisition, real-time processing of large-capacity image data, precise positioning, and pattern recognition technologies. The system comprises equipment for information acquisition, information processing and transmission, and a train inspection and testing center. Using a high-speed camera array, the system captures all visual information from the train's underbody and lower sides. After digital processing, the information is displayed on an information terminal at the inspection center. Train inspection personnel then use the images displayed on the terminal to determine the appearance of key components during EMU operation and determine faults.

[0003] Currently, only a very small number of components in this system have achieved automatic fault diagnosis, but the system still suffers from low fault detection rates and low alarm accuracy. Furthermore, a large number of components on the EMU undercarriage still rely on manual image verification to confirm whether they are faulty. With the continuous increase in EMU mileage and train formations, on-site operators have a heavy workload and require high concentration, which makes them prone to fatigue. At the same time, the number of personnel in TEDS monitoring centers at various railway bureaus has gradually increased, leading to rising labor costs. Faced with these numerous challenges, new technologies and methods are urgently needed to strengthen and optimize the image intelligent recognition algorithm for fault images of key EMU undercarriage components. This will enable on-site operations to gradually transition from manual inspection, supplemented by machine inspection, to machine inspection, supplemented by manual inspection. This will reduce the workload of train inspection personnel, improve fault detection efficiency, and ensure the safe operation of EMUs.

[0004] Currently, most methods for detecting EMU faults in response to the aforementioned issues rely on image processing technology, employing feature extraction and traditional machine learning methods for individual components to extract fault characteristics. This method then combines manual experience and prior knowledge with traditional machine learning classifiers to determine faults. Several related patents have been retrieved, with their general contents as follows:

[0005] The Chinese invention patent application with application number 202310310547.0 and application date is March 27, 2023. The name is "Method, system and equipment for detecting foreign objects in electric vehicles based on target detection network". It discloses that a training data set is first constructed, and then a target detection model is trained based on the data set (the binary cross entropy loss function of the YOLOV5 target detection model is replaced by a single classification loss function). Finally, the image to be tested is detected based on the trained model to determine whether there is a foreign object.

[0006] The application number is 202311742519.2, the application date is 2023.12.18, and the name is a Chinese invention patent application for a semi-supervised railway track foreign object detection method based on inverse perspective images. It discloses obtaining an inverse perspective image with the same resolution as the camera and integrating it into the teacher-student model architecture; in the model training stage, only normal railway line sample data is used, and the inverse perspective image is scaled and multiplied point by point with the intermediate layer feature map of the student network, and then the knowledge distillation operation is performed with the feature map corresponding to the teacher network; in the testing stage, the difference in the intermediate layer features of the teacher network and the student network is used to construct a multi-scale anomaly map, and finally the specific position of the foreign object in the image is determined by the anomaly map.

[0007] Currently, existing technologies and patents, including those used in actual field applications, mostly utilize traditional machine learning methods, such as image processing and recognition, and deep learning object detection methods, such as the YOLO series. Both approaches have the following drawbacks: Traditional machine learning image processing and recognition methods require a combination of prior knowledge and human experience to analyze the morphology of each component. The effectiveness of these methods is heavily dependent on the algorithm, resulting in poor fault accuracy and algorithm robustness, and they are easily disrupted by changes in the external environment. Deep learning object detection methods require the construction of a data training library containing a large amount of fault sample data and annotated files, which consumes considerable manpower and resources, and sample diversity significantly impacts the results. Summary of the Invention

[0008] In view of this, the purpose of the present invention is to provide a method for detecting foreign objects wrapped around EMU bogie components based on image recognition, so as to solve the problem that the existing deep learning target detection method needs to use a large amount of manually labeled fault sample data as a training set to train the model to ensure the accuracy of the detection results of the trained model, which leads to the data labeling process of a large number of fault samples consuming a lot of manpower and material resources.

[0009] To achieve the above objectives, the method of the present invention uses the SimCLR model to implement the training process of the feature extractor using an unlabeled training set. Specifically, the method includes the following steps:

[0010] 1) Input the image to be detected of the EMU bogie component into the encoder pre-trained by SimCLR to obtain the feature map of the image to be detected;

[0011] The SimCLR pre-training process includes data augmentation of the training set, feature extraction of the augmented data using an encoder, and training of the feature extraction model by optimizing a loss function. The training set is an unlabeled foreign object fault dataset of bogie components.

[0012] 2) using the feature map as input to the trained target detector to obtain a target detection result;

[0013] The object detector training process involves replacing the MLP layer in the encoder with an object detection probe based on the pre-trained SimCLR encoder model. This object detection probe is trained using a dataset of labeled foreign object faults on bogie components.

[0014] 3) Determine whether there is a foreign object entanglement fault in the EMU bogie components based on the target detection results.

[0015] The method of the present invention has the following advantages: The method of the present invention uses the SimCLR model to train the encoder, using an unlabeled training set, thereby greatly reducing the human and material costs generated by a large amount of fault sample data and labeled files. By using the model trained by the method of the present invention and applying it to the detection of foreign objects wrapped around the bogie components of the EMU, it can greatly reduce the intensity of manual labor, shorten the maintenance operation time, improve the maintenance work efficiency, reduce the probability of missed detection, reduce labor costs and management costs, thereby ensuring the safe operation of the EMU. The method based on the present invention also uses labeled data to train the target detector to ensure the recognition accuracy of the trained target detector, thereby further improving the accuracy of the detection results.

[0016] Based on the above, in step 1), the encoder is a ResNeXt network model.

[0017] Based on the above, in step 1), the data enhancement processing includes: the data enhancement methods of MixUp and CutMix, and the multimodal data enhancement method designed for dynamic train scenes to simulate rain and fog, low light at night, and dynamic blur conditions.

[0018] The purpose of data enhancement in the present invention is to increase the diversity and robustness of training data. For the original unlabeled image data, different views of the same image can be generated through data enhancement operations, so that the model can learn the invariant characteristics of the image under different transformations, so as to better cope with various situations that may arise in practical applications. The present invention not only adopts the data enhancement methods of MixUp and CutMix to mix images between different classes, thereby expanding the training data set and improving the model's adaptability to different data distributions. It also takes into account the extreme conditions unique to EMUs. Because faults under extreme conditions pose a greater safety risk to EMUs, the present invention adopts a multimodal data enhancement strategy and designs enhancement methods for EMU scenes, such as simulating rain and fog, low light at night, dynamic blur and other extreme conditions, to ensure that the trained model can also obtain accurate detection results in extreme scenes.

[0019] Based on the above, in step 1), the optimization loss function is: maximize the similarity between positive sample pairs, while minimizing the similarity between negative sample pairs;

[0020] The positive sample pairs are: for each original image, multiple views obtained after data enhancement are regarded as positive sample pairs;

[0021] The negative sample pairs are: views from different original images are regarded as negative sample pairs.

[0022] Based on the above, in step 1), the loss function is:

[0023]

[0024] Among them, f(x) is the image feature extracted by the encoder, f(x + ) is the feature representation of the positive sample, f(x i ) is the feature representation of the negative sample, and τ is the temperature parameter. f(x) T f(x + ) is the positive sample similarity, f(x) T f(x i ) is the feature representation of negative samples, and N is the number of negative samples.

[0025] Based on the above, in step 2), the target detection probe adopts the Transformer architecture.

[0026] The ResNeXt network used in this paper introduces a dynamic convolution module that dynamically adjusts the convolution kernel parameters based on the input image, making feature extraction more flexible and adaptable to a wider range of targets. This is particularly useful for the diverse forms of foreign objects on trains. Furthermore, the introduced attention mechanism module can better focus on the target area, improving detection readiness and robustness, and optimizing the detection performance of small-volume foreign objects. Due to its simple and efficient design, the ResNeXt network is generally easier to train than other complex models and is also easier to scale to larger network sizes.

[0027] Based on the above, in step 3), when the foreign object probability of the target detection result is greater than the set foreign object detection judgment threshold, there is a foreign object entanglement fault in the EMU bogie component.

[0028] Based on the above, in step 3), the foreign body probability of the target detection result is: the confidence of the target detection probe that the detected target belongs to the category of foreign body.

[0029] The above description is only an overview of the technical solution of the present invention. In order to more clearly understand the technical means of the present invention, which can be implemented in accordance with the contents of the specification, and to make the above and other purposes, features and advantages of the present invention more obvious and easy to understand, the following specifically cites preferred embodiments and describes them in detail with reference to the accompanying drawings. BRIEF DESCRIPTION OF THE DRAWINGS

[0030] Figure 1 Flowchart of the method for detecting foreign objects wrapped around EMU bogie components of the present invention.

[0031] Figure 2 The SimCLR framework structure diagram adopted by the present invention.

[0032] Figure 3 Schematic diagram of the data enhancement process of the present invention.

[0033] Figure 4 Diagram of the target detection probe architecture used in the present invention. DETAILED DESCRIPTION

[0034] The technical solutions of the present invention will be clearly and completely described below in conjunction with specific embodiments. However, it should be understood by those skilled in the art that the embodiments described below are only used to illustrate the present invention and should not be regarded as limiting the scope of the present invention. Based on the embodiments of the present invention, all other embodiments obtained by those of ordinary skill in the art without creative work are within the scope of protection of the present invention.

[0035] Embodiment of a method for detecting foreign matter wrapped around a motor vehicle bogie component based on image recognition

[0036] The method of this embodiment uses the SimCLR algorithm through contrast learning, and utilizes data enhancement, nonlinear transformation, large batch training and other technologies to learn image feature representations with good generalization ability in an unsupervised manner, and then uses the extracted feature map as the input of the target detector to perform foreign object detection.

[0037] The method of this embodiment uses an encoder pre-trained by SimCLR to obtain a feature map of the image to be detected, and then inputs the feature map into the trained target detector to obtain a target detection result. The result can be used to determine whether there is a foreign object fault wrapped around the EMU bogie component, thereby completing the EMU bogie component wrapped foreign object detection process. Therefore, the detection process of this embodiment uses a pre-trained model. SimCLR is used in model training to reduce the use of training sets of labeled data, thereby reducing the human and material costs generated by a large amount of fault sample data and labeled files. Specifically, model training includes data preparation, building a SimCLR model, model training, and foreign object detection steps:

[0038] 1) Data preparation.

[0039] The method of this embodiment requires training of a feature extraction model and a target detection model. The training of the feature extraction model in this embodiment is performed based on unlabeled data using a self-supervised training method, while the training of the target detection model requires labeled data. In the target detection task, the SimCLR pre-trained model serves as the feature extractor of the target detector. Therefore, in order to ensure the accuracy of detection using the trained model, it is necessary to first train the SimCLR pre-trained feature extractor so that it can fully learn how to distinguish different image views and extract representative and discriminative features to provide a basis for subsequent foreign object detection, thereby better identifying foreign objects. Therefore, the method of this embodiment mainly trains the SimCLR pre-trained model during the training process, and then trains the target detector to complete the fine-tuning of the entire model. Therefore, the training set data used in the SimCLR pre-trained model is more than the training set used in the target detector training.

[0040] The data used in this example comes from over 150 trains passing daily on the Guiyang-Guangzhou railway, capturing foreign object images over the past year. Unlabeled data outnumbers labeled data by at least three times. Approximately 1,000 images are labeled, and 5,000 are unlabeled. The unlabeled data is used for SimCLR model training, while the labeled data is used for detection head training, completing the fine-tuning of the entire model.

[0041] In the foreign object detection task, data enhancement is to increase the diversity and robustness of training data. For the original unlabeled image data, data enhancement operations can generate different views of the same image, allowing the model to learn the invariance characteristics of the image under different transformations, so as to better cope with various situations that may arise in practical applications. Data enhancement methods such as MixUp and CutMix are used to improve the model's adaptability to different data distributions. At the same time, a multimodal data enhancement strategy is adopted, and enhancement methods are designed for dynamic train scenes, such as simulating rain and fog, low light at night, dynamic blur and other extreme conditions. Figure 3 Schematic diagram of the data enhancement strategy of this embodiment.

[0042] 2) Build the SimCLR model.

[0043] like Figure 2 The figure shows the framework diagram of the SimCLR model, which includes data enhancement, encoder, feature projection and loss calculation parts. The encoder of this embodiment adopts the ResNeXt network. By introducing a dynamic convolution module, the ResNeXt network can dynamically adjust the convolution kernel parameters according to the input image, making feature extraction more flexible and adaptable to more diverse targets. It is especially applicable to the different forms of foreign objects on trains. In addition, the introduced attention mechanism module can focus more on the target area, improve detection readiness and robustness, and optimize the detection performance of foreign objects with relatively small volumes. Due to the simplicity and efficiency of its design, the ResNeXt network is usually easier to train than other complex models, and is also easier to expand to a larger network scale.

[0044] The image is fed into the feature extractor ResNeXt network to obtain a high-dimensional feature vector. SimCLR trains this feature extractor in an unsupervised manner, allowing it to learn a universal image feature representation. In foreign object detection, this means that the model can learn the characteristic patterns of normal products or scenes, as well as the potential characteristic patterns related to foreign objects. Through training on a large amount of unlabeled data, the feature extractor can gradually extract representative and discriminative features, providing a basis for subsequent foreign object detection. For each original image, multiple views obtained after data augmentation are regarded as positive sample pairs, while views from different original images are regarded as negative sample pairs. By continuously optimizing the InfoNCE loss function, it is used to maximize the similarity between positive sample pairs and minimize the similarity between negative sample pairs. The model can learn how to distinguish between different image views, thereby better identifying foreign objects.

[0045]

[0046] Among them, f(x) is the image feature extracted by the encoder, f(x + ) is the feature representation of the positive sample, f(xi ) is the feature representation of the negative sample, and τ is the temperature parameter. f(x) T f(x + ) is the positive sample similarity, f(x) T f(x i ) is the feature representation of negative samples, and N is the number of negative samples.

[0047] 3) Model training.

[0048] Unsupervised pre-training is performed using a large amount of unlabeled data. Data-augmented images are fed into the SimCLR model, which is trained by optimizing the contrastive loss function. During training, the model gradually learns common feature representations of images, which are very helpful for object detection tasks.

[0049] 4) Foreign body detection.

[0050] Based on the trained SimCLR model, the MLP layer in the encoder is replaced with an object detection head, which is executed using the Transformer architecture. The architecture diagram is as follows: Figure 4 As shown in Figure 2, the acquired feature map is flattened into a sequence form as the input of the Transformer. Features are extracted through a multi-layer self-attention mechanism and a feedforward neural network. Finally, the target category and bounding box are predicted based on the output of the Transformer.

[0051] In the detection head training part, a small amount of supervised data is used, namely the dataset fine-tuning strategy, to complete the model training.

[0052] Based on the trained model obtained during the training process of this embodiment, when the image to be detected is input into the trained model, the model will output the target's category and location information. Based on this information, a bounding box of the target can be drawn on the image, completing the foreign object detection task. Based on local experiments and actual field application statistics, the foreign object detection threshold of this algorithm is set to 0.65.

[0053] like Figure 1 As shown in the figure, based on the above model training, the method for detecting foreign objects wrapped around EMU bogie components mainly includes two parts: SimCLR image feature extraction and foreign object detection. The process of detecting foreign objects wrapped around EMU bogie components using the trained model is as follows:

[0054] Load pre-trained model: Load a model pre-trained by SimCLR, which includes a pre-trained encoder;

[0055] Extract features: Use the SimCLR pre-trained encoder to extract the feature map of the image;

[0056] Use feature maps: Use the extracted feature maps as the input of the target detector to perform target detection, that is, determine whether there is a foreign object fault entangled in the EMU bogie component based on the target detection results. Specifically, if the foreign object probability corresponding to the detection result is greater than the threshold, there is a foreign object fault entangled in the EMU bogie component. In the actual detection process, the trained model will output the category and location information of the target. Based on this information, the bounding box of the target can be drawn on the image to complete the foreign object detection task. The foreign object probability in this embodiment refers to the model's confidence that the detected target belongs to the category of foreign matter. In this embodiment, the foreign object probability is a Transformer as the detection head, and the confidence is output by the classification branch and normalized by the Softmax function.

[0057] The SimCLR image feature extraction of this embodiment, the training process of this image feature extraction mainly uses the unsupervised learning method of contrastive learning to learn useful features by maximizing the similarity between different data-enhanced versions of the same image. The original image is transformed into different image copies, and then these image pairs are input into the encoder as input samples, and the similarity representation of the encoder output in the high-dimensional feature space is calculated. Foreign object detection is to load a target detection head based on the trained SimCLR model, and then use a small amount of supervised data to complete the training of the detection head. Based on the above training process, the image to be detected is sent to the trained network, and the foreign object location and related alarm information are automatically obtained to complete the anomaly detection task.

[0058] Foreign objects entangled in EMU bogie components are a frequent and potentially dangerous type of failure. This embodiment utilizes unsupervised contrastive learning to enhance the feature extraction capabilities of object detectors, thereby improving object detection accuracy and achieving foreign object detection. SimCLR uses contrastive learning to learn image representations that capture the semantic information and spatial relationships of objects within the image. In object detection tasks, the SimCLR pre-trained model can serve as a feature extractor for the object detector.

[0059] The fault identification method of the present invention has high detection speed and accuracy for foreign object entanglement faults in EMU bogie components. It also significantly reduces the labor and material costs associated with large amounts of fault sample data and annotation files. The application of this intelligent identification method can significantly reduce manual workload, shorten maintenance work time, improve maintenance efficiency, reduce the probability of missed detections, and lower labor and management costs, thereby ensuring the safe operation of EMUs.

[0060] Although the present invention has been described in detail above using general descriptions and specific embodiments, it will be apparent to those skilled in the art that modifications and improvements may be made thereto. Therefore, such modifications and improvements, without departing from the spirit of the present invention, are intended to be within the scope of protection claimed herein.

Claims

1. A method for detecting foreign matter wrapped around a motor vehicle bogie component based on image recognition, characterized in that: The steps include: 1) Input the image to be detected of the EMU bogie component into the encoder pre-trained by SimCLR to obtain the feature map of the image to be detected; The SimCLR pre-training process includes data augmentation of the training set, feature extraction of the augmented data using an encoder, and training of the feature extraction model by optimizing a loss function. The training set is an unlabeled foreign object fault dataset of bogie components. 2) using the feature map as input to the trained target detector to obtain a target detection result; The object detector training process involves replacing the MLP layer in the encoder with an object detection probe based on the pre-trained SimCLR encoder model. This object detection probe is trained using a dataset of labeled foreign object faults on bogie components. 3) Determine whether there is a foreign object entanglement fault in the EMU bogie components based on the target detection results.

2. The method for detecting foreign matter wrapped around a motor vehicle bogie component according to claim 1, characterized in that: In step 1), the encoder is a ResNeXt network model.

3. The method for detecting foreign matter wrapped around a motor vehicle bogie component according to claim 1, characterized in that: In step 1), the data enhancement processing includes: the data enhancement methods of MixUp and CutMix, and a multimodal data enhancement method designed for dynamic train scenes to simulate rain and fog, low light at night, and dynamic blur conditions.

4. The method for detecting foreign matter wrapped around a motor vehicle bogie component according to claim 1, characterized in that: In step 1), the optimization loss function is: maximize the similarity between positive sample pairs and minimize the similarity between negative sample pairs; The positive sample pairs are: for each original image, multiple views obtained after data enhancement are regarded as positive sample pairs; The negative sample pairs are: views from different original images are regarded as negative sample pairs.

5. The method for detecting foreign matter wrapped around a motor vehicle bogie component according to claim 4, characterized in that: In step 1), the loss function is: Among them, f(x) is the image feature extracted by the encoder, f(x + ) is the feature representation of the positive sample, f(x i ) is the feature representation of the negative sample, and τ is the temperature parameter. f(x) T f(x + ) is the positive sample similarity, f(x) T f(x i ) is the feature representation of negative samples, and N is the number of negative samples.

6. The method for detecting foreign matter wrapped around a motor vehicle bogie component according to claim 1, characterized in that: In step 2), the target detection probe adopts the Transformer architecture.

7. The method for detecting foreign matter wrapped around a motor vehicle bogie component according to claim 1, characterized in that: In step 3), when the foreign object probability of the target detection result is greater than the set foreign object detection judgment threshold, there is a foreign object entanglement fault in the EMU bogie component.

8. The method for detecting foreign matter wrapped around a motor vehicle bogie component according to claim 7, characterized in that: In step 3), the foreign body probability of the target detection result is: the confidence of the target detection probe that the detected target belongs to the category of foreign body.

Images