A bullet train bogie component winding foreign matter detection method based on image recognition

Abstract

The application discloses a motor train bogie component winding foreign matter detection method based on image recognition, and belongs to the technical field of railway motor train operation state fault detection. The method trains a feature extractor by using a SimCLR model, so as to train the feature extractor by using an unannotated data set, and then a small amount of annotated data set is used to train a target detector, so as to complete the whole model training process. The trained model is used to input a to-be-detected image into the model, so as to realize motor train bogie component winding foreign matter detection. Most of the training sets used by the method are unannotated training sets, so that the human and material costs generated by a large number of fault sample data and annotation files are greatly reduced.

Description

Technical Field

[0001] This invention relates to the field of fault detection technology for railway EMU operation status, specifically to a method for detecting foreign objects entangled in EMU bogie components based on image recognition. Background Technology

[0002] Currently, to ensure the safe operation of high-speed trains, a Train Operation Fault Image Detection System (TEDS) has been installed on high-speed rail operating lines and train depot entry and exit lines. This system is mainly an intelligent system integrating high-speed digital image acquisition, real-time processing of large-capacity image data, precise positioning, and pattern recognition technology. The system consists of equipment for detection information acquisition, information processing and transmission, and a train inspection center. The system uses a high-speed camera array to capture all visible information about the undercarriage and lower sides of the train. After digital processing, the images are displayed on the information terminal at the inspection center. Then, relevant train inspection personnel use the images displayed on the terminal to determine the appearance and condition of key components during train operation to diagnose faults.

[0003] Currently, only a very small number of components in the system have achieved automatic fault diagnosis, and the system still suffers from low fault detection rate and low alarm accuracy. Furthermore, a large number of components on the underside of the train still rely on manual inspection of images to confirm faults. With the continuous increase in the operating mileage and train formations of high-speed trains, on-site personnel face heavy workloads and require high concentration, leading to fatigue. Simultaneously, the number of personnel in the TEDS monitoring centers of various railway bureaus is gradually increasing, resulting in rising labor costs. Faced with these challenges, there is an urgent need to strengthen and optimize the intelligent image recognition algorithm for faults in key components under the high-speed train through new technologies and methods. This would allow on-site operations to gradually transition from a system primarily based on human inspection with machine inspection as a supplement to one primarily based on machine inspection with human inspection as a supplement, thereby reducing the workload of train inspection personnel, improving fault detection effectiveness, and ensuring the safe operation of high-speed trains.

[0004] Currently, most methods for detecting fault states in high-speed trains addressing the aforementioned issues rely on image processing techniques. For individual components, they employ feature extraction and traditional machine learning methods to extract fault state features. These features are then combined with human experience and prior knowledge, and a traditional machine learning classifier is used for fault diagnosis. Several related patents have been found, with the following general content:

[0005] The Chinese invention patent application with application number 202310310547.0 and application date of March 27, 2023, entitled "Method, System and Device for Foreign Object Detection on EMU Based on Object Detection Network", discloses that a training dataset is first constructed, then an object detection model is trained based on it (the binary cross-entropy loss function of the YOLOv5 object detection model is replaced with a single classification loss function), and finally the trained model is used to detect the image under test to determine whether there is a foreign object.

[0006] Chinese invention patent application No. 202311742519.2, filed on December 18, 2023, entitled "A Semi-Supervised Method for Foreign Object Detection on Railway Track Based on Inverted Perspective View," discloses obtaining an inverted perspective view with the same resolution as the camera and integrating it into a teacher-student model architecture. During the model training phase, only normal railway line sample data is used. The inverted perspective view is scaled and multiplied point-by-point with the intermediate layer feature map of the student network, and then knowledge distillation is performed with the feature map corresponding to the teacher network. During the testing phase, multi-scale anomaly maps are constructed using the differences in intermediate layer features between the teacher and student networks. Finally, the specific location of the foreign object in the image is determined by the anomaly map.

[0007] Currently, most existing technologies and patents, including those related to practical applications, utilize traditional machine learning methods, such as image processing and recognition, and deep learning object detection methods, such as the YOLO series. Both of these methods suffer from the following drawbacks: Traditional machine learning image processing and recognition methods require combining prior knowledge and human experience to analyze the morphology of each component. Their recognition accuracy heavily relies on the algorithm, resulting in poor fault accuracy and algorithm robustness, and they are easily affected by changes in the external environment. Deep learning object detection methods require building a training database containing a large amount of fault sample data and labeled files, which consumes significant human and material resources. Furthermore, sample diversity has a significant impact on the results. Summary of the Invention

[0008] In view of this, the purpose of this invention is to provide an image recognition-based method for detecting foreign objects entangled in the bogie components of high-speed trains, in order to solve the problem that existing deep learning-based target detection methods require a large amount of manually labeled fault sample data as a training set to train the model in order to ensure the accuracy of the detection results of the trained model. This results in a large amount of manpower and resources being consumed in the data labeling process of a large number of fault samples.

[0009] To achieve the above objectives, the method of this invention employs the SimCLR model, enabling the training of the feature extractor using an unlabeled training set. Specifically, the method includes the following steps:

[0010] 1) Input the image of the bogie component to be detected into the encoder pre-trained with SimCLR to obtain the feature map of the image to be detected;

[0011] The SimCLR pre-training process includes data augmentation of the training set, feature extraction of the augmented data using an encoder, and training of the feature extraction model by optimizing the loss function. The training set is a dataset of foreign object faults of unlabeled bogie components.

[0012] 2) The feature map is used as input to the trained target detector, and the target detection result is obtained after passing through the target detector;

[0013] The training process for the target detector includes: on the basis of the pre-trained SimCLR model of the encoder, the MLP layer in the encoder is replaced with the target detection probe, and the target detection probe is trained using a foreign object fault dataset of labeled bogie components.

[0014] 3) Determine whether there is a foreign object entanglement fault in the bogie components of the EMU based on the target detection results.

[0015] The method of this invention has the following advantages: The method uses the SimCLR model for encoder training with an unlabeled training set, thus greatly reducing the manpower and material costs associated with large amounts of fault sample data and labeled files. The model trained using this method, when applied to the detection of foreign objects entangled in train bogie components, significantly reduces manual workload, shortens maintenance time, improves maintenance efficiency, reduces the probability of missed detections, and lowers labor and management costs, thereby ensuring the safe operation of the train. Furthermore, the method of this invention also uses labeled data to train the target detector to ensure the accuracy of the trained target detector, further improving the accuracy of the detection results.

[0016] Based on the above, in step 1), the encoder is a ResNeXt network model.

[0017] Based on the above, in step 1), the data augmentation process includes: data augmentation methods of MixUp and CutMix, and multimodal data augmentation methods designed for high-speed train scenarios under simulated rain and fog, low light at night, and dynamic blur conditions.

[0018] In this invention, data augmentation aims to increase the diversity and robustness of training data. For raw, unlabeled image data, data augmentation operations can generate different views of the same image, allowing the model to learn the invariant features of the image under different transformations, thereby better handling various situations that may arise in practical applications. This invention not only employs MixUp and CutMix data augmentation methods to mix images from different classes, thereby expanding the training dataset and improving the model's adaptability to different data distributions, but also considers the unique extreme conditions of high-speed trains. Because malfunctions under extreme conditions pose a greater safety risk to high-speed trains, this invention adopts a multimodal data augmentation strategy, designing augmentation methods specifically for high-speed train scenarios, such as simulating extreme conditions like rain, fog, low light at night, and dynamic blur, to ensure that the trained model can obtain accurate detection results even in extreme scenarios.

[0019] Based on the above, in step 1), the optimized loss function is: to maximize the similarity between positive sample pairs while minimizing the similarity between negative sample pairs;

[0020] The positive sample pair is defined as follows: for each original image, after data augmentation, multiple views are obtained and regarded as positive sample pairs.

[0021] The negative sample pair is defined as: views from different original images are considered negative sample pairs.

[0022] Based on the above, in step 1), the loss function is:

[0023]

[0024] Where f(x) is the image feature extracted by the encoder, f(x) + f(x) is the feature representation of a positive sample. i f(x) represents the feature representation of the negative samples, and τ is the temperature parameter. T f(x + ) represents the positive sample similarity, f(x) T f(x i ) is the feature representation of the negative sample, and N is the number of negative samples.

[0025] Based on the above, in step 2), the target detection probe adopts the Transformer architecture.

[0026] The ResNeXt network used in this invention introduces a dynamic convolution module, which can dynamically adjust the convolution kernel parameters according to the input image, making feature extraction more flexible and adaptable to more diverse targets, especially suitable for detecting foreign objects of varying shapes on high-speed trains. Furthermore, the introduced attention mechanism module can focus more intently on the target region, improving detection readiness and robustness, and optimizing the detection performance of smaller foreign objects. Due to its simple design and high efficiency, the ResNeXt network is generally easier to train than other complex models and also easier to scale to larger network sizes.

[0027] Based on the above, in step 3), when the probability of foreign objects in the target detection result is greater than the set foreign object detection judgment threshold, there is a foreign object entanglement fault in the bogie component of the EMU.

[0028] Based on the above, in step 3), the probability of the target detection result being a foreign object is: the confidence level of the target detection probe that the detected target belongs to the category of foreign objects.

[0029] The above description is merely an overview of the technical solution of the present invention. In order to better understand the technical means of the present invention and to implement it in accordance with the contents of the specification, and to make the above and other objects, features and advantages of the present invention more apparent and understandable, preferred embodiments are described below in detail with reference to the accompanying drawings. Attached Figure Description

[0030] Figure 1 Flowchart of the method for detecting foreign objects entangled in the bogie components of a high-speed train according to the present invention.

[0031] Figure 2 The SimCLR framework structure diagram used in this invention.

[0032] Figure 3 A schematic diagram of the data augmentation process of this invention.

[0033] Figure 4 The target detection probe architecture diagram used in this invention. Detailed Implementation

[0034] The technical solution of the present invention will be clearly and completely described below with reference to specific embodiments. However, those skilled in the art should understand that the embodiments described below are only for illustrating the present invention and should not be regarded as limiting the scope of the present invention. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0035] Example of an image recognition-based method for detecting foreign objects entangled in train bogie components

[0036] This embodiment utilizes the SimCLR algorithm to learn image feature representations with good generalization ability through contrastive learning, and employs techniques such as data augmentation, nonlinear transformation, and large-batch training in an unsupervised manner. The extracted feature maps are then used as input to the target detector for foreign object detection.

[0037] This embodiment uses a SimCLR pre-trained encoder to obtain a feature map of the image to be detected. The feature map is then input into a trained target detector to obtain the target detection result. This result can determine whether a foreign object entanglement fault exists in the bogie component of the train, thus completing the foreign object entanglement detection process. Therefore, this embodiment utilizes a pre-trained model. SimCLR is used in model training to reduce the need for labeled training data, thereby reducing the manpower and material costs associated with large amounts of fault sample data and labeled files. Specifically, model training includes data preparation, SimCLR model construction, model training, and foreign object detection steps:

[0038] 1) Data preparation.

[0039] The method in this embodiment requires training both the feature extraction model and the object detection model. The feature extraction model is trained using a self-supervised training method based on unlabeled data, while the object detection model requires labeled data. Since the SimCLR pre-trained model serves as the feature extractor for the object detector in the object detection task, to ensure the accuracy of detection using the trained model, the SimCLR pre-trained feature extractor needs to be trained first. This allows it to fully learn how to distinguish different image views and extract representative and discriminative features, providing a foundation for subsequent foreign object detection and thus better identifying foreign objects. Therefore, the training process in this embodiment primarily focuses on training the SimCLR pre-trained model, and the object detector training is used to fine-tune the entire model. Consequently, the training set used for the SimCLR pre-trained model is larger than the training set used for the object detector.

[0040] The data used in this embodiment comes from foreign object data images of more than 150 trains passing through the Guiyang-Guangzhou railway line daily for about a year. The amount of unlabeled data is at least three times that of labeled data. This embodiment uses approximately 1,000 labeled images and 5,000 unlabeled images. The unlabeled data is used for SimCLR model training, and the labeled data is used for detection head training, that is, to complete the fine-tuning of the entire model.

[0041] In foreign object detection tasks, data augmentation aims to increase the diversity and robustness of training data. For raw, unlabeled image data, data augmentation operations can generate different views of the same image, allowing the model to learn the image's invariant characteristics under different transformations, thus better handling various situations that may arise in real-world applications. Data augmentation methods such as MixUp and CutMix are employed to improve the model's adaptability to different data distributions. Simultaneously, multimodal data augmentation strategies are used, with augmentation methods designed specifically for high-speed train scenarios, such as simulating extreme conditions like rain, fog, low-light conditions at night, and dynamic blur. Figure 3 This is a schematic diagram of the data augmentation strategy in this embodiment.

[0042] 2) Construct the SimCLR model.

[0043] like Figure 2 The diagram shows the SimCLR model framework, including data augmentation, encoder, feature projection, and loss calculation. In this embodiment, the encoder uses a ResNeXt network. The ResNeXt network, by introducing a dynamic convolution module, can dynamically adjust the convolution kernel parameters based on the input image, making feature extraction more flexible and adaptable to more diverse targets, especially suitable for detecting foreign objects of varying shapes on high-speed trains. Furthermore, the introduced attention mechanism module can focus more intently on the target region, improving detection readiness and robustness, and optimizing the detection performance of smaller foreign objects. Due to its simplicity and efficiency, the ResNeXt network is generally easier to train than other complex models and also easier to scale to larger network sizes.

[0044] Images are fed into the ResNeXt network, a feature extractor, to obtain high-dimensional feature vectors. SimCLR trains this feature extractor unsupervised, enabling it to learn general image feature representations. In foreign object detection, this means the model can learn feature patterns of normal products or scenes, as well as potential feature patterns related to foreign objects. Through training on a large amount of unlabeled data, the feature extractor gradually extracts representative and discriminative features, providing a foundation for subsequent foreign object detection. For each original image, multiple views obtained after data augmentation are treated as positive sample pairs, while views from different original images are treated as negative sample pairs. By continuously optimizing the InfoNCE loss function to maximize the similarity between positive sample pairs while minimizing the similarity between negative sample pairs, the model learns how to distinguish different image views, thus better identifying foreign objects.

[0045]

[0046] Where f(x) is the image feature extracted by the encoder, f(x) + f(x) is the feature representation of a positive sample.i f(x) represents the feature representation of the negative samples, and τ is the temperature parameter. T f(x + ) represents the positive sample similarity, f(x) T f(x i ) is the feature representation of the negative sample, and N is the number of negative samples.

[0047] 3) Model training.

[0048] Unsupervised pre-training is performed using a large amount of unlabeled data. Data-augmented images are then fed into the SimCLR model, which is trained by optimizing the contrastive loss function. During training, the model gradually learns general feature representations of the images, which are highly helpful for object detection tasks.

[0049] 4) Foreign object detection.

[0050] Based on the trained SimCLR model, the MLP layer in the encoder is replaced with an object detection head, which uses a Transformer architecture. The architecture diagram is as follows. Figure 4 As shown, the acquired feature maps are flattened into a sequence and used as input to the Transformer. Features are extracted through a multi-layer self-attention mechanism and a feedforward neural network. Finally, based on the output of the Transformer, the target category and bounding box are predicted.

[0051] The training of the detection head utilizes a small amount of supervised data, i.e., a dataset fine-tuning strategy, to complete the model training.

[0052] The trained model obtained through the training process in this embodiment outputs the target's category and location information when the image to be detected is input into the trained model. Based on this information, the bounding box of the target can be drawn on the image, completing the foreign object detection task. Based on local experiments and statistical analysis of actual field applications, the foreign object detection threshold of this algorithm is set to 0.65.

[0053] like Figure 1 As shown, based on the above model training, the method for detecting foreign objects wrapped around EMU bogie components mainly includes two parts: SimCLR image feature extraction and foreign object detection. The process of using the trained model to detect foreign objects wrapped around EMU bogie components is as follows:

[0054] Load pre-trained model: Load a model pre-trained by SimCLR, which contains a pre-trained encoder;

[0055] Feature extraction: Extract feature maps from the image using a SimCLR pre-trained encoder;

[0056] Feature Map Usage: The extracted feature maps are used as input to the target detector for target detection. Specifically, the detection results determine whether a foreign object entanglement fault exists on the bogie components of the train. If the probability of the detected foreign object is greater than a threshold, a foreign object entanglement fault exists on the bogie components. In actual detection, the trained model outputs the target's category and location information. Based on this information, the target's bounding box can be drawn on the image, completing the foreign object detection task. In this embodiment, the foreign object probability refers to the model's confidence that the detected target belongs to the foreign object category. In this embodiment, the foreign object probability is generated by the Transformer as the detection head, the confidence is output by the classification branch, and normalized using the Softmax function.

[0057] This embodiment of SimCLR image feature extraction primarily utilizes a contrastive learning-based unsupervised learning method during training. It learns useful features by maximizing the similarity between different data-enhanced versions of the same image. The original image is transformed into different image copies, and these image pairs are then used as input samples into the encoder. The similarity representation of the encoder output in the high-dimensional feature space is calculated. Foreign object detection involves loading a target detection head onto the trained SimCLR model and then training the detection head using a small amount of supervised data. Based on the above training process, the image to be detected is fed into the trained network, which automatically acquires the location of the foreign object and related alarm information, thus completing the anomaly detection task.

[0058] Foreign object entanglement in train bogie components is a frequent and potentially dangerous type of failure. This embodiment utilizes unsupervised contrastive learning to enhance the feature extraction capability of the target detector, thereby improving the accuracy of target detection and achieving the goal of foreign object detection. SimCLR learns image representations through a contrastive learning mechanism, and these representations can capture the semantic information and spatial relationships of objects in the image. In the target detection task, the SimCLR pre-trained model can serve as the feature extractor for the target detector.

[0059] The fault identification method of this invention has a high detection speed and accuracy in detecting foreign object entanglement faults in EMU bogie components. At the same time, it significantly reduces the manpower and material costs associated with large amounts of fault sample data and annotation documents. The application of this intelligent identification method can greatly reduce the workload of manual labor, shorten maintenance operation time, improve maintenance efficiency, reduce the probability of missed detections, and lower labor and management costs, thereby ensuring the safe operation of EMUs.

[0060] Although the present invention has been described in detail above with general descriptions and specific embodiments, modifications or improvements can be made to it, which will be obvious to those skilled in the art. Therefore, all such modifications or improvements made without departing from the spirit of the present invention fall within the scope of protection claimed by the present invention.

Claims

1. A method for detecting foreign objects entangled in a high-speed train bogie component based on image recognition, characterized in that, Includes the following steps: 1) Input the image of the bogie component to be detected into the encoder pre-trained with SimCLR to obtain the feature map of the image to be detected; the encoder is a ResNeXt network model; The SimCLR pre-training process includes data augmentation of the training set, feature extraction of the augmented data using an encoder, and training of the feature extraction model by optimizing the loss function. The training set is a dataset of foreign object faults of unlabeled bogie components. 2) The feature map is used as input to the trained target detector, and the target detection result is obtained after passing through the target detector; The training process for the target detector includes: on the basis of the pre-trained SimCLR model of the encoder, replacing the MLP layer in the encoder with the target detection probe, and training the target detection probe using a labeled foreign object fault dataset of bogie components; the target detection probe adopts the Transformer architecture. 3) Determine whether there is a foreign object entanglement fault in the bogie components of the EMU based on the target detection results.

2. The method for detecting foreign objects entangled in a train bogie component according to claim 1, characterized in that, In step 1), the data augmentation process includes: data augmentation methods MixUp and CutMix, and multimodal data augmentation methods designed for high-speed train scenarios under simulated rain and fog, low light at night, and dynamic blur conditions.

3. The method for detecting foreign objects entangled in a train bogie component according to claim 1, characterized in that, In step 1), the optimized loss function is: to maximize the similarity between positive sample pairs while minimizing the similarity between negative sample pairs; The positive sample pair is defined as follows: for each original image, after data augmentation, multiple views are obtained and regarded as positive sample pairs. The negative sample pair is defined as: views from different original images are considered negative sample pairs.

4. The method for detecting foreign objects entangled in a train bogie component according to claim 3, characterized in that, In step 1), the loss function is: ； in, These are the image features extracted by the encoder. It is a feature representation of positive samples. It is a feature representation of negative samples. It's a temperature parameter. It is positive sample similarity. is the feature representation of the negative samples, and N is the number of negative samples.

5. The method for detecting foreign objects entangled in a train bogie component according to claim 1, characterized in that, In step 3), when the probability of foreign object in the target detection result is greater than the set foreign object detection judgment threshold, there is a foreign object entanglement fault in the bogie component of the EMU.

6. The method for detecting foreign objects entangled in a train bogie component according to claim 5, characterized in that, In step 3), the probability of the target detection result being a foreign object is: the confidence level of the target detection probe in that the detected target belongs to the category of foreign object.

Images