TFDS bogie image fault detection method based on end-to-end model
By directly detecting railway freight car bogie faults using an end-to-end model, the problem of high labor costs caused by the diversity of train models and components in existing technologies is solved, and efficient fault detection is achieved.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- HARBIN KEJIA GENERAL MECHANICAL & ELECTRICAL CO LTD
- Filing Date
- 2026-01-09
- Publication Date
- 2026-04-21
AI Technical Summary
Existing deep learning-based methods for detecting faults in railway freight car bogies cannot meet the diverse needs of different car models and components. They require the use of manually set rules to improve detection accuracy, resulting in high labor costs.
An end-to-end detection method based on ResNet50 network and Faster RCNN model is adopted. By extracting the overall features and vehicle model information of the bogie image and combining the pixel position and color information, the fault detection results are directly output without the need for manual rule setting.
It enables fault detection without the need for manually setting rules, reducing manual intervention and program development costs, and improving detection accuracy and efficiency.
Smart Images
Figure CN121904007A_ABST
Abstract
Description
Technical Field
[0001] This invention relates to a TFDS bogie image fault detection method based on an end-to-end model, belonging to the field of railway freight car fault detection technology. Background Technology
[0002] With the continuous growth of railway freight volume, the safety monitoring of freight cars has become particularly important. Currently, there are products available for fault detection using railway freight car images. These products primarily employ artificial intelligence algorithms to detect fault images, reducing the amount of manual image interpretation. However, due to the large number of railway freight car models and components, and the diverse types of faults, directly using deep learning predictions as output is often insufficient to meet practical needs. Therefore, it is also necessary to manually set detection rules based on the specific train model and component being detected, using logical judgments to improve detection accuracy. This requires programmers to have a deep understanding of the vehicle and component structures and fault types, and necessitates significant human resources for continuous optimization and maintenance of the detection logic. Summary of the Invention
[0003] To address the issue that existing deep learning-based bogie fault detection methods cannot meet the diverse needs of different vehicle models and components, and require the integration of manually defined rules, this invention provides a TFDS bogie image fault detection method based on an end-to-end model.
[0004] The present invention provides a TFDS bogie image fault detection method based on an end-to-end model, comprising,
[0005] The detection model is based on the ResNet50 network and the Faster R-CNN model.
[0006] The overall features of the bogie image were extracted using a ResNet50 network and then input into the detection head of the Faster R-CNN model.
[0007] The vehicle model information is encoded and then input into the detection head of the Faster R-CNN model;
[0008] The pixel location and color information of the bogie image are input into the backbone network of the Faster R-CNN model, and the resulting fused features are input into the detection head.
[0009] After the detection model is trained, the detection head outputs the fault detection results.
[0010] According to the TFDS bogie image fault detection method based on an end-to-end model of the present invention, the method for determining the pixel position information of the bogie image is as follows:
[0011] In the bogie image, the horizontal pixel coordinate of the first axis is set to x1, the horizontal pixel coordinate of the second axis is set to x2, and the horizontal pixel coordinate of the pixel position is represented as x, and the vertical pixel coordinate is represented as y.
[0012] The pixel position information is determined by the relative positional relationship between x and x1 and x2, and the correspondence between y and the bogie image height h.
[0013] According to the TFDS bogie image fault detection method based on the end-to-end model of the present invention, the pixel position information is input into the backbone network of the Faster R-CNN model through three channels, namely:
[0014] Channel 1: (x1-x) / (x2-x1);
[0015] Channel 2: (x2-x) / (x2-x1);
[0016] Channel 3: y / h.
[0017] According to the TFDS bogie image fault detection method based on the end-to-end model of the present invention, the color information of the bogie image includes the RGB information of the pixels, which is input to the backbone network of the Faster R-CNN model through three basic color channels.
[0018] According to the TFDS bogie image fault detection method based on the end-to-end model of the present invention, the vehicle model information is encoded into an encoded vector by one-hot encoding and then input into the detection head of the Faster RCNN model.
[0019] According to the TFDS bogie image fault detection method based on an end-to-end model of the present invention, the vehicle models include 97 types.
[0020] According to the TFDS bogie image fault detection method based on the end-to-end model of the present invention, 97 types of vehicle information are encoded one-hot to obtain a 97-dimensional encoding vector, and then the dimensionality is reduced by a fully connected layer to obtain a 64-dimensional encoding vector, which is then input into the detection head of the Faster R-CNN model.
[0021] According to the TFDS bogie image fault detection method based on the end-to-end model of the present invention, the bogie image is scaled to the target size and then the overall features are extracted by the ResNet50 network.
[0022] According to the TFDS bogie image fault detection method based on the end-to-end model of the present invention, the dimension of the overall feature is 2048.
[0023] According to the TFDS bogie image fault detection method based on the end-to-end model of the present invention, the 2048-dimensional overall features are reduced to 64-dimensional overall features through a fully connected layer and input into the detection head of the Faster R-CNN model.
[0024] The beneficial effects of this invention are as follows: The bogie image can be obtained using a high-definition camera, and the detection model is used to detect bogie faults. The input to the detection model includes the bogie image, the position of each pixel in the image relative to the two axes of the bogie, and vehicle model information. The detection model directly outputs the fault type and fault location based on the input information, without requiring rule-based logical judgments.
[0025] The method of this invention directly learns the hidden relationship between fault mode, location and vehicle model through end-to-end detection model and input data, without the need for manually setting rules, thus reducing manual intervention and program development costs. Attached Figure Description
[0026] Figure 1 This is a flowchart of the TFDS bogie image fault detection method based on an end-to-end model as described in this invention;
[0027] Figure 2 It is an image of a bogie. Detailed Implementation
[0028] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.
[0029] Specific Implementation Method 1: Combination Figure 1 As shown, this invention provides a TFDS bogie image fault detection method based on an end-to-end model, including,
[0030] The detection model is based on the ResNet50 network and the Faster R-CNN model.
[0031] The overall features of the bogie image were extracted using a ResNet50 network and then input into the detection head of the Faster R-CNN model.
[0032] The vehicle model information is encoded and then input into the detection head of the Faster R-CNN model;
[0033] The pixel location and color information of the bogie image are input into the backbone network of the Faster R-CNN model, and the resulting fused features are input into the detection head.
[0034] After the detection model is trained, the detection head outputs the fault detection results.
[0035] This implementation method adds position information and vehicle model information to the model input based on the bogie image, enabling the model to learn the hidden relationship between fault modes, locations, and vehicle models through a large amount of data.
[0036] Furthermore, the method for determining the pixel position information of the bogie image is as follows:
[0037] In the bogie image, the horizontal pixel coordinate of the first axis is set to x1, the horizontal pixel coordinate of the second axis is set to x2, and the horizontal pixel coordinate of the pixel position is represented as x, and the vertical pixel coordinate is represented as y.
[0038] The pixel position information is determined by the relative positional relationship between x and x1 and x2, and the correspondence between y and the bogie image height h.
[0039] The pixel location information is input into the backbone network of the Faster R-CNN model through three channels:
[0040] Channel 1: (x1-x) / (x2-x1);
[0041] Channel 2: (x2-x) / (x2-x1);
[0042] Channel 3: y / h.
[0043] The color information of the bogie image includes the RGB information of the pixels, which is input into the backbone network of the Faster R-CNN model through the three basic color channels.
[0044] In this embodiment, the position of each pixel relative to the two axes of the bogie is used as position information and merged with the normalized image channels, resulting in a total of six channels input to the backbone network. The three channels of position information are the pixel's horizontal position relative to the first axis, the pixel's horizontal position relative to the second axis, and the pixel's vertical position. Combined with... Figure 2 As shown, the image height is h, the first red vertical line corresponds to the horizontal coordinate x1 of the first axis, and the second red vertical line corresponds to the horizontal coordinate x2 of the second axis.
[0045] Furthermore, the vehicle model information is encoded into an encoded vector through one-hot encoding and then input into the detection head of the Faster R-CNN model.
[0046] As an example, the vehicle models mentioned include 97 different models.
[0047] Information on 97 vehicle models is encoded using one-hot encoding to obtain a 97-dimensional encoding vector, which is then reduced to a 64-dimensional encoding vector by a fully connected layer and input into the detector head of the Faster R-CNN model.
[0048] Furthermore, after the bogie image is scaled to the target size, its overall features are extracted using a ResNet50 network.
[0049] As an example, the overall feature has 2048 dimensions.
[0050] The 2048-dimensional overall features are reduced to 64-dimensional overall features through a fully connected layer and then input into the detector head of the Faster R-CNN model.
[0051] Besides vehicle model information, the same vehicle model may correspond to different bogie types. This implementation extracts the overall features of the bogie image using a ResNet50 network.
[0052] The 64-dimensional overall features, the 64-dimensional encoded vector, and the fused features based on image ROI features obtained through the backbone network are combined to form a 384-dimensional vector, which is used as the input to the network detection head.
[0053] For training the detection model, a large number of railway freight car bogie fault images can be prepared, and the fault locations and names in the images can be labeled. The model can then be trained using the training dataset to obtain an end-to-end target detection model.
[0054] The image of the railway freight car bogie is processed by a pre-detection model to obtain the accurate coordinates of the two axes in the image. Based on the coordinates of the two axes, the image pixel position information is obtained through processing. The image, vehicle type information, and position information are input into the detection model to directly obtain the predicted fault location and fault type, which are then used as the final fault alarm output.
[0055] While the invention has been described herein with reference to specific embodiments, it should be understood that these embodiments are merely examples of the principles and applications of the invention. Therefore, it should be understood that many modifications can be made to the exemplary embodiments, and other arrangements can be designed without departing from the spirit and scope of the invention as defined by the appended claims. It should be understood that different dependent claims and features described herein can be combined in ways different from those described in the original claims. It is also understood that features described in conjunction with individual embodiments can be used in other described embodiments.
Claims
1. A TFDS bogie image fault detection method based on an end-to-end model, characterized in that... include, The detection model is based on the ResNet50 network and the Faster R-CNN model. The overall features of the bogie image were extracted using a ResNet50 network and then input into the detection head of the Faster R-CNN model. The vehicle model information is encoded and then input into the detection head of the Faster R-CNN model; The pixel location and color information of the bogie image are input into the backbone network of the Faster R-CNN model, and the resulting fused features are input into the detection head. After the detection model is trained, the detection head outputs the fault detection results.
2. The TFDS bogie image fault detection method based on an end-to-end model according to claim 1, characterized in that, The method for determining the pixel position information of the bogie image is as follows: In the bogie image, the horizontal pixel coordinate of the first axis is set to x1, the horizontal pixel coordinate of the second axis is set to x2, and the horizontal pixel coordinate of the pixel position is represented as x, and the vertical pixel coordinate is represented as y. The pixel position information is determined by the relative positional relationship between x and x1 and x2, and the correspondence between y and the bogie image height h.
3. The TFDS bogie image fault detection method based on an end-to-end model according to claim 2, characterized in that, The pixel location information is input into the backbone network of the Faster R-CNN model through three channels: Channel 1: (x1-x) / (x2-x1); Channel 2: (x2-x) / (x2-x1); Channel 3: y / h.
4. The TFDS bogie image fault detection method based on an end-to-end model according to claim 3, characterized in that, The color information of the bogie image includes the RGB information of the pixels, which is input into the backbone network of the Faster R-CNN model through the three basic color channels.
5. The TFDS bogie image fault detection method based on an end-to-end model according to claim 1, characterized in that, The vehicle model information is encoded into an encoded vector using one-hot encoding and then input into the detection head of the Faster R-CNN model.
6. The TFDS bogie image fault detection method based on an end-to-end model according to claim 5, characterized in that, The vehicle models include 97 different models.
7. The TFDS bogie image fault detection method based on an end-to-end model according to claim 6, characterized in that, Information on 97 vehicle models is encoded using one-hot encoding to obtain a 97-dimensional encoding vector, which is then reduced to a 64-dimensional encoding vector by a fully connected layer and input into the detector head of the Faster R-CNN model.
8. The TFDS bogie image fault detection method based on an end-to-end model according to claim 1, characterized in that, The bogie image is scaled to the target size and then its overall features are extracted using a ResNet50 network.
9. The TFDS bogie image fault detection method based on an end-to-end model according to claim 8, characterized in that, The overall feature has 2048 dimensions.
10. The TFDS bogie image fault detection method based on an end-to-end model according to claim 9, characterized in that, The 2048-dimensional overall features are reduced to 64-dimensional overall features through a fully connected layer and then input into the detector head of the Faster R-CNN model.