Railway facility fault detection and identification system and method based on high-definition image

Abstract

The invention belongs to the technical field of image processing, and provides a railway facility fault detection and recognition system and method based on a high-definition image. The system comprises an 8K ultra-high-definition camera data acquisition unit, a high-precision positioning unit, an image preprocessing unit, an image segmentation unit, a target identification unit, a fastener part detection unit, a fault fastener identification unit and a data processing terminal which are installed in a running train. According to the railway facility fault detection and identification system and method based on the high-definition image, whether the railway facility has a fault or not can be accurately judged, an alarm is given, and the fault is stored in a database, so that the efficiency and the safety coefficient of railway transportation safety detection are effectively improved, the precise management and control capability is improved, and operation accidents are reduced; and meanwhile, the working intensity of front-line workers can be reduced, and the working safety of the front-line workers is guaranteed.

Description

Technical Field

[0001] The present invention belongs to the technical field of image processing, and in particular relates to a fault detection and identification system and method for railway facilities based on high-definition images. Technical Background

[0002] In railway transportation production, inspection work along the railway is the focus of railway transportation work and a major task to ensure the safety of train operation. The traditional railway personnel inspection method has great limitations, low manual inspection efficiency, many safety hazards, frequent accidents, and is not in line with the development direction of smart railways. Summary of the invention

[0003] The purpose of the present invention is to overcome the deficiencies of the above-mentioned prior art and to provide a railway facility fault detection and identification system and method based on high-definition images, which can accurately detect and identify internal and external environmental information along the railway, and alarm for abnormal faults of railway facilities, so as to realize intelligent, digital and real-time inspection operations along the railway.

[0004] To achieve the above objectives, the present invention adopts the following technical solutions.

[0005] The present invention provides a high-definition image-based fault detection and identification system for railway facilities, comprising:

[0006] An 8K ultra-high-definition camera data acquisition unit installed inside a running train to collect environmental data along the railway line;

[0007] The high-precision positioning unit, combined with the sub-meter Beidou high-precision positioning system, is used to give location information to the collected images;

[0008] The image preprocessing unit performs image enhancement processing and divides the pixel areas where the facilities along the railway are located, so as to improve the image segmentation and recognition speed;

[0009] The image segmentation unit performs image segmentation on the entire acquired image and divides various areas, including protective nets, road cuttings, track areas, etc.

[0010] The target recognition unit identifies abnormal areas of protection nets, road cuts, and tracks in the acquired target area;

[0011] The fastener parts detection unit detects the fastener area from the entire captured image;

[0012] A faulty fastener identification unit identifies faulty abnormal fasteners in the acquired fastener area;

[0013] The data processing terminal analyzes the image recognition results and combines them with a high-precision positioning unit to locate and process abnormal parts of facilities along the railway. If there are any abnormal parts, they are stored in the database and an alarm is sounded.

[0014] The present invention also provides a method for fault detection and identification of railway facilities based on high-definition images, comprising the following steps:

[0015] (1) High-definition image acquisition;

[0016] (2) The high-precision positioning unit assigns position information to the collected images;

[0017] (3) Enhance the image and divide the pixel areas where the facilities along the railway are located;

[0018] (4) Use the YOLO series of algorithms to detect fastener parts, collect a large number of normal fastener image samples, annotate them, train the network, obtain the detection model, input the image into the network model, and then detect the fastener area;

[0019] (5) Identify the fault image of abnormal fasteners, and use the image reconstruction method based on generative adversarial network to detect abnormal fasteners in the fastener area; use normal samples in fastener parts to train the network, let the network learn the reconstructed latent variables of normal samples, and automatically encode the network model and the input fastener test image to obtain the encoded latent variable. When the distance between the reconstructed latent variable and the encoded latent variable is less than the threshold, it is considered that there is no abnormality in the image, otherwise it is considered that the part in the image is abnormal;

[0020] (6) Analyze the entire image, segment each region, and use the mask2former method for instance segmentation:

[0021] (6-1) Obtain a large number of protective net, track, and road cutting image samples, annotate them, and train the model;

[0022] (6-2) Input image, the backbone network extracts low-resolution features from the image;

[0023] (6-3) The pixel decoder gradually upsamples and decodes the low-resolution features to obtain a feature pyramid from low resolution to high resolution, which is cyclically used as the input of V and K in the Transformer decoder. The multi-scale features are used to ensure the prediction accuracy of the model for targets of different scales.

[0024] (6-4) The Transformer decoder with mask attention refines the binary mask map layer by layer by querying the multi-scale features obtained from the object and pixel decoder to obtain the final result;

[0025] (7) For the segmented protection net, cutting, and rail area, the deep support vector data description network (DeepSVDD) is used to identify the fault area;

[0026] (8) Issue an alarm for abnormal faults and store them in the database.

[0027] The railway facility fault detection and identification system and method based on high-definition images of the present invention can make accurate judgments on whether there are faults in railway facilities and issue an alarm, which is stored in a database, effectively improving the efficiency and safety factor of railway transportation safety inspection, improving precise management and control capabilities, and reducing the occurrence of operational accidents. At the same time, it can also reduce the workload of front-line operators and ensure the operational safety of front-line employees. BRIEF DESCRIPTION OF THE DRAWINGS

[0028] Figure 1 It is a flow chart of the fault detection and identification method of railway facilities based on high-definition images of the present invention. DETAILED DESCRIPTION

[0029] In order to make the purpose, technical solution and advantages of the present invention more clearly understood, the present invention is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present invention and are not used to limit the present invention.

[0030] like Figure 1 As shown, an embodiment of the present invention provides a fault detection and identification system for railway facilities based on high-definition images, comprising:

[0031] The 8K ultra-high-definition camera data acquisition unit is installed on the front windshield of the train and is used to collect high-definition images of railway facilities such as rails, cuttings, and protective nets in real time during the operation of the train.

[0032] The high-precision positioning unit, combined with the sub-meter Beidou high-precision positioning system, is used to assign location information to the collected images.

[0033] The image preprocessing unit performs image enhancement processing and divides the pixel areas where the facilities along the railway are located.

[0034] The image segmentation unit performs image segmentation on the entire acquired image and divides it into various areas, including protective nets, road cuttings, and track areas.

[0035] The target recognition unit recognizes abnormal areas of the protection net, road cutting and track in the acquired target area.

[0036] The fastener parts detection unit detects the fastener area for the entire captured image.

[0037] The faulty fastener identification unit identifies faulty abnormal fasteners in the acquired fastener area.

[0038] The data processing terminal analyzes the image recognition results and combines them with a high-precision positioning unit to locate and process abnormal parts of facilities along the railway. If there are any abnormal parts, they are stored in the database and an alarm is sounded.

[0039] In the above embodiment, the system backend includes an image processing host unit, a communication module unit and a data storage server unit; the image processing host unit completes image preprocessing, image segmentation, target recognition, detection of fastener parts and identification of faulty fasteners, and the communication module unit transmits the image data with location information collected by the data acquisition unit and the high-precision positioning unit to the image processing host unit and the data storage server unit. The data storage server unit completes the data processing terminal function and stores and backs up the image data and alarm data.

[0040] The embodiment of the present invention also provides a method for fault detection and identification of railway facilities based on high-definition images, comprising the following steps:

[0041] (1) High-definition image acquisition.

[0042] (2) The high-precision positioning unit assigns location information to the captured images.

[0043] (3) Enhance the image and divide the pixel areas where the facilities along the railway are located.

[0044] (4) Use the YOLO series of algorithms to detect fastener parts, collect a large number of normal fastener image samples, annotate them, train the network, obtain the detection model, input the image into the network model, and the fastener area can be detected.

[0045] (5) Identify faulty images of abnormal fasteners. Considering the fact that faulty samples of abnormal fasteners are very rare, an image reconstruction method based on a generative adversarial network is used to detect abnormal fasteners in the fastener area. The network is trained with normal samples from fastener parts to allow the network to learn the reconstructed latent variables of normal samples. The network model and the input fastener test image are automatically encoded to obtain the encoded latent variables. When the distance between the reconstructed latent variable and the encoded latent variable is less than the threshold, it is considered that the image is not abnormal, otherwise it is considered that the parts in the image are abnormal. The autoencoder can be used in conjunction with the generative adversarial network. A discriminator is added on the basis of the autoencoder to distinguish the reconstructed image from the input real image. The quality of the reconstructed image is improved through adversarial training of the discriminator and the reconstruction network. Typical methods include f-AnoGAN and EGBAD.

[0046] (6) Analyze the entire image, segment the protection net, road cutting, railroad tracks and other areas, and use the mask2former method for instance segmentation:

[0047] (6-1) Obtain a large number of protective net, track, and road cutting image samples, annotate them, and train the model;

[0048] (6-2) Input image, the backbone network extracts low-resolution features from the image;

[0049] (6-3) The pixel decoder gradually upsamples and decodes the low-resolution features to obtain a feature pyramid from low resolution to high resolution, which is cyclically used as the input of V and K in the Transformer decoder. The multi-scale features are used to ensure the prediction accuracy of the model for targets of different scales.

[0050] (6-4) The Transformer decoder with mask attention refines the binary mask map layer by layer by querying the multi-scale features obtained from the object and pixel decoder to obtain the final result;

[0051] (7) For the segmented protection net, road cutting, and rail area, a deep support vector data description network is used to identify the fault area. Taking protection net recognition as an example, firstly, a large-scale normal protection net training data is used for learning, then the neural network is used to extract the features of the protection net image, and the minimum hypersphere center of the normal protection net image is found in the feature space. Finally, when a protection net image to be tested is input, the distance from the feature of the test image to the hypersphere center is used as the anomaly score, and the anomaly score of the feature map is mapped back to the input image to realize the location of the abnormal pixel.

[0052] (8) Issue an alarm for abnormal faults and store them in the database.

[0053] In addition, if some defect sample images are available, the detection performance can be improved using a semi-supervised approach. A small number of annotated abnormal samples are added to the training process, and the feature extraction network is trained with the goal of maximizing the entropy of the abnormal sample feature vector distribution. The entropy-based loss function is superior to the traditional distance-based loss function.

[0054] Parts of the present invention that are not described in detail belong to the well-known technology in the art.

[0055] It should be noted that the above-mentioned examples of the present invention are intended to explain the technical features of the present invention in detail. Without departing from the present invention, several improvements and modifications made are also protected by the present invention, so the protection scope of the present invention should be based on the content defined by the claims of this application.

Claims

1. A railway facility fault detection and identification system based on high-definition images, characterized in that The system includes: An 8K ultra-high-definition camera data acquisition unit installed inside a running train to collect environmental data along the railway line; The high-precision positioning unit, combined with the sub-meter Beidou high-precision positioning system, is used to give location information to the collected images; An image preprocessing unit performs image enhancement processing and divides the pixel areas where the facilities along the railway are located; The image segmentation unit performs image segmentation on the entire acquired image and separates various areas, including protection nets, road cuttings, and track areas; The target recognition unit identifies abnormal areas of protection nets, road cuts, and tracks in the acquired target area; The fastener parts detection unit detects the fastener area from the entire captured image; A faulty fastener identification unit identifies faulty abnormal fasteners in the acquired fastener area; The data processing terminal analyzes the image recognition results and combines them with a high-precision positioning unit to locate and process abnormal parts of facilities along the railway. If there are any abnormal parts, they are stored in the database and an alarm is sounded.

2. The high-definition image-based fault detection and identification system for railway facilities according to claim 1, characterized in that: The system backend includes an image processing host, a communication module and a data storage server.

3. A method for fault detection and identification of railway facilities based on high-definition images, characterized in that The method comprises the following steps: (1) High-definition image acquisition; (2) The high-precision positioning unit assigns position information to the collected images; (3) Enhance the image and divide the pixel areas where the facilities along the railway are located; (4) Use the YOLO series of algorithms to detect fastener parts, collect a large number of normal fastener image samples, annotate them, train the network, obtain the detection model, input the image into the network model, and then detect the fastener area; (5) Identify the fault image of abnormal fasteners, and use the image reconstruction method based on generative adversarial network to detect abnormal fasteners in the fastener area; use normal samples in fastener parts to train the network, let the network learn the reconstructed latent variables of normal samples, and automatically encode the network model and the input fastener test image to obtain the encoded latent variable. When the distance between the reconstructed latent variable and the encoded latent variable is less than the threshold, it is considered that there is no abnormality in the image, otherwise it is considered that the part in the image is abnormal; (6) Analyze the entire image, segment each region, and use the mask2former method for instance segmentation: (6-1) Obtain a large number of protective net, track, and road cutting image samples, annotate them, and train the model; (6-2) Input image, the backbone network extracts low-resolution features from the image; (6-3) The pixel decoder gradually upsamples and decodes the low-resolution features to obtain a feature pyramid from low resolution to high resolution, which is cyclically used as the input of V and K in the Transformer decoder. The multi-scale features are used to ensure the prediction accuracy of the model for targets of different scales. (6-4) The Transformer decoder with mask attention refines the binary mask map layer by layer by querying the multi-scale features obtained from the object and pixel decoder to obtain the final result; (7) For the segmented protection net, road cutting, and rail area, a deep support vector data description network is used to identify the fault area. For protection net identification, large-scale normal protection net training data is first used for learning, and then the neural network is used to extract the features of the protection net image. The minimum hypersphere center of the normal protection net image is found in the feature space. When a protection net image to be tested is input, the distance from the feature of the test image to the hypersphere center is used as the anomaly score. The anomaly score of the feature map is mapped back to the input image to achieve abnormal pixel location; (8) Issue an alarm for abnormal faults and store them in the database.

Images