High-speed railway contact network foreign body detection method and system based on single classification and anomaly generation

Abstract

The present invention discloses a method and system for detecting foreign objects in high-speed rail contact networks based on single classification and anomaly generation. The method includes single classification pre-training, anomaly generation, single classification fine-tuning, and testing stages. In the single classification pre-training stage, training is performed only on a normal data set of the contact network, and a masked autoencoder is designed to learn the semantic features of the railway scene from each frame of the image extracted from the surveillance video; in the anomaly generation stage, Poisson fusion is used to migrate foreign objects in other scenes to the high-speed rail scene to generate contact network anomaly data; in the single classification fine-tuning stage, the masked autoencoder is fine-tuned based on the anomaly data; in the testing stage, anomaly detection is performed on each contact network image by calculating an anomaly score. This method does not require real anomaly samples, and the learned model has high detection accuracy and generalization for multiple types of unknown foreign objects in multiple scenes, effectively ensuring the safety of high-speed railway operation.

Description

Technical Field

[0001] The present invention belongs to the field of smart transportation and relates to a high-speed rail contact network foreign object detection method and system based on single classification and anomaly generation. The method and system can learn normal patterns from a data set lacking abnormal samples and automatically identify multiple types of unknown foreign objects invading the contact network. Background Art

[0002] With the continuous development of society and the economy, high-speed rail has become an important mode of transportation. Preventing emergencies during high-speed rail operations has become a crucial issue. In actual operation, high winds along the line can easily cause foreign objects (commonly seen as ground film, plastic bags, and kites) to attach to the high-speed rail contact network, affecting the power supply to electric locomotives and threatening the normal operation of trains.

[0003] At present, my country mainly uses manual observation to detect foreign objects stuck on the contact network. Based on the video of the contact network along the railway captured and transmitted back by the on-board camera of the high-speed train, the naked eye is used to identify whether there are foreign objects attached to the contact network. This detection method is inefficient and consumes a lot of manpower. There is still much room for improvement in reliability.

[0004] In recent years, neural network-based image processing technology has rapidly developed. Object detection, particularly in the areas of video surveillance and autonomous driving, has played a significant role, with significant breakthroughs in small object detection. These new technologies provide powerful tools for foreign object detection in high-speed rail contact lines, significantly reducing the cost of manual observation and improving detection accuracy and timeliness.

[0005] However, compared with other detection tasks, the task of detecting foreign objects in high-speed rail contact networks presents the following challenges. (1) Since the network hanging incident is an emergency, the available abnormal video samples are very limited. Therefore, it is impossible to provide enough target samples during the training process as in general target detection tasks. (2) Foreign objects intruding into the high-speed rail contact network are diverse, ranging from objects that are clearly distinguishable from the contact network background, such as ground film and kites, to inconspicuous small objects such as branches and plastic bags. Therefore, it is necessary to detect the diverse features of abnormal samples. (3) Due to the openness of the real world, some foreign objects that have never been seen before may intrude, such as sky lanterns and drones, so the detection model needs to have better generalization capabilities. Summary of the Invention

[0006] Taking into account the characteristics of foreign object intrusion in high-speed rail contact network scenarios, such as few samples, many types and unknown, the present invention proposes a high-speed rail contact network foreign object detection method and system based on single classification and anomaly generation. This method only uses normal samples during the training process, extracts normal pattern features from normal samples through a neural network, and scores each sample for anomaly during the test, and marks pictures with high anomaly scores as foreign object intrusion events.

[0007] The purpose of the present invention is achieved through the following technical solutions:

[0008] In one aspect, the present invention provides a method for detecting foreign objects in a high-speed railway contact network based on single classification and anomaly generation, the method comprising the following steps:

[0009] S1. Single-classification pre-training: A masked autoencoder is used to extract high-level semantic features of normal catenary images, and the masked autoencoder is pre-trained based on the normal catenary image data.

[0010] S2. Anomaly generation: Use normal images of the contact network as source domain data and images of foreign objects as target domain data. Based on the contact network source domain background modeling and combined with the target domain foreign object images, Poisson fusion is used to generate a contact network foreign object image dataset.

[0011] S3. Single classification fine-tuning: Use the generated abnormal image dataset to fine-tune the pre-trained mask autoencoder;

[0012] S4. During the testing phase, the anomaly score of each contact network image is calculated, and when the score exceeds the threshold, it is judged as abnormal.

[0013] Furthermore, the step S1 is specifically as follows:

[0014] Input the preprocessed RGB image x of the high-speed rail contact network in a normal scene, split it into M rows and N columns, a total of M×N sub-images, and use the random mask matrix I∈R M×N The sub-images are covered. In view of the fact that floating objects are mostly concentrated at the top of the image in the high-speed rail contact network scenario, the probability distribution of the random mask matrix is ​​designed as follows:

[0015]

[0016] Among them I m,n =1 indicates that the sub-images numbered m and n from top to bottom and from left to right are covered; for the covered sub-images, the mask feature learner is used to calculate the replacement value of each pixel:

[0017]

[0018] in Indicates the replacement value of the pixel with coordinates (a, b) in the image, f p Represents the mask feature mapping network from the original pixel value to the mask value, which is shared by all pixel values; after completing the pixel value replacement, it is passed through the encoder f e and decoder f d The reconstructed image generator composed of The optimization goal of masked autoencoder pre-training is:

[0019]

[0020]

[0021]

[0022] where l rec is the mask reconstruction error of the high-speed rail contact network image at the pixel level, D is the contact network training set containing only normal images, and I a,b Indicates whether the pixel (a, b) is masked. When it is 1, it means it is masked, and when it is 0, it means it is not masked. E x～D Represents the mathematical expectation when the image follows the distribution of the training set; l oc is the single classification error of the contact network, λ1 is the single classification error of the contact network l oc The weight of h is the contact network image after encoder f e Output.

[0023] Furthermore, the image preprocessing in step S1 is specifically as follows:

[0024] For the original image X∈R with width W and height H 3×H×W First, capture the sky background image in the upper half of the image, then downsample the captured image and compress the size to 1 / 2 to form the preprocessed image.

[0025] Furthermore, in step S1, the mask feature learner adopts a forward neural network; the encoder adopts a ResNet18 encoding network, and the decoder adopts a ResNet18 decoding network.

[0026] Furthermore, in step S2, the center coordinates of the foreign object are generated in the contact network source domain image according to the size of the foreign object; the foreign object is fused into the original image using Poisson fusion to form the foreign object image dataset D n .

[0027] Furthermore, in step S2, the foreign body image is scaled, and the high-speed rail contact network image after the upper half is cropped is The randomly selected center point (i, j) satisfies:

[0028] w′<i<Ww′

[0029]

[0030] Where H, W are the width and height of the original image of the high-speed rail contact network, and w′, h′ are the width and height of the scaled foreign object image. The center point (i, j) is used as the fusion center, and Poisson fusion is used to add foreign objects to the high-speed rail image to form the contact network foreign object image dataset D n .

[0031] Furthermore, the step S3 is specifically as follows:

[0032] Generate a foreign object detection frame based on the size and center point of the foreign object, and use the pre-trained mask feature learner to generate a replacement mask value for the pixel values ​​in the detection frame;

[0033] For the foreign body image after mask processing, the pre-trained f e , f d Fine-tune and optimize the target:

[0034]

[0035]

[0036]

[0037] where l rec′ is the mask reconstruction error of the high-speed railway contact network image at the pixel level, D n is the contact network training set containing only generated abnormal images, I′ a,b Indicates whether the pixel (a, b) is included in the foreign object detection frame. When it is 1, it means it is included, and when it is 0, it means it is not included. Represents the mathematical expectation when the image follows the distribution of abnormal image training set; l oc′ is the single classification error of the contact network, λ2 is the single classification error of the contact network l oc′ The weight of h is the mask contact network image after encoder f e The output, E x～D Represents the mathematical expectation when the image follows the normal image training set distribution.

[0038] Furthermore, in step S4, the process of calculating the anomaly score is as follows:

[0039] Divide the original image into M×N sub-images, perform T masking according to the masking strategy in step S1, and replace the pixel values ​​based on the mask feature learner. Input the generated T images into the encoder f e and decoder f d , generate the corresponding reconstructed image, and the final anomaly score A is:

[0040]

[0041] where h i and They represent the encoder output features and decoder output reconstructed picture of the image after the i-th mask respectively.

[0042] Furthermore, in step S4, the abnormal score threshold τ is set according to the following formula:

[0043]

[0044] in is the average anomaly score of the abnormal image training set, max x～D A x is the maximum anomaly score of the normal image training set, and λ3 is the weight of the abnormal image training set; when the anomaly score of the test image exceeds the threshold, it is judged as abnormal.

[0045] Another aspect of the present invention provides a high-speed railway contact network foreign body detection system based on single classification and anomaly generation, comprising:

[0046] Single-classification pre-training module: A masked autoencoder is used to extract high-level semantic features of normal catenary images, and the masked autoencoder is pre-trained based on normal catenary image data.

[0047] Anomaly generation module: This module uses normal images of the contact network as source domain data and images of foreign objects as target domain data. Based on the contact network source domain background modeling and combining them with target domain foreign object images, it uses Poisson fusion to generate a contact network foreign object image dataset.

[0048] Single-classification fine-tuning module: Use the generated abnormal image dataset to fine-tune the pre-trained mask autoencoder;

[0049] Anomaly scoring module: During the testing phase, the anomaly score of each contact network image is calculated, and when the score exceeds the threshold, it is judged as an anomaly.

[0050] The beneficial effects of the present invention are:

[0051] 1. The present invention adopts a single classification model for learning. No foreign body samples need to be introduced during the training process. This satisfies the condition that the original data lacks abnormal samples, solves the sample imbalance problem, and greatly enhances the accuracy of foreign body detection.

[0052] 2. The present invention adopts an abnormal data enhancement method to generate high-quality abnormal intrusion contact network data in high-speed rail scenarios, providing an effective detection model verification dataset.

[0053] 3. The present invention utilizes a masked autoencoder to enhance the semantic feature representation capability in the support vector description method, greatly improving the generalization of the model for detecting various foreign objects in actual scenes. BRIEF DESCRIPTION OF THE DRAWINGS

[0054] Figure 1 It is a schematic diagram of a high-speed railway contact network foreign body detection method based on single classification and anomaly generation provided by an exemplary embodiment.

[0055] Figure 22 is a schematic diagram of a masked autoencoder structure provided by an exemplary embodiment.

[0056] Figure 3 This is an abnormal data test effect diagram provided by an exemplary embodiment. DETAILED DESCRIPTION

[0057] According to the technical principles, specific implementation methods of the present invention are described with reference to the above drawings, which should not be regarded as the entirety of the present invention, nor should they be regarded as limiting or restricting the technical solutions of the present invention.

[0058] like Figure 1 As shown, an embodiment of the present invention provides a high-speed railway catenary foreign object detection method based on single classification and anomaly generation, which is used to detect foreign objects hanging on the catenary and floating objects from the catenary monitoring video collected by the train onboard camera, including the following steps:

[0059] S1. Single-category pre-training: A masked autoencoder is used to extract high-level semantic features of normal contact network images, and the masked autoencoder is pre-trained based on the normal contact network image data.

[0060] The masked autoencoder processes the image as follows Figure 2 , as follows:

[0061] For the original image X∈R with width W and height H 3×H×W Preprocessing is performed by first capturing the sky background image in the upper half of the image, then downsampling the captured image and compressing its size to 1 / 2 to form the preprocessed image.

[0062] Input the preprocessed RGB image x of the high-speed rail contact network in a normal scene, split it into M rows and N columns, a total of M×N sub-images, and use the random mask matrix I∈R M×N The sub-images are covered. In view of the fact that floating objects are mostly concentrated at the top of the image in the high-speed rail contact network scenario, the probability distribution of the random mask matrix is ​​designed as follows:

[0063]

[0064] Among them I m,n =1 indicates that the subgraphs numbered m and n from top to bottom and from left to right are covered.

[0065] In this embodiment, the image x is split into 8 sub-images with 2 rows and 4 columns. The probability of each sub-image in the first row being covered is designed to be e -1 , the probability of each sub-image in the second row being covered is

[0066] After the mask image is determined, a feed-forward neural network with an input dimension of 2 and an output dimension of 3 is used to generate replacement values ​​for the masked pixels. The formula is as follows:

[0067]

[0068] in Indicates the replacement value of the pixel with coordinates (a, b) in the image, f p Represents the mask feature mapping network from original pixel values ​​to mask values, which is shared by all pixel values. The final mask image is

[0069] Input the mask image into the ResNet18 encoding network f e , the output dimension is 256-dimensional intermediate feature h∈R 256 .

[0070] Input the intermediate features into the ResNet18 decoding network f d , returns a reconstructed image with the same size as the input image

[0071] The optimization goal for pre-training the masked autoencoder is:

[0072]

[0073]

[0074]

[0075] where l rec is the mask reconstruction error of the high-speed rail contact network image at the pixel level, D is the contact network training set containing only normal images, and I a,b Indicates whether the pixel (a, b) is masked. When it is 1, it means it is masked, and when it is 0, it means it is not masked. E x～D Represents the mathematical expectation when the image follows the distribution of the training set; l oc is the single classification error of the contact network, λ1 is the single classification error of the contact network l oc The weight of h is the mask contact network image after encoder f e Output.

[0076] S2. Anomaly generation: Take normal images of the contact network as source domain data and images of foreign objects as target domain data. Based on the contact network source domain background modeling and combined with the target domain foreign object images, Poisson fusion is used to generate the contact network foreign object image dataset.

[0077] Specifically, obtain images of foreign objects such as balloons, sky lanterns, and plastic bags, scale the images of foreign objects, and control their size w′×h′ within the range of 120×120 pixels.

[0078] Image of the high-speed rail contact network after cropping the upper half The randomly selected center point (i, j) satisfies:

[0079] w′<i<Ww′

[0080]

[0081] The center point (i, j) is used as the fusion center, and Poisson fusion is used to add foreign objects to the high-speed rail image to form the contact network foreign object image dataset D n .

[0082] S3. Single classification fine-tuning: Use the generated abnormal image dataset to fine-tune the pre-trained mask autoencoder.

[0083] Specifically, a 120×120 foreign object detection frame is generated based on the center point of the foreign object, and a pre-trained mask feature learner is used to generate a replacement mask value for the pixel values ​​in the detection frame.

[0084] For the foreign body image after mask processing, the pre-trained f e , f d Fine-tune and optimize the target:

[0085]

[0086]

[0087]

[0088] where l rec′ is the mask reconstruction error of the high-speed railway contact network image at the pixel level, D n is the contact network training set containing only generated abnormal images, I′ a,b Indicates whether the pixel (a, b) is included in the foreign object detection frame. When it is 1, it means it is included, and when it is 0, it means it is not included. Represents the mathematical expectation when the image follows the distribution of abnormal image training set; l oc′ is the single classification error of the contact network, λ2 is the single classification error of the contact network l oc′ The weight of h is the contact network image after encoder f e The output, E x～D Represents the mathematical expectation when the image follows the normal image training set distribution.

[0089] S4. During the testing phase, the anomaly score of each contact network image is calculated, and when the score exceeds the threshold, it is judged as a foreign object.

[0090] Specifically, the test image is divided into 2×4 sub-images, masked 10 times according to the mask strategy in step S1, and pixel values ​​are replaced based on the mask feature learner. The generated 10 images are input into the encoder fe and decoder f d , generate the corresponding reconstructed image, and the final anomaly score A is:

[0091]

[0092] where h i and They represent the encoder output features and decoder output reconstructed picture of the image after the i-th mask respectively.

[0093] The anomaly score threshold τ is set according to the following formula:

[0094]

[0095] in is the average anomaly score of the abnormal image training set, max x～D A x is the maximum anomaly score of the normal image training set, and λ3 is the weight of the abnormal image training set; when the anomaly score of the test image exceeds the threshold, it is judged as abnormal.

[0096] The present invention also provides a high-speed railway contact network foreign body detection system based on single classification and anomaly generation, the system comprising:

[0097] Single-classification pre-training module: A masked autoencoder is used to extract high-level semantic features of normal catenary images, and the masked autoencoder is pre-trained based on normal catenary image data.

[0098] Anomaly generation module: This module uses normal images of the contact network as source domain data and images of foreign objects as target domain data. Based on the contact network source domain background modeling and combining them with target domain foreign object images, it uses Poisson fusion to generate a contact network foreign object image dataset.

[0099] Single-classification fine-tuning module: Use the generated abnormal image dataset to fine-tune the pre-trained mask autoencoder;

[0100] Anomaly scoring module: During the testing phase, the anomaly score of each contact network image is calculated, and when the score exceeds the threshold, it is judged as an anomaly.

[0101] The implementation of each module refers to the steps of the above-mentioned embodiment of the high-speed railway contact network foreign body detection method based on single classification and anomaly generation.

[0102] Figure 3 This is the result of an abnormal data test. When foreign objects such as drones, kites, plastic bags, and balloons appear above the high-speed rail contact network, the method of the present invention can generate a high anomaly score. This method can detect a variety of unknown foreign objects, meeting the needs of railway departments and effectively ensuring railway operation safety.

[0103] The above embodiments are merely preferred embodiments of the present invention and are used to illustrate the technical solutions of the present invention, but are not intended to limit the scope of protection of the present invention. Although the present invention has been described in detail with reference to the above embodiments, the technical solutions described in the above embodiments may be modified or some of the technical features thereof may be replaced by equivalents without departing from the spirit and scope of the claims and their equivalents. Therefore, such modifications or replacements are within the scope of protection of this technical solution.

Claims

1. A method for detecting foreign objects in high-speed railway contact network based on single classification and anomaly generation, characterized in that: The following steps are involved: S1, single classification pre-training: using a masked autoencoder to extract high-level semantic features of a normal catenary image, and pre-training the masked autoencoder based on the normal catenary image data; step S1 is specifically as follows: Input the preprocessed RGB image x of the high-speed rail contact network in a normal scene, split it into M rows and N columns, a total of M×N sub-images, and use the random mask matrix I∈r M×N The sub-images are covered. In view of the fact that floating objects are mostly concentrated at the top of the image in the high-speed rail contact network scenario, the probability distribution of the random mask matrix is ​​designed as follows: Among them I m,n =1 indicates that the sub-images numbered m and n from top to bottom and from left to right are covered; for the covered sub-images, the mask feature learner is used to calculate the replacement value of each pixel: in Indicates the replacement value of the pixel with coordinates (a, b) in the image, f p Represents the mask feature mapping network from the original pixel value to the mask value, which is shared by all pixel values; after completing the pixel value replacement, it is passed through the encoder f e and decoder f d The reconstructed image generator composed of The optimization goal of masked autoencoder pre-training is: where l rec is the mask reconstruction error of the high-speed rail contact network image at the pixel level, D is the contact network training set containing only normal images, and I a,b Indicates whether the pixel (a, b) is masked. When it is 1, it means it is masked, and when it is 0, it means it is not masked. E x～D Represents the mathematical expectation when the image follows the distribution of the training set; l ic is the single classification error of the contact network, λ1 is the single classification error of the contact network l oc The weight of h is the contact network image after encoder f e Output; S2. Anomaly generation: Use normal images of the contact network as source domain data and images of foreign objects as target domain data. Based on the contact network source domain background modeling and combined with the target domain foreign object images, Poisson fusion is used to generate a contact network foreign object image dataset. S3. Single classification fine-tuning: Use the generated abnormal image dataset to fine-tune the pre-trained mask autoencoder; S4. During the testing phase, the anomaly score of each contact network image is calculated, and when the score exceeds the threshold, it is judged as abnormal.

2. The high-speed railway contact network foreign body detection method based on single classification and anomaly generation according to claim 1 is characterized in that: The image preprocessing in step S1 is specifically as follows: For the original image X∈R with width W and height H 3×H×W First, capture the sky background image in the upper half of the image, then downsample the captured image and compress the size to 1 / 2 to form the preprocessed image.

3. The high-speed railway contact network foreign body detection method based on single classification and anomaly generation according to claim 1 is characterized in that: In step S1, the mask feature learner adopts a forward neural network; the encoder adopts a ResNet18 encoding network, and the decoder adopts a ResNet18 decoding network.

4. The high-speed railway contact network foreign body detection method based on single classification and anomaly generation according to claim 1 is characterized in that: In step S2, the center coordinates of the foreign object are generated in the contact network source domain image according to the size of the foreign object; the foreign object is fused into the original image using Poisson fusion to form the foreign object image dataset D n .

5. The high-speed railway contact network foreign body detection method based on single classification and anomaly generation according to claim 4 is characterized in that: In step S2, the foreign body image is scaled, and the high-speed rail contact network image after cropping the upper half is The randomly selected center point (i, j) satisfies: w′ <i<W-w′ Among them, H and W are the width and height of the original image of the high-speed railway contact network, w ′ , h′ is the width and height of the scaled foreign body image; the center point (i, j) is used as the fusion center, and Poisson fusion is used to add foreign bodies to the high-speed rail image to form the contact network foreign body image dataset D n .

6. The method for detecting foreign matter in a high-speed railway contact network based on single classification and anomaly generation according to claim 1, characterized in that: Step S3 is specifically as follows: Generate a foreign object detection frame based on the size and center point of the foreign object, and use the pre-trained mask feature learner to generate a replacement mask value for the pixel values ​​in the detection frame; For the foreign body image after mask processing, the pre-trained f e , f d Fine-tune and optimize the target: where l rec′ is the mask reconstruction error of the high-speed railway contact network image at the pixel level, D n is the contact network training set containing only generated abnormal images, I′ a,b Indicates whether the pixel (a, b) is included in the foreign object detection frame. When it is 1, it means it is included, and when it is 0, it means it is not included. Represents the mathematical expectation when the image follows the distribution of abnormal image training set; l oc, is the single classification error of the contact network, λ2 is the single classification error of the contact network l oc′ The weight of h is the mask contact network image after encoder f e The output, E x～D Represents the mathematical expectation when the image follows the normal image training set distribution.

7. The high-speed railway contact network foreign body detection method based on single classification and anomaly generation according to claim 1 is characterized in that: In step S4, the process of calculating the anomaly score is as follows: Divide the original image into M×N sub-images, perform T masking according to the masking strategy in step S1, and replace the pixel values ​​based on the mask feature learner. Input the generated T images into the encoder f e and decoder f d , generate the corresponding reconstructed image, and the final anomaly score A is: where h i and They represent the encoder output features and decoder output reconstructed picture of the image after the i-th mask respectively.

8. The high-speed railway contact network foreign body detection method based on single classification and anomaly generation according to claim 1 is characterized in that: In step S4, the anomaly score threshold τ is set according to the following formula: in is the average anomaly score of the abnormal image training set, max x～D A x is the maximum anomaly score of the normal image training set, and λ3 is the weight of the abnormal image training set; when the anomaly score of the test image exceeds the threshold, it is judged as abnormal.

9. A high-speed railway contact network foreign body detection system based on single classification and anomaly generation, characterized in that: include: Single-classification pre-training module: A masked autoencoder is used to extract high-level semantic features of normal catenary images, and the masked autoencoder is pre-trained based on normal catenary image data. Specifically: Input the preprocessed RGB image x of the high-speed rail contact network in a normal scene, split it into M rows and N columns, a total of M×N sub-images, and use the random mask matrix I∈R M×N The sub-images are covered. In view of the fact that floating objects are mostly concentrated at the top of the image in the high-speed rail contact network scenario, the probability distribution of the random mask matrix is ​​designed as follows: Among them I m,n =1 indicates that the sub-images numbered m and n from top to bottom and from left to right are covered; for the covered sub-images, the mask feature learner is used to calculate the replacement value of each pixel: in Indicates the replacement value of the pixel with coordinates (a, b) in the image, f p Represents the mask feature mapping network from the original pixel value to the mask value, which is shared by all pixel values; after completing the pixel value replacement, it is passed through the encoder f e and decoder f d The reconstructed image generator composed of The optimization goal of masked autoencoder pre-training is: where l rec is the mask reconstruction error of the high-speed rail contact network image at the pixel level, D is the contact network training set containing only normal images, and I a,b Indicates whether the pixel (a, b) is masked. When it is 1, it means it is masked, and when it is 0, it means it is not masked. E x～D Represents the mathematical expectation when the image follows the distribution of the training set; l oc is the single classification error of the contact network, λ1 is the single classification error of the contact network l oc The weight of h is the contact network image after encoder f e Output; Anomaly generation module: This module uses normal images of the contact network as source domain data and images of foreign objects as target domain data. Based on the contact network source domain background modeling and combining them with target domain foreign object images, it uses Poisson fusion to generate a contact network foreign object image dataset. Single-classification fine-tuning module: Use the generated abnormal image dataset to fine-tune the pre-trained mask autoencoder; Anomaly scoring module: During the testing phase, the anomaly score of each contact network image is calculated, and when the score exceeds the threshold, it is judged as an anomaly.

Images