A visual anomaly detection method and system for rail transit operation and maintenance scene

Abstract

The application relates to the technical field of data processing and analysis, and discloses a visual anomaly detection method and system for a rail transit operation and maintenance scene, image data of rail transit operation and maintenance is collected, the image data is preprocessed, and then is selectively transmitted to a constructed anomaly detector through a target detector, feature extraction and aggregation, feature distribution fitting and anomaly score calculation are performed by using the anomaly detector, an anomaly detection result of the anomaly detector for input image is obtained, and finally, rail transit operation and maintenance anomaly alarm is executed according to the anomaly detection result. Therefore, by adopting optimization design of network structures such as feature breathing mechanism, asymmetric encoder-decoder structure and self-attention decoder layer, the model performance of the anomaly detection algorithm is significantly improved, the anomaly detection of multiple types of components or devices can be realized in one model, meanwhile, the missing detection risk existing in the existing detection system can be improved, and the robustness and adaptability of the system to diversified anomalies are improved.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of data processing and analysis, and particularly relates to a visual anomaly detection method and system for a rail transit operation and maintenance scene. BACKGROUND

[0002] Rail transportation, as the artery of the national economy, is of great importance in terms of safety. In the long-term operation process, due to the influence of complex dynamic load and environment, train components, track lines and trackside equipment will inevitably produce various defects, such as wear, cracks, deformation, corrosion, loosening, missing, etc. Railway bridges, railway tunnels, slopes, etc. may also have cracks, moving chunks, collapse and water accumulation, etc. If these defects and abnormalities cannot be detected and repaired in time, they will affect the efficiency of train operation, or even cause serious safety accidents. Therefore, in the rail transit operation and maintenance scene, visual-based anomaly detection analysis is a key link to ensure the safety of railway transportation and improve the efficiency of operation.

[0003] With the continuous development of artificial intelligence and image processing technology, visual analysis algorithms based on CNN or Transformer have been relatively maturely applied in the intelligent operation and maintenance scene of rail transit. However, the related research schemes still have some deficiencies. On the one hand, some schemes rely heavily on the accumulation of abnormal sample data or completely design algorithms in a supervised learning framework, which leads to the risk of missing detection of the system only sensitive to the types of abnormalities or defects covered in the data set. On the other hand, the existing unsupervised schemes are limited by the performance of the model, and need to set up independent models for detection of different types of components or equipment, thereby increasing the complexity of the system and being not conducive to the expansion and maintenance of functions. SUMMARY

[0004] The present application provides a visual anomaly detection method and system for a rail transit operation and maintenance scene, aiming to solve at least one of the above technical problems.

[0005] To achieve the above-mentioned purpose, the present application provides a visual anomaly detection method for a rail transit operation and maintenance scene, which comprises the following steps:

[0006] S1: collecting rail transit operation and maintenance image data and pre-processing the image data;

[0007] S2: selectively transmitting the pre-processed image data to a target detector for main target detection;

[0008] S3: transmitting the pre-processed image data or the main target output by the target detector to a feature extraction and aggregation unit of an anomaly detector to obtain aggregated features;

[0009] S4: transmit the aggregated features to a feature distribution fitting unit of the anomaly detector, obtain compressed features based on the feature breathing mechanism, establish a loss function based on the compressed features and the aggregated features, and perform model training;

[0010] S5: after model training, generate an anomaly segmentation map of the input image data based on the multi-scale features and the compressed features, and determine an anomaly score of the input image data according to the anomaly segmentation map;

[0011] S6: perform rail transit operation and maintenance anomaly alarm according to the anomaly score.

[0012] Optionally, in step S1, the rail transit operation and maintenance image data is collected, specifically including: for unmanned aerial vehicle line inspection tasks, using a wireless image transmission system to collect data; and / or, for train component defect detection tasks, using an image acquisition subsystem of a TEDS system to collect data.

[0013] Optionally, in step S1, the image data is preprocessed, specifically including: using one or more of image stitching, image cropping, image denoising, or histogram equalization for image preprocessing.

[0014] Optionally, in step S2, the preprocessed image data is selectively transmitted to a target detector for subject target detection, specifically including:

[0015] S21: determine whether the detected object in the image data is an image-level anomaly or a target-level anomaly;

[0016] S22: when the detected object is an image-level anomaly, the preprocessed image data is directly transmitted to the constructed anomaly detector, and the anomaly detection result output by the anomaly detector for the image data is obtained;

[0017] S23: when the detected object is a target-level anomaly, the subject target in the image data is detected by the target detector, the subject target is transmitted to the anomaly detector, and the anomaly detection result output by the anomaly detector for the subject target is obtained.

[0018] Optionally, in step S3, the preprocessed image data or the subject target output by the target detector is transmitted to a feature extraction and aggregation unit of the anomaly detector to obtain aggregated features, specifically including:

[0019] S31: transmit the preprocessed image data or the subject target output by the target detector to the feature extraction and aggregation unit of the anomaly detector, use a pre-trained backbone network based on CNN or Transformer as a feature extractor to extract multi-scale features from the input preprocessed image;

[0020] S32: obtaining aggregated features by resizing and feature concatenation on the multi-scale features; wherein, the expression of obtaining aggregated features by resizing and feature concatenation is specifically as follows:

[0021] ;

[0022] In the formula, aggregated features, Resize represents a feature resizing operation, and Concat represents a feature concatenation operation, , wherein, represents the features of the i-th stage, and m represents the total number of stages.

[0023] Optionally, step S4: transmitting the aggregated features to a feature distribution fitting unit of the anomaly detector, obtaining compressed features based on a feature breathing mechanism, establishing a loss function based on the compressed features and the aggregated features to perform model training, specifically including:

[0024] S41: transmitting the aggregated features to a feature distribution fitting unit of the anomaly detector, performing channel expansion on the aggregated features superimposed with feature perturbations by a linear layer; wherein, the expression of performing channel expansion is specifically as follows:

[0025] ;

[0026] S42: reconstructing the aggregated features after channel expansion by a Transformer architecture-based encoder-decoder to obtain reconstructed features ;

[0027] S43: compressing the channels of the reconstructed features to the original number of channels by a linear layer; wherein, the expression of performing channel compression is specifically as follows:

[0028] ;

[0029] S44: establishing a loss function based on the compressed features and the aggregated features to perform model training; wherein, the expression of the loss function is specifically as follows:

[0030] ;

[0031] In the formula, L represents the loss function, and MSE represents the mean square error.

[0032] Optionally, in step S42, the Transformer architecture-based encoder-decoder is configured to adopt an asymmetric encoder-decoder structure composed of 4 encoder layers and 6 decoder layers; the decoder layer specifically includes:

[0033] The adjacent mask self-attention layer is configured to take the previous decoder layer output as an input representation V vector, take the previous decoder layer output and the query embedding together as an input representation K vector, and take the previous decoder layer output and the query embedding together as an input representation Q query vector.

[0034] The adjacent mask mutual attention layer is configured to take the encoder output as an input representation V vector, take the encoder output and the position embedding together as an input representation K vector, and take the adjacent mask self-attention layer output and the query embedding together as an input representation Q query vector.

[0035] Optionally, step S5: after model training, based on the multi-scale feature and the compressed feature, an abnormal segmentation map of the input image data is generated, and according to the abnormal segmentation map, an abnormal score of the input image data is determined, specifically including:

[0036] S51: after model training, different feature layers are selected according to the characteristics of the detection target to participate in calculation, and distance measurement methods and adjustment parameters are selected according to the richness of the abnormal type;

[0037] S52: based on the selected feature layer, distance measurement method and adjustment parameter, the abnormal segmentation map of the input image data is generated for the multi-scale feature and the compressed feature, and the expression is specifically:

[0038] ;

[0039] In the formula, L2 is an L2 distance, CS represents a cosine similarity, and λ represents an adjustment parameter;

[0040] S53: taking K points with the largest values in the abnormal segmentation map to calculate the mean value, an abnormal score of the input image data is obtained.

[0041] Optionally, step S6, according to the abnormal score, performing rail transit operation and maintenance abnormal alarm, specifically including: judging whether the abnormal score exceeds a preset score threshold, if yes, performing rail transit operation and maintenance abnormal alarm, if not, performing data archiving.

[0042] In addition, in order to achieve the above purpose, the application also provides a visual anomaly detection system for a rail transit operation and maintenance scene, comprising:

[0043] A data acquisition and preprocessing module is configured to acquire rail transit operation and maintenance image data and preprocess the image data;

[0044] An image data transmission module is configured to selectively transmit the preprocessed image data to a target detector for subject target detection.

[0045] a feature extraction aggregation module configured to transmit the preprocessed image data or the subject target output by the target detector to a feature extraction aggregation unit of the anomaly detector, to obtain aggregated features;

[0046] a feature distribution fitting module configured to transmit the aggregated features to a feature distribution fitting unit of the anomaly detector, to obtain compressed features based on a feature breathing mechanism, to establish a loss function based on the compressed features and the aggregated features, and to perform model training;

[0047] an anomaly score calculation module configured to, after the model training, generate an anomaly segmentation map of the input image data based on the multi-scale features and the compressed features, and to determine an anomaly score of the input image data according to the anomaly segmentation map;

[0048] an anomaly alarm module configured to perform rail transit operation and maintenance anomaly alarm according to the anomaly score.

[0049] The present application has the beneficial effect that a visual anomaly detection method and system for a rail transit operation and maintenance scenario are provided, image data of rail transit operation and maintenance is collected, and the image data is preprocessed; the preprocessed image data is transmitted to a constructed anomaly detector, feature extraction aggregation, feature distribution fitting, and anomaly score calculation are performed by using the anomaly detector, anomaly detection results of the input image by the anomaly detector are obtained, and finally, rail transit operation and maintenance anomaly alarm is performed according to the anomaly detection results. Thus, the present application optimizes the network structure by the feature breathing mechanism, the asymmetric encoder-decoder structure, and the self-attention decoder layer, significantly improves the model performance of the anomaly detection algorithm, realizes anomaly detection of multiple types of components or devices in one model, improves the missed detection risk of the existing detection system, and improves the robustness and adaptability of the system to diversified anomalies. BRIEF DESCRIPTION OF DRAWINGS

[0050] Figure 1 A flowchart of the visual anomaly detection method for the rail transit operation and maintenance scenario of the present application;

[0051] Figure 2 A principle diagram of the visual anomaly detection method for the rail transit operation and maintenance scenario of the present application;

[0052] Figure 3 A structure diagram of the anomaly detector proposed by the present application;

[0053] Figure 4 A design diagram of the decoder layer in the anomaly detector proposed by the present application;

[0054] Figure 5 A structure diagram of the visual anomaly detection system for the rail transit operation and maintenance scenario of the present application. DETAILED DESCRIPTION

[0055] In order to make the objects, technical solutions and advantages of the present application clearer, the present application will be further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application and do not limit the present application.

[0056] The embodiment of the present application provides a visual anomaly detection method for a rail transit operation and maintenance scene. Figure 1 , Figure 1 The embodiment of the present application provides a flowchart of the visual anomaly detection method for the rail transit operation and maintenance scene.

[0057] In the embodiment, a visual anomaly detection method for a rail transit operation and maintenance scene comprises the following steps:

[0058] S1: collecting rail transit operation and maintenance image data and pre-processing the image data;

[0059] S2: selectively transmitting the pre-processed image data to a target detector for subject target detection;

[0060] S3: transmitting the pre-processed image data or the subject target output by the target detector to a feature extraction and aggregation unit of an anomaly detector to obtain aggregated features;

[0061] S4: transmitting the aggregated features to a feature distribution fitting unit of the anomaly detector, obtaining compressed features based on a feature breathing mechanism, establishing a loss function based on the compressed features and the aggregated features, and performing model training;

[0062] S5: after the model training, generating an anomaly segmentation map of the input image data based on the multi-scale features and the compressed features, and determining an anomaly score of the input image data according to the anomaly segmentation map;

[0063] S6: performing rail transit operation and maintenance anomaly alarm according to the anomaly score.

[0064] It should be noted that with the continuous development of artificial intelligence and image processing technology, visual analysis algorithms based on CNN or Transformer have been more maturely applied in the intelligent operation and maintenance scene of rail transit. However, the related research scheme still has the following shortcomings: (1) Some schemes still rely heavily on the accumulation of abnormal sample data, or completely design algorithms in a supervised learning framework, which will lead the system to be sensitive only to the types of anomalies or defects covered in the data set, thus there is a risk of missing detection. For example, in the unmanned aerial vehicle line inspection task, the system can identify the anomaly of the Class A equipment beside the track, but cannot identify the anomaly of the Class B equipment beside the track, because there is no abnormal data of the Class B equipment in the data set. (2) In the existing unsupervised scheme, due to the limitation of model performance, an independent model needs to be set up for detection of different types of components or equipment, which increases the complexity of the system and is not conducive to the expansion and maintenance of functions. For example, in the train component defect detection task, the system needs to construct a model for each of the A-type cover plate and the B-type cover plate for detection, and the number of system sub-models is positively correlated with the number of component types.

[0065] To solve the above problems, the embodiment collects rail transit operation and maintenance image data, pre-processes the image data, transmits the pre-processed image data to the constructed anomaly detector, uses the anomaly detector to perform feature extraction and aggregation, feature distribution fitting and abnormal score calculation, obtains the anomaly detection result of the anomaly detector for the input image, and finally executes rail transit operation and maintenance anomaly alarm according to the anomaly detection result. Thus, through the optimization design of network structures such as feature expansion and compression mechanism, asymmetric encoder-decoder structure and DETR decoder layer, the model performance of the anomaly detection algorithm is significantly improved, the anomaly detection of multiple types of components or equipment can be realized in one model, at the same time, the missing detection risk of the existing detection system can be improved, and the robustness and adaptability to diversified anomalies of the system are improved.

[0066] In order to explain the present application more clearly, the specific implementation process of the visual anomaly detection method for the rail transit operation and maintenance scene is provided below.

[0067] As shown in Figure 2 The principle diagram of the visual anomaly detection method for the rail transit operation and maintenance scene provided by the present application, which specifically includes the following execution process:

[0068] 1. Rail transit operation and maintenance data collection: different data collection methods are used according to different operation and maintenance tasks. For example, in the unmanned aerial vehicle line inspection task, a wireless image transmission system is used for data collection; in the train component defect detection task, the image acquisition subsystem of the TEDS system (Trouble of moving EMU Detection System, moving EMU operation fault image detection system) is used for data collection. In the model training stage, normal image data (without abnormal data) is collected to train the target detector (optional) and the anomaly detector. During system detection, real-time detection data is collected.

[0069] 2. Data preprocessing: according to different operation and maintenance tasks, necessary data preprocessing steps such as image stitching, cropping, denoising, histogram equalization, etc. are set.

[0070] 3. Target detector: this step is not necessary, and no specific target detector type is specified. Common YOLO series detectors or DETR series detectors, etc. can be used. When the detection object is image-level anomaly (such as detecting geological disasters in the unmanned aerial vehicle line inspection task), no target detector is needed, and the input data can be directly fed to the anomaly detector; when the detection object is target-level anomaly (such as detecting whether the trackside equipment is abnormal in the unmanned aerial vehicle line inspection task), the target detector is needed to detect the main target first, and then the anomaly detector is used to diagnose the anomaly of the target.

[0071] 4. Anomaly detector: the architecture is shown in Figure 3 , which specifically includes:

[0072] (1) Feature extraction and aggregation unit:

[0073] ;

[0074] (1.1) A pre-trained backbone network based on CNN or Transformer is used as a feature extractor to extract multi-scale features from the input image ( denotes the i-th stage scale feature), it should be noted that the pre-trained backbone network is obtained by pre-training on a large amount of data, and the pre-training method is preferably in a self-supervised manner. In the training stage of the anomaly detector, the weights of the pre-trained backbone network need to be frozen.

[0075] (1.2) The multi-scale features are obtained by size adjustment (i.e. Resize operation) and feature concatenation (i.e. Concat operation) to obtain aggregated features.

[0076] It should be noted that some previous solutions did not include this unit because their models fitted the distribution of the image itself, while this invention fits the distribution of image features in the subsequent feature distribution fitting unit. A major advantage of this invention is that it leverages the feature extraction capabilities of a pre-trained feature extractor to enhance the overall model's perceptual ability, thereby improving anomaly detection performance.

[0077] (2) Feature distribution fitting unit:

[0078] (2.1) Feature breathing mechanism: Before feature reconstruction, feature perturbation is first superimposed through linear layers. aggregation features Channel dilation is performed; after feature reconstruction, the number of channels is compressed back to the original number through a linear layer to enhance the expressive power of the model.

[0079]

[0080] (2.2) Asymmetric codec structure: Aggregation features after channel dilation The model is reconstructed using an encoder-decoder based on the Transformer architecture. In previous solutions, the number of encoder-decoder layers was generally the same, but simultaneously increasing the number of encoder-decoder layers did not improve performance due to increased training difficulty; furthermore, query embeddings existing only in the decoder layer had a significant impact on the model. Therefore, with a fixed number of attention layers, the depth of the decoder should be prioritized over the depth of the encoder. This invention employs a 4-layer encoder + 6-layer decoder architecture to balance performance and training difficulty.

[0081] (2.3) Self-attention decoder layer: such as Figure 4 As shown, in order to fully leverage the potential of query embedding, inspired by DETR, this invention specifically combines the various inputs of the decoder layer and designs the first attention layer as self-attention, thereby increasing the receptive field of the query embedding, enabling it to better learn global image information and produce obvious spatial distinctions.

[0082] (2.4) Training Loss: During training, the model's loss function is defined as the reconstructed features after feature compression. and original aggregation features MSE loss between.

[0083]

[0084] (3) Abnormal score calculation unit:

[0085]

[0086] As shown in the above equation, the anomaly segmentation map is based on the computation of the original multi-scale features. (the subscript b in the upper corner represents a subsequent optional feature layer) and the reconstructed features after feature compression The distance between the two is obtained.

[0087] It should be noted that: when obtaining the anomaly segmentation map, different feature layers can be selected according to the characteristics of the detection target to participate in the calculation. For example, in the unmanned aerial vehicle line inspection task, it is required to identify possible large-area geological disasters, at which time it is more suitable to calculate the distance measure using deep features (b=4); in the train component defect detection task, it is required to identify fine defects of various components, at which time it is more suitable to use full-size features (b=1, 2, 3, 4) for detection.

[0088] At the same time, in the selection of distance measurement methods, L2 distance or cosine similarity (CS) can be selected. When the types of anomalies are rich, the cosine similarity is better, and a larger value of lambda should be used; when the types of anomalies are single, the L2 distance is better, and a smaller value of lambda should be used.

[0089] After that, the mean value of the K largest points in the anomaly segmentation map is calculated, and the anomaly score of the test image can be obtained. The anomaly score can indicate the anomaly probability of the current image or target, and the larger the value, the higher the anomaly probability.

[0090] The present application optimizes the network structure and the strategy optimization of the anomaly score calculation method in three aspects of ① feature breathing mechanism ② asymmetric encoder-decoder structure ③ self-attention decoder layer based on the existing UniAD algorithm, thereby improving the model performance of the anomaly detection algorithm. The AUC performance of the present application in the full category also reaches 99.0%, even exceeding the performance of DRAEM in single category.

[0091] In order to more clearly explain the present application, the following provides an exemplary implementation process description of the visual anomaly detection method for rail transit operation and maintenance scene in two main application scenarios.

[0092] 1. Train component defect detection task (target-level anomaly detection):

[0093] (1) Model training phase:

[0094] 1) The image acquisition device of the TEDS system completes the acquisition of basic data, and the images are spliced in units of carriages;

[0095] 2) The train image data is transversely cropped to adapt to the input of the target detector, and a certain overlap interval needs to be reserved during cropping;

[0096] 3) Data labeling is performed based on the cropped images, and the labeled objects are the train components (such as covers, brake pads, etc.) to be detected, obtaining a data set A;

[0097] 4) Based on the labeling in dataset A, the calibration target is cropped and saved by category to obtain dataset B. By default, the components in dataset B are all normal components without any abnormalities or defects;

[0098] 5) The training of the target detector is completed using dataset A, and the training of the anomaly detector is completed using dataset B. Since there are no abnormal data in dataset B, only normal data exists, so the anomaly detector is an unsupervised learning model. Although the target detector is a supervised learning model, it also only needs normal data, which is relatively easy to obtain.

[0099] (2) Real-time detection phase:

[0100] 1) Real-time data acquisition is performed by the image acquisition device of the TEDS system, and the images are spliced by car;

[0101] 2) The image is cropped and fed to the target detector, and the interval and serial number of the cropped image are recorded;

[0102] 3) The target detector outputs the coordinates and categories of the detected train components, and the component target image is cropped according to the coordinates and fed to the anomaly detector;

[0103] 4) The anomaly detector outputs the abnormality determination result of the target component, and saves the related information to the temporary area;

[0104] 5) After all the target components of the car are determined to be abnormal, all the records in the temporary area are de-duplicated;

[0105] 6) For abnormal targets, the system performs alarm operation and records the time, train number, car number, abnormal component type and coordinates, etc. For normal targets, the system archives the related detection records.

[0106] 2. UAV line inspection task (both image-level anomaly detection and target-level anomaly detection):

[0107] (1) Model training phase:

[0108] 1) The photoelectric pod or image transmission system of the UAV completes the acquisition of basic data, and splices to build a complete orthographic image;

[0109] 2) The orthographic image is cropped to adapt to the subsequent model input, and a certain overlap interval is reserved to obtain dataset A. By default, there are no image-level anomalies such as geological disasters and severe waterlogging in dataset A;

[0110] 3) Data labeling is performed based on the cropped image, and the labeling object is the trackside equipment, steel rail or fastener, etc. to be detected to obtain dataset B;

[0111] 4) Based on the labeling in dataset B, the labeled targets are cropped and saved by category, and dataset C is obtained, in which all targets are normal or defect-free;

[0112] 5) Dataset A is used to train the anomaly detector I (for image-level anomaly), dataset B is used to train the target detector, and dataset C is used to train the anomaly detector II (for target-level anomaly). Similarly, the anomaly detectors I and II are unsupervised learning models, and the target detector is a supervised learning model.

[0113] (2) Real-time detection phase:

[0114] 1) The data of the current inspection area is acquired by the photoelectric pod or the image transmission system of the unmanned aerial vehicle, and a complete orthographic image is constructed by splicing;

[0115] 2) The image is cropped and the interval and serial number of the cropped image are recorded;

[0116] 3) The cropped image is fed to the anomaly detector I to obtain the image-level anomaly determination result and save the related information to the temporary area; the cropped image is fed to the target detector to obtain the coordinates and category of the equipment target, and the equipment target image is cropped according to the coordinates and fed to the anomaly detector II;

[0117] 4) The anomaly detector II outputs the anomaly determination result of the equipment target, and saves the related information to the temporary area;

[0118] 5) After the image-level and target-level anomaly determination of all cropped images in the orthographic image is completed, all records in the temporary area are de-duplicated;

[0119] 6) For abnormal areas or targets, the system performs alarm operation and records the time, anomaly type, coordinates and other information in time; for normal areas or targets, the system archives the related detection records.

[0120] It should be noted that the above method is to construct an orthographic image first and then perform algorithm processing. In actual operation, the above anomaly detection operation can also be performed on a single inspection image acquired in real time.

[0121] Therefore, the present application provides a visual anomaly detection method for a rail transit operation and maintenance scene, which can improve the missed detection risk of the existing detection system, improve the robustness and adaptability of the system to diversified anomalies (compared with the existing supervised learning scheme). For example, in the unmanned aerial vehicle line inspection task, it is required to identify the anomaly of the Class A equipment beside the track, and the present application does not need to include the anomaly data of the Class A equipment in the training data set, but only normal data. On the other hand, based on the improvement of the performance of the anomaly detection model, the system can realize the anomaly detection of multiple types of components or equipment in one model (compared with the existing unsupervised learning scheme). For example, in the train component defect detection task, the present application can complete the detection task of multiple components through one model without the need of stacking redundant sub-models.

[0122] Reference Figure 5 , Figure 5 FIG. 1 is a structural schematic diagram of a visual anomaly detection system for a rail transit operation and maintenance scene according to an embodiment of the present application.

[0123] As shown in FIG. 1, the visual anomaly detection system for the rail transit operation and maintenance scene according to the embodiment of the present application comprises: Figure 5 A data acquisition and preprocessing module 10 is configured to acquire rail transit operation and maintenance image data and pre-process the image data.

[0124] An image data transmission module 20 is configured to selectively transmit the pre-processed image data to a target detector for main target detection.

[0125] A feature extraction and aggregation module 30 is configured to transmit the pre-processed image data or the main target output by the target detector to a feature extraction and aggregation unit of an anomaly detector to obtain aggregated features.

[0126] A feature distribution fitting module 40 is configured to transmit the aggregated features to a feature distribution fitting unit of the anomaly detector, obtain compressed features based on a feature breathing mechanism, and establish a loss function based on the compressed features and the aggregated features to perform model training.

[0127] An anomaly score calculation module 50 is configured to, after the model training, generate an anomaly segmentation map of the input image data based on the multi-scale features and the compressed features, and determine an anomaly score of the input image data according to the anomaly segmentation map.

[0128] An anomaly alarm module 60 is configured to perform rail transit operation and maintenance anomaly alarm according to the anomaly score.

[0129] Other embodiments or specific implementation manners of the visual anomaly detection system for the rail transit operation and maintenance scene according to the present application can refer to the above-mentioned method embodiments, which will not be described here again.

[0130]

[0131] ​It is to be understood that the terms "one embodiment", "another embodiment", "other embodiments", "first embodiment", "second embodiment", etc. as may be found in the specification and / or in the claims, indicate that the alternative is included in at least one embodiment. These terms only specify the scope of claimable subject matter; and do not necessarily affect the scope of the application. Further, these terms only indicate particular embodiments of the applications. Other embodiments of the present application can be derived from the description, experimental results and / or the claims, without departing from the scope of the present application.

[0132] It is to be understood that the terms "including", "comprising", "consisting" and "consisting essentially of" or any other variation thereof, are intended to cover a non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements does not include only those elements but can include other elements not expressly listed or inherent to such process, method, article, or apparatus. An element proceeded by "comprises a... " does not, without more constraints, exclude the existence of additional identical elements in the process, method, article, or apparatus that comprises the element.

[0133] The above only describes the preferred embodiments of the present application, and does not limit the patent scope of the present application. Any equivalent structure or equivalent process transformation, or direct or indirect application in other related technical fields, which is made according to the content of the specification and drawings of the present application, is also included in the patent protection scope of the present application.

Claims

1. A visual anomaly detection method for rail transit operation and maintenance scenarios, characterized in that, The method comprises the following steps: S1: collecting rail transit operation image data, and pre-processing the image data; S2: selectively transmitting the pre-processed image data to a target detector for subject target detection; S3: transmitting the pre-processed image data or the subject target output by the target detector to a feature extraction and aggregation unit of an anomaly detector to obtain aggregated features; specifically comprising: S31: transmitting the pre-processed image data or the subject target output by the target detector to the feature extraction and aggregation unit of the anomaly detector, using a pre-trained backbone network based on CNN or Transformer as a feature extractor to extract multi-scale features from the input pre-processed image; S32: obtaining aggregated features by size adjustment and feature splicing; wherein the expression of the aggregated features obtained by size adjustment and feature splicing is specifically: ； In the formula, Resize indicates feature aggregation, Concat indicates feature concatenation, and Resize indicates feature resizing. Let m represent the characteristics of the i-th stage, and m represent the total number of stages. S4: transmitting the aggregated features to a feature distribution fitting unit of the anomaly detector, obtaining compressed features based on a feature breathing mechanism, and establishing a loss function based on the compressed features and the aggregated features to perform model training; wherein the compressed features obtained based on the feature breathing mechanism specifically comprise: S41: transmit the aggregated features to a feature distribution fitting unit of the anomaly detector, perform channel expansion on the aggregated features superimposed with the feature disturbance by a linear layer wherein the expression for performing the channel expansion is specifically:​ ； S42: aggregate features after channel expansion reconstructing by a Transformer architecture-based encoder-decoder to obtain reconstructed features ; S43: compress the reconstructed features to the original number of channels through a linear layer to obtain compressed features ; wherein the expression for channel compression is specific​ ； S44: Establishing a loss function based on compressed features with aggregated features to perform model training; wherein an expression of the loss function is specifically: ； wherein L represents the loss function, and MSE represents the mean square error; S5: after model training, generating an anomaly segmentation map of the input image data based on the multi-scale features and the compressed features, and determining an anomaly score of the input image data according to the anomaly segmentation map; specifically comprising: S51: after model training, selecting different feature layers to participate in calculation according to the characteristics of the detected target, and selecting distance measurement methods and adjustment parameters according to the richness of the anomaly types; S52: based on the selected feature layers, distance measurement methods and adjustment parameters, generating an anomaly segmentation map of the input image data for the multi-scale features and the compressed features, and the expression is specifically: ； In the formula, L2 is an L2 distance, CS represents a cosine similarity, and λ represents an adjustment parameter. denotes a multi-scale feature, and the superscript b denotes the bth feature layer, denotes a compressed feature of the bth feature layer. S53: calculating the mean value of the K points with the largest values in the anomaly segmentation map to obtain the anomaly score of the input image data; S6: performing rail transit operation anomaly alarm according to the anomaly score.

2. The visual anomaly detection method for rail transit operation and maintenance scene according to claim 1, wherein, In step S1, the rail transit operation image data is collected, specifically including: for unmanned aerial vehicle line inspection tasks, using a wireless image transmission system to collect data; and / or, for train component defect detection tasks, using an image acquisition subsystem of a TEDS system to collect data.

3. The visual anomaly detection method for rail transit operation and maintenance scene according to claim 1, wherein, In step S1, the image data is pre-processed, specifically including: using one or more of image stitching, image cropping, image denoising or histogram equalization for image preprocessing.

4. The visual anomaly detection method for rail transit operation and maintenance scene according to claim 1, wherein, In step S2, the pre-processed image data is selectively transmitted to the target detector for subject target detection, specifically including: S21: determining whether the detected object in the image data is an image-level anomaly or a target-level anomaly; S22: when the detected object is an image-level anomaly, directly transmitting the pre-processed image data to the constructed anomaly detector, and using the anomaly detector to output an anomaly detection result for the image data; S23: when the detected object is a target-level anomaly, detecting the subject target in the image data using the target detector, transmitting the subject target to the anomaly detector, and obtaining an anomaly detection result output by the anomaly detector for the subject target.

5. The visual anomaly detection method for rail transit operation and maintenance scene according to claim 1, wherein, In step S42, the Transformer architecture-based encoder-decoder is configured to adopt an asymmetric encoder-decoder structure composed of 4-layer encoder layers and 6-layer decoder layers; The decoder layer specifically comprises: The adjacent mask self-attention layer is configured to take the previous decoder layer output as an input V value vector, take the previous decoder layer output and the query embedding as an input K value vector, and take the previous decoder layer output and the query embedding as an input Q query vector; The adjacent mask mutual attention layer is configured to take the encoder output as an input V value vector, take the encoder output and the position embedding as an input K value vector, and take the adjacent mask self-attention layer output and the query embedding as an input Q query vector.

6. The visual anomaly detection method for rail transit operation and maintenance scene according to claim 1, wherein, In step S6, according to the anomaly score, track traffic operation and maintenance anomaly alarm is performed, specifically comprising: judging whether the anomaly score exceeds a preset score threshold, if yes, performing track traffic operation and maintenance anomaly alarm, if no, performing data archiving.

7. A visual anomaly detection system for rail transit operation and maintenance scenarios, characterized in that, The system comprises: A data acquisition and preprocessing module is configured to acquire track traffic operation image data and pre-process the image data; An image data transmission module is configured to selectively transmit the pre-processed image data to a target detector for subject target detection; A feature extraction and aggregation module is configured to transmit the pre-processed image data or the subject target output by the target detector to a feature extraction and aggregation unit of an anomaly detector to obtain aggregated features; specifically comprising: The pre-processed image data or the subject target output by the target detector is transmitted to the feature extraction and aggregation unit of the anomaly detector, and a pre-trained backbone network based on CNN or Transformer is used as a feature extractor to extract multi-scale features from the input pre-processed image; The multi-scale features are adjusted in size and spliced to obtain aggregated features; wherein the expression of the aggregated features obtained by size adjustment and feature splicing is specifically as follows: ； In the formula, Resize indicates feature aggregation, Concat indicates feature concatenation, and Resize indicates feature resizing. Let m represent the characteristics of the i-th stage, and m represent the total number of stages. A feature distribution fitting module is configured to transmit the aggregated features to a feature distribution fitting unit of the anomaly detector, obtain compressed features based on a feature breathing mechanism, and establish a loss function based on the compressed features and the aggregated features to perform model training; Wherein, the compressed features obtained based on the feature breathing mechanism specifically comprise: The aggregated features are transmitted to a feature distribution fitting unit of the anomaly detector, and the aggregated features superimposed with feature perturbations are processed through a linear layer The aggregated features Channel expansion is performed; wherein the expression of performing channel expansion is specifically as follows: ； Polymeric characteristics of the channel after expansion reconstructed by a Transformer architecture-based decoder, to obtain reconstructed features ; The reconstructed features are processed through a linear layer. The number of channels is compressed to the original number of channels to obtain the compression feature. The expression for channel compression is as follows: ； Establishing a loss function based on compressed features with aggregated features to perform model training; wherein an expression of the loss function is specifically: ； In the formula, L represents the loss function, and MSE represents the mean square error; An anomaly score calculation module is configured to, after model training, generate an anomaly segmentation map of input image data based on the multi-scale features and the compressed features, and determine an anomaly score of the input image data according to the anomaly segmentation map; specifically comprising: After model training, different feature layers are selected according to the characteristics of the detection target to participate in calculation, and distance measurement methods and adjustment parameters are selected according to the richness of the anomaly type; Based on the selected feature layers, distance measurement methods and adjustment parameters, an anomaly segmentation map of input image data is generated for the multi-scale features and the compressed features, and the expression is specifically as follows: ； In the formula, L2 is an L2 distance, CS represents a cosine similarity, and λ represents an adjustment parameter. denotes a multi-scale feature, and the superscript b denotes the bth feature layer, denotes a compressed feature of the bth feature layer. The maximum value of the K points in the anomaly segmentation map is calculated to obtain the anomaly score of the input image data; An anomaly alarm module is configured to perform track traffic operation and maintenance anomaly alarm according to the anomaly score.

Images