Multi-mode anomaly detection method for high-speed rail train control system

By adopting multimodal anomaly detection method in the high-speed rail train control system, combining log text and visual data, a text-visual multimodal learning model is established, and the problem of difficulty in troubleshooting and large impact in the existing technology is solved, and the problem of failure diagnosis and large impact range is achieved. Predict and narrow the effects of failures in advance.

CN118915692APending Publication Date: 2024-11-08CHINA STATE RAILWAY GRP CO LTD +4
View PDF 0 Cites 1 Cited by

Patent Information

Application Number
CN202410964359.4
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Priority Date
2024-03-13
Filing Date
2024-07-18
Publication Date
2024-11-08

AI Technical Summary

Technical Problem

The abnormal detection of existing high-speed rail train control systems is mainly carried out after the event, lacks an early prevention mechanism, has difficulty in troubleshooting and positioning, is inefficient, has low accuracy, and is unable to effectively combine the coupling and mutual influence relationship between equipment, resulting in the impact of the fault. The range is relatively large.

Method used

The multimodal anomaly detection method of the high-speed rail train control system is adopted, and the abnormality detection model based on text-visual multimodal learning is established by obtaining and processing the log text and lamp display visual data of the train control system, and the dual-flow architecture and attention mechanism are used. Coordinate the performance operation status of functions and performance to achieve fault prediction and narrow the scope of fault impact.

Benefits of technology

It realizes prediction before a system failure, narrows the scope of the fault impact, improves the timeliness and accuracy of fault diagnosis, solves the shortcomings of single mode detection, and enhances the overall status monitoring capability of the train control system.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN118915692A_ABST
    Figure CN118915692A_ABST
Patent Text Reader

Abstract

The invention discloses a multi-mode anomaly detection method for a high-speed rail train control system, and is applied to the technical field of train safety detection. The method comprises the following steps: acquiring a log text and lamp display visual data in the running process of a train control system as an original data set; the method comprises the following steps: preprocessing an original data set, labeling various types of anomalies in the original data set, and dividing according to a proportion to obtain a training set, a verification set and a test set with labels; establishing an anomaly detection model based on text-vision multi-mode learning of the high-speed rail train control system; inputting the training set and the verification set into an anomaly detection model to realize model training; inputting the test set into an anomaly detection model to realize performance evaluation; and applying the anomaly detection model to the high-speed rail train control system to realize system anomaly detection. According to the method, a text-vision fused double-flow multi-mode neural network anomaly detection model is built from the two aspects of function and performance, prediction is carried out before a system breaks down, and the purpose of narrowing the fault influence range is achieved.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present invention relates to the technical field of rail transit train safety detection, and more particularly to a method for multi-modal anomaly detection of a high-speed railway train control system. Background Art

[0002] The train control system is the nerve center that ensures the safe and efficient operation of high-speed trains. It works closely with the vehicle system to control the running interval and speed of the trains to prevent train rear-end collisions and derailments. For such an important and safety-critical system, any system abnormality caused by random failure of software or hardware or illegal attacks during system operation may have a great impact. It can be seen that accurate abnormality detection and fault diagnosis are inevitable requirements for ensuring driving safety.

[0003] Unfortunately, the current abnormal detection and response of my country's train control system are carried out after the fact, and there are no technical means to achieve advance prevention. There are deficiencies such as difficulty in fault diagnosis and positioning, low efficiency, and low accuracy. During the operation of the system, although there is monitoring of the equipment operating status, it mainly monitors the business functions of the equipment, does not involve the degradation of the performance status of the equipment, the monitoring information is limited, and there are no conditions for accurate fault positioning. In addition, fault location is basically completed manually, which is very time-consuming. It only considers the operating status of a single device, and cannot take into account the coupling and mutual influence relationship between the equipment to grasp the status of the system as a whole, or only uses the current value for a simple threshold comparison, without combining historical data, resulting in a large scope of fault impact. The two system failures that occurred on February 26, 2021 and May 19, 2023 caused train delays of more than 180 minutes, which is a typical case of a large impact range caused by this fault handling method.

[0004] At present, there are only two ways to characterize the health status of train control system equipment: logs generated in real time during system operation and equipment operation indicator lights. The former is a text record, and the latter is a visual record. There is heterogeneity and interconnectivity between these two different modalities. First, the modalities are heterogeneous, because information presented in different modalities usually exhibits different qualities, structures, and representations. Text logs focus on the business processing logic at the functional level of the equipment, while visual records focus on the degradation state at the performance level of the equipment. At the same time, the structural forms of the two data are also different. Secondly, the two modalities are not independent entities, but are interrelated. They serve the same goal, that is, to jointly characterize the operating status of the system equipment, and have an obvious dependency relationship in the same time dimension.

[0005] Accurate anomaly perception and detection is a prerequisite for the normal execution of railway transportation process control and operation plan. With the development of emerging technologies such as machine learning, most anomaly detection methods are currently based on data-driven deep learning methods. Due to the particularity of scenarios and environments, it is a general consensus to study targeted anomaly detection methods for specific equipment.

[0006] For train control systems, there is no technical means to monitor equipment use and performance degradation. Only natural language logs and visual indicator light information can be used as data sources for anomaly detection. At present, there is no literature report on the anomaly detection method of train control systems that integrates text-visual multimodal learning.

[0007] Therefore, it is an urgent problem for technical personnel in this field to propose a multimodal anomaly detection method for a high-speed railway train control system to solve the difficulties existing in the prior art. Summary of the invention

[0008] In view of this, the present invention provides a multi-modal anomaly detection method for a high-speed railway train control system, which organically integrates the two levels of function and performance, predicts system failures before they occur, and achieves the purpose of reducing the scope of the failure impact.

[0009] In order to achieve the above object, the present invention provides the following technical solutions:

[0010] A multi-modal anomaly detection method for a high-speed railway train control system comprises the following steps:

[0011] S1, data acquisition step: obtaining log text and light display visual data during the operation of the train control system as the original data set;

[0012] S2, data processing step: structure the original data set in S1, remove duplicate data, mark various anomalies in the original data set, and divide it into training set, validation set and test set with annotations in proportion;

[0013] S3, model building step: building an anomaly detection model based on text-visual multimodal learning of high-speed rail train control system;

[0014] S4, model training and evaluation steps: input the training set and validation set in S2 into the anomaly detection model in S3 to implement model training; input the test set in S2 into the anomaly detection model in S3 to implement performance evaluation;

[0015] S5, system anomaly detection step: Apply the anomaly detection model trained and evaluated in S4 to the high-speed rail train control system to realize system anomaly detection.

[0016] Optionally, the anomaly detection model in S3 collaboratively characterizes the functional and performance operating status of the train control system from the two dimensions of text and vision, designs a dual-stream text-visual learning architecture, introduces independent log text and visual processing streams, and then performs modal interaction through a common decision layer.

[0017] Optionally, the anomaly detection model in S3 is divided into three layers: an embedding layer, a feature layer, and an output layer; wherein the embedding layer includes a text embedder and a visual embedder; the feature layer includes an attention module; and the output layer includes a decision layer fusion module.

[0018] Optionally, the attention module includes a multi-head attention, a forward propagation network, a linear layer and a Softmax layer; both the multi-head attention and forward propagation network modules apply residual connections, followed by a layer normalization module, and the linear layer obtains an output of a specified dimension by a linear transformation of the previous step, and the converted dimension corresponds to the number of output categories; the Softmax layer further outputs the classification probability.

[0019] Optionally, the device anomaly detection model in S5 implements the specific steps of device anomaly detection as follows:

[0020] The system operation log text data and light display visual data in the input sequence are converted into corresponding text vectors and image vectors through their respective embedders, and the position vector of each input is added to reflect the position of the text or image in the sequence; then the text vector and image vector of the integrated position vector are respectively input into the attention module, and the relationship between the inputs is learned through the self-attention mechanism to encode its contextual information, and then the vector representation of each log or image with integrated contextual features is output through a feedforward network; finally, the vector representation of each output log or image is respectively input into the decision layer fusion module, and the vector representation of each input log or image is integrated through the algebraic combination rules of multiple prediction class labels to obtain the final device abnormality discrimination result.

[0021] Optionally, the system operation log text data and light display visual data in the input sequence are converted into corresponding text vectors and image vectors through their respective embedders. The specific steps are as follows:

[0022] In the text embedder: an embedding layer is used to map the input text into a low-dimensional dense vector space, and a tokenizer is designed to convert the log text into an embedding layer text vector with subwords as the minimum granularity for model learning;

[0023] In the visual embedder: the image data format is a three-dimensional matrix of [H, W, C]. The H*W*C image is divided into several P*P blocks to obtain a discrete sequence with semantic information. The number of small blocks is N = H / P*W / P. Then each block is mapped to a one-dimensional vector through linear mapping. The conversion process is implemented through a convolutional layer.

[0024] It can be seen from the above technical solutions that, compared with the prior art, the present invention discloses a method for multi-modal anomaly detection of a high-speed railway train control system, which has the following beneficial effects:

[0025] (1) This paper proposes a unified text-visual multimodal learning anomaly detection method for high-speed rail train control systems, which for the first time collaboratively characterizes the function and performance status of the train control system from two dimensions: text and vision.

[0026] (2) Based on the complex coupling scenarios and characteristics of log text and visual images, which are both independent and related, a dual-stream text-visual learning architecture is designed to introduce independent log text and visual processing streams, and then perform modal interaction through a common decision layer;

[0027] (3) The attention mechanism is used throughout the process to build a fully connected neural network model, focusing on the long-distance contextual dependencies of text sequences and tracking of small targets with visual changes at specific locations; through multimodal anomaly detection with organic fusion of text and vision, the problem of low timeliness and accuracy of fault diagnosis in high-speed rail train control systems is solved;

[0028] (4) The present invention proposes a text-vision based multimodal anomaly detection method, which independently learns the information of the text modality and the visual modality, and then realizes the communication and conversion of the two modal information, organically integrating them from the functional and performance levels, and predicts the system failure before it occurs, so as to achieve the purpose of reducing the scope of the failure. BRIEF DESCRIPTION OF THE DRAWINGS

[0029] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on the provided drawings without paying creative work.

[0030] Figure 1 A flow chart of a method for multi-modal anomaly detection of a high-speed railway train control system provided by the present invention;

[0031] Figure 2 A system architecture diagram of a multi-modal anomaly detection model for a high-speed railway train control system provided by the present invention;

[0032] Figure 3 A schematic diagram of a multi-modal feature representation of a high-speed railway train control system provided by the present invention;

[0033] Figure 4 A schematic diagram of multi-head attention calculation for multi-modal anomaly detection in a high-speed railway train control system provided by the present invention. DETAILED DESCRIPTION

[0034] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0035] like Figure 1 As shown, the present invention discloses a multi-modal anomaly detection method for a high-speed railway train control system, comprising the following steps:

[0036] S1, data acquisition step: obtaining log text and light display visual data during the operation of the train control system as the original data set;

[0037] S2, data processing step: structure the original data set in S1, remove duplicate data, mark various anomalies in the original data set, and divide it into training set, validation set and test set with annotations in proportion;

[0038] S3, model building step: building an anomaly detection model based on text-visual multimodal learning of high-speed rail train control system;

[0039] S4, model training and evaluation steps: input the training set and validation set in S2 into the anomaly detection model in S3 to implement model training; input the test set in S2 into the anomaly detection model in S3 to implement performance evaluation;

[0040] S5, system anomaly detection step: Apply the anomaly detection model trained and evaluated in S4 to the high-speed rail train control system to realize system anomaly detection.

[0041] Furthermore, the anomaly detection model in S3 collaboratively characterizes the function and performance status of the train control system from two dimensions: text and vision. A dual-stream text-visual learning architecture is designed to introduce independent log text and visual processing streams, and then perform modal interaction through a common decision layer. Figure 2 shown.

[0042] Furthermore, the device anomaly detection model in S3 is divided into three layers: embedding layer, feature layer, and output layer; the embedding layer includes a text embedder and an image embedder; the feature layer includes an attention module; and the output layer includes a decision layer fusion module.

[0043] Furthermore, the attention module includes multi-head attention, forward propagation network, linear layer and Softmax layer; both the multi-head attention and forward propagation network modules apply residual connection, followed by a layer normalization module, and the linear layer obtains the output of the specified dimension through the linear transformation of the previous step, and the converted dimension corresponds to the number of output categories; the Softmax layer further outputs the classification probability.

[0044] Furthermore, the device anomaly detection model in S5 implements the specific steps of device anomaly detection as follows:

[0045] The train log text data and train equipment visual images in the input sequence are converted into corresponding text vectors and image vectors through their respective embedders, and the position vector of each input is added to reflect the position of the text or image in the sequence; then the text vector and image vector of the integrated position vector are respectively input into the attention module, and the relationship between the inputs is learned through the self-attention mechanism to encode its contextual information, and then the vector representation of each log or image with integrated contextual features is output through a feedforward network; finally, the vector representation of each output log or image is respectively input into the decision layer fusion module, and the vector representation of each input log or image is integrated through the algebraic combination rules of multiple prediction class labels to obtain the final equipment abnormality discrimination result.

[0046] Furthermore, the system operation log text data and light display visual data in the input sequence are converted into corresponding text vectors and image vectors through their respective embedders. The specific steps are as follows:

[0047] In the text embedder: an embedding layer is used to map the input text into a low-dimensional dense vector space, and a tokenizer is designed to convert the log text into an embedding layer text vector with subwords as the minimum granularity for model learning;

[0048] In the visual embedder: the image data format is a three-dimensional matrix of [H, W, C]. The H*W*C image is divided into several P*P blocks to obtain a discrete sequence with semantic information. The number of small blocks is N = H / P*W / P. Then each block is mapped to a one-dimensional vector through linear mapping. The conversion process is implemented through a convolutional layer.

[0049] Furthermore, the anomaly detection model of this patent is based on a dual-stream architecture to ensure the independent representation of each modality, and the tokens of each modality, text and vision, only focus on the information within its own modality. In addition, a separate fusion information stream is added to deal with the decision-making information interaction between the two modalities. This design scheme not only ensures the purity of the single-modal information flow, but also allows multi-modal interaction and fusion, decoupling the intra-modal and inter-modal interactions. It can be seen that the anomaly detection model of the present invention is guided by actual business needs, and effectively depicts the complex scenarios in which the equipment log text and visual images of the train control system are independent and related to each other.

[0050] In addition to the special architecture design adapted to the actual scenario, special designs are also made in the following aspects:

[0051] (1) The entire network structure consists only of self-attention and feedforward networks, which can simultaneously calculate the representations of all input positions, realize parallel computing, and greatly improve computing efficiency.

[0052] (2) The self-attention mechanism is used to process each input as a fully connected graph through global aggregation characteristics, directly capturing the dependency between any two positions and realizing the long-distance dependency of text sequences.

[0053] (3) Divide the visual image into blocks and then compress the blocks into a sequence to achieve parallel training while making full use of global information.

[0054] (4) By using the multi-head attention idea, multiple virtual subspaces are set up to characterize the behavior of different devices generating logs and images at the same time, thereby enhancing the performance of the model.

[0055] (5) Process the characteristics of multiple streams and fused information streams separately, so that the detection errors of single streams are independent of each other and will not cause the superposition of errors. Specifically include:

[0056] 1. Multimodal Learning Architecture

[0057] Operation data can reflect the dynamic characteristics of the system and the context information of system failures, and has good expression and detection capabilities for unknown failures. The operation data of the train control system mainly includes real-time system logs and indicator light information of the equipment operation status.

[0058] Abnormal state detection based on operating data is a key step in the safe and reliable operation of the train control system. Its fundamental purpose is to find abnormal patterns in the system operating data that do not conform to the expected behavior. In order to systematically solve the problem of a large range of fault impact, the information value existing in the text and visual operating data is mined to achieve accurate anomaly detection.

[0059] The essence of text-visual multimodal anomaly detection is to use neural networks to estimate the conditional probability p(x t |x 1:(t-1) ). p(x t |x 1:(t-1) ) indicates that according to historical data x 1:(t-1) Predict future data x t The conditional probability of , 1:(t-1) represents the 1st moment to the t-1th moment, and t represents a future moment, which is the idea of ​​a deep sequence model. Convolutional or recurrent neural networks can only establish local dependencies between input information. To establish long-distance dependencies, the most direct way is to use a fully connected network. In actual transportation tasks, the lengths of log texts and visual images generated by the train control system are not the same. For different input lengths, the connection weights should also be different. The patent of this invention "dynamically" generates weights of different connections through the attention mechanism. Therefore, the key to establishing a multimodal anomaly detection model for a train control system is to design a fully connected network and an attention layer.

[0060] The overall architecture design of the network depends on the relationship between the two modes in the actual business. With the help of collaborative thinking, the information of each mode is mapped separately, but it is guaranteed that there are certain constraints between the two modes after mapping, so that they enter the parallel mutually mapped collaborative space. The "constraint" here refers to the functional and performance coupling relationship of the same train control device at the same time, that is, there is a correlation between the logical function and performance degradation. On the one hand, the train control system cannot complete the specified vehicle control function due to the performance degradation of electronic components, which is manifested as abnormal visual information and log text information of the device. On the other hand, the equipment has good performance, but it is interfered by the outside world or the information interaction with other devices is abnormal, which is manifested as the functional logic of the device that should be output is disordered. In this case, only the text information is abnormal but there is no visual abnormality such as indicator light display. Therefore, although there is no essential one-to-one causal relationship between the text and visual modes of the train control system, they are not completely independent of each other. The two show a complex coupling relationship.

[0061] In view of the relationship between the two modalities, a dual-stream modal learning architecture is designed with two parallel streams, each of which focuses on one modality, text or image. The prediction results of each modality model are then fused to make the final decision. The advantage of this fusion strategy is that the errors of the fusion model come from two different classifiers, text and visual, and the errors of different classifiers are independent of each other and do not affect each other, so they will not cause further accumulation of errors.

[0062] Figure 2In the , the attention module is mainly composed of multi-head attention, forward propagation network, linear layer and Softmax layer. In order to effectively solve the gradient vanishing problem and make the neural network converge quickly, the multi-head attention and forward propagation network modules are both applied with residual connection, followed by a layer normalization module. The linear layer obtains the output of the specified dimension by linear transformation of the previous step, and the transformed dimension corresponds to the number of output categories. The Softmax layer further outputs the classification probability.

[0063] 2. Multimodal feature representation

[0064] Modal representation is the basis of multimodal deep learning. The neural network model requires the input to be a real vector, so some conversion needs to be done on the log text and visual images before inputting into the model.

[0065] On the text side, it is implemented by word segmentation and embedding. The overall idea is to map the input text to a low-dimensional dense vector space through an embedding layer. Design a word segmenter to convert the log text into an embedding layer with subwords as the minimum granularity for model learning. This solution retains semantic independence to a certain extent, while ensuring that the vocabulary is not too large. Specifically, the WordPiece algorithm is used to construct a subword vocabulary using the combination with the highest probability of occurrence. That is, WordPiece selects two subwords to be merged each time, which usually appear simultaneously in the corpus in an adjacent manner. In addition to completing the basic word segmentation function, the word segmenter also uses a unified method to add tokens that appear in the training corpus and are not originally in the vocabulary to the vocabulary, and then identifies and processes special tokens. Special tokens include [CLS], [SEP], etc. [CLS] represents the beginning of a sentence, and [SEP] represents the end of a sentence. After word segmentation, the one-hot vector encoding of the token is converted into a denser embedding encoding. The essence of its core embedding matrix is ​​a lookup table. Each word will locate a row in this table, and this row is the semantics of the word learned in the embedding space.

[0066] Visual modalities are divided into image modalities and video modalities. The video modality is an image sequence after being expanded in the time dimension. Therefore, the key to learning the vector representation of visual modalities is the vector representation of image modalities. For image data, its data format is a three-dimensional matrix of [H, W, C]. In the performance representation image of train control equipment, the physical position representing specific information is fixed. This feature is conducive to image recognition. It only needs to recognize the color change of the indicator light at a specific position. Therefore, the goal of visual input is lightweight and fast. The specific approach is to remove the deep embedding used for visual input and implement shallow non-convolutional embedding at the pixel level. The overall idea is to divide the image of size H*W*C into several blocks of size P*P to obtain a discrete sequence with semantic information. The number of small blocks is N=H / P*W / P, and then each block is mapped to a one-dimensional vector through linear mapping. The conversion process is implemented through a convolutional layer. A D-dimensional vector is obtained, which is the block embedding code.

[0067] Since the self-attention mechanism ignores the x in the sequence X i Therefore, it is necessary to add position coding correction when initializing the sequence. Let H (0) =[e1+p1,...,e t +p t ] In the formula, e t For text and images x t The embedding vector representation of t For position encoding, the vector representation at position t is defined as follows:

[0068] In the formula, p t,2i is the 2ith dimension of the encoding vector at the tth position, and D is the dimension of the encoding vector.

[0069] Each input is a complete transport task performed by the train control system, that is, the log text and visual image corresponding to the time period from system startup to shutdown. First, the inputs of both are aligned on a unified time dimension. Second, the text and visual embeddings are added with their corresponding modality type embedding vectors and position embedding vectors. Finally, the text and visual vector sequences are input into the two-stream model respectively. The feature representation diagram of the sequence input is shown in the figure below. Figure 3 shown.

[0070] Figure 3 In the figure, the * part indicates the [CLS] token, where the top line is the text token and the bottom line is the visual token. The yellow part indicates the category of the modality, where 0 indicates text and 1 indicates visual. The light blue and dark blue parts indicate the positional embeddings of text and visual, respectively. The light gray and dark gray parts indicate the tokens obtained after the text and visual pass through the embedding layer, respectively.

[0071] With the idea of ​​collaborative representation, a separate embedding model is designed for each modality, and the complementarity of multimodal data is fully utilized by extracting and representing the feature information of text + visual modality data.

[0072] 3. Multimodal Fusion

[0073] The main goal of text and visual modality fusion is to reduce the heterogeneity between modalities while maintaining the integrity of the specific semantics of each modality and achieving optimal performance in the deep learning model. For train control equipment, the model learns independently at the feature layer, and the fusion of the two modalities occurs later, which belongs to the back-end fusion mode. An "or" logic operation is performed at the output layer to output the final fusion result.

[0074] In the task of abnormality detection of train control system, the input source is log text and visual image sequence, and the output target is probability distribution. It can be seen that the input and output are different. Therefore, the attention mechanism should act between the source and the target, and should also act between the internal elements of the source. In addition, the network connection weights of variable-length input sequences are different, and the attention mechanism needs to be used to dynamically generate the weights of different connections. The traditional attention mechanism ignores the dependency between the elements of the source or target itself, while the self-attention mechanism calculates the attention function at the source and target according to their respective inputs, captures the mutual dependency between the elements of the sequence itself, and then adds the correlation obtained at the source to the target to capture the dependency between the elements of the source and target. Therefore, the self-attention mechanism can not only obtain the dependency between the source and the target, but also obtain the dependency between the elements of the source itself, which is better. Therefore, the self-attention mechanism is introduced to characterize the dependency between the system log text and visual images.

[0075] The input of the model feature layer is the text vector and image vector of the comprehensive position vector, and the output is a vector representing historical information. The core of the mapping from input to output is a series of self-attention layers. The specific implementation steps are as follows:

[0076] (1) All input sequences X are mapped into an abstract continuous representation that contains the learning information of the entire sequence. The output of the attention module network is denoted as H = [h1,h2,…,h t ], any h i ∈H represents the hidden state of the network at time i.

[0077] First, X is input into three different fully connected feedforward networks to obtain the key vector matrix K corresponding to the s-th sub-device s , value vector matrix V s and the query vector matrix Q s , calculated as:

[0078]

[0079]

[0080] in, and is the network parameter. We can get: K s =[k s1 ,k s2 ,...,k st ]V s =[v s1 ,v s2 ,...,v st ]Q s =[q s1 ,q s2 ,...,q st ]in, q si ∈Q s .

[0081] Then, for time l, calculate the attention function corresponding to the s-th sub-device

[0082] Among them, h sl-1 It represents the network output state corresponding to the s-th sub-device at time l-1, which is used as the query vector. The query vector at the initial time is q s1 . It represents the attention distribution corresponding to the s-th sub-device, and its calculation formula is as follows:

[0083] Softmax is the activation function, s(k si ,h sl-1 ) is the attention scoring function, usually calculated by the scaled dot product model:

[0084] Where D is the vector k si Dimension.

[0085] During the operation of the train control system, multiple sub-devices such as the on-board ATP security computer (Vital Computer, VC), speed and distance processing unit (Speed ​​& Distance Processing Unit, SDU), and balise transmission module (BTM) generate operation logs and image data simultaneously. In order to process this information in parallel, the multi-head self-attention is used to calculate the attention function ca l The calculation is shown as follows Figure 4 shown.

[0086] It should be noted that during the calculation of the attention function corresponding to each sub-device, different parameter matrices W will be randomly initialized. k With W v , so different key vector matrices and value vector matrices will be obtained, and thus, the query vectors used as the hidden state at the previous moment will also be different. Therefore, although the input X of each attention mechanism is the same, the attention function of each device is calculated are different.

[0087] (2) Given the output hidden state H′ of the previous layer, the output hidden state H of this layer is obtained through a multi-head attention layer and a nonlinear forward propagation network. During the calculation, residual connection and layer normalization operations are required, specifically:

[0088] H′=LayerNorm(SelfAttention(X)+X)H=LayerNorm(FFN(H′)+H′)Where SelfAttention(·) represents the attention module, the attention function As shown in the above formula, LayerNorm(·) represents the layer normalization operation.

[0089] The forward propagation network consists of a fully connected layer plus a nonlinear activation function, which can act on each position separately, that is,

[0090] FFN(H′)=GELU(H′W1+b1)W2+b2, where H′ represents the output of the previous layer, GELU(·) is the Gaussian error linear activation function, and W1, b1, W2 and b2 represent the weights and biases of the two linear transformations.

[0091] (3) The attention network is stacked in N layers, with each layer receiving input from the previous layer. By stacking, the model can extract and focus on different attention combinations from its attention heads, thereby potentially improving the prediction ability. Finally, the output probability is calculated through a fully connected feedforward neural network.

[0092] (4) Decision-making level fusion. The prediction results of the textual modality and the visual modality are logically ORed to make the final decision. That is, as long as the prediction result of any modality is abnormal, the model will output an abnormality. Based on the abnormal detection results of the model and combined with the existing resource information, a series of equipment maintenance guarantee suggestions are proactively provided to the driver or dispatcher.

[0093] In summary, the sequence encoding based on the self-attention mechanism is actually a fully connected feedforward neural network, where each position in the lth layer receives the output of all positions in the l-1th layer. The difference is that its connection weights are dynamically calculated through the attention mechanism, thereby obtaining the long-distance dependency relationship between each element in the input sequence and other elements. Due to the focus feature of the attention mechanism and the parallel feature of multi-head attention, the feature extraction and fusion problem of the two modes in the abnormal detection of the train control system is solved.

[0094] In this specification, each embodiment is described in a progressive manner, and each embodiment focuses on the differences from other embodiments. The same or similar parts between the embodiments can be referred to each other. For the device disclosed in the embodiment, since it corresponds to the method disclosed in the embodiment, the description is relatively simple, and the relevant parts can be referred to the method part.

[0095] The above description of the disclosed embodiments enables one skilled in the art to implement or use the present invention. Various modifications to these embodiments will be apparent to one skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of the present invention. Therefore, the present invention will not be limited to the embodiments shown herein, but rather to the widest scope consistent with the principles and novel features disclosed herein.

Claims

1. A multi-modal anomaly detection method for a high-speed railway train control system, characterized in that: The following steps are involved: S1, data acquisition step: obtaining log text and light display visual data during the operation of the train control system as the original data set; S2, data processing step: structure the original data set in S1, remove duplicate data, mark various anomalies in the original data set, and divide it into training set, validation set and test set with annotations in proportion; S3, model building step: building an anomaly detection model based on text-visual multimodal learning of high-speed rail train control system; S4, model training and evaluation steps: input the training set and validation set in S2 into the anomaly detection model in S3 to implement model training; input the test set in S2 into the anomaly detection model in S3 to implement performance evaluation; S5, system anomaly detection step: Apply the anomaly detection model trained and evaluated in S4 to the high-speed rail train control system to realize system anomaly detection.

2. The multi-modal anomaly detection method for a high-speed railway train control system according to claim 1, characterized in that: The anomaly detection model in S3 collaboratively characterizes the functional and performance operating status of the train control system from two dimensions: text and vision. A dual-stream text-visual learning architecture is designed, which introduces independent log text and visual processing streams, and then performs modal interaction through a common decision layer.

3. The multi-modal anomaly detection method for a high-speed railway train control system according to claim 2 is characterized in that: The anomaly detection model in S3 is divided into three layers: embedding layer, feature layer, and output layer; the embedding layer includes text embedder and visual embedder; the feature layer includes attention module; and the output layer includes decision layer fusion module.

4. The multi-modal anomaly detection method for a high-speed railway train control system according to claim 3 is characterized in that: The attention module includes multi-head attention, forward propagation network, linear layer and Softmax layer; both multi-head attention and forward propagation network modules apply residual connection, followed by a layer normalization module. The linear layer obtains the output of the specified dimension by linear transformation of the previous step, and the transformed dimension corresponds to the number of output categories. The Softmax layer further outputs classification probabilities.

5. The method for detecting multi-modal anomaly of a high-speed railway train control system according to claim 1, characterized in that: The specific steps of implementing device anomaly detection in the device anomaly detection model in S5 are as follows: The system operation log text data and light display visual data in the input sequence are converted into corresponding text vectors and image vectors through their respective embedders, and the position vector of each input is added to reflect the position of the text or image in the sequence; then the text vector and image vector of the integrated position vector are respectively input into the attention module, and the relationship between the inputs is learned through the self-attention mechanism to encode its contextual information, and then the vector representation of each log or image with integrated contextual features is output through a feedforward network; finally, the vector representation of each output log or image is respectively input into the decision layer fusion module, and the vector representation of each input log or image is integrated through the algebraic combination rules of multiple prediction class labels to obtain the final device abnormality discrimination result.

6. A high-speed railway train control system multi-modal anomaly detection method according to claim 5, characterized in that: The system operation log text data and light display visual data in the input sequence are converted into corresponding text vectors and image vectors through their respective embedders. The specific steps are as follows: In the text embedder: an embedding layer is used to map the input text into a low-dimensional dense vector space, and a tokenizer is designed to convert the log text into an embedding layer text vector with subwords as the minimum granularity for model learning; In the visual embedder: the image data format is a three-dimensional matrix of [H, W, C]. The H*W*C image is divided into several P*P blocks to obtain a discrete sequence with semantic information. The number of small blocks is N = H / P*W / P. Then each block is mapped to a one-dimensional vector through linear mapping. The conversion process is implemented through a convolutional layer.

Citation Information

Cited By

  • Rail locomotive operation and maintenance data intelligent monitoring method and system and electronic equipment

    CN120296430A