High-speed rail signal system fault identification method based on deep learning

Abstract

The invention discloses a high-speed rail signal system fault identification method based on deep learning, and relates to the technical field of natural language processing and high-speed rail traffic safety. Comprising the following steps that fault data of the high-speed rail signal system are collected, a Token sequence including fault phenomenon description and fault type labels is formed, a deep learning model suitable for fault recognition of the high-speed rail signal system is constructed, the model comprises a BERT word embedding layer, a BiGRU context semantic obtaining layer, an MHSA self-attention mechanism layer and a full connection layer, and the model is used for recognizing the fault of the high-speed rail signal system. Inputting the Token sequence into the deep learning model to train and optimize the model weight, and obtaining a trained deep learning model; and inputting to-be-detected data into the trained model, and outputting to obtain the fault category of the real high-speed rail signal system. The high-speed rail signal system fault diagnosis method can improve the accuracy and efficiency of high-speed rail signal system fault diagnosis.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of natural language processing and high-speed rail traffic safety, and particularly relates to a high-speed rail signal system fault identification method based on deep learning. BACKGROUND

[0002] The high-speed rail signal system is the core technology to ensure the safe and efficient operation of trains, responsible for train positioning, speed control and safety distance maintenance. It includes track circuits, transponders, train control centers, wireless block centers, interlocking, etc. Fault data is usually recorded in the form of natural language description, containing professional terms and context information. In recent years, deep learning technology has been applied in fault diagnosis, and through the analysis of fault description text, automatic classification can be realized, reducing manual intervention and improving maintenance efficiency.

[0003] The existing technology has many problems in high-speed rail signal system fault identification, such as insufficient semantic understanding, rules-based methods that are difficult to capture professional terms and context relationships in fault descriptions, leading to low recognition accuracy in complex fault scenarios; low recognition rate for minority classes: due to the imbalance of high-speed rail signal system fault data, existing methods have not effectively optimized minority classes, and the recognition rate is not ideal; limited generalization ability: existing methods lack adaptability to diverse fault scenarios and cannot handle texts with multiple descriptions or noise interference.

[0004] Therefore, to solve the problems existing in the prior art, a high-speed rail signal system fault identification method based on deep learning is provided, which is a problem that needs to be solved by those skilled in the art. SUMMARY

[0005] Therefore, the present application provides a high-speed rail signal system fault identification method based on deep learning to solve the problems of insufficient semantic understanding, weak time series modeling, low recognition rate for minority classes, and limited generalization ability in the prior art, and to improve the accuracy and efficiency of high-speed rail signal system fault diagnosis.

[0006] In order to achieve the above-mentioned purpose, the present application adopts the following technical solutions: A high-speed rail signal system fault identification method based on deep learning, comprising the following steps: Collecting high-speed rail signal system fault data to form a Token sequence including fault phenomenon description and fault type label, which is used as an initial data set; Building a deep learning model suitable for high-speed rail signal system fault identification, the model including a BERT word embedding layer, a BiGRU context semantic acquisition layer, an MHSA self-attention mechanism layer, and a full connection layer; Inputting the initial data set into the deep learning model for training, using Focal Loss to optimize the model weight, and obtaining the trained deep learning model; The data to be detected is input into the trained model, and the output obtains the fault categories of the real high-speed rail signal system.

[0007] Optionally, there are nine types of faults, including: track circuits, transponders, train control center, radio block center, interlocking, centralized dispatching system, onboard problems, temporary speed limit server and others.

[0008] Optionally, the BERT word embedding layer is used to encode the token sequence to generate a high-dimensional semantic feature vector, including: BertTokenizer is used to convert the segmented fault descriptions into a sequence of token IDs. Using the BERT-base-chinese model, the token ID sequence is input into the 12-layer Transformer encoder of the BERT-base-chinese model to generate a 768-dimensional feature vector containing contextual semantics. and CLS features The expression is:

[0009] in, This is a sequence of fault descriptions after word segmentation. For batch size, The sequence length is given.

[0010] Optionally, the BiGRU context semantic acquisition layer employs a bidirectional GRU network to model the temporal dependency of fault descriptions, expressed as: ; in, and These are the hidden states of the forward and backward GRU, respectively. It is a bidirectional hidden state sequence, which makes up for BERT's weak modeling of sequence order and enhances the extraction of causal relationships.

[0011] Optionally, the MHSA self-attention mechanism layer is used to highlight key fault characteristics and improve the ability to distinguish between similar categories, including: The MHSA layer embeds a dimension of 768, with 12 headers, each with a dimension of 64, and outputs the BiGRU. Weighted features are generated using average pooling, with the following formula:

[0012] in, The query, key, and value vectors output by BiGRU , respectively Size corresponds to Projection matrix of direction, h represents the number of attention heads. To output the projection matrix, For average pooling operation, It is a feature vector of fixed length.

[0013] Optionally, the fully connected layer is used to concatenate the CLS token features of BERT. and MHSA pooling characteristics Generate a 1536-dimensional feature vector using the following formula:

[0014] in Pooling features for multi-head self-attention mechanisms This is the 1536-dimensional feature vector obtained by concatenating these two features; The first fully connected layer maps the 1536-dimensional features to 384 dimensions, using ReLU activation and BatchNorm, as shown in the formula:

[0015] in This is the weight matrix of the first fully connected layer. , For activation functions; The second fully connected layer maps the 384-dimensional features to 9-dimensional logits, as shown in the formula:

[0016] in This is the weight matrix of the second fully connected layer. For bias terms, The output consists of 9 categories of logits, where 9 corresponds to the number of fault type categories; Nine types of fault probabilities are generated using the softmax function, with the following formula:

[0017] in, Let be the probability vector for the fault category. For the first The probability of a type of failure.

[0018] As can be seen from the above technical solutions, compared with the prior art, the present invention provides a fault identification method for high-speed rail signaling systems based on deep learning, which has the following beneficial effects: 1) The present invention uses the BERT word embedding layer and the bert-base-chinese model to generate 768-dimensional semantic vectors, capture professional terms and contextual relationships, solve the problem of insufficient semantic understanding, and improve the identification accuracy of complex fault descriptions; 2) The present invention uses the BiGRU context semantic acquisition layer to model temporal dependencies through bidirectional GRU, make up for the weak modeling of sequence order by BERT, extract causal relationships, and enhance the ability to understand multi-segment faults; 3) The present invention's MHSA layer and fully connected layer focus on key fault features, improve the ability to distinguish similar categories, optimize the identification performance of a few categories, and improve the robustness and maintenance efficiency of the system. Attached Figure Description

[0019] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on the provided drawings without creative effort.

[0020] Figure 1 This is a flowchart of a high-speed rail signaling system fault identification method based on deep learning disclosed in this invention; Figure 2 This is a schematic diagram of the deep learning model for fault identification in high-speed rail signaling systems disclosed in this invention. Detailed Implementation

[0021] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0022] Reference Figure 1 As shown, this invention discloses a fault identification method for high-speed rail signaling systems based on deep learning, comprising the following steps: Collect fault data from the high-speed rail signaling system to form a token sequence that includes fault phenomenon descriptions and fault type labels, and use it as the initial dataset; Constructing a deep learning model suitable for fault identification in high-speed rail signaling systems, such as... Figure 2 As shown, the model includes a BERT word embedding layer, a BiGRU context semantic acquisition layer, an MHSA self-attention mechanism layer, and a fully connected layer; The initial dataset is input into the deep learning model for training, and Focal Loss is used to optimize the model weights to obtain a trained deep learning model. The data to be detected is input into the trained model, and the output obtains the fault categories of the real high-speed rail signal system.

[0023] Furthermore, the fault types are divided into nine categories, including: track circuits, transponders, train control center, radio block center, interlocking, centralized dispatching system, onboard problems, temporary speed limit server, and others.

[0024] Furthermore, generating the token sequence includes: Collect descriptions of fault symptoms and fault type labels, and generate a CSV file; BertTokenizer is used to convert the segmented text into a sequence of token IDs, generating input with dimensions [B, L].

[0025] Furthermore, the BERT word embedding layer is used to encode the token sequence to generate a high-dimensional semantic feature vector, including: BertTokenizer is used to convert the segmented fault descriptions into a sequence of token IDs. Using the BERT-base-chinese model, the token ID sequence is input into the 12-layer Transformer encoder of the BERT-base-chinese model to generate a 768-dimensional feature vector containing contextual semantics. and CLS features The expression is: ; in, .

[0026] Furthermore, the BiGRU context semantic acquisition layer employs a bidirectional GRU network to model the temporal dependency of fault descriptions, expressed as: ; in, and These are the hidden states of the forward and backward GRU, respectively. It is a bidirectional hidden state sequence, which makes up for BERT's weak modeling of sequence order and enhances the extraction of causal relationships.

[0027] Furthermore, the MHSA self-attention mechanism layer is used to highlight key fault characteristics and improve the ability to distinguish between similar categories, including: The MHSA layer embeds a dimension of 768, with 12 headers, each with a dimension of 64, and outputs the BiGRU. Weighted features are generated using average pooling, with the following formula: ; ; ; in, The query, key, and value vectors output by BiGRU , respectively Size corresponds to Projection matrix of direction, h represents the number of attention heads. To output the projection matrix, For average pooling operation, It is a feature vector of fixed length.

[0028] Furthermore, the fully connected layer is used to concatenate the CLS token features of BERT. and MHSA pooling characteristics Generate a 1536-dimensional feature vector using the following formula:

[0029] in Pooling features for multi-head self-attention mechanisms This is the 1536-dimensional feature vector obtained by concatenating these two features; The first fully connected layer maps the 1536-dimensional features to 384 dimensions, using ReLU activation and BatchNorm, as shown in the formula:

[0030] in This is the weight matrix of the first fully connected layer. , For activation functions; The second fully connected layer maps the 384-dimensional features to 9-dimensional logits, as shown in the formula:

[0031] in This is the weight matrix of the second fully connected layer. For bias terms, The output consists of 9 categories of logits, where 9 corresponds to the number of fault type categories; Nine types of fault probabilities are generated using the softmax function, with the following formula:

[0032] in, Let be the probability vector for the fault category. For the first The probability of a type of failure.

[0033] In one specific embodiment, the high-speed rail signaling system fault identification method based on deep learning includes the following steps: Collect fault data of high-speed rail signaling system, including fault phenomenon description and fault type label data, and store the data in CSV file. Use Bert Tokenizer (bert-base-chinese) to convert the segmented text into a token ID sequence. A deep learning model suitable for fault identification in high-speed rail signaling systems is constructed. The model includes a BERT word embedding layer, a BiGRU context semantic acquisition layer, an MHSA self-attention mechanism layer, and a fully connected layer. The BERT word embedding layer uses a pre-trained BERT-based-Chinese model, with Dropout set to 0.1 to prevent overfitting. The input is a sequence of token IDs, and the output is a semantic feature tensor. The BiGRU context semantic acquisition layer receives BERT output and uses a two-layer bidirectional GRU to model temporal dependencies, generating a sequence of hidden states. The MHSA self-attention mechanism layer receives the BiGRU output, focuses on key tokens through multi-head self-attention weighting, and generates fixed-length features using average pooling. The fully connected layer is used to concatenate BERT CLS features and MHSA pooling features to generate a 1536-dimensional input. The two fully connected layers generate classification logits, and finally the softmax function converts the logits into 9 types of fault probabilities. The initial dataset is input into the deep learning model for training, and Focal Loss is used to optimize the model weights to obtain a trained deep learning model. The model was trained using PyTorch 1.12, and the dataset was divided into a training set (80%), a validation set (10%), and a test set (10%). The training parameters are as follows: The batch size (batch_size) is set to 8, Focal Loss (Gamma=3.0) is used as the loss function, AdamW is used as the optimizer to optimize the class imbalance problem, the BERT learning rate is set to 1e-5, the learning rate of other layers is set to 1e-4, and the weight decay is 0.01. The data to be detected is input into the trained model, and the output obtains the fault categories of the real high-speed rail signal system.

[0034] The above description of the disclosed embodiments enables those skilled in the art to make or use the invention. Various modifications to these embodiments will be readily apparent to those skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of the invention. Therefore, the invention is not to be limited to the embodiments shown herein, but is to be accorded the widest scope consistent with the principles and novel features disclosed herein.

Claims

1. A fault identification method for high-speed rail signaling systems based on deep learning, characterized in that, Includes the following steps: Collect fault data from the high-speed rail signaling system to form a token sequence that includes fault phenomenon descriptions and fault type labels, and use it as the initial dataset; A deep learning model suitable for fault identification in high-speed rail signaling systems is constructed. The model includes a BERT word embedding layer, a BiGRU context semantic acquisition layer, an MHSA self-attention mechanism layer, and a fully connected layer. The initial dataset is input into the deep learning model for training, and Focal Loss is used to optimize the model weights to obtain a trained deep learning model. The data to be detected is input into the trained model, and the output obtains the fault categories of the real high-speed rail signal system.

2. The fault identification method for high-speed rail signaling systems based on deep learning according to claim 1, characterized in that, The fault types are divided into nine categories, including: track circuits, transponders, train control center, radio block center, interlocking, centralized dispatching system, onboard problems, temporary speed limit server, and others.

3. The fault identification method for high-speed rail signaling systems based on deep learning according to claim 1, characterized in that, The BERT word embedding layer is used to encode the token sequence and generate a high-dimensional semantic feature vector, including: BertTokenizer is used to convert the segmented fault descriptions into a sequence of token IDs. Using the BERT-base-chinese model, the token ID sequence is input into the 12-layer Transformer encoder of the BERT-base-chinese model to generate a 768-dimensional feature vector containing contextual semantics. and CLS features The expression is: in, This is a sequence of fault descriptions after word segmentation. For batch size, The sequence length is given.

4. The fault identification method for high-speed rail signaling systems based on deep learning according to claim 1, characterized in that, The BiGRU context semantic acquisition layer uses a bidirectional GRU network to model the temporal dependency of fault descriptions, expressed as follows: in, and These are the hidden states of the forward and backward GRU, respectively. It is a bidirectional hidden state sequence, which makes up for BERT's weak modeling of sequence order and enhances the extraction of causal relationships.

5. The fault identification method for high-speed rail signaling systems based on deep learning according to claim 1, characterized in that, The MHSA self-attention mechanism layer is used to highlight key fault characteristics and improve the ability to distinguish between similar categories, including: The MHSA layer is embedded with a dimension of 768, and 12 attention heads are set. Each head has 64 dimensions, and the output of the BiGRU is... Weighted features are generated using average pooling, with the following formula: in, The query, key, and value vectors output by BiGRU , respectively Size corresponds to Projection matrix of direction, h represents the number of attention heads. To output the projection matrix, For average pooling operation, It is a feature vector of fixed length.

6. The fault identification method for high-speed rail signaling system based on deep learning according to claim 1, characterized in that... Fully connected layers are used to concatenate the CLS token features of BERT. and MHSA pooling characteristics Generate a 1536-dimensional feature vector using the following formula: in Pooling features for multi-head self-attention mechanisms This is the 1536-dimensional feature vector obtained by concatenating these two features; The first fully connected layer maps the 1536-dimensional features to 384 dimensions, using ReLU activation and BatchNorm, as shown in the formula: in This is the weight matrix of the first fully connected layer. , For activation functions; The second fully connected layer maps the 384-dimensional features to 9-dimensional logits, as shown in the formula: in This is the weight matrix of the second fully connected layer. For bias terms, The output consists of 9 categories of logits, where 9 corresponds to the number of fault type categories; Nine types of fault probabilities are generated using the softmax function, with the following formula: in, Let be the probability vector for the fault category. For the first The probability of a type of failure.

Images