Track circuit fault label classification method and device based on BERT and multi-module fusion and medium

Abstract

The invention relates to a BERT and multi-module fusion-based track circuit fault label classification method and device and a medium, and the method comprises the steps: obtaining a to-be-classified track circuit fault text, and carrying out the cleaning and standardization processing of the to-be-classified track circuit fault text; performing word segmentation and vectorization processing on the track circuit fault text to generate a lexical symbol sequence conforming to BERT encoder input specifications; extracting a context semantic vector representation of the input lexical symbol sequence through a BERT encoder; performing feature extraction through a TextCNN module and a self-attention module which are parallel to obtain local features and weighted global features, and splicing the local features and the weighted global features to obtain a comprehensive feature vector; and synchronously generating a multi-label classification result of a fault phenomenon, a fault reason and a solution measure corresponding to the current fault text through a multi-label classification head based on the comprehensive feature vector. Compared with the prior art, the method has the advantages of high discrimination, high accuracy, high practicability and the like.

Description

Technical Field

[0001] This invention relates to the fields of natural language processing and fault diagnosis technology, and in particular to a method, device and medium for classifying track circuit fault labels based on BERT and multi-module fusion. Background Technology

[0002] As a key component of the railway signaling system, the track circuit system's operational status directly impacts the safety and efficiency of railway transportation. In actual operation and maintenance, after a fault occurs, the fault phenomenon, cause analysis, and handling measures are typically recorded in text form. Maintenance personnel need to rely on these fault reports for rapid diagnosis and decision-making. Traditional fault diagnosis methods heavily depend on human experience, resulting in low efficiency, poor consistency, and susceptibility to subjective factors, making them unsuitable for the high demands of modern railway systems for fault response speed and accuracy.

[0003] In recent years, with the development of natural language processing technology, text-based automatic fault classification methods have been gradually applied in the industrial field. Existing methods mostly employ a single model structure, such as text classification models based on convolutional neural networks or recurrent neural networks, which can extract semantic information from fault texts and perform classification to a certain extent. However, these methods generally have limitations.

[0004] First, traditional models have shortcomings in feature extraction. While convolutional neural networks can effectively capture local semantic features, their ability to model long-distance dependencies and global contextual information in text is limited. Recurrent neural networks and their variants can handle sequential information, but their training complexity is high, and they are prone to gradient vanishing or exploding problems when processing long texts. A single model cannot adequately consider both local details and global semantic relationships, resulting in insufficient semantic understanding of complex fault descriptions.

[0005] Secondly, most existing methods only support single-label classification, meaning each text corresponds to only one category label. However, in real-world fault scenarios, a fault event often involves multiple phenomena, multiple causes, and multiple handling measures, making it a typical multi-label classification problem. Existing technologies lack the ability to model multi-label co-occurrence situations, failing to meet the needs of recognizing multiple semantic information in practical fault diagnosis.

[0006] Furthermore, traditional methods perform poorly in terms of domain adaptability. The field of rail circuits contains a large number of technical terms and equipment names, which general-purpose text processing tools struggle to accurately identify and segment, leading to the loss or misunderstanding of semantic information and affecting classification accuracy.

[0007] In summary, existing technologies have significant shortcomings in terms of the comprehensiveness of feature extraction, multi-label classification capabilities, and domain adaptability, which limits their practical application in the automatic classification of railway circuit fault texts. Therefore, how to enable fault classification methods to fully integrate local and global text features, support multi-label output, and possess good domain adaptability, in order to improve the accuracy, efficiency, and automation level of fault diagnosis, is a technical problem that needs to be solved. Summary of the Invention

[0008] The purpose of this invention is to overcome the shortcomings of the existing technology by providing a method, device and medium for classifying track circuit fault labels based on BERT and multi-module fusion. By using the TextCNN module and the lightweight self-attention module in parallel to perform collaborative processing and feature fusion on the context semantic vector generated by BERT, the technical limitation of not being able to take into account both local features and global dependencies is overcome, thereby improving the accuracy and robustness of multi-label classification of fault text.

[0009] The objective of this invention can be achieved through the following technical solutions: According to a first aspect of the present invention, a method for classifying track circuit fault tags based on BERT and multi-module fusion is provided, the specific steps of which include: S1. Obtain the track circuit fault text to be classified, and perform cleaning and standardization processing; S2. The processed track circuit fault text is segmented and quantized to generate a word sequence that conforms to the input specification of the BERT encoder, and then input into the pre-trained track circuit fault label classification model. The track circuit fault label classification model includes: BERT encoder, parallel TextCNN module and self-attention module, feature fusion layer and multi-label classification head. S3. Extract the contextual semantic vector representation of the input word sequence through the BERT encoder; perform feature extraction through the parallel TextCNN module and self-attention module to obtain local features and weighted global features respectively, and concatenate the local features and weighted global features to obtain a comprehensive feature vector; S4. Based on the comprehensive feature vector, the multi-label classification results corresponding to the current fault text, including the fault phenomenon, fault cause, and solution measures, are generated synchronously through the multi-label classification head.

[0010] Furthermore, the specific steps for word segmentation and vectorization of the text in S2 include: The multi-label fault text is segmented using a pre-built dictionary specifically for the field of track circuits and a maximum positive matching algorithm. The segmented sequence is input into the WordPiece segmenter of the BERT encoder for fine-grained word segmentation to obtain a word sequence; By querying the word embedding matrix of the BERT encoder, each word in the word sequence is mapped to a corresponding fusion vector, which includes word embedding information and positional encoding information, to form an input sequence representation; All input sequences are standardized to a preset fixed length by padding or truncation operations, and a corresponding attention mask matrix is ​​generated to identify the actual words and padding positions.

[0011] Furthermore, the specific steps for obtaining local features in S3 include: The context semantic vector representation is input into the TextCNN module, and convolution operations are performed through multiple convolution kernels of different scales in the TextCNN module. The outputs of each convolution are then max-pooled and concatenated to obtain local features.

[0012] Furthermore, the specific data processing steps in the TextCNN module include: The context semantic vector representation is input into multiple parallel one-dimensional convolutional layers. Each one-dimensional convolutional layer uses a convolutional kernel of different height. By sliding along the sequence dimension, the inner product of the kernel with the local word vector fragment is calculated. Combined with the activation function, the features of local word order fragments of different lengths in the context semantic vector are scanned and extracted. For each of the local word order fragment features, a max pooling operation is performed along the sequence dimension to extract the most significant semantic feature signals from local word order fragments of different lengths; The most prominent semantic feature signals in local word order segments of different lengths are concatenated to form local feature vectors.

[0013] Furthermore, the specific steps for obtaining global features in S3 include: The context semantic vector representation is input into the lightweight self-attention module. The compatibility score between each position vector in the sequence and the learnable global query vector is calculated and normalized to obtain the attention weight. Then, the attention weight is used to sum all position vectors in a weighted manner to obtain the weighted global feature.

[0014] Furthermore, the specific data processing steps of the lightweight self-attention module include: The predefined learnable global query vector in the lightweight self-attention module is invoked to represent the key information paradigm in the fault classification task. Calculate the compatibility score between the semantic unit vector of each position vector in the context semantic vector representation and the global query vector; The compatibility scores of all positions are normalized to obtain the importance weight of the semantic unit vector of each position vector in the global semantics. The importance weights are used to perform a weighted summation on all semantic unit vectors, and the summation result is used as the weighted global feature vector.

[0015] Furthermore, in step S4, the comprehensive feature vector is simultaneously input into a multi-label classification head that includes three independent fully connected classifiers; the first fully connected classifier outputs the classification probability distribution of the fault phenomenon; the second fully connected classifier outputs the classification probability distribution of the fault cause; and the third fully connected classifier outputs the classification probability distribution of the solution.

[0016] Furthermore, the operation steps of each fully connected classifier include: The composite feature vector is mapped to the hidden layer vector through a linear transformation layer; Apply the ReLU activation function to the hidden layer vector; The vector processed by the ReLU activation function is then passed through an output layer and activated by the Sigmoid function to obtain the predicted probability distribution of all possible labels for the corresponding task.

[0017] According to a second aspect of the present invention, an electronic device is provided, including a memory and a processor, wherein the memory stores a computer program, and the processor executes the program to implement the method described thereon.

[0018] According to a third aspect of the present invention, a computer-readable storage medium is provided having a computer program stored thereon, which, when executed by a processor, implements the method described thereon.

[0019] Compared with the prior art, the present invention has the following beneficial effects: (1) By using parallel multi-module feature fusion, the comprehensiveness and discriminative power of the semantic features of fault text are improved: This invention uses the TextCNN module and the lightweight self-attention module to process the context semantic vector output by the BERT encoder in parallel. TextCNN extracts local phrase features in the text, such as voltage instability, through multi-scale convolution kernels, while the lightweight self-attention module weights the semantics of the whole text through learnable global query vectors, highlights key information, and concatenates local features with weighted global features to form a comprehensive feature vector. At the same time, it captures the detailed local patterns and overall logical connections in the fault description, overcomes the limitations of a single model in feature extraction, and thus enhances the semantic understanding and feature discriminative power of fault classification as a whole.

[0020] (2) The multi-label classification head is used to output synchronously, which realizes accurate structured analysis of complex fault scenarios: The multi-label classification head of the present invention contains three independent fully connected classifiers. In the data processing flow, the comprehensive feature vector is simultaneously input to these three classifiers, which output the probability distribution of fault phenomena, fault causes and solutions synchronously. Based on the same underlying comprehensive feature, the mechanism of multi-task parallel reasoning accurately reflects the complex situation of multiple phenomena, causes and solutions coexisting in actual faults, realizes the structured analysis of fault text, and improves the accuracy and practicality of automated processing of fault diagnosis reports.

[0021] (3) Combining domain-adaptive preprocessing and end-to-end model enhances applicability and reliability: In the early stage of data processing, this invention uses a pre-built domain-specific dictionary for track circuits and the maximum positive matching algorithm for word segmentation, ensuring the correct segmentation and retention of professional terms such as "red light band" and "compensation capacitor". Together with the subsequent BERT vectorization and multi-module fusion model, it forms an end-to-end classification process, which improves the accuracy of the model's semantic understanding of professional texts for track circuits and reduces the semantic deviation caused by the missegmentation of professional words by general word segmentation tools, thereby ensuring high reliability and strong adaptability. Attached Figure Description

[0022] Figure 1 A flowchart of a track circuit fault label classification method based on BERT and multi-module fusion; Figure 2 This is a schematic diagram illustrating the data flow of a track circuit fault label classification method based on BERT and multi-module fusion. Figure 3 This is a structural diagram of the BERT encoder; Figure 4 Here is a structural diagram of the TextCNN module; Figure 5 This is a structural diagram of a lightweight self-attention module; Figure 6 This is a structural diagram of a multi-label classification head. Detailed Implementation

[0023] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of the present invention. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort should fall within the scope of protection of the present invention.

[0024] like Figure 1 As shown, this embodiment provides a track circuit fault label classification method based on BERT and multi-module fusion. The specific steps include: S1. Obtain the track circuit fault text to be classified, and perform cleaning and standardization processing; S2. The processed track circuit fault text is segmented and quantized to generate a word sequence that conforms to the input specification of the BERT encoder, and then input into the pre-trained track circuit fault label classification model. The track circuit fault label classification model includes: BERT encoder, parallel TextCNN module and self-attention module, feature fusion layer and multi-label classification head. S3. Extract the contextual semantic vector representation of the input word sequence through the BERT encoder; perform feature extraction through the parallel TextCNN module and self-attention module to obtain local features and weighted global features respectively, and concatenate the local features and weighted global features to obtain a comprehensive feature vector; S4. Based on the comprehensive feature vector, the multi-label classification results corresponding to the current fault text, including the fault phenomenon, fault cause, and solution measures, are generated synchronously through the multi-label classification head.

[0025] The training steps for the track circuit fault label classification model used in this embodiment specifically include: Historical fault texts of the track circuit system were collected, and after cleaning and standardization, each fault text was labeled with three categories of tags: fault phenomenon, fault cause, and solution, to construct a multi-label training dataset. The text in the multi-label training dataset is segmented and vectorized to generate a word sequence that conforms to the BERT encoder input specification; Construct a track circuit fault label classification model, including: The BERT encoder is used to extract a contextual semantic vector representation of the input word sequence; The parallel TextCNN module and self-attention module are used to extract local features and weighted global features from the context semantic vector representation, respectively; The feature fusion layer is used to concatenate the local features and weighted global features into a comprehensive feature vector. The multi-label classification head includes three independent fully connected classifiers for simultaneously predicting labels for fault phenomena, fault causes, and solutions based on the comprehensive feature vector. The processed word sequence is input into the classification model for forward propagation to obtain the predicted probability distribution of the three types of labels; The total loss function of the calculation model is optimized and updated using the backpropagation algorithm until the model converges, resulting in a well-trained track circuit fault label classification model.

[0026] The data flow diagram of the method in this embodiment is shown below. Figure 2As shown, a multi-label fault text dataset was first constructed. Historical fault texts, totaling 5836 original data entries, were exported from the fault information management system of the subway operating company. The raw data was first cleaned and standardized: irrelevant characters, garbled text, and special symbols were removed using regular expressions; spelling correction and terminology standardization were performed based on a predefined dictionary of track circuit terminology, including terms such as "red light band," "transformer," "compensation capacitor," and "insulation joint." Subsequently, according to a predefined 32-category labeling system, including 15 types of fault phenomena, 7 types of fault causes, and 10 types of solutions, each cleaned text was independently labeled with multiple tags, forming standardized multi-label fault texts. These were then randomly divided in a 7:2:1 ratio to obtain training, validation, and test sets, and stored and managed using JSON format.

[0027] The specific steps for text segmentation and vectorization in S2 include: Using a pre-built dictionary specifically for the field of track circuits, the maximum positive matching algorithm is employed to segment multi-label fault texts. The segmented sequence is input into the WordPiece segmenter of the BERT encoder for fine-grained word segmentation to obtain a word sequence; By querying the word embedding matrix of the BERT encoder, each word in the word sequence is mapped to a corresponding fusion vector. In this embodiment, it is a 768-dimensional fusion vector. The fusion vector includes word embedding information and position encoding information to form an input sequence representation. All input sequences are standardized to a preset fixed length by padding or truncation operations, and a corresponding attention mask matrix is ​​generated to identify the actual words and padding positions.

[0028] In S3, a BERT pre-trained model is used as the encoder. For example... Figure 3 As shown, in the BERT encoder, the sequence of words is used to generate a contextual semantic vector representation through a self-attention mechanism: Its core multi-head self-attention expression is: , Among them, the Size The expression is: , In the formula, It is the output projection matrix. These are query, key, and value matrices, respectively. , and These are the weight matrices for the query, key, and value, respectively. is the dimension of the key vector.

[0029] The specific steps for obtaining local features in S3 include: The context semantic vector represents the input as follows: Figure 4 The TextCNN module shown performs convolution operations using multiple convolution kernels of different scales within the TextCNN module, and then concatenates the outputs of each convolution after max pooling to obtain local features.

[0030] The specific data processing steps in the TextCNN module include: The context semantic vector representation is input into multiple parallel one-dimensional convolutional layers. Each one-dimensional convolutional layer uses a convolutional kernel of different height. By sliding along the sequence dimension, the inner product of the kernel with the local word vector fragment is calculated. Combined with the activation function, the features of local word order fragments of different lengths in the context semantic vector are scanned and extracted. In this embodiment, convolution kernels with heights of 2, 3, and 4 are used to perform convolution operations on the context semantic vector, as expressed in the following expression: , in It is the ReLU activation function. For convolution kernel weights, For bias terms; For each local word order segment feature, perform max pooling along the sequence dimension to extract the most significant semantic feature signals from local word order segments of different lengths; The most prominent semantic feature signals in local word order segments of different lengths are concatenated to form local feature vectors.

[0031] The specific steps for obtaining global features in S3 include: The context semantic vector represents the input as follows: Figure 5 The lightweight self-attention module shown calculates the compatibility score between each position vector in the sequence and the learnable global query vector, normalizes it to obtain the attention weight, and then uses the attention weight to perform a weighted sum of all position vectors to obtain the weighted global features.

[0032] The specific data processing steps of the lightweight self-attention module include: Call the predefined learnable global query vector in the lightweight self-attention module to represent the key information paradigm in the fault classification task; Calculate the compatibility score between the semantic unit vector of each position vector in the context semantic vector representation and the global query vector; In this embodiment, a global query vector can be learned. The semantic unit vector of each position vector With global query vector Compatibility score The expression is: , in, It is a trainable linear transformation matrix; The compatibility scores for all locations are normalized using softmax to obtain the importance weight of the semantic unit vector of each location vector in the global semantics. ; All semantic unit vectors are weighted and summed using importance weights. The sum is then used as a weighted global feature vector, expressed as follows: .

[0033] In S4, the comprehensive feature vector is simultaneously input into a multi-label classification head that includes three independent fully connected classifiers. The structure of the multi-label classification head is as follows: Figure 6 As shown, the first fully connected classifier outputs the probability distribution of the fault phenomenon; the second fully connected classifier outputs the probability distribution of the fault cause; and the third fully connected classifier outputs the probability distribution of the solution.

[0034] The operation steps of each fully connected classifier include: The linear transformation layer maps the combined feature vector to the hidden layer vector, as shown in the expression: , in, This is the weight matrix. For bias vectors, For the hidden layer dimension, This is a comprehensive feature vector.

[0035] Apply the ReLU activation function to the hidden layer vectors; The vector processed by the ReLU activation function is then passed through an output layer and activated by the Sigmoid function to obtain the predicted probability distribution of all possible labels for the corresponding task: , in, , , For the first The total number of labels for a phenomenon, cause, or measure; output. It is A dimensional vector, where each element has a value range of [0, 1], representing the independent probability of the corresponding label existing; This is the output of the ReLU activation function.

[0036] In the training process of the overall model used in this embodiment, a binary cross-entropy loss function is employed for all three classification tasks: fault phenomenon, fault cause, and solution, and the loss is calculated per label. The loss function for each task is as follows: , in, This represents the number of samples in the batch. Indicates the first The sample at the th The first task The true value of each label (0 or 1). The corresponding predicted probability, This represents the loss value for the k-th classification task.

[0037] The model's total loss function is the sum of the losses from the three tasks: , in, Losses in the task of classifying fault phenomena; The loss of tasks categorizing the causes of failures; To address the losses in the categorization of measures and tasks.

[0038] By minimizing the total loss function, the backpropagation algorithm is used to simultaneously optimize all parameters of the three classification tasks and the model front end.

[0039] The parameter update rules for model training are as follows: First, calculate the first moment estimate of the gradient. and second-order moment estimation The expression is: , , in, Current time step t The gradient; and The decay rate hyperparameter is estimated by moment. =0.9, =0.999.

[0040] Then, bias corrections are applied to the first and second moment estimates to compensate for their bias at the initial time step, expressed as: , , Finally, the parameters are updated based on weight decay: , in, Correction for first-order moment estimation, For second-order moment estimation correction, ϵ is the learning rate, ε is the numerical stability constant (usually 1e−8), and λ is the weight decay coefficient, which is set to 0.01 in this embodiment.

[0041] The learning rate scheduling uses a cosine annealing strategy with hot restart, and the variation expression is as follows: , in, and These are the minimum and maximum values ​​of the learning rate, respectively (in this embodiment). =2e−5, =0), This represents the number of iterations performed within the current period. For the first The total number of iterations in each restart cycle; each time a restart occurs... Reset to 0.

[0042] After training, new faulty texts are preprocessed and the model is used for inference, and multi-label classification results are output.

[0043] The model performance of the method proposed in this embodiment was evaluated on the test set, and the evaluation criteria were the macro-average precision, recall, and F1 score for each task. Table 1 shows a performance comparison of the method of this embodiment and other methods on the multi-label classification task of track circuit faults (F1 score, %); Table 2 shows the detailed evaluation results of the method of this embodiment on each task (macro-average, %).

[0044] Specifically, macro average accuracy Recall rate The expression for the F1 value is: , , , in, The number of samples in which the model correctly predicts a positive example, i.e., a certain fault label exists; The number of samples in which the model incorrectly predicts a positive example, i.e., a certain fault label exists; This represents the number of samples where the model incorrectly predicts a negative example, i.e., a fault label does not exist, but the label actually exists.

[0045] Table 1. Performance comparison of different methods on multi-label classification of track circuit faults Table 2. Detailed evaluation results of the method in this embodiment on various tasks. As shown in Table 1, the fusion model proposed in this embodiment achieves a macro-average F1 score of 90.4% across the three sub-tasks, consistently outperforming all comparative models. Compared to the powerful BERT-Base baseline, it demonstrates a performance improvement of 4.2 percentage points; compared to BERT+TextCNN and BERT+Self-Attention models, which also utilize feature fusion, it shows improvements of 2.9 and 2.5 percentage points, respectively. This proves that the multi-module feature fusion strategy can effectively integrate local features and global dependencies, improving the overall model performance to an excellent level of over 90%.

[0046] As shown in Table 2, the F1 score for fault phenomenon classification is the highest at 92.1%, because its description is usually the most direct and obvious in the text. The solution classification achieved an F1 score of 89.8%, comparable to the fault cause classification, demonstrating the model's good synergy. With only a slight increase in the number of parameters, a significant performance improvement was achieved, with the macro-average F1 score exceeding 90.4%, proving that the method in this embodiment achieves the best balance between model efficiency and performance, and has high engineering application value.

[0047] In summary, the experimental results demonstrate that the method described in this embodiment achieves an excellent performance of over 90% in the multi-label classification task of track circuit faults, which has significant advantages over existing methods and can provide reliable technical support for railway fault diagnosis.

[0048] This embodiment creatively constructs a track circuit fault label classification method based on BERT and multi-module fusion. It designs a parallel-connected TextCNN and a lightweight self-attention module as the core of feature extraction. TextCNN is specifically responsible for capturing local key signals and phrase features in the fault text, while the lightweight self-attention effectively focuses on full-text semantic dependencies and long-range contextual associations. The feature vectors output by both are concatenated to form a more comprehensive and discriminative integrated feature representation, solving the technical challenge of a single model failing to simultaneously consider local and global semantic information, becoming the core driving force for a fundamental leap in the overall classification accuracy of the model. Secondly, it adopts an architecture of shared low-level feature encoding and collaborative training of a top-level multi-task classifier, enabling the three highly related tasks of fault phenomenon, cause, and measures to jointly learn and transfer knowledge in a shared semantic space. This not only significantly improves the efficiency of model parameters but also implicitly enhances the model's causal inference ability for the complex task of "fault cause" through mutual regularization between tasks. Finally, through adaptive dictionary preprocessing and a cosine annealing training strategy with hot restart, it ensures the model's accurate understanding of technical terms and rapid and stable convergence of the training process. Experimental results show that the macro-average F1 score of this invention exceeds 90.4% in the multi-label classification task of track circuit faults, and performs particularly well in key and difficult tasks such as fault cause diagnosis. It is significantly better than existing baseline models and provides an efficient and reliable technical solution for intelligent railway operation and maintenance.

[0049] Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the specific working process of the described module can be referred to the corresponding process in the foregoing method embodiments, and will not be repeated here.

[0050] The electronic device of this invention includes a central processing unit (CPU), which can perform various appropriate actions and processes according to computer program instructions stored in read-only memory (ROM) or loaded from a storage unit into random access memory (RAM). The RAM may also store various programs and data required for device operation. The CPU, ROM, and RAM are interconnected via a bus. Input / output (I / O) interfaces are also connected to the bus.

[0051] Multiple components in the device are connected to an I / O interface, including: input units such as a keyboard, mouse, etc.; output units such as various types of displays, speakers, etc.; storage units such as disks, optical disks, etc.; and communication units such as network interface cards, modems, wireless transceivers, etc. The communication unit allows the device to exchange information / data with other devices through computer networks such as the Internet and / or various telecommunications networks. The processing unit performs the various methods and processes described above, such as the method of the present invention. For example, in some embodiments, the method of the present invention may be implemented as a computer software program tangibly contained in a machine-readable medium, such as a storage unit. In some embodiments, part or all of the computer program may be loaded and / or installed on the device via ROM and / or the communication unit. When the computer program is loaded into RAM and executed by the CPU, one or more steps of the method of the present invention described above may be performed. Alternatively, in other embodiments, the CPU may be configured to execute the method of the present invention by any other suitable means (e.g., by means of firmware).

[0052] The functions described above in this document can be performed, at least in part, by one or more hardware logic components. For example, exemplary types of hardware logic components that can be used, without limitation, include: Field Programmable Gate Arrays (FPGAs), Application-Specific Integrated Circuits (ASICs), Application Standard Products (ASSPs), System-on-Chip (SoCs), Complex Programmable Logic Devices (CPLDs), and so on.

[0053] The program code used to implement the methods of the present invention can be written in any combination of one or more programming languages. This program code can be provided to a processor or controller of a general-purpose computer, special-purpose computer, or other programmable data processing device, such that when executed by the processor or controller, the program code causes the functions / operations specified in the flowcharts and / or block diagrams to be implemented. The program code can be executed entirely on the machine, partially on the machine, as a standalone software package partially on the machine and partially on a remote machine, or entirely on a remote machine or server.

[0054] In the context of this invention, a machine-readable medium can be a tangible medium that may contain or store a program for use by or in conjunction with an instruction execution system, apparatus, or device. A machine-readable medium can be a machine-readable signal medium or a machine-readable storage medium. Machine-readable media can include, but are not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, apparatus, or devices, or any suitable combination of the foregoing. More specific examples of machine-readable storage media include electrical connections based on one or more wires, portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fibers, portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination of the foregoing.

[0055] The above description is merely a specific embodiment of the present invention, but the scope of protection of the present invention is not limited thereto. Any person skilled in the art can easily conceive of various equivalent modifications or substitutions within the technical scope disclosed in the present invention, and these modifications or substitutions should all be covered within the scope of protection of the present invention. Therefore, the scope of protection of the present invention should be determined by the scope of the claims.

Claims

1. A track circuit fault label classification method based on BERT and multi-module fusion, characterized in that, The specific steps include: S1. Obtain the track circuit fault text to be classified, and perform cleaning and standardization processing; S2. The processed track circuit fault text is segmented and vectorized to generate a word sequence that conforms to the input specification of the BERT encoder, and then input into the pre-trained track circuit fault label classification model. The track circuit fault label classification model includes: BERT encoder, parallel TextCNN module and self-attention module, feature fusion layer and multi-label classification head. S3. Extract the contextual semantic vector representation of the input word sequence through the BERT encoder; perform feature extraction through the parallel TextCNN module and self-attention module to obtain local features and weighted global features respectively, and concatenate the local features and weighted global features to obtain a comprehensive feature vector; S4. Based on the comprehensive feature vector, the multi-label classification results corresponding to the current fault text, including the fault phenomenon, fault cause, and solution measures, are generated synchronously through the multi-label classification head.

2. The track circuit fault label classification method based on BERT and multi-module fusion according to claim 1, characterized in that, The specific steps for text segmentation and vectorization in S2 include: The multi-label fault text is segmented using a pre-built dictionary specifically for the field of track circuits and a maximum positive matching algorithm. The segmented sequence is input into the WordPiece segmenter of the BERT encoder for fine-grained word segmentation to obtain a word sequence; By querying the word embedding matrix of the BERT encoder, each word in the word sequence is mapped to a corresponding fusion vector, which includes word embedding information and positional encoding information, to form an input sequence representation; All input sequences are standardized to a preset fixed length by padding or truncation operations, and a corresponding attention mask matrix is ​​generated to identify the actual words and padding positions.

3. The track circuit fault label classification method based on BERT and multi-module fusion according to claim 1, characterized in that, The specific steps for obtaining local features in S3 include: The context semantic vector representation is input into the TextCNN module, and convolution operations are performed through multiple convolution kernels of different scales in the TextCNN module. The outputs of each convolution are then max-pooled and concatenated to obtain local features.

4. The track circuit fault label classification method based on BERT and multi-module fusion according to claim 3, characterized in that, The specific data processing steps in the TextCNN module include: The context semantic vector representation is input into multiple parallel one-dimensional convolutional layers. Each one-dimensional convolutional layer uses a convolutional kernel of different height. By sliding along the sequence dimension, the inner product of the kernel with the local word vector fragment is calculated. Combined with the activation function, the features of local word order fragments of different lengths in the context semantic vector are scanned and extracted. For each of the local word order fragment features, a max pooling operation is performed along the sequence dimension to extract the most significant semantic feature signals from local word order fragments of different lengths; The most prominent semantic feature signals in local word order segments of different lengths are concatenated to form local feature vectors.

5. The track circuit fault label classification method based on BERT and multi-module fusion according to claim 1, characterized in that, The specific steps for obtaining global features in S3 include: The context semantic vector representation is input into the lightweight self-attention module. The compatibility score between each position vector in the sequence and the learnable global query vector is calculated and normalized to obtain the attention weight. Then, the attention weight is used to sum all position vectors in a weighted manner to obtain the weighted global feature.

6. The track circuit fault label classification method based on BERT and multi-module fusion according to claim 5, characterized in that, The specific data processing steps of the lightweight self-attention module include: The predefined learnable global query vector in the lightweight self-attention module is invoked to represent the key information paradigm in the fault classification task. Calculate the compatibility score between the semantic unit vector of each position vector in the context semantic vector representation and the global query vector; The compatibility scores of all positions are normalized to obtain the importance weight of the semantic unit vector of each position vector in the global semantics. The importance weights are used to perform a weighted summation on all semantic unit vectors, and the summation result is used as the weighted global feature vector.

7. The track circuit fault label classification method based on BERT and multi-module fusion according to claim 1, characterized in that, In step S4, the comprehensive feature vector is simultaneously input into a multi-label classification head that includes three independent fully connected classifiers; the first fully connected classifier outputs the classification probability distribution of the fault phenomenon; the second fully connected classifier outputs the classification probability distribution of the fault cause; and the third fully connected classifier outputs the classification probability distribution of the solution.

8. The track circuit fault label classification method based on BERT and multi-module fusion according to claim 7, characterized in that, The operation steps of each fully connected classifier include: The composite feature vector is mapped to the hidden layer vector through a linear transformation layer; Apply the ReLU activation function to the hidden layer vector; The vector processed by the ReLU activation function is then passed through an output layer and activated by the Sigmoid function to obtain the predicted probability distribution of all possible labels for the corresponding task.

9. An electronic device comprising a memory and a processor, wherein the memory stores a computer program, characterized in that, When the processor executes the program, it implements the method as described in any one of claims 1 to 8.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the program is executed by the processor, it implements the method as described in any one of claims 1 to 8.

Images