Subway equipment fault maintenance information-oriented named entity identification method and system

Abstract

The invention discloses a metro equipment fault maintenance information-oriented named entity identification method and system, and relates to the technical field of named entity identification, and the method comprises the following steps: preprocessing: carrying out sequence labeling and labeling classification on original text data; constructing a subway train fault maintenance text data set; the method comprises the following steps: constructing a RoBERTa-WWM-BiLSTMKAN-CRF model for named entity recognition; the RoBERTa-WWM-BiLSTMKAN-CRF model is trained and evaluated, and the performance of the model is evaluated by testing loss, a prediction result and a real label; and named entity identification of the subway train fault maintenance text is realized through the model. According to the method, the subway train fault maintenance text data set is constructed, the KAN layer is added to be responsible for performing spatial transformation on features and adding noise in the model, the expression ability and the generalization ability of the model are improved, and better effects are achieved in the aspects of recognition accuracy, text utilization rate and recognition effect.

Description

Technical Field

[0001] The present invention relates to the technical field of named entity recognition, and in particular to a named entity recognition method and system for subway equipment fault maintenance information. Background Art

[0002] Named Entity Recognition (NER) is a key step in train fault text mining, also known as concept extraction. It aims to extract entities from large amounts of text to help computers understand the semantics and contextual information of entities in the text. As one of the subtasks of information extraction, it is an important foundation for natural language processing (NLP) tasks.

[0003] Urban rail transit operating companies collect and record fault information of train onboard equipment during daily operations. This information usually includes but is not limited to important contents such as fault phenomenon, fault location, fault handling method and fault cause. At present, these data are mainly recorded manually in text form and kept for record. However, due to the lack of in-depth analysis of these text data, the potential value of fault information has not been fully explored. One of the reasons for this problem is that the existing text analysis methods have great limitations, and the research on text analysis in the field of rail transit is relatively scarce. In order to meet the above challenges, the present invention proposes a named entity recognition method specifically for subway equipment fault maintenance information. This method aims to improve the efficiency and accuracy of data analysis by automatically and structuredly identifying and extracting key information in the fault text, thereby providing more powerful support for rail transit equipment maintenance and management. Summary of the invention

[0004] In view of the above-mentioned problems, the present invention is proposed.

[0005] Therefore, the technical problem solved by the present invention is that the existing subway train operation and maintenance text method has the problems of low utilization rate, limited analysis method, and inability to accurately identify entity labels in this field.

[0006] To solve the above technical problems, the present invention provides the following technical solutions: a named entity recognition method for subway equipment fault maintenance information, including preprocessing, sequence labeling and label classification of original text data; constructing a subway train fault maintenance text dataset; constructing a RoBERTa-WWM-BiLSTM_KAN-CRF model for named entity recognition; training and evaluating the RoBERTa-WWM-BiLSTM_KAN-CRF model, and evaluating the model performance through test loss, prediction results and true labels; and realizing named entity recognition of subway train fault maintenance text through the model.

[0007] As a preferred solution of the method for named entity recognition for subway equipment fault maintenance information described in the present invention, the preprocessing, sequence labeling and label classification of the original text data includes preprocessing, semantic classification labeling of the training corpus, and the steps include:

[0008] Preprocess the subway train fault maintenance text dataset and perform sequence annotation on the training corpus;

[0009] Construct a subway train failure maintenance sample dataset with named entity recognition labels;

[0010] Convert annotated BIO format data into a format suitable for RoBERTa input.

[0011] As a preferred solution of the method for named entity recognition for subway equipment fault maintenance information described in the present invention, wherein: the construction of the RoBERTa-WWM-BiLSTM_KAN-CRF model for named entity recognition includes constructing a RoBERTa-WWM-BiLSTM_KAN-CRF model to identify named entities of subway train fault maintenance text:

[0012] It consists of the RoBERTa-WWM Chinese pre-trained language model, BiLSTM_KAN layer and CRF layer; it can recognize the named entities in the subway train fault maintenance text.

[0013] As a preferred solution of the method for named entity recognition for subway equipment fault maintenance information described in the present invention, wherein: the construction of the RoBERTa-WWM-BiLSTM_KAN-CRF model for named entity recognition includes constructing a model including using RoBERTa-WWM to character encode the input text, extracting contextual semantic features, and outputting a tensor containing the context representation of each input tag.

[0014] Convert the input text T into a word embedding matrix. The input text T contains N tokens, and the word embedding dimension of each token is d. The input is represented as a matrix X:

[0015]

[0016] Among them, e i Represents the word embedding of the i-th token;

[0017] Add positional encoding P to capture sequence information:

[0018]

[0019] The positional encoding and word embedding matrices are added to get the input matrix H 0 :

[0020] H 0 =X+P

[0021] The RoBERTa model is composed of a stack of Transformer layers. Each layer includes a multi-head self-attention mechanism and a feedforward neural network. There are L Transformer layers, and the output of layer l is H l , the self-attention mechanism output of each Transformer layer is expressed as:

[0022]

[0023] Where Q, K, and V represent query, key, and value matrices, respectively, and Q = K = V = H l-1 ;

[0024] Multi-head self-attention concatenates the outputs of the attention heads as:

[0025] MultiHead(Q,K,V)=Concat(head1,…,head h )W O

[0026] Among them, the output of the i-th head is expressed as:

[0027]

[0028] in, represents the learnable projection matrix, W O represents the output projection matrix;

[0029] The output of the feedforward neural network applied to each position is expressed as:

[0030] FFN(x)=max(0,xW1+b1)W2+b2

[0031] Among them, W1, W2, b1, b2 represent learnable parameters;

[0032] Layer normalization and residual connections are used for training as:

[0033] H l =LayerNorm(H l-1 +MultiHead(Q,K,V)

[0034] H l =LayerNorm(H l +FFN(H l ))

[0035] After L layers of Transformer layer calculations, the RoBERTa model outputs the final context representation H L:

[0036]

[0037] Among them, h i Represents the context representation of the i-th token, with dimensions (N, T, d), where N represents the batch size, T represents the sequence length, and d represents the hidden layer dimension;

[0038] The output sequence vector representation is passed to the BiLSTM layer, and the contextual semantic features are obtained through the forward LSTM layer and the backward LSTM layer. The BiLSTM layer captures the bidirectional long-range dependencies in the input sequence by processing the contextual representation;

[0039] The output sequence vector H of RoBERTa is passed to the BiLSTM layer;

[0040] The forward LSTM layer processes the input sequence from left to right and outputs a forward hidden state sequence. It is expressed as:

[0041]

[0042] The backward LSTM layer processes the input sequence from right to left to obtain the backward hidden state sequence It is expressed as:

[0043]

[0044] Concatenate the hidden states of the forward and backward LSTMs and output the output H of the BiLSTM BiLSTM , expressed as:

[0045]

[0046] in, Represents the BiLSTM hidden state at the tth time step, with dimension 2d hidden ;

[0047] The KAN layer further processes the representation by adding spatial transformation and noise;

[0048] The KAN layer performs spatial transformation on the output of BiLSTM. The KAN layer has L sub-layers, and the spatial transformation matrix of each sub-layer is W l , the dimension is d×d, then the output of the lth sub-layer is expressed as:

[0049]

[0050] For each sub-layer, adding noise is expressed as:

[0051] Z l =H l+σN l

[0052] Among them, σ represents the standard deviation of the noise, and the noise matrix obeys the standard normal distribution, that is,

[0053] Applying the activation function f(·) to the noised feature is expressed as:

[0054] A l =f(Z l )

[0055] Aggregation of multi-sublayer outputs: Aggregate the outputs of all sublayers, take the average value, and the final output of the KAN layer is expressed as:

[0056]

[0057] Use a linear layer to map the output of the LSTM to the label space, that is, the output dimension of each time step is consistent with the size of the label set;

[0058] The role of the linear layer is to transform the hidden state h at each time step t Mapped to the label space, the mathematical expression of the linear layer is:

[0059] O t =Wh t +b

[0060] Among them, C represents the size of the label set, h t represents the output of BiLSTM at time step t, with dimension d, W represents the weight matrix of the linear layer, with dimension C×d, b represents the bias vector of the linear layer, with dimension C, O t represents the output score at time step t, with dimension C;

[0061] Applying the linear layer to all time steps in the batch results in an output tensor O of dimension (N, T, C) represented as:

[0062]

[0063] in, represents the output of BiLSTM at the i-th sample and t-th time step in the batch, with dimension d, O i,t represents the output score of the linear layer at the i-th sample and t-th time step in the batch, with dimension C;

[0064] In the CRF layer, the prediction sequence is output through the constraint information between adjacent labels, thereby outputting the category of each character;

[0065] The CRF layer defines a transfer matrix A with a dimension of C×C. ijrepresents the score of transferring from label i to label j;

[0066] For each time step t and each label y t , the label score is the output of the linear layer given;

[0067] Given an input sequence x and a label sequence y, the sequence score of CRF is defined as:

[0068]

[0069] in, Indicates the label y corresponding to time step t t The score, Represents the label y t to y t+1 transfer score;

[0070] The normalization factor Z(x) of CRF is the sum of the scores of the label sequence and is expressed as:

[0071]

[0072] in, represents all possible tag sequences;

[0073] Given an input sequence x and a label sequence y, the CRF layer calculates the conditional probability as:

[0074]

[0075] To find the sequence that maximizes the conditional probability, the Viterbi algorithm is used for decoding:

[0076]

[0077] As a preferred solution of the method for named entity recognition for subway equipment fault maintenance information described in the present invention, wherein: the training and evaluation of the second model, and the evaluation of model performance through testing include training and evaluating the second model, and evaluating the model performance through testing loss, prediction results and true labels, and expressing it as:

[0078] Encode the input data through the pre-trained RoBERTa model;

[0079] Pass the output of RoBERTa to the BiLSTM layer;

[0080] Pass the output of BiLSTM to the KAN layer;

[0081] Use a linear layer to map the output of the LSTM to the label space;

[0082] The output mapped to the label space is passed to the CRF layer, which outputs the negative log-likelihood of the target label sequence based on the dependency between labels.

[0083] As a preferred solution of the method for named entity recognition for subway equipment fault maintenance information described in the present invention, wherein: the RoBERTa-WWM-BiLSTM_KAN-CRF model is trained and evaluated, and the model performance is evaluated by testing loss, prediction results and true labels, including encoding the input data through a pre-trained RoBERTa model;

[0084] Pass the output of RoBERTa to the BiLSTM layer;

[0085] Pass the output of BiLSTM to the KAN layer;

[0086] Use a linear layer to map the output of the LSTM to the label space;

[0087] The output mapped to the label space is passed to the CRF layer, which outputs the negative log-likelihood of the target label sequence based on the dependency between labels.

[0088] Output the negative log-likelihood of the target label sequence, including:

[0089] Given an input sequence x and a target label sequence y, the negative log-likelihood loss of the CRF layer is defined as:

[0090]

[0091] in, represents the negative log-likelihood loss (Loss), x represents the input sequence, y represents the target label sequence, corresponding to the true label of the input sequence x, p(y|x) represents the conditional probability of generating the label sequence y, and s(x,y) represents the score of the input sequence x and the label sequence y.

[0092] As a preferred solution of the named entity recognition method for subway equipment fault maintenance information described in the present invention, wherein: the named entity recognition of subway train fault maintenance text realized by the model includes utilizing the model to realize named entity recognition of subway train fault maintenance text, performing named entity recognition on the fault information text based on the subway train fault maintenance sample data set, and identifying the faulty device, fault cause, and processing operation entity.

[0093] Another object of the present invention is to provide a named entity recognition system for subway equipment fault maintenance information, which can construct a subway train fault maintenance text dataset through sequence labeling, solving the problem that the current subway train operation and maintenance text method contains limitations of the analysis method.

[0094] As a preferred solution of the named entity recognition system for subway equipment fault maintenance information described in the present invention, it includes: a data preprocessing module, a model building and training module, and a named entity recognition module; the data preprocessing module is used to perform sequence annotation and label classification on the original text data to construct a text data set for subway train fault maintenance; the model building and training module is used to construct a RoBERTa-WWM-BiLSTM_KAN-CRF model; the named entity recognition module is used to perform named entity recognition on subway train fault maintenance text.

[0095] A computer device comprises a memory and a processor, wherein the memory stores a computer program, and the processor executes the computer program to implement a method for named entity recognition for subway equipment fault maintenance information.

[0096] A computer-readable storage medium stores a computer program, which, when executed by a processor, implements the steps of a named entity recognition method for subway equipment fault maintenance information.

[0097] Beneficial effects of the present invention: The method for named entity recognition of subway equipment fault maintenance information provided by the present invention constructs a subway train fault maintenance text dataset through sequence labeling. In order to improve the named entity recognition effect, based on the above-mentioned optimal model BiLSTM-CRF, a KAN layer is added in the model to be responsible for spatial transformation of features and adding noise, aiming to improve the expression and generalization capabilities of the model. The named entity recognition of subway train fault maintenance text is realized, and the present invention achieves better results in recognition accuracy, text utilization and recognition effect. BRIEF DESCRIPTION OF THE DRAWINGS

[0098] In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the accompanying drawings required for use in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other accompanying drawings can be obtained based on these accompanying drawings without paying creative work.

[0099] Figure 1 An overall flow chart of a named entity recognition method for subway equipment fault maintenance information provided by the first embodiment of the present invention.

[0100] Figure 2 A structural diagram of a named entity recognition method for subway equipment fault maintenance information provided by the first embodiment of the present invention.

[0101] Figure 3A method flow chart of a method for named entity recognition for subway equipment fault maintenance information provided by the first embodiment of the present invention.

[0102] Figure 4 An overall flow chart of a named entity recognition system for subway equipment fault maintenance information provided in the third embodiment of the present invention. DETAILED DESCRIPTION

[0103] In order to make the above-mentioned purposes, features and advantages of the present invention more obvious and easy to understand, the specific implementation methods of the present invention are described in detail below in conjunction with the drawings of the specification. Obviously, the described embodiments are part of the embodiments of the present invention, but not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary persons in the art without creative work should fall within the scope of protection of the present invention.

[0104] Example 1, reference Figure 1-Figure 3 , as an embodiment of the present invention, provides a method for named entity recognition for subway equipment fault maintenance information, comprising:

[0105] S1: Preprocessing: sequence labeling and label classification of raw text data; constructing a subway train fault maintenance text dataset.

[0106] Furthermore, the input text T is converted into a word embedding matrix. The input text T contains N tokens, and the word embedding dimension of each token is d. The input is represented as a matrix X:

[0107] Among them, e i is the word embedding of the i-th token.

[0108] Add positional encoding P to capture sequence information:

[0109] The positional encoding and word embedding matrices are added to get the input matrix H 0 :H 0 =X+P

[0110] The RoBERTa model is composed of multiple Transformer layers. Each layer includes a multi-head self-attention mechanism and a feedforward neural network. There are L Transformer layers, and the output of layer l is H l The self-attention mechanism of each Transformer layer is calculated as follows:

[0111]

[0112] Among them, Q, K, V are query, key and value matrices respectively, usually Q = K = V = H l-1.

[0113] Multi-head self-attention concatenates the outputs of multiple attention heads:

[0114] MultiHead(Q,K,V)=Concat(head1,…,head h )W O

[0115] Among them, the calculation of the i-th head is: is the learnable projection matrix, W O is the output projection matrix.

[0116] A feed-forward neural network is applied to the output at each position:

[0117] FFN(x)=max(0,xW1+b1)W2+b2, where W1, W2, b1, b2 are learnable parameters.

[0118] Layer normalization and residual connections are used to stabilize training:

[0119] H l =LayerNorm(H l-1 +MultiHead(Q,K,V)

[0120] H l =LayerNorm(H l +FFN(H l ))

[0121] After calculations by L layers of Transformer layers, the RoBERTa model outputs the final context representation H L :

[0122] Among them, h i It is the contextual representation of the i-th token, and its dimension is (N, T, d), where: N is the batch size, T is the sequence length, and d is the hidden layer dimension (i.e., the output dimension of RoBERTa).

[0123] S2: Build the RoBERTa-WWM-BiLSTM_KAN-CRF model for named entity recognition.

[0124] Furthermore, the output sequence vector H of RoBERTa is passed to the BiLSTM layer.

[0125] The forward LSTM layer processes the input sequence from left to right to obtain a forward hidden state sequence

[0126]

[0127] The backward LSTM layer processes the input sequence from right to left to obtain the backward hidden state sequence

[0128]

[0129] Concatenate the hidden states of the forward and backward LSTMs to get the output H of the BiLSTM BiLSTM :

[0130] in, is the BiLSTM hidden state at the tth time step, with dimension 2d hidden

[0131] S3: Train and evaluate the RoBERTa-WWM-BiLSTM_KAN-CRF model, and evaluate the model performance through test loss, prediction results and true labels.

[0132] Furthermore, the KAN layer performs spatial transformation on the output of BiLSTM. The KAN layer has L sub-layers, and the spatial transformation matrix of each sub-layer is W l (dimension is d×d), then the output of the lth sub-layer is:

[0133]

[0134] For each sub-layer, adding noise can be expressed as: Z l =H l +σN l , where: σ is the standard deviation of the noise; N l is a noise matrix that follows a standard normal distribution (similar to H l dimensions are the same), i.e.

[0135] Apply activation function f(·) to the noised features: l =f(Z l )

[0136] Aggregation of multi-sublayer outputs. Aggregate the outputs of all sublayers and usually take the average value to get the final output of the KAN layer:

[0137] S4: The model is used to realize named entity recognition of subway train failure maintenance text.

[0138] Furthermore, the role of the linear layer is to transform the hidden state h at each time step t Mapped to the label space. The mathematical expression of the linear layer is: t =Wh t +b

[0139] Among them: the size of the label set is C; h t is the output of BiLSTM at time step t, with dimension d; W is the weight matrix of the linear layer, with dimension C×d; b is the bias vector of the linear layer, with dimension C; O t is the output score at time step t, with dimension C.

[0140] Applying the Linear layer to all time steps in the batch results in an output tensor O of dimension (N, T, C):

[0141] in: is the output of BiLSTM at the ith sample and tth time step in the batch, with dimension d; O i,t is the output score of the linear layer at the i-th sample and t-th time step in the batch, with dimension C.

[0142] It should be noted that the CRF layer defines a transfer matrix A with dimensions C×C, where A ij represents the score of transferring from label i to label j;

[0143] For each time step t and each label y t , the label score is the output of the linear layer given;

[0144] Given an input sequence x and a label sequence y, the sequence score of CRF is defined as:

[0145]

[0146] in: is the label y corresponding to time step t t score; is the label y t to y t+1 transfer score.

[0147] The normalization factor Z(x) of CRF is the sum of the scores of all possible label sequences:

[0148] in represents all possible tag sequences.

[0149] Given an input sequence x and a label sequence y, the CRF layer computes the conditional probability:

[0150]

[0151] To find the most likely tag sequence (i.e. the sequence that maximizes the conditional probability), the Viterbi algorithm is used for decoding:

[0152] Example 2, an embodiment of the present invention, provides a named entity recognition method for subway equipment fault maintenance information. In order to verify the beneficial effects of the present invention, scientific demonstration is carried out through economic benefit calculation and simulation experiments.

[0153] First, the data set comes from a subway company's many years of fault maintenance records. After preprocessing, the text data contains key information such as equipment fault name, fault cause and treatment method. In the preprocessing stage, BIO format tags are used to perform sequence annotation and semantic classification on the text to ensure that the data format is suitable for the input of the RoBERTa model.

[0154] The experiment is divided into several main steps:

[0155] Preprocessing stage: The text data is converted into a format suitable for model input, including adding positional encoding, and sequence annotation and classification of the training corpus. The key to this step is to normalize the input data through the BIO format to ensure that the RoBERTa model can fully capture the contextual semantic features.

[0156] Model training: A pre-trained model based on RoBERTa-WWM was constructed, combined with bidirectional LSTM to capture contextual bidirectional dependencies. The KAN layer is responsible for spatial transformation and adding noise to enhance the generalization ability of the model. Finally, the CRF layer further improves the recognition accuracy through the correlation between labels. After the model is fully trained, its performance is rigorously evaluated through the test set. The evaluation indicators include prediction accuracy, entity recall, entity precision, and F1 score.

[0157] Model testing: The trained model was run on different types of fault texts, and the preprocessing time, model training time, and final named entity recognition performance (including indicators such as accuracy, recall, and precision) of each type of text were recorded.

[0158] Table 1 Experimental data table

[0159]

[0160] It can be seen from the experimental data that the named entity recognition method of the present invention shows excellent performance on different types of subway equipment failure texts. First, the prediction accuracy of the model is maintained above 91%, with the highest reaching 92.7%, proving that the combination of RoBERTa-WWM, BiLSTM, KAN and CRF multi-layer models has extremely strong accuracy and robustness in processing complex subway equipment failure texts.

[0161] By comparing with the existing technology, traditional named entity recognition methods (such as LSTM or simple CRF-based models) often suffer from poor recognition accuracy when facing complex industrial texts due to insufficient context understanding or inability to capture bidirectional dependencies. This paper introduces the RoBERTa pre-training model to comprehensively capture context information, and combines the BiLSTM and KAN layers to not only enhance the model's global semantic understanding capabilities, but also effectively handle long-range dependencies in the text. In addition, the CRF layer further improves the output consistency of the model through dependency constraints between labels, so that in practical applications, the model can stably identify key entities such as fault causes, equipment names, and treatment plans.

[0162] The experimental data also show that the model of the present invention has achieved a high level of recall and precision of over 90% in entity recognition. When processing subway equipment fault texts, the traditional method often has insufficient recall and precision, resulting in the omission of a large amount of key information. The present invention processes the noise through the KAN layer to ensure the robustness of the model in complex texts. This is reflected in the recall rate and F1 score, especially the recall rate of Fault Type 6 is 90.3% and the F1 score is 90.9%, indicating that the model can still maintain a high performance in fault types that are more difficult to identify.

[0163] In summary, through the analysis of the table data, it can be clearly seen that the present invention not only has a higher recognition accuracy rate when processing subway equipment failure maintenance text, but also maintains a stable recall rate and precision, which fully demonstrates its innovation and application advantages in this field. Compared with the prior art, the multi-level model structure of the present invention significantly improves the practicality and effectiveness of the model in actual application scenarios.

[0164] Example 3, reference Figure 4 , which is an embodiment of the present invention, provides a named entity recognition system for subway equipment fault maintenance information, including a data preprocessing module, a model building and training module, and a named entity recognition module.

[0165] The data preprocessing module is used to perform sequence annotation and label classification on the original text data to construct a text dataset of subway train fault maintenance. The model construction and training module is used to build the RoBERTa-WWM-BiLSTM_KAN-CRF model. The named entity recognition module is used to perform named entity recognition on subway train fault maintenance text.

[0166] If the function is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, or the part that contributes to the prior art, or the part of the technical solution, can be embodied in the form of a software product. The computer software product is stored in a storage medium, including several instructions for a computer device (which can be a personal computer, server, or network device, etc.) to perform all or part of the steps of the methods of each embodiment of the present invention. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), disk or optical disk, etc. Various media that can store program codes.

[0167] The logic and / or steps represented in the flowchart or otherwise described herein, for example, can be considered as an ordered list of executable instructions for implementing logical functions, and can be embodied in any computer-readable medium for use by an instruction execution system, device or apparatus (such as a computer-based system, a system including a processor, or other system that can fetch instructions from an instruction execution system, device or apparatus and execute instructions), or in conjunction with such instruction execution systems, devices or apparatuses. For the purposes of this specification, "computer-readable medium" can be any device that can contain, store, communicate, propagate or transmit a program for use by an instruction execution system, device or apparatus, or in conjunction with such instruction execution systems, devices or apparatuses.

[0168] More specific examples of computer-readable media (a non-exhaustive list) include the following: an electrical connection with one or more wires (electronic device), a portable computer disk case (magnetic device), a random access memory (RAM), a read-only memory (ROM), an erasable and programmable read-only memory (EPROM or flash memory), an optical fiber device, and a portable compact disk read-only memory (CDROM). In addition, the computer-readable medium may even be a paper or other suitable medium on which the program is printed, since the program may be obtained electronically, for example, by optically scanning the paper or other medium, followed by editing, deciphering, or processing in another suitable manner as necessary, and then stored in a computer memory.

[0169] It should be understood that the various parts of the present invention can be implemented by hardware, software, firmware or a combination thereof. In the above-mentioned embodiments, multiple steps or methods can be implemented by software or firmware stored in a memory and executed by a suitable instruction execution system. For example, if implemented by hardware, as in another embodiment, it can be implemented by any one of the following technologies known in the art or their combination: a discrete logic circuit having a logic gate circuit for implementing a logic function for a data signal, a dedicated integrated circuit having a suitable combination of logic gate circuits, a programmable gate array (PGA), a field programmable gate array (FPGA), etc. It should be noted that the above embodiments are only used to illustrate the technical solution of the present invention and are not limited. Although the present invention is described in detail with reference to the preferred embodiments, it should be understood by those skilled in the art that the technical solution of the present invention can be modified or replaced by equivalents without departing from the spirit and scope of the technical solution of the present invention, which should be included in the scope of the claims of the present invention.

[0170] It should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention rather than to limit it. Although the present invention has been described in detail with reference to the preferred embodiments, those skilled in the art should understand that the technical solutions of the present invention may be modified or replaced by equivalents without departing from the spirit and scope of the technical solutions of the present invention, which should all be included in the scope of the claims of the present invention.

Claims

1. A named entity recognition method for subway equipment fault maintenance information, characterized in that: include: Preprocessing: sequence labeling and label classification of raw text data; constructing a subway train fault maintenance text dataset; Build the RoBERTa-WWM-BiLSTM_KAN-CRF model for named entity recognition; Train and evaluate the RoBERTa-WWM-BiLSTM_KAN-CRF model, and evaluate the model performance through test loss, prediction results, and true labels; The model is used to realize named entity recognition of subway train fault maintenance text.

2. The method for named entity recognition for subway equipment failure maintenance information according to claim 1, characterized in that: The preprocessing, sequence labeling and labeling classification of the original text data includes preprocessing, semantic classification labeling of the training corpus, and the steps include: Preprocess the subway train fault maintenance text dataset and perform sequence annotation on the training corpus; Construct a subway train failure maintenance sample dataset with named entity recognition labels; Convert annotated BIO format data into a format suitable for RoBERTa input.

3. The method for named entity recognition for subway equipment failure maintenance information according to claim 2, characterized in that: The construction of the RoBERTa-WWM-BiLSTM_KAN-CRF model for named entity recognition includes constructing a RoBERTa-WWM-BiLSTM_KAN-CRF model to identify named entities in subway train fault maintenance text: It consists of the RoBERTa-WWM Chinese pre-trained language model, BiLSTM_KAN layer and CRF layer; it can recognize the named entities in the subway train fault maintenance text.

4. The method for named entity recognition for subway equipment failure maintenance information according to claim 3, characterized in that: The construction of the RoBERTa-WWM-BiLSTM_KAN-CRF model for named entity recognition includes building a model that uses RoBERTa-WWM to character encode the input text, extract contextual semantic features, and outputs a tensor containing the contextual representation of each input token. Convert the input text T into a word embedding matrix. The input text T contains N tokens, and the word embedding dimension of each token is d. The input is represented as a matrix X: Among them, e i Represents the word embedding of the i-th token; Add positional encoding P to capture sequence information: The positional encoding and word embedding matrix are added to get the input matrix H 0 : H 0 =X+P The RoBERTa model is composed of a stack of Transformer layers. Each layer includes a multi-head self-attention mechanism and a feedforward neural network. There are L Transformer layers, and the output of layer l is H l , the self-attention mechanism output of each Transformer layer is expressed as: Where Q, K, and V represent query, key, and value matrices, respectively, and Q = K = V = H l-1 ; Multi-head self-attention concatenates the outputs of the attention heads as: MultiHead(Q,K,V)=Concat(head1,…,head h )W O Among them, the output of the i-th head is expressed as: in, represents the learnable projection matrix, W O represents the output projection matrix; The output of the feedforward neural network applied to each position is expressed as: FFN(x)=max(0,xW1+b1)W2+b2 Among them, W1, W2, b1, b2 represent learnable parameters; Layer normalization and residual connections are used for training as: H l =LayerNorm(H l-1 +MultiHead(Q,K,V)) H l =LayerNorm(H l +FFN(H l )) After L layers of Transformer layer calculations, the RoBERTa model outputs the final context representation H L : Among them, h i Represents the context representation of the i-th token, with dimensions (N, T, d), where N represents the batch size, T represents the sequence length, and d represents the hidden layer dimension; The output sequence vector representation is passed to the BiLSTM layer, and the contextual semantic features are obtained through the forward LSTM layer and the backward LSTM layer. The BiLSTM layer captures the bidirectional long-range dependencies in the input sequence by processing the contextual representation; The output sequence vector H of RoBERTa is passed to the BiLSTM layer; The forward LSTM layer processes the input sequence from left to right and outputs a forward hidden state sequence. It is expressed as: The backward LSTM layer processes the input sequence from right to left to obtain the backward hidden state sequence It is expressed as: Concatenate the hidden states of the forward and backward LSTMs and output the output H of the BiLSTM BiLSTM , expressed as: in, Represents the BiLSTM hidden state at the tth time step, with dimension 2d hidden ; The KAN layer further processes the representation by adding spatial transformation and noise; The KAN layer performs spatial transformation on the output of BiLSTM. The KAN layer has L sub-layers, and the spatial transformation matrix of each sub-layer is W l , the dimension is d×d, then the output of the lth sub-layer is expressed as: For each sub-layer, adding noise is expressed as: Z l =H l +σN l Among them, σ represents the standard deviation of the noise, and the noise matrix obeys the standard normal distribution, that is, Applying the activation function f(·) to the noised feature is expressed as: A l =f(Z l ) Aggregation of multi-sublayer outputs: Aggregate the outputs of all sublayers, take the average value, and the final output of the KAN layer is expressed as: Use a linear layer to map the output of the LSTM to the label space, that is, the output dimension of each time step is consistent with the size of the label set; The role of the linear layer is to transform the hidden state h at each time step t Mapped to the label space, the mathematical expression of the linear layer is: O t =Wh t +b Among them, C represents the size of the label set, h t represents the output of BiLSTM at time step t, with dimension d, W represents the weight matrix of the linear layer, with dimension C×d, b represents the bias vector of the linear layer, with dimension C, O t represents the output score at time step t, with dimension C; Applying the linear layer to all time steps in the batch results in an output tensor O of dimension (N, T, C) represented as: in, represents the output of BiLSTM at the ith sample and tth time step in the batch, with dimension d, O i,t represents the output score of the linear layer at the i-th sample and t-th time step in the batch, with dimension C; In the CRF layer, the prediction sequence is output through the constraint information between adjacent labels, thereby outputting the category of each character; The CRF layer defines a transfer matrix A with a dimension of C×C. ij represents the score of transferring from label i to label j; For each time step t and each label y t , the label score is the output of the linear layer given; Given an input sequence x and a label sequence y, the sequence score of CRF is defined as: in, Indicates the label y corresponding to time step t t The score, Represents the label y t to y t+1 transfer score; The normalization factor Z(x) of CRF is the sum of the scores of the label sequence and is expressed as: in, represents all possible tag sequences; Given an input sequence x and a label sequence y, the CRF layer calculates the conditional probability as: To find the sequence that maximizes the conditional probability, the Viterbi algorithm is used for decoding:

5. The method for named entity recognition for subway equipment failure maintenance information according to claim 4, characterized in that: The training and evaluation of the second model and evaluating the model performance by testing include training and evaluating the second model, and evaluating the model performance by testing loss, prediction results and true labels as follows: Encode the input data through the pre-trained RoBERTa model; Pass the output of RoBERTa to the BiLSTM layer; Pass the output of BiLSTM to the KAN layer; Use a linear layer to map the output of the LSTM to the label space; The output mapped to the label space is passed to the CRF layer, which outputs the negative log-likelihood of the target label sequence based on the dependency between labels.

6. The method for named entity recognition for subway equipment failure maintenance information according to claim 5, characterized in that: The RoBERTa-WWM-BiLSTM_KAN-CRF model is trained and evaluated, and the model performance is evaluated by testing the loss, prediction results and true labels, including encoding the input data through the pre-trained RoBERTa model; Pass the output of RoBERTa to the BiLSTM layer; Pass the output of BiLSTM to the KAN layer; Use a linear layer to map the output of the LSTM to the label space; The output mapped to the label space is passed to the CRF layer, which outputs the negative log-likelihood of the target label sequence based on the dependency between labels. Output the negative log-likelihood of the target label sequence, including: Given an input sequence x and a target label sequence y, the negative log-likelihood loss of the CRF layer is defined as: in, represents the negative log-likelihood loss (Loss), x represents the input sequence, y represents the target label sequence, corresponding to the true label of the input sequence x, p(y|x) represents the conditional probability of generating the label sequence y, and s(x,y) represents the score of the input sequence x and the label sequence y.

7. The method for named entity recognition for subway equipment failure maintenance information according to claim 6, characterized in that: The RoBERTa-WWM-BiLSTM_KAN-CRF model is trained and evaluated, and the model performance is evaluated by testing loss, prediction results and true labels, including using the model to realize named entity recognition of subway train fault maintenance text, performing named entity recognition on fault information text based on a subway train fault maintenance sample data set, and identifying faulty components, fault causes, and processing operation entities.

8. A system using the named entity recognition method for subway equipment fault maintenance information according to any one of claims 1 to 7, characterized in that: Including data preprocessing module, model building and training module, named entity recognition module; The data preprocessing module is used to perform sequence annotation and label classification on the original text data to construct a text data set for subway train fault maintenance; The model building and training module is used to build the RoBERTa-WWM-BiLSTM_KAN-CRF model; The named entity recognition module is used to perform named entity recognition on subway train fault maintenance text.

9. A computer device comprising a memory and a processor, wherein the memory stores a computer program, wherein: When the processor executes the computer program, the steps of the method for named entity recognition for subway equipment fault maintenance information described in any one of claims 1 to 7 are implemented.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the method for named entity recognition for subway equipment fault maintenance information described in any one of claims 1 to 7 are implemented.

Images