A named entity recognition method and a terminal

Abstract

The application discloses a named entity recognition method and a terminal. The method comprises the following steps: performing word segmentation on obtained text to obtain a character sequence; performing feature enhancement on the character sequence based on a feature enhancement algorithm; obtaining a feature vector based on the enhanced multi-feature; performing time sequence coding on the feature vector using a multi-head attention mechanism to output time sequence features; predicting the time sequence features to obtain an optimal annotation sequence; and obtaining a named entity recognition result corresponding to the text according to the optimal annotation sequence. The method can fuse multi-feature, utilize multi-feature to help the model learn more semantic information, effectively obtain attention between character sequences, directly link any two characters in the sequence in the calculation process, shorten distance dependence, effectively utilize feature to express the features and context features of the entire text, and associate the characters with each other, thereby effectively improving the accuracy of named entity recognition.

Description

Technical Field

[0001] This invention relates to the field of natural language processing technology, and in particular to a named entity recognition method and terminal. Background Technology

[0002] Named Entity Recognition (NER) is a fundamental tool in applications such as information extraction, question answering systems, syntactic analysis, and knowledge graph construction, playing a crucial role in the practical application of natural language processing technology. Simply put, it involves identifying relevant entities from a natural language text and labeling their location and type. Compared to classification problems, sequence labeling problems involve the current predicted label being related not only to the current input features but also to previous predicted labels, meaning there is a strong interdependence between the predicted label sequences. In traditional machine learning, LSTM (Long Short-Term Memory) + CRF (Conditional Random Field) is currently the mainstream NER model. Its objective function considers not only the input state feature function but also the label transition feature function. Solving a CRF is a dynamic programming problem, which can be decoded using the Viterbi algorithm. The most mainstream NER models currently consist of an embedding layer, a bidirectional RNN (Recurrent Neural Network) layer, a tanh (hyperbolic tangent function) activation layer, and a final CRF layer. The advantage of this model is that it can utilize rich internal and contextual features during the labeling process for a given location. However, it still has the following shortcomings:

[0003] 1. Sequence calculations need to be performed sequentially, so when faced with long sentences, it is relatively difficult to capture dependencies that are far apart.

[0004] 2. Lack of connection between characters. Summary of the Invention

[0005] The technical problem to be solved by the present invention is to provide a named entity recognition method and terminal that can effectively improve the accuracy of named entity recognition.

[0006] To solve the above-mentioned technical problems, the present invention adopts the following technical solution:

[0007] A named entity recognition method includes the following steps:

[0008] Obtain text and segment the text into words to obtain a text sequence, the text sequence including a character text sequence;

[0009] Based on the character text sequence, feature enhancement algorithm is used to enhance features to obtain enhanced multi-dimensional features, and feature vectors are obtained based on the enhanced multi-dimensional features;

[0010] The feature vector is temporally encoded using a multi-head attention mechanism to output temporal features.

[0011] The temporal features are predicted to obtain the optimal annotation sequence, and the named entity recognition result corresponding to the text is obtained based on the optimal annotation sequence.

[0012] To solve the above-mentioned technical problems, another technical solution adopted by the present invention is as follows:

[0013] A named entity recognition terminal includes a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the computer program, it performs the following steps:

[0014] Obtain text and segment the text into words to obtain a text sequence, the text sequence including a character text sequence;

[0015] Based on the character text sequence, feature enhancement algorithm is used to enhance features to obtain enhanced multi-dimensional features, and feature vectors are obtained based on the enhanced multi-dimensional features;

[0016] The feature vector is temporally encoded using a multi-head attention mechanism to output temporal features.

[0017] The temporal features are predicted to obtain the optimal annotation sequence, and the named entity recognition result corresponding to the text is obtained based on the optimal annotation sequence.

[0018] The beneficial effects of this invention are as follows: Feature enhancement algorithms are used to enhance the features of character text sequences, resulting in enhanced multi-dimensional features. Feature vectors are then obtained based on these enhanced multi-dimensional features. A multi-head attention mechanism is used to temporally encode the feature vectors, outputting temporal features. By obtaining feature vectors based on enhanced multi-dimensional features, multi-dimensional features can be fused. Utilizing multi-dimensional features helps the model learn more semantic information. Furthermore, the multi-head attention mechanism effectively captures the attention between character sequences, directly linking any two characters in the sequence during computation, shortening distance dependencies, and effectively utilizing features to fully express the features of the entire text and its context. Moreover, characters can be interconnected, thereby effectively improving the accuracy of named entity recognition. Attached Figure Description

[0019] Figure 1 This is a flowchart illustrating the steps of a named entity recognition method according to an embodiment of the present invention;

[0020] Figure 2 This is a schematic diagram of the structure of a named entity recognition terminal according to an embodiment of the present invention;

[0021] Figure 3 This is a schematic diagram illustrating the steps of acquiring temporal features in the named entity recognition method according to an embodiment of the present invention;

[0022] Figure 4 This is a schematic diagram illustrating feature enhancement and fusion in the named entity recognition method of this invention.

[0023] Figure 5 This is a schematic diagram of the attention calculation network in the named entity recognition method of this invention.

[0024] Figure 6 This is a schematic diagram of the structure for obtaining temporal features in the named entity recognition method of this invention. Detailed Implementation

[0025] To explain in detail the technical content, objectives, and effects of the present invention, the following description is provided in conjunction with the embodiments and accompanying drawings.

[0026] Please refer to Figure 1 This invention provides a named entity recognition method, including the following steps:

[0027] Obtain text and segment the text into words to obtain a text sequence, the text sequence including a character text sequence;

[0028] Based on the character text sequence, feature enhancement algorithm is used to enhance features to obtain enhanced multi-dimensional features, and feature vectors are obtained based on the enhanced multi-dimensional features;

[0029] The feature vector is temporally encoded using a multi-head attention mechanism to output temporal features.

[0030] The temporal features are predicted to obtain the optimal annotation sequence, and the named entity recognition result corresponding to the text is obtained based on the optimal annotation sequence.

[0031] As can be seen from the above description, the beneficial effects of the present invention are as follows: feature enhancement is performed based on character text sequences using a feature enhancement algorithm to obtain enhanced multi-dimensional features, and feature vectors are obtained based on the enhanced multi-dimensional features. The feature vectors are then temporally encoded using a multi-head attention mechanism to output temporal features. By obtaining feature vectors based on enhanced multi-dimensional features, multi-dimensional features can be fused. Utilizing multi-dimensional features helps the model learn more semantic information. Furthermore, the multi-head attention mechanism can effectively acquire attention between character sequences, directly linking any two characters in the sequence during the calculation process, shortening distance dependencies, effectively utilizing features to fully express the features of the entire text and contextual features, and ensuring that characters can be interconnected, thereby effectively improving the accuracy of named entity recognition.

[0032] Furthermore, the step of segmenting the text to obtain a text sequence includes:

[0033] The text is segmented into characters according to character segmentation rules to obtain a character text sequence;

[0034] The text is segmented into words using a word segmentation tool to obtain a word sequence.

[0035] As described above, character segmentation and word segmentation are performed simultaneously on the text to obtain character text sequences and word text sequences. Segmentation is performed with different text lengths so that features can be learned from each other in the future, thereby improving the accuracy of named entity recognition.

[0036] Furthermore, the step of using a word segmentation tool to segment the text into words to obtain a word sequence includes:

[0037] The text is segmented into words using either the jieba or hanlp word segmentation tool to obtain a word sequence.

[0038] As described above, either the jieba or hanlp word segmentation tool can be used to segment words in the text, depending on the actual situation, which improves flexibility and applicability.

[0039] Furthermore, the feature enhancement algorithm used to enhance the features based on the character text sequence to obtain the enhanced multi-dimensional features includes:

[0040] The character sequence is converted to obtain multiple features;

[0041] The multivariate features are enhanced using a feature enhancement algorithm to obtain the enhanced multivariate features.

[0042] As described above, using feature enhancement algorithms to enhance multi-dimensional features makes it easier for the network to learn the enhanced multi-dimensional features and obtain more semantic information.

[0043] Furthermore, the step of obtaining the feature vector based on the enhanced multivariate features includes:

[0044] The enhanced multivariate features are expanded in feature dimension using a linear mapping to obtain the expanded multivariate features.

[0045] Based on the expanded multi-features, feature fusion is performed to obtain the fused multi-features;

[0046] The fused multi-dimensional features are input into a character embedding network, and the output is a character feature vector.

[0047] The word sequence is input into a pre-trained BERT model, and the output is a word feature vector.

[0048] As described above, by using the fused multi-features as network input and outputting character feature vectors, the model can learn more textual semantic information by utilizing character feature vectors and word feature vectors.

[0049] Furthermore, the char embedding network comprises a fully connected neural network, a random deactivation layer, and an activation function connected in sequence;

[0050] The step of inputting the fused multi-dimensional features into the character embedding network and outputting a character feature vector includes:

[0051] The fused multi-dimensional features are input into the fully connected neural network, and the first vector is output.

[0052] The first vector is input into the random deactivation layer, and the second vector is output.

[0053] The second vector is input into the activation function, and the output is the character feature vector.

[0054] As described above, a fully connected neural network is used to linearly transform one feature space into another. The random deactivation layer prevents the neural network from favoring a particular node feature, thus reducing overfitting. Finally, the activation function completes the nonlinear transformation of the data, solving the problem of insufficient expressive and classification capabilities of linear models, thereby improving the accuracy of named entity recognition.

[0055] Furthermore, the step of using a multi-head attention mechanism to perform temporal encoding on the feature vector, and outputting temporal features, includes:

[0056] The character feature vector is input into the first attention calculation network, the word feature vector is input into the second attention calculation network, and the first attention calculation network and the second attention calculation network are updated using a loss function to output the temporal features.

[0057] As described above, the dual-network structure of the first attention computing network and the second attention computing network enables the multi-dimensional features in the two networks to learn from each other, achieve parallel computing and network deepening, thereby constraining and enhancing the model performance. This can effectively improve the named entity recognition performance, especially in handling boundary problems, and also improve the performance of multiple entity types.

[0058] Furthermore, the first attention computation network includes a first multi-head attention coding layer and a first bidirectional LSTM network;

[0059] The second attention computation network includes a second multi-head attention coding layer and a second bidirectional LSTM network;

[0060] The process of inputting the character feature vector into a first attention network, inputting the word feature vector into a second attention network, updating the first and second attention networks using a loss function, and outputting temporal features includes:

[0061] The character feature vector is input into the first multi-head attention encoding layer for encoding to obtain character attention features, and the character attention features are input into the first bidirectional LSTM network to output character temporal features;

[0062] The word feature vector is input into the second multi-head attention encoding layer for encoding to obtain word attention features, and the word attention features are input into the second bidirectional LSTM network to output word temporal features;

[0063] The first attention computing network and the second attention computing network are updated using a loss function, and the temporal features are obtained by outputting the character temporal encoding and the word temporal encoding.

[0064] As described above, using a bidirectional LSTM network instead of the original Position-wise FFN in Transformer can obtain contextual relationships rather than simply positional sequence information, thereby improving the accuracy of named entity recognition.

[0065] Furthermore, the loss function includes a first loss function and a second loss function;

[0066] The step of updating the first attention calculation network and the second attention calculation network using a loss function includes:

[0067] The first attention calculation network is updated using the first loss function, and the second attention calculation network is updated using the second loss function.

[0068] The first loss function for:

[0069]

[0070] In the formula, This represents the loss of the multi-head attention network for the character at time t-1, where β represents the first weight parameter. This represents the loss of the multi-head attention network for the words mentioned at time t-1;

[0071] The second loss function for:

[0072]

[0073] In the formula, α represents the second weighting parameter.

[0074] As described above, during model training, the losses of the first attention computation network and the second attention computation network are minimized simultaneously. These two networks will pass loss information to each other and update at the same time, so that the losses of the two networks are minimized at the same time.

[0075] Please refer to Figure 2 Another embodiment of the present invention provides a named entity recognition terminal, including a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the computer program, it implements each step of the named entity recognition method described above.

[0076] The named entity recognition method and terminal described above are applicable to scenarios involving named entity recognition of natural language. The following detailed embodiments illustrate these methods:

[0077] Example 1

[0078] Please refer to Figure 1 and Figures 3-6 This embodiment of a named entity recognition method includes the following steps:

[0079] S1. Obtain the text and segment it into words to obtain a text sequence, which includes a character text sequence, specifically including:

[0080] S11, Get the text;

[0081] S12. The text is segmented according to character segmentation rules to obtain a character text sequence, such as... Figure 3As shown;

[0082] The character segmentation rules include English character segmentation rules, Chinese character segmentation rules, and symbol character segmentation rules. The English character segmentation rules segment words according to words, the Chinese character segmentation rules segment words according to individual characters, and the symbol character segmentation rules segment words according to characters.

[0083] Specifically, the English text in the text is segmented according to English character segmentation rules, the Chinese text in the text is segmented according to Chinese character segmentation rules, and the symbols in the text are segmented according to symbol character segmentation rules to obtain a character text sequence;

[0084] S13. Use a word segmentation tool to segment the text into words to obtain a word sequence;

[0085] Specifically, the text is segmented into words using either the jieba or hanlp word segmentation tool to obtain a word sequence.

[0086] S2. Based on the character sequence, feature enhancement algorithms are used to perform feature enhancement to obtain enhanced multi-dimensional features, and feature vectors are obtained based on the enhanced multi-dimensional features, such as... Figure 3 As shown, it specifically includes:

[0087] S21. Convert the character sequence to obtain multiple features;

[0088] The multi-dimensional features include pre-trained BERT features, stroke features, pinyin features, character structure features, and Zheng code features, such as... Figure 4 As shown;

[0089] Specifically, the character text sequence is transformed according to the pre-trained BERT model to obtain pre-trained BERT features, namely pre-trained BERT embeddings;

[0090] S22. Use a feature enhancement algorithm to enhance the multi-dimensional features to obtain the enhanced multi-dimensional features;

[0091] S23. The enhanced multi-dimensional features are expanded using linear mapping to obtain the expanded multi-dimensional features;

[0092] Specifically, the enhanced multivariate features are expanded in feature dimension using linear mapping and combined with pre-trained BERT embedding. The mapping dimension is determined to be the same to obtain the expanded multivariate features, thus avoiding attention shift.

[0093] S24. Based on the expanded multi-dimensional features, feature fusion is performed to obtain the fused multi-dimensional features;

[0094] S25. Input the fused multi-dimensional features into the character embedding network, and output the character feature vector, such as... Figure 4 As shown, it specifically includes:

[0095] The char embedding network comprises a fully connected neural network, a random deactivation layer, and an activation function connected in sequence, and the char embedding network is trained and optimized simultaneously with the entire network model;

[0096] S251. Input the fused multi-dimensional features into the fully connected neural network and output the first vector.

[0097] S252. Input the first vector into the random deactivation layer and output the second vector.

[0098] S253. Input the second vector into the activation function and output the character feature vector.

[0099] S26. Input the word sequence into the pre-trained BERT model, and output the word feature vector, such as... Figure 3 As shown;

[0100] In another optional implementation, the word sequence is input into a preset Word2vec model, and word feature vectors are output.

[0101] S3. Use a multi-head attention mechanism to perform temporal encoding on the feature vector and output the temporal features.

[0102] Specifically, the character feature vector is input into a first attention computing network, and the word feature vector is input into a second attention computing network. A loss function is then used to update both the first and second attention computing networks, and the temporal features are output, such as... Figure 5 As shown, it specifically includes:

[0103] The first attention computation network includes a first multi-head attention coding layer and a first bidirectional LSTM network. Figure 5 The second attention computation network includes a second multi-head attention coding layer and a second bidirectional LSTM network.

[0104] S31. Input the character feature vector into the first multi-head attention coding layer for encoding to obtain character attention features, and input the character attention features into the first bidirectional LSTM network to output character temporal features;

[0105] Specifically, the character feature vector is input into the first multi-head attention encoding layer for encoding and then normalized to obtain character attention features. The character attention features are then input into the first bidirectional LSTM network for layer normalization, and then the output of the Softmax layer is used to obtain character temporal features.

[0106] S32. Input the word feature vector into the second multi-head attention encoding layer for encoding to obtain word attention features, and input the word attention features into the second bidirectional LSTM network to output word temporal features;

[0107] Specifically, the word feature vector is input into the second multi-head attention encoding layer for encoding and then normalized to obtain word attention features. The word attention features are then input into the second bidirectional LSTM network for layer normalization, and then the output of the Softmax layer is used to obtain word temporal features.

[0108] The attention calculation formula is as follows:

[0109] Multihead = Concat(head) i ,....head h ,)W o ;

[0110]

[0111]

[0112] In the formula, Multihead represents multi-head attention, W o This represents the weight matrix parameter, used to concatenate multiple matrices. This parameter is also continuously updated during training. (head) i Let head represent the output vector of the i-th attention head. h Let Q represent the output vector of the h-th attention head, K represent the query vector, V represent the key vector, and W represent the value vector. i Q This represents the first parameter obtained during training. This represents the second parameter obtained during training. This represents the third parameter obtained during training. These parameters are typically initialized using Gaussian, Xavier, or random methods and are continuously updated as training progresses. K K represents the dimension of the query vector or the key vector, Attention() represents attention calculation, and K... T This represents the transpose of the key vector;

[0113] S33. Update the first attention calculation network and the second attention calculation network using the loss function, and obtain the temporal features based on the character temporal encoding and the word temporal encoding, such as... Figure 6 As shown, it specifically includes:

[0114] The loss function includes a first loss function and a second loss function, and the loss function adopts the binary cross-entropy loss.

[0115] S331. Update the first attention calculation network using the first loss function, and update the second attention calculation network using the second loss function;

[0116] Wherein, the first loss function for:

[0117]

[0118] In the formula, This represents the loss of the multi-head attention network for the character at time t-1, where β represents the first weight parameter. This represents the loss of the multi-head attention network for the words mentioned at time t-1;

[0119] The second loss function for:

[0120]

[0121] In the formula, α represents the second weighting parameter;

[0122] S332. Output the temporal features based on the character temporal encoding and the word temporal encoding;

[0123] The final output of the time-series features is at the character level.

[0124] Steps S31 to S33 are performed simultaneously to minimize the loss of the dual network structure (first attention calculation network and second attention calculation network) at the same time, and the two networks will pass loss information to each other and be trained and updated at the same time.

[0125] S4. Predict the temporal features to obtain the optimal annotation sequence, and obtain the named entity recognition result corresponding to the text based on the optimal annotation sequence;

[0126] Example 2

[0127] Please refer to Figure 2This embodiment of a named entity recognition terminal includes a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the computer program, it implements the various steps of the named entity recognition method in Embodiment 1.

[0128] In summary, the present invention provides a named entity recognition method and terminal, which acquires text and segments the text to obtain a text sequence, the text sequence including a character text sequence; performs feature enhancement based on the character text sequence using a feature enhancement algorithm to obtain enhanced multi-dimensional features, and obtains feature vectors based on the enhanced multi-dimensional features; uses a multi-head attention mechanism to perform temporal encoding on the feature vectors, outputting temporal features; predicts the temporal features to obtain an optimal annotation sequence, and obtains the named entity recognition result corresponding to the text based on the optimal annotation sequence; specifically, the character feature vectors are input into a first attention computing network, the word feature vectors are input into a second attention computing network, and a loss function is used to update the first attention computing network and the second attention computing network to output temporal features; a bidirectional LSTM network is used to replace the original position-wise Transformer. FFN (Famous Entity Recognition) can capture contextual relationships rather than simply positional sequence information. It can also fuse multiple features, which helps the model learn more semantic information. Furthermore, by using a multi-head attention mechanism, it can effectively capture the attention between character sequences. During the calculation process, it can directly link any two characters in the sequence, shortening distance dependencies. It effectively uses features to fully express the features of the entire text and contextual features, and characters can be correlated with each other, thereby effectively improving the accuracy of named entity recognition.

[0129] The above description is merely an embodiment of the present invention and does not limit the patent scope of the present invention. Any equivalent modifications made based on the content of the present invention specification and drawings, or direct or indirect applications in related technical fields, are similarly included within the patent protection scope of the present invention.

Claims

1. A named entity recognition method, characterized in that, Including the following steps: Obtain text and segment the text into words to obtain a text sequence, the text sequence including a character text sequence; Based on the character sequence, feature enhancement algorithms are used to enhance features, resulting in enhanced multi-dimensional features. The feature vector is obtained based on the enhanced multi-dimensional features; The feature vector is temporally encoded using a multi-head attention mechanism to output temporal features. The process involves inputting character feature vectors into a first attention computing network, inputting word feature vectors into a second attention computing network, updating the first and second attention computing networks using loss functions, and outputting temporal features. The loss functions include a first loss function and a second loss function. The first loss function is used to update the first attention computing network, and the second loss function is used to update the second attention computing network. for: In the formula This represents the loss of the character multi-head attention network at time t-1, where β represents the first weight parameter. The second loss function represents the loss of the word multi-head attention network at time t-1. for: In the formula, α represents the second weighting parameter; The temporal features are predicted to obtain the optimal annotation sequence, and the named entity recognition result corresponding to the text is obtained based on the optimal annotation sequence.

2. The named entity recognition method according to claim 1, characterized in that, The step of segmenting the text to obtain a text sequence includes: The text is segmented into characters according to character segmentation rules to obtain a character text sequence; the text is then segmented into words using a word segmentation tool to obtain a word text sequence.

3. The named entity recognition method according to claim 2, characterized in that, The step of using a word segmentation tool to segment the text into word sequences includes: The text is segmented into words using either the jieba or hanlp word segmentation tool to obtain a word sequence.

4. The named entity recognition method according to claim 2, characterized in that, The character text The sequence is enhanced using a feature enhancement algorithm to obtain enhanced multi-dimensional features, including: transforming the character text sequence to obtain multi-dimensional features; The multivariate features are enhanced using a feature enhancement algorithm to obtain the enhanced multivariate features.

5. The named entity recognition method according to claim 2, characterized in that, The process of obtaining the feature vector based on the enhanced multivariate features includes: The enhanced multivariate features are expanded in feature dimension using a linear mapping to obtain expanded multivariate features; feature fusion is then performed based on the expanded multivariate features to obtain fused multivariate features. The fused multi-dimensional features are input into a character embedding network to output character feature vectors; the word text sequence is input into a pre-trained BERT model to output word feature vectors.

6. The named entity recognition method according to claim 5, characterized in that, The char embedding network comprises a fully connected neural network, a random deactivation layer, and an activation function connected in sequence. The step of inputting the fused multi-dimensional features into the character embedding network and outputting a character feature vector includes: The fused multi-dimensional features are input into the fully connected neural network, and a first vector is output; the first vector is input into the random deactivation layer, and a second vector is output. The second vector is input into the activation function, and the output is the character feature vector.

7. The named entity recognition method according to claim 6, characterized in that, The first attention computation network includes a first multi-head attention coding layer and a first bidirectional LSTM network; The second attention computation network includes a second multi-head attention coding layer and a second bidirectional LSTM network; The character feature vector is input into a first attention computing network, the word feature vector is input into a second attention computing network, and a loss function is used to update the first and second attention computing networks. The output temporal features include: The character feature vector is input into the first multi-head attention encoding layer for encoding to obtain character attention features, and the character attention features are input into the first bidirectional LSTM network to output character temporal features; The word feature vector is input into the second multi-head attention encoding layer for encoding to obtain word attention features, and the word attention features are input into the second bidirectional LSTM network to output word temporal features; The first attention calculation network and the second attention calculation network are updated using a loss function, and the temporal features are output based on the character temporal features and the word temporal features.

8. A named entity recognition terminal, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the computer program, it implements each step of the named entity recognition method according to any one of claims 1 to 7.

Images