Named entity identification method, device and equipment, and storage medium thereof

Abstract

The invention discloses a named entity recognition method, device and equipment and a storage medium thereof, and the method comprises the steps: receiving a to-be-recognized text, carrying out the feature coding processing, generating a dynamic feature vector of a lexical element through the combination of a dynamic word vector model and context information, and obtaining an input sequence; performing feature extraction on the input sequence by using the pre-training model, and extracting context information in the input sequence by introducing dynamic position information and an optimized multi-head self-attention mechanism; and on the basis of the extracted context information, predicting the feature representation corresponding to the input sequence, using a conditional random field to adjust the weight according to the complexity of the named entity label through an adaptive weight adjustment mechanism, and combining a named entity recognition rule in the field of power equipment operation and maintenance to adapt to the named entity relationship and label and classify the named entities. The named entity recognition accuracy is improved; by combining the Bi-RNN and CRF structures, the over-fitting phenomenon of the model is reduced, and stable performance can be kept on different data sets.

Description

Technical Field

[0001] The present invention belongs to the technical field of power equipment operation and maintenance, and more specifically, to a named entity recognition method, device, equipment, and storage medium thereof. Background Art

[0002] Nowadays, although named entity recognition (NER) of English text has been widely studied and has made significant progress, Chinese NER still faces many challenges, especially the problem of Chinese word segmentation, where it is often difficult to define the composition of a word. Traditional Chinese NER tasks usually use recurrent neural networks (RNNs) to improve the performance of models on entity classification tasks. However, RNNs have gradient vanishing and gradient exploding problems when dealing with long-distance dependencies. Although the long short-term memory model (LSTM) performs better in longer sequences and can significantly reduce overfitting through simple tuning techniques, its effect is still limited.

[0003] The neural network model of BiLSTM combined with Conditional Random Field (CRF) performs better than simple CRF or BiLSTM in NER and part-of-speech tagging tasks. This bidirectional structure can obtain contextual sequence information, so it has been widely used in named entity recognition tasks. However, since the entity boundaries of Chinese named entity recognition are difficult to divide, there are problems such as word segmentation errors and out-of-vocabulary (OOV), so the effect of BiLSTM as a feature extractor is not ideal.

[0004] In addition, traditional word vector pre-training methods are usually based on word or character feature extraction, ignoring the relevant information of the word in the context, and extracting a static word vector that does not contain the context. This method leads to a decrease in the model's entity recognition ability. Although the BERT pre-training model solves the above problems well through a multi-layer bidirectional Transformer encoder, improves the generalization ability of the word vector model, fully describes the character-level, word-level, sentence-level and even inter-sentence relationship features, and combines the contextual and semantic information of the context, thereby improving the model's entity recognition ability, it still faces challenges when the data set is limited.

[0005] In order to further improve the performance of NER models, researchers have proposed various attempts, including integrating the latest methods into existing tools. However, further improving the performance of named entity recognition based on existing methods, especially when the dataset is limited, remains an urgent problem to be solved.

[0006] The above contents are only used to assist in understanding the technical solution of the present invention and do not constitute an admission that the above contents are prior art.

[0007] Prior art document 1 (CN114707509A) discloses a method, apparatus, computer device and storage medium for traffic named entity recognition, the method comprising: obtaining a corpus, preprocessing the corpus, the preprocessing comprising using regular expressions to divide each corpus into multiple text sequences; inputting the multiple text characters into a trained bidirectional recurrent neural network and conditional random field to obtain traffic named entities.

[0008] The shortcoming of the prior art document 1 is that the Transformer model in the comparative document 1 mainly focuses on long-distance dependencies, which leads to a more complex attention mechanism and unbalanced attention weights; the significant difference between the present invention and the prior art document 1 is that the scaling factor in the Transformer model is deleted to make the Transformer model pay even attention to the words that need to be paid attention to; the beneficial effect achieved by the present invention relative to the prior art document 1 is that a relatively simple attention mechanism relative to the comparative document 1 is used to improve the calculation speed, and the accuracy of the named entity recognition method of this embodiment is improved by deleting the scaling factor and combining the attention mechanism of the present invention.

[0009] Prior art document 2 (CN115033659A) discloses a clause-level automatic summary model system and summary generation method based on deep learning. When the model system is used, the sentence-level extraction unit is first used to split the original data into words and sentences to construct input words and sentences; the encoder based on the BERT pre-trained model and the Transformer model is used to process the input words and sentences constructed by the sentence-level extraction unit to obtain a vector representation C″ of the clause containing contextual semantic information. i,j ; Use the classifier to preliminarily screen the vector representation and output the preliminary candidate clause summaries; finally, use the summary matcher based on BERTScore to calculate the semantic similarity between the candidate clause summaries and the original text, and obtain the summary that best matches the original text semantics.

[0010] The shortcoming of the prior art document 2 is that the entity corresponding to the long sentence cannot be extracted and named by splitting the long sentence into clauses and refining the summary. Summary of the invention

[0011] In order to solve the deficiencies in the prior art, the present invention provides a method, apparatus, device, and storage medium for named entity recognition, aiming to solve the technical problem of how to improve the accuracy of named entity recognition.

[0012] The present invention adopts the following technical solution.

[0013] A first aspect of the present invention provides a method for named entity recognition, comprising the following steps:

[0014] Step 1: Receive the text to be recognized and perform feature encoding processing, combine the dynamic word vector model, use the context information to generate the dynamic feature vector of the word unit, and obtain the input sequence;

[0015] Step 2: Use the pre-trained model to extract features from the input sequence, and extract contextual information from the input sequence by introducing dynamic position information and an optimized multi-head self-attention mechanism;

[0016] Step 3: Based on the extracted context information, predict the feature representation corresponding to the input sequence, use the conditional random field to adjust the weight according to the complexity of the named entity label through an adaptive weight adjustment mechanism, and combine the named entity recognition rules in the field of power equipment operation and maintenance to optimize the transition probability between labels.

[0017] Preferably, step 1 comprises:

[0018] Step 1.1, receiving input text and decomposing it into word units; the word units include basic units constituting the text;

[0019] Step 1.2, using a dynamic word vector model to perform feature encoding on the word element to generate a dynamic feature vector sequence containing context information;

[0020] Step 1.3, arranging the feature vectors according to the order of the word units in the text to generate an input sequence; the input sequence includes a corresponding feature vector sequence; the feature vector sequence includes subsequences, each subsequence corresponds to a word unit.

[0021] Preferably, the pre-training model includes a bidirectional recurrent neural network Bi-RNN and a modified Transformer model;

[0022] Step 2 includes:

[0023] Step 2.1: The input sequence is processed by the forward and backward RNN layers respectively to generate the forward and reverse feature representations of each word, and the forward and reverse feature representations are concatenated and fused to form a comprehensive feature vector.

[0024] Step 2.2: The comprehensive feature vector introduces relative position information through position encoding. Position encoding combined with dynamic position information enables the pre-trained model to retain the order and position information of word units when processing long sequences. After being processed by the multi-head self-attention mechanism and feedforward neural network, the final feature vector of each subsequence is generated, including the context information and relative position information of each word unit in the entire sequence.

[0025] Step 2.3, combining the processing results of the Bi-RNN and Transformer models, capturing the long-term dependencies between subsequences through Bi-RNN, and capturing the short-term dependencies between subsequences through the multi-head self-attention mechanism of the Transformer model;

[0026] In step 2.4, direction information and relative position information are added to train the context understanding ability of the pre-trained model, and the context information is optimized through the improved attention mechanism.

[0027] Preferably, in step 2.2, after being processed by a multi-head self-attention mechanism and a feedforward neural network, a final feature vector of each subsequence is generated, including:

[0028] The comprehensive feature vector generated by the bidirectional recurrent neural network Bi-RNN is input into the multi-attention head mechanism. The multi-attention heads independently calculate the attention score between each subsequence, and the results of all attention heads are concatenated to generate the final feature representation.

[0029] The position encoding generated by the sine and cosine functions is added to the feature representation generated by the multi-head self-attention mechanism to retain the relative position information of the word unit;

[0030] The feature representation processed by the multi-head self-attention mechanism is nonlinearly transformed through a feedforward neural network to extract the final feature vector of each subsequence.

[0031] Preferably, step 3 comprises:

[0032] Step 3.1, the feature representations of the Bi-RNN and Transformer models are integrated to form the final context feature vector;

[0033] Step 3.2, use a linear layer to map the final context feature vector to the new feature space and convert the context feature vector of each subsequence into a feature representation;

[0034] Step 3.3, input the feature representation into the conditional random field (CRF) model. The CRF model analyzes the transfer relationship between named entity labels and dynamically adjusts the weight parameters based on the frequency, context dependency, and complexity of label occurrence to optimize the calculation of label transfer probability and predict the optimal label sequence.

[0035] Step 3.4: label the subsequences in the input sequence according to the optimal label sequence output by the CRF model, identify the named entities, and convert the optimized feature representation into specific named entity labels.

[0036] A second aspect of the present invention provides a named entity recognition device, comprising: an input sequence construction module, an information extraction module and a feature representation module:

[0037] The input sequence construction module is used to receive the text to be recognized, perform feature coding on the text to be recognized, and generate an input sequence; the feature coding is used to convert each word element in the text to be recognized into a feature vector to form an input sequence; the text to be recognized includes word elements, and each word element corresponds to a subsequence in the input sequence;

[0038] The information extraction module is used to extract information from the input sequence through a pre-trained model to capture context information of the subsequence;

[0039] The feature representation module is used to map the context feature vector generated by the feature extraction module to a new feature space to generate a feature representation; the feature representation includes a subsequence corresponding to the preset entity label.

[0040] A third aspect of the present invention provides a named entity recognition device, the device comprising: a memory, a processor, and a computer program stored in the memory and executable on the processor, the computer program being configured to implement the steps of the named entity recognition method described above.

[0041] A fourth aspect of the present invention provides a storage medium, which is a computer-readable storage medium, and stores a computer program on the storage medium. When the computer program is executed by a processor, the steps of the named entity recognition method described above are implemented.

[0042] A fifth aspect of the present invention provides a computer program product, which includes a computer program, and when the computer program is executed by a processor, the steps of the named entity recognition method described above are implemented.

[0043] Compared with the prior art, the beneficial effects of the present invention include at least:

[0044] (1) The present invention ensures the accuracy and integrity of the input sequence through effective feature coding processing, providing a high-quality data foundation;

[0045] (2) The present invention can capture rich contextual information in the input sequence through the pre-trained model, including character-level, word-level and sentence-level features, effectively overcoming the word segmentation and out-of-vocabulary problems in named entities; through the Transformer structure, long-distance dependencies are handled, the gradient problem in traditional RNN is avoided, and the overall performance and generalization ability of the model are improved; among them, the Transformer model does not contain a scaling factor and the attention mechanism with direction and relative position information, which improves the accuracy of named entity recognition by improving the translation model;

[0046] (3) The present invention utilizes predicted feature representation and combines it with conditional random field (CRF) optimization to effectively improve the precision and accuracy of named entity recognition. By considering the transition probability between labels and maximizing the conditional probability of the global label sequence, the adaptability and robustness of the model under complex text structures are further enhanced. Combining the structure of Bi-RNN and CRF, the overfitting phenomenon of the model is effectively reduced, so that it can maintain stable performance on different data sets. BRIEF DESCRIPTION OF THE DRAWINGS

[0047] Figure 1 is a flow chart of a method for named entity recognition provided according to an embodiment of the present invention;

[0048] Figure 2 is a brief flow chart of a method for named entity recognition provided according to an embodiment of the present invention;

[0049] Figure 3 is a module structure diagram of a named entity recognition device provided according to an embodiment of the present invention;

[0050] Figure 4 It is a device structure diagram of a hardware operating environment involved in a named entity recognition method provided in accordance with an embodiment of the present invention. DETAILED DESCRIPTION

[0051] In order to make the purpose, technical solution and advantages of the present invention clearer, the technical solution of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. The embodiments described in the present invention are only part of the embodiments of the present invention, not all of the embodiments. Based on the spirit of the present invention, other embodiments obtained by ordinary technicians in this field without creative work are all within the scope of protection of the present invention.

[0052] like Figure 1 As shown, embodiment 1 of the present invention provides a method for named entity recognition, comprising the following steps:

[0053] Step 1, receiving the text to be recognized and performing feature encoding processing to obtain an input sequence;

[0054] Preferably, step 1 specifically includes:

[0055] Receive a text to be recognized, perform feature encoding processing on the text to be recognized, and obtain an input sequence, wherein the text to be recognized includes one or more word elements, and the input sequence refers to a feature vector sequence corresponding to the text to be recognized, and the input sequence includes one or more subsequences, and the subsequences correspond to the word elements of the text to be recognized;

[0056] It should be noted that the text to be recognized refers to the text input into the pre-trained model. Feature encoding is a key step in machine learning and data analysis. It involves converting raw data into a format that the model can understand and process. Feature encoding is particularly important in natural language processing (NLP) and named entity recognition (NER). A word element refers to a single word that makes up the text to be recognized. The input sequence refers to the data that the machine can recognize after the text to be recognized is processed through feature encoding.

[0057] It can be understood that, through step 1, the text to be recognized input by the user can be converted into an input sequence that can be processed by the machine and the model, which provides a basis for implementing subsequent steps.

[0058] Step 2: Use the pre-trained model to extract features from the input sequence to capture the contextual information of the subsequences in the input sequence;

[0059] Preferably, the pre-training model includes a bidirectional recurrent neural network Bi-RNN and a modified Transformer model; the context information refers to the long-term dependency and short-term dependency of a subsequence in the input sequence;

[0060] Step 2 specifically includes the following steps:

[0061] It should be noted that the pre-trained model refers to a machine learning model that has been pre-trained on a large amount of data, including a bidirectional recurrent neural network and a translation model. Contextual information refers to additional information that can help better understand and interpret the data in addition to the direct input data. Long-term dependencies refer to the relationship or influence between subsequences at distant positions in the input sequence, and short-term dependencies refer to the relationship or influence between subsequences at closer positions in the input sequence.

[0062] Step 2.1: The input sequence is processed by the forward and backward RNN layers respectively to generate the forward and reverse feature representations of each word, and the forward and reverse feature representations are concatenated or fused to form a comprehensive feature vector.

[0063] Step 2.2: The comprehensive feature vector introduces relative position information through position encoding, and after being processed by the multi-head self-attention mechanism and feedforward neural network, the final feature vector of each subsequence is generated, including the context information and relative position information of each word in the entire sequence;

[0064] Preferably, in step 2.2, the comprehensive feature vector generated by Bi-RNN is input into a multi-attention head mechanism, and the multi-attention heads independently calculate the attention scores between each subsequence, and the results of all attention heads are concatenated to generate the final feature representation;

[0065] The position encoding generated by the sine and cosine functions is added to the feature representation generated by the multi-head self-attention mechanism to retain the relative position information of the word unit;

[0066] The feature representation processed by the multi-head self-attention mechanism is nonlinearly transformed through a feedforward neural network to extract the final feature vector of each subsequence.

[0067] It is worth noting that the final feature representation refers to the feature representation of each subsequence (for example, a word or a token) after the multi-head self-attention mechanism. Multiple attention heads independently calculate the attention scores between each subsequence, and concatenate these results to generate the final feature representation of each subsequence;

[0068] Relative position information: The Transformer model does not use sequential information like traditional RNNs, so position encoding needs to be introduced to preserve the position information of the word unit. The position encoding is generated by sine and cosine functions and added to the initial feature representation, thereby introducing relative position information in the feature representation. This ensures that the model can distinguish the different meanings of the same word unit in different positions.

[0069] Final feature vector: refers to the final feature representation of each subsequence after passing through the feedforward neural network. The feedforward neural network consists of two linear transformation layers and an activation function, which further extracts and optimizes the feature representation through nonlinear transformation to generate the final feature vector of each subsequence.

[0070] The significant difference between the present invention and the prior art is that the present invention analyzes the entire paragraph through an additional attention mechanism to further extract and name the entity names.

[0071] Step 2.3, combining the processing results of the Bi-RNN and Transformer models, capturing the long-term dependencies between subsequences through Bi-RNN, and capturing the short-term dependencies between subsequences through the multi-head self-attention mechanism of the Transformer model;

[0072] In step 2.4, direction information and relative position information are added to train the context understanding ability of the pre-trained model, and the context information is optimized through the improved attention mechanism.

[0073] In a possible embodiment, no scaling factor is included in the translation model;

[0074] The step of extracting features from the input sequence by using a pre-trained model to capture the context information of the subsequence includes step A10:

[0075] Step A10, capture the necessary subsequence of the input sequence through the translation model, where the necessary subsequence refers to the subsequence corresponding to the tokens that the pre-trained model focuses on.

[0076] It should be noted that the preset entity label refers to a specific entity name in a certain field. For example, in power operation and maintenance, according to the work order elements, preset entity labels such as "equipment name", "equipment voltage level", "line to which the equipment belongs", "substation to which the equipment belongs", "equipment damage location", "equipment maintenance situation", and "equipment maintenance time" are summarized.

[0077] By deleting the scaling factor, the attention weights of the pre-trained model are changed. Since more attention is paid to the important words in the text to be recognized, the accuracy of the named entity recognition method in this embodiment is improved.

[0078] In a feasible embodiment, the translation model includes an attention mechanism with direction and relative position information, and the attention mechanism includes an additional attention mechanism and a position encoding method;

[0079] The steps of capturing the context information of the subsequence by extracting features from the input sequence through the pre-trained model include Step A101:

[0080] Step A101, through the attention mechanism, extract the direction and relative position relationship between the necessary subsequences in the input sequence to generate context information.

[0081] It should be noted that the direction refers to the sequentiality of the subsequences in the sequence, that is, whether the subsequence is forward (future) or backward (past). For example, in a sentence, "he" is before "playing football", indicating a temporal or logical order. The relative position information refers to the distance or position relationship between the subsequences in the sequence, regardless of their absolute positions in the sequence. This helps the model understand the proximity or interval between the subsequences.

[0082] It can be understood that combining direction and relative position information can help the model better understand the context. For example, determining whether a word modifies the previous noun or the following noun, thereby generating context information.

[0083] Furthermore, the necessary subsequence is associated with the preset entity label. For example, when the text content to be recognized corresponding to the subsequence is the Chinese character "electricity", then this subsequence is a necessary subsequence. If the corresponding text content to be recognized is the Chinese character "I", then this subsequence is not a necessary subsequence.

[0084] Through the attention mechanism with direction and relative position information, the context understanding ability of the named entity recognition method of this embodiment is enhanced. The model can more comprehensively understand the context of words or phrases in the sequence, including the semantic relationship and syntactic structure between them, so as to more accurately identify entity boundaries and categories, thereby improving the accuracy of the named entity recognition method of this embodiment.

[0085] Step 3: Based on the extracted context information, predict the feature representation corresponding to the input sequence, use the conditional random field to adjust the weight according to the complexity of the named entity label through an adaptive weight adjustment mechanism, and combine the named entity recognition rules in the field of power equipment operation and maintenance to optimize the transition probability between labels.

[0086] Preferably, step 3 specifically includes the following steps:

[0087] It should be noted that feature representation refers to the combination of named entities. For example, the content of an operation and maintenance work order for power equipment includes: equipment No. 1, low voltage level, intact chassis, good machine condition, and capacity kilovolt-amperes. The corresponding feature representation is equipment No. 1, low voltage, good, and kilovolt-amperes, where equipment No. 1 corresponds to the equipment name, low voltage corresponds to the voltage level, good corresponds to the equipment operation status, and kilovolt-amperes corresponds to the equipment capacity. The above is a popular explanation of feature representation. The actual feature representation is vector data, which is data that can be recognized by the machine.

[0088] In a feasible embodiment, the step B10 is also included:

[0089] Step B10, inputting the feature representation into a conditional random field, constraining the feature representation, and outputting a constrained feature representation, wherein the conditional random field calculates the transition probability between preset entity labels in the feature representation.

[0090] The output constraint feature representation refers to the feature representation of the output after the conditional random field constraint. This feature representation is the feature representation that conforms to the meaning of the input text after the transfer probability is calculated.

[0091] Errors may occur when performing feature representation on the text to be recognized. For example, if the input text is "Alibaba Group was founded in 1999 and is headquartered in Hangzhou, China", the headquarters may be mistakenly identified as a location. If the transition probability is calculated by inputting a conditional random field, this feature representation will be excluded, thereby outputting a better feature representation.

[0092] In a feasible embodiment, the bidirectional recurrent neural network adopted by the pre-training model includes steps C10 to C20:

[0093] Step C10, two independent recurrent neural network layers, processing the forward and reverse information of the input sequence respectively;

[0094] It should be noted that the two independent recurrent neural network layers include a forward neural network layer and a reverse neural network layer, wherein the forward neural network layer processes information in the order from the first subsequence to the last subsequence, and the reverse neural network layer processes information in the order from the last subsequence to the first subsequence.

[0095] It can be understood that forward information refers to information processed in the natural order of the input sequence (such as the time sequence in a time series or the order of words in a text), and reverse information refers to information processed in the opposite direction to the natural order of the input sequence.

[0096] Step C20, concatenating or fusing the information in the two directions to generate feature information.

[0097] It is understandable that in this way, the model can more comprehensively understand the information in the input sequence, capture richer contextual information, and then generate more accurate and robust feature representations to support downstream tasks.

[0098] In a feasible embodiment, steps D10 to D20 are also included:

[0099] Step D10, receiving a cluster of power equipment operation and maintenance work orders, and annotating the cluster of operation and maintenance work orders by using preset entity tags to construct a training set;

[0100] It should be noted that the power equipment operation and maintenance work order cluster refers to the work order data related to the operation and maintenance of power equipment, which may include equipment maintenance records, inspection reports, fault logs, etc. Preset entity tags refer to determining the types of named entities that need to be identified and defining preset entity tags for these types, such as "equipment name", "voltage level", "belonging line", etc. Entity labeling refers to the use of text labeling tools or manual methods to perform entity labeling on the text data in the operation and maintenance work order set, that is, marking each named entity in the text and assigning corresponding labels. The training set refers to the power equipment operation and maintenance work order cluster after entity labeling.

[0101] By combining the labeled operation and maintenance work order data into a training set, this training set will be used for subsequent model training and is the basis for model learning entity recognition.

[0102] In a feasible embodiment, in the process of constructing the training set, necessary data cleaning is performed, including removing irrelevant information, correcting labeling errors, etc., to ensure data quality.

[0103] Step D20, pre-training is performed using the training set to obtain a pre-trained model.

[0104] Pre-training is performed on the training set to enable the model to learn the language patterns and entity features in the field of power equipment operation and maintenance.

[0105] The named entity recognition method of this embodiment can build a model that can understand and identify key information in power equipment operation and maintenance work orders. The pre-trained model not only learns common language features, but also learns professional knowledge through domain-specific training data, thereby improving the applicability and accuracy of the model in actual operation and maintenance tasks.

[0106] For example, to help understand the technical concept or technical principle of the named entity recognition method after combining this embodiment with the above embodiment, please refer to Figure 2 , Figure 2 A brief flowchart of a named entity recognition method is provided, as follows:

[0107] Among them, Encoders refers to encoders, Decoders refers to improved transformer decoders, Linear refers to linear data structure arrays and linked lists, and Softmax is a function widely used in machine learning and deep learning, especially when dealing with classification problems. It is often used in the output layer of multi-classification models to convert the raw scores (also called logits) output by the model into probability distributions. Pre-trained word vectors are equivalent to preset entity labels.

[0108] Step 1: Obtain relevant data sets (text to be input) as input values ​​and put them into the input layer, and preprocess the obtained relevant data sets;

[0109] Step 2: Build an improved transformer-crf network structure including parallel encoder / decoder, multi-head self-attention mechanism, position encoding and residual connection;

[0110] Step 3: Use the parallelized improved transformer as a feature extractor, extract features from the data set through a multi-head self-attention mechanism and a feedforward neural network, and use the position encoding information to obtain the feature vector;

[0111] Step 4: Input the obtained feature vector into the improved transformer decoder for information extraction, and parallelize the decoding part;

[0112] Step 5: Input the training results of the transformer into the conditional random field layer crf, constrained by the conditional random field layer crf, and output the optimal label sequence.

[0113] It should be noted that the above examples are only used to assist in understanding the present invention and do not constitute a limitation on the named entity identification method of the present invention. More simple transformations based on this technical concept are all within the scope of protection of the present invention.

[0114] like Figure 3 As shown, embodiment 2 of the present invention provides a named entity recognition device, the named entity recognition device comprising:

[0115] An input sequence construction module 10 is used to receive a text to be recognized including one or more named entities, perform feature encoding processing on the text to be recognized, and obtain an input sequence, wherein the text to be recognized includes one or more word elements, and the input sequence refers to a feature vector sequence corresponding to the text to be recognized, and the input sequence includes one or more subsequences, and the subsequences correspond to the word elements of the text to be recognized;

[0116] An information extraction module 20 is used to extract information from the input sequence through a pre-trained model to capture context information of the subsequence, wherein the pre-trained model includes a bidirectional recurrent neural network and a translation model, and the context information refers to the long-term dependency and short-term dependency of a subsequence in the input sequence;

[0117] The feature representation module 30 is used to make a prediction based on the context information to obtain a feature representation corresponding to the input sequence, wherein the feature representation includes a subsequence corresponding to the preset entity tag.

[0118] Optionally, the translation model does not include a scaling factor;

[0119] The information extraction module 20 is also used for:

[0120] The necessary subsequences of the input sequence are captured through the translation model, where the necessary subsequences refer to the subsequences corresponding to the word units that the pre-training model focuses on.

[0121] Optionally, the translation model includes an attention mechanism with direction and relative position information, and the attention mechanism includes an additional attention mechanism and a position encoding method;

[0122] The information extraction module 20 is also used for:

[0123] Through the attention mechanism, the directions and relative position relationships between the necessary subsequences in the input sequence are extracted to generate context information.

[0124] Optionally, the feature representation module 30 is further used to: input the feature representation into a conditional random field, constrain the feature representation, and output a constrained feature representation, wherein the conditional random field calculates the transition probability between preset entity labels in the feature representation.

[0125] Optionally, the input sequence construction module 10 is further used to: construct a bidirectional recurrent neural network;

[0126] The bidirectional recurrent neural network used in the pre-training model includes:

[0127] Two independent recurrent neural network layers, processing the forward and reverse information of the input sequence respectively;

[0128] The information from two directions is spliced ​​or fused to generate feature information.

[0129] Optionally, the input sequence construction module 10 is further used to: receive a cluster of power equipment operation and maintenance work orders, and perform entity labeling on the cluster of operation and maintenance work orders using preset entity tags to construct a training set;

[0130] Pre-training is performed using the training set to obtain a pre-training model.

[0131] The named entity recognition device provided in the embodiment of the present invention adopts the named entity recognition method in the above embodiment, which can solve the technical problem of how to improve the accuracy of named entity recognition. Compared with the prior art, the beneficial effects of the named entity recognition device provided in the embodiment of the present invention are the same as the beneficial effects of the named entity recognition method provided in the above embodiment, and other technical features in the named entity recognition device are the same as the features disclosed in the above embodiment method, which will not be repeated here.

[0132] like Figure 4 As shown, the present invention provides a named entity recognition device, which includes: at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor so that the at least one processor can execute the named entity recognition method in the above-mentioned embodiment one.

[0133] Reference below Figure 4 , which shows a schematic diagram of the structure of a named entity recognition device suitable for implementing an embodiment of the present invention. The named entity recognition device in the embodiment of the present invention may include, but is not limited to, mobile terminals such as mobile phones, laptop computers, digital broadcast receivers, PDAs (Personal Digital Assistants), PADs (Portable Application Descriptions), PMPs (Portable Media Players), vehicle-mounted terminals (such as vehicle-mounted navigation terminals), etc., and fixed terminals such as digital TVs, desktop computers, etc. Figure 4 The named entity recognition device shown is only an example and should not bring any limitation to the functions and scope of use of the embodiments of the present invention.

[0134] like Figure 4As shown, the named entity recognition device may include a processing device 1001 (e.g., a central processing unit, a graphics processing unit, etc.), which can perform various appropriate actions and processes according to a program stored in a read-only memory (ROM: Read Only Memory) 1002 or a program loaded from a storage device 1003 to a random access memory (RAM: Random Access Memory) 1004. In RAM1004, various programs and data required for the operation of the xxx device are also stored. The processing device 1001, ROM1002, and RAM1004 are connected to each other through a bus 1005. An input / output (I / O) interface 1006 is also connected to the bus. Generally, the following systems can be connected to the I / O interface 1006: an input device 1007 including, for example, a touch screen, a touch pad, a keyboard, a mouse, an image sensor, a microphone, an accelerometer, a gyroscope, etc.; an output device 1008 including, for example, a liquid crystal display (LCD: Liquid Crystal Display), a speaker, a vibrator, etc.; a storage device 1003 including, for example, a magnetic tape, a hard disk, etc.; and a communication device 1009. The communication device 1009 can allow the named entity recognition device to communicate with other devices wirelessly or by wire to exchange data. Although the figure shows a named entity recognition device with various systems, it should be understood that it is not required to implement or have all the systems shown. More or fewer systems can be implemented or provided alternatively.

[0135] In particular, according to the embodiments disclosed in the present invention, the process described above with reference to the flowchart can be implemented as a computer software program. For example, the embodiments disclosed in the present invention include a computer program product, which includes a computer program carried on a computer-readable medium, and the computer program contains program code for executing the method shown in the flowchart. In such an embodiment, the computer program can be downloaded and installed from the network through a communication device, or installed from the storage device 1003, or installed from the ROM 1002. When the computer program is executed by the processing device 1001, the above-mentioned functions defined in the method of the embodiment disclosed in the present invention are executed.

[0136] The named entity recognition device provided by the present invention adopts the named entity recognition method in the above embodiment, which can solve the technical problem of how to improve the accuracy of named entity recognition. Compared with the prior art, the beneficial effects of the named entity recognition device provided by the present invention are the same as the beneficial effects of the named entity recognition method provided by the above embodiment, and other technical features in the named entity recognition device are the same as the features disclosed in the method of the previous embodiment, which will not be repeated here.

[0137] It should be understood that the various parts disclosed in the present invention can be implemented by hardware, software, firmware or a combination thereof. In the description of the above embodiments, specific features, structures, materials or characteristics can be combined in any one or more embodiments or examples in a suitable manner.

[0138] The above is only a specific embodiment of the present invention, but the protection scope of the present invention is not limited thereto. Any person skilled in the art can easily think of changes or substitutions within the technical scope disclosed by the present invention, which should be included in the protection scope of the present invention. Therefore, the protection scope of the present invention should be based on the protection scope of the claims.

[0139] The present invention provides a computer-readable storage medium having computer-readable program instructions (ie, computer programs) stored thereon, wherein the computer-readable program instructions are used to execute the named entity recognition method in the above-mentioned embodiment.

[0140] The beneficial effects of the present invention include at least:

[0141] (1) The present invention ensures the accuracy and integrity of the input sequence through effective feature coding processing, providing a high-quality data foundation;

[0142] (2) The present invention can capture rich contextual information in the input sequence through the pre-trained model, including character-level, word-level and sentence-level features, effectively overcoming the word segmentation and out-of-vocabulary problems in named entities; through the Transformer structure, long-distance dependencies are handled, the gradient problem in traditional RNN is avoided, and the overall performance and generalization ability of the model are improved; among them, the Transformer model does not contain a scaling factor and the attention mechanism with direction and relative position information, which improves the accuracy of named entity recognition by improving the translation model;

[0143] (3) The present invention utilizes predicted feature representation and combines it with conditional random field (CRF) optimization to effectively improve the precision and accuracy of named entity recognition. By considering the transition probability between labels and maximizing the conditional probability of the global label sequence, the adaptability and robustness of the model under complex text structures are further enhanced. Combining the structure of Bi-RNN and CRF, the overfitting phenomenon of the model is effectively reduced, so that it can maintain stable performance on different data sets.

[0144] The computer-readable storage medium provided by the present invention may be, for example, a USB flash drive, but is not limited to electrical, magnetic, optical, electromagnetic, infrared, or semiconductor systems, systems or devices, or any combination thereof. More specific examples of computer-readable storage media may include, but are not limited to: an electrical connection with one or more wires, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination thereof. In this embodiment, the computer-readable storage medium may be any tangible medium containing or storing a program that may be used by or in combination with an instruction execution system, system or device. The program code contained on the computer-readable storage medium may be transmitted using any suitable medium, including but not limited to: wires, optical cables, RF (Radio Frequency), etc., or any suitable combination thereof.

[0145] The computer-readable storage medium may be included in the named entity recognition device; or may exist independently without being assembled into the named entity recognition device.

[0146] The computer-readable storage medium carries one or more programs. When the one or more programs are executed by the named entity recognition device, the named entity recognition device: receives the text to be recognized, performs feature encoding processing on the text to be recognized, and obtains an input sequence, wherein the text to be recognized includes one or more word units, and the input sequence refers to a feature vector sequence corresponding to the text to be recognized, and the input sequence includes one or more subsequences, and the subsequences correspond to the word units of the text to be recognized; extracts features from the input sequence through a pre-trained model to capture the context information of the subsequence, wherein the pre-trained model includes a bidirectional recurrent neural network and a translation model, and the context information refers to the long-term dependency and short-term dependency of a subsequence in the input sequence; predicts according to the context information to obtain a feature representation corresponding to the input sequence, wherein the feature representation includes a subsequence corresponding to a preset entity tag. The computer program code for performing the operation of the present invention can be written in one or more programming languages ​​or a combination thereof, and the programming language includes an object-oriented programming language such as Java, Smalltalk, C++, and a conventional procedural programming language such as "C" language or a similar programming language. The program code may be executed entirely on the user's computer, partially on the user's computer, as a separate software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In the case of a remote computer, the remote computer may be connected to the user's computer through any type of network, including a local area network (LAN) or a wide area network (WAN), or may be connected to an external computer (e.g., through the Internet using an Internet service provider).

[0147] The flow chart and block diagram in the accompanying drawings illustrate the possible architecture, function and operation of the equipment, method and computer program product according to various embodiments of the present invention. In this regard, each box in the flow chart or block diagram can represent a module, a program segment or a part of a code, and the module, the program segment or a part of the code contains one or more executable instructions for realizing the specified logical function. It should also be noted that in some alternative implementations, the functions marked in the box can also occur in a different order from the order marked in the accompanying drawings. For example, two boxes represented in succession can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each box in the block diagram and / or flow chart, and the combination of the boxes in the block diagram and / or flow chart can be implemented with a dedicated hardware-based system that performs the specified function or operation, or can be implemented with a combination of dedicated hardware and computer instructions.

[0148] The modules involved in the embodiments of the present invention may be implemented by software or hardware, wherein the name of the module does not limit the unit itself in some cases.

[0149] The readable storage medium provided by the present invention is a computer-readable storage medium, which stores computer-readable program instructions (i.e., computer programs) for executing the above-mentioned named entity recognition method, and can solve the technical problem of how to improve the accuracy of named entity recognition. Compared with the prior art, the beneficial effects of the computer-readable storage medium provided by the present invention are the same as the beneficial effects of the named entity recognition method provided by the above-mentioned embodiment, and will not be repeated here.

[0150] An embodiment of the present invention provides a computer program product, including a computer program, and when the computer program is executed by a processor, the steps of the named entity recognition method as described above are implemented.

[0151] The computer program product provided by the present invention can solve the technical problem of how to improve the accuracy of named entity recognition. Compared with the prior art, the beneficial effects of the computer program product provided by the embodiment of the present invention are the same as the beneficial effects of the named entity recognition method provided by the above embodiment, which will not be repeated here.

[0152] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention rather than to limit it. Although the present invention has been described in detail with reference to the above embodiments, ordinary technicians in the relevant field should understand that the specific implementation methods of the present invention can still be modified or replaced by equivalents, and any modifications or equivalent replacements that do not depart from the spirit and scope of the present invention should be covered within the scope of protection of the claims of the present invention.

Claims

1. A method for named entity recognition, characterized in that: The following steps are involved: Step 1: Receive the text to be recognized and perform feature encoding processing, combine the dynamic word vector model, use the context information to generate the dynamic feature vector of the word unit, and obtain the input sequence; Step 2: Use the pre-trained model to extract features from the input sequence, and extract contextual information from the input sequence by introducing dynamic position information and an optimized multi-head self-attention mechanism; Step 3: Based on the extracted context information, predict the feature representation corresponding to the input sequence, use the conditional random field to adjust the weight according to the complexity of the named entity label through an adaptive weight adjustment mechanism, and combine the named entity recognition rules in the field of power equipment operation and maintenance to optimize the transition probability between labels to adapt to the named entity relationship and label and classify the named entities.

2. A method for named entity recognition according to claim 1, characterized in that: Step 1 includes: Step 1.1, receiving input text and decomposing it into word units; the word units include basic units constituting the text; Step 1.2, using a dynamic word vector model to perform feature encoding on the word element to generate a dynamic feature vector sequence containing context information; Step 1.3, arranging the feature vectors according to the order of the word units in the text to generate an input sequence; the input sequence includes a corresponding feature vector sequence; the feature vector sequence includes subsequences, each subsequence corresponds to a word unit.

3. A method for named entity recognition according to claim 1, characterized in that: The pre-training model includes a bidirectional recurrent neural network Bi-RNN and an improved Transformer model; Step 2 includes: Step 2.1: The input sequence is processed by the forward and backward RNN layers respectively to generate the forward and reverse feature representations of each word, and the forward and reverse feature representations are concatenated and fused to form a comprehensive feature vector. Step 2.2: The comprehensive feature vector introduces relative position information through position encoding. Position encoding combined with dynamic position information enables the pre-trained model to retain the order and position information of word units when processing long sequences. After being processed by the multi-head self-attention mechanism and feedforward neural network, the final feature vector of each subsequence is generated, including the context information and relative position information of each word unit in the entire sequence. Step 2.3, combining the processing results of the Bi-RNN and Transformer models, capturing the long-term dependencies between subsequences through Bi-RNN, and capturing the short-term dependencies between subsequences through the multi-head self-attention mechanism of the Transformer model; In step 2.4, direction information and relative position information are added to train the context understanding ability of the pre-trained model, and the context information is optimized through the improved attention mechanism.

4. A method for named entity recognition according to claim 3, characterized in that: In step 2.2, after processing by the multi-head self-attention mechanism and the feedforward neural network, the final feature vector of each subsequence is generated, including: The comprehensive feature vector generated by the bidirectional recurrent neural network Bi-RNN is input into the multi-attention head mechanism. The multi-attention heads independently calculate the attention score between each subsequence, and the results of all attention heads are concatenated to generate the final feature representation. The position encoding generated by the sine and cosine functions is added to the feature representation generated by the multi-head self-attention mechanism to retain the relative position information of the word unit; The feature representation processed by the multi-head self-attention mechanism is nonlinearly transformed through a feedforward neural network to extract the final feature vector of each subsequence.

5. A method for named entity recognition according to claim 1, characterized in that: Step 3 includes: Step 3.1, the feature representations of the Bi-RNN and Transformer models are integrated to form the final context feature vector; Step 3.2, use a linear layer to map the final context feature vector to the new feature space and convert the context feature vector of each subsequence into a feature representation; Step 3.3, input the feature representation into the conditional random field (CRF) model. The CRF model analyzes the transfer relationship between named entity labels and dynamically adjusts the weight parameters based on the frequency, context dependency, and complexity of label occurrence to optimize the calculation of label transfer probability and predict the optimal label sequence. Step 3.4: label the subsequences in the input sequence according to the optimal label sequence output by the CRF model, identify the named entities, and convert the optimized feature representation into specific named entity labels.

6. A method for named entity recognition according to claim 5, characterized in that: In step 3.3, the feature representation output by the linear layer is input into the conditional random field model. The CRF model optimizes the feature representation and predicts the optimal label sequence by considering the transition probability between labels. In the CRF model, each feature representation and the corresponding label are modeled as a state, and the relationship between the modeled labels is the transition probability. By maximizing the conditional probability of the global label sequence, the CRF model optimizes the prediction of the input sequence.

7. A named entity recognition device, comprising: Input sequence building module, information extraction module and feature representation module; characterized by: The input sequence construction module is used to receive the text to be recognized, perform feature coding on the text to be recognized, and generate an input sequence; the feature coding is used to convert each word element in the text to be recognized into a feature vector to form an input sequence; the text to be recognized includes word elements, and each word element corresponds to a subsequence in the input sequence; The information extraction module is used to extract information from the input sequence through a pre-trained model to capture context information of the subsequence; The feature representation module is used to map the context feature vector generated by the feature extraction module to a new feature space to generate a feature representation; the feature representation includes a subsequence corresponding to the preset entity label.

8. A named entity recognition device, characterized in that: The device comprises: a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the computer program is configured to implement the steps of the named entity recognition method according to any one of claims 1 to 6.

9. A storage medium, characterized in that: The storage medium is a computer-readable storage medium, and a computer program is stored on the storage medium. When the computer program is executed by a processor, the steps of the named entity recognition method according to any one of claims 1 to 6 are implemented.

10. A computer program product, characterized in that: The computer program product comprises a computer program, and when the computer program is executed by a processor, the steps of the method for named entity recognition according to any one of claims 1 to 6 are implemented.

Images