Named entity identification method and device, medium and equipment

Abstract

The invention discloses a named entity recognition method and device, a medium and equipment, and relates to the technical field of natural language processing. According to the method, an ALBERT model is used as an embedded presentation layer of text data, a parallel DoouBiLSTM model and a multi-window CNN model form a dual-channel coding layer, the dual-channel coding layer is serially connected to the embedded presentation layer and then serially connected to a tag decoding layer formed based on a conditional random field, a combined model is constructed, named entity recognition training is performed through a sample text, and the recognition efficiency of the named entity is improved. And finally, for a target text to be recognized, generating a dynamic representation vector of the target text through an ALBERT model of the named entity recognition model, extracting global semantic features and multi-granularity local semantic features of the target text through a dual-channel coding layer, and decoding through a tag decoding layer to obtain a named entity. According to the method, the advantages of the BiLSTM model and the CNN model are fully combined, and the recognition effect of the fine-grained named entities is improved.

Description

Technical Field

[0001] The present invention relates to the technical field of natural language processing, and in particular to a method, device, medium and equipment for named entity recognition. Background Art

[0002] At present, named entity recognition (NER) is an important task in the field of natural language processing. It aims to identify entities with specific meanings from text, such as names of people, places, and organizations. With the rapid development of information technology, named entity recognition plays an increasingly important role in many application scenarios such as information extraction, question-answering systems, and machine translation. English named entity recognition developed earlier and has achieved good results, but the recognition of Chinese named entities is more complicated. This is mainly because compared with English, Chinese has richer vocabulary meanings and more flexible sentence patterns. The effect of Chinese named entity recognition is greatly affected by the results of word segmentation, and the phenomenon of polysemy in Chinese is very common, which brings more challenges to Chinese named entity recognition. Therefore, due to the complexity of the Chinese language structure and the polysemy of vocabulary, higher granularity is required in Chinese named entity recognition.

[0003] Compared with traditional entity recognition, fine-grained named entity recognition for Chinese has the following two main features: First, there are more entity categories, such as book titles, scenic spots, animals, plants, etc., and new categories will continue to emerge over time; second, fine-grained entity categories are more hierarchical, for example, traditional place names can be further subdivided into city names, university names, community names, etc., and hospitals can be subdivided into general hospitals, specialized hospitals, teaching hospitals, etc. This fine-grained entity division provides more semantic information and can provide more possibilities for downstream tasks in the field of natural language processing.

[0004] However, traditional named entity recognition methods often perform poorly in fine-grained entity recognition. Summary of the invention

[0005] Based on this, it is necessary to provide a method, device, medium and equipment for named entity recognition in response to the above technical problems.

[0006] This manual adopts the following technical solutions:

[0007] This specification provides a method for named entity recognition, including:

[0008] Acquire text data as sample text, and annotate real named entities in the sample text;

[0009] The ALBERT model is used as the embedding representation layer of text data. The parallel DouBiLSTM model and multi-window CNN model form a dual-channel encoding layer and are connected in series to the embedding representation layer. The label decoding layer formed based on the conditional random field is then connected in series to construct a combined model.

[0010] Input the sample text into the combined model to obtain predicted named entities; and train the combined model based on the real named entities and the predicted named entities to obtain a named entity recognition model;

[0011] The target text to be recognized by named entity is obtained and input into the named entity recognition model, and the dynamic representation vector corresponding to the target text is generated by the ALBERT model; the global semantic features of the target text are extracted according to the dynamic representation vector by the DouBiLSTM model, and the multi-granularity local semantic features of the target text are extracted according to the dynamic representation vector by the multi-window CNN model; the global semantic features and the multi-granularity local semantic features are spliced ​​through the label decoding layer, and the spliced ​​semantic features are mapped to the classification label space to obtain prediction information, so as to decode the prediction information through the conditional random field to obtain the named entity.

[0012] Optionally, generating a dynamic representation vector corresponding to the target text through the ALBERT model specifically includes:

[0013] The target text is segmented by the word segmenter of the ALBERT model to obtain multiple subwords; each subword is mapped to the corresponding word vector through the TokenEmbeddings module of the ALBERT model;

[0014] The target text is segmented into sentences through the SegmentEmbeddings module of the ALBERT model, and sentence embedding representations are generated for each sentence.

[0015] Generate position embedding representation for each character position in the target text through the PositionEmbeddings module of the ALBERT model;

[0016] A comprehensive embedding representation of the target text is obtained based on the word vector, sentence embedding representation, and position embedding representation, and is input into the Transformer encoder of the ALBERT model to obtain a context-based dynamic representation vector of the target text.

[0017] Optionally, extracting the global semantic features of the target text according to the dynamic representation vector by using the DouBiLSTM model specifically includes:

[0018] The global semantic features of the target text are extracted based on the dynamic representation vector of the target text based on the DouBiLSTM model through the following formula:

[0019] G = H(V) + H(H(V));

[0020] Among them, V represents the dynamic representation vector of the target text, H() represents the output of the BiLSTM network, and G represents the global semantic features of the target text.

[0021] Optionally, extracting multi-granularity local semantic features of the target text according to the dynamic representation vector through the multi-window CNN model specifically includes:

[0022] The dynamic representation vector corresponding to each character in the target text is convolved based on the convolution kernels of different sizes in the multi-window CNN model through the following formula, and activated with the ReLU activation function to extract the multi-granularity local semantic features of the target text:

[0023]

[0024] Among them, C k Represents the multi-granular local semantic features of the target text, represents the multi-granularity local features of the i-th character, k represents the window size of the convolution kernel, and W c represents the weight matrix parameters of the convolution kernel, b c represents the bias vector of the convolution kernel, Indicates Characters to The input word vector sequence of characters, ReLU() represents the activation function.

[0025] Optionally, mapping the spliced ​​semantic features obtained by splicing to a classification label space to obtain prediction information, and decoding the prediction information through a conditional random field to obtain a named entity, specifically includes:

[0026] The concatenated semantic features are mapped to multiple classification spaces based on the fully connected layer in the label decoding layer through the following formula to obtain prediction information:

[0027] N=MA c +b c ;

[0028] The predicted information is decoded and recognized based on the conditional random field through the following formula:

[0029]

[0030]

[0031] Among them, N represents the prediction information, M represents the spliced ​​semantic features, and A c represents the feature weight matrix of the fully connected layer, b crepresents the bias vector, Y represents the predicted named entity sequence, Sorce(N,Y) represents the score of the predicted named entity sequence based on the prediction information, Represents the predicted named entity Y i To predict named entity Y i+1 The transition probability score, Indicates that the predicted named entity is Y i The softmax output score of the i-th character, Y * Represents the highest scoring named entity sequence.

[0032] This specification provides a named entity recognition device, including:

[0033] An acquisition module, used for acquiring text data as sample text and marking real named entities in the sample text;

[0034] A construction module is used to use the ALBERT model as the embedding representation layer of text data, form a dual-channel encoding layer with a parallel DouBiLSTM model and a multi-window CNN model and connect them in series after the embedding representation layer, and then connect them in series with a label decoding layer formed based on a conditional random field to build a combined model;

[0035] A training module is used to input sample text into the combined model to obtain predicted named entities; and train the combined model according to the real named entities and the predicted named entities to obtain a named entity recognition model;

[0036] The recognition module is used to obtain the target text to be recognized by named entity and input it into the named entity recognition model, generate the dynamic representation vector corresponding to the target text through the ALBERT model; extract the global semantic features of the target text according to the dynamic representation vector through the DouBiLSTM model, and extract the multi-granularity local semantic features of the target text according to the dynamic representation vector through the multi-window CNN model; splice the global semantic features and the multi-granularity local semantic features through the label decoding layer, and map the spliced ​​semantic features to the classification label space to obtain prediction information, so as to decode the prediction information through the conditional random field to obtain the named entity.

[0037] This specification provides a computer-readable storage medium, which stores a computer program. When the computer program is executed by a processor, the above-mentioned named entity recognition method is implemented.

[0038] This specification provides a computer device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the above-mentioned named entity recognition method when executing the program.

[0039] At least one of the above technical solutions adopted in this specification can achieve the following beneficial effects:

[0040] The present invention uses an ALBERT model as an embedding representation layer for text data, forms a dual-channel encoding layer with a parallel DouBiLSTM model and a multi-window CNN model and connects the dual-channel encoding layer in series to the embedding representation layer, and then connects a label decoding layer formed based on a conditional random field in series to construct a combined model; trained named entity recognition through sample text to obtain a named entity recognition model; finally, for a target text to be recognized, a dynamic representation vector of the target text is generated through the ALBERT model of the named entity recognition model, and global semantic features and multi-granularity local semantic features of the target text are extracted through the dual-channel encoding layer, and then the named entity is decoded through the label decoding layer to obtain the named entity.

[0041] The present invention fully combines the advantages of the BiLSTM model and the CNN model to fully extract the global semantic features and local semantic features of the target text, thereby improving the recognition effect of fine-grained named entities. BRIEF DESCRIPTION OF THE DRAWINGS

[0042] The drawings described herein are used to provide a further understanding of the present invention and constitute a part of the present invention. The exemplary embodiments of the present invention and their descriptions are used to explain the present invention and do not constitute an improper limitation of the present invention. In the drawings:

[0043] Figure 1 A flowchart of a named entity recognition method provided in this specification;

[0044] Figure 2 A schematic diagram of a constructed model framework provided in this specification;

[0045] Figure 3 A schematic diagram of an embedded representation process provided for this specification;

[0046] Figure 4 A basic structural diagram of a global feature perception unit provided in this specification;

[0047] Figure 5 A schematic diagram of a maximum pooling layer structure provided in this specification;

[0048] Figure 6 A schematic diagram of a named entity recognition device provided in this specification;

[0049] Figure 7 A schematic diagram of a computer device for implementing a named entity recognition method provided in this specification. DETAILED DESCRIPTION

[0050] In order to make the purpose, technical solution and advantages of the present invention clearer, the technical solution of the present invention will be clearly and completely described below in conjunction with the specific embodiments of the present invention and the corresponding drawings. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative work are within the scope of protection of the present invention.

[0051] At present, traditional named entity recognition methods are mainly based on rules and statistical machine learning. Although these traditional methods are easier to explain, their recognition effect depends largely on the modeling capabilities and rules for specific tasks. In particular, the task of fine-grained NER places higher requirements on the modeling capabilities of traditional methods. In recent years, with the rapid development of artificial intelligence, traditional NER methods have been unable to meet the needs, and NER methods based on deep learning have become a research hotspot. This type of method does not require manual feature selection, which makes up for the shortcomings of traditional methods. First, the text sequence is vectorized through the BERT model, and then the neural network is used to automatically extract features. After the neural network encodes the text sequence, the predicted label sequence is finally output through a decoder such as CRF.

[0052] Aiming at the task of Chinese fine-grained entity recognition, the present invention proposes a named entity recognition model based on a hybrid neural network. The model combines the advantages of pre-trained models, deep neural networks, residual connection mechanisms and conditional random fields. The model adopts a three-layer module structure. In the embedding representation layer, the AlBERT pre-trained model is used to encode the text sequence. In the dual-channel encoding layer, on the one hand, a combination of two BiLSTM networks is used to hierarchically process the encoded information output by the previous layer to capture the global semantic information of the text; on the other hand, three convolution kernels of different sizes are set to extract features, and then the local features of the text are obtained by maximum pooling. The feature vectors of the two channels are added and input into the label decoding layer. In the label decoding layer, the CRF model is used to decode the encoded information to obtain the global optimal sequence, thereby ensuring the effect of fine-grained named entity recognition.

[0053] The technical solutions provided by various embodiments of the present application are described in detail below in conjunction with the accompanying drawings.

[0054] Figure 1 The following is a flowchart of a method for named entity recognition in this specification, which specifically includes the following steps:

[0055] S101: Acquire text data as sample text, and annotate real named entities in the sample text.

[0056] Generally, when performing text named entity recognition, the server of the business platform may first train a pre-adopted model based on sample text to execute the business through the trained model.

[0057] Based on this, in one or more embodiments of this specification, the server may first obtain text data as sample text. The specific sample text may be determined as needed. It may be data collected by the business platform during the execution of the business, or it may be sample text obtained from a third-party platform. This specification does not limit this. Then, the real named entities in the sample text may be annotated to assist in the subsequent model training process.

[0058] The server mentioned in this specification can be a server set up on a business platform, or a device such as a desktop computer, a laptop computer, etc. that can execute the solution of this specification. For the convenience of description, the following description is based on the server as the execution subject.

[0059] S102: Use the ALBERT model as the embedding representation layer of text data, use the parallel DouBiLSTM model and multi-window CNN model to form a dual-channel encoding layer and connect it in series after the embedding representation layer, and then connect it in series with the label decoding layer formed based on the conditional random field to build a combined model.

[0060] S103: Input the sample text into the combined model to obtain predicted named entities; and train the combined model according to the real named entities and the predicted named entities to obtain a named entity recognition model.

[0061] S104: Obtain the target text to be subjected to named entity recognition and input it into the named entity recognition model, generate a dynamic representation vector corresponding to the target text through the ALBERT model; extract the global semantic features of the target text according to the dynamic representation vector through the DouBiLSTM model, and extract the multi-granularity local semantic features of the target text according to the dynamic representation vector through the multi-window CNN model; concatenate the global semantic features and the multi-granularity local semantic features through the label decoding layer, and map the concatenated semantic features to the classification label space to obtain prediction information, so as to decode the prediction information through the conditional random field to obtain the named entity.

[0062] This paper focuses on the Chinese fine-grained named entity recognition task and proposes an ALBERT-ThCNN-DouBiLSTM-CRF fine-grained named entity recognition model. The overall architecture of the model is as follows: Figure 2 As shown, Figure 2This is a schematic diagram of a model framework constructed in this specification. As can be seen from the figure, the model consists of three parts: an embedding representation layer, a dual-channel encoding layer, and a label decoding layer. First, the embedding representation layer uses the ALBERT model to generate a dynamic representation vector for the text, which effectively solves the problem of polysemy in Chinese. The vector sequence is then input into the dual-channel encoding layer for feature extraction. The dual-channel encoding layer contains two sub-units, DouBiLSTM and ThCNN, which are responsible for extracting the global semantic features and multi-granularity local features of the text, respectively. Finally, the semantic features extracted by the DouBiLSTM unit and the ThCNN unit are spliced ​​and then input into the conditional random field for decoding to obtain the optimal sequence.

[0063] Among them, for the embedding representation layer, although the BERT model solves the problem that static word vectors cannot be dynamically adjusted according to the context, the BERT model has the problem of large parameters and long training time. ALBERT optimizes it, and ALBERT can be understood as a simplified version of BERT. The ALBERT model mainly improves it in the following three aspects:

[0064] (1) Embedding Matrix Decomposition

[0065] In the BERT model, the dimensions of the embedding layer and the hidden layer are always consistent, which means that the embedding layer dimension will increase as the hidden layer dimension increases, resulting in too large an embedding matrix parameter. The ALBERT model optimizes this by decomposing the embedding layer and the hidden layer, first projecting the vector into a low-dimensional space E, and then projecting it into a high-dimensional space H. Finally, the number of parameters in the embedding matrix changes from O(V×H) to O(V×E+E×H). When the hidden layer dimension H is much larger than the embedding layer dimension E, the model parameters will be greatly reduced.

[0066] (2) Cross-layer parameter sharing

[0067] Both ALBERT and BERT use multi-layer Transformer encoders, but parameters are not shared in BERT, which results in an increase in model parameters as the number of layers increases. ALBERT shares parameters, using only one layer of parameters to represent the original multi-layer parameters, which is the main reason for the reduction in model parameters compared to BERT. After adding parameter sharing, the cosine similarity curve between ALBERT's input embedding and output embedding is also smoother, indicating that the introduction of parameter sharing has also improved the stability of the model.

[0068] (3) Sentence Order Prediction

[0069] BERT's next sentence prediction task (NSP) includes two tasks: topic prediction and coherence prediction. Topic prediction is relatively simple and often overshadows coherence prediction, resulting in NSP being unable to judge the relevance between sentences. ALBERT focuses on the sentence order prediction (Sentence Order Prediction, SOP) task. Its positive examples are the same as BERT, that is, two sentences in the correct order, and the negative examples are sentences in the opposite order. If "Xiao Ming went to the supermarket" and "bought some fruits" are positive examples, then "bought some fruits" and "Xiao Ming went to the supermarket" are negative examples. The ALBERT model using the SOP task has achieved better results in a variety of downstream tasks.

[0070] The ALBERT model is used to encode characters to generate dynamic word vectors. For a given character, the ALBERT model concatenates its TokenEmbeddings, SegmentEmbeddings, and PositionEmbeddings as input, and then performs deep bidirectional training on the sentence through the Transformer encoder to generate context-based character embeddings. For a sentence of length n S = {S1, S2, ..., S n}, after deep characterization using the ALBERT model, a dynamic word vector matrix is ​​formed in is the word vector of the ith character, d c represents the word vector dimension. The working process of the embedding representation layer is as follows Figure 3 As shown, Figure 3 This is a schematic diagram of an embedded representation process in this specification.

[0071] For the dual-channel encoding layer, the dual-channel encoding layer uses DouBiLSTM units and ThCNN units to perceive the global features and local features of the text sequence respectively. The specific structure is as follows Figure 1 As shown in the dual-channel encoding layer.

[0072] 1. Global feature perception unit

[0073] Generally, Chinese text contains a large number of contextual features, which are important for identifying various named entities in the text. In order to fully extract contextual features and enhance the global feature extraction capability of the model, two BiLSTM networks are used to extract contextual features in layers. The BiLSTM network contains two LSTM networks in opposite directions. After forward and reverse encoding of the text sequence, the hidden state sequence H is obtained. x and H y , H x and H y The final output H is obtained by splicing, and the calculation formula is shown as follows:

[0074] H=H x +H y

[0075] The global feature perception unit adopts a residual connection mechanism, which can effectively alleviate the gradient vanishing problem while improving the model performance. The hidden state sequences of the two BiLSTM networks are merged point by point. That is, the dimensions after merging do not change, only the weight values ​​are changed. Its basic structure is as follows Figure 4 As shown, Figure 4 This is a basic structure diagram of a global feature perception unit in this manual. In order to prevent overfitting, a Dropout layer is added after each BiLSTM network. As the input of the global feature perception unit, if H() is used to represent the output of each BiLSTM network, the final output G of the global feature perception unit is as follows:

[0076] G=H(V)+H(H(V))

[0077] The final global features represents the output result of the i-th character after passing through the global perception unit, d h Represents the hidden layer dimension of BiLSTM.

[0078] 2. Multi-window CNN unit

[0079] In Chinese named entity recognition, if we only focus on the global features of the entire sentence and ignore the important information in the local context, it will also lead to recognition errors. For example, in the sentence "Xiaomi Group recently released ten new products", the suffix "Group" often appears in named entities such as organizations, which can help the model to recognize the company entity "Xiaomi", so the ability to extract local features is also very important for NER tasks.

[0080] To extract multi-granularity local features, a 3×d c , 5×d c ,7×d c Three sizes of convolution kernels are used for feature extraction, and dc is the word vector dimension. When convolving text, Samepadding is used to fill the text matrix with zero vectors, so that after convolution kernels of different sizes are convolved on the text sequence, the sentence length is consistent with the input sentence. The input of the multi-window CNN unit is the same as the global feature perception unit, which is also a dynamic word vector matrix. Use convolution kernels of different sizes to convolve the i-th character, and use the ReLU activation function for activation to accelerate the convergence speed of the model, as shown in the following formula.

[0081]

[0082] in, represents the multi-granularity local features of the i-th character, k represents the window size of the convolution kernel, Wc represents the weight matrix parameter of the convolution kernel, and b c represents the bias vector of the convolution kernel, Indicates Characters to The input word vector sequence of characters, ReLU() represents the activation function.

[0083] After the input word vector matrix V is convolved, a multi-granularity local feature matrix can be obtained As shown below:

[0084]

[0085] In order to filter out the most important local features from the multi-granularity local features and reduce the number of model parameters, maximum pooling is used to filter the multi-granularity local features. First, for the local features C of different granularities, k , take the maximum value of the corresponding dimension of each character to get x i，j ; Then we can get the local features of each character Finally, the n characters are concatenated to get the output of the ThCNN unit. Figure 5 This is a schematic diagram of the maximum pooling layer structure in this specification, and the specific calculation formula is shown as follows:

[0086]

[0087]

[0088] X={x1,x2,…,x i , …, x n}

[0089] Among them, x i，j Represents the feature value of the jth dimension of the i-th character.

[0090] For the label decoding layer, the label decoding layer is the last layer of the model and is responsible for accepting the global features generated by DouBiLSTM. Multi-granularity local features generated by ThCNN And concatenate the global feature G and the local feature X to get As shown below:

[0091]

[0092] Then, the concatenated semantic features M are mapped to y classification label spaces using a fully connected layer to obtain the prediction information N, as shown in the following formula:

[0093] N=MA c +b c

[0094] Where N∈R n×y , n is the sentence length, y is the number of tags in the tag set, is the feature weight matrix of the fully connected layer, b c is the bias vector.

[0095] Finally, CRF is used to decode the predicted information N to obtain the optimal sequence and the final entity recognition result. CRF can effectively consider the relationship between adjacent labels and learn constraint rules during training, thereby filtering out illegal label sequences and calculating the global optimal label sequence.

[0096] For the prediction information N, there may be a predicted named entity sequence Y = {Y1,…,Y i ,…,Y n}, its score function is as follows:

[0097] In the formula, Y represents the predicted named entity sequence, Sorce(N,Y) represents the score of the predicted named entity sequence based on the prediction information, Represents the predicted named entity Y i To predict named entity Y i+1 The transition probability score, Indicates that the predicted named entity is Y i The softmax output score of the i-th character.

[0098] The scores of the predicted named entity sequences can be used to calculate the conditional probability of the predicted named entity sequence:

[0099] In the formula, Represents any possible predicted named entity sequence corresponding to the predicted information, and finally uses the Viterbi algorithm to decode to obtain the output sequence Y with the highest score * :

[0100] Where Y * Represents the highest-scoring named entity sequence.

[0101] based on Figure 1The method for named entity recognition shown in the invention first uses the ALBERT model as the embedding representation layer of text data, forms a dual-channel encoding layer with a parallel DouBiLSTM model and a multi-window CNN model and connects it in series after the embedding representation layer, and then connects the label decoding layer formed based on the conditional random field in series, constructs a combined model, and trains named entity recognition through sample text to obtain a named entity recognition model, and finally, for the target text to be recognized, generates a dynamic representation vector of the target text through the ALBERT model of the named entity recognition model, and extracts the global semantic features and multi-granularity local semantic features of the target text through the dual-channel encoding layer, and then decodes the named entity through the label decoding layer. The invention fully combines the advantages of the BiLSTM model and the CNN model to fully extract the global semantic features and local semantic features of the target text, and improves the recognition effect of fine-grained named entities.

[0102] The present invention constructs an ALBERT-ThCNN-DouBiLSTM-CRF fine-grained named entity recognition model. The model gives full play to the advantages of both CNN and BiLSTM, and adopts a feature encoding layer with a dual-channel structure to capture the semantic features of different types of entities in the text, thereby improving the recognition effect of fine-grained named entities.

[0103] Among them, the global feature perception unit adopts the idea of ​​residual connection, which improves the global feature extraction capability of the model while alleviating the gradient vanishing problem; a multi-window CNN unit is constructed, and convolution kernels of different sizes are used to obtain local features of different granularities. The local features are integrated and screened through maximum pooling, which improves the local feature extraction capability of the model.

[0104] The model performance was verified on multiple datasets. The experimental results show that the model has better recognition effect in Chinese fine-grained named entity recognition. The F1 values ​​reached 87.64% and 91.47% on the CLUENER2020 dataset containing 10 entity types and the cMedQANER dataset containing 11 entity types, respectively.

[0105] When applying the named entity recognition method provided in this specification, it is not necessary to Figure 1 The steps are executed in the order shown. The specific execution order of the steps can be determined according to needs, and this manual does not limit this.

[0106] The above is a named entity recognition method provided by one or more embodiments of this specification. Based on the same idea, this specification also provides a corresponding named entity recognition device, such as Figure 6 shown.

[0107] Figure 6 A schematic diagram of a named entity recognition device provided in this specification includes:

[0108] An acquisition module 201 is used to acquire text data as sample text and mark real named entities in the sample text;

[0109] A construction module 202 is used to use the ALBERT model as the embedding representation layer of the text data, form a dual-channel encoding layer with a parallel DouBiLSTM model and a multi-window CNN model and connect them in series after the embedding representation layer, and then connect them in series with a label decoding layer formed based on a conditional random field to construct a combined model;

[0110] The training module 203 is used to input the sample text into the combined model to obtain the predicted named entities; and train the combined model according to the real named entities and the predicted named entities to obtain a named entity recognition model;

[0111] The recognition module 204 is used to obtain the target text to be subjected to named entity recognition and input it into the named entity recognition model, generate a dynamic representation vector corresponding to the target text through the ALBERT model; extract the global semantic features of the target text according to the dynamic representation vector through the DouBiLSTM model, and extract the multi-granularity local semantic features of the target text according to the dynamic representation vector through the multi-window CNN model; concatenate the global semantic features and the multi-granularity local semantic features through the label decoding layer, and map the concatenated semantic features to the classification label space to obtain prediction information, so as to decode the prediction information through the conditional random field to obtain the named entity.

[0112] For the specific definition of the named entity recognition device, please refer to the definition of the named entity recognition method above, which will not be repeated here. The various modules in the above-mentioned named entity recognition device can be implemented in whole or in part by software, hardware and a combination thereof. The above-mentioned modules can be embedded in or independent of the processor in the computer device in the form of hardware, or can be stored in the memory of the computer device in the form of software, so that the processor can call and execute the operations corresponding to the above modules.

[0113] This specification also provides a computer-readable storage medium, which stores a computer program, which can be used to execute the above Figure 1 Provided named entity recognition methods.

[0114] This manual also provides Figure 7 The structural diagram of the computer device shown in FIG. Figure 7 As shown in the figure, at the hardware level, the computer device includes a processor, an internal bus, a network interface, a memory, and a non-volatile memory, and may also include other hardware required for the business. The processor reads the corresponding computer program from the non-volatile memory into the memory and then runs it to achieve the above Figure 1 Provided named entity recognition methods.

[0115] Those of ordinary skill in the art can understand that all or part of the processes in the above-mentioned embodiment methods can be completed by instructing the relevant hardware through a computer program, and the computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, any reference to memory, storage, database or other media used in the embodiments provided in this application can include at least one of non-volatile and volatile memory. Non-volatile memory may include read-only memory (ROM), magnetic tape, floppy disk, flash memory or optical memory, etc. Volatile memory may include random access memory (RAM) or external cache memory. As an illustration and not limitation, RAM can be in various forms, such as static random access memory (SRAM) or dynamic random access memory (DRAM).

[0116] The technical features of the above embodiments may be combined arbitrarily. To make the description concise, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

Claims

1. A method for named entity recognition, characterized in that: include: Acquire text data as sample text, and annotate real named entities in the sample text; The ALBERT model is used as the embedding representation layer of text data. The parallel DouBiLSTM model and multi-window CNN model form a dual-channel encoding layer and are connected in series to the embedding representation layer. The label decoding layer formed based on the conditional random field is then connected in series to construct a combined model. Input the sample text into the combined model to obtain predicted named entities; and train the combined model based on the real named entities and the predicted named entities to obtain a named entity recognition model; The target text to be recognized by named entity is obtained and input into the named entity recognition model, and the dynamic representation vector corresponding to the target text is generated by the ALBERT model; the global semantic features of the target text are extracted according to the dynamic representation vector by the DouBiLSTM model, and the multi-granularity local semantic features of the target text are extracted according to the dynamic representation vector by the multi-window CNN model; the global semantic features and the multi-granularity local semantic features are spliced ​​through the label decoding layer, and the spliced ​​semantic features are mapped to the classification label space to obtain prediction information, so as to decode the prediction information through the conditional random field to obtain the named entity.

2. The method according to claim 1, characterized in that The step of generating a dynamic representation vector corresponding to the target text through the ALBERT model specifically includes: The target text is segmented by the word segmenter of the ALBERT model to obtain multiple subwords; each subword is mapped to the corresponding word vector through the TokenEmbeddings module of the ALBERT model; The target text is segmented into sentences through the SegmentEmbeddings module of the ALBERT model, and sentence embedding representations are generated for each sentence. Generate position embedding representation for each character position in the target text through the PositionEmbeddings module of the ALBERT model; A comprehensive embedding representation of the target text is obtained based on the word vector, sentence embedding representation, and position embedding representation, and is input into the Transformer encoder of the ALBERT model to obtain a context-based dynamic representation vector of the target text.

3. The method for named entity recognition according to claim 1, wherein: The method of extracting the global semantic features of the target text according to the dynamic representation vector by using the DouBiLSTM model specifically includes: The global semantic features of the target text are extracted based on the dynamic representation vector of the target text based on the DouBiLSTM model through the following formula: G = H(V) + H(H(V)); Among them, V represents the dynamic representation vector of the target text, H() represents the output of the BiLSTM network, and G represents the global semantic features of the target text.

4. The method for named entity recognition according to claim 1, wherein: The method of extracting multi-granularity local semantic features of the target text according to the dynamic representation vector by using the multi-window CNN model specifically includes: The dynamic representation vector corresponding to each character in the target text is convolved based on the convolution kernels of different sizes in the multi-window CNN model through the following formula, and activated with the ReLU activation function to extract the multi-granularity local semantic features of the target text: Among them, C k Represents the multi-granular local semantic features of the target text, represents the multi-granularity local features of the i-th character, k represents the window size of the convolution kernel, and W c represents the weight matrix parameters of the convolution kernel, b c represents the bias vector of the convolution kernel, Indicates Characters to The input word vector sequence of characters, ReLU() represents the activation function.

5. The method for named entity recognition according to claim 1, wherein: The step of mapping the spliced ​​semantic features obtained by splicing to the classification label space to obtain prediction information, and decoding the prediction information through the conditional random field to obtain the named entity, specifically includes: The concatenated semantic features are mapped to multiple classification spaces based on the fully connected layer in the label decoding layer through the following formula to obtain prediction information: N=MA c +b c ; The predicted information is decoded and recognized based on the conditional random field through the following formula: Among them, N represents the prediction information, M represents the spliced ​​semantic features, and A c represents the feature weight matrix of the fully connected layer, b c represents the bias vector, Y represents the predicted named entity sequence, Sorce(N,Y) represents the score of the predicted named entity sequence based on the prediction information, Represents the predicted named entity Y i To predict named entity Y i+1 The transition probability score, Indicates that the predicted named entity is Y i The softmax output score of the i-th character, Y * Represents the highest scoring named entity sequence.

6. A named entity recognition device, characterized in that: include: An acquisition module, used for acquiring text data as sample text and marking real named entities in the sample text; A construction module is used to use the ALBERT model as the embedding representation layer of text data, form a dual-channel encoding layer with a parallel DouBiLSTM model and a multi-window CNN model and connect them in series after the embedding representation layer, and then connect them in series with a label decoding layer formed based on a conditional random field to build a combined model; A training module is used to input sample text into the combined model to obtain predicted named entities; and train the combined model according to the real named entities and the predicted named entities to obtain a named entity recognition model; The recognition module is used to obtain the target text to be recognized by named entity and input it into the named entity recognition model, generate the dynamic representation vector corresponding to the target text through the ALBERT model; extract the global semantic features of the target text according to the dynamic representation vector through the DouBiLSTM model, and extract the multi-granularity local semantic features of the target text according to the dynamic representation vector through the multi-window CNN model; splice the global semantic features and the multi-granularity local semantic features through the label decoding layer, and map the spliced ​​semantic features to the classification label space to obtain prediction information, so as to decode the prediction information through the conditional random field to obtain the named entity.

7. A computer-readable storage medium, characterized in that: The storage medium stores a computer program, and when the computer program is executed by a processor, the method according to any one of claims 1 to 5 is implemented.

8. A computer device, characterized in that: The method comprises a memory, a processor and a computer program stored in the memory and executable on the processor, wherein when the processor executes the program, the method described in any one of claims 1 to 5 is implemented.

Images