A named entity recognition method, apparatus, medium, and device

Abstract

The application discloses a named entity recognition method and device, a medium and equipment, and relates to the technical field of natural language processing. The application takes an ALBERT model as an embedding representation layer of text data, forms a double-channel coding layer by using a parallel DouBiLSTM model and a multi-window CNN model, and is connected in series after the embedding representation layer, then a label decoding layer based on a conditional random field is connected in series, a combined model is constructed, named entity recognition is trained through sample text, a named entity recognition model is obtained, finally, for a target text to be recognized, a dynamic representation vector of the target text is generated by the ALBERT model of the named entity recognition model, global semantic features and multi-granularity local semantic features of the target text are extracted through the double-channel coding layer, and then the named entity is decoded through the label decoding layer. The application fully combines the advantages of the BiLSTM model and the CNN model, and improves the recognition effect of fine-grained named entities.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of natural language processing, and in particular to a named entity recognition method, device, medium and equipment. BACKGROUND

[0002] At present, named entity recognition (NER) is an important task in the field of natural language processing, which aims to identify entities with specific meanings from text, such as names, place names, organization names, etc. With the rapid development of information technology, named entity recognition plays an increasingly important role in information extraction, question and answer systems, machine translation and many other application scenarios. The development of English named entity recognition is earlier and has achieved good results, but the recognition of Chinese named entities is more complex, mainly because compared with English, Chinese vocabulary has more meanings and sentence patterns are more flexible and variable. The effect of Chinese named entity recognition is greatly affected by the segmentation result, and the phenomenon of polysemy is very common in Chinese, which brings more challenges to Chinese named entity recognition. Therefore, due to the complexity of Chinese language structure and the polysemy of vocabulary, higher granularity is required in Chinese named entity recognition.

[0003] Compared with traditional entity recognition, the following two characteristics are mainly used to deal with fine-grained named entity recognition in Chinese: first, there are more entity categories, such as possible categories including: book name, scenic spot, animal, plant, etc., and new categories will continue to appear over time; second, fine-grained entity categories have more hierarchical sense, for example, traditional place names can be further divided into city names, university names, community names, etc., and for example, hospitals can be divided into general hospitals, specialized hospitals, teaching hospitals, etc. This fine-grained entity division provides more semantic information and can provide more possibilities for downstream tasks in the field of natural language processing.

[0004] However, the traditional named entity recognition method often has poor effect in fine-grained entity recognition. SUMMARY

[0005] Therefore, it is necessary to provide a named entity recognition method, device, medium and equipment in view of the above technical problems.

[0006] The present application adopts the following technical solutions:

[0007] The present application provides a named entity recognition method, comprising:

[0008] Obtaining text data as a sample text, and labeling real named entities in the sample text;

[0009] An ALBERT model is used as an embedding representation layer of text data, a DouBiLSTM model and a multi-window CNN model are used to form a double-channel encoding layer and are connected in series after the embedding representation layer, and a label decoding layer based on a conditional random field is connected in series to construct a combined model;

[0010] The sample text is input into the combined model to obtain a predicted named entity, and the combined model is trained according to a real named entity and the predicted named entity to obtain a named entity recognition model;

[0011] The target text to be subjected to named entity recognition is input into the named entity recognition model, a dynamic representation vector corresponding to the target text is generated by the ALBERT model, global semantic features of the target text are extracted from the dynamic representation vector by the DouBiLSTM model, and multi-granularity local semantic features of the target text are extracted from the dynamic representation vector by the multi-window CNN model, the global semantic features and the multi-granularity local semantic features are spliced by the label decoding layer, the spliced semantic features are mapped to a classification label space to obtain prediction information, and the named entity is obtained by decoding the prediction information by the conditional random field.

[0012] Optionally, the dynamic representation vector corresponding to the target text is generated by the ALBERT model, and specifically includes:

[0013] The target text is subjected to tokenization processing by a tokenizer of the ALBERT model to obtain a plurality of subwords, and each subword is mapped to a corresponding word vector by a TokenEmbeddings module of the ALBERT model;

[0014] Each sentence of the target text is subjected to sentence processing by a SegmentEmbeddings module of the ALBERT model to generate sentence embedding representation for each sentence;

[0015] Each character position in the target text is subjected to position embedding representation processing by a PositionEmbeddings module of the ALBERT model;

[0016] The comprehensive embedding representation of the target text is obtained according to the word vector, the sentence embedding representation and the position embedding representation, and is input into a Transformer encoder of the ALBERT model to obtain a dynamic representation vector of the target text based on context.

[0017] Optionally, the global semantic features of the target text are extracted from the dynamic representation vector by the DouBiLSTM model, and specifically includes:

[0018] The global semantic features of the target text are extracted from the dynamic representation vector of the target text by the DouBiLSTM model based on the following formula:

[0019] G = H(V) + H(H(V));

[0020] Wherein, V represents the dynamic representation vector of the target text, H() represents the output of the BiLSTM network, and G represents the global semantic feature of the target text.

[0021] Optionally, the multi-granularity local semantic features of the target text are extracted from the dynamic representation vector by the multi-window CNN model, and specifically include:

[0022] The dynamic representation vector corresponding to each character in the target text is convolved based on different size convolution kernels in the multi-window CNN model, and a ReLU activation function is used for activation, so as to extract the multi-granularity local semantic features of the target text:

[0023]

[0024] Wherein, C k represents the multi-granularity local semantic features of the target text, represents the multi-granularity local features of the i-th character, k represents the window size of the convolution kernel, W c represents the weight matrix parameter of the convolution kernel, b c represents the bias vector of the convolution kernel, represents the input word vector sequence from the i-th character to the j-th character, and ReLU() represents the activation function.

[0025] Optionally, the spliced semantic features obtained by splicing are mapped to a classification label space to obtain prediction information, so as to decode the prediction information by a conditional random field to obtain a named entity, and specifically include:

[0026] The spliced semantic features obtained by splicing are mapped to a plurality of classification spaces to obtain prediction information based on the full connection layer in the label decoding layer by the following formula:

[0027] N = MA c +b c ;

[0028] The prediction information is decoded and recognized by a conditional random field by the following formula:

[0029]

[0030]

[0031] Wherein, N represents the prediction information, M represents the spliced semantic features obtained by splicing, A c represents the feature weight matrix of the full connection layer, and b c ​​denotes a bias vector, Y denotes a predicted named entity sequence, Sorce(N, Y) denotes a score of the predicted named entity sequence based on prediction information, denotes a transition probability score from the predicted named entity Y i to the predicted named entity Y i+1 , denotes a softmax output score of the i-th character of the predicted named entity Y i , * denotes a highest-scored named entity sequence.

[0032] The specification provides a named entity recognition device, comprising:

[0033] An acquisition module is configured to acquire text data as sample text and label real named entities in the sample text;

[0034] A construction module is configured to use an ALBERT model as an embedding representation layer of the text data, use a parallel DouBiLSTM model and a multi-window CNN model to form a double-channel encoding layer and connect the double-channel encoding layer after the embedding representation layer, and then connect a label decoding layer formed based on a conditional random field to construct a combined model;

[0035] A training module is configured to input the sample text into the combined model to obtain predicted named entities, and train the combined model according to the real named entities and the predicted named entities to obtain a named entity recognition model;

[0036] An identification module is configured to input a target text to be subjected to named entity recognition into the named entity recognition model, generate a dynamic representation vector corresponding to the target text through the ALBERT model, extract global semantic features of the target text according to the dynamic representation vector through the DouBiLSTM model, and extract multi-granularity local semantic features of the target text according to the dynamic representation vector through the multi-window CNN model, splice the global semantic features and the multi-granularity local semantic features through the label decoding layer, map spliced semantic features obtained by splicing to a classification label space to obtain prediction information, and decode the prediction information through the conditional random field to obtain named entities.

[0037] The specification provides a computer-readable storage medium, which stores a computer program, and the computer program is executed by a processor to implement the named entity recognition method.

[0038] The specification provides a computer device, which comprises a memory, a processor, and a computer program stored in the memory and executable on the processor, and the processor implements the named entity recognition method when executing the program.

[0039] The above at least one technical solution adopted by the specification can achieve the following beneficial effects:

[0040] The application takes the ALBERT model as an embedding representation layer of text data, forms a double-channel encoding layer by a parallel DouBiLSTM model and a multi-window CNN model, and is connected in series after the embedding representation layer, and then a label decoding layer based on a conditional random field is connected in series to construct a combined model; a named entity recognition model is obtained by training the named entity recognition of sample text; finally, for the target text to be recognized, the ALBERT model of the named entity recognition model generates the dynamic representation vector of the target text, and the double-channel encoding layer extracts the global semantic features and multi-granularity local semantic features of the target text, and then the label decoding layer is decoded to obtain the named entity.

[0041] The application fully combines the advantages of the BiLSTM model and the CNN model to fully extract the global semantic features and local semantic features of the target text, and improves the recognition effect of the fine-grained named entity. BRIEF DESCRIPTION OF DRAWINGS

[0042] The accompanying drawings, which are included to provide a further understanding of the application and are incorporated in and constitute a part of this application, illustrate embodiments of the application and serve to explain the principles of the application, and do not limit the application in any way. In the drawings:

[0043] Figure 1 A named entity recognition method flowchart is provided for the present specification;

[0044] Figure 2 A model framework obtained is provided for the present specification;

[0045] Figure 3 An embedding representation flowchart is provided for the present specification;

[0046] Figure 4 A global feature perception unit basic structure diagram is provided for the present specification;

[0047] Figure 5 A maximum pooling layer structure diagram is provided for the present specification;

[0048] Figure 6 A named entity recognition device diagram is provided for the present specification;

[0049] Figure 7 A computer device for implementing the named entity recognition method is provided for the present specification. DETAILED DESCRIPTION

[0050] In order to make the objects, technical solutions and advantages of the present application clearer, the technical solutions of the present application will be described below in connection with specific embodiments of the present application and corresponding drawings. Obviously, the described embodiments are only some of the embodiments of the present application, not all. Based on the embodiments in the present application, all other embodiments obtained by those of ordinary skill in the art without creative labor fall within the scope of protection of the present application.

[0051] Currently, traditional named entity recognition methods are mainly based on rules and statistical machine learning. Although these traditional methods are easier to explain, their recognition effect depends largely on the modeling ability and rules for specific tasks. In particular, the task of fine-grained NER puts higher requirements on the modeling ability of traditional methods. In recent years, with the rapid development of artificial intelligence, the traditional method of NER has been difficult to meet the demand, and the NER method based on deep learning has become a research hotspot. This kind of method does not need to select artificial features, which makes up for the shortcomings of traditional methods. First, the text sequence is represented by a vector through a BERT model, and then the neural network automatically extracts features. After the neural network encodes the text sequence, the final predicted label sequence is output through a decoder such as CRF.

[0052] For the task of Chinese fine-grained entity recognition, the present application proposes a named entity recognition model based on a hybrid neural network. The model combines the advantages of pre-trained models, deep neural networks, residual connection mechanisms and conditional random fields. The model uses a three-layer module structure. In the embedding representation layer, the AlBERT pre-trained model is used to encode the text sequence. In the double-channel encoding layer, on the one hand, the combination of two BiLSTM networks is used to process the encoding information output by the previous layer in layers to capture the global semantic information of the text. On the other hand, three different size convolution kernels are set to extract features, and then the local features of the text are obtained through maximum pooling. The feature vectors of the two channels are added and input into the label decoding layer. In the label decoding layer, the CRF model is used to decode the encoded information to obtain the globally optimal sequence, thereby ensuring the effect of fine-grained named entity recognition.

[0053] The technical solutions provided by the embodiments of the present application will be described in detail below in connection with the drawings.

[0054] Figure 1 For the named entity recognition method flowchart in the present specification, the specific steps include the following steps:

[0055] S101: Obtain text data as sample text, and label the real named entity in the sample text.

[0056] Generally, when performing text named entity recognition, the server of the business platform can first train a pre-adopted model based on sample text, so as to execute the business by the trained model.

[0057] Based on this, in one or more embodiments of the present specification, the server can first obtain text data as sample text, and the specific sample text can be determined as needed, which can be data collected by the business platform during the execution of the business, or sample text obtained from a third-party platform, and the present specification does not limit this. Then, the real named entity in the sample text can be labeled to assist the subsequent model training process.

[0058] The server mentioned in the present specification can be a server arranged in a business platform, or a device such as a desktop computer, a notebook computer, etc. capable of executing the scheme of the present specification. For the convenience of description, the following will be described only by taking the server as the execution subject.

[0059] S102: The ALBERT model is used as an embedding representation layer of the text data, a DouBiLSTM model and a multi-window CNN model are used to form a double-channel encoding layer and are connected in series after the embedding representation layer, and a label decoding layer based on a conditional random field is connected in series to construct a combined model.

[0060] S103: The sample text is input into the combined model to obtain a predicted named entity, and the combined model is trained according to the real named entity and the predicted named entity to obtain a named entity recognition model.

[0061] S104: A target text to be subjected to named entity recognition is input into the named entity recognition model, a dynamic representation vector corresponding to the target text is generated by the ALBERT model, global semantic features of the target text are extracted from the dynamic representation vector by the DouBiLSTM model, and multi-granularity local semantic features of the target text are extracted from the dynamic representation vector by the multi-window CNN model, the global semantic features and the multi-granularity local semantic features are spliced by the label decoding layer, and the spliced semantic features are mapped to a classification label space to obtain prediction information, so that the named entity is obtained by decoding the prediction information by the conditional random field.

[0062] The present application focuses on the task of Chinese fine-grained named entity recognition, and proposes an ALBERT-ThCNN-DouBiLSTM-CRF fine-grained named entity recognition model. The overall architecture of the model is as shown in Figure 2 Figure 2 ​A model framework constructed in the specification is shown in the figure. As can be seen from the figure, the model is composed of three parts of embedding representation layer, double-channel encoding layer and label decoding layer. First, the embedding representation layer uses the ALBERT model to generate dynamic representation vectors of the text, effectively solving the problem of Chinese polysemy. Then the vector sequence is input to the double-channel encoding layer for feature extraction. The double-channel encoding layer includes DouBiLSTM and ThCNN two subunits, which are responsible for extracting the global semantic features and multi-granularity local features of the text respectively. Finally, the semantic features extracted by the DouBiLSTM unit and the ThCNN unit are spliced and input into the conditional random field to decode the optimal sequence.

[0063] Among them, for the embedding representation layer, although the BERT model solves the problem that the static word vector cannot be dynamically adjusted according to the context, the BERT model has the problems of large parameters and long training time. ALBERT optimizes it, and ALBERT can be understood as a simplified version of BERT. The ALBERT model mainly improves in the following three aspects:

[0064] (1) Embedding matrix decomposition

[0065] In the BERT model, the dimensions of the embedding layer and the hidden layer are always consistent, which makes the dimension of the embedding layer increase with the increase of the dimension of the hidden layer, resulting in too large embedding matrix parameters. The ALBERT model optimizes it by decomposing the embedding layer and the hidden layer, first projecting the vector to a low-dimensional space E, and then projecting it to a high-dimensional space H. Finally, the parameter quantity of the embedding matrix is changed from O(VxH) to O(VxE+ExH). When the dimension of the hidden layer H is much larger than the dimension of the embedding layer E, the model parameters will be greatly reduced.

[0066] (2) Cross-layer parameter sharing

[0067] ALBERT and BERT both use multi-layer Transformer encoder, but the parameters in BERT are not shared, which leads to an increase in model parameters when the number of layers increases. ALBERT shares parameters, and the parameter quantity of one layer can represent the parameter quantity of the original multiple layers, which is the main reason for the reduction of the parameters of the BERT model. After adding parameter sharing, the cosine similarity curve between the input embedding and the output embedding of ALBERT is also smoother, indicating that the stability of the model has also been improved after introducing parameter sharing.

[0068] (3) Sentence order prediction

[0069] The next sentence prediction task (NSP) of BERT includes two tasks of topic prediction and coherence prediction, the topic prediction is relatively simple, and often masks the coherence prediction, resulting in that the NSP cannot judge the relevance between sentences. ALBERT focuses on the sentence order prediction (SOP) task, and its positive examples and BERT are the same, that is, two sentences with correct order, and the negative examples are the sentences with opposite order. If "Xiaoming went to the supermarket" and "bought some fruits" are positive examples, then "bought some fruits" and "Xiaoming went to the supermarket" are negative examples. The ALBERT model using the SOP task achieves better results in various downstream tasks.

[0070] The ALBERT model is used to encode characters to generate dynamic word vectors. For a given character, the ALBERT model connects TokenEmbeddings, SegmentEmbeddings and PositionEmbeddings as input, and then performs deep bidirectional training on the sentence through the Transformer encoder to generate context-based character embeddings. For a sentence S = {S1, S2, …, Sn} of length n, after deep representation using the ALBERT model, a dynamic word vector matrix is formed n wherein is the word vector of the i-th character, d c represents the dimension of the word vector. The working process of the embedding representation layer is shown in Figure 3 Figure 3 FIG. 1 is a schematic diagram of an embedding representation process in the present specification.

[0071] For the double-channel encoding layer, the double-channel encoding layer perceives the global features and local features of the text sequence through DouBiLSTM units and ThCNN units respectively. The specific structure is shown in Figure 1

[0072] 1. Global feature perception unit

[0073] Generally, a large amount of context features are contained in Chinese text, and these features play an important role in identifying various named entities in the text. In order to fully extract the context features and enhance the global feature extraction capability of the model, two BiLSTM networks are used to extract context features in layers. The BiLSTM network contains two opposite LSTM networks, and the hidden state sequence H x and H y is obtained after forward and reverse encoding of the text sequence. H x and H y are spliced to obtain the final output H, and the calculation formula is shown in the following formula: ​​​

[0074] H = H x + H y

[0075] The global feature perception unit adopts a residual connection mechanism to improve model performance while effectively alleviating the gradient vanishing problem. The hidden state sequences of the two BiLSTM networks are merged point by point, that is, the dimension does not change after merging, only the weight value changes, and the basic structure is as shown in Figure 4 Figure 4 Figure 1 is a schematic diagram of a basic structure of a global feature perception unit in the present specification. In order to prevent overfitting, a Dropout layer is also added after each BiLSTM network. The dynamic word vector matrix is input into the global feature perception unit, and if H() represents the output of each BiLSTM network, the final output G of the global feature perception unit is as follows:

[0076] G = H(V) + H(H(V))

[0077] The final global feature represents the output result of the i-th character after global perception, d h represents the hidden layer dimension of BiLSTM.

[0078] 2. Multi-window CNN unit

[0079] In Chinese named entity recognition, if only the global features of the entire sentence are focused on, and important information in the local context is ignored, it will also lead to recognition errors. For example, in the sentence "Xiaomi Group recently released ten new products", the suffix "Group" often appears in organizational named entities, which can help the model to identify the "Xiaomi" company entity, so the local feature extraction capability is also very important for the NER task.

[0080] In order to extract multi-granularity local features, three sizes of convolution kernels, 3xd c , 5xd c , and 7xd c , are used for feature extraction, and dc is the word vector dimension. When performing convolution on the text, zero vector padding is used for the text matrix to make the sentence length consistent with the input sentence after convolution of different size convolution kernels on the text sequence. The input of the multi-window CNN unit is the same as that of the global feature perception unit, which is the dynamic word vector matrix The i-th character is convolved using different size convolution kernels, and the ReLU activation function is used for activation operation to speed up the convergence speed of the model, which is as follows. ​

[0081]

[0082] in, Let represent the multi-granularity local features of the i-th character, k represent the window size of the convolution kernel, Wc represent the weight matrix parameters of the convolution kernel, and b represent the multi-granularity local features of the i-th character. c This represents the bias vector of the convolution kernel. Indicates the first characters to The input word vector sequence of n characters, where ReLU() represents the activation function.

[0083] After convolution, the input character vector matrix V yields a multi-granularity local feature matrix. As shown in the following formula:

[0084]

[0085] To filter out the most important local features from multi-granularity local features and reduce the number of model parameters, max pooling is used to filter and select multi-granularity local features. First, for local features C of different granularities... k x is obtained by taking the maximum value of the corresponding dimension for each character. i，j Then the local features of each character can be obtained. Finally, the n characters are concatenated to obtain the output of the ThCNN unit. Figure 5 This is a schematic diagram of one maximum pooling layer structure in this specification. The specific calculation formula is shown below:

[0086]

[0087]

[0088] X = {x1, x2, ..., x} i , ..., x n}

[0089] Where, x i，j This represents the feature value of the j-th dimension of the i-th character.

[0090] The label decoding layer is the last layer of the model and is responsible for receiving the global features generated by DouBiLSTM. Multi-granularity local features generated by ThCNN Then, the global feature G and the local feature X are concatenated to obtain... As shown in the following formula:

[0091]

[0092] Then the full connection layer is used to map the spliced semantic features M to the y classification label space to obtain the prediction information N, as shown in the following formula:

[0093] N = MA c +b c

[0094] Wherein, N ∈ R n×y , n is the length of the sentence, y is the number of labels in the label set, is the feature weight matrix of the full connection layer, b c is the bias vector.

[0095] Finally, the CRF is used to decode the prediction information N to obtain the optimal sequence to obtain the final entity recognition result. The CRF can effectively consider the relationship between adjacent labels, and can learn the constraint rules in the training, so as to filter out the illegal label sequence and calculate the globally optimal label sequence.

[0096] For the prediction information N, there may be a predicted named entity sequence Y = {Y1,…, Y i ,…,Y n}, and its score function is as shown in the following formula:

[0097] In the formula, Y represents the predicted named entity sequence, Sorce(N, Y) represents the score of the predicted named entity sequence based on the prediction information, represents the transition probability score from the predicted named entity Y i to the predicted named entity Y i+1 , represents the softmax output score of the i-th character of the predicted named entity Y i .

[0098] The score of the predicted named entity sequence can be used to calculate the conditional probability of the predicted named entity sequence:

[0099] In the formula, represents any one of the possible predicted named entity sequences corresponding to the prediction information, and finally the Viterbi algorithm is used to decode to obtain the highest score output sequence Y * :

[0100] In the formula, Y * represents the highest score named entity sequence.

[0101] Based on Figure 1The shown named entity recognition method, first, the application takes the ALBERT model as the embedding representation layer of the text data, forms a double-channel coding layer with a parallel DouBiLSTM model and a multi-window CNN model, and then connects the label decoding layer based on the conditional random field after the embedding representation layer, constructs a combined model, and trains the named entity recognition model through sample text for named entity recognition. Finally, for the target text to be recognized, the ALBERT model of the named entity recognition model generates the dynamic representation vector of the target text, and the double-channel coding layer extracts the global semantic features and multi-granularity local semantic features of the target text, and then decodes the label decoding layer to obtain the named entity. The application fully combines the advantages of the BiLSTM model and the CNN model to fully extract the global semantic features and local semantic features of the target text, and improves the recognition effect of the fine-grained named entity.

[0102] The application constructs an ALBERT-ThCNN-DouBiLSTM-CRF fine-grained named entity recognition model, which fully utilizes the advantages of CNN and BiLSTM, adopts a feature coding layer with a double-channel structure to capture the semantic features of different types of entities in the text, and improves the recognition effect of the fine-grained named entity.

[0103] Among them, the global feature perception unit adopts the idea of residual connection, which improves the global feature extraction ability of the model and relieves the gradient vanishing problem; a multi-window CNN unit is constructed, different size convolution kernels are used to obtain local features of different granularities, and the local features are integrated and selected through maximum pooling, which improves the local feature extraction ability of the model.

[0104] The performance of the model is verified on multiple data sets, and the experimental results show that the model has better recognition effect in Chinese fine-grained named entity recognition, and the F1 value on the CLUENER2020 data set containing 10 entity types and the cMedQANER data set containing 11 entity types is 87.64% and 91.47% respectively.

[0105] When applying the named entity recognition method provided in the specification, the order of the steps shown can not be executed according to Figure 1 The order of execution of each step can be determined as needed, and the specification does not limit this.

[0106] The above is the named entity recognition method provided by one or more embodiments of the specification, based on the same idea, the specification also provides a corresponding named entity recognition device, as Figure 6 shown.

[0107] Figure 6 A named entity recognition device provided by the specification is shown in the schematic diagram, which comprises:

[0108] The acquisition module 201 is used to acquire text data as sample text and label the real named entities in the sample text;

[0109] Module 202 is used to construct a combined model by using the ALBERT model as the embedding representation layer for text data, forming a dual-channel encoding layer with parallel DouBiLSTM and multi-window CNN models and concatenating it after the embedding representation layer, and then concatenating it with a label decoding layer based on conditional random fields.

[0110] The training module 203 is used to input sample text into the ensemble model to obtain predicted named entities; and to train the ensemble model based on the real named entities and the predicted named entities to obtain a named entity recognition model.

[0111] The recognition module 204 is used to obtain the target text input for the named entity recognition model, generate the dynamic representation vector corresponding to the target text through the ALBERT model, extract the global semantic features of the target text based on the dynamic representation vector through the DouBiLSTM model, and extract the multi-granularity local semantic features of the target text based on the dynamic representation vector through the multi-window CNN model. The global semantic features and multi-granularity local semantic features are concatenated through the label decoding layer, and the concatenated semantic features are mapped to the classification label space to obtain the prediction information, so as to decode the prediction information through the conditional random field to obtain the named entity.

[0112] Specific limitations regarding the named entity recognition device can be found in the limitations of the named entity recognition method above, and will not be repeated here. Each module in the aforementioned named entity recognition device can be implemented entirely or partially through software, hardware, or a combination thereof. These modules can be embedded in or independent of the processor in a computer device in hardware form, or stored in the memory of a computer device in software form, so that the processor can call and execute the operations corresponding to each module.

[0113] This specification also provides a computer-readable storage medium storing a computer program that can be used to execute the above-described... Figure 1 The provided named entity recognition method.

[0114] This instruction manual also provides Figure 7 The schematic diagram of the computer device shown is as follows: Figure 7 As shown, at the hardware level, this computer device includes a processor, internal bus, network interface, memory, and non-volatile memory, and may also include other hardware required for business operations. The processor reads the corresponding computer program from the non-volatile memory into memory and then executes it to achieve the above. Figure 1 The provided named entity recognition method.

[0115] Those skilled in the art can understand that all or part of the processes in the above-mentioned embodiment methods can be completed by instructing the relevant hardware through a computer program. The computer program can be stored in a non-volatile computer readable storage medium, and when executed, can include the processes of the above-mentioned embodiments of the methods. Any reference to memory, storage, database or other medium used in the embodiments provided in the present application can include at least one of non-volatile and volatile memory. The non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory or optical memory, etc. The volatile memory can include random access memory (RAM) or external cache memory. As an illustration but not limitation, the RAM can be in various forms, such as static random access memory (SRAM) or dynamic random access memory (DRAM), etc.

[0116] The technical features of the above embodiments can be combined in any manner. In order to make the description simple, all possible combinations of the technical features in the above embodiments are not described, however, as long as the combinations of the technical features do not exist contradictory, they should be considered as the scope of the present disclosure.

Claims

1. A method of named entity recognition, characterized by, The application relates to a named entity recognition method and device. The method comprises the following steps: acquiring text data as sample texts, and labeling real named entities in the sample texts; taking an ALBERT model as an embedding representation layer of the text data, forming a double-channel encoding layer by using a parallel DouBiLSTM model and a multi-window CNN model, and then connecting the double-channel encoding layer and a label decoding layer based on a conditional random field to form a combined model; the DouBiLSTM model comprises two BiLSTM networks adopting a residual connection mechanism; The sample texts are input into the combined model to obtain predicted named entities, and the combined model is trained according to the real named entities and the predicted named entities to obtain a named entity recognition model; The target text is input into the named entity recognition model to generate a dynamic representation vector corresponding to the target text by using the ALBERT model; global semantic features of the target text are extracted from the dynamic representation vector by using the DouBiLSTM model, and multi-granularity local semantic features of the target text are extracted from the dynamic representation vector by using the multi-window CNN model; the global semantic features and the multi-granularity local semantic features are spliced by using the label decoding layer, and the spliced semantic features are mapped to a classification label space to obtain predicted information, so that the predicted information is decoded by using the conditional random field to obtain named entities. The dynamic representation vector corresponding to the target text is generated by using the ALBERT model, and the method comprises the following steps:

2. The method of claim 1, wherein, The target text is segmented by using a segmenter of the ALBERT model to obtain a plurality of subwords, and each subword is mapped to a corresponding word vector by using a TokenEmbeddings module of the ALBERT model; Each sentence of the target text is segmented by using a SegmentEmbeddings module of the ALBERT model to generate sentence embedding representations; Position embedding representations of each character position in the target text are generated by using a PositionEmbeddings module of the ALBERT model; Comprehensive embedding representations of the target text are obtained according to the word vectors, the sentence embedding representations and the position embedding representations, and the comprehensive embedding representations are input into a Transformer encoder of the ALBERT model to obtain a dynamic representation vector of the target text based on context. The global semantic features of the target text are extracted from the dynamic representation vector of the target text by using the DouBiLSTM model, and the method comprises the following steps: 3.The named entity recognition method of claim 1, wherein, The global semantic features of the target text are extracted from the dynamic representation vector of the target text by using the DouBiLSTM model based on the following formula: G = H(V) + H(H(V)); wherein V represents the dynamic representation vector of the target text, H() represents the output of the BiLSTM network, and G represents the global semantic features of the target text. The multi-granularity local semantic features of the target text are extracted from the dynamic representation vector of the target text by using the multi-window CNN model, and the method comprises the following steps:

4. The named entity recognition method of claim 1, wherein, The multi-granularity local semantic features of the target text are extracted from the dynamic representation vector of the target text by using the multi-window CNN model based on different sizes of convolution kernels, and the dynamic representation vector of each character in the target text is convolved by using a ReLU activation function to activate the dynamic representation vector, so that the multi-granularity local semantic features of the target text are extracted: ReLU ， ； wherein, C k denotes a multi-granularity local semantic feature of the target text, denotes a multi-granularity local feature of the th character, k denotes a window size of the convolution kernel, W c denotes a weight matrix parameter of the convolution kernel, b c denotes a bias vector of the convolution kernel, denotes an input word vector sequence of the th character to the th character, ​ denotes an activation function.

5. The named entity recognition method of claim 1, wherein, The spliced semantic features are mapped to a classification label space to obtain prediction information, and the named entity is obtained by decoding the prediction information through a conditional random field. The spliced semantic features are mapped to multiple classification spaces to obtain prediction information based on a full connection layer in the label decoding layer through the following formula: N = MA c + b c ； The named entity is recognized by decoding the prediction information through the conditional random field through the following formula: ， Y = { Y 1,…, Y i ,…, Y n }； ； wherein, N denotes prediction information, M denotes a spliced semantic feature obtained by splicing, A c denotes a feature weight matrix of a full connection layer, b c denotes a bias vector, Y denotes a predicted named entity sequence, Sorce ( N , Y ) denotes a score of the predicted named entity sequence based on the prediction information, denotes a transition probability score from a predicted named entity Y i to a predicted named entity Y i+1 , denotes a softmax output score of a predicted named entity being Y i the i th character, denotes a highest-scored named entity sequence.

6. A named entity recognition apparatus characterized by comprising: Comprise: The acquisition module is configured to acquire text data as a sample text and label real named entities in the sample text; The construction module is configured to use an ALBERT model as an embedding representation layer of the text data, use a parallel DouBiLSTM model and a multi-window CNN model to form a double-channel encoding layer and sequentially connect the double-channel encoding layer after the embedding representation layer, and sequentially connect a label decoding layer based on a conditional random field to construct a combined model; the DouBiLSTM model comprises two BiLSTM networks adopting a residual connection mechanism; The training module is configured to input the sample text into the combined model to obtain predicted named entities, and train the combined model according to the real named entities and the predicted named entities to obtain a named entity recognition model; The recognition module is configured to input a target text to be subjected to named entity recognition into the named entity recognition model to generate a dynamic representation vector corresponding to the target text through the ALBERT model, extract global semantic features of the target text according to the dynamic representation vector through the DouBiLSTM model, and extract multi-granularity local semantic features of the target text according to the dynamic representation vector through the multi-window CNN model, splice the global semantic features and the multi-granularity local semantic features through the label decoding layer, map spliced semantic features obtained through the splicing to a classification label space to obtain prediction information, and decode the prediction information through the conditional random field to obtain the named entity.

7. A computer-readable storage medium, characterized in that, The storage medium stores a computer program, and the computer program is executed by the processor to implement the method in any one of claims 1-5.

8. A computer device, comprising: The computer program is stored in the memory and executable on the processor, and the processor implements the method in any one of claims 1-5 when executing the program. The computer program is stored in the memory and executable on the processor, and the processor implements the method in any one of claims 1-5 when executing the program.

Images