Named entity recognition method, storage medium and electronic equipment

Abstract

The invention relates to the technical field of natural language processing, in particular to a named entity recognition method, a storage medium and electronic equipment. The method comprises the following steps: determining a word vector of each character in a target text and a word vector of each vocabulary in the target text; according to the character vectors of the characters and the word vectors of the vocabularies to which the characters belong, determining fusion vectors of the characters; and according to the fusion vector, determining a named entity tag of the character. Thus, on the basis of the word vectors, through fusion of the word vectors, the fusion vectors can contain richer information, the recognition precision of named entity recognition is improved, the possibility of misrecognition is reduced, and the robustness and universality of the system are improved.

Description

Technical Field

[0001] The present disclosure relates to the technical field of natural language processing, and in particular to a named entity recognition method, a storage medium, and an electronic device. Background Art

[0002] At present, there are two types of classic models for solving nested entity recognition tasks: cascading models and region-based models. The cascading model refers to decomposing the nested entity recognition task into multiple subtasks, each of which is responsible for identifying a layer of entities, and then using the results of the previous layer as the input of the next layer. The region-based model refers to dividing the text into multiple regions, each region corresponds to a candidate entity, and then using graph neural networks or other methods to determine whether each region is a valid entity. The disadvantage of the cascading model is that it is prone to error propagation and inconsistent annotations. The disadvantage of the region-based model is that it is computationally intensive and requires more training data. The above classic models for solving nested entity recognition tasks all have significant shortcomings to varying degrees, making it difficult to ensure the accuracy of the results of named entity recognition. Summary of the invention

[0003] The purpose of the present disclosure is to provide a named entity recognition method, a storage medium, and an electronic device to improve the recognition accuracy of named entity recognition, as well as the robustness and versatility of the system.

[0004] In order to achieve the above object, the present disclosure provides a method for named entity recognition, comprising:

[0005] Determine the word vector for each character in the target text;

[0006] Determining a word vector for each word in the target text;

[0007] Determining a fusion vector of the character according to the word vector of the character and the word vector of the vocabulary to which it belongs;

[0008] A named entity tag of the character is determined according to the fusion vector.

[0009] Optionally, determining a word vector for each character in the target text includes:

[0010] The target text is input into the BERT model to obtain a first word vector sequence, wherein the first word vector sequence includes a word vector for each of the characters.

[0011] Optionally, determining a word vector for each character in the target text includes:

[0012] The word vector of the character is determined according to the word embedding information, sentence embedding information and position embedding information of the character, wherein the word embedding information is used to represent the meaning of the character, the sentence embedding information is used to represent the clause of the character in the target text, and the position embedding information is used to represent the position of the character in the target text.

[0013] Optionally, determining the word vector of the character includes:

[0014] The word vector e of the nth character is determined by the following formula n :

[0015] e n =Embedding tok (x n )+Embedding seg (x n )+Embedding pos (x n )

[0016] Among them, x n For the nth character, Embedding tok (x n ) is the nth character x n Word embedding information, Embedding seg (x n ) is the nth character x n Sentence embedding information, Embedding pos (x n ) is the nth character x n The location of the embed information.

[0017] Optionally, dimensions of the word embedding information, the sentence embedding information and the position embedding information are consistent.

[0018] Optionally, determining a word vector for each word in the target text includes:

[0019] Inputting the target text into a target word segmentation tool to obtain a plurality of words in the target text;

[0020] The word vector of each of the words is determined by a word vector model, wherein the dimension of each of the word vectors is the same.

[0021] Optionally, the word vector model is a Word2vec model.

[0022] Optionally, the target word segmentation tool is obtained by importing a pre-constructed target domain professional dictionary into the original word segmentation tool.

[0023] Optionally, the target domain professional dictionary is constructed in the following manner:

[0024] Obtain reference vocabulary from a reference database corresponding to the target domain;

[0025] The reference vocabulary is preprocessed to obtain the target domain professional dictionary, wherein the preprocessing includes at least one of deduplication, entity category labeling and regularization processing.

[0026] Optionally, the regularization processing includes:

[0027] If the reference vocabulary includes the target symbol, the target symbol and the content in the target symbol are deleted.

[0028] Optionally, determining the fusion vector of the character according to the word vector of the character and the word vector of the corresponding vocabulary includes:

[0029] The word vector of the character and the word vector of the vocabulary to which the character belongs are concatenated to obtain a fusion vector of the character, wherein the fusion vector is used to represent the character information of the character and the word boundary information of the vocabulary to which the character belongs.

[0030] Optionally, the concatenating the word vector of the character and the word vector of the corresponding vocabulary to obtain a fusion vector of the character includes:

[0031] The fusion vector is determined by the following formula:

[0032]

[0033] Among them, q n is the fusion vector of the nth character, e n is the word vector of the nth character, w n is the word vector of the vocabulary to which the nth character belongs, is the splicing symbol, the fusion vector q n The dimension of is the sum of the corresponding character vector dimension and the word vector dimension.

[0034] Optionally, determining the named entity tag of the character according to the fusion vector includes:

[0035] According to the BiLSTM model and the fusion vector, a named entity label of each of the characters is determined.

[0036] Optionally, determining the named entity label of each of the characters according to the BiLSTM model and the fusion vector includes:

[0037] Inputting a second word vector sequence into the BiLSTM model to obtain a first label sequence, wherein the second word vector sequence includes a fusion vector of each of the characters, and the first label sequence includes at least one reference label of each of the characters and a reference probability of the reference label;

[0038] Inputting the first label sequence into the CRF model to obtain a second label sequence, wherein the second label sequence includes an output label for each character and an output probability of the output label;

[0039] The output label of each character in the second label sequence is determined as the named entity label of the corresponding character.

[0040] Optionally, the second label sequence is obtained in the CRF model in the following manner:

[0041] Determine a state transfer matrix according to the first tag sequence;

[0042] Determining a score for each possible tag sequence according to the state transition matrix;

[0043] The possible tag sequence with the largest score is determined as the second tag sequence.

[0044] Optionally, determining the score of each possible tag sequence according to the state transfer matrix includes:

[0045] Determining a transfer score according to the state transfer matrix;

[0046] determining an emission fraction according to the first tag sequence;

[0047] The score of the possible tag sequence is determined according to the sum of the transfer score and the emission score.

[0048] Optionally, the loss function loss in the CRF model is constructed according to the following formula:

[0049] loss=-score(x,y)+log∑ y′ e score(x，y′)

[0050] Wherein, x is the first label sequence, y is the second label sequence, y′ is a possible label sequence, score(x, y) is the score of the second label sequence, and score(x, y′) is the score of the possible label sequence.

[0051] Optionally, the method further comprises:

[0052] Get the text to be recognized;

[0053] The target text is determined according to the text to be recognized and the target length.

[0054] Optionally, determining the target text according to the text to be recognized and the target length includes:

[0055] If the length of the text to be recognized is less than the target length, padding processing is performed on the text to be recognized to obtain the target text of the target length.

[0056] Optionally, determining the target text according to the text to be recognized and the target length includes:

[0057] If the length of the text to be recognized is greater than the target length, the text to be recognized is cut to obtain the target text of the target length.

[0058] A second aspect of the present disclosure provides a non-transitory computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the steps of the method provided in the first aspect of the present disclosure.

[0059] A third aspect of the present disclosure provides an electronic device, including:

[0060] a memory having a computer program stored thereon;

[0061] A controller, wherein when the computer program is executed by the controller, the steps of the method provided in the first aspect of the present disclosure are implemented.

[0062] In the above technical solution, the word vector of each character in the target text and the word vector of each vocabulary in the target text are determined; the fusion vector of the character is determined based on the word vector of the character and the word vector of the vocabulary to which it belongs; and the named entity label of the character is determined based on the fusion vector. In this way, on the basis of the word vector, by fusing the word vector, the fusion vector can contain richer information, improve the recognition accuracy of named entity recognition, reduce the possibility of misrecognition, and improve the robustness and versatility of the system.

[0063] Other features and advantages of the present disclosure will be described in detail in the following detailed description. BRIEF DESCRIPTION OF THE DRAWINGS

[0064] The accompanying drawings are used to provide a further understanding of the present disclosure and constitute a part of the specification. Together with the following specific embodiments, they are used to explain the present disclosure but do not constitute a limitation of the present disclosure. In the accompanying drawings:

[0065] Figure 1 It is a flowchart of a method for named entity recognition provided by an exemplary embodiment of the present disclosure.

[0066] Figure 2It is a schematic diagram of vector stitching provided by an exemplary embodiment of the present disclosure.

[0067] Figure 3 It is a schematic diagram of BERT model vector extraction provided by an exemplary embodiment of the present disclosure.

[0068] Figure 4 Schematic diagram of an LSTM unit of a BiLSTM model provided by an exemplary embodiment of the present disclosure.

[0069] Figure 5 It is a flowchart of a method for named entity recognition provided by an exemplary embodiment of the present disclosure.

[0070] Figure 6 It is a framework diagram of a named entity recognition model provided by an exemplary embodiment of the present disclosure.

[0071] Figure 7 It is a block diagram of a named entity recognition device provided by an exemplary embodiment of the present disclosure.

[0072] Figure 8 is a block diagram of an electronic device provided by an exemplary embodiment of the present disclosure. DETAILED DESCRIPTION

[0073] The specific implementation of the present disclosure is described in detail below in conjunction with the accompanying drawings. It should be understood that the specific implementation described herein is only used to illustrate and explain the present disclosure, and is not used to limit the present disclosure.

[0074] Figure 1 FIG. 1 is a flowchart of a method for identifying named entities provided by an exemplary embodiment of the present disclosure. Figure 1 As shown, the method may include S101 to S104.

[0075] S101, determining the word vector of each character in the target text.

[0076] For example, the word vector of each character in the target text can be determined by using models such as BERT (Bidirectional Encoder Representation from Transformers, a pre-trained language representation model) and Word2vec (World embeddings, word vectors).

[0077] S102, determining the word vector of each word in the target text.

[0078] For example, the word vector of each word in the target text can be determined by one-hot encoding or Word2vec model. Taking the determination of the word vector of each word in the target text by Word2vec model as an example, the target text can be first segmented, and then the segmentation result is input into the Word2vec model, and the content output by the Word2vec model is the word vector of each word in the target text.

[0079] S103, determining a fusion vector of the character according to the character vector and the word vector of the vocabulary to which the character belongs.

[0080] For example, the word vector of a character and the word vector of the vocabulary to which it belongs can be concatenated to obtain a fusion vector of the character. The word vector of each character in the vocabulary is taken from the word vector of the vocabulary to which it belongs. For each character, the fusion vector can be the concatenation result of the word vector of the character and the word vector of the vocabulary to which it belongs. Taking the target text "Whooping cough is an acute respiratory infectious disease" as an example, the vocabulary obtained includes "whooping cough", "is" and "acute respiratory infectious disease", and the word vectors corresponding to these words are w 1 、w 2 and w 3 , "Whooping cough is an acute respiratory infectious disease" has 12 characters, and the word vector of each character is e 1 、e 2 、e 3 、e 4 、e 5 、e 6 、e 7 、e 8 、e 9 、e 10 、e 11 、e 12 Among them, the fusion vector of "hundred" is e 1 +w 1 , the fusion vector of "day" is e 2 +w 1 The fusion vector of “cough” is e 3 +w 1 , the fusion vector of "yes" is e 4 +w 2 The fusion vector of "urgent" is e 5 +w 3 , the fusion vector of "sex" is e 6 +w 3 The fusion vector of “hu” is e 7 +w 3 The fusion vector of "absorption" is e 8 +w 3 , the fusion vector of "Tao" is e 9 +w 3The fusion vector of "transmission" is e 10 +w 3 , the fusion vector of "dying" is e 11 +w 3 , the fusion vector of “disease” is e 12 +w 3 The splicing process can be as follows Figure 2 As shown in the figure, the dimension of each word vector can be made the same, and the dimension of each character vector can be made the same, so that the length of the fused vector can be ensured to be consistent.

[0081] If the target sentence includes a professional term that is a nested entity, the fusion vector corresponding to the professional term includes the character information of each character in the professional term and the word boundary information of the nested entity. In this way, on the basis of the character vector, by fusing the word vector, the fusion vector can contain richer information, improve the recognition accuracy of named entity recognition, and avoid misrecognition.

[0082] S104: Determine the named entity tag of the character according to the fused vector.

[0083] For example, the determined fusion vector can be input into a BiLSTM (Bidirectional Long Short-Term Memory) model to obtain the named entity label of each character through the BiLSTM model. In this way, through the BiLSTM model, the correlation between the context in the sequence including the fusion vector of each character can be learned, and rich context information can be captured to improve the accuracy of the determined named entity label of the character.

[0084] In the present disclosure, the traditional named entity recognition method can be optimized by fusing the word vector and term vector of a single character. The traditional named entity recognition method can be applied to the nested named entity recognition task without affecting its performance in the general entity recognition task. The robustness is better than that of the traditional entity recognition method alone. It is more versatile in processing daily entity relationship extraction.

[0085] In the above technical solution, the word vector of each character in the target text and the word vector of each vocabulary in the target text are determined; the fusion vector of the character is determined based on the word vector of the character and the word vector of the vocabulary to which it belongs; and the named entity label of the character is determined based on the fusion vector. In this way, on the basis of the word vector, by fusing the word vector, the fusion vector can contain richer information, improve the recognition accuracy of named entity recognition, reduce the possibility of misrecognition, and improve the robustness and versatility of the system.

[0086] In an optional embodiment, in S101, determining the word vector of each character in the target text includes:

[0087] Input the target text into the BERT model to obtain a first sequence of word vectors, where the first sequence of word vectors includes the word vectors of each character.

[0088] Exemplarily, the target text can be input into the BERT model in the form of a sequence, and the input sequence can be: input = ([CLS], s 1 , s 2 , …, s u , [SEP], p 1 , p 2 , …, p v , [SEP]). Where u and v are the sequence numbers in the input symbol table, and the subsequence (s 1 , s 2 , …, s u ) is the preceding sentence in the sentence pair, and the subsequence (p 1 , p 2 , …, p v ) is the subsequent sentence in the sentence pair. The input sequence start marker [CLS] is used for classification tasks, and the special marker [SEP] is used to distinguish each clause. Taking the target text "Lung cancer occurs in the bronchial mucosa epithelium, also known as bronchial carcinoma" as an example, the schematic diagram of vector extraction by the BERT model is as Figure 3 shown, where each vector in the word embedding information Token Embeddings from E [CLS] to the last E [SEP] in this line represents the meaning of the corresponding character, the E in the sentence embedding information Segment Embeddings A indicates that the corresponding character is in the preceding sentence, and E B indicates that the corresponding character is in the subsequent sentence; the E in the position embedding information Position Embeddings 1 - E 19 represents the position of the corresponding character in the target text. Taking the character "发" as an example, its corresponding word vector is composed of three parts: E 发 , E A and E 3 . Inputting the target text into the BERT model can obtain a first sequence of word vectors including the word vectors of each character in the target text. The order of each word vector in the first sequence of word vectors is the same as the order of the corresponding character in the target text. Based on the BERT model, the context semantics and sentence associations can be better understood, so that the word vectors extracted contain richer information, thereby improving the accuracy of the subsequent named entity recognition results.

[0089] In an optional embodiment, in S101, determining the word vectors of each character in the target text includes: ​​

[0090] The word vector of the character is determined according to the word embedding information, sentence embedding information and position embedding information of the character, wherein the word embedding information is used to represent the meaning of the character, the sentence embedding information is used to represent the clause of the character in the target text, and the position embedding information is used to represent the position of the character in the target text.

[0091] For example, the word vector e of the nth character can be determined by the following formula n :

[0092] e n =Embedding tok (x n )+Embedding seg (x n )+Embedding pos (x n )

[0093] Among them, x n For the nth character, Embedding tok (x n ) is the nth character x n Word embedding information, Embedding seg (x n ) is the nth character x n Sentence embedding information, Embedding pos (x n ) is the nth character x n The location of the embed information.

[0094] The dimensions of word embedding information, sentence embedding information, and position embedding information are consistent. For example, the dimensions d bert can all be set to 768. The dimensions of word embedding information, sentence embedding information, and position embedding information are consistent, which can facilitate the combination of information to obtain word vectors simply and quickly.

[0095] In an optional embodiment, in S102, determining the word vector of each word in the target text includes:

[0096] Input the target text into the target word segmentation tool to obtain multiple words in the target text;

[0097] The word vector of each vocabulary word is determined by a word vector model, wherein the dimension of each word vector is the same.

[0098] The target word segmentation tool may be obtained by importing a pre-built target domain professional dictionary into the original word segmentation tool.

[0099] For example, the target word segmentation tool can be a Jieba word segmentation tool, a THULAC word segmentation tool, or an LTP word segmentation tool, which is not limited here. The target text is input into the target word segmentation tool, and the target text can be segmented to obtain multiple words. In various professional fields, there are often problems with professional terms that are difficult to identify, as well as nested entities. This often leads to a reduction in the reliability of named entity recognition results. To solve the above problems, a professional dictionary for the target field can be constructed and imported into the word segmentation tool so that the word segmentation tool can accurately realize the division of vocabulary, that is, the reliability of the word segmentation results of the word segmentation tool can be improved.

[0100] Taking the target field as the medical field as an example, as shown in Table 1, the word segmentation result obtained by the word segmentation tool without importing the professional dictionary of the target field is word segmentation result 1, including "pertussis", "is", "acute", "respiratory tract" and "infectious disease"; and the word segmentation result obtained by the word segmentation tool with importing the professional dictionary of the target field is word segmentation result 2, including "pertussis", "is", and "acute respiratory infectious disease". The word segmentation result of the word segmentation tool with importing the professional dictionary of the target field is more professional and can better identify nested entities in the professional field.

[0101] Table 1

[0102]

[0103] In one embodiment, the target domain professional dictionary can be constructed in the following manner:

[0104] Obtain reference vocabulary from a reference database corresponding to the target domain;

[0105] The reference vocabulary is preprocessed to obtain a target domain professional dictionary, wherein the preprocessing includes at least one of deduplication, entity category labeling, and regularization processing.

[0106] Still taking the target field as the medical field as an example, multiple thesaurus related to the medical field can be determined as reference databases, medical terms in the reference databases can be determined as reference vocabulary, and the reference vocabulary can be pre-processed. Multiple reference databases belong to the same field, and there is a high probability that there are repeated medical terms. Therefore, the determined reference vocabulary can be deduplicated. Entity categories can include medical entities related to diseases, symptoms, drugs, etc., which are used to classify reference vocabulary. Regularization processing is to organize non-standard vocabulary. In an optional embodiment, regularization processing may include:

[0107] If the target symbol is included in the reference vocabulary, the target symbol and the content in the target symbol are deleted.

[0108] For example, the target symbol may be a bracket. Some entities in the reference vocabulary contain content supplemented by brackets. The process of deleting brackets and the content in the brackets can be called regularization of the reference vocabulary. In the above manner, a medical field professional dictionary as shown in Table 2 can be constructed.

[0109] Table 2

[0110] Medical terminology category Children's Qingfei Oral Liquid drug Children's Cold Syrup drug Enlarged spleen symptom Psoriasis disease tonsillitis disease Children's cold disease

[0111] In this way, by constructing a professional dictionary in the target field, it is possible to recognize new named entities by updating the professional dictionary in the target field without retraining the BERT model, the subsequent BiLSTM model, and the CRF (Conditional Random Fields) model, saving manpower and material resources. In particular, for the high frequency of new named entities in certain professional fields, it can greatly shorten the adaptation time of named entity recognition to newly emerging named entities, better handle the recognition problem of multiple entity types corresponding to one named entity, and provide strong support for data analysis and decision-making in related fields.

[0112] For example, the word vector model can be a one-hot encoding model or a Word2vec model. Preferably, the word vector model can be a Word2vec model. On the one hand, by constructing word vectors based on the Word2vec method, word vectors that express more information can be obtained. On the other hand, by constructing word vectors based on the Word2vec method, the one-hot encoder can be converted into a low-dimensional continuous value through its Embedding layer, avoiding the problems of vocabulary gap and dimensionality disaster.

[0113] Among them, the word vector dimension d output by the Word2vec model w2v It can be 200, the window size can be set to 5, and the learning rate can be set to 0.001, where the learning rate determines the convergence effect of the Word2vec model.

[0114] In addition, if the Word2vec model cannot obtain the word vector corresponding to the vocabulary, the zero vector that is consistent with the determined word vector dimension can be used as the word vector and concatenated with the corresponding character vector to ensure that the dimensions of the fusion vectors of each character are consistent.

[0115] In an optional embodiment, in S103, determining the fusion vector of the character according to the character vector and the word vector of the corresponding vocabulary may include:

[0116] The word vector of the character and the word vector of the vocabulary to which it belongs are concatenated to obtain a fusion vector of the character. The fusion vector is used to represent the character information of the character and the word boundary information of the vocabulary to which the character belongs.

[0117] Among them, character information can be determined based on word vectors, and word boundary information can be determined based on word vectors. The fusion vector can be determined by the following formula:

[0118]

[0119] Among them, q n is the fusion vector of the nth character, e n is the word vector of the nth character, w n is the word vector of the vocabulary to which the nth character belongs, ⊕ is the concatenation symbol, and the fusion vector q n The dimension of is the sum of the corresponding character vector dimension and the word vector dimension.

[0120] In an optional embodiment, in S104, determining the named entity tag of the character according to the fusion vector includes:

[0121] Based on the BiLSTM model and fusion vector, the named entity label of each character is determined.

[0122] For example, the second word vector sequence constructed based on the fusion vector can be input into the BiLSTM model to obtain a first label sequence, wherein the first label sequence includes at least one reference label and a reference probability of the reference label for each character of the target text, and the order of each vector in the second word vector sequence is the same as the order of the corresponding characters in the target text. The reference label corresponding to the maximum reference probability of each character in the first label sequence can be determined as the named entity label of the character.

[0123] The BiLSTM model can extract text information containing context for entity classification. The BiLSTM model consists of two LSTM layers, a forward LSTM layer (i.e., forward propagation layer) and a reverse LSTM layer (i.e., reverse propagation layer). The forward LSTM layer processes data from the beginning to the end of the sequence, while the reverse LSTM layer processes data from the end to the beginning of the sequence. This bidirectional processing mechanism allows the BiLSTM model to consider past and future information at the same time, learn the correlation between the context in the sequence, and capture richer context information than the unidirectional LSTM model. Therefore, the BiLSTM model has better performance when processing sequence data.

[0124] The LSTM layer is composed of LSTM units. The schematic diagram of the LSTM unit structure can be shown as follows: Figure 4 As shown in Figure 1. The gating mechanism of the LSTM unit is a technology used to control the flow of information. The three gates are input gate i t, forget gate f t and output gate o t , input gate i t Control the input of new information, forget gate f t Controls the retention or forgetting of old information, output gate o t Controls the selection of output information. Figure 4 x in t is the input at time t, c t-1 is the internal information at time t-1, h t-1 is the external information at time t-1, according to x t 、c t-1 and h t-1 The internal information c at time t can be determined t and external information t , c t and h t On the one hand, it is used as the output of the hidden layer at the next moment, and on the other hand, it is used as the input of the LSTM unit at the next moment. The LSTM unit divides the linear part and the nonlinear part, and the output c t The linear part is saved, and the output h t What is retained is the nonlinear part.

[0125] The output y of BiLSTM at time t t It can be determined by the following formula:

[0126]

[0127] in, is the output of the forward propagation layer, is the output of the back-propagation layer, represents vector concatenation, and V is a linear matrix.

[0128] For the BiLSTM model, the text library used for training is generally a general text library. The state transfer matrix obtained by training this model can achieve excellent results in non-professional fields. However, due to the small amount of training text related to professional terms and nested entities, when faced with professional terms, the state transfer matrix obtained by the general text library may be difficult to identify professional terms or nested entities. Taking "acute respiratory infectious diseases" as an example, without adding word vectors, the corresponding label is output according to the minimum word. The above sequence will be split into three parts according to the state transfer matrix obtained through training, among which "acute" is an irrelevant adjective, "respiratory tract" is judged as a body part, and "infectious disease" is judged as a disease. The output entity labels are (O, O, B-bod, I-bod, E-bod, B-dis, I-dis, E-dis); after adding word vectors, each word vector carries the word boundary information of the vocabulary to which it belongs. The BiLSTM model can capture the front and back information of the input sequence at the same time, so it can recognize the word boundary information and judge it as a whole to obtain the corresponding label. The final output form is (B-dis, I-dis, I-dis, I-dis, I-dis, I-dis, E-dis). Among them, O represents a non-entity label, B-bod represents the beginning of the body part label, I-bod represents the middle of the body part label, E-bod represents the end of the body part label, B-dis represents the beginning of the disease label, I-dis represents the middle of the disease label, and E-dis represents the end of the disease label.

[0129] In an optional embodiment, determining the named entity label of each character according to the BiLSTM model and the fusion vector may include:

[0130] Inputting the second word vector sequence into the BiLSTM model to obtain a first label sequence, wherein the second word vector sequence includes a fusion vector of each character, and the first label sequence includes at least one reference label of each character and a reference probability of the reference label;

[0131] Input the first label sequence into the CRF model to obtain a second label sequence, wherein the second label sequence includes an output label of each character and an output probability of the output label;

[0132] The output label of each character in the second label sequence is determined as the named entity label of the corresponding character.

[0133] For example, the BiLSTM model and the CRF model can be considered as a complete model. The CRF model can better handle the dependency of labels by considering the correlation between each label in the sequence. Therefore, the CRF model can constrain the content output by the BiLSTM model and limit the label transfer, thereby obtaining a more accurate named entity recognition result. In the CRF model, the second label sequence y can be obtained by the following formula:

[0134] y=argmax score(x,y′)y′∈Y X

[0135] Among them, x is the first label sequence, y′ is the possible label sequence, and Y X are all possible label sequences, and score(x,y′) is the score of the possible label sequence determined based on the first label sequence x.

[0136] In an optional embodiment, the second label sequence is obtained in the CRF layer in the following manner:

[0137] Determine a state transfer matrix according to the first tag sequence;

[0138] According to the state transition matrix, determine the score of each possible label sequence;

[0139] The possible label sequence with the largest score is determined as the second label sequence.

[0140] For example, a framework of a state transfer matrix can be established based on the first label sequence, and the state transfer matrix can be regarded as a parameter of the CRF model. Before training the model, the values ​​of each element in the state transfer matrix can be randomly initialized. These values ​​will be updated with the iteration process of training. In other words, the CRF model can learn these constraints by itself. For example, the state transfer matrix P is as follows:

[0141]

[0142] Among them, p ij is the score (i.e., probability) from the i-th label to the j-th label, where b is the length of the first label sequence and a is the number of non-repeated words in the text library used by the CRF model.

[0143] The score for each possible tag sequence can be determined as follows:

[0144] According to the state transfer matrix, determine the transfer score;

[0145] determining an emission fraction based on the first tag sequence;

[0146] The score of the possible label sequence is determined based on the sum of the transfer score and the emission score.

[0147] The character corresponding to the first label sequence output by the BiLSTM model is (x 1 ,x 2 ,x 3 ), the corresponding entity label is (B-dis, I-dis, E-dis) as an example, the emission score EmissionScore can be determined by the following formula:

[0148] EmissionScore = x 0,start +x 1,B-dis +x 2,I-dis +x 3,E-dis +x 4,end

[0149] Among them, x index,label Indicates the score (i.e., probability) of the index-th word being labeled label, x 1,B-dis For the BiLSTM model with character x 1 The corresponding output, x 2,I-dis For the BiLSTM model with character x 2 The corresponding output, x 3,E-dis For the BiLSTM model with character x 3 The corresponding output, x 0,start and x 4,end is 0.

[0150] The transition score TransitionScore can be determined by the following formula:

[0151] TransitionScore = t start→B-dis +t B-dis→I-dis +t I-dis→E-dis +t E-dis→end

[0152] Among them, t label1→label2 Indicates the score of label1 transferred to label2, t start→B-dis is the score of the transfer from label start to label B-dis in the state transfer matrix, t B-dis→I-dis is the score of label B-dis transferred to label I-dis in the state transfer matrix, t I-dis→E-dis is the score of the transfer from label I-dis to label E-dis in the state transfer matrix, t E-dis→end is the score of the transfer from label E-dis to label end in the state transfer matrix.

[0153] For the score of each possible tag sequence, the score of the possible tag sequence may be determined based on iterative calculation of the sum of the transfer score and the emission score.

[0154] In an optional embodiment, the loss function loss in the CRF model can be constructed according to the following formula:

[0155] loss=-score(x,y)+log∑ y′ e score(x，y′)

[0156] Wherein, x is the first label sequence, y is the second label sequence, y′ is a possible label sequence, score(x, y) is the score of the second label sequence, and score(x, y′) is the score of the possible label sequence.

[0157] After the second word vector sequence is input into BiLSTM, the hidden state results of the forward and backward directions are combined to generate the output of BiLSTM. Finally, the output of BiLSTM is used as the input of CRF, thus forming a BiLSTM-CRF structure. This structure combines the characteristics and advantages of BiLSTM and CRF: BiLSTM can effectively save the front and back information of the whole sentence and extract the feature information in the sentence; CRF can use the context information to perform sequence labeling with high accuracy. That is, the BiLSTM model and CRF model can obtain more accurate named entity tags.

[0158] In one embodiment, the named entity recognition method provided by the present disclosure may further include:

[0159] Get the text to be recognized;

[0160] Determine the target text based on the text to be recognized and the target length.

[0161] For example, determining the target text according to the text to be recognized and the target length may include:

[0162] If the length of the text to be recognized is less than the target length, padding is performed on the text to be recognized to obtain the target text of the target length; or

[0163] If the length of the text to be recognized is greater than the target length, the text to be recognized is cut to obtain the target text of the target length.

[0164] For example, the target length can be set based on the actual computing power of the system, for example, it can be set to 256 characters. If the length of the text to be recognized is less than the target length, it can be determined that the current system computing power can meet the current processing requirements. In order to standardize the processing of texts of different lengths, the text to be recognized that is less than the target length can be padded so that its text length reaches the target length. If the length of the text to be recognized is greater than the target length, it can be determined that the current system computing power is difficult to meet the current processing requirements. In order to avoid occupying too many machine resources, the text to be recognized that is greater than the target length can be cut so that its text length reaches the target length.

[0165] Figure 5 is a flowchart of a method for named entity recognition provided by an exemplary embodiment of the present disclosure. Figure 5 , the implementation process of the named entity recognition method provided by the present disclosure can be more clearly understood. The corresponding framework diagram of the named entity recognition model is as follows Figure 6 As shown in Figure 1, it includes the BERT model, vector embedding layer, BiLSTM model and CRF model. Figure 5 As shown, the method may include S501 to S510.

[0166] S501, obtaining reference vocabulary in a reference database corresponding to a target domain.

[0167] S502, pre-processing the reference vocabulary to obtain a professional dictionary in the target field.

[0168] The preprocessing includes at least one of deduplication, entity category labeling and regularization processing.

[0169] S503, importing the target domain professional dictionary into the original word segmentation tool to obtain the target word segmentation tool.

[0170] S504: Input the target text into a target word segmentation tool to obtain a plurality of words in the target text.

[0171] S505, determining the word vector of each word through the Word2vec model.

[0172] S506: Input the target text into the BERT model to obtain a first word vector sequence.

[0173] The first word vector sequence includes a word vector for each character.

[0174] S507, the vector embedding layer concatenates the character vector and the word vector of the corresponding vocabulary to obtain a second character vector sequence.

[0175] The second character vector sequence includes a fused vector of each character.

[0176] S508: Input the second word vector sequence into the BiLSTM model to obtain a first label sequence.

[0177] The first label sequence includes at least one reference label for each character and a reference probability of the reference label.

[0178] S509: Input the first label sequence into the CRF model to obtain a second label sequence.

[0179] The second label sequence includes an output label of each character and an output probability of the output label;

[0180] S510: Determine the output label of each character in the second label sequence as the named entity label of the corresponding character.

[0181] In this way, the BERT model can better understand the contextual semantics and sentence associations, so that the extracted word vectors contain richer information, thereby improving the accuracy of subsequent named entity recognition results. On the basis of word vectors, by fusing word vectors, the fused vectors can contain richer information, improve the recognition accuracy of named entity recognition, and reduce the possibility of misidentification. More accurate named entity labels can be obtained through the BiLSTM model and CRF model. In addition, by constructing and updating professional dictionaries in the target field, the adaptation time of named entity recognition to newly emerging named entities can be greatly shortened.

[0182] Based on the same inventive concept, the present disclosure also provides a named entity recognition device. Figure 7 is a block diagram of a named entity recognition apparatus 700 provided by an exemplary embodiment of the present disclosure. Figure 7 , the named entity recognition device 700 may include:

[0183] A first determination module 701 is used to determine a word vector for each character in the target text;

[0184] A second determination module 702 is used to determine the word vector of each word in the target text;

[0185] A fusion module 703, configured to determine a fusion vector of the character according to the character vector and the word vector of the corresponding vocabulary;

[0186] The recognition module 704 is used to determine the named entity tag of the character according to the fusion vector.

[0187] In the above technical solution, the word vector of each character in the target text and the word vector of each vocabulary in the target text are determined; the fusion vector of the character is determined based on the word vector of the character and the word vector of the vocabulary to which it belongs; and the named entity label of the character is determined based on the fusion vector. In this way, on the basis of the word vector, by fusing the word vector, the fusion vector can contain richer information, improve the recognition accuracy of named entity recognition, reduce the possibility of misrecognition, and improve the robustness and versatility of the system.

[0188] Optionally, the first determination module 701 is used to determine the word vector of each character in the target text by:

[0189] The target text is input into the BERT model to obtain a first word vector sequence, wherein the first word vector sequence includes a word vector for each of the characters.

[0190] Optionally, the first determining module 701 is used to determine the word vector of each character in the target text in the following manner:

[0191] The word vector of the character is determined according to the word embedding information, sentence embedding information and position embedding information of the character, wherein the word embedding information is used to represent the meaning of the character, the sentence embedding information is used to represent the clause of the character in the target text, and the position embedding information is used to represent the position of the character in the target text.

[0192] Optionally, the first determining module 701 is used to determine the word vector of the character by the following formula, including:

[0193] The word vector e of the nth character is determined by the following formula n :

[0194] e n =Embedding tok (x n )+Embedding seg (x n )+Embedding pos (x n )

[0195] Among them, x n For the nth character, Embedding tok (x n ) is the nth character x n Word embedding information, Embedding seg (x n ) is the nth character x n Sentence embedding information, Embedding pos (xn ) is the nth character x n The location of the embed information.

[0196] Optionally, dimensions of the word embedding information, the sentence embedding information and the position embedding information are consistent.

[0197] Optionally, the second determination module 702 is used to determine the word vector of each word in the target text by:

[0198] Inputting the target text into a target word segmentation tool to obtain a plurality of words in the target text;

[0199] The word vector of each of the words is determined by a word vector model, wherein the dimension of each of the word vectors is the same.

[0200] Optionally, the target word segmentation tool is obtained by importing a pre-constructed target domain professional dictionary into the original word segmentation tool.

[0201] Optionally, the word vector model is a Word2vec model.

[0202] Optionally, the target domain professional dictionary is constructed in the following manner:

[0203] Obtain reference vocabulary from a reference database corresponding to the target domain;

[0204] The reference vocabulary is preprocessed to obtain the target domain professional dictionary, wherein the preprocessing includes at least one of deduplication, entity category labeling and regularization processing.

[0205] Optionally, the regularization processing includes:

[0206] If the reference vocabulary includes the target symbol, the target symbol and the content in the target symbol are deleted.

[0207] Optionally, the fusion module 703 is used to determine the fusion vector of the character in the following manner:

[0208] The word vector of the character and the word vector of the vocabulary to which the character belongs are concatenated to obtain a fusion vector of the character, wherein the fusion vector is used to represent the character information of the character and the word boundary information of the vocabulary to which the character belongs.

[0209] Optionally, the fusion module 703 is used to determine the fusion vector by the following formula:

[0210]

[0211] Among them, q n is the fusion vector of the nth character, en is the word vector of the nth character, w n is the word vector of the vocabulary to which the nth character belongs, is the splicing symbol, the fusion vector q n The dimension of is the sum of the corresponding character vector dimension and the word vector dimension.

[0212] Optionally, the recognition module 704 is used to determine the named entity tag of each character in the following manner:

[0213] According to the BiLSTM model and the fusion vector, a named entity label of each of the characters is determined.

[0214] Optionally, the identification module 704 includes:

[0215] A first recognition submodule, configured to input a second word vector sequence into the BiLSTM model to obtain a first label sequence, wherein the second word vector sequence includes a fusion vector of each of the characters, and the first label sequence includes at least one reference label of each of the characters and a reference probability of the reference label;

[0216] A second recognition submodule is used to input the first label sequence into the CRF model to obtain a second label sequence, wherein the second label sequence includes an output label of each character and an output probability of the output label;

[0217] The first determination submodule is used to determine the output label of each character in the second label sequence as the named entity label of the corresponding character.

[0218] Optionally, the second label sequence is obtained in the CRF model in the following manner:

[0219] Determine a state transfer matrix according to the first tag sequence;

[0220] Determining a score for each possible tag sequence according to the state transition matrix;

[0221] The possible tag sequence with the largest score is determined as the second tag sequence.

[0222] Optionally, the CRF model determines the score of each possible label sequence by:

[0223] Determining a transfer score according to the state transfer matrix;

[0224] determining an emission fraction according to the first tag sequence;

[0225] The score of the possible tag sequence is determined according to the sum of the transfer score and the emission score.

[0226] Optionally, the loss function loss in the CRF model is constructed according to the following formula:

[0227] loss=-score(x,y)+log∑ y′ e score(x，y′)

[0228] Wherein, x is the first label sequence, y is the second label sequence, y′ is a possible label sequence, score(x, y) is the score of the second label sequence, and score(x, y′) is the score of the possible label sequence.

[0229] Optionally, the device 700 further includes:

[0230] An acquisition module is used to acquire the text to be recognized;

[0231] The second determination module is used to determine the target text according to the text to be recognized and the target length.

[0232] Optionally, the second determining module includes:

[0233] The second determination submodule is used to fill the text to be recognized if the length of the text to be recognized is less than the target length, so as to obtain the target text of the target length.

[0234] Optionally, the second determining module includes:

[0235] The third determination submodule is used to cut the text to be recognized if the length of the text to be recognized is greater than the target length, so as to obtain the target text of the target length.

[0236] Figure 8 1 is a block diagram of an electronic device 1900 according to an exemplary embodiment. For example, the electronic device 1900 may be provided as a server. Figure 8 The electronic device 1900 includes a processor 1922, which may be one or more, and a memory 1932 for storing a computer program executable by the processor 1922. The computer program stored in the memory 1932 may include one or more modules each corresponding to a set of instructions. In addition, the processor 1922 may be configured to execute the computer program to perform the above-mentioned named entity recognition method.

[0237] In addition, the electronic device 1900 may further include a power supply component 1926 and a communication component 1950, wherein the power supply component 1926 may be configured to perform power management of the electronic device 1900, and the communication component 1950 may be configured to implement communication, for example, wired or wireless communication, of the electronic device 1900. In addition, the electronic device 1900 may further include an input / output (I / O) interface 1958. The electronic device 1900 may operate based on an operating system stored in the memory 1932.

[0238] In another exemplary embodiment, a computer-readable storage medium including program instructions is also provided, and when the program instructions are executed by a processor, the steps of the above-mentioned named entity recognition method are implemented. For example, the non-transitory computer-readable storage medium can be the above-mentioned memory 1932 including program instructions, and the above-mentioned program instructions can be executed by the processor 1922 of the electronic device 1900 to complete the above-mentioned named entity recognition method.

[0239] In another exemplary embodiment, a computer program product is further provided. The computer program product includes a computer program that can be executed by a programmable device. The computer program has a code portion for executing the above-mentioned named entity recognition method when executed by the programmable device.

[0240] The preferred embodiments of the present disclosure are described in detail above in conjunction with the accompanying drawings; however, the present disclosure is not limited to the specific details in the above embodiments. Within the technical concept of the present disclosure, a variety of simple modifications can be made to the technical solution of the present disclosure, and these simple modifications all fall within the protection scope of the present disclosure.

[0241] It should also be noted that the various specific technical features described in the above specific embodiments can be combined in any suitable manner without contradiction. In order to avoid unnecessary repetition, the present disclosure will not further describe various possible combinations.

[0242] In addition, various embodiments of the present disclosure may be arbitrarily combined, and as long as they do not violate the concept of the present disclosure, they should also be regarded as the contents disclosed by the present disclosure.

Claims

1. A method for named entity recognition, characterized in that: The method comprises: Determine the word vector for each character in the target text; Determining a word vector for each word in the target text; Determining a fusion vector of the character according to the word vector of the character and the word vector of the vocabulary to which it belongs; A named entity tag of the character is determined according to the fusion vector.

2. The method according to claim 1, characterized in that The step of determining a word vector for each character in the target text includes: The target text is input into the BERT model to obtain a first word vector sequence, wherein the first word vector sequence includes a word vector for each of the characters.

3. The method according to claim 1, characterized in that The step of determining a word vector for each character in the target text includes: The word vector of the character is determined according to the word embedding information, sentence embedding information and position embedding information of the character, wherein the word embedding information is used to represent the meaning of the character, the sentence embedding information is used to represent the clause of the character in the target text, and the position embedding information is used to represent the position of the character in the target text.

4. The method according to claim 3, characterized in that The determining of the word vector of the character comprises: The word vector e of the nth character is determined by the following formula n : e n =Embedding tok (x n )+Embedding seg (x n )+Embedding pos (x n ) Among them, x n For the nth character, Embedding tok (x n ) is the nth character x n Word embedding information, Embedding seg (x n ) is the nth character x n Sentence embedding information, Embedding pos (x n ) is the nth character x n The location of the embed information.

5. The method according to claim 3, characterized in that: The dimensions of the word embedding information, the sentence embedding information and the position embedding information are consistent.

6. The method according to claim 1, characterized in that Determining the word vector of each word in the target text includes: Inputting the target text into a target word segmentation tool to obtain a plurality of words in the target text; The word vector of each of the words is determined by a word vector model, wherein the dimension of each of the word vectors is the same.

7. The method according to claim 6, characterized in that The word vector model is a Word2vec model.

8. The method according to claim 6, characterized in that The target word segmentation tool is obtained by importing a pre-constructed target domain professional dictionary into the original word segmentation tool.

9. The method according to claim 8, characterized in that The target domain professional dictionary is constructed in the following way: Obtain reference vocabulary from a reference database corresponding to the target domain; The reference vocabulary is preprocessed to obtain the target domain professional dictionary, wherein the preprocessing includes at least one of deduplication, entity category labeling and regularization processing.

10. The method according to claim 9, characterized in that The regularized processing includes: If the reference vocabulary includes the target symbol, the target symbol and the content in the target symbol are deleted.

11. The method according to claim 1, characterized in that: The step of determining the fusion vector of the character according to the word vector of the character and the word vector of the vocabulary to which the character belongs comprises: The word vector of the character and the word vector of the vocabulary to which the character belongs are concatenated to obtain a fusion vector of the character, wherein the fusion vector is used to represent the character information of the character and the word boundary information of the vocabulary to which the character belongs.

12. The method according to claim 11, characterized in that The step of concatenating the word vector of the character and the word vector of the corresponding vocabulary to obtain a fusion vector of the character includes: The fusion vector is determined by the following formula: Among them, q n is the fusion vector of the nth character, e n is the word vector of the nth character, w n is the word vector of the vocabulary to which the nth character belongs, is the splicing symbol, the fusion vector q n The dimension of is the sum of the corresponding character vector dimension and the word vector dimension.

13. The method according to claim 1, characterized in that Determining the named entity tag of the character according to the fusion vector includes: According to the BiLSTM model and the fusion vector, a named entity label of each of the characters is determined.

14. The method according to claim 13, characterized in that Determining the named entity label of each character according to the BiLSTM model and the fusion vector includes: Inputting a second word vector sequence into the BiLSTM model to obtain a first label sequence, wherein the second word vector sequence includes a fusion vector of each of the characters, and the first label sequence includes at least one reference label of each of the characters and a reference probability of the reference label; Inputting the first label sequence into the CRF model to obtain a second label sequence, wherein the second label sequence includes an output label for each character and an output probability of the output label; The output label of each character in the second label sequence is determined as the named entity label of the corresponding character.

15. The method according to claim 14, characterized in that The CRF model obtains the second label sequence in the following way: Determine a state transfer matrix according to the first tag sequence; Determining a score for each possible tag sequence according to the state transition matrix; The possible tag sequence with the largest score is determined as the second tag sequence.

16. The method according to claim 15, characterized in that Determining the score of each possible tag sequence according to the state transfer matrix includes: Determining a transfer score according to the state transfer matrix; determining an emission fraction according to the first tag sequence; The score of the possible tag sequence is determined according to the sum of the transfer score and the emission score.

17. The method according to claim 14, characterized in that The loss function loss in the CRF model is constructed according to the following formula: loss=-score(x,y)+log∑ y′ e score(x，y′) Wherein, x is the first label sequence, y is the second label sequence, y′ is a possible label sequence, score(x, y) is the score of the second label sequence, and score(x, y′) is the score of the possible label sequence.

18. The method according to claim 1, characterized in that The method further comprises: Get the text to be recognized; The target text is determined according to the text to be recognized and the target length.

19. The method according to claim 18, characterized in that The step of determining the target text according to the text to be recognized and the target length includes: If the length of the text to be recognized is less than the target length, padding processing is performed on the text to be recognized to obtain the target text of the target length.

20. The method according to claim 18, characterized in that The step of determining the target text according to the text to be recognized and the target length includes: If the length of the text to be recognized is greater than the target length, the text to be recognized is cut to obtain the target text of the target length.

21. A non-transitory computer-readable storage medium having a computer program stored thereon, characterized in that: When the program is executed by a processor, the steps of the method described in any one of claims 1 to 20 are implemented.

22. An electronic device, characterized in that: include: a memory having a computer program stored thereon; A processor, configured to execute the computer program in the memory to implement the steps of the method according to any one of claims 1 to 20.

Images