A nested named entity recognition method and device based on prototype network

Through the nested named entity recognition method based on the prototype network, the prototype network is optimized by using BIOES annotation method and loss function, the problem of insufficient recognition accuracy of named entity recognition with multiple types and complex nested relationships in the existing technology is solved, and high accuracy is achieved multi-category nested entity recognition and structure of unstructured text.

CN118364822BActive Publication Date: 2024-11-01NO 15 INST OF CHINA ELECTRONICS TECH GRP
View PDF 5 Cites 0 Cited by

Patent Information

Application Number
CN202410776086.0
Authority / Receiving Office
CN · China
Patent Type
Patents(China)
Current Assignee / Owner
Filing Date
2024-06-17
Publication Date
2024-11-01
Estimated Expiration
2044-06-17

AI Technical Summary

Technical Problem

Existing named entity recognition technology is difficult to effectively identify named entities with many types, long lengths and nested relationships. There are limitations in recognition accuracy and number of categories, especially under the conditions of many entity category labels and complex entity nesting, it is accurate. Insufficient rate.

Method used

Using nested named entity recognition method based on prototype networks, the prototype network includes encoder, fully connected network and prototype parameter mapping network is constructed, and the label group is constructed using BIOES annotation method, the sample set and loss function are trained to optimize the prototype network, and the character and Learn the mapping relationship of tag groups to improve the recognition accuracy of nested named entities.

Benefits of technology

Improve the recognition accuracy of multi-category nested named entities, and is suitable for the high accuracy recognition of multi-category nested entities in practical applications, enhances the structured ability of unstructured text, and optimizes the model's distribution data in small samples and long-tail on processing capability.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN118364822B_ABST
    Figure CN118364822B_ABST
Patent Text Reader

Abstract

The present invention provides a nested named entity recognition method and device based on a prototype network, which belongs to the technical field of natural language processing. The method of the present invention adopts a prototype network as a recognition structure; when constructing a training sample, a label is formed according to the position of each character in the training text sequence in a known entity and the entity type; one character may correspond to multiple labels, forming an ideal label group, forming a training sample, and training the prototype network; when performing recognition, the text sequence to be recognized is input into the prototype network to generate a label group corresponding to each character; the label groups of all characters in the text to be recognized form a label sequence, and finally the label sequence is parsed to identify the entity. The use of the present invention can effectively support the recognition of multi-category nested entities.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present invention relates to the technical field of natural language processing, and in particular to a method and device for recognizing nested named entities based on a prototype network. Background Art

[0002] Named Entity Recognition (NER), also known as entity recognition, entity chunking, and entity extraction, is a subtask of information extraction. Its purpose is to identify named entities in text and classify them into predefined entity types, such as people, organizations, locations, time expressions, quantities, monetary values, percentages, etc. Named Entity Recognition is an important foundation for many natural language processing (NLP) tasks, such as information extraction, question-answering systems, syntactic analysis, and machine translation.

[0003] Traditional named entity recognition technology only supports the recognition of non-nested, few-category named entities, and cannot support the recognition of entities with many types, long lengths, and nested relationships in actual applications. More and more scholars have begun to invest in the research of nested named entity recognition technology. Nested named entity recognition technology is a type of named entity recognition technology that aims to realize the recognition of multi-type, nested named entities for practical application needs. Current nested named entity recognition methods are mainly divided into: sequence labeling-based methods, SPAN-based methods, hypergraph-based methods, etc., but current methods still have certain limitations in recognition accuracy and the number of categories.

[0004] For example, Takashi et al. (TAKASHI S, EDUARD H. Nested Named Entity Recognition via Second-best Sequence Learning and Decoding [J]. Transactions of the Association for Computational Linguistics, 2020, 8: 605-620) used a hierarchical approach to identify entities of different categories. Each entity type was identified using a separate conditional random field (CRF), and the Viterbi algorithm was used to search and identify the nested entities. Since different CRF networks in this method are trained separately, the recognition effect is greatly affected by the sample distribution and has certain limitations. Liu Feng et al. (Chinese patent, authorization announcement number CN114692636B) proposed a nested named entity recognition method based on relationship recognition and sequence labeling, using sequence labeling tasks to assist deep networks in generating embedding vectors, which are input into multiple Self-Attention modules to predict different entity boxes. This method uses multi-category classification, and the number of entity types supported is still limited.

[0005] In fact, existing technologies mainly obtain entity recognition results directly by using neural networks or by parsing the label sequence predicted by the neural network. The current methods define fewer entity label categories, and many methods first locate the entity through the boundary label and then identify the entity type, which leads to the problem of missing semantic information in the input vector of the classification model; in addition, after the increase of entity categories, there may be a long-tail problem in data distribution, resulting in inaccurate boundary positioning and entity category recognition.

[0006] Therefore, the existing technology still lacks high-accuracy entity recognition under conditions where there are more entity category labels and more complex entity nesting. Summary of the invention

[0007] In view of this, the present invention provides a nested named entity recognition method and device based on a prototype network, which effectively utilizes the advantages of the prototype network on small sample data and improves the recognition accuracy of named entities with multiple types and nested relationships.

[0008] A nested named entity recognition method based on a prototype network of the present invention comprises:

[0009] Step 1: Construct a prototype network including an encoder, a fully connected network, and a prototype parameter mapping network; the prototype parameter mapping network includes 4m+1 prototype groups, each of which contains multiple prototypes; 4m+1 prototype groups correspond to 4m+1 labels; wherein 4m labels correspond to m known entity categories S Entity ; Each entity class S Entity There are four types of labels, namely B label, I label, E label and S label in BIOES labeling method. These four types of labels are marked by BIOES I BIOES and entity class S Entity Composition; the 4m+1th label is the O label in the BIOES annotation method;

[0010] Step 2: Construct a training sample set: Given a training text sequence and a corresponding entity, the entity includes an entity character sequence and an entity category S Entity ; Based on the training text sequence and known entities, determine each character in the training text sequence W i To which entity and its position in the entity character sequence, using the entity category S of the entity to which it belongs Entity and the BIOES identification of the position BIOES Constructs a label; one character W i Corresponding to at least one label, forming an ideal label group label i ;character W i and ideal tag group label i Composition of training samples;

[0011] Step 3: Train the prototype network: Input the training samples into the prototype network, calculate the loss function value between the output of the prototype network and the ideal label group in the training samples, and optimize the prototype network.

[0012] Step 4: When performing recognition, input the text sequence to be recognized into the prototype network to generate each character W i Corresponding tag group l i ; The label groups of all characters in the text sequence to be recognized constitute a label sequence;

[0013] Step 5: Perform label parsing based on the text sequence to be identified and its label sequence to identify the entity set corresponding to the text sequence to be identified.

[0014] Preferably, in step 2, the construction tag is:

[0015] For each character in the training text sequence Wi , determine whether it is included in the known entity;

[0016] If the character W i Contained in a known entity, then based on the character W i The position and length of the entity character sequence, marked with BIOES identifier I BIOES , and get the character W i The entity type S of the entity Entity , composed of labels [I BIOES -S Entity ]; specifically:

[0017] ① Characters W i is the starting point of the entity character sequence and the length of the entity character sequence is greater than 1, then the corresponding label is [BS Entity ];

[0018] ②Characters W i is the end point of the entity character sequence and the length of the entity character sequence is greater than 1, then the corresponding label is [ES Entity ];

[0019] ③ Characters W i Inside the entity character sequence, and not at the start or end of the entity, the corresponding label is [IS Entity ];

[0020] ④ Characters W i is the starting point of the entity character sequence and the length of the entity character sequence is 1, then the corresponding label is [SS Entity ];

[0021] For characters W i , traverse all known entities and get the characters W i All corresponding labels constitute the ideal label group label i ; If for a character, no corresponding label can be found by traversing all known entities, the corresponding label of the character is set to O.

[0022] Preferably, in step 3, the optimization of the prototype network is as follows: the prototype parameter mapping network calculates all implicit features output by the fully connected network h i The distance from all prototypes, the loss function value of the prototype network is calculated through the loss function, and the prototype network is optimized according to the loss function value;

[0023] First, calculate each latent feature h i Belong to the prototype m pq Probability P for:

[0024] (1)

[0025] in, x Representing characters W i ;prototype m pq Indicates p The first q A prototype, p [1,C], q [1, K ]; C The number of labels in the prototype parameter mapping network is also the number of prototype groups. C =4m+1; K The number of prototypes in the prototype group corresponding to each label; d (·) is the distance function; γ is a constant;

[0026] Counting Characters W i The probability value belonging to each label is:

[0027] (2)

[0028] The loss function value is calculated as:

[0029] (3)

[0030] In the above formula, N The number of characters in the text sequence of the input prototype parameter mapping network, C The number of labels in the prototype parameter mapping network is also the number of prototype groups; label i For characters W i The corresponding ideal label group;

[0031] The above formula (3) shows that if C of the tags k Tags in characters W i The corresponding ideal label group label i In Wi Belong to k The probability of the label participates in the accumulation of the loss function, otherwise, it does not participate in the accumulation of the loss function; traverse the characters W i ~ W N , use the above formula (3) to obtain the loss function value of the prototype network;

[0032] Optimize the prototype network according to the loss function value.

[0033] Preferably, the tag parsing method in step 5 is:

[0034] Define the text entity generation list templist, the tag group entity generation list templist_new and the entity set EntitySet; traverse each tag group in the tag sequence respectively, and create a tag group entity generation list templist_new when starting to process a new tag group; for each tag in the tag group, according to the entity type S Entity , In the previous step, set the content of the text entity generation list templist and the entity set EntitySet to the label group entity generation list templist_new and update the entity set EntitySet content;

[0035] After all tags of a certain tag group are processed, the entity process sequences that have not been updated in the text entity generation list templist are added to the tag group entity generation list templist_new, and the entity process sequences whose end positions of the entity character sequences in the tag group entity generation list templist_new are less than the positions of the corresponding characters of the current tag group in the text sequence to be recognized are added to the entity set EntitySet, and these entity character sequences are deleted from the tag group entity generation list templist_new;

[0036] After processing each tag group, assign the tag group entity generation list templist_new to the text entity generation list templist, and clear templist_new;

[0037] When traversing each tag in the tag group, the processing method is as follows:

[0038] ① When the currently traversed label is "BX", X represents the entity category S Entity , create a new entity process sequence Y X , add the character corresponding to the current label to the entity process sequence Y X In the process sequence Y X Add to the tag group entity generation list templist_new;

[0039] ② When the currently traversed tag is "IX", if there is an entity process sequence Y of the same category X in the text entity generation list templist X , the entity process sequence Y X Record it as to be updated, and add the character corresponding to the current label to the entity process sequence Y X tail, and the entity process sequence Y X Store it in the tag group entity generation list templist_new; otherwise, create a new entity process sequence and store it in the tag group entity generation list templist_new;

[0040] ③When the currently traversed tag is "EX", if there is a same type of entity process sequence Y in the text entity generation list templist X , the entity process sequence Y X Record it as to be updated, and add the character corresponding to the current label to the entity process sequence Y X The end, then the entity process sequence Y X Add to the entity set EntitySet;

[0041] ④ When the currently traversed tag is "SX", a new entity process sequence Y is created X , stored in the entity set EntitySet.

[0042] Preferably, the fully connected network consists of a Bi-LSTM network and a fully connected layer.

[0043] Preferably, optimizing the prototype network according to the loss function value is: synchronously optimizing the prototype network through an Adam iterative optimizer.

[0044] The present invention also provides a nested named entity recognition device based on a prototype network, comprising an encoder, a fully connected network, a prototype parameter mapping network, a label parsing module, a training sample construction module, and a network optimization module;

[0045] The encoder is used to encode the characters in the text sequence W i Encode and get the character encoding e i ;

[0046] The fully connected network is used to encode characters e i Extract contextual semantic information and map the dimensions through the fully connected layer to obtain the implicit vector h i ;

[0047] The prototype parameter mapping network includes 4m+1 prototype groups, each of which contains multiple prototypes; 4m+1 prototype groups correspond to 4m+1 labels; wherein 4m labels correspond to m known entity categories S Entity ; Each entity class S Entity There are four types of labels, namely B label, I label, E label and S label in BIOES labeling method. These four types of labels are marked by BIOES I BIOES and entity class S Entity Composition; the 4m+1th label corresponds to the O label in the BIOES annotation method;

[0048] The prototype parameter mapping network calculates all the implicit features output by the fully connected network h i Distance to all prototypes; calculate characters using distance W i Belongs to a tag y If the probability value is greater than the set threshold, the current character W i belong y Label, thus getting the character W i All corresponding tags form a tag group label i ; The tag groups of all characters in the text to be recognized form a tag sequence;

[0049] The label parsing module is used to perform label parsing according to the text sequence to be identified and the label sequence thereof, and identify the entity set corresponding to the text sequence to be identified;

[0050] The network optimization module is used to train the prototype network; the training samples provided by the training sample construction module are input into the prototype network, and all implicit features are obtained from the prototype parameter mapping network. h i The distance from all prototypes; substituting the distance into the loss function to calculate the loss function value, and optimizing the prototype network according to the loss function value;

[0051] The training sample construction module is used to construct a training sample set: a known training text sequence and a corresponding entity, wherein the entity includes an entity character sequence and an entity category S Entity ; Based on the training text sequence and known entities, determine each character in the training text sequence W i To which entity and its position in the entity character sequence, using the entity category S of the entity to which it belongs Entity and the BIOES identification of the position BIOES Constructs a label; one character W iCorresponding to at least one label, forming an ideal label group label i ;character W i and ideal tag group label i Composition of training samples.

[0052] Preferably, the training sample construction module determines the character W i The ideal tag set label i The method is:

[0053] For each character in the training text sequence W i , determine whether it is included in the known entity;

[0054] If the character W i Contained in a known entity, then based on the character W i The position and length of the entity character sequence, marked with BIOES identifier I BIOES , and get the character W i The entity type S of the entity Entity , composed of labels [I BIOES —S Entity ]; specifically:

[0055] ① Characters W i is the starting point of the entity character sequence and the length of the entity character sequence is greater than 1, then the corresponding label is [BS Entity ];

[0056] ② Characters W i is the end point of the entity character sequence and the length of the entity character sequence is greater than 1, then the corresponding label is [ES Entity ];

[0057] ③ Characters W i Inside the entity character sequence, and not at the start or end of the entity, the corresponding label is [IS Entity ];

[0058] ④ Characters W i is the starting point of the entity character sequence and the length of the entity character sequence is 1, then the corresponding label is [SS Entity ];

[0059] For characters W i, traverse all known entities and get the characters W i All corresponding labels constitute the ideal label group label i ; If for a character, no corresponding label can be found by traversing all known entities, the corresponding label of the character is set to O.

[0060] Preferably, the network optimization module optimizes the prototype network in the following manner:

[0061] Calculate the latent vector for each text h i Belong to the prototype m pq Probability P for:

[0062] (1)

[0063] in, x Corresponding characters W i , which also corresponds to the implicit vector h i ;prototype m pq Indicates p The first q A prototype, p [1,C], q [1, K ]; C The number of labels in the prototype parameter mapping network is also the number of prototype groups. C =4m+1; K The number of prototypes in the prototype group corresponding to each label; d (·) is the distance function; γ is a constant;

[0064] Counting Characters W i The probability value belonging to each label is:

[0065] (2)

[0066] The loss function value is calculated as:

[0067] (3)

[0068] In the above formula, N The number of characters in the text sequence of the input prototype parameter mapping network, C The number of labels in the prototype parameter mapping network is also the number of prototype groups;label i For the i The label group corresponding to the characters;

[0069] The above formula (3) shows that if C of the tags k Tags in characters W i The corresponding ideal label group label i In W i Belong to k The probability of the label participates in the accumulation of the loss function, otherwise, it does not participate in the accumulation of the loss function; traverse the characters W i ~ W N , use the above formula (3) to obtain the loss function value of the prototype network;

[0070] Optimize the prototype network according to the loss function value.

[0071] Preferably, the tag parsing method of the tag parsing module is:

[0072] Define the text entity generation list templist, the tag group entity generation list templist_new and the entity set EntitySet; traverse each tag group in the tag sequence respectively, and create a tag group entity generation list templist_new when starting to process a new tag group; for each tag in the tag group, according to the entity type S Entity , In the previous step, set the content of the text entity generation list templist and the entity set EntitySet to the label group entity generation list templist_new and update the entity set EntitySet content;

[0073] After all tags of a certain tag group are processed, the entity process sequences that have not been updated in the text entity generation list templist are added to the tag group entity generation list templist_new, and the entity process sequences whose end positions of the entity character sequences in the tag group entity generation list templist_new are less than the positions of the corresponding characters of the current tag group in the text sequence to be recognized are added to the entity set EntitySet, and these entity character sequences are deleted from the tag group entity generation list templist_new;

[0074] After processing each tag group, assign the tag group entity generation list templist_new to the text entity generation list templist, and clear templist_new;

[0075] When traversing each tag in the tag group, the processing method is as follows:

[0076] ① When the currently traversed label is "BX", X represents the entity category S Entity , create a new entity process sequence Y X , add the character corresponding to the current label to the entity process sequence Y X In the process sequence Y X Add to the tag group entity generation list templist_new;

[0077] ② When the currently traversed tag is "IX", if there is an entity process sequence Y of the same category X in the text entity generation list templist X , the entity process sequence Y X Record it as to be updated, and add the character corresponding to the current label to the entity process sequence Y X tail, and the entity process sequence Y X Store it in the tag group entity generation list templist_new; otherwise, create a new entity process sequence and store it in the tag group entity generation list templist_new;

[0078] ③When the currently traversed tag is "EX", if the text entity generates a list templist There is a sequence of entity processes of the same type Y X , the entity process sequence Y X Record it as to be updated, and add the character corresponding to the current label to the entity process sequence Y X The end, then the entity process sequence Y X Add to the entity set EntitySet;

[0079] ④ When the currently traversed tag is "SX", a new entity process sequence Y is created X , stored in the entity set EntitySet.

[0080] Beneficial effects:

[0081] (1) The present invention is aimed at the task of multi-category nested named entity recognition, and adopts a prototype network as a recognition structure. The recognition structure learns the mapping relationship between characters and label groups. The label group includes multiple labels rather than a single label. Combined with label parsing, that is, building a reverse interpretation of the label, the recognition of multi-category nested named entities is realized. This solution takes advantage of the prototype network's advantages in handling tasks with few samples / uneven sample distribution, and achieves high-accuracy recognition of multi-category nested entity recognition labels; at the same time, the present invention designs a nested entity multi-label sequence labeling method, which has richer sequence labels than traditional sequence labeling tasks, and achieves high-accuracy entity recognition under conditions with more entity category labels and more complex entity nesting. Furthermore, the solution provides unstructured text data structuring capabilities that are more in line with actual application needs, and assists in the implementation of text processing intelligent applications.

[0082] (2) The present invention proposes a prototype network optimization loss function for multi-category nested entity tag recognition. In the probability calculation formula in the loss function, the numerator corresponds to the prototype to be optimized, and the denominator corresponds to all prototypes. During the optimization process, all prototypes can be realized at the same time to make the model converge. Moreover, the training method can also optimize the basic network and prototype parameters at the same time to improve the model training efficiency.

[0083] (3) The present invention uses a prototype network for processing. The prototype network has advantages in processing small samples and long-tail distribution data. Therefore, the method based on the prototype network technology in this article naturally has the advantage of processing small samples and long-tail distribution data. Through the feature extraction network and the Bi-LSTM layer, the feature vector of the semantics of the fusion context is obtained, and the prototype network is used to directly identify the label of each character. Since different types of samples correspond to different prototypes in the prototype network, and the prototypes are independent, the result is not easily affected by the data distribution.

[0084] (4) The existing neural network solutions locate the entity and then identify the entity category. The present invention directly identifies the entity and category. For example, [CD City, LOC] is an entity and a category, thereby improving the accuracy and speed of recognition.

[0085] (5) The solution of the present invention is applicable to any natural language processing task. BRIEF DESCRIPTION OF THE DRAWINGS

[0086] Figure 1 Technical roadmap for nested named entity recognition methods based on prototype networks. DETAILED DESCRIPTION

[0087] The present invention is described in detail below with reference to the accompanying drawings and embodiments.

[0088] The present invention provides a nested named entity recognition method based on a prototype network, the basic idea of ​​which is to use a prototype network as a recognition structure, the recognition structure learns the mapping relationship between characters and label groups, and solves the problem of named entity recognition with multiple types and nested relationships. When constructing a training sample, the method constructs a label according to the position of each character in a known entity in a training text sequence and the entity type, and the label includes a BIOES annotation representing the position and the entity type. A character may correspond to multiple labels to form an ideal label group; the character and the ideal label group constitute a training sample to train the prototype network. When performing recognition, the text sequence to be recognized is input into the prototype network to generate a label group corresponding to each character; the label groups of all characters in the text to be recognized constitute a label sequence, and finally the label sequence is parsed to identify the entity. The present invention can effectively utilize the advantages of the prototype network on small sample data, improve the recognition accuracy of named entities with multiple types and nested relationships, and effectively support the recognition of multi-category nested entities.

[0089] Combine the following Figure 1 The implementation mode of the present invention is described in detail.

[0090] Step 1: Build a prototype network.

[0091] like Figure 1 As shown in FIG. 1 , the prototype network of the present invention includes an encoder, a fully connected network, and a prototype parameter mapping network. The encoder can be a Transformer encoder, and the fully connected network is composed of a Bi-LSTM network and a fully connected layer. After the text sequence is input into the prototype network, the Transformer encoder is first used to obtain the W i The encoding, i The value range is 1~ N , N is the number of characters in the text sequence. The embedding vector is input into the Bi-LSTM network to obtain the hidden feature vector, and then mapped to the same dimension as the prototype network feature through the fully connected layer, that is, the hidden feature corresponding to each word is obtained. h i .

[0092] The input of the prototype parameter mapping network is the latent feature h i , the output is a label group. The prototype parameter mapping network includes 4m+1 prototype groups, each of which contains multiple prototypes (also prototype parameters). m is the known entity category S Entity 4m+1 prototype groups correspond to 4m+1 tags. Among them, the first 4m tags correspond to m known entity categories S Entity , each entity category S EntityThere are four types of labels, namely B label, I label, E label and S label in BIOES labeling. The 4m+1th label corresponds to the O label in BIOES labeling. The label is represented by the BIOES label I BIOES and entity class S Entity Combination of. Figure 1 As shown, the entity category S Entity As shown in the figure, LOC, ORG, GPE, etc. The label is [I BIOES -S Entity ] recording methods, such as B-LOC, I-GPE, etc.

[0093] The prototype parameter mapping network also includes distance calculation and label recognition functions. Distance calculation is to calculate all the implicit features of the fully connected network output. h i The distance to all prototypes, using the distance to calculate the character W i Belongs to a tag y If the probability value is greater than the set threshold, the character W i belong y Tag. Traverse all tags to get characters W i All tags belonging to form a tag group l i . The label group of all characters in the text to be recognized l i ~ l N Composed label sequence; label group of all characters of input text sequence l 1~ l N Composition tag sequence , which is the output of the prototype parameter mapping network. Only by performing label recognition on the label sequence can the entity corresponding to the input text sequence be obtained.

[0094] The encoder, fully connected network, and prototype parameter mapping network in the prototype network need to be trained jointly to obtain parameters for effective processing and mapping.

[0095] Step 2: Construct a training sample set.

[0096] Given a training text sequence and a corresponding entity, the entity includes an entity character and an entity category S Entity , which can be expressed as E=[entity character, S Entity ], for example [CD City, LOC]. Extract a character from the training text sequence W i , such as "C", the characterW i Compare with known entities to determine the character W i To which entity it belongs (in which entity it appears), and the position in the entity character sequence where it appears. This position can determine the BIOES identifier I BIOES . Using characters W i The entity type S of the entity Entity and the identified BIOES identity I BIOES , build label [I BIOES -S Entity ]. One character W i It may belong to one or more labels. Find all the labels to which the character belongs and form a label group. Because it is training data, it is called an ideal label group. label i . A character W i And its ideal tag set label i It constitutes a training sample. Traverse each character in the training text sequence W i , using the same operation, multiple ideal label groups can be obtained label i .

[0097] Specifically, how to construct a label [I BIOES -S Entity ] and the specific steps of the tag group are:

[0098] For each character in the training text sequence W i , determine whether it is included in the known entity;

[0099] If the character W i Contained in a known entity, then based on the character W i The position and length of the entity character sequence, marked with BIOES identifier I BIOES , and get the character W i The entity type S of the entity Entity , composed of labels [I BIOES —S Entity ]; specifically:

[0100] ① Characters W i is the starting point of the entity character sequence and the length of the entity character sequence is greater than 1, then the corresponding label is [BS Entity];

[0101] ② Characters W i is the end point of the entity character sequence and the length of the entity character sequence is greater than 1, then the corresponding label is [ES Entity ];

[0102] ③ Characters W i Inside the entity character sequence, and not at the start or end of the entity, the corresponding label is [IS Entity ];

[0103] ④ Characters W i is the starting point of the entity character sequence and the entity length is 1, then the corresponding label is [SS Entity ];

[0104] For characters W i , traverse all known entities and get the characters W i All corresponding labels form an ideal label group label i ; If no corresponding label can be found for a certain character by traversing all known entities, the corresponding label of the character is set to O.

[0105] For example, Figure 1 As shown, multiple entities are shown on the right side of the picture. For the input text "CD City PQ District XY Government", the first character "C" is obtained, and the entity in which "C" appears is determined. It is found that it appears in three entities: [CD City, LOC], [CD City PQ District, LOC], and [CD City PQ District XY Government, GPE]. The positions are all at the beginning, and the entity character sequences are all greater than 1, so the BIOES identifier I BIOES All are B, entity type S Entity LOC and GPE respectively, then the labels corresponding to the character "C" are [B-LOC] and [B-GPE], and these two labels form a label group label i ={[B-LOC],[B-GPE}. "C" and label i ={[B-LOC], [B-GPE} constitutes a training sample. The label group of all characters in "CD City PQ District XY Government" can be obtained through the same operation.

[0106] Step 3: Train the prototype network.

[0107] The training samples are input into the prototype network, the loss function value between the prototype network output and the ideal label group in the training samples is calculated, and the prototype network is optimized.

[0108] In this step, the characters in the training sample are input into the prototype network, and the prototype network outputs a label group. The label group is inaccurate and needs to be optimized through residual feedback. The optimization objects include the parameters in the encoder, fully connected network and prototype parameter mapping network, so that the output results converge to the true value.

[0109] The training optimization principle is to use the distance function to calculate all the hidden features separately The distance from all prototypes is calculated through the loss function, and the loss function value of the prototype network is optimized using the loss function value. The distance function used can be the Euclidean distance or the cosine distance function, and the Adam iterative optimizer is used to synchronously optimize the prototype network.

[0110] The loss function definition and loss value calculation are as follows:

[0111] For an input text sequence , processed by the Transformer encoder to obtain the encoding of each word e i express:

[0112]

[0113] in E Represents a sequence of text T A sequence of character codes. e i Indicates i Characters W i The encoding, , s is the dimension of the embedding vector, Transformer (·) represents the embedding vector encoding extracted from the input text sequence using the Transformer encoder.

[0114] The embedded vector encoding is input into the BiLSTM network to obtain more contextual semantic information, and the dimensions are mapped through the fully connected layer to obtain the implicit vector:

[0115]

[0116] in h i Indicates i Words W i The corresponding implicit vector, , mis the dimension of the implicit vector (i.e. the prototype parameter dimension), BiLSTM (·) represents the extraction of context semantic vector, FC (·) Map the embedding vector to m Dimension.

[0117] First, calculate each latent vector h i Belong to the prototype m pq Probability P for:

[0118] (1)

[0119] in, x Representing characters W i ;prototype m pq Indicates p The first q A prototype, p [1,C], q [1, K ]; C The number of labels in the prototype parameter mapping network is also the number of prototype groups. C =4m+1; K The number of prototypes in the prototype group corresponding to each label; d (·) is the distance function; γ is a constant. In this formula, the numerator corresponds to the prototype to be optimized, and the denominator corresponds to all prototypes. During the optimization process, all prototypes can be realized at the same time to make the model converge.

[0120] Then character W i ( x ) belongs to a certain label:

[0121] (2)

[0122] The loss function value is calculated as follows:

[0123] (3)

[0124] In the above formula, N The number of characters in the text sequence of the input prototype parameter mapping network, C The number of labels in the prototype parameter mapping network is also the number of prototype groups; label i For characters W iThe corresponding ideal label group.

[0125] The above formula (3) shows that if C of the tags k Tags in characters W i The corresponding ideal label group label i In W i Belong to k The probability of the label participates in the accumulation of the loss function, otherwise, it does not participate in the accumulation of the loss function; traverse the characters W i ~ W N , use the above formula (3) to obtain the loss function value of the prototype network.

[0126] After training, testing is required. For each text sequence in the test set, the prototype network is used to predict the label group of each character to form a multi-category entity label sequence. Determine whether the recognition accuracy meets the requirements. Based on the judgment result, further training is performed, or training is completed and the next step is entered.

[0127] Step 4: When actually performing recognition, the text sequence to be recognized is input into the prototype network to generate each character W i Corresponding tag group l i ; Label group for all characters of the text to be recognized l i ~ l N Composed tag sequence.

[0128] In this step, for the text sequence to be recognized, the implicit vector is obtained by using the above steps , and calculate each character separately (here using x The corresponding hidden features h i With all prototypes m pq Distance d ( h i , m pq ), using distance to calculate characters x Belongs to a tag y The probability value .like If it is greater than the threshold, the current character x Belongs to the tag y ,Will y Record as character x A label belonging to ;Otherwise it does not belong to the tag y . Traverse all tags to get characters x All tags belonging to form a tag group . Perform the above processing on each character in the input text sequence to obtain a multi-category entity label sequence In this tag sequence l i For characters W i Then this step obtains the text sequence to be recognized. W 1 ~W N The corresponding tag sequence .

[0129] Step 5: Perform label parsing on the label sequence to obtain multi-category nested named entities.

[0130] Step 4: Get the text sequence to be recognized W 1 ~W N The corresponding tag sequence After that, it is not possible to directly identify W 1 ~W N Corresponding entities need to be parsed for the label sequence. This step uses an entity generation list to record possible entity information, distinguishes different entity categories, creates, expands, and ends entity information of the same category, parses to obtain nested entities of different categories, and stores them in the entity collection. By traversing all multi-category entity label sequences, all entities in the sequence can be obtained.

[0131] The specific parsing process is:

[0132] Define the text entity generation list templist, the tag group entity generation list templist_new and the entity set EntitySet. The text entity generation list templist records the entity process sequence generated in the process of processing the tag groups corresponding to each character in the text sequence, the tag group entity generation list templist_new records the entity process sequence generated in the process of a tag group, and the entity set EntitySet stores the entities that have been parsed before parsing the current character. Initialize the text entity generation list templist, the tag group entity generation list templist_new and the entity set EntitySet to be empty. Traverse each tag group in the tag sequence separately. It is known that each tag group corresponds to a character in the text sequence to be recognized. When processing each new tag group, create a tag group entity generation list templist_new. For each tag in the tag group, according to its entity type S Entity, update the entity generation list templist_new and the entity set EntitySet with the content of the previous-step entity generation list templist and the entity set;

[0133] Further, after processing all the tags in a certain tag group, add the entity process sequences in the text entity generation list templist that have not been updated to the tag group entity generation list templist_new, and add the entity process sequences in the tag group entity generation list templist_new whose ending position of the entity character sequence is less than the position of the corresponding character of the current tag group in the text sequence to be recognized to the entity set EntitySet. For example, if the currently processed character "市" is the third character, and there are still other entity process sequences with 1 or 2 characters that have not been updated in the templist_new list, such as [CD, LOC], then add these shorter entities to the EntitySet and delete these entity character sequences from the tag group entity generation list templist_new.

[0134] After processing each tag group, assign the content in the tag group entity generation list templist_new to the text entity generation list templist and clear templist_new;

[0135] When traversing each tag in the tag group, the processing method is as follows:

[0136] Case (1) For the tags corresponding to the four BIOES annotation methods of B, I, E, and S:

[0137] ① When the currently traversed tag is "B-X", where X represents the entity category S Entity , create a new entity process sequence Y X (here it is called "entity process sequence" in view of the current intermediate process of parsing entities), add the character corresponding to the tag (such as "C") to the entity process sequence Y X into it, and add the entity process sequence Y X to the tag group entity generation list templist_new; the format of this entity process sequence is the same as that of the前述 entity, including the entity character sequence and the corresponding entity category S Entity , and the recording method can still be [entity character sequence, LOC], such as [CD, LOC];

[0138] ② When the currently traversed tag is "I-X", if there is an entity process sequence Y of the same category X in the text entity generation list templist , then add the character corresponding to the tag to this entity process sequence Y X , XThe tail is stored in the tag group entity generation list templist_new; otherwise, a new entity process sequence is created and stored in the tag group entity generation list templist_new;

[0139] ③When the currently traversed tag is "EX", if the text entity generates a list templist There is a sequence of entity processes of the same type Y X , then add the character corresponding to the label to the entity process sequence Y X The end, then the entity process sequence Y X Add to the entity set EntitySet;

[0140] ④ When the currently traversed tag is "SX", a new entity process sequence Y is created X , this Yx is an entity recognition result, which is stored in the entity set EntitySet;

[0141] Case (2) When the currently traversed tag is the O tag, all sequences in the text entity generation list templist are added to the entity collection, and the temporary entity list is cleared.

[0142] See below Figure 1 , give an example to illustrate the above parsing process:

[0143] (1) For the character "C", it includes two labels: B-LOC and B-GPE:

[0144] ——B-LOC belongs to the "BX" class tag, create a new entity process sequence Y LOC ={[C,LOC]}, Y LOC Add to templist_new; templist_new = {[C, LOC]};

[0145] ——B-GPE belongs to the "BX" class tag, create a new entity process sequence Y GPE ={[C, GPE]}, Y GPE Add to templist_new; templist_new = {[C, LOC], [C, GPE]};

[0146] ——Let templist=templist_new, templist={[C, LOC], [C, GPE]}.

[0147] (2) For the character "D", it includes two labels: I-LOC and I-GPE:

[0148] ——I-LOC belongs to the "I-X" type of label, and there is already an entity process sequence Y of the same type in templist LOC ={[C, LOC]}, add "D" to Y LOC , Y LOC ={[CD, LOC]}, add to templist_new; templist_new = {[CD, LOC]};

[0149] ——I-GPE belongs to the "I-X" type of label, and there is already an entity sequence Y of the same type in templist GPE ={[C, GPE]}, add "D" to Y GPE , Y GPE ={[CD, GPE]}, add to templist_new; templist_new = {[CD, LOC], [CD, GPE]};

[0150] ——Let templist = templist_new, templist = {[CD, LOC], [CD, GPE]}.

[0151] (3)For the character "市", it contains three labels: I-LOC, E-LOC, and I-GPE

[0152] ——I-LOC belongs to the "I-X" type of label, and there is already an entity sequence Y of the same type in templist LOC ={[CD, LOC]}, add "市" to Y LOC , Y LOC ={[CD市, LOC]}, add to templist_new; templist_new = {[CD市, LOC]};

[0153] ——E-LOC belongs to the "E-X" type of label, and there is already an entity sequence Y of the same type in templist LOC ={[CD市, LOC]}, add "市" to Y LOC , Y LOC ={[CD市, LOC], add to the entity set EntitySet = {[CD市, LOC]}; templist_new remains {[CD市, LOC]};

[0154] ——I-GPE belongs to the "I-X" type of label, and there is already an entity sequence Y of the same type in templist GPE ={[CD, GPE]}, add "市" to Y GPE , Y GPE={[CD City, GPE], add to templist_new; templist_new={[CD City, LOC], [CD City, GPE]};

[0155] ——Let templist=templist_new, now templist={[CD City, LOC], [CD City, GPE]}, EntitySet={[CD City, LOC]}.

[0156] (4) For the character "P", it contains the labels B-LOC, I-LOC, B-GPE, I-GPE

[0157] ——B-LOC belongs to the "BX" class tag, create a new entity process sequence Y LOC , add "P" to Y LOC , Y LOC ={[P,LOC]}, add to templist_new; templist_new={[P,LOC]};

[0158] ——I-LOC belongs to the "IX" class tag, and the same type of entity sequence Y already exists in templist LOC ={[CD City, LOC]}, add "P" to Y LOC , Y LOC ={[CD City P, LOC], Y LOC =[CD City P, LOC]} add to templist_new; templist_new={[P, LOC],[CD City P, LOC]};

[0159] ——B-GPE belongs to the "BX" class tag, create a new entity process sequence Y GPE , add "P" to Y GPE , Y GPE ={[P, GPE]}, add to templist_new; templist_new={[P, LOC], [CD City P, LOC], [P, GPE]};

[0160] ——I-GPE belongs to the "IX" class tag, and the same type of entity process sequence Y already exists in templist GPE ={[CD City, GPE]}, add "P" to Y GPE , Y GPE ={[CD City P, GPE]}, Y GPE Add to templist_new; templist_new = {[P, LOC], [CD City P, LOC], [P, GPE], [CD City P, GPE]};

[0161] - Update templist_new to templist, templist = {[P, LOC], [CD City P, LOC], [P, GPE], [CD City P, GPE]}, EntitySet = {[CD City, LOC]};

[0162] (5)For the character "Q", it contains tags I-LOC, I-GPE

[0163] - I-LOC belongs to the "I-X" type of tags, and Y already exists in templist LOC ={[P, LOC], [CD City P, LOC]}, add "Q" to Y LOC in, Y LOC ={[PQ, LOC], [CD City PQ, LOC]}, Y LOC Add to templist_new; templist_new = {[PQ, LOC], [CD City PQ, LOC]};

[0164] - I-GPE belongs to the "I-X" type of tags, and Y already exists in templist GPE ={[P, GPE], [CD City P, GPE]}, add "Q" to Y GPE in; Y GPE ={[PQ, GPE], [CD City PQ, GPE]}, Y GPE Add to templist_new; templist_new = {[PQ, LOC], [CD City PQ, LOC], [PQ, GPE], [CD City PQ, GPE]};

[0165] - templist = templist_new (update operation); templist = {[PQ, LOC], [CD City PQ, LOC], [PQ, GPE], [CD City PQ, GPE]}, EntitySet = {[CD City, LOC]}

[0166] (6)For the character "District", it includes tags E-LOC, I-GPE:

[0167] - E-LOC, belongs to the "E-X" type of tags, and Y already exists in templist LOC ={[PQ, LOC], [CD City PQ, LOC]}, add "District" to Y LOC , Y LOC ={[PQ District, LOC], [CD City PQ District, LOC]}, add Y LOCAdded to the entity set EntitySet = {[CD City, LOC], [PQ District, LOC], [CD City PQ District, LOC]};

[0168] ——I-GPE, belonging to the "I-X" type of label, and there is already Y in templist GPE ={[PQ, GPE], [CD City PQ, GPE]}, add "District" to Y GPE , Y GPE ={[PQ District, GPE], [CD City PQ District, GPE]}, Y GPE Added to templist_new. templist_new = {[PQ District, GPE], [CD City PQ District, GPE]};

[0169] ——templist = templist_new; templist = {[PQ District, GPE], [CD City PQ District, GPE]}, EntitySet = {[CD City, LOC], [PQ District, LOC], [CD City PQ District, LOC]}.

[0170] The processing method of "XY Government" in the text sequence is similar and will not be elaborated.

[0171] (7) After processing all the labels in a certain label group, the following operations can solve the situation of lacking the label "E-X". For example:

[0172] If the character "City" includes one label of I-GPE, at this time templist = {[CD, LOC], [CD, GPE]}, EntitySet = {empty}.

[0173] ——I-GPE belongs to the "I-X" type of label, and there is already an entity sequence Y of the same type in templist GPE ={[CD, GPE]}, add "City" to Y GPE , Y GPE ={[CD City, GPE]}, added to templist_new; templist_new = {[CD City, GPE]}; [CD, GPE] in templist is the entity process sequence to be updated, while [CD, LOC] is the entity process sequence not to be updated;

[0174] Add the unupdated entity process sequence [CD, LOC] to templist_new, where templist_new = {[CD City, GPE], [CD, LOC]}. Through post-processing operations, add [CD, LOC] to EntitySet and delete it from templist_new, so templist_new = {[CD City, GPE]}. After assignment, templist = {[CD City, GPE]}, and EntitySet = {[CD, LOC]}.

[0175] (8) When the label is "O":

[0176] If the character "City" corresponds to the label O, templist = {[CD, LOC], [CD, GPE]}. Add all sequences in templist to the entity set and clear templist. Get templist = {}, and EntitySet = {[CD, LOC], [CD, GPE]}.

[0177] So far, this process ends.

[0178] Based on the above solution, the present invention also provides a nested named entity recognition device based on a prototypical network, including an encoder, a fully connected network, a prototype parameter mapping network, a label parsing module, a training sample construction module, and a network optimization module;

[0179] The encoder is used to encode the characters in the text sequence W i to obtain character encodings e i ;

[0180] The fully connected network is used to extract context semantic information from the character encodings e i and map the dimensions through a fully connected layer to obtain hidden vectors h i ;

[0181] The prototype parameter mapping network includes 4m + 1 prototype groups, and each prototype group contains multiple prototypes; the 4m + 1 prototype groups correspond to 4m + 1 labels; among them, 4m labels correspond to m known entity categories S Entity ; each entity category S Entity corresponds to 4 labels, namely the B label, I label, E label, and S label in the BIOES annotation method. These four types of labels are composed of the BIOES identifier I BIOES and the entity category S Entity ; the 4m + 1th label corresponds to the O label in the BIOES annotation method;

[0182] The prototype parameter mapping network calculates all the hidden features output by the fully connected network h i Distance to all prototypes; calculate characters using distance W i Belongs to a tag If the probability value is greater than the set threshold, the current character W i belong Label, thus getting the character W i All corresponding tags form a tag group label i ; The tag groups of all characters in the text to be recognized form a tag sequence;

[0183] A label parsing module is used to perform label parsing according to the text sequence to be identified and the label sequence thereof, and identify the entity set corresponding to the text sequence to be identified;

[0184] The network optimization module is used to train the prototype network; the training samples provided by the training sample construction module are input into the prototype network, and all implicit features are obtained from the prototype parameter mapping network. h i The distance from all prototypes; substituting the distance into the loss function to calculate the loss function value, and optimizing the prototype network according to the loss function value;

[0185] The training sample construction module is used to construct a training sample set: the training text sequence and the corresponding entity are known, and the entity includes the entity character sequence and the entity category S Entity ; Based on the training text sequence and known entities, determine each character in the training text sequence W i To which entity and its position in the entity character sequence, using the entity category S of the entity to which it belongs Entity and the BIOES identification of the position BIOES Constructs a label; one character W i Corresponding to at least one label, forming an ideal label group label i ;character W i and ideal tag group label i Composition of training samples.

[0186] Training sample building module determines the character W i The ideal tag set label iThe method of optimizing the prototype network by the network optimization module and the label parsing method of the label parsing module have been described above and will not be repeated here.

[0187] The above specific embodiments only describe the design principle of the present invention. The shapes and names of the components in the description may be different and are not limited. Therefore, those skilled in the art in the field of the present invention may modify or replace the technical solutions recorded in the above embodiments; and these modifications and replacements do not deviate from the creative purpose and technical solutions of the present invention and should all fall within the protection scope of the present invention.

Claims

1. A nested named entity recognition method based on prototype network, It is characterized in that include: Step 1: Construct a prototype network including an encoder, a fully connected network, and a prototype parameter mapping network; The prototype parameter mapping network includes 4m+1 prototype groups, each of which contains multiple prototypes; 4m+1 prototype groups correspond to 4m+1 labels; wherein 4m labels correspond to m known entity categories S Entity ; Each entity class S Entity There are four types of labels, namely B label, I label, E label and S label in BIOES labeling method. These four types of labels are marked by BIOES I BIOES and entity class S Entity Composition; the 4m+1th label is the O label in the BIOES annotation method; Step 2: Construct a training sample set: Given a training text sequence and a corresponding entity, the entity includes an entity character sequence and an entity category S Entity ; Based on the training text sequence and known entities, determine each character in the training text sequence W i To which entity and its position in the entity character sequence, using the entity category S of the entity to which it belongs Entity and the BIOES identification of the position BIOES Constructs a label; one character W i Corresponding to at least one label, forming an ideal label group label i ;character W i and ideal tag group label i Composition of training samples; Step 3: Train the prototype network: Input the training samples into the prototype network, calculate the loss function value between the output of the prototype network and the ideal label group in the training samples, and optimize the prototype network. The optimization of the prototype network is as follows: the prototype parameter mapping network calculates all the implicit features output by the fully connected network h i The distance from all prototypes, the loss function value of the prototype network is calculated through the loss function, and the prototype network is optimized according to the loss function value; First, calculate each latent feature h i Belong to the prototype m pq Probability P for: (1) in, x Representing characters W i ;prototype m pq Indicates p The first q A prototype, p [1,C], q [1, K ]; C The number of labels in the prototype parameter mapping network is also the number of prototype groups. C =4m+1; K The number of prototypes in the prototype group corresponding to each label; d (·) is the distance function; γ is a constant; Counting Characters W i The probability value belonging to each label is: (2) The loss function value is calculated as: (3) In the above formula, N The number of characters in the text sequence of the input prototype parameter mapping network, C The number of labels in the prototype parameter mapping network is also the number of prototype groups; label i For characters W i The corresponding ideal label group; The above formula (3) shows that if C of the tags k Tags in characters W i The corresponding ideal label group label i In W i Belong to k The probability of the label participates in the accumulation of the loss function, otherwise, it does not participate in the accumulation of the loss function; traverse the characters W i ~ W N , use the above formula (3) to obtain the loss function value of the prototype network; Optimize the prototype network according to the loss function value; Step 4: When performing recognition, input the text sequence to be recognized into the prototype network to generate each character W i Corresponding tag group l i ; The label groups of all characters in the text sequence to be recognized constitute a label sequence; Step 5: Perform label parsing based on the text sequence to be identified and its label sequence to identify the entity set corresponding to the text sequence to be identified.

2. The nested named entity recognition method based on prototype network as claimed in claim 1, It is characterized in that In step 2, the build tag is: For each character in the training text sequence W i , determine whether it is included in the known entity; If the character W i Contained in a known entity, then based on the character W i The position and length of the entity character sequence, marked with BIOES identifier I BIOES , and get the character W i The entity type S of the entity Entity , composed of labels [I BIOES -S Entity ]; specifically: ① Character W i is the starting point of the entity character sequence and the length of the entity character sequence is greater than 1, then the corresponding label is [BS Entity ]; ②Characters W i is the end point of the entity character sequence and the length of the entity character sequence is greater than 1, then the corresponding label is [ES Entity ]; ③ Characters W i Inside the entity character sequence, and not at the start or end of the entity, the corresponding label is [IS Entity ]; ④ Characters W i is the starting point of the entity character sequence and the length of the entity character sequence is 1, then the corresponding label is [SS Entity ]; For characters W i , traverse all known entities and get the characters W i All corresponding labels constitute the ideal label group label i ; If for a character, no corresponding label can be found by traversing all known entities, the corresponding label of the character is set to O.

3. The nested named entity recognition method based on prototype network as claimed in claim 1, It is characterized in that The tag parsing method described in step 5 is: Define the text entity generation list templist, the tag group entity generation list templist_new and the entity set EntitySet; traverse each tag group in the tag sequence respectively, and create a tag group entity generation list templist_new when starting to process a new tag group; for each tag in the tag group, according to the entity type S Entity , In the previous step, set the content of the text entity generation list templist and the entity set EntitySet to the label group entity generation list templist_new and update the entity set EntitySet content; After all tags of a certain tag group are processed, the entity process sequences that have not been updated in the text entity generation list templist are added to the tag group entity generation list templist_new, and the entity process sequences whose end positions of the entity character sequences in the tag group entity generation list templist_new are less than the positions of the corresponding characters of the current tag group in the text sequence to be recognized are added to the entity set EntitySet, and these entity character sequences are deleted from the tag group entity generation list templist_new; After processing each tag group, assign the tag group entity generation list templist_new to the text entity generation list templist, and clear templist_new; When traversing each tag in the tag group, the processing method is as follows: ①When the currently traversed tag is "BX", X represents the entity category S Entity , create a new entity process sequence Y X , add the character corresponding to the current label to the entity process sequence Y X In the process sequence Y X Add to the tag group entity generation list templist_new; ② When the currently traversed tag is "IX", if there is an entity process sequence Y of the same category X in the text entity generation list templist X , the entity process sequence Y X Record it as to be updated, and add the character corresponding to the current label to the entity process sequence Y X tail, and the entity process sequence Y X Store it in the tag group entity generation list templist_new; otherwise, create a new entity process sequence and store it in the tag group entity generation list templist_new; ③When the currently traversed tag is "EX", if there is a same type of entity process sequence Y in the text entity generation list templist X , the entity process sequence Y X Record it as to be updated, and add the character corresponding to the current label to the entity process sequence Y X The end, then the entity process sequence Y X Add to the entity set EntitySet; ④ When the currently traversed tag is "SX", a new entity process sequence Y is created X , stored in the entity set EntitySet.

4. The nested named entity recognition method based on prototype network as claimed in claim 1, It is characterized in that The fully connected network consists of a Bi-LSTM network and a fully connected layer.

5. The method for nested named entity recognition based on prototype network as claimed in claim 1, It is characterized in that Optimizing the prototype network according to the loss function value is: synchronously optimizing the prototype network through the Adam iterative optimizer.

6. A nested named entity recognition device based on prototype network, It is characterized in that It includes encoder, fully connected network, prototype parameter mapping network, label parsing module, training sample construction module, and network optimization module; The encoder is used to encode the characters in the text sequence W i Encode and get the character encoding e i ; The fully connected network is used to encode characters e i Extract contextual semantic information and map the dimensions through the fully connected layer to obtain the implicit vector h i ; The prototype parameter mapping network includes 4m+1 prototype groups, each of which contains multiple prototypes; 4m+1 prototype groups correspond to 4m+1 labels; wherein 4m labels correspond to m known entity categories S Entity ; Each entity class S Entity There are four types of labels, namely B label, I label, E label and S label in BIOES labeling method. These four types of labels are marked by BIOES I BIOES and entity class S Entity Composition; the 4m+1th label corresponds to the O label in the BIOES annotation method; The prototype parameter mapping network calculates all the implicit features output by the fully connected network h i Distance to all prototypes; calculate characters using distance W i Belongs to a tag y If the probability value is greater than the set threshold, the current character W i belong y Label, thus getting the character W i All corresponding tags form a tag group label i ; The tag groups of all characters in the text to be recognized form a tag sequence; The label parsing module is used to perform label parsing according to the text sequence to be identified and the label sequence thereof, and identify the entity set corresponding to the text sequence to be identified; The network optimization module is used to train the prototype network; the training samples provided by the training sample construction module are input into the prototype network, and all implicit features are obtained from the prototype parameter mapping network. h i The distance from all prototypes; substituting the distance into the loss function to calculate the loss function value, and optimizing the prototype network according to the loss function value; The training sample construction module is used to construct a training sample set: a known training text sequence and a corresponding entity, wherein the entity includes an entity character sequence and an entity category S Entity ; Based on the training text sequence and known entities, determine each character in the training text sequence W i To which entity and its position in the entity character sequence, using the entity category S of the entity to which it belongs Entity and the BIOES identification of the position BIOES Constructs a label; one character W i Corresponding to at least one label, forming an ideal label group label i ;character W i and ideal tag group label i Composition of training samples; The network optimization module optimizes the prototype network in the following way: Calculate the latent vector for each text h i Belong to the prototype m pq Probability P for: (1) in, x Corresponding characters W i , which also corresponds to the implicit vector h i ;prototype m pq Indicates p The first q A prototype, p [1,C], q [1, K ]; C The number of labels in the prototype parameter mapping network is also the number of prototype groups. C =4m+1; K The number of prototypes in the prototype group corresponding to each label; d (·) is the distance function; γ is a constant; Counting Characters W i The probability value belonging to each label is: (2) The loss function value is calculated as: (3) In the above formula, N The number of characters in the text sequence of the input prototype parameter mapping network, C The number of labels in the prototype parameter mapping network is also the number of prototype groups; label i For the i The label group corresponding to the characters; The above formula (3) shows that if C of the tags k Tags in characters W i The corresponding ideal label group label i In W i Belong to k The probability of the label participates in the accumulation of the loss function, otherwise, it does not participate in the accumulation of the loss function; traverse the characters W i ~ W N , use the above formula (3) to obtain the loss function value of the prototype network; Optimize the prototype network according to the loss function value.

7. The nested named entity recognition device based on prototype network as claimed in claim 6, It is characterized in that The training sample building module determines the character W i The ideal tag set label i The method is: For each character in the training text sequence W i , determine whether it is included in the known entity; If the character W i Contained in a known entity, then based on the character W i The position and length of the entity character sequence, marked with BIOES identifier I BIOES , and get the character W i The entity type S of the entity Entity , composed of labels [I BIOES —S Entity ]; specifically: ① Character W i is the starting point of the entity character sequence and the length of the entity character sequence is greater than 1, then the corresponding label is [BS Entity ]; ②Characters W i is the end point of the entity character sequence and the length of the entity character sequence is greater than 1, then the corresponding label is [ES Entity ]; ③ Characters W i Inside the entity character sequence, and not at the start or end of the entity, the corresponding label is [IS Entity ]; ④ Characters W i is the starting point of the entity character sequence and the length of the entity character sequence is 1, then the corresponding label is [SS Entity ]; For characters W i , traverse all known entities and get the characters W i All corresponding labels constitute the ideal label group label i ; If for a character, no corresponding label can be found by traversing all known entities, the corresponding label of the character is set to O.

8. The nested named entity recognition device based on prototype network as claimed in claim 6, It is characterized in that The tag parsing method of the tag parsing module is: Define the text entity generation list templist, the tag group entity generation list templist_new and the entity set EntitySet; traverse each tag group in the tag sequence respectively, and create a tag group entity generation list templist_new when starting to process a new tag group; for each tag in the tag group, according to the entity type S Entity , In the previous step, set the content of the text entity generation list templist and the entity set EntitySet to the label group entity generation list templist_new and update the entity set EntitySet content; After all tags of a certain tag group are processed, the entity process sequences that have not been updated in the text entity generation list templist are added to the tag group entity generation list templist_new, and the entity process sequences whose end positions of the entity character sequences in the tag group entity generation list templist_new are less than the positions of the corresponding characters of the current tag group in the text sequence to be recognized are added to the entity set EntitySet, and these entity character sequences are deleted from the tag group entity generation list templist_new; After processing each tag group, assign the tag group entity generation list templist_new to the text entity generation list templist, and clear templist_new; When traversing each tag in the tag group, the processing method is as follows: ①When the currently traversed tag is "BX", X represents the entity category S Entity , create a new entity process sequence Y X , add the character corresponding to the current label to the entity process sequence Y X In the process sequence Y X Add to the tag group entity generation list templist_new; ② When the currently traversed tag is "IX", if there is an entity process sequence Y of the same category X in the text entity generation list templist X , the entity process sequence Y X Record it as to be updated, and add the character corresponding to the current label to the entity process sequence Y X tail, and the entity process sequence Y X Store it in the tag group entity generation list templist_new; otherwise, create a new entity process sequence and store it in the tag group entity generation list templist_new; ③When the currently traversed tag is "EX", if the text entity generates a list templist There is a sequence of entity processes of the same type Y X , the entity process sequence Y X Record it as to be updated, and add the character corresponding to the current label to the entity process sequence Y X The end, then the entity process sequence Y X Add to the entity set EntitySet; ④ When the currently traversed tag is "SX", a new entity process sequence Y is created X , stored in the entity set EntitySet.

Citation Information

Patent Citations

  • A Nested Named Entity Recognition Method Based on Relation Classification and Sequence Labeling

    CN114692636B

  • Named entity identification method oriented to field of network security

    CN111709241A

  • Chinese nested named entity recognition method integrated with word boundary information

    CN114386417A

  • Named entity recognition method capable of being used for small sample scene

    CN118114674A

  • Aggregation cross-entropy loss function-based sequence recognition method

    WO2020248471A1