Discontinuous entity recognition method and device based on perception enhancement and storage medium

Abstract

The invention discloses a non-continuous entity recognition method and device based on perception enhancement and a storage medium, belongs to the technical field of natural language processing, and solves the problems that an existing non-continuous named entity recognition method is large in annotation data resource demand quantity and has decoding ambiguity. According to the method, discontinuous entity recognition is divided into an entity detection part and an entity classification part, and the entity detection part fuses word continuous type prompt perception embedding in an embedding layer of an encoder so as to ensure that an entity sequence obtained through prediction is output according to the sequence in a text sequence; the entity classification part provides additional prompt knowledge based on the entity sequence obtained by the entity detection part, performs entity information completion on the text sequence, enriches discontinuous entity information, reduces the difficulty of the entity classification stage, can effectively avoid the problems of overlarge consumption of annotated data resources, decoding ambiguity and the like, and improves the decoding efficiency. And the discontinuous entity recognition accuracy in a low data resource scene is improved.

Description

Technical Field

[0001] The present invention belongs to the technical field of natural language processing, and relates to a discontinuous entity recognition method, device and storage medium based on perception enhancement. Background Art

[0002] Named entity recognition refers to identifying entities with special meanings in text. As a basic work in the field of natural language processing, named entity recognition is an important preprocessing step for downstream tasks such as relationship extraction, text summarization, and intelligent question answering. At present, most named entity recognition methods assume that entities are composed of continuous words, use sequence labeling based on BIO tags, classify each word in the text, and obtain entities of specific types by combining word classification results.

[0003] Different from ordinary named entities, non-continuous named entities refer to entities composed of discontinuous words. However, most of the above-mentioned sequence labeling methods based on BIO tags are difficult to solve such irregular entity recognition tasks. In recent years, the mainstream methods for discontinuous entity recognition mainly include entity parsing methods based on state transition and sequence labeling methods based on label expansion. The entity parsing method based on state transition realizes the parsing of discontinuous entities by predicting different actions of different words. For example, the document "An effective transition-based model for discontinuous NER" (Dai X, Karimi S, Hachey B, et al. arXiv preprint arXiv:2004.13454, 2020.) uses six types of action labels such as SHIFT, OUT, COMPLETE, REDUCE, RIGHT-REDUCE, and LEFT-REDUCE to predict different actions of words, realize the merging of different words, and obtain non-continuous entities. Sequence annotation methods based on label extension extend BIO labels to represent non-continuous named entities. For example, the paper "Concept Identification and Normalisation for Adverse Drug Event Discovery in Medical Forums" (Metke-Jimenez A, Karimi S. BMDID@ISWC.2016.) extends BIO labels into four types of labels, BI, IH, BD and ID, to represent the different head and tail parts of non-continuous named entities. However, the above methods either require a large amount of labeled data for training, which requires a large amount of data resources, or there is decoding ambiguity, and cannot be applied to low-data resource scenarios with strong characteristics in the fields of biomedicine, military defense, etc. Summary of the invention

[0004] The technical solution of the present invention is used to solve the problems of large demand for annotation data resources and decoding ambiguity in existing non-continuous named entity recognition methods.

[0005] The present invention solves the above technical problems through the following technical solutions:

[0006] A non-continuous entity recognition method based on perception enhancement comprises the following steps:

[0007] S1. Use the Embedding-1 embedding layer on the text sequence to map the words of the text sequence into embedding vectors.

[0008] S2. Use the Transformer-encoder-1 encoder to encode the embedding vector output by the Embedding-1 embedding layer;

[0009] S3. Decode the encoder output and entity sequence using the Transformer-decoder-1 decoder;

[0010] S4, pass the decoder output through a linear mapping layer, use the softmax function to get the decoded character, and add it to the decoded entity sequence;

[0011] S5, looping steps S3 to S4 until the end marker is decoded to obtain an entity sequence corresponding to the text sequence;

[0012] S6, supplementing entity information according to the text sequence and the entity sequence, converting the non-continuous entity segments in the text sequence into continuous entity segments, and obtaining an entity text sequence;

[0013] S7. Use the Embedding-2 embedding layer on the entity text sequence to map the words of the entity text sequence into embedding vectors;

[0014] S8. Encode the embedding vector output by the Embedding-2 embedding layer using the Transformer-encoder-2 encoder, and then average the entity component character vectors to obtain the entity type feature vector;

[0015] S9. Pass the entity type feature vector through a linear mapping layer, use the softmax function to obtain the entity category prediction value, and obtain the entity prediction result.

[0016] Furthermore, the mapping of words in the text sequence into embedding vectors in step S1 is specifically as follows:

[0017] E=Word_Emb(X)+Position_Emb(X)+Perception_Emb(X) (1)

[0018] Where E = {e1, e2, ..., e n} represents the embedding vector sequence after the mapping of the Embedding-1 embedding layer, Word_Emb(·) represents the word semantic embedding of the text sequence, Position_Emb(·) represents the word position embedding of the text sequence, Perception_Emb(·) represents the word cue-aware embedding of the text sequence, X = {x1, x2, ..., x n} represents a text sequence, and n represents the length of the text sequence.

[0019] Furthermore, the encoding of the embedding vector output by the Embedding-1 embedding layer using the Transformer-encoder-1 encoder in step S2 is specifically as follows:

[0020] H=Transformer-encoder-1(E) (2)

[0021] Where H = {h1, h2, ..., h n} represents the output of the embedding vector sequence after being encoded by the Transformer-encoder-1 encoder.

[0022] Furthermore, the use of the Transformer-decoder-1 decoder to decode the encoder output and the entity sequence in step S3 is specifically as follows:

[0023]

[0024] in, represents the output of the Transformer-decoder-1 decoder, Represents the decoded entity sequence, <bos>Represents the start marker, and t represents the length of the decoded entity sequence.

[0025] Furthermore, the characters decoded in step S4 are specifically:

[0026]

[0027] in, represents the character decoded in step t+1, and Represents a trainable parameter.

[0028] Furthermore, the mapping of words of the entity text sequence into embedding vectors in step S7 is specifically as follows:

[0029]

[0030] in, represents the embedding vector sequence after being mapped by the Embedding-2 embedding layer, Word_Emb(·) represents the word semantic embedding of the text sequence, Position_Emb(·) represents the word position embedding of the text sequence, and Entity_Position_Emb(·) represents the entity position-aware embedding of the text sequence. represents the entity text sequence, and m represents the length of the entity text sequence.

[0031] Furthermore, the calculation formula of the entity type feature vector in step S8 is as follows:

[0032]

[0033] in, represents the entity type feature vector, k represents the number of entities, and Mean(·) represents the average of the entity component character vectors.

[0034] Furthermore, the calculation formula of the entity category prediction value in step S9 is as follows:

[0035]

[0036] in, represents the predicted value of entity category, and Represents a trainable parameter.

[0037] A device includes a memory and a processor, wherein the memory is used to store a program that supports the processor to execute the non-continuous entity recognition method based on perception enhancement, and the processor is configured to execute the program stored in the memory.

[0038] A storage medium stores a computer program, which executes the steps of the discontinuous entity recognition method based on perception enhancement when the computer program is executed by a processor.

[0039] The advantages of the present invention are as follows: the present invention divides non-continuous entity recognition into two parts: entity detection and entity classification. The entity detection part integrates word continuous prompt-aware embedding in the embedding layer of the encoder to ensure that the predicted entity sequence is output in the order of precedence in the text sequence; further utilizing the encoder-decoder architecture, it is possible to realize entity fragment prediction based on a small amount of labeled data; the entity classification part provides additional prompt knowledge based on the entity sequence obtained by the entity detection part, completes the entity information of the text sequence, enriches the non-continuous entity information, obtains the entity text sequence, and reduces the difficulty of the entity classification stage; at the same time, it further integrates the entity position-aware embedding based on the entity position-aware embedding, enhances the representation information of the entity text sequence, and realizes entity type prediction based on the encoder architecture, which can effectively avoid the problems of excessive consumption of labeled data resources and decoding ambiguity, and improve the accuracy of non-continuous entity recognition in low data resource scenarios. BRIEF DESCRIPTION OF THE DRAWINGS

[0040] Figure 1 is a flow chart of a non-continuous entity recognition method based on perception enhancement according to an embodiment of the present invention;

[0041] Figure 2 It is a structural diagram of a discontinuous entity recognition model according to an embodiment of the present invention. DETAILED DESCRIPTION

[0042] In order to make the purpose, technical solution and advantages of the embodiments of the present invention clearer, the technical solution in the embodiments of the present invention will be clearly and completely described below in combination with the embodiments of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0043] The technical solution of the present invention is further described below in conjunction with the accompanying drawings and specific embodiments:

[0044] Embodiment 1

[0045] like Figure 1 Specifically, a method for discontinuous entity recognition based on perception enhancement is disclosed, comprising the following steps:

[0046] S1. Use the Embedding-1 embedding layer on the text sequence to map the words of the text sequence into embedding vectors.

[0047] S2. Use the Transformer-encoder-1 encoder to encode the embedding vector output by the Embedding-1 embedding layer;

[0048] S3. Decode the encoder output and entity sequence using the Transformer-decoder-1 decoder;

[0049] S4, pass the decoder output through a linear mapping layer, use the softmax function to get the decoded character, and add it to the decoded entity sequence;

[0050] S5, looping steps S3 to S4 until the end marker is decoded to obtain an entity sequence corresponding to the text sequence;

[0051] S6, supplementing entity information according to the text sequence and the entity sequence, converting the non-continuous entity segments in the text sequence into continuous entity segments, and obtaining an entity text sequence;

[0052] S7. Use the Embedding-2 embedding layer on the entity text sequence to map the words of the entity text sequence into embedding vectors;

[0053] S8. Encode the embedding vector output by the Embedding-2 embedding layer using the Transformer-encoder-2 encoder, and then average the entity component character vectors to obtain the entity type feature vector;

[0054] S9. Pass the entity type feature vector through a linear mapping layer, use the softmax function to obtain the entity category prediction value, and obtain the entity prediction result.

[0055] The present invention divides non-continuous entity recognition into two parts: entity detection and entity classification. The entity detection part integrates word continuous prompt-aware embedding in the Embedding-1 embedding layer to ensure that the predicted entity sequence is output in the order of the text sequence; further using the encoder-decoder architecture, it can realize entity fragment prediction based on a small amount of annotated data;

[0056] The entity classification part is based on the entity sequence obtained in the entity detection part, provides additional prompt knowledge, completes the entity information of the text sequence, enriches the non-continuous entity information, and obtains the entity text sequence, which reduces the difficulty of the entity classification stage; at the same time, the entity text sequence is further embedded using the Embedding-2 embedding layer to incorporate entity position perception information, enhance the representation information of the entity text sequence, and realize entity type prediction based on the encoder architecture, which can effectively avoid problems such as excessive consumption of annotation data resources and decoding ambiguity, and improve the accuracy of non-continuous entity recognition in low data resource scenarios.

[0057] S1, use the Embedding-1 embedding layer on the text sequence X to map words into embedding vectors;

[0058] Specifically, the mapping of words of the text sequence X into embedding vectors in step S1 is specifically as follows:

[0059] E=Word_Emb(X)+Position_Emb(X)+Perception_Emb(X) (1)

[0060] Where E = {e1, e2, ..., e n } represents the embedding vector sequence after the mapping of the Embedding-1 embedding layer, Word_Emb(·) represents the word semantic embedding of the text sequence, Position_Emb(·) represents the word position embedding of the text sequence, Perception_Emb(·) represents the word cue-aware embedding of the text sequence, X = {x1, x2, ..., x n } represents a text sequence, and n represents the length of the text sequence.

[0061] In this embodiment, given a text sequence [I have leg and arm pain now], the entity is leg pain and arm pain of type ADR, and the Embedding-1 embedding layer is used for the given text sequence. The words are mapped to embedding vectors E = {e1, e2, ..., e8} by adding word semantic embedding, word position embedding and word cue-aware embedding.

[0062] S2. Use the Transformer-encoder-1 encoder to encode the embedding vector E output by the Embedding-1 embedding layer;

[0063] Specifically, the encoding of the embedding vector E output by the embedding-1 embedding layer using the Transformer-encoder-1 encoder in step S2 is specifically as follows:

[0064] H=Transformer-encoder-1(E) (2)

[0065] Where H = {h1,h2,…,h n } represents the output of the embedded vector sequence after being encoded by the Transformer-encoder-1 encoder, and H = {h1,h2,…,h8} is obtained.

[0066] S3, use Transformer-decoder-1 decoder to decode the encoder output H and entity sequence Decode;

[0067] Specifically, in step S3, the Transformer-decoder-1 decoder is used to decode the encoder output H and entity sequence The decoding is as follows:

[0068]

[0069] in, represents the output of the Transformer-decoder-1 decoder, Represents the decoded entity sequence, <bos>Represents the start marker, and t represents the length of the decoded entity sequence.

[0070] Based on the decoded entity sequence And the encoder output H, decoded by Transformer-decoder-1 decoder, get the decoder output

[0071] S4, decoder output Through a linear mapping layer, the decoded characters are obtained using the softmax function and added to the decoded entity sequence. middle;

[0072] Specifically, the characters decoded in step S4 are:

[0073]

[0074] in, represents the character decoded in step t+1, and Represents a trainable parameter.

[0075] In this embodiment, Through a linear mapping layer, the softmax function is used to obtain the decoded characters. For example, the decoded character "leg" is added to the decoded entity sequence. get

[0076] S5, loop steps S3 to S4 until the end marker is decoded <end>, get the entity sequence corresponding to the text sequence X After multiple cycles, the entity sequence corresponding to the text sequence X in this embodiment is obtained

[0077] S6. According to the text sequence X and entity sequence Perform entity information supplementation, convert the non-continuous entity fragments in X in the text sequence into continuous entity fragments, and obtain the entity text sequence

[0078] In this embodiment, according to the text sequence X and the entity sequence Supplement entity information and transform the discontinuous entity fragment "leg, arm pain" in the text sequence X into a continuous entity fragment "leg pain, arm pain" to obtain the entity text sequence The entity text sequence obtained by supplementing the entity information with the text sequence and the entity sequence can enrich the non-continuous entity information and reduce the difficulty of the entity classification stage.

[0079] S7. Entity text sequence Use the Embedding-2 embedding layer to embed entity text sequences The word is mapped to an embedding vector

[0080] Specifically, in step S7, the entity text sequence The word is mapped to an embedding vector Specifically:

[0081]

[0082] in, represents the embedding vector sequence after being mapped by the Embedding-2 embedding layer, Word_Emb(·) represents the word semantic embedding of the text sequence, Position_Emb(·) represents the word position embedding of the text sequence, and Entity_Position_Emb(·) represents the entity position-aware embedding of the text sequence. represents the entity text sequence, and m represents the length of the entity text sequence.

[0083] In this embodiment, the entity text sequence Use the Embedding-2 embedding layer to map words into embedding vectors by summing word semantic embedding, word position embedding, and entity position-aware embedding.

[0084] S8. Use the Transformer-encoder-2 encoder to embed the embedding vector output by the Embedding-2 embedding layer Encode, then average the entity components into character vectors to obtain the entity type feature vector

[0085] The entity type feature vector in step S8 The calculation formula is as follows:

[0086]

[0087] in, represents the entity type feature vector, k represents the number of entities, and Mean(·) represents the average of the entity component character vectors to obtain the entity type feature vector

[0088] S9. Entity type feature vector Through a linear mapping layer, the softmax function is used to obtain the entity category prediction value Get entity prediction results;

[0089] The entity category prediction value in step S9 The calculation formula is as follows:

[0090]

[0091] in, represents the predicted value of entity category, and Represents a trainable parameter.

[0092] In this embodiment, Through a linear mapping layer, the softmax function is used to obtain the entity category prediction value Get the predicted entities of ADR type: leg pain and arm pain.

[0093] Embodiment 2

[0094] A device includes a memory and a processor, wherein the memory is used to store a program that supports the processor to execute the discontinuous entity recognition method based on perception enhancement in Example 1, and the processor is configured to execute the program stored in the memory.

[0095] Embodiment 3

[0096] A storage medium stores a computer program, which, when executed by a processor, executes the steps of the method for discontinuous entity recognition based on perception enhancement in embodiment 1.

[0097] The above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit the same. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that the technical solutions described in the aforementioned embodiments may still be modified, or some of the technical features may be replaced by equivalents. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present invention.< / end> < / bos> < / bos>

Claims

1. A non-continuous entity recognition method based on perception enhancement, characterized in that: The following steps are involved: S1. Use the Embedding-1 embedding layer on the text sequence to map the words of the text sequence into embedding vectors. S2. Use the Transformer-encoder-1 encoder to encode the embedding vector output by the Embedding-1 embedding layer; S3. Decode the encoder output and entity sequence using the Transformer-decoder-1 decoder; S4, pass the decoder output through a linear mapping layer, use the softmax function to get the decoded character, and add it to the decoded entity sequence; S5, looping steps S3 to S4 until the end marker is decoded to obtain an entity sequence corresponding to the text sequence; S6, supplementing entity information according to the text sequence and the entity sequence, converting the non-continuous entity segments in the text sequence into continuous entity segments, and obtaining an entity text sequence; S7. Use the Embedding-2 embedding layer on the entity text sequence to map the words of the entity text sequence into embedding vectors; S8. Encode the embedding vector output by the Embedding-2 embedding layer using the Transformer-encoder-2 encoder, and then average the entity component character vectors to obtain the entity type feature vector; S9. Pass the entity type feature vector through a linear mapping layer, use the softmax function to obtain the entity category prediction value, and obtain the entity prediction result.

2. The method for non-continuous entity recognition based on perception enhancement according to claim 1, characterized in that: The mapping of words in the text sequence into embedding vectors described in step S1 is specifically as follows: E=Word_Emb(X)+Position_Emb(X)+Perception_Emb(X) (1) Where E = {e1, e2, …, e n } represents the embedding vector sequence after the mapping of the Embedding-1 embedding layer, Word_Emb(·) represents the word semantic embedding of the text sequence, Position_Emb(·) represents the word position embedding of the text sequence, Perception_Emb(·) represents the word prompt perception embedding of the text sequence, X = {x1, x2, …, x n } represents a text sequence, and n represents the length of the text sequence.

3. The method for non-continuous entity recognition based on perception enhancement according to claim 2, characterized in that: The specific encoding of the embedding vector output by the Embedding-1 embedding layer using the Transformer-encoder-1 encoder in step S2 is: H=Transformer-encoder-1(E) (2) Where H = {h1,h2,…,h n } represents the output of the embedding vector sequence after being encoded by the Transformer-encoder-1 encoder.

4. The method for non-continuous entity recognition based on perception enhancement according to claim 3 is characterized in that: The specific method of decoding the encoder output and entity sequence using the Transformer-decoder-1 decoder in step S3 is as follows: in, represents the output of the Transformer-decoder-1 decoder, Represents the decoded entity sequence, <bos> Represents the start marker, and t represents the length of the decoded entity sequence.< / bos> 5. The method for non-continuous entity recognition based on perception enhancement according to claim 4, characterized in that: The characters decoded in step S4 are specifically: in, represents the character decoded in step t+1, and Represents a trainable parameter.

6. The method for non-continuous entity recognition based on perception enhancement according to claim 1, characterized in that: The mapping of words of the entity text sequence into embedding vectors in step S7 is specifically as follows: in, represents the embedding vector sequence after being mapped by the Embedding-2 embedding layer, Word_Emb(·) represents the word semantic embedding of the text sequence, Position_Emb(·) represents the word position embedding of the text sequence, and Entity_Position_Emb(·) represents the entity position-aware embedding of the text sequence. represents the entity text sequence, and m represents the length of the entity text sequence.

7. The method for non-continuous entity recognition based on perception enhancement according to claim 6, characterized in that: The calculation formula of the entity type feature vector in step S8 is as follows: in, represents the entity type feature vector, k represents the number of entities, and Mean(·) represents the average of the entity component character vectors.

8. The method for non-continuous entity recognition based on perception enhancement according to claim 7, characterized in that: The calculation formula of the entity category prediction value in step S9 is as follows: in, represents the predicted value of entity category, and Represents a trainable parameter.

9. A device comprising a memory and a processor, characterized in that: The memory is used to store a program that supports the processor to execute the non-continuous entity recognition method based on perception enhancement as described in any one of claims 1 to 8, and the processor is configured to execute the program stored in the memory.

10. A storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the non-continuous entity recognition method based on perception enhancement according to any one of claims 1 to 8 are executed.

Images