Discontinuous entity recognition method, device and storage medium based on perception enhancement

Abstract

A non-continuous entity recognition method, device and storage medium based on perception enhancement belong to the field of natural language processing technology, and solve the problems of large demand for annotation data resources and decoding ambiguity in existing non-continuous named entity recognition methods. The present invention divides non-continuous entity recognition into two parts: entity detection and entity classification. The entity detection part integrates word continuous prompt perception embedding in the embedding layer of the encoder to ensure that the predicted entity sequence is output in the order of the text sequence; the entity classification part provides additional prompt knowledge based on the entity sequence obtained by the entity detection part, completes the entity information of the text sequence, enriches the non-continuous entity information, reduces the difficulty of the entity classification stage, can effectively avoid the problems of excessive consumption of annotation data resources and decoding ambiguity, and improves the accuracy of non-continuous entity recognition in low data resource scenarios.

Description

Technical Field

[0001] The present invention belongs to the technical field of natural language processing and relates to a discontinuous entity recognition method, device and storage medium based on perception enhancement. Background Art

[0002] Named entity recognition (NER) involves identifying entities with special meaning in text. As a fundamental task in natural language processing, NER is a crucial preprocessing step for downstream tasks such as relation extraction, text summarization, and intelligent question answering. Currently, most NER methods assume that entities consist of consecutive words. Using sequence annotation based on BIO tags, they classify each word in the text and combine the word classification results to identify entities of a specific type.

[0003] Different from ordinary named entities, non-continuous named entities refer to entities composed of discontinuous words. However, most of the above-mentioned sequence labeling methods based on BIO tags have difficulty in solving such irregular entity recognition tasks. In recent years, the mainstream methods for discontinuous entity recognition mainly include entity parsing methods based on state transition and sequence labeling methods based on label expansion. The entity parsing method based on state transition realizes the parsing of discontinuous entities by predicting different actions of different words. For example, the paper "An effective transition-based model for discontinuous NER" (Dai X, Karimi S, Hachey B, et al. arXiv preprint arXiv:2004.13454, 2020.) uses six types of action labels such as SHIFT, OUT, COMPLETE, REDUCE, RIGHT-REDUCE, and LEFT-REDUCE to predict different actions of words, realize the merging of different words, and obtain discontinuous entities. Sequence annotation methods based on label extension extend the BIO label to represent non-continuous named entities. For example, the paper "Concept Identification and Normalization for Adverse Drug Event Discovery in Medical Forums" (Metke-Jimenez A, Karimi S. BMDID@ISWC.2016.) expands the BIO label into four types of labels: BI, IH, BD, and ID, to represent the different head and tail parts of non-continuous named entities. However, these methods either require a large amount of labeled data for training, which requires a lot of data resources, or suffer from decoding ambiguity, making them unsuitable for low-data-resource scenarios in fields such as biomedicine and military defense. Summary of the Invention

[0004] The technical solution of the present invention is used to solve the problems of large demand for annotation data resources and decoding ambiguity in existing non-continuous named entity recognition methods.

[0005] The present invention solves the above technical problems through the following technical solutions:

[0006] A non-continuous entity recognition method based on perception enhancement includes the following steps:

[0007] S1. Use the Embedding-1 embedding layer on the text sequence to map the words in the text sequence into embedding vectors;

[0008] S2. Use the Transformer-encoder-1 encoder to encode the embedding vector output by the Embedding-1 embedding layer;

[0009] S3. Decode the encoder output and entity sequence using the Transformer-decoder-1 decoder;

[0010] S4. Pass the decoder output through a linear mapping layer, use the softmax function to obtain the decoded character, and add it to the decoded entity sequence;

[0011] S5, looping steps S3 to S4 until the end marker is decoded to obtain the entity sequence corresponding to the text sequence;

[0012] S6. Supplement entity information based on the text sequence and the entity sequence, converting discontinuous entity segments in the text sequence into continuous entity segments to obtain an entity text sequence;

[0013] S7. Use the Embedding-2 embedding layer on the entity text sequence to map the words in the entity text sequence into embedding vectors;

[0014] S8. Use the Transformer-encoder-2 encoder to encode the embedding vector output by the Embedding-2 embedding layer, and then average the entity component character vectors to obtain the entity type feature vector;

[0015] S9. Pass the entity type feature vector through a linear mapping layer and use the softmax function to obtain the entity category prediction value to obtain the entity prediction result.

[0016] Furthermore, the mapping of words in the text sequence into embedding vectors in step S1 is specifically as follows:

[0017] E=Word_Emb(X)+Position_Emb(X)+Perception_Emb(X) (1)

[0018] Where E={e1,e2,...,e n} represents the embedding vector sequence after the mapping of the Embedding-1 embedding layer, Word_Emb(·) represents the word semantic embedding of the text sequence, Position_Emb(·) represents the word position embedding of the text sequence, Perception_Emb(·) represents the word prompt perception embedding of the text sequence, X = {x1, x2, ..., x n} represents a text sequence, and n represents the length of the text sequence.

[0019] Furthermore, the encoding of the embedding vector output by the Embedding-1 embedding layer using the Transformer-encoder-1 encoder in step S2 is specifically as follows:

[0020] H=Transformer-encoder-1(E) (2)

[0021] Where H = {h1, h2, ..., h n} represents the output of the embedding vector sequence after being encoded by the Transformer-encoder-1 encoder.

[0022] Furthermore, the decoding of the encoder output and the entity sequence using the Transformer-decoder-1 decoder in step S3 is specifically as follows:

[0023]

[0024] in, represents the output of the Transformer-decoder-1 decoder, Represents the decoded entity sequence, <bos>Represents the start marker, and t represents the length of the decoded entity sequence.

[0025] Furthermore, the characters decoded in step S4 are specifically:

[0026]

[0027] in, Indicates the character decoded in step t+1, and Represents a trainable parameter.

[0028] Furthermore, the mapping of words in the entity text sequence into embedding vectors in step S7 is specifically as follows:

[0029]

[0030] in, Represents the embedding vector sequence after the mapping of the Embedding-2 embedding layer, Word_Emb(·) represents the word semantic embedding of the text sequence, Position_Emb(·) represents the word position embedding of the text sequence, and Entity_Position_Emb(·) represents the entity position-aware embedding of the text sequence. Represents the entity text sequence, and m represents the length of the entity text sequence.

[0031] Furthermore, the calculation formula of the entity type feature vector in step S8 is as follows:

[0032]

[0033] in, represents the entity type feature vector, k represents the number of entities, and Mean(·) represents the average of the entity component character vectors.

[0034] Furthermore, the calculation formula of the entity category prediction value in step S9 is as follows:

[0035]

[0036] in, represents the entity category prediction value, and Represents a trainable parameter.

[0037] A device includes a memory and a processor, wherein the memory is used to store a program that supports the processor to execute the non-continuous entity recognition method based on perception enhancement, and the processor is configured to execute the program stored in the memory.

[0038] A storage medium stores a computer program, which, when executed by a processor, executes the steps of the discontinuous entity recognition method based on perception enhancement.

[0039] The advantages of the present invention are: the present invention divides non-continuous entity recognition into two parts: entity detection and entity classification. The entity detection part integrates word continuous prompt-aware embedding in the embedding layer of the encoder to ensure that the predicted entity sequence is output in the order of the text sequence; further utilizing the encoder-decoder architecture, it can realize entity fragment prediction based on a small amount of labeled data; the entity classification part provides additional prompt knowledge based on the entity sequence obtained by the entity detection part, completes the entity information of the text sequence, enriches the non-continuous entity information, obtains the entity text sequence, and reduces the difficulty of the entity classification stage; at the same time, it further integrates the entity position-aware embedding based on the entity position-aware embedding, enhances the representation information of the entity text sequence, and realizes entity type prediction based on the encoder architecture, which can effectively avoid the problems of excessive consumption of labeled data resources and decoding ambiguity, and improve the accuracy of non-continuous entity recognition in low data resource scenarios. BRIEF DESCRIPTION OF THE DRAWINGS

[0040] Figure 1 is a flow chart of a method for discontinuous entity recognition based on perception enhancement according to an embodiment of the present invention;

[0041] Figure 2 4 is a structural diagram of a discontinuous entity recognition model according to an embodiment of the present invention. DETAILED DESCRIPTION

[0042] To make the objectives, technical solutions, and advantages of the embodiments of the present invention more clear, the technical solutions in the embodiments of the present invention will be clearly and completely described below in conjunction with the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. All other embodiments obtained by ordinary technicians in this field based on the embodiments of the present invention without making any creative efforts shall fall within the scope of protection of the present invention.

[0043] The technical solution of the present invention is further described below with reference to the accompanying drawings and specific embodiments:

[0044] Example 1

[0045] like Figure 1 Specifically, a method for discontinuous entity recognition based on perception enhancement is disclosed, comprising the following steps:

[0046] S1. Use the Embedding-1 embedding layer on the text sequence to map the words in the text sequence into embedding vectors;

[0047] S2. Use the Transformer-encoder-1 encoder to encode the embedding vector output by the Embedding-1 embedding layer;

[0048] S3. Decode the encoder output and entity sequence using the Transformer-decoder-1 decoder;

[0049] S4. Pass the decoder output through a linear mapping layer, use the softmax function to obtain the decoded character, and add it to the decoded entity sequence;

[0050] S5, looping steps S3 to S4 until the end marker is decoded to obtain the entity sequence corresponding to the text sequence;

[0051] S6. Supplement entity information based on the text sequence and the entity sequence, converting discontinuous entity segments in the text sequence into continuous entity segments to obtain an entity text sequence;

[0052] S7. Use the Embedding-2 embedding layer on the entity text sequence to map the words in the entity text sequence into embedding vectors;

[0053] S8. Use the Transformer-encoder-2 encoder to encode the embedding vector output by the Embedding-2 embedding layer, and then average the entity component character vectors to obtain the entity type feature vector;

[0054] S9. Pass the entity type feature vector through a linear mapping layer and use the softmax function to obtain the entity category prediction value to obtain the entity prediction result.

[0055] The present invention divides non-continuous entity recognition into two parts: entity detection and entity classification. The entity detection part integrates word continuous hint-aware embedding in the Embedding-1 embedding layer to ensure that the predicted entity sequence is output in the order of the text sequence. Furthermore, the encoder-decoder architecture is used to achieve entity segment prediction based on a small amount of annotated data.

[0056] The entity classification part is based on the entity sequence obtained in the entity detection part, provides additional prompt knowledge, completes the entity information of the text sequence, enriches the non-continuous entity information, obtains the entity text sequence, and reduces the difficulty of the entity classification stage; at the same time, it further uses the Embedding-2 embedding layer to perceive the entity text sequence, incorporates entity position perception information, enhances the representation information of the entity text sequence, and realizes entity type prediction based on the encoder architecture, which can effectively avoid problems such as excessive consumption of annotation data resources and decoding ambiguity, and improve the accuracy of non-continuous entity recognition in low data resource scenarios.

[0057] S1. Use the Embedding-1 embedding layer on the text sequence X to map words into embedding vectors.

[0058] Specifically, the mapping of words in the text sequence X into embedding vectors in step S1 is as follows:

[0059] E=Word_Emb(X)+Position_Emb(X)+Perception_Emb(X) (1)

[0060] Where E={e1,e2,...,e n } represents the embedding vector sequence after the mapping of the Embedding-1 embedding layer, Word_Emb(·) represents the word semantic embedding of the text sequence, Position_Emb(·) represents the word position embedding of the text sequence, Perception_Emb(·) represents the word prompt perception embedding of the text sequence, X = {x1, x2, ..., x n } represents a text sequence, and n represents the length of the text sequence.

[0061] In this embodiment, given the text sequence [I have leg and arm pain now], the entity is leg pain and arm pain of type ADR. The Embedding-1 embedding layer is used for the given text sequence. By adding word semantic embedding, word position embedding and word cue-aware embedding, the word is mapped to an embedding vector E = {e1, e2, ..., e8}.

[0062] S2. Use the Transformer-encoder-1 encoder to encode the embedding vector E output by the Embedding-1 embedding layer;

[0063] Specifically, the encoding of the embedding vector E output by the Embedding-1 embedding layer using the Transformer-encoder-1 encoder in step S2 is specifically as follows:

[0064] H=Transformer-encoder-1(E) (2)

[0065] Where H={h1,h2,…,h n } represents the output of the embedded vector sequence after being encoded by the Transformer-encoder-1 encoder, and the result is H = {h1,h2,…,h8}.

[0066] S3, use Transformer-decoder-1 decoder to output H and entity sequence of encoder Decode;

[0067] Specifically, the encoder output H and entity sequence are processed using the Transformer-decoder-1 decoder in step S3. The decoding is as follows:

[0068]

[0069] in, represents the output of the Transformer-decoder-1 decoder, Represents the decoded entity sequence, <bos>Represents the start marker, and t represents the length of the decoded entity sequence.

[0070] Based on the decoded entity sequence And the encoder output H, decoded by Transformer-decoder-1 decoder, get the decoder output

[0071] S4, decoder output Through a linear mapping layer, the softmax function is used to obtain the decoded characters and add them to the decoded entity sequence. middle;

[0072] Specifically, the characters decoded in step S4 are:

[0073]

[0074] in, Indicates the character decoded in step t+1, and Represents a trainable parameter.

[0075] In this embodiment, Through a linear mapping layer, the softmax function is used to obtain the decoded characters. For example, the decoded character "leg" is added to the decoded entity sequence. get

[0076] S5, loop steps S3 to S4 until the end marker is decoded <end>, get the entity sequence corresponding to the text sequence X After multiple cycles, the entity sequence corresponding to the text sequence X in this embodiment is obtained

[0077] S6. According to the text sequence X and entity sequence Perform entity information supplementation, convert the non-continuous entity segments in X in the text sequence into continuous entity segments, and obtain the entity text sequence

[0078] In this embodiment, according to the text sequence X and the entity sequence Perform entity information supplementation and transform the discontinuous entity segment "leg, arm pain" in the text sequence X into a continuous entity segment "leg pain, arm pain" to obtain the entity text sequence The entity text sequence obtained by supplementing the entity information with the text sequence and entity sequence can enrich the non-continuous entity information and reduce the difficulty of the entity classification stage.

[0079] S7. Entity text sequence Use the Embedding-2 embedding layer to embed entity text sequences The word is mapped to an embedding vector

[0080] Specifically, in step S7, the entity text sequence The word is mapped to an embedding vector Specifically:

[0081]

[0082] in, Represents the embedding vector sequence after the mapping of the Embedding-2 embedding layer, Word_Emb(·) represents the word semantic embedding of the text sequence, Position_Emb(·) represents the word position embedding of the text sequence, and Entity_Position_Emb(·) represents the entity position-aware embedding of the text sequence. Represents the entity text sequence, and m represents the length of the entity text sequence.

[0083] In this embodiment, by Use the Embedding-2 embedding layer to map words into embedding vectors by summing word semantic embedding, word position embedding, and entity position-aware embedding.

[0084] S8, using the Transformer-encoder-2 encoder to embed the embedding vector output by the Embedding-2 embedding layer Encode, then average the entity composition character vector to obtain the entity type feature vector

[0085] The entity type feature vector in step S8 The calculation formula is as follows:

[0086]

[0087] in, Represents the entity type feature vector, k represents the number of entities, and Mean(·) represents the average of the entity component character vectors to obtain the entity type feature vector

[0088] S9. Entity type feature vector Through a linear mapping layer, the softmax function is used to obtain the entity category prediction value Get entity prediction results;

[0089] The entity category prediction value in step S9 The calculation formula is as follows:

[0090]

[0091] in, represents the entity category prediction value, and Represents a trainable parameter.

[0092] In this embodiment, Through a linear mapping layer, the softmax function is used to obtain the entity category prediction value Get the predicted entities of ADR type: leg pain and arm pain.

[0093] Example 2

[0094] A device includes a memory and a processor, wherein the memory is used to store a program that supports the processor to execute the discontinuous entity recognition method based on perception enhancement in embodiment 1, and the processor is configured to execute the program stored in the memory.

[0095] Example 3

[0096] A storage medium stores a computer program, which, when executed by a processor, executes the steps of the method for discontinuous entity recognition based on perception enhancement in embodiment 1.

[0097] The above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit the same. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the various embodiments of the present invention.< / end> < / bos> < / bos>

Claims

1. A non-continuous entity recognition method based on perception enhancement, characterized in that: The following steps are involved: S1. Use the Embedding-1 embedding layer on the text sequence to map the words in the text sequence into embedding vectors; S2. Use the Transformer-encoder-1 encoder to encode the embedding vector output by the Embedding-1 embedding layer; S3. Decode the encoder output and entity sequence using the Transformer-decoder-1 decoder; S4. Pass the decoder output through a linear mapping layer, use the softmax function to obtain the decoded character, and add it to the decoded entity sequence; S5, looping steps S3 to S4 until the end marker is decoded to obtain the entity sequence corresponding to the text sequence; S6. Supplement entity information based on the text sequence and the entity sequence, converting discontinuous entity segments in the text sequence into continuous entity segments to obtain an entity text sequence; S7. Use the Embedding-2 embedding layer on the entity text sequence to map the words in the entity text sequence into embedding vectors; S8. Use the Transformer-encoder-2 encoder to encode the embedding vector output by the Embedding-2 embedding layer, and then average the entity component character vectors to obtain the entity type feature vector; S9. Pass the entity type feature vector through a linear mapping layer and use the softmax function to obtain the entity category prediction value to obtain the entity prediction result.

2. The method for discontinuous entity recognition based on perception enhancement according to claim 1, characterized in that: The mapping of words in the text sequence into embedding vectors in step S1 is specifically as follows: E=Word_Emb(X)+Position_Emb(X)+Perception_Emb(X) (1) Where E={e1,e2,…,e n } represents the embedding vector sequence after the mapping of the Embedding-1 embedding layer, Word_Emb(·) represents the word semantic embedding of the text sequence, Position_Emb(·) represents the word position embedding of the text sequence, Perception_Emb(·) represents the word prompt perception embedding of the text sequence, X={x1,x2,…,x n } represents a text sequence, and n represents the length of the text sequence.

3. The method for discontinuous entity recognition based on perception enhancement according to claim 2, characterized in that: The specific encoding of the embedding vector output by the Embedding-1 embedding layer using the Transformer-encoder-1 encoder in step S2 is as follows: H=Transformer-encoder-1(E) (2) Where H={h1,h2,…,h n } represents the output of the embedding vector sequence after being encoded by the Transformer-encoder-1 encoder.

4. The method for discontinuous entity recognition based on perception enhancement according to claim 3, characterized in that: The specific decoding of the encoder output and entity sequence using the Transformer-decoder-1 decoder in step S3 is as follows: in, represents the output of the Transformer-decoder-1 decoder, Represents the decoded entity sequence, <bos> Represents the start marker, and t represents the length of the decoded entity sequence.< / bos> 5. The method for discontinuous entity recognition based on perception enhancement according to claim 4, characterized in that: The characters decoded in step S4 are specifically: in, Indicates the character decoded in step t+1, and Represents a trainable parameter.

6. The method for discontinuous entity recognition based on perception enhancement according to claim 1, characterized in that: The mapping of words in the entity text sequence into embedding vectors in step S7 is specifically as follows: in, Represents the embedding vector sequence after the mapping of the Embedding-2 embedding layer, Word_Emb(·) represents the word semantic embedding of the text sequence, Position_Emb(·) represents the word position embedding of the text sequence, and Entity_Position_Emb(·) represents the entity position-aware embedding of the text sequence. Represents the entity text sequence, and m represents the length of the entity text sequence.

7. The method for discontinuous entity recognition based on perception enhancement according to claim 6, characterized in that: The calculation formula of the entity type feature vector in step S8 is as follows: in, represents the entity type feature vector, k represents the number of entities, and Mean(·) represents the average of the entity component character vectors.

8. The method for discontinuous entity recognition based on perception enhancement according to claim 7, characterized in that: The calculation formula of the entity category prediction value in step S9 is as follows: in, represents the entity category prediction value, and Represents a trainable parameter.

9. A device comprising a memory and a processor, characterized in that: The memory is used to store a program that supports the processor to execute the perception-enhanced discontinuous entity recognition method according to any one of claims 1 to 8, and the processor is configured to execute the program stored in the memory.

10. A storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the non-continuous entity recognition method based on perception enhancement according to any one of claims 1 to 8 are executed.

Images