Nested entity recognition method and system based on double tri- affine mechanism and storage medium

Abstract

The application discloses a kind of based on double tri- affine mechanism's nested entity identification method, system and storage medium, it is related to named entity recognition technical field.Mainly include the following steps: on the basis of using tri- affine mechanism to do entity span representation, entity boundary relative position and span area information are fused into entity span representation;Then again using tri- affine mechanism attention mechanism, with entity boundary as query, with new span representation as key and value, the information in entity is interacted with boundary word to obtain entity span representation;Finally, entity class prediction is carried out using entity span joint double affine mechanism.The application realizes the accurate identification of nested entity, under the condition of same batch size, parameter quantity and occupied resources are relatively less, provide reference for efficient nested named entity recognition, and the accuracy of entity recognition is improved compared with general tri- affine mechanism model.

Description

Technical Field

[0001] This invention relates to the field of named entity recognition technology, and more specifically to a nested entity recognition method, system, and storage medium based on a double triaxonic mechanism. Background Technology

[0002] The task of identifying the boundaries and types of named entities from a given text is called named entity recognition, and it is one of the standard tasks of information extraction.

[0003] Before the large-scale application of deep learning, named entity recognition methods mainly relied on dictionary-based and rule-based approaches, which, along with machine learning methods, resulted in slow improvements in recognition accuracy and failed to achieve ideal results. The introduction and development of deep learning have significantly improved the accuracy of named entity recognition.

[0004] However, existing research largely focuses on non-nested entity recognition, with insufficient attention paid to nested entity recognition. Nested named entity structures are complex and varied, with no regularity in nesting granularity and nesting levels. Therefore, for those skilled in the art, the key to the comprehensive application of natural language processing lies in how to quickly and efficiently extract nested named entity structure information from open-domain text to achieve more accurate semantic understanding. Summary of the Invention

[0005] In view of this, the present invention provides a nested entity recognition method, system and storage medium based on a double tri-affine mechanism to solve the problems existing in the background art.

[0006] To achieve the above objectives, the present invention adopts the following technical solution: a nested entity recognition method based on a bi-triangular affine mechanism, the specific steps of which include the following: Retrieve the target nested entity; Each character of the target nested entity is encoded to obtain a vector representation of the character; The vector representation of the character is processed by a nested entity recognition model that fuses position and content information based on a bi-triangular affine mechanism to obtain all the span representations of the character for entity recognition, and a first probability value is obtained. The vector representation of the character is mapped using a double affine mechanism to obtain a span representation, thereby obtaining the second probability value for entity recognition; The first probability value and the second probability value are added together to predict the category of each sub-entity in the target nested entity.

[0007] Optionally, the steps for entity recognition using the nested entity recognition model are as follows: The span representation is calculated based on the vector representation of the word through a three-affine mechanism. The relative position information of the entity boundary and the span region information are then integrated into the span representation to obtain fused information. By utilizing a three-affine mechanism, the entity boundary interacts with the fused information to obtain an entity span representation; The entity span representation is used in conjunction with a dual affine mechanism to predict the category of each sub-entity in the target nested entity.

[0008] Optionally, the nested entity recognition model includes an encoding layer, a three-affine mechanism layer, and a prediction layer; wherein, Encoding layer: Using pre-trained language models BERT and BiLSTM as encoders, the target nested entities are encoded to produce vector representations of words containing context; Three-affine mechanism layer: In the three-affine mechanism layer, the first stage uses a three-affine attention mechanism to create a span representation. The second stage fuses the span representation with the relative position and region information of the entity boundary. The three-affine mechanism is then used again to interact the boundary with the fused information in a higher order to obtain the entity span representation and obtain the first probability value. Prediction layer: Entity prediction is performed by using the first probability value together with a dual affine predictor.

[0009] Optionally, the expression representing the span is: ; ; ; in, Indicates the start and end positions of the span. This indicates information from the start position to the end position. This indicates a three-affine mechanism. Representing three-affine attention, with entity boundaries and the dimension parameter represented by entity span ( As an attention mechanism, queries for each word ( The keys and values ​​used in the attention mechanism are generated in dimensional form. span representation .

[0010] Optionally, the expression for the fused information is: ; in, Indicates the span. Indicates the relative position information of entity boundaries. , Indicates span area information, , , The span represents the dimension. This indicates the embedding dimension of the relative position information of entity boundaries. This indicates the dimension of information embedding across the span of the region.

[0011] Optionally, the expression for the entity span obtained using the three-affine mechanism is: ; in, Represents entity boundaries, Indicates the label size parameter This refers to the fused information.

[0012] Optionally, the entity span representation obtained based on the three-affine mechanism is used to calculate the span from the first... i The word to the 1 j The span score of each word.

[0013] On the other hand, a nested entity recognition system based on a bi-triangular affine mechanism is provided, comprising a data acquisition module, an encoding module, a first recognition module, a second recognition module, and a prediction module; wherein, The data acquisition module is used to acquire the target nested entity; The encoding module is used to encode the target nested entity to obtain a vector representation of the words of the target nested entity; The first recognition module is used to perform entity recognition on the vector representation of the word using a nested entity recognition model that fuses position and content information based on a bi-triangular affine mechanism, and obtain a first probability value; The second recognition module is used to recognize the vector representation of the word through a dual affine mechanism to obtain a second probability value; The prediction module is used to add the first probability value and the second probability value to predict the category of each sub-entity in the target nested entity.

[0014] Finally, a computer storage medium is provided, on which a computer program is stored, and when the computer program is executed by a processor, it implements the steps of the nested entity recognition method based on a bi-triangular affine mechanism.

[0015] As can be seen from the above technical solutions, compared with the prior art, the present invention discloses a nested entity recognition method, system and storage medium based on a double tri-affine mechanism, which has the following beneficial technical effects: it achieves accurate recognition of nested entities, and with the same batch size, the number of parameters and the resources occupied are relatively small, providing a reference for efficient nested named entity recognition, and improving the accuracy of entity recognition compared with the general tri-affine mechanism model. Attached Figure Description

[0016] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on the provided drawings without creative effort.

[0017] Figure 1 This is a flowchart of the method of the present invention; Figure 2 This is a schematic diagram of the nested named entity recognition model framework of the present invention; Figure 3 This is a diagram showing the span of the three affine lines of the present invention; Figure 4 This is a schematic diagram illustrating the fusion of relative position information and region information according to the present invention; Figure 5 This is a system structure diagram of the present invention. Detailed Implementation

[0018] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0019] This invention discloses a nested entity recognition method based on a bi-triangular affine mechanism, such as... Figure 1 As shown, the specific steps include the following: S1. Obtain the target nested entity; S2. Encode the target nested entity to obtain the vector representation of the target nested entity's words; S3. The vector representation of the word is used to perform entity recognition by a nested entity recognition model that integrates position and content information based on a bi-triangular affine mechanism, and the first probability value is obtained. S4. The vector representation of the word is identified through a dual affine mechanism to obtain the second probability value; S5. Add the first probability value and the second probability value to predict the category of each sub-entity in the target nested entity.

[0020] Furthermore, the steps for entity recognition using a nested entity recognition model are as follows: S31. The word-based vector representation is calculated using a three-affine mechanism to obtain a span representation. The relative position information of entity boundaries and span region information are then integrated into the span representation to obtain fused information. S32. Using a three-affine mechanism, the entity boundary and fused information interact to obtain an entity span representation; S33. Use entity span representation and a combined double affine mechanism to predict the category of each sub-entity in a nested target entity.

[0021] Furthermore, such as Figure 2 As shown, the nested entity recognition model includes an encoding layer, a three-affine mechanism layer, and a prediction layer.

[0022] Encoding Layer: Using pre-trained language models BERT and BiLSTM as encoders, the target nested entities are encoded to produce vector representations of words containing context. Using BERT as model input, for a given sentence x={x1,x2, …,xn}, where xi represents the i-th word in the sentence, the BERT input vector is composed of word embedding vectors, word segmentation embedding vectors, and positional encoding vectors. After BERT computation, the last four layers of vectors are max-pooled to obtain the word vector for each word. To further enhance context modeling, two layers of bidirectional LSTM are used after the BERT output to generate the final word representation, i.e., H={h1,h2,...,hN}∈ N×dh where N represents the dimension of a word, and N represents the length of a sentence; Three-affine mechanism layer: In the three-affine mechanism layer, the first stage uses a three-affine attention machine to create a span representation. The second stage fuses the span representation with the relative position and region information of the entity boundary. The three-affine mechanism is then used again to interact the boundary with the fused information in a higher order to obtain the entity span representation and the first probability value.

[0023] Definition of the three-affine mechanism: such as Figure 3 The diagram shown illustrates the span representation of a triaffine mechanism. Defining a triaffine mechanism requires three vectors u, v, w ∈ R. d and a parameter tensor It is used to compute the input vector applied to different MLP (Multilayer Perceptron) transformations and to compute tensor vector multiplication to output a scalar.

[0024] ; ; ; Where a, b, and c represent the layers of the MLP, and in this embodiment, the boundary words of the word pair table are represented as u and v, and the words inside the word pair are represented as w.

[0025] Two-stage three-affine attention: The first stage: Three-affine attention represents entity span. To obtain a better span representation through word pair grids, this invention adopts a three-affine attention mechanism to realize entity span representation and entity boundary interaction, by introducing a third dimension to represent entity span. i,j,d,m Using a three-affine mechanism, the span expression is as follows: ; ; ; in, Indicates the start and end positions of the span. This indicates information from the start position to the end position. This indicates a three-affine mechanism. Representing three-affine attention, with entity boundaries and the dimension parameter represented by entity span ( As an attention mechanism, queries for each word ( The keys and values ​​used in the attention mechanism are generated in dimensional form. span representation

[0026] The second stage involves a tri-affine attention representation of entity categories that integrates region information and relative positions. To enrich the information in the word pair grid, region information and the relative positions between each word pair are added, such as... Figure 4As shown. The left image shows the relative position information of the word pair grid. In the word pair table, the row starts at the beginning and the column ends at the end. The distance from the beginning to the end of an entity is the relative position information of the entity boundary. In the image, for the entity "the secretary of homeland security", the distance from "the" to "security" is 4, so the corresponding value in the table is 4. The right image shows the region information within the span. In this example sentence, the entity "homeland security" is nested within the entity "the secretary of homeland security", so the region information for "homeland security" is 2, and the region information for the other positions is 1. Here, 2 indicates that the word "homeland security" is inside two entities. Similarly, 1 indicates that the word within the span is inside one entity. The vectors used as the relative position information embedding representation for each pair of words, the vector used as the region information embedding representation for the upper triangle of the word pair table, and the entity span representation obtained in the first stage are concatenated in the third dimension to obtain the fused information. The expression is: ; in, Indicates the span. Indicates the relative position information of entity boundaries. , Indicates span area information, , , The span represents the dimension. This indicates the embedding dimension of the relative position information of entity boundaries. This indicates the dimension of information embedding across the span of the region.

[0027] In the second stage, the output of the LSTM is biaffine again to obtain a new span boundary representation. Then, with the new span representation Using a three-affine attention mechanism for span category prediction, the expression for the entity span representation obtained using the three-affine mechanism is as follows: ; in, Represents entity boundaries, Indicates the label size parameter This refers to the fused information.

[0028] Prediction layer: Entity prediction is performed by combining the first probability value with a dual affine predictor. After the above processing, the probability of each word pair corresponding to the label is obtained. It has been proven that predicting together with a dual affine predictor used for relation classification can improve the performance of entity prediction. Therefore, in this invention, both predictor factors are used simultaneously to calculate the span ( Two independent relational distributions, where , The first and last parts of the span are represented respectively, and they are combined to form the final prediction.

[0029] Biaffine prediction: The input to the biaffine predictor is the output of the coding layer. For a given sentence of length N, two MLPs are used to compute the head and tail representations of the entities, respectively. Then, a dual affine classifier is used to compute the head and tail representations of this pair of head and tail words. The entity label score, composed of ( ), is shown in the following formula: ; ; ; in, , These are learnable parameters. and express and The beginning and end of the text are indicated. This indicates the label score.

[0030] Span Prediction: Entity Span Representation Based on Three-Affine Attention Mechanism Use an MLP to calculate the span from the first The word to the 1 Span score for each word: ; The final label probability of the word pair combines the scores from the dual affine predictor and the span predictor: .

[0031] Embodiment 2 of the present invention provides a nested entity recognition system based on a bi-triangular affine mechanism, such as... Figure 5 As shown, it includes a data acquisition module, an encoding module, a first recognition module, a second recognition module, and a prediction module; wherein, The data acquisition module is used to acquire target nested entities; The encoding module is used to encode the target nested entity to obtain the vector representation of the target nested entity's words; The first recognition module is used to perform entity recognition on the vector representation of words using a nested entity recognition model that fuses position and content information based on a bi-triangular affine mechanism, and obtain the first probability value. The second recognition module is used to recognize the vector representation of the word through a dual affine mechanism to obtain a second probability value; The prediction module is used to add the first probability value and the second probability value to predict the category of each sub-entity in the target nested entity.

[0032] Finally, a computer storage medium is provided, on which a computer program is stored, and when the computer program is executed by a processor, it implements the steps of a nested entity recognition method based on a bi-triangular affine mechanism.

[0033] The various embodiments in this specification are described in a progressive manner, with each embodiment focusing on its differences from other embodiments. Similar or identical parts between embodiments can be referred to interchangeably. For the apparatus disclosed in the embodiments, since they correspond to the methods disclosed in the embodiments, the description is relatively simple; relevant parts can be referred to the method section.

[0034] The above description of the disclosed embodiments enables those skilled in the art to make or use the invention. Various modifications to these embodiments will be readily apparent to those skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of the invention. Therefore, the invention is not to be limited to the embodiments shown herein, but is to be accorded the widest scope consistent with the principles and novel features disclosed herein.

Claims

1. A nested entity recognition method based on a bi-triangular affine mechanism, characterized in that, The specific steps include the following: Retrieve the target nested entity; Each character of the target nested entity is encoded to obtain a vector representation of the character; The vector representation of the character is processed by a nested entity recognition model that fuses position and content information based on a bi-triangular affine mechanism to obtain all the span representations of the character for entity recognition, and a first probability value is obtained. The vector representation of the character is mapped using a double affine mechanism to obtain a span representation, thereby obtaining the second probability value for entity recognition; The first probability value and the second probability value are added together to predict the category of each sub-entity in the target nested entity; The steps for entity recognition using the nested entity recognition model are as follows: The vector representation based on the character is used to calculate the span representation through a three-affine mechanism. The relative position information of the entity boundary and the span region information are then integrated into the span representation to obtain fused information. By utilizing a three-affine mechanism, the entity boundary interacts with the fused information to obtain an entity span representation; The entity span representation is used in conjunction with a biaffine mechanism to predict the category of each sub-entity in the target nested entity; The expression representing the span is: ； ； ； in, This indicates information from the start position to the end position. This indicates a three-affine mechanism. Representing three-affine attention, with entity boundaries Dimensional parameters representing entity span The queries used as an attention mechanism are generated in dimensionality . span representation ; The expression for the fused information is: ； in, Indicates the span. Indicates the relative position information of entity boundaries. , Indicates span area information, , , The span represents the dimension. This indicates the embedding dimension of the relative position information of entity boundaries. This indicates the embedding dimension of the span region information, the span region information This is a feature that represents the number of nested entity levels of each word within the span of a word pair. Its value corresponds to the number of nested entity levels of the word.

2. The nested entity recognition method based on a bi-triangular affine mechanism according to claim 1, characterized in that, The nested entity recognition model includes an encoding layer, a bi-trigonal affine mechanism layer, and a prediction layer; wherein... Encoding layer: Using pre-trained language models BERT and BiLSTM as encoders, the target nested entities are encoded to produce vector representations of words containing context; Three-affine mechanism layer: In the three-affine mechanism layer, the first stage uses a three-affine attention mechanism to create a span representation. The second stage fuses the span representation with the relative position and region information of the entity boundary. The three-affine mechanism is then used again to interact the boundary with the fused information in a higher order to obtain the entity span representation and obtain the first probability value. Prediction layer: Entity prediction is performed by using the first probability value together with a dual affine predictor.

3. The nested entity recognition method based on a bi-triangular affine mechanism according to claim 1, characterized in that, The expression for the entity span obtained using the three-affine mechanism is as follows: ； in, Represents entity boundaries, This indicates the label size parameter. This refers to the fused information.

4. The nested entity recognition method based on a bi-triangular affine mechanism according to claim 1, characterized in that, Based on the entity span representation obtained from the three affine mechanism, the span is calculated from the first... i The word to the 1 j The span score of each word.

5. A nested entity recognition system based on a bi-triangular affine mechanism, characterized in that, It includes a data acquisition module, an encoding module, a first recognition module, a second recognition module, and a prediction module; among which, The data acquisition module is used to acquire the target nested entity; The encoding module is used to encode each character of the target nested entity to obtain a vector representation of the character; The first recognition module is used to process the vector representation of the character using a nested entity recognition model that fuses position and content information based on a bi-triple affine mechanism to obtain all the span representations of the character for entity recognition, and to obtain a first probability value; The steps for entity recognition using the nested entity recognition model are as follows: The vector representation based on the character is used to calculate the span representation through a three-affine mechanism. The relative position information of the entity boundary and the span region information are then integrated into the span representation to obtain fused information. By utilizing a three-affine mechanism, the entity boundary interacts with the fused information to obtain an entity span representation; The entity span representation is used in conjunction with a biaffine mechanism to predict the category of each sub-entity in the target nested entity; The expression representing the span is: ； ； ； in, This indicates information from the start position to the end position. This indicates a three-affine mechanism. Representing three-affine attention, with entity boundaries Dimensional parameters representing entity span The queries used as an attention mechanism are generated in dimensionality . span representation ; The expression for the fused information is: ； in, Indicates the span. Indicates the relative position information of entity boundaries. , Indicates span area information, , , The span represents the dimension. This indicates the embedding dimension of the relative position information of entity boundaries. This indicates the embedding dimension of the span region information, the span region information This is a feature information representing the number of nested entity levels of each word within the span of a word pair; its value corresponds to the number of entity nesting levels where the word is located. The second recognition module is used to map the vector representation of the character through a double affine mechanism to obtain a span representation, thereby obtaining a second probability value for entity recognition; The prediction module is used to add the first probability value and the second probability value to predict the category of each sub-entity in the target nested entity.

6. A computer storage medium, characterized in that, The computer storage medium stores a computer program, which, when executed by a processor, implements the steps of a nested entity recognition method based on a bi-triangular affine mechanism as described in any one of claims 1-4.

Images