A unified named entity recognition method and device based on a two-stage architecture

Abstract

The application discloses a unified named entity recognition method and device based on a two-stage architecture, and the method comprises the following steps: in the first stage, a segment extractor enumerates and classifies all segments in a text to extract entity segments; the entity segment is defined as being capable of constituting an entity alone or being a constituent segment of an entity; in the second stage, a segment pair classifier combines entity segments two by two and classifies the entity segment combination; the relation classification comprises a Next-Fragment relation type and an Overlapped relation type, which are respectively used for discontinuous NER and nested NER; in the training process, multi-task learning is adopted to jointly train the first stage and the second stage.

Description

Technical Field

[0001] This invention relates to the field of named entity recognition technology in natural language processing, and in particular to a unified named entity recognition method and apparatus based on a two-stage architecture. Background Technology

[0002] Named entity recognition (NER) aims to identify text segments that represent entities. It has become a fundamental task in the field of natural language processing, playing a crucial role in various knowledge-based applications such as entity linking and data mining.

[0003] Research on named entity recognition has evolved from early conventional named entity recognition to nested named entity recognition, and more recently, discontinuous named entity recognition. Specifically, conventional named entity recognition simply detects entity references in text and their types; nested named entity recognition detects potentially nested named entities in text and their types; and discontinuous named entity recognition detects named entities in text that contain many discontinuous segments and their types.

[0004] Nested named entity recognition and discontinuous named entity recognition are more complex problems because the entity references to be identified may contain multiple irregular nested or discontinuous segments. Discontinuous named entity recognition is very important, especially in the medical field, where many medical and disease datasets, including CADEC and ShARe13 datasets, contain a large number of discontinuous entities.

[0005] In recent years, numerous methods have been researched to solve the three Named Entity Recognition (NER) tasks. Most of these methods focus on solving regular and overlapping NER problems, with very few addressing discontinuous NER. Named Entity Recognition aims to develop a unified framework capable of simultaneously solving regular, nested, and discontinuous NER. For NER, the BART-LARGE model employs a generative approach, generating positional indices of entity fragments end-to-end and combining these indices to obtain the NER result. However, generative methods often suffer from exposure bias; inconsistencies between text generation during training and inference can severely impact the generated results. In contrast, the W2NER model achieves better results. This model transforms the NER task into a word pair classification task, classifying two relationships between words in a sentence and then combining the words to obtain the NER result. However, this method also faces problems such as low efficiency and word pair redundancy.

[0006] In existing technologies, fragment-based models can naturally solve the named entity recognition problem, but they cannot handle the recognition of discontinuous named entities. Therefore, we propose a unified named entity recognition method based on a two-stage architecture and provide the relevant apparatus. Summary of the Invention

[0007] This invention aims to address at least one of the technical problems existing in the prior art. To this end, this invention discloses a unified named entity recognition method and apparatus based on a two-stage architecture. The method models the named entity recognition problem as a pairwise classification problem of entity fragments. The relationships between entity fragment pairs describe the semantic connections between different fragments, and are therefore crucial for recognizing nested and discontinuous entities. The unified named entity recognition problem is solved through the following two stages: In the first stage, a fragment extractor enumerates and classifies all fragments in the text to extract entity fragments; in the second stage, a fragment pairwise classifier combines entity fragments pairwise and classifies their relationships; during training, multi-task learning is employed to jointly train the two stages.

[0008] The technical solution of the present invention is as follows: a unified named entity recognition method based on a two-stage architecture, the method comprising:

[0009] In the first stage, the fragment extractor enumerates and classifies all fragments in the text to extract entity fragments; the entity fragment is defined as: capable of forming an entity on its own, or a component fragment of an entity.

[0010] In the second stage, the fragment pairwise classifier combines entity fragments in pairs and performs relation classification on the entity fragment combinations; the relation classification includes Next-Fragment relation type and Overlapped relation type, which are used for discontinuous NER and nested NER, respectively;

[0011] During the training process, multi-task learning is used to jointly train the first and second phases.

[0012] Specifically, the fragment extractor aims to find all text fragments and determine whether these fragments constitute entities, including the following steps:

[0013] Given input text X = {x1, x2, ..., x...} N}, where N is the text length, and a maximum segment length L is set. First, enumerate the text segments to obtain the candidate segment set S(X) = {s (1,1) ,…,s (1,L) ,…,s (N-L+1,N) ,…,s (N,N)}, s (i,j)Candidate segments starting at position i and ending at position j;

[0014] Group adjacent segments with the same starting position into one group;

[0015] For each group, a template is constructed. The template is the concatenation of the segment identifiers corresponding to all segments in that group. For candidate segment s i The identifier corresponding to the segment has the same positional embedding as the start and end words of the segment;

[0016] After inserting the template, the embedding position of each word in the text remains unchanged;

[0017] Each template is expanded to the input text, and the final sequence is then input into the first BERT encoder module;

[0018] The embedding of the entire sequence was obtained. in It is the length of the expanded sequence; fragment s (a,b) The context embedding of the identifier is and Next, the embedding of the fragment is calculated as follows:

[0019]

[0020] Where w represents the embedding of the fragment length feature, and [;] represents the vector concatenation operation;

[0021] Using a multilayer perceptron for fragment type classification:

[0022] p1(e|s (a,b) =Softmax(MLP1(h(s)) (a,b) )))

[0023] Where MLP1 represents a multilayer perceptron, p1 represents the probability distribution of entity fragment type e∈ε∪{none}, ε represents a predefined set of entity types, and if e∈none, it indicates that fragment s (a,b) "None" indicates a non-entity fragment.

[0024] Specifically, the purpose of the segment pairwise classifier is to determine the pairwise relationships between segments, including the following steps:

[0025] Given all the entity fragments identified in the first stage, one fragment is sequentially designated as the preceding fragment, and the fragments following it in the sentence are designated as the following fragments. The preceding fragment and all its corresponding following fragments are then grouped together, i.e., the preceding fragment s is grouped into a single group. (a,b) and all corresponding subsequent fragments Package them into a group;

[0026] For each group, a typed template is constructed, which is a concatenation of the identifiers of all fragment pairs in that group; for each group... The previous segment s (a,b) logo <F:e x >and < / F:e x Insert before or after this segment into the text;

[0027] Then, the identifiers of the subsequent segments are concatenated in sequence, and the identifier pairs still have the same positional embedding as the start and end words of the corresponding segments. The typified template consists of the preceding identifier and all the following identifiers fixed in the text, and the template can capture the dependencies between candidate segment pairs.

[0028] Append a typed template to the input text to obtain the entire sequence;

[0029] The entire sequence is input into the second BERT encoder module. After BERT computation, the embedding of the entire sequence is obtained. For segment pairs s (a,b) and s (c,d) , the previous fragments (a,b) The embedding of the identifier is x a-1 and x b+1 The following fragment s (c,d) The embedding of the identifier is and Therefore, the embedding of the fragment pair is calculated as follows:

[0030]

[0031] Where [;] denotes vector concatenation operation;

[0032] Next, a multilayer perceptron is used to classify the types of fragment pairs:

[0033] p2(r|X)=Softmax(MLP2(h(s (a,b) ,s (c,d) )))

[0034] Where MLP2 represents a multilayer perceptron, p2 represents the probability distribution of the type r∈R of entity fragment pairs, and R is a predefined set of relation types, including Next-Fragment, Overlapped, and OTHER relations.

[0035] Preferably, during training, the segment extractor and the segment pair classifier are jointly trained through multi-task learning, and the training objective is to minimize the negative log-likelihood loss in both stages:

[0036]

[0037] Among them, e* and r * The standard labels represent entity fragments and entity pairs, and α and β are hyperparameters that control the influence of the fragment extractor and fragment pair classifier on the overall training objective.

[0038] Furthermore, a syntactic dependency parser is used to convert the text sequence after template concatenation into an adjacency matrix A, where A ij =1 indicates that word x in the text i To x j There is a syntactic dependency edge between them, otherwise A ij =0;

[0039] The syntactic adjacency matrix is ​​encoded using the attention-guided graph convolutional network AGGCN; the AGGCN module transforms the adjacency matrix A into an attention-guided adjacency matrix. To improve the encoding capability of the original graph convolutional neural network module, among which From word x i To x j The weight of the syntactic dependency edges between them;

[0040] The adjacency matrix of the t-th attention head The following multi-head attention calculation is used:

[0041]

[0042] in and It is X t The parameter matrix projected onto the query value and key value, where d is the dimension of the input text embedding. It is the adjacency matrix guided by the attention of the t-th attention head, t≤N head And N head It is a hyperparameter, X t It is the embedding of word x at the t-th attention head;

[0043] The AGGCN module uses a dense connection layer for... Update X, output X represents the original embedding of word x calculated by BERT. The word x is used Perform the updated embedding at layer t, and then integrate the information from different layers:

[0044]

[0045] Where W1 is the weight, This represents the embedding of word x after information integration through multiple densely connected layers;

[0046] Will Concatenate with the original word embedding to obtain the final word embedding. Where W2 represents the linear transformation used for dimensionality reduction;

[0047] The final syntactic enhancement embedding X' is used for the segment and segment pair representations in the first and second stages.

[0048] A named entity recognition device based on a two-stage architecture includes:

[0049] processor;

[0050] And, a memory for storing the executable instructions of the processor;

[0051] The processor is configured to execute the named entity recognition method described above by executing executable instructions.

[0052] Compared with existing methods, the advantages of the present invention are as follows: It identifies the lack of effective solutions for the current named entity recognition task and proposes a unified named entity recognition method, T2NER, based on a two-stage architecture, along with related devices, to promote the development of the named entity recognition field; by modeling the unified NER task as a pairwise classification problem of entity fragment pairs, the named entity recognition task can be naturally solved using a model based on a two-stage architecture; to address the inefficiency of traditional fragment-based models, a 'template' is configured in this architecture, enabling parallel training and inference of the model, thus improving model efficiency; furthermore, a neighbor-packaging strategy and a latter-packaging strategy are designed to model entity fragments and fragment pairs, achieving more accurate named entity recognition. Attached Figure Description

[0053] Figure 1 A flowchart illustrating an embodiment of the present invention is shown;

[0054] Figure 2 The following are schematic diagrams of the structures of various parts in an embodiment of the present invention;

[0055] Figure 3 The algorithm flowchart for the decoding process is shown. Detailed Implementation

[0056] To make the objectives, technical solutions, and advantages of this invention clearer, the invention will be further described in detail below with reference to the accompanying drawings. Obviously, the described embodiments are merely some embodiments of this invention, and not all embodiments. Based on the embodiments of this invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this invention.

[0057] It should be understood that the specific embodiments described herein are merely illustrative of the invention and are not intended to limit the invention.

[0058] First, we define the entity fragment relationship types and the identifiers that constitute the template. Then, we formalize the named entity recognition problem.

[0059] Definition 1. Entity Fragment Pair Relationship: Defines three types of relationships between entity fragment pairs: OTHER relationship indicates that there are other relationships between entity fragment pairs or that there is no nesting or discontinuous relationship; Next-Fragment indicates that entity fragment pairs belong to the same entity and are consecutive; Overlapped indicates that the two entity fragments are nested.

[0060] Definition 2. Fragment Identifier: An explicit pair of identifiers inserted before and after the i-th candidate entity fragment. and Used to indicate this segment.

[0061] Definition 3. Fragment Pair Identifier: Explicitly inserting two pairs of identifiers before and after a pair of candidate entity fragments. <F:e x >, < / F:e x >, <L:e y >and < / L:e y >, where e x ,e y ∈ε, where ε represents a predefined set of entity types.

[0062] T2NER Task: Regular, nested, and discontinuous NER can be formalized as follows: Given an input text X = {x1, x2, ..., xn} with N words... N The goal of the first-stage fragment extractor is to detect all entity fragments s in the input text X with a length less than L by encoding and classifying fragment identifiers. (a,b) ={x a ,x a+1 ,…,x b}, and its entity type e∈ε. After the first stage, regular entities and nested entities can be identified. Then, the second stage fragment pair classifier aims to identify entity fragments s. (a,b) and s (c,d) As input, the semantic relationship r∈R between fragment pairs is identified by classifying fragment pair identifiers, where R is a predefined set of relation types, including Next-Fragment, Overlapped, and OTHER relations. After the second stage, discontinuous entities can be identified, and the regular and nested entities identified in the first stage can be reconfirmed.

[0063] like Figure 1 The above describes a unified named entity recognition method based on a two-stage architecture, the method comprising:

[0064] In the first stage, the fragment extractor enumerates and classifies all fragments in the text to extract entity fragments; the entity fragment is defined as: capable of forming an entity on its own, or a component fragment of an entity.

[0065] In the second stage, the fragment pairwise classifier combines entity fragments in pairs and performs relation classification on the entity fragment combinations; the relation classification includes Next-Fragment relation type and Overlapped relation type, which are used for discontinuous NER and nested NER, respectively;

[0066] During the training process, multi-task learning is used to jointly train the first and second phases.

[0067] like Figure 2 As shown, the embodiments of the present invention mainly consist of four parts: (1) a fragment extractor, (2) a fragment pair classifier, (3) fragment modeling enhanced with syntactic information, and (4) decoding and joint training. The input is “I am having aching in legs and shoulders.” (a) The fragment extractor in the first stage uses a neighbor-packing strategy to construct templates; (b) The fragment pair classifier in the second stage uses the latter packing strategy to construct templates; (c) We use the AGGCN module to encode syntactic information into vector embeddings to enhance the representation of fragments and fragment pairs. (d) The two stages are jointly trained, and the entity results are decoded using Next-Fragment relations.

[0068] (1) Fragment Extractor

[0069] The purpose of fragment extraction is to find all text fragments and determine whether these fragments constitute entities.

[0070] Templates are constructed for each input text to perform approximation calculations, thereby accelerating the reasoning process. These templates consist of span markers. Specifically, given the input text X = {x1, x2, ..., x...} N Given a maximum segment length L, we first enumerate text segments to obtain a candidate segment set S(X) = {s (1,1) ,…,s (1,L) ,…,s (N-L+1,N) ,…,s (N,N)}

[0071] Training and inferring each candidate span separately requires significant computational cost. To mitigate this issue, these spans are first grouped into multiple instances. To fully model span boundaries, a proximity-packing strategy is proposed. Adjacent segments with the same start position are grouped together. For example, segments {s} are grouped... (1,1) ,s (1,2) ,…,s(1,L)} are grouped into group S1.

[0072] Then, a template is constructed for each group. The template is a concatenation of the fragment identifiers corresponding to all fragments in that group. Specifically, for candidate fragment s... i The identifier corresponding to the segment has the same positional embedding as the start and end words of the segment, that is:

[0073] p( ),p( :=p(x start(i) ),p(x end(i) )

[0074] Therefore, after inserting the template, the embedding positions of each word in the original text remain unchanged. The template for group S1 is... <m1>< / m1> <m2>< / m2> … <ml>< / ml> .

[0075] Finally, each template is augmented onto the input text, and the final sequence is input into the BERT encoder module. To reuse text word representations, we use a directed attention mask matrix in BERT's attention layer. Specifically, text word embeddings focus only on the text words themselves, not on fragment identifiers, while fragment identifiers can focus on all text words and their associated identifiers. Therefore, we can achieve distributed processing of all groups across multiple runs, and batch processing of fragments within each group in a single run.

[0076] After BERT computation, we obtain the embedding of the entire sequence. in This is the length of the expanded sequence. (Segment s) (a,b) The context embedding of the identifier is and Next, the embedding of the fragment is calculated as follows:

[0077]

[0078] Where w represents the embedding of the fragment length feature, and [;] represents the vector concatenation operation.

[0079] Next, a multilayer perceptron is used to classify fragment types:

[0080] p1(e|s (a,b) =Softmax(MLP1(h(s)) (a,b) ))

[0081] Where p1 represents the probability distribution of entity fragment type e∈ε∪{none}, ε represents a predefined set of entity types, and if e∈none, it indicates that fragment s (a,b) It is not a physical fragment.

[0082] Finally, the identified entity fragments are used as input for the second stage.

[0083] (2) Pair classifier

[0084] The segment pairwise classifier takes candidate segments as input and aims to determine the pairwise relationships between segments. To achieve overall modeling and representation of different segment pairs, we construct typed templates for each input sequence and learn the embeddings of segment pairs through these templates.

[0085] Similarly, to control computational costs, we group all candidate fragment pairs into multiple groups. To learn discriminative representations of different fragment pairs, we propose the latter grouping strategy. As shown, fragment pairs with the same preceding fragment are clustered together. This strategy enables holistic modeling of different fragment pairs.

[0086] Specifically, given all the entity fragments identified in the first stage, we sequentially designate one fragment as the preceding fragment and the fragments following it in the sentence as the following fragments. We then group the preceding fragment and all its corresponding following fragments into a single group. That is, we group the preceding fragment s... (a,b) and all corresponding subsequent fragments Package them into a group.

[0087] Then, a typed template is constructed for each group, which is a concatenation of the identifiers of all fragment pairs in that group. Specifically, for group... We will use the previous segment s (a,b) logo <F:e x >and < / F:e x Insert the fragment before or after the text. Then concatenate the identifiers of the later fragments in order, resulting in:

[0088] <l1:e1>< / l1:e1> <l2:e2>< / l2:e2> ... <Lm:e m > < / Lm:e m >

[0089] These identifiers still have the same positional embeddings as the start and end words of the corresponding segments. The typed templates consist of pre-identifiers and all post-identifiers fixed in the text. These templates can capture the dependencies between candidate segment pairs.

[0090] Finally, the typed template is appended to the input text. The entire sequence is:

[0091]

[0092] We input the sequence into another BERT encoder module and used an attention mask matrix similar to that used in Stage 1. After BERT computation, we obtained the embedding of the entire sequence. in is the length of the expanded sequence. For a fragment pair s... (a,b) and s (c,d) , the previous fragments (a,b) The embedding of the identifier is x a-1 and x n+1 The following fragment s (c,d) The embedding of the identifier is and Therefore, the embedding of the fragment pair is calculated as follows:

[0093]

[0094] Where [;] represents vector concatenation operation.

[0095] Next, a multilayer perceptron is used to classify the types of fragment pairs:

[0096] p2(r|X)=Softmax(MLP2(h(s (a,b) ,s (c,d) )))

[0097] Where p2 represents the probability distribution of the type r∈R of the entity fragment pair, and R is a predefined set of relation types, including Next-Fragment, Overlapped and OTHER relations.

[0098] (3) Enhance the representation of segments and segment pairs using syntactic information

[0099] In previous named entity recognition work, syntactic dependency information has often been ignored. Here, we use syntactic dependency information to enhance our model. Specifically, we utilize a syntactic dependency parser to convert the text sequence after template concatenation into an adjacency matrix A, where A... ij =1 indicates that word x in the text i To x j There is a syntactic dependency edge between them, otherwise A ij =0. It is worth noting that each identifier word and its corresponding text word share the same syntactic information.

[0100] We then encode the syntactic adjacency matrix using an attention-guided graph convolutional network (AGGCN). To illustrate the AGGCN module, we first introduce the traditional graph convolutional neural network (GCN). Given the embedding of the concatenated text... in This is the length of the concatenated text. Next, the GCN module updates the text embedding using syntactic information, as follows:

[0101]

[0102] Among them W(l) and b (l) is the weight matrix and bias vector of the l-th layer of the GCN module. A is the adjacency matrix of the obtained syntactic information.

[0103] The AGGCN module transforms the adjacency matrix A into an attention-guided adjacency matrix. To improve the encoding capabilities of the original GCN module, among which From word x i To x j The weights of the syntactic dependency edges between attention heads. The adjacency matrix of the t-th attention head. The following multi-head attention calculation is used:

[0104]

[0105] in and It is X t The parameter matrix projected onto the query value and key value, where d is the dimension of the input text embedding. It is the adjacency matrix guided by the attention of the t-th attention head, t≤N head And N head It is a hyperparameter, X t It is the embedding of word x in the t-th attention head.

[0106] The AGGCN module uses a dense connection layer for... Update X, output X represents the original embedding of word x calculated by BERT. The word x is used Perform the updated embedding at layer t, and then integrate the information from different layers:

[0107]

[0108] Where W1 is the weight, This indicates the embedding of word x after information integration through multiple densely connected layers.

[0109] Then Concatenate with the original word embedding to obtain the final word embedding. Where W2 represents the linear transformation used for dimensionality reduction. The final syntax-enhanced embedding X' is used for the segment and segment pair representations in the first and second stages.

[0110] (4) Decoding and Joint Training

[0111] Our model predicts entity fragments and the pairwise relationships between them, which can be viewed as a directed fragment graph. The decoding goal is to find subgraphs within the fragment graph, where each entity fragment is connected to other fragments via Next-Fragment relationships. Each subgraph corresponds to one entity, and only the subgraph of a single fragment itself constitutes an entity. Figure 3 The algorithm in the image demonstrates the decoding process.

[0112] During training, we jointly train the segment extractor and the segment pair classifier through multi-task learning. The training objective is to minimize the negative log-likelihood loss in both stages:

[0113]

[0114] Among them, e * and r * The standard labels represent entity fragments and entity pairs, and α and β are hyperparameters that control the influence of the fragment extractor and fragment pair classifier on the overall training objective.

[0115] The present invention also discloses an electronic device, comprising:

[0116] processor;

[0117] And, a memory for storing the executable instructions of the processor;

[0118] The processor is configured to execute the aforementioned unified entity recognition method by executing the executable instructions.

[0119] The main technical effects of this patent application include the following:

[0120] This invention addresses the unified named entity recognition (NER) problem through two stages: In the first stage, a fragment extractor enumerates and classifies all fragments in the text to extract entity fragments (an entity fragment is defined as either a single entity or a component of an entity); in the second stage, a fragment pairwise classifier combines entity fragments in pairs and classifies their relationships (for this purpose, we define two types of relationships between fragment pairs: Next-Fragment relationships and Overlapped relationships, used for discontinuous NER and nested NER, respectively). During training, multi-task learning is employed to jointly train both stages of the architecture.

[0121] However, this two-stage architecture has three problems: 1) Inefficiency: the framework needs to classify all candidate fragments and fragment pairs separately, resulting in high model complexity; 2) Insufficient modeling of fragment boundaries in the first stage: fragment boundaries are very important for entity classification in NER, but the framework has not yet been able to learn fragment boundary information well; 3) Failure to learn discriminative information of fragment pairs in the second stage: fragment pair representation directly affects the classification of fragment pairs, but the framework has failed to learn the differential representation of fragment pairs from other fragment pairs.

[0122] To address these issues, this method proposes configuring 'templates' for the two-stage architecture, essentially a set of 'tags' to indicate the start and end positions of segments. The insertion of 'templates' allows the model to reuse word embeddings for each word in the text across multiple training batches, enabling batch processing of the data. For issue 2), a proximity-packaging strategy is proposed, which models the boundaries of neighboring segments by packing them into a single training instance in the first stage. For issue 3), a second packing strategy is proposed, which models and represents these segment pairs by packing them into a single training instance in the second stage. Furthermore, syntactic information is utilized to enhance the embedding representations of segments and segment pairs.

[0123] Those skilled in the art will understand that embodiments of this application can be provided as methods, systems, or computer program products. Therefore, this application can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, this application can take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

Claims

1. A unified named entity recognition method based on a two-stage architecture, characterized in that, The method includes: In the first stage, the fragment extractor enumerates and classifies all fragments in the text to extract entity fragments; the entity fragment is defined as: capable of forming an entity on its own, or a component fragment of an entity. In the second stage, the fragment pairwise classifier combines entity fragments in pairs and performs relation classification on the entity fragment combinations; the relation classification includes Next-Fragment relation type and Overlapped relation type, which are used for discontinuous NER and nested NER, respectively; During the training process, multi-task learning is used to jointly train the first and second phases. The purpose of the fragment extractor is to find all text fragments and determine whether these fragments constitute entities, including the following steps: Given input text , This refers to the text length; setting a maximum segment length. First, enumerate the text segments to obtain a set of candidate segments. , The starting position is End position is Candidate fragments; Group adjacent segments with the same starting position into one group; For each group, a template is constructed. The template is a concatenation of the segment identifiers corresponding to all segments in that group. For candidate segments... The identifier corresponding to the segment has the same positional embedding as the start and end words of the segment; After inserting the template, the embedding position of each word in the text remains unchanged; Each template is expanded to the input text, and the final sequence is then input into the first BERT encoder module; The embedding of the entire sequence was obtained. It is the length of the expanded sequence; fragment The context embedding of the identifier is and Next, the embedding of the fragment is calculated as follows: in, Embedding representing fragment length features, This represents a vector concatenation operation; Using a multilayer perceptron for fragment type classification: in, This represents a multilayer perceptron. Indicates entity fragment type The probability distribution, Represents a predefined set of entity types, if This indicates a fragment It is not a physical fragment. This represents a non-entity fragment.

2. The unified named entity recognition method based on a two-stage architecture according to claim 1, characterized in that, The purpose of the segment pairwise classifier is to determine the pairwise relationships between segments, including the following steps: Given all the entity fragments identified in the first stage, one fragment is sequentially designated as the preceding fragment, and the fragments following it in the sentence are designated as the following fragments. The preceding fragment and all its corresponding following fragments are then grouped together; that is, the preceding fragment is grouped... and all corresponding subsequent fragments Package them into a group; For each group, a typed template is constructed, which is a concatenation of the identifiers of all fragment pairs in that group; for each group... , the previous segment logo <F: >and < / F: Insert before or after this segment into the text; Then, the identifiers of the subsequent segments are concatenated in sequence, and the identifier pairs still have the same positional embedding as the start and end words of the corresponding segments. The typified template consists of the preceding identifier and all the following identifiers fixed in the text, and the template can capture the dependencies between candidate segment pairs. Append a typed template to the input text to obtain the entire sequence; The entire sequence is input into the second BERT encoder module. After BERT computation, the embedding of the entire sequence is obtained, for each segment pair. and Pre-recorded fragments The embedding of the identifier is and The following segment The embedding of the identifier is and Therefore, the embedding calculation for fragment pairs is as follows: in This represents a vector concatenation operation; Next, a multilayer perceptron is used to classify the types of fragment pairs: in, This represents a multilayer perceptron. Indicates the type of entity fragment pairs The probability distribution, It is a predefined set of relationship types, including Next-Fragment, Overlapped, and OTHER relationships.

3. The unified named entity recognition method based on a two-stage architecture according to claim 2, characterized in that, During training, the fragment extractor and the fragment pairwise classifier are jointly trained through multi-task learning. The training objective is to minimize the negative log-likelihood loss in both stages. in, and Standard labels representing entity fragments and entity pairs and These are hyperparameters that control the impact of the fragment extractor and the fragment pair classifier on the overall training objective.

4. The unified named entity recognition method based on a two-stage architecture according to claim 3, characterized in that, A syntactic dependency parser is used to convert the concatenated text sequence into an adjacency matrix. ,in Indicates words in the text arrive There is a syntactic dependency edge between them, otherwise ; The syntactic adjacency matrix is ​​encoded using the attention-guided graph convolutional network AGGCN; The AGGCN module uses the adjacency matrix Transformed into an attention-guided adjacency matrix To improve the encoding capability of the original graph convolutional neural network module, among which From words arrive The weight of the syntactic dependency edges between them; No. Adjacency matrix of attention heads The following multi-head attention calculation is used: in and It is The parameter matrix projected onto the query value and the key value. It is the dimension of the input text embedding. It is the first Attention-guided adjacency matrix for each attention head. and It is a hyperparameter. It is a word In the Embedding of attention heads; The AGGCN module uses a dense connection layer for... renew Output , Indicator The original embedding calculated by BERT Indicator use Conduct the first The updated embeddings are then processed, and information from different layers is integrated. in, It's weight. This refers to words that have undergone information integration through multiple densely connected layers. Embedding; Will Concatenate with the original embedding to obtain the final word embedding. ,in, This represents a linear transformation used for dimensionality reduction; Final word embedding Segment and segment pair representations used for the first and second phases.

5. A named entity recognition device based on a two-stage architecture, characterized in that, include: processor; And, a memory for storing the executable instructions of the processor; The executable instructions configured in the processor are the unified named entity recognition method based on a two-stage architecture as described in any one of claims 1 to 4.

Images