N-gram cross attention enhancement method and device for complex nested named entity recognition

Abstract

The invention discloses a complex nested named entity recognition-oriented N-gram cross attention enhancement method and device. The method comprises the following steps of: encoding an input original sentence; segmenting the encoded sentence into a sequence with a fixed length by using an N-grams method; respectively mapping each N-gram sequence group and the original sentence to a new space by using set linear mapping in each category space; calculating the zoom dot product similarity between each N-gram sequence group matrix and the original sentence matrix; and performing maximum pooling operation to obtain a similarity matrix of the N-gram sequence group matrix, and further processing to output a logits score. According to the method, a multi-label and single-label multi-classification task frame is designed, global and local context information can be considered at the same time, and the accuracy and robustness of an NER task in processing nested and multi-label entities are improved.

Description

Technical Field

[0001] The present invention belongs to the technical field of natural language processing, and in particular relates to an N-gram cross-attention enhancement method and device for complex nested named entity recognition. Background Art

[0002] Named entity recognition (NER) is a basic task in the field of natural language processing (NLP). It aims to identify entities with specific meanings from text, such as names of people, places, and organizations. These entities are crucial for understanding text content and extracting information. For example, in a news report, identifying names of people can help people understand the main characters in the report, while place names can help people determine the location of the event. With the deepening of research, nested named entity recognition (NNER), as a special subclass of named entity recognition, focuses on identifying entities nested inside other external entities. This recognition is particularly important for some fields. For example, in the biomedical field, a molecule name may belong to both the "protein" and "DNA" categories. This requires a technology that can identify and distinguish entities at different levels. Early implementation technologies mainly used conditional random fields (CRFs), a statistical modeling method, to identify entities through sequence annotation (BIO). In BIO annotation, B (Begin) marks the beginning of the entity, I (Inside) marks the middle or end of the entity, and O (Other) represents the non-entity part. For example Figure 1 As shown, for a sentence "The Wall Street Journal", it can be identified that "Wall Street" is an entity "Street" (STREET), and "The Wall Street Journal" is an "Organization" entity (ORG).

[0003] In recent years, with the development of deep learning technology, deep learning-based models, such as bidirectional long short-term memory networks (LSTM) or convolutional neural networks (CNN) with conditional random fields (CRF), have significantly improved the performance of named entity recognition tasks by enhancing character-level features and contextual information. These models are better able to capture complex patterns and dependencies in text. However, with the development of nested named entity recognition, traditional conditional random field methods face challenges because they are limited by problems such as "single label constraints" and "hierarchical errors".

[0004] In order to solve these problems, scholars proposed to introduce a multi-head attention mechanism to handle complex named entity recognition tasks, where each "head" represents a different "entity category label" and the named entity recognition task is regarded as a binary multi-label classification problem between the start-end character similarities. This method can more flexibly handle the nested relationship and multi-label characteristics of entities, such as Figure 2These models usually treat the sequence as a whole when calculating and use global context information, but this approach ignores the locally available context information.

[0005] Existing technologies have made some progress in NER tasks, but there are still some problems and shortcomings when dealing with nested entities and multi-label entities:

[0006] (1) Lack of local context information:

[0007] When dealing with NER tasks, existing models often rely on global context information to capture the semantics of entities in text, ignoring local context information closely related to specific entities, resulting in inaccurate or incomplete recognition of entities. Although this approach can provide a certain degree of contextual understanding, it may not be sufficient to handle complex nested relationships.

[0008] Local context information, that is, words and phrases closely related to a specific entity, is crucial to understanding the specific meaning and boundaries of the entity. For example, in fields such as biomedicine, the recognition of an entity is highly dependent on the words and phrases surrounding it, and existing technologies cannot effectively capture this local context information, limiting the performance of the model in professional domain texts. In a biomedical sentence, a gene name may consist of only a few letters, and its meaning and function may be clarified only by a few words around it. If the model cannot effectively capture this local context information, it may lead to inaccurate or incomplete recognition of entities.

[0009] In addition, global context information may become obscure or irrelevant when dealing with long-distance dependencies, especially in long sentences or complex-structured texts. This may cause the model to have difficulty recognizing nested entities that span multiple parts of a sentence. Existing models have difficulty maintaining the clarity of long-distance dependencies when dealing with long sentences or complex-structured texts, which affects the recognition of nested entities that span multiple parts of a sentence. Therefore, the inadequacy of existing technologies in capturing local context information limits their effectiveness in dealing with complex NER tasks.

[0010] (2) Unified paradigm of multi-label and single-label:

[0011] Existing named entity recognition models are usually based on single-label multi-classification tasks, which is widely adopted due to its low model optimization complexity. Therefore, when facing multi-label datasets, existing models usually need to be constantly adjusted and optimized, which not only increases the complexity of the model, but may also affect the performance of the model. Summary of the invention

[0012] Technical purpose: In view of the above technical problems, the present invention proposes an N-gram cross-attention enhancement method and device for complex nested named entity recognition, aiming to solve the problem of missing local context information in existing named entity recognition technology when processing complex text data. The present invention designs a multi-label and single-label multi-classification task framework, which can simultaneously consider global and local context information, and improve the accuracy and robustness of NER tasks when processing nested and multi-label entities.

[0013] Technical solution: To achieve the above technical objectives, the present invention adopts the following technical solution:

[0014] An N-gram cross-attention enhancement method for complex nested named entity recognition, characterized in that the nested named entity to be recognized is input into a pre-trained recognition model to obtain a recognition result;

[0015] The recognition model processes the nested named entity to be recognized, including the following steps:

[0016] (1) Encode the original input sentence and map each word in the original sentence into a high-dimensional space to form a word embedding;

[0017] (2) Use the N-grams method to split the encoded sentences into sequences of fixed length to obtain multiple N-gram sequence groups;

[0018] (3) Using the set linear mapping in each category space, each N-gram sequence group and the original sentence are mapped to a new space respectively, to obtain the original sentence matrix and multiple N-gram sequence group matrices;

[0019] (4) Calculate the scaled dot product similarity between each N-gram sequence group matrix and the original sentence matrix;

[0020] (5) Based on the calculated scaled dot product similarities of all N-gram sequence group matrices, a maximum pooling operation is performed to obtain a similarity matrix of the N-gram sequence group matrices;

[0021] (6) The similarity matrix of the obtained N-gram sequence group matrix is ​​processed through the Talking module and the logits score is output.

[0022] Preferably, in step (1), a pre-trained BERT model is used to encode the original sentence.

[0023] Preferably, in step (2), the encoded sentence is divided into L-n+1 sequences using the N-grams method, each sequence has n words, n is a predefined positive value ranging from 1 to L, and L represents the length of the sentence.

[0024] Preferably, in step (3), the set linear mapping is a set weight matrix, and the mapping is performed according to the following formula:

[0025] Q i =N i ·W Q

[0026] K=H·W K +bias

[0027] Among them, W Q ,W K They represent weight matrices, and bias represents bias values;

[0028] N i represents an N-gram sequence group, Q i Represents the weight matrix W Q N-gram sequence group matrix after linear mapping;

[0029] H represents the original sentence after encoding, and K represents the weight matrix W K The original sentence matrix after linear mapping.

[0030] Preferably, in step (4), the scaled dot product similarity between each N-gram sequence group matrix and the original sentence matrix is ​​calculated according to the following formula:

[0031]

[0032] in, It represents the dot product similarity between each N-gram sequence group matrix and each word in the original sentence matrix in each category space, K T Represents the weight matrix W K The transpose of the original sentence matrix after linear mapping, d k Represents the weight matrix W K The dimension size of the original sentence matrix after linear mapping, c i Indicates that in the c i The cross attention mechanism is calculated in the category space, and cross represents cross attention.

[0033] Preferably, in step (5), the maximum pooling operation is performed in the following steps:

[0034] Loop through each N-gram sequence group matrix corresponding to the high-dimensional matrix H of the encoded original sentence;

[0035] Calculate the dot product similarity between the currently extracted N-gram sequence group matrix and the original sentence matrix and put them into a list;

[0036] When the length of the list is equal to the preset value, the pooling algorithm is called to process the current data in the list to obtain the similarity matrix R of the current N-gram sequence group matrix, and the similarity matrix R of the current N-gram sequence group matrix is ​​put into the list to update the list;

[0037] After completing the pooling operation of all N-gram sequence group matrices, all similarity matrices R are obtained and output as the maximum pooling result, i.e., the similarity matrix of the N-gram sequence group;

[0038] Among them, the steps of the pooling algorithm include:

[0039] Get the current data in the list, including multiple N-gram sequence group matrices;

[0040] Concatenate the repeated positions in the obtained N-gram sequence group matrix and perform maximum pooling to eliminate the representation of repeated characters;

[0041] Reconstruct the matrices to obtain the similarity matrix R.

[0042] Preferably, in step (6), the Talking module includes a plurality of convolutional neural network units, an activation function module and an addition unit, which are expressed as follows:

[0043] R = Conv2d(R)

[0044] R ′ =GeLU(Conv2d(R))

[0045] Logits = R + R ′

[0046] Among them, Conv2d represents a two-dimensional convolution operation, GelU represents an activation function, and R ′ represents the output of the activation function unit, and logits represents the output of the addition unit.

[0047] Preferably, the loss function of the recognition model is designed as follows:

[0048]

[0049] Among them, i, j are the starting and ending indexes of the span, Pos α Represents the positive sample of entity type α, Neg α Represents the negative sample of entity type α, L α (i ,j) represents the Logits score of entity type α.

[0050] An N-gram cross-attention enhancement device for complex nested named entity recognition uses a pre-trained recognition model to process the nested named entity to be recognized and obtains the recognition result;

[0051] The recognition model includes:

[0052] The encoding module is used to encode the input original sentence and map each word in the original sentence into a high-dimensional space to form a word embedding;

[0053] The segmentation module uses the N-grams method to segment the encoded sentences into sequences of fixed length and obtain multiple N-gram sequence groups;

[0054] A mapping module is used to use a set linear mapping in each category space to map each N-gram sequence group and the original sentence to a new space, respectively, to obtain an original sentence matrix and multiple N-gram sequence group matrices of the same dimension;

[0055] A similarity calculation module is used to calculate the scaled dot product similarity between each N-gram sequence group matrix and the original sentence matrix;

[0056] The pooling module performs a maximum pooling operation based on the calculated scaled dot product similarities of all N-gram sequence group matrices to obtain a similarity matrix of the N-gram sequence group matrices;

[0057] The output module processes the similarity matrix of the obtained N-gram sequence group matrix through the Talking module and outputs the logits score.

[0058] A computer-readable storage medium having a computer program stored thereon, wherein the program, when executed by a processor, implements the method according to any one of claims 1 to 8.

[0059] Beneficial effects: Due to the adoption of the above technical solution, the present invention has the following beneficial effects:

[0060] 1. Improve recognition accuracy: By integrating local and global context information, the present invention enables the model to more accurately identify nested and multi-label entities, especially in texts in professional fields such as biomedicine and law.

[0061] 2. Enhance the generalization ability of the model: The present invention designs an optimized attention mechanism and multi-label processing strategy, so that the model has good generalization ability in different fields and different types of texts.

[0062] 3. Simplify model training and deployment: The present invention designs a unified multi-label classification framework, which reduces the frequent adjustment of the model architecture and makes the model training and deployment process simpler. BRIEF DESCRIPTION OF THE DRAWINGS

[0063] Figure 1 This is an example of a sentence to be recognized and processed;

[0064] Figure 2 A schematic diagram of the recognition result using a binary multi-label classification method in the prior art;

[0065] Figure 3 It is a schematic diagram of the overall framework of the recognition model of the present invention;

[0066] Figure 4 Flowchart of the N-gram Cross Attention calculation method;

[0067] Figure 5 This is the Talking module diagram;

[0068] Figure 6 This is a diagram of the experimental results of the nested named body recognition dataset;

[0069] Figure 7 This is the experimental result diagram of the flat named body recognition dataset. DETAILED DESCRIPTION

[0070] The embodiments of the present invention are described in detail below with reference to the accompanying drawings.

[0071] The present invention proposes a named entity recognition model based on N-gram cross attention mechanism, such as Figure 3 As shown in the figure, the model aims to improve the recognition effect of nested and multi-label entities. It optimizes the limitations of existing NER technology by integrating local context information and multi-label classification strategies, especially when dealing with complex and nested entity relationships.

[0072] The specific implementation method of the present invention uses the Pytorch deep learning training architecture and the Adam optimizer for training. Its specific representation can be described in natural language and expressed in mathematical formulas.

[0073] Assume that the input is a sentence X = (x1, x2..., xL) of length L tokens, and an entity can be represented as a tuple (c, s, e). Among them, s and e are the starting and ending positions of the entity in the sentence, and c represents the entity type. Then the final model output is a C×L×L label matrix, where L represents the length of the sentence and C represents the number of entity types. It is worth noting that there are L(L+1) / 2 valid spans in this case.

[0074] Then, its specific implementation method can be expressed as:

[0075] 1. Sentence encoding: Convert the sentence H into a matrix vector that can be recognized by a computer, usually through a pre-trained model such as BERT. Models such as BERT can map each word into a high-dimensional space to form word embeddings. These embeddings can capture word meaning and context information. When a specific sentence X consisting of L words is input, the specific process can be expressed by the following mathematical formula:

[0076] H=Encoder(X)

[0077] H is the vector representation of the sentence, and its vector dimension is L×V, where L is the number of words in the sentence and V is the dimension of each word vector. Encoder represents a public pre-trained model, such as "Bert-base-uncased".

[0078] 2. N-grams segmentation: This is a method of segmenting text into fixed-length sequences. For each word in a sentence, it is combined with the following n-1 words into an N-grams sequence group. The N-grams segmentation method is used to divide the sentence into L-n+1 groups, each with n words, such as (word 1, word 2...word n), (word 2, word 3...word n+1), etc. This segmentation method can capture local context information. The specific process can be expressed by the following mathematical formula:

[0079] N=N1,N2…,N L-n+1

[0080] N i =(H i ,H i+1 ..,H i+n-1 )

[0081] Where N i The dimension of is n×V, where n is a predefined positive value ranging from 1 to L.

[0082] 3. Linear mapping: For each N-gram sequence group and the entire sentence, a linear mapping (e.g., a weight matrix) is used to map them to a new space. This mapping can help extract and emphasize features that are more important for the task.

[0083] Q i =N i ·W Q

[0084] K=H·W K +bias

[0085] Where WQ ,W K is a differentiable weight matrix of dimension d×V, bias represents an additional bias. After linear mapping, Q i Represents the N-sequence group matrix N after linear transformation i , whose dimension is C×N×d; similarly, K represents the original sentence matrix H after linear transformation, whose dimension is C×L×d, and C represents the number of label categories, that is, the category space, which is set artificially. In the multi-label classification scenario, each category represents a category space.

[0086] 4. Scaled dot product similarity: This is a method to measure the similarity between two vectors. For each N-gram sequence group and the original sentence, calculate the scaled dot product similarity between them. Scaled dot product similarity is calculated by dividing the dot product of two vectors by the product of their norms, which avoids the similarity calculation deviation caused by different vector lengths. For L-n+1 N-gram sequence groups, L-n+1 scaled dot product similarities will be generated, where each scaled dot product similarity has a dimension of C×n×L, where C represents the number of categories. The specific calculation formula is as follows:

[0087]

[0088] in It represents the dot product similarity between each N-gram sequence group matrix and each word in the original sentence matrix in each category space. Its dimension is C×n×L, where C represents the number of categories. The processing flow is as follows: Figure 4 As shown, K T Represents the weight matrix W K The transpose of the original sentence matrix after linear mapping, d k Represents the weight matrix W K The dimension size of the original sentence matrix after linear mapping, c i Indicates that in the c i The cross attention mechanism is calculated in the category space, and cross represents cross attention.

[0089] 5. Max pooling: Max pooling is an aggregation operation used to extract the most important information from multiple similarity scores / dot product similarities. During max pooling, the similarity scores of corresponding positions are compared and the largest one is selected. For example, if there are two N-gram sequence groups with similarity scores [(word 1, word 2, word 3)] and [(word 2', word 3', word 4)], the result after max pooling will be [(word 1, max(word 2, word 2'), max(word 3, word 3'), word 4)]. This process can reduce the dimensionality of the output while retaining the most important information. The operation can be represented by the pseudo code shown in Tables 1 and 2.

[0090] Table 1 Aggregation algorithm

[0091]

[0092] Table 2 Pooling algorithm

[0093]

[0094]

[0095] List is an empty list when initialized. Whenever the length of the list is 2, the pooling algorithm will be used to merge the two inputs in the list into one output. That is, whenever there are two similarity matrices in the list, the pooling algorithm will be used to aggregate the two matrices and finally output one similarity matrix. R is the total matrix after multiple similarity matrices are concatenated and pooled.

[0096] 6. Output similarity matrix: After maximum pooling, we get an L×L similarity matrix R. Each element in this matrix represents the similarity between two words in the sentence.

[0097] 7. Talking module

[0098] The present invention particularly designs an innovative "talking" module, which can be used as Figure 5 As shown. This module is processed by multi-convolutional neural network (Multi-CNN), which enhances the model's ability to recognize the interaction between entity labels. It can be expressed as:

[0099] R = Conv2d(R)

[0100] R ′ =GeLU(Conv2d(R))

[0101] Logits = R + R ′

[0102] Conv2d stands for 2D convolution module, and GelU is the activation function. The final output logits dimension is C×L×L, where C represents the number of categories and L represents the length of the sentence. After the similarity matrix R passes through the Talking module, the score matrix P is obtained.

[0103] In the Talking module, in each category space, the convolutional neural network CNN can treat the L×L similarity matrix R as an image to capture more fine-grained local information; by interacting in multiple category spaces to capture local features, it is like "talking" to each space.

[0104] Finally, the model is back-propagated through the following loss function, which can be expressed as:

[0105]

[0106] Among them, i, j are the starting and ending indexes of the span, Pos α Represents the positive sample of entity type α, Neg α Represents the negative sample of entity type α, L α (i,j) represents the Logits score of entity type α. A sentence can be divided into multiple spans. For example, in the sentence "Nanjing Audit University", the span is set to [0,1], and the word segmentation can get "Nanjing". The span is set to [0,5], and the word segmentation can get "Nanjing Audit University".

[0107] In order to demonstrate the effectiveness of the above model, the present invention conducts experiments on two public biomedical nested datasets (The Chilean Waiting List corpus and GENIA) and four public biomedical English flat datasets (AnatEM, BC5CDR, BC4CHEMD, BC5CDR-chemical), and compares them with the best baseline model in the past. Figure 6 and Figure 7 shown.

[0108] It can be seen that in the GENIA dataset, the experimental results of the model of the present invention are the second best, slightly lower than the biaffine model by 1.11% in the F1 index. In the Chilean Waiting List corpus dataset, the model of the present invention obtained 79.63 points, which is 0.33 points higher than the current optimal model MLC model in the F1 index when using BERT as the encoder. In summary, the model of the present invention has achieved comparable performance to other powerful models, further demonstrating the strong potential of this multi-label model in nested entity recognition.

[0109] The F1 score of the model of the present invention in the data set AnatEM is close to the current most advanced system, with a difference of only -0.1%. In addition, compared with the current optimal system, the F1 score of the model of the present invention in the BC5CDR-chemical data set is 1.34% lower; the F1 score in the BC4CHEMD data set is 1.47% lower. At the same time, the F1 score in the BC5CDR data set is slightly lower than the current optimal system by 1.58%. This performance shows its robustness on different data sets. Although it did not surpass the performance on all data sets, it showed competitiveness on all data sets.

[0110] Examples:

[0111] This example will explain in detail how to implement an N-gram cross-attention enhancement method and unified paradigm for complex nested named entity recognition.

[0112] 1. Sentence encoding:

[0113] Input: a sentence of indefinite length, such as "The Wall Street Journal";

[0114] Steps: Use a pre-trained model (such as BERT) to encode the sentence and map each word into a high-dimensional space to form a word embedding. The encoded sentence is represented as a high-dimensional matrix with a dimension of L×V, where L is the number of words in the sentence and V is the dimension of each word vector.

[0115] In this example, when bert-base-uncased is used as the pre-trained model, the output dimension is 5 × 768. That is, each word corresponds to a high-dimensional matrix vector.

[0116] 2. N-grams segmentation:

[0117] Steps: Use the N-grams method to split the encoded sentences into fixed-length sequences. For each word, combine it with the following n-1 words to form N sequence groups. For example, when n=3, the sequence groups after the "Wall Street Journal" segmentation are: (Wall, Er, Street), (Earth, Street, Day), (Street, Day, Newspaper), and so on.

[0118] 3. Linear mapping:

[0119] Steps: For each N-gram sequence group and the entire sentence, use a linear mapping (for example, a weight matrix) in each category space to map them to a new space. This mapping can help extract and emphasize features that are more important to the task. For example, if the new weight matrix dimension is V×d, then for the vector of the original sentence, its dimension becomes C×L×d, and the dimension of each N-gram sequence group after mapping is C×n×d, where C represents the number of categories.

[0120] 4. Scaling dot product similarity

[0121] Steps: Calculate the scaled dot product similarity between each N-gram sequence group and the original sentence. The scaled dot product similarity is calculated by dividing the dot product of two vectors by the product of their norms, which can avoid the similarity calculation bias caused by different vector lengths. For example, for one of the N-gram sequence groups ("Wall, Street") and the original sentence ("The Wall Street Journal"), calculate the dot product attention / dot product similarity in each category space. Finally, for each N-gram sequence group, a vector matrix of dimension C×L×n can be obtained. Each value represents the dot product similarity between each word in the original sentence and each N-gram sequence group in different category spaces.

[0122] 5. Max Pooling:

[0123] Steps: Since the scaled dot product similarity is calculated for the original sentence and the N-gram sequence group, for a sentence of length L, L-n+1 dot product similarity matrices will be generated. The goal of the present invention is an L×L matrix, not L-n+1 L×n matrices. Therefore, for each dot product similarity matrix, the most important information is extracted from multiple similarity scores through the maximum pooling operation. Compare the similarity scores of corresponding positions, and select the one with the largest similarity score to retain and splice.

[0124] For example, for the matrix "Wall Street Journal" with an original sentence length of 5, the dot product similarity is calculated with each sequence group of n=3 in each dimension, resulting in 3 matrices such as ("Wall Street Journal", "Wall Street"), ("Wall Street Journal", "Street Journal"), then we use the maximum pooling and splicing algorithm to merge them into ("Wall Street Journal", ("Hua", maximum pooling ("Er, Er"), maximum pooling ("Jie, Jie"), maximum pooling ("Ri, Ri"), "Journal")), and finally obtain ("Wall Street Journal", ("Hua", "Er", "Jie", "Ri", "Journal") with a dimension of L×L.

[0125] 6. Talking module:

[0126] Steps: After obtaining the final rating matrix, use a multi-convolutional neural network (Multi-CNN) to enhance the model's ability to recognize interactions between entity labels. Multi-convolutional neural networks do not change the dimension of the original matrix, so the final output rating matrix dimension is C×L×L, where C represents the number of categories and L represents the sentence length.

[0127] 7. Loss function and back propagation:

[0128] Steps: After obtaining the final output matrix, backpropagation can be performed through the loss function in the deep learning framework to optimize the model parameters. The loss function takes into account the positive and negative samples of the entity type, as well as the score of the entity type. Then, the corresponding label has the same dimension as the output score matrix, C×L×L, where each value in the matrix is ​​0 or 1.

[0129] For example, in the example "Wall Street Journal", for the two categories "street" and "organization", the dimensions of the final scoring matrix and label are 2×5×5. For example, when the label "street" is index 0 and the label "organization" is index 1, in this example, the labels [0,0,2] and [1,0,4] are 1, and the rest are 0. Then, through the loss of this label and scoring function, the derivative matrix can be optimized through back propagation, thereby further optimizing the model.

[0130] 8. Implementation effect: Through the above implementation steps, the present invention can improve the recognition effect of nested and multi-label entities when processing complex and nested entity relationships. Experimental results on public biomedical nested datasets and flat datasets show that the model has comparable performance compared with other powerful models, proving its potential in nested entity recognition. The above examples of the present invention take sentence processing as an example, which can actually be extended to the processing of any continuous text.

[0131] The above shows and describes the basic principles, main features and advantages of the present invention. Those skilled in the art should understand that the above embodiments do not limit the present invention in any form, and any technical solution obtained by equivalent replacement or equivalent transformation falls within the protection scope of the present invention.

Claims

1. An N-gram cross-attention enhancement method for complex nested named entity recognition, characterized in that: Input the nested named entity to be recognized into the pre-trained recognition model to obtain the recognition result; The recognition model processes the nested named entity to be recognized, including the following steps: (1) Encode the original input sentence and map each word in the original sentence into a high-dimensional space to form a word embedding; (2) Use the N-grams method to split the encoded sentences into sequences of fixed length to obtain multiple N-gram sequence groups; (3) Using the set linear mapping in each category space, each N-gram sequence group and the original sentence are mapped to a new space respectively, to obtain the original sentence matrix and multiple N-gram sequence group matrices; (4) Calculate the scaled dot product similarity between each N-gram sequence group matrix and the original sentence matrix; (5) Based on the calculated scaled dot product similarities of all N-gram sequence group matrices, a maximum pooling operation is performed to obtain a similarity matrix of the N-gram sequence group matrices; (6) The similarity matrix of the obtained N-gram sequence group matrix is ​​processed through the Talking module and the logits score is output.

2. According to claim 1, the N-gram cross-attention enhancement method for complex nested named entity recognition is characterized by: In the step (1), the original sentence is encoded using a pre-trained BERT model.

3. The N-gram cross-attention enhancement method for complex nested named entity recognition according to claim 1, characterized in that: In the step (2), the encoded sentence is divided into L-n+1 sequences using the N-grams method, each sequence has n words, n is a predefined positive value ranging from 1 to L, and L represents the length of the sentence.

4. The N-gram cross-attention enhancement method for complex nested named entity recognition according to claim 1, characterized in that: In the step (3), the linear mapping is set to be a set weight matrix, and the mapping is performed using the following formula: Q i =N i ·W Q K=H·W K +bias Among them, W Q ,W K They represent weight matrices, and bias represents bias values; N i represents an N-gram sequence group, Q i Represents the weight matrix W Q N-gram sequence group matrix after linear mapping; H represents the original sentence after encoding, and K represents the weight matrix W K The original sentence matrix after linear mapping.

5. The N-gram cross-attention enhancement method for complex nested named entity recognition according to claim 4, characterized in that: In step (4), the scaled dot product similarity between each N-gram sequence group matrix and the original sentence matrix is ​​calculated according to the following formula: in, It represents the dot product similarity between each N-gram sequence group matrix and each word in the original sentence matrix in each category space. Represents the weight matrix W K The transpose of the original sentence matrix after linear mapping, d k Represents the weight matrix W K The dimension size of the original sentence matrix after linear mapping, c i Indicates that in the c i The cross attention mechanism is calculated in the category space, and cross represents cross attention.

6. The N-gram cross-attention enhancement method for complex nested named entity recognition according to claim 1, characterized in that: In step (5), the maximum pooling operation is performed in the following steps: Loop through each N-gram sequence group matrix corresponding to the high-dimensional matrix H of the encoded original sentence; Calculate the dot product similarity between the currently extracted N-gram sequence group matrix and the original sentence matrix and put them into a list; When the length of the list is equal to the preset value, the pooling algorithm is called to process the current data in the list to obtain the similarity matrix R of the current N-gram sequence group matrix, and the similarity matrix R of the current N-gram sequence group matrix is ​​put into the list to update the list; After completing the pooling operation of all N-gram sequence group matrices, all similarity matrices R are obtained and output as the maximum pooling result, i.e., the similarity matrix of the N-gram sequence group; Among them, the steps of the pooling algorithm include: Get the current data in the list, including multiple N-gram sequence group matrices; Concatenate the repeated positions in the obtained N-gram sequence group matrix and perform maximum pooling to eliminate the representation of repeated characters; Reconnect the matrices to obtain the similarity matrix R and output it.

7. The N-gram cross-attention enhancement method for complex nested named entity recognition according to claim 1, characterized in that: In the step (6), the Talking module includes a plurality of convolutional neural network units, an activation function module and an addition unit, which are expressed as follows: R = Conv2d(R) R ′ =GeLU(Conv2d(R)) Logits=R+R ′ Among them, Conv2d represents a two-dimensional convolution operation, GelU represents an activation function, and R ′ represents the output of the activation function unit, and logits represents the output of the addition unit.

8. The N-gram cross-attention enhancement method for complex nested named entity recognition according to claim 1, characterized in that: The loss function of the recognition model is designed as follows: Among them, i, j are the starting and ending indexes of the span, Pos α Represents the positive sample of entity type α, Neg α Represents the negative sample of entity type α, L α (i,j) represents the Logits score of entity type α.

9. An N-gram cross-attention enhancement device for complex nested named entity recognition, characterized in that: Use the pre-trained recognition model to process the nested named entities to be recognized and obtain the recognition results; The recognition model includes: The encoding module is used to encode the input original sentence and map each word in the original sentence into a high-dimensional space to form a word embedding; The segmentation module uses the N-grams method to segment the encoded sentences into sequences of fixed length and obtain multiple N-gram sequence groups; A mapping module is used to use a set linear mapping in each category space to map each N-gram sequence group and the original sentence to a new space, respectively, to obtain an original sentence matrix and multiple N-gram sequence group matrices of the same dimension; A similarity calculation module is used to calculate the scaled dot product similarity between each N-gram sequence group matrix and the original sentence matrix; The pooling module performs a maximum pooling operation based on the calculated scaled dot product similarities of all N-gram sequence group matrices to obtain a similarity matrix of the N-gram sequence group matrices; The output module processes the similarity matrix of the obtained N-gram sequence group matrix through the Talking module and outputs the logits score.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the program is executed by a processor, the method described in any one of claims 1 to 8 is implemented.

Images