Entity semantic enhancement method based on multi-scale frame fusion
Through multi-scale bounding box fusion and position-weighted multi-head self-attention mechanism, the semantic features of bounding boxes in named entity recognition are enhanced, the problems of nested named entity recognition and weakening of long entity semantics are solved, and a more efficient named entity recognition effect is achieved.
Patent Information
- Application Number
- CN202510761708.7
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2025-06-09
- Publication Date
- 2025-09-19
AI Technical Summary
Existing named entity recognition technologies have difficulty in effectively identifying nested named entities, and when faced with long entities, the semantics are weakened and the boundaries are inaccurate.
An entity semantic enhancement method based on multi-scale bounding box fusion is adopted. The semantic features of the bounding box are enhanced through the multi-scale bounding box fusion mechanism and the multi-head self-attention mechanism based on position weights, and boundary regression and entity classification are performed through the entity classification module.
The model's ability to recognize named entities of different lengths, especially long ones, is improved, the connection between boundary position information and entity classification is enhanced, and the performance of named entity recognition is improved.
Smart Images
Figure CN120671675A_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the technical field of entity semantic enhancement, and in particular to an entity semantic enhancement method based on multi-scale bounding box fusion. Background Art
[0002] Named Entity Recognition (NER) involves identifying specific entity information, such as names of people, organizations, and locations, from a given text. As a core task in Natural Language Processing (NLP), NER is widely used to understand entities and relationships within text. It plays a crucial role in many downstream tasks, including relation extraction, information retrieval, and entity linking.
[0003] Currently, deep learning-based named entity recognition methods can be divided into sequence-based, hypergraph-based, and span-based methods. Sequence-based labeling methods mainly label each element in the sentence to indicate its role in forming the named entity. For example, in the "BIO" encoding, "B", "I", or "O" respectively represent the "beginning", "inside", or "outside" of the named entity. However, this method can only assign one label to each word, can only solve non-nested structures, and has difficulty in identifying nested named entities.
[0004] To address the difficulty of nested named entity recognition, some researchers have designed proprietary recognition structures using hypergraph-based approaches. For example, the method proposed by Huang et al. allows an edge to connect to multiple nodes to represent different entities, uses a neural network to encode these edges and nodes, and finally recovers the entities from the hypergraph labels. Although hypergraph-based methods can resolve nested structures, they inevitably require complex transformations and expensive decoding processes, which can introduce additional errors.
[0005] Span-based methods are a common approach for nested named entity recognition. They typically generate entity spans from a sentence through boundary concatenation and output a class label for each possible span. Entities are then identified by enumerating all candidate spans in the sentence and classifying them. This approach has the advantages of addressing nested structures and fully leveraging the character characteristics of spans, but it increases computational complexity and lacks supervision of entity boundaries, which can lead to inaccurate boundaries. Furthermore, span-based methods fail to effectively utilize features between spans when dealing with longer entities, resulting in weakened semantics.
[0006] Recently, some studies have noted the formal consistency between object detection and NER tasks, such as boundary regression models. By regressing the locations of candidate entity boundaries, supervised information can be better utilized and the entity nesting problem can be effectively addressed. However, boundary regression models require pre-setting dense text bounding boxes of a fixed length and number for locating entities and performing regression operations. This single-scale feature map with bounding boxes is difficult to adapt to ambiguous or complex contexts and cannot fully capture the representation of entities in different contexts, thus limiting the accurate recognition of entities of different scales during entity recognition. Summary of the Invention
[0007] The purpose of the present invention is to provide an entity semantic enhancement method based on multi-scale bounding box fusion, which can solve the problem of entity semantics being dispersed in spans and semantic ambiguity, capture the global information of two-dimensional sentences, and realize the recognition of long named entities.
[0008] To achieve the above object, the present invention provides an entity semantic enhancement method based on multi-scale bounding box fusion, comprising the following steps:
[0009] S1. The sentence is abstractly represented by the basic network, which consists of the abstract representation of all characters, and a bounding box is generated for each character, including the bounding box features and the bounding box position; the bounding box position includes the start position and the end position;
[0010] S2. Use convolution kernels of different sizes to generate border features of different scales, and introduce a multi-scale border fusion mechanism. By calculating the weight vector of each border and fusing the multi-scale border features, the initial border position of each character is obtained;
[0011] S3. Through the multi-head self-attention mechanism based on position weights, the starting and ending positions of the initial bounding box are adjusted, and a position weight is generated for the bounding box of each attention head using a quasi-Gaussian distribution to obtain the multi-head attention result of the fused bounding box position;
[0012] S4. Based on the multi-head attention results of the fused border position, the border features are updated through the entity classification module, and the total loss function is constructed by combining the negative log-likelihood of class prediction and boundary loss to enhance the entity semantics.
[0013] Preferably, in S2, the multi-scale bounding box fusion mechanism includes obtaining score vectors of corresponding features based on bounding box features of different scales, and calculating a weight vector for each bounding box as follows:
[0014]
[0015] Where, For character t i Feature representation at the kth layer; Representation characteristics The score vector of The vector representations of the start and end positions of the k-th layer border respectively; For character t i The weight vector of the jth span at the kth layer; Indicates the position of the multi-scale bounding box, which is used to indicate the character t i The position information of all spans j of the feature;
[0016] Then, use the weight vector of each border to initialize the corresponding border position vector as follows:
[0017]
[0018] where · represents the element-wise product, Indicates the corresponding features The initial border position, n j A position vector representing a multi-scale bounding box.
[0019] Preferably, in S2, the multi-scale bounding box features are fused, and the expression is as follows:
[0020]
[0021] Where, For character t i The j-th span of the k-th layer represents h' j is the fused multi-scale bounding box feature.
[0022] Preferably, S3 includes using the fused multi-scale bounding box features to calculate the offset of the bounding box boundary relative to the real entity position, and then updating the position of the multi-scale bounding box as follows:
[0023]
[0024] d j =n j +Δn j ;
[0025] Where, d j is the position vector of the updated multi-scale bounding box, Δn j is the offset vector of the multi-scale bounding box relative to the real entity position, which is determined by the offset of the starting boundary Offset from the span's end boundary express.
[0026] Preferably, S3 includes generating a position weight for each attention head bounding box using a Gaussian-like distribution as follows:
[0027]
[0028] in,
[0029]
[0030] η j =MLP(h′ i );
[0031] Where w j,m Represents the position weight of the multi-scale bounding box; Σ j is a positive definite matrix, and h′ i Constructed; n j,m d j,m ,Σ j,m Respectively represent n j d j ,Σ j Linear transformation.
[0032] Preferably, in step S3, the multi-head self-attention mechanism based on position weight is expressed as follows:
[0033] a=[head1;head2;...;head m ]W a ;
[0034]
[0035] Where a is the multi-head self-attention result; and Represent the query vector, key vector and value vector initialized in the attention mechanism, respectively, and are represented by h′ j Perform linear transformation to obtain; [head1; head2; ...; head m ] represents the calculation result of each attention head, m represents the number of attention heads; d k is a hyperparameter; W a It is the weight matrix after the calculation results of the attention heads are concatenated.
[0036] Preferably, S4 includes obtaining the best pairing of the real entity set and the predicted entity set using a bipartite matching algorithm.
[0037] Preferably, in S4, the expression of the total loss function is:
[0038]
[0039] Where, is the total loss function, y is the real entity set, To predict the entity set, Indicates that the prediction of index σ(i) is category ci The probability of L boundry (.,.) is the boundary loss, is the L1 loss, g i is the position of the real entity, d σ(i) The position of the predicted bounding box, N is the number of prediction results.
[0040] Therefore, the present invention adopts the above-mentioned entity semantic enhancement method based on multi-scale border fusion, which has the following technical effects:
[0041] (1) The present invention represents the span as a bounding box with boundary position information and integrates multi-scale entity features within the bounding box to enhance the semantic features in the bounding box, thereby improving the model's recognition of entities of different lengths, especially long entities.
[0042] (2) This paper introduces a multi-head attention mechanism based on position weights to strengthen the connection between boundary position information and entity classification, effectively improving performance.
[0043] The technical solution of the present invention is further described in detail below through the accompanying drawings and embodiments. BRIEF DESCRIPTION OF THE DRAWINGS
[0044] Figure 1 2 is a schematic diagram of the structure of the model network in an embodiment of the entity semantic enhancement method based on multi-scale bounding box fusion. DETAILED DESCRIPTION
[0045] The present invention can be explained in more detail by the following examples. The purpose of disclosing the present invention is to protect all changes and improvements within the scope of the present invention. The present invention is not limited to the following examples.
[0046] Example 1
[0047] like Figure 1 As shown, the present invention provides an entity semantic enhancement method based on multi-scale border fusion. The model architecture adopted by this method is mainly divided into five parts: basic network, multi-scale border fusion, boundary regression, attention mechanism and category judgment. In the basic network part, the word or character tag is converted into a context feature map to capture the semantic and syntactic information of the sentence. In the multi-scale border fusion part, a border is obtained to facilitate the subsequent classification process. By performing multi-scale feature fusion on the features of the text border, a border with richer semantic information is obtained. The boundary regression part mainly updates the position of the border by capturing external semantics. Then, based on the multi-head attention mechanism of the position weight, the adjusted position information can be used as supervisory information to assist subsequent classification. The fifth part is the category judgment layer. The following is a detailed analysis of each part of the model.
[0048] (1) Basic network:
[0049] The basic network is the core part of the abstract representation of sentences, which converts word or character tokens into context feature maps to capture the semantic and syntactic information of sentences. Let T = [t1, t2, t3, ..., t N ], where t i (1≤i≤N) represents the token of a word or character, and N is the length of the input sentence T. To avoid the difficulty of finding entity boundaries when they are far away from any real entity boundaries, we use multi-granularity sentence representation in the basic network, t i The abstract representation h is obtained by the following method i :
[0050] h i =[h i char ;h i bert ;h i word ;h i pos ];
[0051] Among them, the character-level embedding h i char Each character t i Embedded into a vector, and then converted to a high-dimensional vector through Bi-LSTM; h i bert The BERT-Large network converts the word into a vector, effectively encoding the semantic information of the word; word-level embedding h i word It is t i After pre-training word vector mapping such as Glove; to introduce the semantic information of part of speech, each t i Use the part-of-speech tagging tool to generate POS tags and map them into vectors to get h i pos ;[;] means to splice several representations; splicing to get the mark t i The abstract representation h i The abstract representation of the entire sentence consists of all character abstract representations, expressed as H = [h1,h2,h3,…,h N ].
[0052] 2. Multi-scale bounding box fusion:
[0053] In this stage, a bounding box is generated for each character, including the bounding box features and the bounding box position. The bounding box position vector consists of the start position and the end position. The multi-scale bounding box features are generated directly from the first layer by convolutional layers with kernel sizes of 1, 2, and 3. The first layer uses the output H of the basic network. The k-th feature layer is expressed as:
[0054]
[0055] Through the above convolution process, a multi-scale feature map layer is generated. This multi-scale sentence representation method helps to better capture the semantic information of different scales in the sentence, thereby enhancing the performance of named entity recognition. To further improve the rationality of the named entity recognition task in terms of representation, a multi-scale bounding box fusion mechanism is introduced as follows:
[0056]
[0057] In the formula, through Calculate the score vector for each feature and The vector representations of the start and end positions are used to calculate the weight vector of each bounding box; Represents the character t i The weight vector of the jth span in the kth feature map layer. Contains character t i The location of the multi-scale bounding box of the feature is expressed as Where i is t i The location information of k is the number of t i For example, the position corresponding to the first character t1 feature is The vectors of these three positions are fused to generate a new fused border. In order to obtain richer semantic information, this embodiment also fuses the weight vector of the border and the border features to generate a global weight representation. The specific process is as follows:
[0058] First, fuse multi-scale bounding box features:
[0059]
[0060] Where, is the bounding box feature of the jth span of the kth feature layer, h' j This is the fused multi-scale bounding box feature. This fusion method comprehensively considers the semantic information of different scales and enhances the semantic information of the bounding box.
[0061] Then, using the weight vector The border position of the j-th span Initialize it as follows:
[0062]
[0063] Among them, n j is the position vector of the j-th span border, · represents the element-wise product; Representative corresponding to the feature This method can obtain the feature The initial bounding box position vector of the character t i The leftmost position vector and the rightmost position vector, this restriction is helpful for model understanding and reasonable adjustment of subsequent boundary regression methods.
[0064] 3. Multi-head self-attention mechanism based on position weight:
[0065] (1) Boundary regression:
[0066] Although many bounding boxes are not recognized as negative samples by the model, they overlap and share the same context, leading to boundary mismatches in the entity recognition task. To address this, this embodiment designs a boundary regression module that captures external semantics to update the position of bounding boxes, effectively improving span accuracy and improving entity localization tasks.
[0067] In the specific implementation process, the starting and ending positions of the border are adjusted, and MLP is applied to learn their position offset relative to the nearest true border, and the features h' of the starting and ending positions of the border are converted into j Enter the boundary regression module to calculate the offset of the bounding box relative to the real entity position. The update formula of the bounding box position is as follows:
[0068]
[0069] d j =n j +Δn j ;
[0070] Where, d j is the jth span border position vector after update, Δn j is the offset of the border boundary relative to the real entity position, which is the offset of the starting boundary Δn j Offset from the span's end boundary Indicates that W∈R 3D*2 , b∈R 2 are all learnable parameters.
[0071] (2) Attention mechanism:
[0072] After boundary regression adjustment, the adjusted position information can be incorporated into the border features through a multi-head attention mechanism based on position weights. Using the original border features to obtain the class logarithm will lead to model error propagation. To this end, this embodiment uses a Gaussian-like distribution to generate a position weight for each border of each attention head. The existing Gaussian-like distribution G(x|μ,Σ) is formulated as follows:
[0073]
[0074] Where x∈R 2 belongs to the independent variable, μ belongs to the expected vector, that is, the center of the distribution. -1 ∈R 2×2 represents the inverse of the covariance matrix, which is a positive definite matrix that meets the precision matrix requirements of the Gaussian distribution.
[0075] During the application process, a series of parameters need to be generated to complete the construction of the Gaussian distribution. The method is as follows:
[0076]
[0077] η j =MLP(h′ i );
[0078] Reversible matrix Σ j Can pass h′ i To construct; h′ j 、n j d j and Σ j Perform linear transformation to obtain h′ j,m 、n j,m d j,m and Σ j,m , and then use n j,m and d j,m The difference between the multi-scale border position vector and the expected value is expressed as w j,m , the calculation formula is as follows:
[0079]
[0080] The calculation formula of the position-aware multi-head self-attention mechanism is as follows:
[0081] a=[head1;head2;...;head m ]W a ;
[0082]
[0083] in, and Represent the query vector, key vector and value vector initialized in the attention mechanism, respectively, and are represented by h′ j Perform linear transformation to obtain; head m represents the calculation result of the i-th attention head, m represents the number of attention heads; d k is a hyperparameter; W aIt is the weight matrix after the calculation results of the attention heads are spliced together; the multi-head self-attention result a is sent to the next layer for the final classification prediction.
[0084] 4. Entity Classification Module:
[0085] The flexibility of the border is increased through the position-based multi-head self-attention mechanism. On this basis, the multi-scale border feature h′ is fused with the multi-head self-attention results of the border position. j Update so that the adjusted position information can well supervise the subsequent classification task, and then input the classifier to calculate the probability that the span is a named entity The calculation formula is as follows:
[0086]
[0087] Where, is the updated multi-scale bounding box feature, is the output of the attention mechanism, are all learnable parameters, a j ∈A represents the result of the attention mechanism in the previous stage, d1 is the number of attention heads and head m The product of dimensions, d2 represents the vector dimension of the bounding box feature.
[0088] The output of the model is d j and is a set of N predictions containing entity boundary locations and entity classification probabilities, where N is set to a value significantly larger than the number of named entities in the text. In this embodiment, N is set to the length of the sentence. Let y represent the true entity set of the object, which is considered as (No object) Fill to a set of size N; the set of N predictions is represented by To find a bipartite matching between these two sets, search N elements σ∈Ψ with minimum cost N The arrangement is as follows:
[0089]
[0090] in, represents the total matching cost, is the true value y i The pairwise matching cost between the prediction indexed by σ(i) and the prediction indexed by σ(i) can be calculated effectively using the Hungarian algorithm in this embodiment to calculate the optimal allocation, which is used to comprehensively consider the matching cost of the category prediction and its boundary prediction. Each element of the real entity set can be regarded as y i =(c i ,g i ), c i Indicates the entity tag (which may be ), gi ∈[0,1] 2 Is the starting position of the real entity and end position For the prediction indexed by σ(i), the category c i The probability is defined as The predicted bounding box is defined as d σ(i) .match is defined as follows:
[0091]
[0092] Boundary loss L boundry (.,.) is defined as follows:
[0093]
[0094] in, is a robust L1 loss that quantifies g i and d σ(i) The difference between.
[0095] Total loss function The calculation of is a linear combination of the negative log-likelihood of the class prediction and the margin loss, as follows:
[0096]
[0097] Example 2
[0098] In order to verify the effectiveness of the entity semantic enhancement method based on multi-scale bounding box fusion provided by the present invention, a set of comparative examples is also set up: on the ACE2004, ACE2005, and Weibo datasets, the existing models Layered model, W2Ner, BS, Full-span, Locate-and-Label, and DiffusionNER are used for comparative analysis. The results are shown in Table 1.
[0099] Table 1 Comparison of experimental results on datasets
[0100]
[0101] As shown in Table 1, the F1 values of our method on the ACE2004 and ACE2005 datasets reached 88.63% and 88.53%, respectively, achieving the best results compared to previous work. This demonstrates that our method is capable of effectively identifying nested named entities. Furthermore, a comparative analysis on the Weibo dataset shows that our method is not limited to English datasets but also performs well on Chinese datasets, demonstrating excellent cross-lingual capabilities.
[0102] In the existing boundary regression model, good results are achieved by refining the spatial position of the entity span in the sentence through regression operations, and all possible named entities in the sentence are found. However, the model adjusts the entity position based on a single scale feature and cannot fully capture the performance of the entity in different contexts. The span-based named entity recognition model obtains the span by splicing the representation of the entity boundary. When faced with long entities, it is easy to cause the semantic association between the two entity boundaries to be weakened. In contrast, the model adopted in this embodiment solves the above problem well through the method of multi-scale border fusion, comprehensively considers the semantic information of different scales, and enhances the semantic information of the border; at the same time, the multi-head self-attention mechanism based on position weights can effectively combine the boundary position information to improve the accuracy of the model in identifying entities, especially in the ability to recognize long entities.
[0103] Therefore, the present invention adopts the above-mentioned entity semantic enhancement method based on multi-scale border fusion. By fusing the feature representations of borders at different scales, it can reasonably generate the left and right boundaries of possible candidate entities, so as to improve the accuracy of the border position representation, help capture features of different scales, and effectively support subsequent boundary regression operations; at the same time, the introduction of position weights enables the model to pay more attention to the boundary detail information of the entity during classification, thereby improving the accuracy of entity classification.
[0104] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention rather than to limit the same. Although the present invention has been described in detail with reference to the preferred embodiments, those skilled in the art should understand that they can still modify or replace the technical solutions of the present invention with equivalents, and these modifications or equivalent replacements cannot cause the modified technical solutions to deviate from the spirit and scope of the technical solutions of the present invention.
Claims
1. Entity semantic enhancement method based on multi-scale bounding box fusion, characterized by: The following steps are involved: S1. The sentence is abstractly represented by the basic network, which consists of the abstract representation of all characters, and a bounding box is generated for each character, including the bounding box features and the bounding box position; the bounding box position includes the start position and the end position; S2. Use convolution kernels of different sizes to generate border features of different scales, and introduce a multi-scale border fusion mechanism. By calculating the weight vector of each border and fusing the multi-scale border features, the initial border position of each character is obtained; S3. Through the multi-head self-attention mechanism based on position weights, the starting and ending positions of the initial bounding box are adjusted, and a position weight is generated for the bounding box of each attention head using a quasi-Gaussian distribution to obtain the multi-head attention result of the fused bounding box position; S4. Based on the multi-head attention results of the fused border position, the border features are updated through the entity classification module, and the total loss function is constructed by combining the negative log-likelihood of class prediction and boundary loss to enhance the entity semantics.
2. The entity semantic enhancement method based on multi-scale bounding box fusion according to claim 1, characterized in that: In S2, the multi-scale bounding box fusion mechanism includes obtaining the score vector of the corresponding features based on the bounding box features of different scales, and calculating the weight vector of each bounding box as follows: Where, For character t i Feature representation at the kth layer; Representation characteristics The score vector of The vector representations of the start and end positions of the k-th layer border respectively; For character t i The weight vector of the jth span at the kth layer; Indicates the position of the multi-scale bounding box, which is used to indicate the character t i The position information of all spans j of the feature; Then, use the weight vector of each border to initialize the corresponding border position vector as follows: where · represents the element-wise product, Indicates the corresponding features The initial border position, n j A position vector representing a multi-scale bounding box.
3. The entity semantic enhancement method based on multi-scale bounding box fusion according to claim 1, characterized in that: In S2, the multi-scale bounding box features are fused and the expression is as follows: Where, For character t i The j-th span of the k-th layer represents h' j is the fused multi-scale bounding box feature.
4. The entity semantic enhancement method based on multi-scale bounding box fusion according to claim 1, characterized in that: S3 involves using the fused multi-scale bounding box features to calculate the offset of the bounding box boundary relative to the real entity position, and then updating the position of the multi-scale bounding box as follows: d j =n j +Δn j ; Where, d j is the position vector of the updated multi-scale bounding box, Δn j is the offset vector of the multi-scale bounding box relative to the real entity position, which is determined by the offset of the starting boundary Offset from the span's end boundary express.
5. The entity semantic enhancement method based on multi-scale bounding box fusion according to claim 4 is characterized in that: S3 involves using a Gaussian-like distribution to generate a position weight for each attention head bounding box as follows: in, n j =MLP(h′ i ); Where w j,m Represents the position weight of the multi-scale bounding box; Σ j is a positive definite matrix, and h i 'Constructed; n j,m d j,m ,Σ j,m Represents n j d j ,Σ j Linear transformation.
6. The entity semantic enhancement method based on multi-scale bounding box fusion according to claim 1, characterized in that: In S3, the multi-head self-attention mechanism based on position weight is expressed as follows: a=[head1;head2;...;head m ]W a ; Where a is the multi-head self-attention result; and Represent the query vector, key vector and value vector initialized in the attention mechanism, respectively, and are represented by h′ j Perform linear transformation to obtain; [head1; head2; ...; head m ] represents the calculation result of each attention head, m represents the number of attention heads; d k is a hyperparameter; W a It is the weight matrix after the calculation results of the attention heads are concatenated.
7. The entity semantic enhancement method based on multi-scale bounding box fusion according to claim 1, characterized in that: S4 includes using a bipartite matching algorithm to obtain the best pairing of the real entity set and the predicted entity set.
8. The entity semantic enhancement method based on multi-scale bounding box fusion according to claim 1, characterized in that: In S4, the expression of the total loss function is: Where, is the total loss function, y is the real entity set, To predict the entity set, Indicates that the prediction of index σ(i) is category c i The probability of L boundry (.,.) is the boundary loss, is the L1 loss, g i is the position of the real entity, d σ(i) The position of the predicted bounding box, N is the number of prediction results.