Named entity identification method and system based on long-distance information enhancement and boundary smoothing

Abstract

The invention relates to the technical field of named entity recognition, in particular to a named entity recognition method and system based on long-distance information enhancement and boundary smoothing. Encoding an input sentence X by using a pre-trained Chine BERT encoder, and generating context representation information hi; carrying out feature extraction on the corpus comprising the context representation information by using a bidirectional long short-term memory network and a SwinTransform with a mask; the extracted features are sent to two independent multi-layer perceptron (MLP) to represent starting and ending of the entities respectively, and the span score of each entity is calculated; performing convolution on all entity span scores by using a convolutional neural network, and converting spatial information among words into spatial features; assigning a small part of probability Epsilon to a span adjacent to the current label, assigning the remaining probability 1-Epsilon to the span of the current label, performing boundary smoothing operation on each label to obtain a smoothed label # imgabs0 #

Description

Technical Field

[0001] The present invention relates to the technical field of named entity recognition, and in particular to a method and system for named entity recognition based on long-distance information enhancement and boundary smoothing. Background Art

[0002] Named entity recognition (NER) is a core task in natural language processing, aiming to detect and classify entity spans in text. Entity spans are phrases or words in text that have specific semantics or categories. When entity spans in NER overlap, it is called nested NER. The challenge of nested NER lies in accurately identifying and distinguishing complex, overlapping entities in text. To address this problem, span-based methods are widely used. These methods represent entities in text by constructing a span matrix, where each entry corresponds to a span, and identify relationships between entities by analyzing the matrix.

[0003] Span-based methods construct an n×n matrix to represent entity spans in a text, where each entry corresponds to the boundary of an entity span. This allows for accurate identification of nested entities in a text and handling overlapping relationships between entities, avoiding the shortcomings of traditional sequence labeling methods in nested structures. Existing technologies use span-based methods to introduce convolutional neural networks (CNNs) to model spatial relationships in the score matrix, addressing the problem of traditional span-based methods ignoring spatial relationships. Modeling spatial relationships through convolutional neural networks effectively captures the relationships between adjacent spans in the matrix, enhancing the model's understanding of long-range dependencies and contextual information, thereby optimizing the ability to capture entity boundaries and avoiding the shortcomings of traditional methods in dealing with fuzzy boundaries.

[0004] The existing technology has the following technical problems: (1) Long-distance information is lost or underutilized. When dealing with long-distance dependencies, the model often cannot effectively capture the semantic connection between distant characters in the text, thereby affecting the recognition accuracy of entities; (2) Data labels are crucial in the named entity recognition task, but the current methods have the problem of being overconfident in boundary information. When the model predicts the entity boundary, it usually assumes that the boundary is absolute, thereby ignoring the uncertainty of the boundary. This will cause the model to make overconfident predictions when faced with blurred boundaries, reducing the overall performance of the model.

[0005] At the same time, Chinese named entity recognition presents additional challenges. This is due to differences between Chinese and other languages ​​in grammatical structure, named entity categories, and highly ambiguous named entities, particularly in the handling of nested entities. These characteristics of Chinese make it more difficult to extract long-range information and accurately predict boundaries. Summary of the Invention

[0006] In order to solve the above technical problems, the present invention provides a named entity recognition method and system based on long-distance information enhancement and boundary smoothing, which improves the existing technology, alleviates the model's dependence on absolute boundaries, makes boundary prediction smoother, and further improves the performance of nested named entity recognition tasks.

[0007] To achieve the above object, the technical solution adopted by the present invention is: first, a method for named entity recognition based on long-distance information enhancement and boundary smoothing is provided, comprising the following steps:

[0008] S1 preprocessing:

[0009] Use the pre-trained Chinese BERT encoder to input sentence X=[x1,x2,…,x n ] is encoded to generate context representation information h i ;

[0010] S2 feature extraction:

[0011] Use bidirectional long short-term memory network and masked SwinTransformer to extract features from corpus including contextual representation information;

[0012] S3 calculates span score:

[0013] The features extracted by S2 are fed into two independent multi-layer perceptrons (MLPs) to represent the start and end of the entity, respectively, and the span score of each entity is calculated;

[0014] S4 convolution operation:

[0015] Use convolutional neural networks to convolve all entity span scores and convert the spatial information between words into spatial features;

[0016] S5 Smooth Boundary:

[0017] Assign a small part of the probability ε to the span adjacent to the current label, and the remaining probability 1-ε to the span of the current label. Perform boundary smoothing on each label to obtain the smoothed label.

[0018] Preferably, the encoding process in S1 is: for the input sentence X=[x1, x2, ..., x n Each character x in ] n , the encoder first generates its corresponding character embedding, pronunciation embedding and glyph embedding, with embedding dimensions of D; then concatenates the character embedding, pronunciation embedding and glyph embedding, and maps them into a D-dimensional fusion embedding through a fully connected layer, adds position embedding information to the embedding, and sends it to the multi-layer Transformer layer to generate the character xn Context representation information h i ∈R D , the fusion representation information is H = {h1,h2,…,h i}.

[0019] Preferably, the feature extraction process in S2 is as follows:

[0020] S2.1 constructs a relationship matrix to represent the degree of association between characters and encodes the semantic similarity and structural dependency implicit features;

[0021] S2.2 divides the relationship matrix into blocks, and each block is used as a basic processing unit and input into SwinTransformer for feature modeling and attention calculation;

[0022] S2.3 performs downsampling through block merging, extracts high-dimensional features, generates hierarchical representation, gradually reduces the dimension of the feature matrix, and extracts the feature matrix.

[0023] Preferably, the entity span score calculation process is as follows:

[0024] The start and end scores of the entity are vectors h s and h e :

[0025] h s (i) = LeakyReLU(h i w s ),

[0026] h e (i) = LeakyReLU(h i w e ),

[0027] Among them, h i Represents the context representation information of the nth basic unit token; w s With w e is a learnable weight matrix; LeakyReLU is a nonlinear activation function, h s (i) represents the feature representation of the starting position of the i-th candidate entity, h e (i) Feature representation of the end position of the i-th candidate entity;

[0028] Get the score of each entity through the biaffine attention matrix to get the span score of each entity:

[0029] r(i)=h s (i) T U m h e (i)+W m (hs (i)⊕h e (i))+b m ,

[0030] Among them, r represents the score of all possible named entity spans when the starting position of the span is before the ending position, and b m is the bias term, U m is a learnable bilinear mapping matrix used to model the bilinear similarity between the starting vector and the ending vector; W m It is a linear transformation weight matrix used to map the concatenated vector to a scalar score, which is the scoring weight after feature concatenation.

[0031] Preferably, the convolution process is as follows:

[0032] R′=Conv2d(R),

[0033] R″=GeLU(LayerNorm(R ′ +R)),

[0034] Among them, R represents the span score matrix of all entities, R ′ Represents the result of a layer of convolution, and R″ represents the result after the activation function.

[0035] Preferably, the smoothed label It is expressed as follows:

[0036]

[0037] in, is the smoothed label, representing the target probability distribution of span (i, j), y ij is the label in the original dataset, ε is the smoothing factor, which takes the value of ε∈{0.1,0.2,0.3}, D is the smoothing range, which takes the value of D∈{1,2}, y i+d,j with y i,j+d Indicates the entity distribution after extending d basic unit tokens outward from the span boundary. If (i, j) is an entity, then y ij is 1, otherwise y ij is 0.

[0038] Preferably, the smoothing range D is set to 2, and the smoothing factor ε is set to 0.2.

[0039] Preferably, step S5 further includes:

[0040] Use the smoothed labels to calculate the binary cross entropy loss to optimize the model:

[0041]

[0042] Among them, y′ ij Represents the probability of being predicted as an entity.

[0043] The present invention also proposes a named entity recognition system based on long-distance information enhancement and boundary smoothing, comprising:

[0044] Preprocessing module:

[0045] Used to use the pre-trained Chinese BERT encoder to input sentence X = [x1, x2, ..., x n ] is encoded to generate context representation information h i ;

[0046] Feature extraction module:

[0047] Used to extract features from corpus including contextual representation information using a bidirectional long short-term memory network and a masked SwinTransformer;

[0048] Span score calculation module:

[0049] It is used to feed the features extracted by S2 into two independent multi-layer perceptrons (MLPs), which represent the start and end of the entity respectively, and calculate the span score of each entity;

[0050] Convolution operation module:

[0051] It is used to convolve all entity span scores using convolutional neural networks to convert the spatial information between words into spatial features;

[0052] Smooth Boundary Module:

[0053] It is used to assign a small part of the probability ε to the span adjacent to the current label, and the remaining probability 1-ε is assigned to the span of the current label. The boundary smoothing operation is performed on each label to obtain the smoothed label.

[0054] The above technical solution has the following advantages or beneficial effects:

[0055] The span-based method of the present invention has significant advantages in the nested named entity recognition task. Compared with the traditional sequence labeling method, the span-based method can more effectively capture the nested relationship between entities, thereby improving the accuracy and precision of the entity. Especially when dealing with multi-level entity structures, the span-based method can avoid the entity boundary fuzzy problem existing in the sequence method, and show a higher recognition ability. Secondly, SwinTransformer can effectively enhance the ability to extract long-distance features, thereby better capturing the long-distance dependency information between entities. Finally, by using the boundary smoothing method to assign certain weights to the adjacent spans of the entity, the model avoids overconfidence in boundary prediction, alleviates the model's dependence on absolute boundaries, makes boundary predictions smoother, and further improves the performance of the nested named entity recognition task.

[0056] The above summary is for illustrative purposes only and is not intended to be limiting in any way. In addition to the illustrative aspects, embodiments and features described above, further aspects, embodiments and features of the present invention will be readily apparent by reference to the accompanying drawings and the following detailed description. BRIEF DESCRIPTION OF THE DRAWINGS

[0057] In the accompanying drawings, unless otherwise specified, the same reference numerals throughout the multiple drawings represent the same or similar components or elements. These drawings are not necessarily drawn to scale. It should be understood that these drawings only depict some embodiments disclosed herein and should not be construed as limiting the scope of the invention.

[0058] Figure 1 A flowchart of a named entity recognition method based on long-distance information enhancement and boundary smoothing provided by the present invention;

[0059] Figure 2 A feature extraction module architecture diagram of an embodiment of a named entity recognition method based on long-distance information enhancement and boundary smoothing provided by the present invention;

[0060] Figure 3 An example diagram of window attention in an embodiment of a method for named entity recognition based on long-distance information enhancement and boundary smoothing provided by the present invention;

[0061] Figure 4 An example diagram of a sliding window attention mechanism in an embodiment of a method for named entity recognition based on long-distance information enhancement and boundary smoothing provided by the present invention;

[0062] Figure 5 An example diagram of a sliding window mask in an embodiment of a method for named entity recognition based on long-distance information enhancement and boundary smoothing provided by the present invention;

[0063] Figure 6 An example diagram of a position mask strategy in an embodiment of a method for named entity recognition based on long-distance information enhancement and boundary smoothing provided by the present invention;

[0064] Figure 7 An example diagram of experimental results of an embodiment of a named entity recognition method based on long-distance information enhancement and boundary smoothing provided by the present invention;

[0065] Figure 8 An example diagram of performance impact analysis of the boundary smoothing and feature extraction modules in an embodiment of a method for named entity recognition based on long-distance information enhancement and boundary smoothing provided by the present invention;

[0066] Figure 9 This is an example diagram of analyzing the impact of window size on performance in an embodiment of a named entity recognition method based on long-distance information enhancement and boundary smoothing provided by the present invention;

[0067] Figure 10 An example diagram showing the impact of smoothing parameters on performance in an embodiment of a named entity recognition method based on long-distance information enhancement and boundary smoothing provided by the present invention. DETAILED DESCRIPTION

[0068] Hereinafter, only certain exemplary embodiments are briefly described. As will be appreciated by those skilled in the art, the described embodiments may be modified in various ways without departing from the spirit or scope of the present invention. Therefore, the drawings and description are to be considered as illustrative in nature and not restrictive.

[0069] The present invention first provides a method for named entity recognition based on long-distance information enhancement and boundary smoothing. The process is as follows: Figure 1 As shown, the following steps are included:

[0070] S1 preprocessing, using the pre-trained Chinese BERT encoder to process the input sentence X = [x1, x2, ..., x n ] is encoded to generate context representation information h i ;

[0071] The encoding process is as follows: For the input sentence X=[x1, x2,…, x n Each character x in ] n , the encoder first generates its corresponding character embedding, pronunciation embedding and glyph embedding, with embedding dimensions of D; then concatenates the character embedding, pronunciation embedding and glyph embedding, and maps them into a D-dimensional fusion embedding through a fully connected layer, adds position embedding information to the embedding, and sends it to the multi-layer Transformer layer to generate the character x n Context representation information hi ∈R D , the fusion representation information is H = {h1,h2,…,h i}.

[0072] S2 feature extraction uses a bidirectional long short-term memory network and a masked SwinTransformer to extract features from the corpus including context representation information. The process is as follows Figure 2 As shown;

[0073] The feature extraction process is as follows:

[0074] To fully leverage the local and global information between characters, S2.1 compresses word embeddings and constructs a relationship matrix to represent the degree of association between characters. This matrix can encode implicit features such as semantic similarity and structural dependencies. This matrix provides a wealth of information for SwinTransformer feature extraction. Furthermore, the construction of the relationship matrix can incorporate prior knowledge to highlight specific entity boundary information using different masking methods, thereby enhancing the model's adaptability to nested structures.

[0075] S2.2 In order to adapt to the processing method of SwinTransformer, the input relationship matrix is ​​partitioned (Patch Partition) to divide the overall relationship matrix into several non-overlapping blocks of fixed size. Each block serves as a basic processing unit and contains relationship information within a local range. Compared with the method of directly processing the overall matrix, block partitioning can reduce the computational complexity while retaining the integrity of local information, providing a reasonable processing method for the calculation of subsequent window attention mechanisms. SwinTransformer extracts features at different levels to form a hierarchical feature representation. After the block partitioning is completed, it is input into SwinTransformer for feature modeling. Figure 3 As shown in the figure, the SwinTransformer module is mainly composed of window multi-head self-attention (W-MSA) and sliding window multi-head self-attention (SW-MSA). Compared with global self-attention, it reduces a lot of calculations. The two work together to fully model the local area and efficiently interact with information across windows.

[0076] Windowed Multi-Head Self-Attention (W-MSA) calculates self-attention within a fixed window, allowing the model to focus more on key information in the local window. At the same time, because Windowed Multi-Head Self-Attention (W-MSA) is only calculated within the local window, the computational complexity is reduced, which can improve computational efficiency.

[0077] Although Window Multi-Head Self-Attention (W-MSA) reduces the computational complexity, its fixed window division limits the information interaction between different windows. To solve this problem, the sliding window attention (SW-MSA) mechanism is introduced, such as Figure 4 As shown in Figure 2, SW-MSA shifts the attention window across the input feature map, allowing information between different windows to be transferred in a hierarchical manner. This allows the unattended portion to be computed after several shifts, improving the model's ability to perceive nested named entities with a certain spatial structure.

[0078] The method to implement the attention window is as follows: Figure 4 Take the window where "block 4" is located as an example: First, all blocks in the window are converted into a vector, and blocks in different regions are distinguished by assigning different masks. When performing matrix calculations, parts from the same region will be retained, and results from different regions will be discarded. Figure 5 As shown in the figure, the pure color part comes from the same area, and the matrix coefficient is set to zero when masking; the gradient part comes from different areas, and the matrix coefficient is set to a negative value when masking. After calculation, the size of the gray part after the activation function approaches zero, and the effect of window sliding is achieved.

[0079] Using the method of realizing the attention window, masking is performed according to the distance between entities to reduce the influence of long-distance characters on the current entity. Figure 6 Taking the example of "Is China Mobile's headquarters in Beijing?" in the example, when modeling the relational features of this sentence, according to the window size and label span, the mask weight within the window and the mask weight near the label are both 1, and the mask weight of the remaining part decreases with distance. Figure 6 The color depth of the square represents the weight. "China" and "Beijing" are entity labels. When calculating local attention within the SwinTransformer window, the weight within the window remains unchanged, the weight near the label outside the window remains unchanged, and the weight away from the label decreases with distance.

[0080] After performing attention calculation, S2.3 performs downsampling through patch merging to extract high-dimensional features and generate hierarchical representations. As the network goes deeper, the dimension of the feature matrix is ​​gradually reduced through patch merging, and finally the feature matrix is ​​extracted to the convolutional neural network to extract spatial information.

[0081] A bidirectional long short-term memory network is introduced for feature extraction. Unlike the traditional unidirectional recurrent neural network, the bidirectional long short-term memory network solves the gradient vanishing problem of the recurrent neural network. Through two independent long short-term memory networks: a forward long short-term memory network that processes the input in chronological order, and a backward long short-term memory network that processes the input in reverse chronological order, it can well obtain the feature information of the context and obtain dynamic word vector representation.

[0082] While extracting feature information from the bidirectional long short-term memory network, a masked SwinTransformer is introduced to extract features from the relationship matrix. SwinTransformer is widely used in the field of image processing and has successfully solved the problems of high time complexity and missing multi-scale information of VisionTransformer (VIT). For the extraction of spatial feature information in named entity recognition tasks, masks can be used flexibly to shield data, while also alleviating the problem of high time complexity of the model when processing long text sentences. Therefore, the masked SwinTransformer is used for spatial feature extraction between words, and the sliding window mechanism of SwinTransformer is used to reduce time complexity, so that the model has good time complexity while improving the effect of nested named entity recognition.

[0083] S3 calculates the span score and feeds the features extracted by S2 into two independent multi-layer perceptrons (MLPs), which represent the start and end of the entity respectively, and calculates the span score of each entity.

[0084] The entity span score calculation process is as follows:

[0085] The start and end scores of the entity are vectors h s and h e :

[0086] h s (i) = LeakyReLU(h i w s ),

[0087] h e (i) = LeakyReLU(h i w e ),

[0088] Among them, h i Represents the context representation information of the nth basic unit token; w s With w e is a learnable weight matrix; LeakyReLU is a nonlinear activation function, h s (i) represents the feature representation of the starting position of the i-th candidate entity, he (i) Feature representation of the end position of the i-th candidate entity;

[0089] Get the score of each entity through the biaffine attention matrix to get the span score of each entity:

[0090] r(i)=h s (i) T U m h e (i)+W m (h s (i)⊕h e (i))+b m ,

[0091] Among them, r represents the score of all possible named entity spans when the starting position of the span is before the ending position, and b m is the bias term, U m is a learnable bilinear mapping matrix used to model the bilinear similarity between the starting vector and the ending vector; W m It is a linear transformation weight matrix used to map the concatenated vector to a scalar score, which is the scoring weight after feature concatenation.

[0092] S4 convolution operation uses a convolutional neural network to convolve the entity span score and convert the spatial information between words into spatial features;

[0093] The convolution process is as follows:

[0094] R′=Conv2d(R),

[0095] R″=GeLU(LayerNorm(R ′ +R)),

[0096] Among them, R represents the span score matrix of all entities, R ′ Represents the result of a layer of convolution, and R″ represents the result after the activation function.

[0097] The number of characters n in a sentence is different, and R has different shapes. To ensure the same result when batching R, the positions without corresponding spans in the matrix R are filled with 0 to adapt to the batch processing operation of sentences of different lengths. After obtaining the score matrix R", the entity span scores of different spans are:

[0098] y=sigmoid(W o (R+R″)+b).

[0099] S5 smoothes the boundaries, assigns a small portion of the probability ε to the span adjacent to the current label, and the remaining probability 1-ε is assigned to the current label span. The boundary smoothing operation is performed on each label to obtain the smoothed label

[0100] Smoothed labels It is expressed as follows:

[0101]

[0102] in, is the smoothed label, representing the target probability distribution of span (i, j), y ij is the label in the original dataset, ε is the smoothing factor, which takes the value of ε∈{0.1,0.2,0.3}, D is the smoothing range, which takes the value of D∈{1,2}, and d is the Manhattan distance to the label entity, d <D,y i+d,j with y i,j+d Indicates the entity distribution after extending d basic unit tokens outward from the span boundary. If (i, j) is an entity, then y ij is 1, otherwise y ij is 0.

[0103] Use the smoothed labels to calculate the binary cross entropy loss to optimize the model:

[0104]

[0105] Among them, y′ ij Represents the probability of being predicted as an entity.

[0106] A named entity recognition system based on long-distance information enhancement and boundary smoothing, comprising:

[0107] Preprocessing module:

[0108] Used to use the pre-trained Chinese BERT encoder to input sentence X = [x1, x2, ..., x n ] is encoded to generate context representation information h i ;

[0109] Feature extraction module:

[0110] Used to extract features from corpus including contextual representation information using a bidirectional long short-term memory network and a masked SwinTransformer;

[0111] Span score calculation module:

[0112] It is used to feed the features extracted by S2 into two independent multi-layer perceptrons (MLPs), which represent the start and end of the entity respectively, and calculate the span score of each entity;

[0113] Convolution operation module:

[0114] It is used to convolve all entity span scores using convolutional neural networks to convert the spatial information between words into spatial features;

[0115] Smooth Boundary Module:

[0116] It is used to assign a small part of the probability ε to the span adjacent to the current label, and the remaining probability 1-ε is assigned to the span of the current label. The boundary smoothing operation is performed on each label to obtain the smoothed label.

[0117] This paper combines local and global position information with a boundary smoothing classification model. To verify the effectiveness of this paper, several effective methods are selected as baselines for comparison. The relevant baseline models are introduced as follows:

[0118] (1) Biaffine+CNN model: This model proposes a method based on biaffine and convolutional neural networks (CNNs) to solve the problem of nested named entity recognition in Chinese. The model effectively captures the spatial relationships in the score matrix through convolutional neural networks, especially in the case of overlapping entity spans, which enhances the model's ability to recognize nested entities. Experiments show that the use of convolutional neural networks can help the model discover more nested entities and surpass several recently proposed methods on three commonly used nested named entity recognition datasets.

[0119] (2) Triffine model: The Triffine model proposes a novel triple affine mechanism (Triaffine) to enhance the span representation capability in nested named entity recognition. This mechanism effectively integrates heterogeneous information such as boundaries, labels, internal tags, and related spans through triple affine attention and scoring, thereby improving classification performance. The model achieves state-of-the-art F1 scores on the GENIA and KBP2017 datasets and achieves comparable results on the ACE2004 and ACE2005 datasets.

[0120] (3) BILSTM+MutiCRF Suboptimal Path Model: This model proposes a suboptimal path method based on a bidirectional long short-term memory network and a multi-layer conditional random field (CRF) to identify nested entities. By treating the annotation sequence of the nested entity as a suboptimal path within the span of the parent entity, the model is able to gradually identify nested entities from the outer layer to the inner layer. This method does not require additional hyperparameters and performs well on the ACE-2004, ACE-2005, and GENIA datasets, with F1 scores of 85.82%, 84.34%, and 77.36%, respectively.

[0121] (4) Boundary prediction model based on position embedding and multi-level prediction: This model solves the problems of inaccurate boundary positioning and blurred boundaries between Chinese characters and words in Chinese nested named entity recognition. Through position embedding and multi-level boundary prediction mechanism, the model simultaneously encodes the position information of nested entities and text, thereby enhancing the relationship between nested entities and original text. At the decoding layer, the model re-determines the entity boundary by calculating the offset, improving the accuracy and recall of Chinese nested entity recognition. Experiments show that it significantly improves performance compared with the baseline model on medical and daily datasets.

[0122] like Figure 7 As shown in the figure, compared with existing models, the proposed method outperforms other models and shows improvements on both datasets. On the CNERTA dataset, the model achieves an F1 score of 83.27%, a precision of 86.08%, and a recall of 80.63%, representing improvements of 4.8, 4.99, and 4.92 percentage points, respectively. On the CMQNN dataset, the model achieves an F1 score of 97.90, a precision of 97.36, and a recall of 98.34 on the test set, representing improvements of 0.21, 0.33, and 0.27 percentage points, respectively.

[0123] Experimental results show that after combining feature extraction and boundary smoothing technology, the performance of the present invention is better than that of the model that only uses long short-term memory network encoding. This is due to the strong adaptability of the long short-term memory network and SwinTransformer encoder to nested entities, which can deeply capture the detailed information inside the entity. In addition, the use of boundary smoothing technology can improve the accuracy of nested entity recognition. Compared with Shibuya's model, the F1 score of the improved model on the CNERTA dataset increased by 24.88%; compared with Yan's model, the F1 score on the CNERTA dataset increased by 4.8%. Compared with Yua's model, the F1 score on the CNERTA dataset increased by 5.16%. Compared with Duan's model, it increased by 0.21% on the CMQNN dataset. These improvements indicate that the improved model better extracts the features of Chinese nested entities and can help the model improve its ability to extract nested named entities.

[0124] In order to verify the effectiveness of the present invention, an ablation experiment was conducted on the feature fusion layer and boundary smoothing method of the model on CNERTA to see how it affects the model effect. Figure 8 shown.

[0125] Ablation experiments validated the effectiveness of the feature fusion layer and boundary smoothing method for the nested named entity recognition task. After removing the SwinTransformer from the feature extraction module, the model's F1 score dropped to 82.15, demonstrating that the SwinTransformer effectively captures local and global information in the text through its multi-scale modeling capabilities during feature extraction, improving the model's understanding of complex entity structures. After removing the Long Short-Term Memory (LSTM) network module, the F1 score dropped to 82.68, demonstrating that the LSTM network plays a positive role in modeling sequential information. Removing the boundary smoothing mechanism had the greatest impact on performance, with a particularly significant drop in recall (from 80.63 to 79.2). This demonstrates that the boundary smoothing strategy effectively alleviates the problem of blurred entity boundaries, reduces false and missed recognitions, and thus improves the model's recall. Experimental results show that removing any module impacts model performance, indirectly demonstrating the effectiveness of the proposed method.

[0126] (2) Study the impact of sliding window size on model performance

[0127] Through experiments, we further explore the impact of sliding window size on model performance and conduct comparative analysis in the range of window size from 3 to 9. Figure 9 As shown, a window size of 6 achieves the best balance between capturing local details and integrating global semantics, demonstrating strong nested named entity recognition capabilities. Smaller windows (e.g., 3 or 4) help capture local information, but their limited field of view makes it difficult to fully capture long-range dependencies. Larger windows may introduce excessive redundant noise, increasing computational burden and exacerbating the risk of overfitting. Taking all these factors into consideration, a window size of 6 can more comprehensively extract relational information from the sequence, maintaining a balance between computational complexity and model generalization.

[0128] (3) Study the impact of boundary smoothing parameters on model performance

[0129] In the process of tuning the boundary smoothing parameters, the effects of different combinations of smoothing distance D and smoothing strength ε on the model performance were explored. Figure 10As shown, the smoothing distance D in the parameter experiment is taken as 1 and 2 respectively, and the smoothing strength ε is selected in the range of 0.1, 0.2, and 0.3. Under the premise of keeping other parameters unchanged, by comparing the performance of each combination in terms of classification accuracy, recall rate, F1 value and model robustness, it is found through experiments that there is an obvious synergistic effect between the parameters. When D is 2 and ε is 0.2, the model reaches the optimal state between smoothing boundary effects and balancing local details and global structures. Compared with the combination of D=1 and ε taking other values, this parameter combination can more effectively alleviate noise interference, reduce the risk of overfitting, and show a more stable improvement in overall performance, and provides a theoretical basis and parameter tuning reference for the model optimization of the present invention.

[0130] In the description of this specification, reference to the terms "one embodiment," "a preferred embodiment," "example," "specific example," or "some examples" means that the specific features, structures, materials, or characteristics described in conjunction with the embodiment or example are included in at least one embodiment or example of the present invention. Moreover, the specific features, structures, materials, or characteristics described may be combined in any suitable manner in any one or more embodiments or examples. In addition, those skilled in the art may combine and integrate different embodiments or examples described in this specification, as well as features of different embodiments or examples, unless otherwise inconsistent.

[0131] Any process or method description in a flowchart or otherwise described herein may be understood to represent a module, segment or portion of code comprising one or more executable instructions for implementing the steps of a specific logical function or process, and the scope of the preferred embodiments of the present invention includes alternative implementations in which functions may be performed out of the order shown or discussed, including performing functions in a substantially simultaneous manner or in the reverse order depending on the functions involved, which should be understood by those skilled in the art to which the embodiments of the present invention pertain.

[0132] The logic and / or steps represented in the flowcharts or otherwise described herein, for example, can be considered as a sequenced list of executable instructions for implementing the logical functions, and can be embodied in any computer-readable medium for use by, or in conjunction with, an instruction execution system, apparatus, or device (e.g., a computer-based system, a system including a processor, or other system that can fetch and execute instructions from an instruction execution system, apparatus, or device). For purposes of this specification, a "computer-readable medium" can be any device that can contain, store, communicate, propagate, or transport a program for use by, or in conjunction with, an instruction execution system, apparatus, or device. More specific examples (a non-exhaustive list) of computer-readable media include the following: an electrical connection with one or more wires (electronic devices), a portable computer disk cartridge (magnetic device), a random access memory (RAM), a read-only memory (ROM), an erasable and programmable read-only memory (EPROM or flash memory), a fiber optic device, and a portable read-only memory (CDROM). Furthermore, the computer-readable medium may even be paper or other suitable medium on which the program is printed, since the program may be obtained electronically, for example, by optically scanning the paper or other medium and then editing, interpreting or processing it in another suitable manner if necessary, and then storing it in a computer memory.

[0133] It should be understood that various parts of the present invention can be implemented using hardware, software, firmware, or a combination thereof. In the above embodiments, multiple steps or methods can be implemented using software or firmware stored in a memory and executed by a suitable instruction execution system. For example, if implemented using hardware, as in another embodiment, any one of the following technologies known in the art or a combination thereof can be used: a discrete logic circuit having a logic gate circuit for implementing a logic function on a data signal, an application-specific integrated circuit having a suitable combination of logic gate circuits, a programmable gate array (PGA), a field programmable gate array (FPGA), etc.

[0134] Those skilled in the art will appreciate that all or part of the steps in the above-described method embodiments can be accomplished by instructing the relevant hardware through a program, which can be stored in a computer-readable storage medium. When executed, the program includes one or a combination of the steps in the method embodiments. The device embodiments correspond to the method embodiments, so the description of the device embodiments is relatively brief, and the relevant description can refer to the description of the method embodiments.

[0135] Furthermore, the functional units in the various embodiments of the present invention may be integrated into a single processing module, each unit may exist physically separately, or two or more units may be integrated into a single module. The aforementioned integrated modules may be implemented in the form of hardware or software functional modules. If the integrated modules are implemented as software functional modules and sold or used as independent products, they may also be stored in a computer-readable storage medium. The storage medium may be a read-only memory, a magnetic disk, or an optical disk, etc.

[0136] The above description is merely a specific embodiment of the present invention, but the scope of protection of the present invention is not limited thereto. Any person skilled in the art can easily conceive of various modifications and substitutions within the technical scope disclosed in the present invention, and such modifications and substitutions are intended to be within the scope of protection of the present invention. Therefore, the scope of protection of the present invention shall be subject to the scope of protection of the claims.

Claims

1. A named entity recognition method based on long-distance information enhancement and boundary smoothing, characterized in that: The steps include: S1 preprocessing: Use the pre-trained Chinese BERT encoder to input sentence X=[x1,x2,…,x n ] is encoded to generate context representation information h i ; S2 feature extraction: Use bidirectional long short-term memory network and masked SwinTransformer to extract features from corpus including contextual representation information; S3 calculates span score: The features extracted by S2 are fed into two independent multi-layer perceptrons (MLPs) to represent the start and end of the entity, respectively, and the span score of each entity is calculated; S4 convolution operation: Use convolutional neural networks to convolve all entity span scores and convert the spatial information between words into spatial features; S5 Smooth Boundary: Assign a small part of the probability ε to the span adjacent to the current label, and the remaining probability 1-ε to the span of the current label. Perform boundary smoothing on each label to obtain the smoothed label.

2. The method for named entity recognition based on long-distance information enhancement and boundary smoothing according to claim 1, characterized in that: The encoding process in S1 is: for the input sentence X=[x1,x2,…,x n Each character x in ] n , the encoder first generates its corresponding character embedding, pronunciation embedding and glyph embedding, with embedding dimensions of D; then concatenates the character embedding, pronunciation embedding and glyph embedding, and maps them into a D-dimensional fusion embedding through a fully connected layer, adds position embedding information to the embedding, and sends it to the multi-layer Transformer layer to generate the character x n Context representation information h i ∈R D , the fusion representation information is H = {h1,h2,…,h i }.

3. The method for named entity recognition based on long-distance information enhancement and boundary smoothing according to claim 1, characterized in that: The feature extraction process described in S2 is as follows: S2.1 constructs a relationship matrix to represent the degree of association between characters and encodes implicit features such as semantic similarity and structural dependency; S2.2 divides the relationship matrix into blocks, and each block is used as a basic processing unit and input into SwinTransformer for feature modeling and attention calculation; S2.3 performs downsampling through block merging, extracts high-dimensional features, generates hierarchical representation, gradually reduces the dimension of the feature matrix, and extracts the feature matrix.

4. The method for named entity recognition based on long-distance information enhancement and boundary smoothing according to claim 1, characterized in that: The entity span score calculation process is as follows: The start and end scores of the entity are vectors h s and h e : h s (i)=LeakyReLU(h i In s ), h e (i)=LeakyReLU(h i In e ), Among them, h i Represents the context representation information of the nth basic unit token; w s With w e is a learnable weight matrix; LeakyReLU is a nonlinear activation function, h s (i) represents the feature representation of the starting position of the i-th candidate entity, h e (i) Feature representation of the end position of the i-th candidate entity; Get the score of each entity through the biaffine attention matrix to get the span score of each entity: Among them, r represents the score of all possible named entity spans when the starting position of the span is before the ending position, and b m is the bias term, U m is a learnable bilinear mapping matrix used to model the bilinear similarity between the starting vector and the ending vector; W m It is a linear transformation weight matrix used to map the concatenated vector to a scalar score, which is the scoring weight after feature concatenation.

5. The method for named entity recognition based on long-distance information enhancement and boundary smoothing according to claim 1, characterized in that: The convolution process described in S4 is as follows: R′=Conv2d(R), R″=GeLU(LayerNorm(R′+R)), Among them, R represents the span score matrix of all entities, R′ represents the result of one layer of convolution, and R″ represents the result after the activation function.

6. The method for named entity recognition based on long-distance information enhancement and boundary smoothing according to claim 1, characterized in that: The smoothed labels described in S5 It is expressed as follows: in, is the smoothed label, representing the target probability distribution of span (i, j), y ij is the label in the original dataset, ε is the smoothing factor, which takes the value of ε∈{0.1,0.2,0.3}, D is the smoothing range, which takes the value of D∈{1,2}, y i+d,j with y i,j+d Indicates the entity distribution after extending d basic unit tokens outward from the span boundary. If (i, j) is an entity, then y ij is 1, otherwise y ij is 0.

7. The method for named entity recognition based on long-distance information enhancement and boundary smoothing according to claim 6, characterized in that: The smoothing range D is set to 2, and the smoothing factor ε is set to 0.

2.

8. The method for named entity recognition based on long-distance information enhancement and boundary smoothing according to claim 1, characterized in that: Step S5 further includes: Use the smoothed labels to calculate the binary cross entropy loss to optimize the model: Among them, y′ ij Represents the probability of being predicted as an entity.

9. A named entity recognition system based on long-distance information enhancement and boundary smoothing, characterized in that: include: Preprocessing module: Used to use the pre-trained Chinese BERT encoder to input sentence X = [x1, x2, ..., x n ] is encoded to generate context representation information h i ; Feature extraction module: Used to extract features from corpus including contextual representation information using a bidirectional long short-term memory network and a masked SwinTransformer; Span score calculation module: It is used to feed the features extracted by S2 into two independent multi-layer perceptrons (MLPs), which represent the start and end of the entity respectively, and calculate the span score of each entity; Convolution operation module: It is used to convolve all entity span scores using a convolutional neural network to convert the spatial information between words into spatial features; Smooth Boundary Module: It is used to assign a small part of the probability ε to the span adjacent to the current label, and the remaining probability 1-ε is assigned to the span of the current label. The boundary smoothing operation is performed on each label to obtain the smoothed label.

Images