Chinese named entity recognition method based on data enhancement and feature enhancement

Abstract

The invention relates to the technical field of natural language processing, in particular to a data enhancement and feature enhancement-based Chinese named entity recognition method, which is based on a feature enhancement double-attention named entity recognition model, and is characterized in that the model comprises an embedded layer, a data enhancement module, a multi-scale convolution fusion attention layer and a prediction layer; the embedded layer uses a dual-channel attention fusion module to process texts in parallel, fuses multi-dimensional information features of Chinese characters, combines an error correction type mask language model, a pre-training model and a bidirectional gating loop unit, fuses local and global text features, and obtains text representation from multiple dimensions and multiple levels; according to the method, by introducing an innovative model mechanism or training strategy, challenges such as label sparsity and text noise existing in a Chinese named entity recognition task can be effectively handled, the learning ability and generalization performance of the model for long-tail entity categories are improved, and the robustness of the model in a real and non-ideal data environment is enhanced.

Description

Technical Field

[0001] This invention relates to the field of natural language processing technology, specifically to a Chinese named entity recognition method that combines data augmentation and feature enhancement. Background Technology

[0002] Named entity recognition (NEntity Recognition) is an important task in Natural Language Processing (NLP), involving the identification and classification of named entities. NEntity Recognition plays an important role in many downstream tasks, such as recommendation systems, question answering systems, and event extraction.

[0003] Chinese is the second most spoken language in the world. With the acceleration of globalization and increasingly close cross-cultural exchanges, the application demand for Chinese named entity recognition is showing a continuous growth trend. Due to the polysemy and complex grammatical structure of Chinese, the task of identifying entity types and boundaries has become more difficult. Early named entity recognition methods were primarily rule-based and statistical, relying heavily on manual annotation and rules, requiring significant manpower and time, and exhibiting poor portability. With the continuous development of artificial intelligence technologies such as deep learning, deep learning models can automatically learn text feature representations and semantic information, rather than relying on pre-defined rules. These models mainly include Recurrent Neural Networks (RNNs) and Long Short-Term Memory (LSTM) neural networks. These methods not only enhance the expressive power of the models but also effectively compensate for the shortcomings of traditional models in capturing semantic features. RNNs achieve sequence dependency modeling through their recurrent connection mechanism, effectively capturing contextual information, but they are susceptible to gradient explosion and are mostly unidirectional. Therefore, researchers have introduced bidirectional long short-term memory networks with gating mechanisms. Bi-LSTM models text sequences bidirectionally, enabling each character to incorporate bidirectional contextual information for richer and more accurate representations. However, in a temporal structure, the computation at the current moment depends on the output of the previous moment, thus hindering parallel computation. Subsequently, the transformer introduces a novel self-attention mechanism that can compute the correlation between each element in the sequence and other elements, assigning weights to each element to achieve context awareness. This eliminates the temporal structure, making parallel computation possible. Therefore, transformer-based models are widely used in named entity recognition tasks. Pre-trained models have proven their effectiveness in various Chinese natural language processing tasks. These models learn rich language representations through pre-training on large-scale texts and then fine-tuning on named entity annotation datasets, thereby achieving significant performance improvements in named entity recognition tasks.

[0004] However, many Chinese named entity recognition datasets suffer from label sparsity, with insufficient samples for some entity categories or long-tail entities, which limits the model's learning and generalization of these categories. At the same time, Chinese text contains a large number of polysemous words and ambiguous boundaries. In the Chinese word segmentation process, word segmentation errors and manual annotation errors can easily be introduced due to blurred boundaries, which can affect the robustness of the model. Most of the methods mentioned above have not solved the problems of text noise and data label sparsity. In addition, in order to capture the multi-dimensional features and dependencies in Chinese text sequences and thus provide reliable support for subsequent entity recognition and classification, the field of Chinese named entity recognition has adopted a variety of feature enhancement methods, such as methods based on external knowledge assistance, methods based on word fusion, and methods based on character shape and radicals. These methods effectively improve the model's ability to understand Chinese text and the accuracy of entity boundary recognition by introducing rich semantic and structural information. However, the above feature enhancement methods are mostly based on a single channel and a single convolutional structure, which has the problem of single-view modeling and limits the ability to integrate complex feature relationships and structural information such as character shape, phonology, and semantics. Summary of the Invention

[0005] (a) Technical problems to be solved To address the shortcomings of existing technologies, this invention provides a Chinese named entity recognition method with data augmentation and feature enhancement to solve the problems mentioned in the background art.

[0006] (II) Technical Solution To achieve the above objectives, the present invention provides the following technical solution: a Chinese named entity recognition method based on data augmentation and feature enhancement, the method being based on a feature-enhanced dual-attention named entity recognition model, the model comprising: The embedding layer uses a dual-channel attention fusion module to process text in parallel, fusing multi-dimensional information features of Chinese characters, and combining error-correcting masking language models, pre-trained models and bidirectional gated recurrent units to fuse local and global text features, thereby obtaining text representations from multiple dimensions and levels. The data augmentation module encodes the local contextual semantics of entities using a pre-trained language model during the data preprocessing stage, selects candidate entities from the entity dictionary based on semantic similarity, randomly replaces data samples to enhance their diversity, and uses adversarial training to perturb the generated word vectors to generate adversarial examples. The multi-scale convolutional fusion attention layer uses conditional normalization to construct and improve the grid representation of word pairs, while the multi-scale dilated convolutional fusion module extracts character dependencies and global features at different distances in the word pair grid, and combines spatial and channel attention to enhance feature channels. The prediction layer performs inference and decodes to predict all possible entity labels.

[0007] Furthermore, the embedding layer employs whole-word masking and substitution masking strategies, along with an improved next-sentence prediction task, to alleviate the semantic bias problem of traditional random masking and sentence pair prediction in Chinese tasks. This enhances the ability to model word representations and sentence relationships, resulting in superior performance in Chinese named entity recognition tasks. Therefore, this paper uses the embedding layer to process the input text to obtain the initial embedding representation of the text characters. For each character in a given Chinese sentence... The corresponding character embedding is obtained using the following formula: in, , This indicates the output dimension of the error-correcting masking language.

[0008] In Chinese named entity recognition tasks, relying solely on character embeddings is insufficient to fully cover all linguistic features, especially posing challenges for recognizing homographs, polyphonic characters, and similar-looking characters. Furthermore, the Chinese character system contains rich phonological and glyphic information. Therefore, this approach first converts each Chinese character into its tonal form (pinyin), constructs a pinyin vocabulary, and maps it to an index ID. Then, a pinyin embedding vector is generated using a trainable embedding matrix. Similarly, based on a pre-constructed Chinese character radical dictionary, each character is mapped to its corresponding radical category, and then a radical vector is generated through an embedding matrix. The three types of features are concatenated along the embedding dimension to obtain the multimodal fusion representation of the character: in, , Embedding dimensions for pinyin; , Embedding dimensions for radicals; ,in express , and The sum of dimensions.

[0009] Furthermore, the dual-channel attention fusion module includes a bidirectional gated recurrent unit and a dual-channel attention structure. The bidirectional gated recurrent unit, based on a single gated recurrent unit, further considers the forward and reverse information of the sequence. It consists of two independent forward-gated recurrent units and a reverse-gated recurrent unit. One processes the sequence in forward order, and the other in reverse order. In the Chinese named entity recognition task, the bidirectional gated recurrent unit can fuse forward and reverse text sequence information, effectively capturing contextual dependencies, thereby extracting richer local sequence features and contextual information. The hidden state vector generated by the bidirectional gated recurrent unit is denoted as... , used to represent the context-related features of the t-th character in the sequence, the multimodal fused embedding representation is input to the bidirectional gated recurrent unit encoding to obtain the hidden state containing the sequence context information. ; A single gated loop unit mainly consists of two gating mechanisms: a reset gate and an update gate. Both gates are constructed using two fully connected layers with logical activation functions. The reset gate determines how much hidden state information from the previous time step needs to be forgotten, while the update gate determines how much information from the previous time step needs to be retained in the hidden state of the current unit. The specific mathematical expressions are as follows: in, This indicates the current moment when the model is processing the sequence data. These are the output vectors for the reset gate and the update gate, respectively. Represents the hidden state vector; It is a candidate hidden state. Output the hidden state at the current moment. and These are the weight matrices and bias parameters corresponding to the reset gate, update gate, and candidate hidden states, respectively. , These are the activation functions.

[0010] Furthermore, the dual-channel attention structure extracts multi-dimensional features from Chinese text from different angles and levels through two channels with independent parameters but identical architecture. In addition, by fusing pinyin embedding and radical embedding on top of character embedding, the model obtains a more comprehensive multimodal input, enabling the two channels to focus on different dimensions of linguistic information. Subsequently, combined with a multi-head attention mechanism, the dual-channel features are dynamically fused, utilizing the complementarity of the two channels to enhance semantic and representational information. Specifically, the multimodal representation of the text is input into the bidirectional gated recurrent units of the two channels to obtain hidden states containing contextual information of the text sequence, which are then labeled as follows: and Subsequently, the output states of the two channels are concatenated as follows to obtain the joint feature vector of the two-channel information. , in, The dual-channel joint feature representing each sequence position fuses complementary information from both channels, resulting in a spliced ​​feature representation. It needs to undergo multi-head attention processing to further dynamically fuse and enhance feature representations, specifically by splicing features. Linear mapping to , , : in, Indicates the first One point of attention, The corresponding learnable parameters are then used. Subsequently, the attention matrix is ​​split into multiple heads using multi-head attention, with each head learning different semantic and representational information from the text embedding. The formula for calculating the attention points is: in, Indicates the first The output of each attention head, For each head dimension, the outputs of all attention heads are then concatenated and linearly mapped: in, The learnable parameters are represented, and finally, through residual connections and layer normalization, the dual-channel feature representation after dynamic fusion of multi-head attention is obtained: in, , This indicates the embedding dimension after fusion.

[0011] Furthermore, the data augmentation module includes semantic entity replacement and adversarial training, wherein the semantic entity replacement provides the entity set of the given text. For each entity, select its context text according to the set window size. The entity context is encoded into a vector through an encoding model. Simultaneously, by utilizing a pre-built entity context dictionary, information related to the entity is obtained. A set of candidate entities of the same type and its candidate entity context set The candidate context is encoded to obtain the corresponding vector. Subsequently, the similarity between the query entity and the candidate entity's context vector is calculated using cosine similarity and compared with a preset threshold. To select the entity to replace, follow these steps: in, This represents the set of valid replacement entities after similarity filtering. As a similarity threshold, for each entity in the set of valid replacement entities, with probability... Randomly replace the original entity The replacement strategy for the newly generated sample is as follows: in, This represents the entity that has been replaced.

[0012] Furthermore, the adversarial examples introduced in the adversarial training are equivalent to artificially creating "noise" in the training data. This noise simulates Chinese word segmentation errors and similar natural annotation errors that may exist in real-world scenarios. By exposing the model to both original and adversarial examples simultaneously, it can adapt to the shifts caused by parameter perturbations, thereby improving the entity boundary perception ability and the robustness of the model in the Chinese named entity recognition task. FGM is an adversarial training method that quickly generates adversarial examples. In Chinese named entity recognition, the FGM strategy can be applied to pre-trained text embeddings as a regularization method to enhance the model's robustness to input perturbations. Let the set of pre-trained character embedding vectors be... The specific method for generating the disturbance is as follows: in, This represents the generated perturbation. The disturbance intensity coefficient is... It is the gradient of the loss function L with respect to the embedding vector E. Let L2 be the gradient vector norm. Here, L2 normalization not only accurately preserves the gradient direction but also eliminates the influence of the original gradient magnitude while retaining the gradient direction information. Based on the generated perturbation, the process of constructing adversarial examples and performing adversarial forward propagation and parameter updates is as follows: in, These are adversarial examples generated through perturbation. The adversarial loss is calculated based on adversarial examples. This represents the updated parameters, and the update process takes into account the gradient loss of the original samples. and counter-gradient loss The parameters are optimized by weighted summation.

[0013] Furthermore, the multi-scale convolutional fusion attention layer includes conditional layer normalization, multi-scale dilated convolutional layers, and channel attention and spatial attention mechanisms. Conditional layer normalization is used to construct a grid matrix representation of Chinese character pairs, resulting in a three-dimensional matrix. Each element in the matrix Represents corresponding Chinese character pairs Specifically, The calculation formula is as follows: in, As conditional inputs, used to generate two location-dependent hyperparameters. and , , and , This represents the learnable weight matrix and bias terms; and They represent respectively to The mean and standard deviation of a vector along its dimension are defined as follows: in, express The nth dimension.

[0014] Furthermore, a multi-scale dilated convolutional layer is used. This layer employs a four-layer dilated convolutional neural network structure, including three convolutional branches at different scales and one global average pooling branch, to extract relationships between characters at different distances. This better captures long-distance dependencies and determines entity boundary relationships. Specifically, the convolutional branches at different scales use different kernel sizes and dilation rates. The calculation process is as follows: in, ,in Represents the convolution channel dimension. The expansion rate is d and convolution kernel size k The convolution output of the dilated convolution operation, As the activation function, the global average pooling branch can extract global features between characters, integrating overall information into local features. This helps the network better understand the relationship between parts and the whole of characters, thereby improving the performance of entity boundary discrimination and recognition. The process is represented as follows: Among them, the global features P of the word pair matrix AvgPool(⋅) represents the global pooling operation, which uses bilinear interpolation to restore the global feature space dimension to 1. get Although all branch features are spliced ​​and fused: Among them, Q This represents the relationship between Chinese characters after integrating information at different granularities, effectively combining local and global feature information.

[0015] Furthermore, the channel attention and spatial attention mechanisms enhance the expressive power of character relationship representation. The channel attention mechanism aggregates information from each channel through global average pooling, and then... Convolutional transformations and nonlinear activation functions are used to generate channel weight vectors. These weight vectors are then used to perform a weighted summation of the original feature channels to enhance important feature channels and suppress irrelevant channels, thereby obtaining a weighted channel feature representation. Spatial attention mechanisms, on the other hand, focus on input features. conduct Convolution is used to integrate channel information from local spatial locations. Then, a spatial weight matrix is ​​generated by sigmoid activation. Finally, the original feature channels are weighted to obtain a spatially weighted feature representation. Ultimately, the channel attention output will be... Spatial attention output The characters are then merged to obtain the final character relation representation: Where max represents taking the larger of the two values ​​element by element. This is the final character relation grid representation that simultaneously integrates channel and spatial attention.

[0016] Furthermore, the feature is that the prediction layer employs a multilayer perceptron, inputting the final character relationship representation into the multilayer perceptron to calculate the character relationship score, the specific calculation formula of which is as follows: in, For the score matrix, Represents the weight matrix. This is the bias term. Subsequently, the diagonal portion is extracted from this matrix as the final score representation for each character in the sequence, combining local and global features. Next, the final classification probability for each Chinese character is calculated using the normalized exponential function, as shown in the formula: in, Let represent the predicted probability distribution of each character in the sequence across all categories. We define the learning objective as the cross-entropy loss function, which is widely used in classification tasks and is defined as follows: in, For sequence length, For the number of categories, Indicates the true label, This represents the probability of the corresponding class predicted by the model.

[0017] Beneficial effects Compared with existing technologies, this invention provides a Chinese named entity recognition method with data augmentation and feature enhancement, which has the following beneficial effects: 1. This invention, by introducing innovative model mechanisms or training strategies, can effectively address challenges such as label sparsity and text noise in Chinese named entity recognition tasks, improve the model's learning ability and generalization performance for long-tail entity categories, and enhance the model's robustness in real, non-ideal data environments.

[0018] 2. This invention employs a multi-channel, multi-scale, or heterogeneous fusion architecture, which can collaboratively extract and integrate complex features of Chinese text from different granularities and perspectives, significantly enhancing the model's ability to understand the characteristics of the Chinese language. In particular, when dealing with problems such as polysemous words and ambiguous boundaries, it can more accurately locate and identify entity boundaries.

[0019] 3. This invention combines the self-attention mechanism of the transformer with the advantages of recurrent or convolutional networks. While retaining the high efficiency of parallel computing, it further enhances the ability to capture long-distance contextual dependencies in Chinese. This enables the model to understand text semantics more comprehensively and improves the recognition accuracy of complex sentence structures and nested entities.

[0020] 4. This invention uses full-word masking and substitution masking strategies, as well as an improved next-sentence prediction task, to alleviate the semantic bias problem of traditional random masking and sentence pair prediction in Chinese tasks, and enhances the ability to model word representation and sentence relationships. Attached Figure Description

[0021] The accompanying drawings, which are included to provide a further understanding of the invention and form part of this application, illustrate exemplary embodiments of the invention and, together with their description, serve to explain the invention and do not constitute an undue limitation thereof. In the drawings: Figure 1 This is a diagram illustrating the overall framework of the feature-enhanced dual-attention named entity recognition model of the present invention. Figure 2 This is a flowchart of the overall SER operation of the present invention. Detailed Implementation

[0022] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0023] Specific implementation examples are given below.

[0024] Example Please see Figure 1 and Figure 2 This invention provides a Chinese named entity recognition method based on data augmentation and feature enhancement, using a feature-enhanced dual-attention named entity recognition model. The model includes: Data preprocessing and data augmentation module: Before training the model, the original Chinese text dataset is preprocessed. For a given sentence, the entity set is identified and extracted. For each entity Centered on the entity, select the five characters before and after it as the context window. This forms local contextual text. Then, the data augmentation phase begins: Semantic entity replacement: using a pre-trained error-correcting masked language model for entity replacement. and its context Encode to obtain the context semantic vector Filter out entities from the pre-built entity dictionary that are related to All candidate entities of the same type (e.g., all are "person names" or "place names") They also use error-correcting masked language models to obtain their context encoding vectors. ,calculate With each Cosine similarity, setting a threshold Only candidate entities with a similarity greater than or equal to the threshold are retained to form an effective replacement set. Finally, based on the probability of replacement For the original entity Random replacements are performed to generate new training samples, thereby increasing the diversity of the data. Adversarial training: During model training, adversarial training is performed using the fast gradient sign method. Let the current input character embedding set be... The model loss function is Calculate the loss Compared to embedding gradient Generate disturbance The disturbance intensity coefficient Set as , to embed the original With disturbance Add them together to get the adversarial example. The model's total loss consists of the original sample loss and the adversarial sample loss. By jointly optimizing these losses, the model's robustness to small input perturbations is improved.

[0025] Embedding Layer and Feature Enhancement Dual-channel Attention Fusion Module: Multimodal Embedding: For each Chinese character in the input sentence , first obtain its character embedding through an error-correction masked language model . At the same time, convert the Chinese character into its pinyin with tones (e.g., "中" is converted to "zhōng"), and generate pinyin embedding through a trainable pinyin embedding matrix . In addition, query the pre-constructed radical table to obtain the radical category corresponding to the Chinese character (e.g., the radical of "中" is "丨"), and generate radical embedding through a trainable radical embedding matrix . Concatenate the three in dimension to obtain multimodal fusion embedding . Dual-channel Attention Fusion: Input the multimodal fusion embedding sequence into the dual-channel attention fusion module. Both channels contain a bidirectional gated recurrent unit. The hidden state obtained after processing by the bidirectional gated recurrent unit in the first channel , and the hidden state obtained after processing by the bidirectional gated recurrent unit in the second channel . Concatenate the outputs of the two channels in the feature dimension to obtain joint features . Subsequently, input into the multi-head attention layer, calculate the attention weights and perform feature fusion, and finally output the fused feature representation through residual connection and layer normalization .

[0026] Multi-scale Convolutional Fusion Attention Layer: Conditional Layer Normalization and Grid Construction: Utilize the conditional layer normalization mechanism, and use each hidden state in as a condition to perform normalization and affine transformation on all other states , and construct a three-dimensional character pair grid matrix of . This matrix contains the potential relationships between all character pairs Multi-scale Feature Extraction: Apply multi-scale dilated convolution to the grid matrix . Set three convolutional branches: The first branch uses dilated convolution with a dilation rate of 1 and a convolution kernel of 3×3; the second branch uses dilated convolution with a dilation rate of 2 and a convolution kernel of 3×3; the third branch uses dilated convolution with a dilation rate of 3 and a convolution kernel of 3×3. These three branches capture local character dependencies at different distances. At the same time, a global average pooling branch extracts the global features of the entire grid and expands it back to size through bilinear interpolation. Finally, concatenate the outputs of the four branches in the channel dimension to obtain fused features . Channel and Spatial Attention: On Fusion Features Channel attention and spatial attention mechanisms are applied separately. Channel attention generates channel weights through global average pooling and 1×1 convolution, which are then used to weight the features. Spatial attention generates a spatial weight matrix through 1×1 convolution and a sigmoid function. The outputs of the two attention mechanisms are then maximized element-wise to obtain the final character relation representation. .

[0027] Prediction layer: The final character relationship representation The input is fed into a multilayer perceptron, and through a linear transformation and a sigmoid activation function, a score matrix is ​​calculated. ,from Extract the diagonal elements from the matrix to obtain the final score for each character in the sequence. Finally, the Softmax function is used to... Normalization is performed to obtain the predicted probability distribution of each character in each entity category (such as "person name", "place name", "organization name" etc.). The training objective of the model is to minimize the cross-entropy loss between the true label and the predicted probability.

[0028] The foregoing has shown and described the basic principles, main features, and advantages of the present invention. Those skilled in the art should understand that the present invention is not limited to the above embodiments. The embodiments and descriptions in the specification are merely illustrative of the principles of the invention. Various changes and modifications can be made to the invention without departing from its spirit and scope, and all such changes and modifications fall within the scope of the claimed invention.

Claims

1. A method for Chinese named entity recognition that combines data augmentation and feature enhancement, characterized in that, The method is based on a feature-enhanced dual-attention named entity recognition model, which includes: The embedding layer uses a dual-channel attention fusion module to process text in parallel, fusing multi-dimensional information features of Chinese characters, and combining error-correcting masking language models, pre-trained models and bidirectional gated recurrent units to fuse local and global text features, thereby obtaining text representations from multiple dimensions and levels. The data augmentation module encodes the local contextual semantics of entities using a pre-trained language model during the data preprocessing stage, selects candidate entities from the entity dictionary based on semantic similarity, randomly replaces data samples to enhance their diversity, and uses adversarial training to perturb the generated word vectors to generate adversarial examples. The multi-scale convolutional fusion attention layer uses conditional normalization to construct and improve the grid representation of word pairs, while the multi-scale dilated convolutional fusion module extracts character dependencies and global features at different distances in the word pair grid, and combines spatial and channel attention to enhance feature channels. The prediction layer performs inference and decodes to predict all possible entity labels.

2. The Chinese named entity recognition method with data augmentation and feature enhancement according to claim 1, characterized in that, The embedding layer employs whole-word masking and substitution masking strategies, along with an improved next-sentence prediction task. This paper uses the embedding layer to process the input text to obtain initial embedding representations of the text characters. For each character in a given Chinese sentence... The corresponding character embedding is obtained using the following formula: in, , This indicates the output dimension of the error-correcting masking language.

3. In Chinese named entity recognition tasks, relying solely on character embeddings is insufficient to fully cover all linguistic features, especially posing challenges for the recognition of homographs, polyphonic characters, and similar-looking characters. The Chinese character system also contains rich phonological and glyphic information. Therefore, each Chinese character is first converted into its tonal form (pinyin), a pinyin vocabulary is constructed and mapped to an index ID, and then a pinyin embedding vector is generated using a trainable embedding matrix. Similarly, based on a pre-constructed Chinese character radical dictionary, each character is mapped to its corresponding radical category, and then a radical vector is generated through an embedding matrix. The three types of features are concatenated along the embedding dimension to obtain the multimodal fusion representation of the character: in, , Embedding dimensions for pinyin; , Embedding dimensions for radicals; ,in express , and The sum of dimensions.

4. The Chinese named entity recognition method with data augmentation and feature enhancement according to claim 1, characterized in that, The dual-channel attention fusion module includes a bidirectional gated recurrent unit and a dual-channel attention structure. The bidirectional gated recurrent unit, based on a single gated recurrent unit, further considers both forward and reverse information of the sequence. It consists of two independent forward-gated and reverse-gated recurrent units, one processing the sequence in forward order and the other in reverse order. In Chinese named entity recognition tasks, the bidirectional gated recurrent unit can fuse forward and reverse text sequence information, effectively capturing contextual dependencies and thus extracting richer local sequence features and contextual information. The hidden state vector generated by the bidirectional gated recurrent unit is denoted as... , used to represent the context-related features of the t-th character in the sequence, the multimodal fused embedding representation is input to the bidirectional gated recurrent unit encoding to obtain the hidden state containing the sequence context information. ; A single gated loop unit mainly consists of two gating mechanisms: a reset gate and an update gate. Both gates are constructed using two fully connected layers with logical activation functions. The reset gate determines how much hidden state information from the previous time step needs to be forgotten, while the update gate determines how much information from the previous time step needs to be retained in the hidden state of the current unit. The specific mathematical expressions are as follows: in, This indicates the current moment when the model is processing the sequence data. These are the output vectors for the reset gate and the update gate, respectively. Represents the hidden state vector; It is a candidate hidden state. Output the hidden state at the current moment. and These are the weight matrices and bias parameters corresponding to the reset gate, update gate, and candidate hidden states, respectively. , These are the activation functions.

5. The Chinese named entity recognition method with data augmentation and feature enhancement according to claim 3, characterized in that, The dual-channel attention structure extracts multi-dimensional features from Chinese text from different angles and levels through two parameter-independent but architecturally identical channels. Furthermore, by fusing pinyin embedding and radical embedding on top of character embedding, the model obtains a more comprehensive multimodal input, allowing the two channels to focus on different dimensions of linguistic information. Subsequently, combined with a multi-head attention mechanism, the dual-channel features are dynamically fused, leveraging the complementarity of the two channels to enhance semantic and representational information. Specifically, the multimodal representation of the text is input into the bidirectional gated recurrent units of the two channels to obtain hidden states containing contextual information of the text sequence, which are then labeled as follows: and Subsequently, the output states of the two channels are concatenated as follows to obtain the joint feature vector of the two-channel information. , in, The dual-channel joint feature representing each sequence position fuses complementary information from both channels, resulting in a spliced ​​feature representation. It needs to undergo multi-head attention processing to further dynamically fuse and enhance feature representations, specifically by splicing features. Linear mapping to , , : in, Indicates the first One point of attention, The corresponding learnable parameters are then used. Subsequently, the attention matrix is ​​split into multiple heads using multi-head attention, with each head learning different semantic and representational information from the text embedding. The formula for calculating the attention points is: in, Indicates the first The output of each attention head, For each head dimension, the outputs of all attention heads are then concatenated and linearly mapped: in, The learnable parameters are represented, and finally, through residual connections and layer normalization, the dual-channel feature representation after dynamic fusion of multi-head attention is obtained: in, , This indicates the embedding dimension after fusion.

6. The Chinese named entity recognition method with data augmentation and feature enhancement according to claim 1, characterized in that, The data augmentation module includes semantic entity replacement and adversarial training. The semantic entity replacement provides a set of entities for the given text. For each entity, select its context text according to the set window size. The entity context is encoded into a vector through an encoding model. Simultaneously, by utilizing a pre-built entity context dictionary, information related to the entity is obtained. A set of candidate entities of the same type and its candidate entity context set The candidate context is encoded to obtain the corresponding vector. Subsequently, the similarity between the query entity and the candidate entity's context vector is calculated using cosine similarity and compared with a preset threshold. To select the entity to replace, follow these steps: in, This represents the set of valid replacement entities after similarity filtering. As a similarity threshold, for each entity in the set of valid replacement entities, with probability... Randomly replace the original entity The replacement strategy for the newly generated sample is as follows: in, This represents the entity that has been replaced.

7. The Chinese named entity recognition method with data augmentation and feature enhancement according to claim 5, characterized in that, The adversarial examples introduced in the adversarial training are equivalent to artificially creating "noise" in the training data. This noise simulates Chinese word segmentation errors and similar natural annotation errors that may exist in real-world scenarios. By exposing the model to both original and adversarial examples simultaneously, it can adapt to the shifts caused by parameter perturbations, thereby improving the model's ability to perceive entity boundaries and its robustness in Chinese named entity recognition tasks. FGM is an adversarial training method that quickly generates adversarial examples. In Chinese named entity recognition, the FGM strategy can be applied to pre-trained text embeddings as a regularization method to enhance the model's robustness to input perturbations. Let the set of pre-trained character embedding vectors be... The specific method for generating the disturbance is as follows: in, This represents the generated perturbation. The disturbance intensity coefficient is... It is the gradient of the loss function L with respect to the embedding vector E. Let L2 be the gradient vector norm. Here, L2 normalization not only accurately preserves the gradient direction but also eliminates the influence of the original gradient magnitude while retaining the gradient direction information. Based on the generated perturbation, the process of constructing adversarial examples and performing adversarial forward propagation and parameter updates is as follows: in, These are adversarial examples generated through perturbation. The adversarial loss is calculated based on adversarial examples. This represents the updated parameters, and the update process takes into account the gradient loss of the original samples. and counter-gradient loss The parameters are optimized by weighted summation.

8. The Chinese named entity recognition method with data augmentation and feature enhancement according to claim 1, characterized in that, The multi-scale convolutional fusion attention layer includes conditional layer normalization, multi-scale dilated convolutional layers, and channel and spatial attention mechanisms. Conditional layer normalization is used to construct a grid matrix representation of Chinese character pairs, resulting in a three-dimensional matrix. Each element in the matrix Represents corresponding Chinese character pairs Specifically, The calculation formula is as follows: in, As conditional inputs, used to generate two location-dependent hyperparameters. and , , and , This represents the learnable weight matrix and bias terms; and They represent respectively to The mean and standard deviation of a vector along its dimension are defined as follows: in, express The nth dimension.

9. The Chinese named entity recognition method with data augmentation and feature enhancement according to claim 1, characterized in that, The multi-scale dilated convolutional layer uses a four-layer dilated convolutional neural network structure, including three convolutional branches at different scales and one global average pooling branch, to extract relationships between characters at different distances, thereby better capturing long-range dependencies and determining entity boundary relationships. Specifically, the convolutional branches at different scales use different kernel sizes and dilation rates, and the calculation process is as follows: in, ,in Represents the convolution channel dimension. The expansion rate is d and convolution kernel size k The convolution output of the dilated convolution operation, As the activation function, the global average pooling branch can extract global features between characters, integrating overall information into local features. This helps the network better understand the relationship between parts and the whole of characters, thereby improving the performance of entity boundary discrimination and recognition. The process is represented as follows: Among them, the global features P of the word pair matrix , This represents a global pooling operation, which uses bilinear interpolation to restore the global feature space dimension to [value missing]. get Although all branch features are spliced ​​and fused: Among them, Q This represents the relationship between Chinese characters after integrating information at different granularities, effectively combining local and global feature information.

10. The Chinese named entity recognition method with data augmentation and feature enhancement according to claim 1, characterized in that, The channel attention and spatial attention mechanisms enhance the expressive power of character relationship representations. The channel attention mechanism aggregates information from each channel through global average pooling, and then... Convolutional transformations and nonlinear activation functions are used to generate channel weight vectors. These weight vectors are then used to perform a weighted summation of the original feature channels to enhance important feature channels and suppress irrelevant channels, thereby obtaining a weighted channel feature representation. Spatial attention mechanisms, on the other hand, focus on input features. conduct Convolution is used to integrate channel information from local spatial locations. Then, a spatial weight matrix is ​​generated by sigmoid activation. Finally, the original feature channels are weighted to obtain a spatially weighted feature representation. Ultimately, the channel attention output will be... Spatial attention output The characters are then merged to obtain the final character relation representation: Where max represents taking the larger of the two values ​​element by element. This is the final character relation grid representation that simultaneously integrates channel and spatial attention.

11. The Chinese named entity recognition method with data augmentation and feature enhancement according to claim 1, characterized in that, The prediction layer employs a multilayer perceptron. The final character relationship representation is input into the multilayer perceptron to calculate the score of the character relationship. The specific calculation formula is as follows: in, For the score matrix, Represents the weight matrix. This is the bias term. Subsequently, the diagonal portion is extracted from this matrix as the final score representation for each character in the sequence, combining local and global features. Next, the final classification probability for each Chinese character is calculated using the normalized exponential function, as shown in the formula: in, Let represent the predicted probability distribution of each character in the sequence across all categories. We define the learning objective as the cross-entropy loss function, which is widely used in classification tasks and is defined as follows: in, For sequence length, For the number of categories, Indicates the true label, This represents the probability of the corresponding class predicted by the model.

Images