A data augmentation method and system for named entity recognition

Abstract

The application discloses a kind of named entity recognition data enhancement method and system, method includes: obtaining named entity recognition corpus;Named entity recognition corpus is preprocessed, and training data set is obtained;Training data set is input into untrained data enhancement model, and prompt parameter is generated by first data set to prompt generator generation processing, and pre-training model is updated according to prompt parameter, and the data enhancement model of training completion is obtained;Obtain the label filling matrix, and the data enhancement model is input into the label filling matrix and carries out data enhancement processing, and obtains data enhancement sentence;Data enhancement sentence is labeled conversion processing, and the third data set is obtained;Data extraction is carried out from the third data set to the first data set, and the data enhancement data set is obtained.The embodiment of the application can generate high-quality data, so as to reduce noise, improve model performance, and can be widely applied in artificial intelligence technical field.

Description

Technical Field

[0001] This invention relates to the field of artificial intelligence technology, and in particular to a data augmentation method and system for named entity recognition. Background Technology

[0002] Named entity recognition (NER) is an important branch of natural language processing and a foundation for tasks such as knowledge graph construction and information extraction. Currently, deep learning models for NER have achieved good performance. However, achieving good performance requires a large amount of manually labeled data. Data labeling is extremely time-consuming and labor-intensive, and the process is also quite challenging. In small sample sizes, NER models are prone to overfitting. To improve performance in small sample situations, existing techniques often use simple text transformations for data augmentation. However, these methods can easily destroy semantic information in the text, introducing significant noise into the model and resulting in unstable performance improvements. In summary, the technical problems in these technologies urgently need to be addressed. Summary of the Invention

[0003] In view of this, embodiments of the present invention provide a data augmentation method and system for named entity recognition to achieve the generation of high-quality data and reduce data noise.

[0004] On one hand, the present invention provides a data augmentation method for named entity recognition, comprising:

[0005] Obtain named entity recognition corpus;

[0006] The named entity recognition corpus is preprocessed to obtain a training dataset, which includes a first dataset and a second dataset. The second dataset is used to further pre-train the pre-trained model.

[0007] The training dataset is input into the untrained data augmentation model. The prompt generator is processed using the first dataset to generate prompts and obtain prompt parameters. The pre-trained model is then updated based on the prompt parameters to obtain the trained data augmentation model.

[0008] Obtain the annotation-fill matrix and input the annotation-fill matrix into the data augmentation model for data augmentation processing to obtain the data-augmented sentence;

[0009] The data-enhanced sentences are then labeled and transformed to obtain the third dataset.

[0010] Data is extracted from the third dataset and added to the first dataset to obtain a data augmentation dataset.

[0011] Optionally, the named entity recognition corpus is preprocessed to obtain a training dataset, which includes a first dataset and a second dataset, comprising:

[0012] A portion of the named entity recognition corpus is annotated to obtain the first dataset;

[0013] Another portion of the data in the named entity recognition corpus was identified as the second dataset.

[0014] Optionally, the step of obtaining the prompt parameters by performing prompt generation processing on the prompt generator using the first dataset includes:

[0015] The first dataset is processed by sentence expansion to obtain expanded data;

[0016] The expanded data is input into the prompt generator to obtain the prompt parameters.

[0017] Optionally, the step of performing sentence expansion processing on the first dataset to obtain expanded data includes:

[0018] The non-entity labels in the first dataset are removed to obtain the entity label dataset.

[0019] Move the entity labels in the entity label dataset before the corresponding words to obtain expanded data.

[0020] Optionally, before updating the pre-trained model according to the prompt parameters, the method further includes continuing the pre-training process of the pre-trained model, specifically including:

[0021] The second dataset is input into the pre-trained model, and encoded data is obtained through word embedding and position encoding.

[0022] The encoded data is input into the decoder, and the probability distribution of the label is calculated through the activation function.

[0023] Optionally, updating the pre-trained model based on the prompt parameters includes:

[0024] Obtain the attention vector of the pre-trained model;

[0025] The cue parameters are concatenated with the attention vector to obtain the updated pre-trained model.

[0026] Optionally, the step of annotating and transforming the data-enhanced sentences to obtain a third dataset includes:

[0027] The data-enhanced sentences are then filtered to obtain selected sentences;

[0028] The selected sentences are labeled to obtain the third dataset.

[0029] On the other hand, embodiments of the present invention also provide a data augmentation system for named entity recognition, comprising:

[0030] The first module is used to acquire named entity recognition corpus;

[0031] The second module is used to preprocess the named entity recognition corpus to obtain a training dataset. The training dataset includes a first dataset and a second dataset. The second dataset is used to further pre-train the pre-trained model.

[0032] The third module is used to input the training dataset into the untrained data augmentation model, use the first dataset to perform prompt generation processing on the prompt generator to obtain prompt parameters, and update the pre-trained model according to the prompt parameters to obtain the trained data augmentation model.

[0033] The fourth module is used to obtain the annotation-fill matrix and input the annotation-fill matrix into the data augmentation model for data augmentation processing to obtain the data-augmented sentence;

[0034] The fifth module is used to perform annotation and transformation processing on the data-enhanced sentences to obtain the third dataset;

[0035] The sixth module is used to extract data from the third dataset into the first dataset to obtain a data augmentation dataset.

[0036] Optionally, the second module is used to preprocess the named entity recognition corpus to obtain a training dataset, the training dataset including a first dataset and a second dataset, the second dataset being used to further pre-train the pre-trained model, including:

[0037] The first unit is used to annotate a portion of the data in the named entity recognition corpus to obtain a first dataset.

[0038] The second unit is used to determine another part of the data in the named entity recognition corpus as the second dataset.

[0039] Optionally, the third module is used to input the training dataset into an untrained data augmentation model, perform prompt generation processing on the prompt generator using the first dataset to obtain prompt parameters, and update the pre-trained model according to the prompt parameters to obtain a trained data augmentation model, including:

[0040] The third unit is used to perform sentence expansion processing on the first dataset to obtain expanded data;

[0041] The fourth unit is used to input the expanded data into the prompt generator to obtain prompt parameters.

[0042] Compared with existing technologies, the present invention employs the above technical solutions and has the following technical effects: In this embodiment, the training dataset is input into an untrained data augmentation model, and the first dataset is used to process the prompt generator to obtain prompt parameters. The pre-trained model is then updated based on these prompt parameters to obtain a trained data augmentation model. The prompt parameters can be used to leverage the knowledge learned by the pre-trained model during the continued pre-training phase to generate diverse and high-quality data to expand the dataset, thereby mitigating the impact of a lack of labeled data on the named entity recognition model and improving its robustness. Attached Figure Description

[0043] To more clearly illustrate the technical solutions in the embodiments of this application, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0044] Figure 1 This is a flowchart of a data augmentation method for named entity recognition provided in an embodiment of this application;

[0045] Figure 2 This is a system framework diagram of a data augmentation model provided in an embodiment of this application;

[0046] Figure 3 This is a schematic diagram illustrating the process of expanding labeled data according to an embodiment of this application. Detailed Implementation

[0047] To make the objectives, technical solutions, and advantages of this application clearer, the following detailed description is provided in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the scope of this application.

[0048] First, let's analyze some of the terms used in this application:

[0049] Named Entity Recognition (NER), also known as proper name recognition, is a fundamental task in Natural Language Processing (NLP) with a wide range of applications. Named entities generally refer to entities in text that have specific meaning or strong referentiality. NER models extract named entities from unstructured input text and can identify more categories of entities according to business needs. It can be widely applied in various NLP technologies such as information extraction, information retrieval, knowledge graphs, machine translation, and question answering systems.

[0050] Generative Pre-Training (GPT) is a type of generative pre-trained model based on the Transformer architecture. It involves unsupervised pre-training on a large-scale corpus followed by fine-tuning on a much smaller supervised dataset for specific tasks. By training a general-purpose model first and then fine-tuning it on individual tasks, this task-independent model design technique can achieve excellent performance across multiple tasks simultaneously.

[0051] In related technologies, data augmentation methods typically involve generating data by simply deleting text randomly, randomly swapping word positions in sentences, or randomly selecting and inserting words from sentences. However, these methods easily destroy the semantic information in the text, introducing significant noise into the model and resulting in unstable performance improvements. To address these issues, this invention provides a data augmentation method for named entity recognition based on a language generation model, generating high-quality and diverse data for named entity recognition data augmentation.

[0052] Reference Figure 1 This invention provides a data augmentation method for named entity recognition, comprising:

[0053] S101. Obtain named entity recognition corpus;

[0054] S102. Preprocess the named entity recognition corpus to obtain a training dataset. The training dataset includes a first dataset and a second dataset. The second dataset is used to further pre-train the pre-trained model.

[0055] S103. Input the training dataset into the untrained data augmentation model, use the first dataset to perform prompt generation processing on the prompt generator to obtain prompt parameters, and update the pre-trained model according to the prompt parameters to obtain the trained data augmentation model.

[0056] S104. Obtain the annotation filling matrix and input the annotation filling matrix into the data augmentation model for data augmentation processing to obtain the data augmented sentence;

[0057] S105. The data-enhanced sentences are labeled and transformed to obtain the third dataset;

[0058] S106. Extract data from the third dataset into the first dataset to obtain a data augmentation dataset.

[0059] This invention provides a method for data augmentation of named entity recognition based on a language generation model. It primarily uses the original training samples, i.e., named entity recognition corpora, as training data for the data augmentation model to generate a high-quality and diverse augmentation dataset. This expands the training set while reducing noise in the generated data, thereby mitigating the impact of insufficient labeled data on the named entity recognition model and improving its robustness. This invention acquires named entity recognition corpora, which can be obtained from various large corpora and is not limited in this embodiment. Next, this embodiment preprocesses the named entity recognition corpora by labeling a portion of the data to obtain a training dataset. The training dataset includes a first dataset and a second dataset. The first dataset is labeled data used to train the named entity recognition model; the second dataset is an unlabeled dataset used to further pre-train the pre-trained model. Then, this embodiment inputs the training dataset into the untrained data augmentation model. The first dataset is used to generate prompts for the prompt generator to obtain prompt parameters, and the pre-trained model is updated based on these prompt parameters to obtain a trained data augmentation model. The data augmentation model includes a prompt generator and a pre-trained model. Finally, in this embodiment of the invention, sentences are generated in batches by inputting a label-filling matrix filled with start tokens and completion tokens into a data augmentation model to obtain data-augmented sentences. These data-augmented sentences are then subjected to label transformation processing to obtain a third dataset. Data is extracted from the third dataset and added to the first dataset to obtain the data-augmented dataset. This embodiment of the invention generates diverse and high-quality data through a data augmentation model to expand the dataset, mitigating the impact of insufficient labeled data on the named entity recognition model and improving its robustness.

[0060] Reference Figure 2 In this embodiment of the invention, the GPT model is used as the language generation model, that is... Figure 2 The pre-trained model in the middle, through a small neural network, namely Figure 2The prompt generator generates prompt information and concatenates the prompt information to the Key and Value of the attention layer of the GPT model. In this way, the data generation task of this embodiment of the invention is adapted to the pre-training task of the GPT model, thereby utilizing the powerful language generation capability of the GPT model to generate high-quality and diverse data for data augmentation of named entity recognition.

[0061] As a further preferred embodiment, the named entity recognition corpus is preprocessed to obtain a training dataset, which includes a first dataset and a second dataset, comprising:

[0062] A portion of the named entity recognition corpus is annotated to obtain the first dataset;

[0063] Another portion of the data in the named entity recognition corpus was identified as the second dataset.

[0064] In this embodiment of the invention, named entity recognition corpus is collected and partially labeled. This labeled named entity recognition dataset is denoted as the first dataset. The first dataset is used to train the named entity recognition model and the data augmentation model of this embodiment. The unlabeled corpus is denoted as the second dataset, which is used to further pre-train the pre-trained model in the data augmentation model. In this embodiment, the GPT model can be used as the pre-trained model in the data augmentation model.

[0065] Further, as a preferred embodiment, the step of obtaining prompt parameters by performing prompt generation processing on the prompt generator using the first dataset includes:

[0066] The first dataset is processed by sentence expansion to obtain expanded data;

[0067] The expanded data is input into the prompt generator to obtain the prompt parameters.

[0068] In this embodiment of the invention, the first dataset, i.e., the annotated data in the named entity recognition corpus, is expanded and used as the training data for the data augmentation model. The expanded data is then used to train the prompt generator. During this training process, the parameters of the GPT model in the data augmentation model are frozen and remain unchanged during training. By training the neural network that generates prompt information, i.e., the prompt generator, the training objective is maximized.

[0069]

[0070] In the formula, This represents the first dataset. Let Θ represent the probability of the i-th output given the k inputs before it is determined, and let Θ represent the neural network parameters.

[0071] The mathematical expression of the neural network model of the prompt generator designed in this embodiment of the invention is as follows:

[0072] P e =E M W pe ;

[0073] θ=tanh(P e W1+b1)W2+b2;

[0074] In the formula, E M It is an identity matrix of size M, where M is the length of the prompt parameter. b1 and b2 are trainable parameters, d model d represents the dimension of the input and output of the transformer block layer in a neural network. prompt It is a hyperparameter, d θ =2×N×d model .

[0075] In this embodiment of the invention, the neural network parameter θ is divided into two parts, used to prompt the key and value respectively, formalized as follows: and Where l represents the attention mechanism of the l-th layer, W represents the weight matrix, and X represents the input matrix.

[0076] As a further preferred embodiment, the step of performing sentence expansion processing on the first dataset to obtain expanded data includes:

[0077] The non-entity labels in the first dataset are removed to obtain the entity label dataset.

[0078] Move the entity labels in the entity label dataset before the corresponding words to obtain expanded data.

[0079] In this embodiment of the invention, the labeled data, in addition to being provided as the first dataset for training the NER model, is also expanded to serve as training data for the data augmentation model of this embodiment. This embodiment expands the labeled data into sentences by ignoring the non-entity label "O" and placing the entity label before the word. In one feasible implementation, an example of how to expand data labeled using the BIO format is given; other labeling methods are similar. For example, given labeled data like "Summer\tB-season\nday\tI-season\nmost\tO\nsuit\tO\neat\tO\nwater\tB-fruit\nmelon\tI-fruit\nle\tO\n", where "B-season", "I-season", "B-fruit", "I-fruit", and "O" represent labels, the expanded result is "B-season summer I-seasonday is the most suitable day to eat B-fruit watermelon, such as..." Figure 3 As shown in the figure, this embodiment of the invention removes non-entity labels from the first dataset to obtain an entity label dataset, and moves the entity labels in the entity label dataset before the corresponding words to obtain expanded data.

[0080] As a further preferred embodiment, before updating the pre-trained model according to the prompt parameters, the method further includes continuing the pre-training process of the pre-trained model, specifically including:

[0081] The second dataset is input into the pre-trained model, and encoded data is obtained through word embedding and position encoding.

[0082] The encoded data is input into the decoder, and the probability distribution of the label is calculated through the activation function.

[0083] In this embodiment of the invention, the collected unlabeled corpus, namely the second dataset, is used to further pre-train the GPT model in an unsupervised manner, allowing GPT to learn more domain-specific knowledge and giving it a more powerful generative capability in that domain. Specifically, in this embodiment of the invention, the second dataset is input into the pre-trained model, and encoded data is obtained through word embedding and positional encoding; the encoded data is then input into the decoder, and the probability distribution of the labels is calculated through an activation function.

[0084] In this embodiment of the invention, the input is an unsupervised corpus, and the second dataset is a sequence of tokens. Our goal is to maximize:

[0085]

[0086] Where k represents the window size and Θ represents the neural network parameters.

[0087] The input token sequence, after word embedding and positional encoding, is passed into multiple transformer docoder blocks. Finally, the probability of the corresponding token sequence label is calculated using softmax, expressed by the formula:

[0088] h0=UW e +W p ;

[0089]

[0090]

[0091] where U=(u -k ,…,u -1 ) represents the second dataset, h represents the transformer_block layer, and W e W represents the word embedding matrix. p This represents the position embedding matrix.

[0092] As a further preferred embodiment, the step of updating the pre-trained model according to the prompt parameters includes:

[0093] Obtain the attention vector of the pre-trained model;

[0094] The cue parameters are concatenated with the attention vector to obtain the updated pre-trained model.

[0095] In this embodiment of the invention, the first dataset is input into the untrained data augmentation model using the above method to calculate the attention vector of the pre-trained model. The attention vector includes a query vector, a key vector, and a value vector. Then, this embodiment of the invention concatenates the prompt parameters generated by the prompt generator with the key and value of the pre-trained model to obtain the updated pre-trained model.

[0096]

[0097] In the formula, Attention represents the attention layer in the pre-trained model, softmax represents the activation function, and Q, K, and V represent the query vector, key vector, and value vector, respectively. This represents the concatenated key vector. d represents the concatenated value vector, and d represents the dimension of the attention vector.

[0098] As a further preferred embodiment, the step of annotating and transforming the data-enhanced sentences to obtain a third dataset includes:

[0099] The data-enhanced sentences are then filtered to obtain selected sentences;

[0100] The selected sentences are labeled to obtain the third dataset.

[0101] In this embodiment of the invention, to further improve the quality of the generated sentences, a filter is set up to delete sentences that do not meet the requirements. The deletion rules are as follows:

[0102] (1) Delete sentences containing only the non-entity tag "O";

[0103] (2) Delete sentences containing the unknown token;

[0104] (3) Delete sentences that do not conform to the annotation rules;

[0105] (4) Delete identical sentences but with different annotations;

[0106] (5) Delete “B-word” and “I-word” when “word” is not in the tag set.

[0107] In this embodiment of the invention, after filtering the data-enhanced sentences according to the above rules, the selected sentences are obtained. Then, the selected sentences are labeled. By manually labeling the selected sentences, the labeling method can be BIO format labeling, etc., ignoring non-entity tags "O", and then placing entity tags at the beginning of the words to expand the labeled data into sentences, thus obtaining the third dataset.

[0108] Combined with appendix Figure 1The specific process of this invention includes: First, acquiring named entity recognition corpus, preprocessing the corpus to obtain a training dataset. Next, inputting the training dataset into an untrained data augmentation model, using the first dataset to generate prompts for the prompt generator to obtain prompt parameters, and updating the pre-trained model based on these parameters to obtain a trained data augmentation model. Then, acquiring a label-filling matrix and inputting it into the data augmentation model for data augmentation to obtain data-augmented sentences. Finally, performing label transformation on the data-augmented sentences to obtain a third dataset, and then extracting data from the third dataset into the first dataset to obtain a data-augmented dataset. This invention expands the training set by extracting data from the third dataset into the first dataset, thereby mitigating the impact of insufficient labeled data on the named entity recognition model and improving its robustness. By adopting the above technical solution, this invention can generate diverse and high-quality data, expanding the dataset while reducing noise generation, thus more stably and effectively improving the performance of the named entity recognition model. Compared with related technologies, the embodiments of the present invention can generate more diverse and higher quality data, and the improvement of named entity recognition models is more stable and effective.

[0109] On the other hand, embodiments of the present invention also provide a data augmentation system for named entity recognition, comprising:

[0110] The first module is used to acquire named entity recognition corpus;

[0111] The second module is used to preprocess the named entity recognition corpus to obtain a training dataset. The training dataset includes a first dataset and a second dataset. The second dataset is used to further pre-train the pre-trained model.

[0112] The third module is used to input the training dataset into the untrained data augmentation model, use the first dataset to perform prompt generation processing on the prompt generator to obtain prompt parameters, and update the pre-trained model according to the prompt parameters to obtain the trained data augmentation model.

[0113] The fourth module is used to obtain the annotation-fill matrix and input the annotation-fill matrix into the data augmentation model for data augmentation processing to obtain the data-augmented sentence;

[0114] The fifth module is used to perform annotation and transformation processing on the data-enhanced sentences to obtain the third dataset;

[0115] The sixth module is used to extract data from the third dataset into the first dataset to obtain a data augmentation dataset.

[0116] Further, as a preferred embodiment, optionally, the second module is used to preprocess the named entity recognition corpus to obtain a training dataset, the training dataset including a first dataset and a second dataset, the second dataset being used to further pre-train the pre-trained model, including:

[0117] The first unit is used to annotate a portion of the data in the named entity recognition corpus to obtain a first dataset.

[0118] The second unit is used to determine another part of the data in the named entity recognition corpus as the second dataset.

[0119] Further, as a preferred embodiment, the third module is used to input the training dataset into the untrained data augmentation model, perform prompt generation processing on the prompt generator using the first dataset to obtain prompt parameters, and update the pre-trained model according to the prompt parameters to obtain the trained data augmentation model, including:

[0120] The third unit is used to perform sentence expansion processing on the first dataset to obtain expanded data;

[0121] The fourth unit is used to input the expanded data into the prompt generator to obtain prompt parameters.

[0122] In summary, the embodiments of the present invention have the following advantages: The present invention uses a cue-based learning method to make good use of the knowledge learned by the pre-trained model during the pre-training stage, and can generate diverse and high-quality data to expand the dataset, thereby mitigating the impact of a lack of labeled data on the named entity recognition model and improving the robustness of the named entity recognition model.

[0123] In some alternative embodiments, the functions / operations mentioned in the block diagrams may not occur in the order shown in the operation diagrams. For example, depending on the functions / operations involved, two consecutively shown blocks may actually be executed substantially simultaneously, or the blocks may sometimes be executed in reverse order. Furthermore, the embodiments presented and described in the flowcharts of this invention are provided by way of example to provide a more comprehensive understanding of the technology. The disclosed methods are not limited to the operations and logic flows presented herein. Alternative embodiments are contemplated in which the order of various operations is altered and sub-operations described as part of a larger operation are executed independently.

[0124] Furthermore, although the invention has been described in the context of functional modules, it should be understood that, unless otherwise stated, one or more of the described functions and / or features may be integrated into a single physical device and / or software module, or one or more functions and / or features may be implemented in a separate physical device or software module. It is also understood that a detailed discussion of the actual implementation of each module is unnecessary for understanding the invention. Rather, given the properties, functions, and internal relationships of the various functional modules in the apparatus disclosed herein, the actual implementation of the module will be understood within the scope of conventional skill of an engineer. Therefore, those skilled in the art can implement the invention as set forth in the claims using ordinary techniques without excessive experimentation. It is also understood that the specific concepts disclosed are merely illustrative and not intended to limit the scope of the invention, which is determined by the full scope of the appended claims and their equivalents.

[0125] If the aforementioned functions are implemented as software functional units and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this invention, essentially, or the part that contributes to the prior art, or a portion of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of this invention. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0126] The logic and / or steps represented in the flowchart or otherwise described herein, for example, can be considered as a sequenced list of executable instructions for implementing logical functions, and can be embodied in any computer-readable medium for use by, or in conjunction with, an instruction execution system, apparatus, or device (such as a computer-based system, a processor-included system, or other system that can fetch and execute instructions from, an instruction execution system, apparatus, or device). For the purposes of this specification, "computer-readable medium" can be any means that can contain, store, communicate, propagate, or transmit programs for use by, or in conjunction with, an instruction execution system, apparatus, or device.

[0127] In the description of this specification, references to terms such as "one embodiment," "some embodiments," "example," "specific example," or "some examples," etc., indicate that a specific feature, structure, material, or characteristic described in connection with that embodiment or example is included in at least one embodiment or example of the invention. In this specification, the illustrative expressions of the above terms do not necessarily refer to the same embodiment or example. Furthermore, the specific features, structures, materials, or characteristics described may be combined in any suitable manner in one or more embodiments or examples.

[0128] Although embodiments of the invention have been shown and described, those skilled in the art will understand that various changes, modifications, substitutions and alterations can be made to these embodiments without departing from the principles and spirit of the invention, the scope of which is defined by the claims and their equivalents.

[0129] The above is a detailed description of the preferred embodiments of the present invention, but the present invention is not limited to the embodiments described. Those skilled in the art can make various equivalent modifications or substitutions without departing from the spirit of the present invention, and these equivalent modifications or substitutions are all included within the scope defined by the claims of this application.

Claims

1. A data augmentation method for named entity recognition, characterized in that, The method includes: Obtain named entity recognition corpus; The named entity recognition corpus is preprocessed to obtain a training dataset, which includes a first dataset and a second dataset. The second dataset is used to further pre-train the pre-trained model. The training dataset is input into the untrained data augmentation model. The prompt generator is processed using the first dataset to generate prompts and obtain prompt parameters. The pre-trained model is then updated based on the prompt parameters to obtain the trained data augmentation model. Obtain the annotation-fill matrix and input the annotation-fill matrix into the data augmentation model for data augmentation processing to obtain the data-augmented sentence; The data-enhanced sentences are then labeled and transformed to obtain the third dataset. Data is extracted from the third dataset and fed into the first dataset to obtain a data augmentation dataset; The step of obtaining prompt parameters by processing the prompt generator with the first dataset includes: The first dataset is processed by sentence expansion to obtain expanded data; The expanded data is input into the prompt generator to obtain the prompt parameters; The mathematical expression for the neural network model of the prompt generator is: ； ； In the formula, It is an identity matrix of size M, where M is the length of the prompt parameter. , , b1 and b2 are trainable parameters. The dimensions representing the input and output of a transformer block in a neural network. It is a hyperparameter. .

2. The method according to claim 1, characterized in that, The named entity recognition corpus is preprocessed to obtain a training dataset, which includes a first dataset and a second dataset, comprising: A portion of the named entity recognition corpus is annotated to obtain the first dataset; Another portion of the data in the named entity recognition corpus was identified as the second dataset.

3. The method according to claim 1, characterized in that, The process of expanding sentences in the first dataset to obtain expanded data includes: The non-entity labels in the first dataset are removed to obtain the entity label dataset. Move the entity labels in the entity label dataset before the corresponding words to obtain expanded data.

4. The method according to claim 1, characterized in that, Before updating the pre-trained model according to the prompt parameters, the method further includes continuing the pre-training process of the pre-trained model, specifically including: The second dataset is input into the pre-trained model, and encoded data is obtained through word embedding and position encoding. The encoded data is input into the decoder, and the probability distribution of the label is calculated through the activation function.

5. The method according to claim 1, characterized in that, The step of updating the pre-trained model based on the prompted parameters includes: Obtain the attention vector of the pre-trained model; The cue parameters are concatenated with the attention vector to obtain the updated pre-trained model.

6. The method according to claim 1, characterized in that, The process of annotating and transforming the data-enhanced sentences to obtain a third dataset includes: The data-enhanced sentences are then filtered to obtain selected sentences; The selected sentences are labeled to obtain the third dataset.

7. A data augmentation system for named entity recognition, characterized in that, The system includes: The first module is used to acquire named entity recognition corpus; The second module is used to preprocess the named entity recognition corpus to obtain a training dataset. The training dataset includes a first dataset and a second dataset. The second dataset is used to further pre-train the pre-trained model. The third module is used to input the training dataset into the untrained data augmentation model, use the first dataset to perform prompt generation processing on the prompt generator to obtain prompt parameters, and update the pre-trained model according to the prompt parameters to obtain the trained data augmentation model. The fourth module is used to obtain the annotation-fill matrix and input the annotation-fill matrix into the data augmentation model for data augmentation processing to obtain the data-augmented sentence; The fifth module is used to perform annotation and transformation processing on the data-enhanced sentences to obtain the third dataset; The sixth module is used to extract data from the third dataset into the first dataset to obtain a data augmentation dataset; The third module is used to input the training dataset into the untrained data augmentation model, perform prompt generation processing on the prompt generator using the first dataset to obtain prompt parameters, and update the pre-trained model according to the prompt parameters to obtain the trained data augmentation model, including: The third unit is used to perform sentence expansion processing on the first dataset to obtain expanded data; The fourth unit is used to input the expanded data into the prompt generator to obtain prompt parameters; The mathematical expression for the neural network model of the prompt generator is: ； ； In the formula, It is an identity matrix of size M, where M is the length of the prompt parameter. , , b1 and b2 are trainable parameters. The dimensions representing the input and output of a transformer block in a neural network. It is a hyperparameter. .

8. The system according to claim 7, characterized in that, The second module is used to preprocess the named entity recognition corpus to obtain a training dataset, which includes a first dataset and a second dataset. The second dataset is used to further pre-train the pre-trained model, including: The first unit is used to annotate a portion of the data in the named entity recognition corpus to obtain a first dataset. The second unit is used to determine another part of the data in the named entity recognition corpus as the second dataset.

Images