Few-shot entity recognition method based on automatic prompt

Abstract

The application discloses a few-sample entity recognition method based on automatic prompt, specifically includes automatic template generation process, constructing prompt decoding process, interactive detection module entity prediction process; the application is based on prompt, and the generative method is used to remodel the named entity recognition task, which can reduce the dependence of the entity recognition task on sample data; the application adopts the automatically generated template based on entity knowledge enhancement, directly generates the optimal prompt template by the model prediction, does not need manual design and test template, and better entity recognition effect can be obtained in the few-sample scene; meanwhile, the application only needs to construct the prompt for each entity type, directly predicts the entity contained in the input text by the model, does not need to enumerate all text segments, and greatly reduces the time complexity.

Description

Technical Field

[0001] This invention belongs to the field of information extraction technology and relates to a few-sample entity recognition method based on automatic prompt. Background Technology

[0002] Named entity recognition (NER) is an important research topic in natural language processing. Its goal is to identify named entities (boundaries) in text and categorize them into their corresponding entity types, such as person names, place names, and organization names. NER is often treated as a sequence labeling task and typically employs a modified autoencoder language model, i.e., a pre-training + fine-tuning paradigm. The basic model structure is a pre-trained model (+BiLSTM) + CRF. However, the effectiveness of NER under the pre-training + fine-tuning paradigm depends on the availability of a large amount of labeled data. Currently, there is sufficient labeled data available in general domains. However, in many specialized fields, such as cybersecurity, due to their high level of expertise, labeling requires personnel with relevant professional knowledge, making the acquisition of labeled data very costly. Furthermore, with technological advancements and business changes, entity types frequently need to be updated, making the re-labeling of large amounts of data unacceptable. Therefore, researching NER methods for scenarios with limited labeled data, i.e., few-sample scenarios, is crucial.

[0003] Researchers are exploring ways to better transfer knowledge from pre-trained models by designing prompts (prompts) to guide the model to output specific answers. This approach is known as prompt learning. The advantage of this method is that it doesn't introduce any additional parameters, and the goal of the downstream task remains consistent with the goal of the pre-training task. Therefore, only a small number of samples are needed; by constructing appropriate prompts, the model can adapt to the downstream task. Constructing a prompt first requires designing a template, typically a piece of natural language text containing a mask for the position to be filled. This template is then concatenated with the original text, fed into a language model, and the model predicts the text that will produce the highest score at the position to be filled. Prompts are mostly manually designed. The advantages of manual templates are obvious: strong interpretability. However, they have the following problems: 1) High construction cost. Manual templates need to be designed by professionals with domain-related experience. Researchers typically manually design multiple experimental templates and then select the best-performing template through experiments. This requires significant manpower and domain knowledge to design and test these templates. 2) No guarantee of obtaining the optimal solution. Even experienced designers may struggle to create optimal templates, and even with significant effort, manually crafted templates can be suboptimal, impacting entity recognition performance. 3) High computational complexity. Some methods require enumerating all n-gram text fragments to find all possible entities, drastically reducing entity prediction speed. To address these issues, we need a few-shot entity recognition method based on automatic prompts, where the optimal prompt template is directly generated by model prediction, achieving excellent entity recognition results even in few-shot scenarios within specialized domains. Summary of the Invention

[0004] The purpose of this invention is to provide a few-shot entity recognition method based on automatic prompt, which solves the problems of existing technologies that require a large amount of labeled data and that the results of few-shot entity recognition using manually designed templates are poor and complex.

[0005] The technical solution adopted in this invention is a few-sample entity recognition method based on automatic prompt, which specifically includes an automatic template generation process, a prompt construction and decoding process, and an interactive detection module to predict entities. The specific steps are as follows: Step 1, Automatic template generation process: Entity knowledge is used as a prompt, that is, the entity category entity_type and the entity to be extracted candidate_span are known, the template position is filled with the special symbol [mask], and the model predicts the template in reverse; Step 2, constructing a prompt for decoding: Use the template automatically generated in Step 1 to construct a prompt for each entity type, and then send the input text and the prompt information together into the BART model for encoding; Step 3, Interactive detection module predicts entity process: In the entity prediction stage, an interactive detection module is designed to limit the output entities of the model to come from the input text sequence, so as to achieve controllable and structured entity prediction; Step 1 is implemented in the following steps: Step 1.1 involves constructing a prompt and injecting entity knowledge, specifically knowing the entity type (entity_type) and the entity to be extracted (candidate_span). The template position is filled with the special symbol [mask]. This is for a piece of network security text. X The prompt is constructed as follows: ; [CLS] and [SEP] indicate the beginning and end of the text, respectively; Step 1.2, input text sequence The inputs are fed into the model to predict the masked template portion. ,Right now [mask1]、[mask2] and [mask3] The generative model BART is used for prediction, and the search method is beam search. Step 1.3, by constructing N lines sequence The input sequence is fed into the model for prediction. The model prediction is a softmax multi-class classification, performed on each input sequence. The predicted probability is obtained from formula (2). ,at last Predicted probability Take the average of the model's N predictions, and select the top k tokens with the highest probabilities as candidate prediction results; (2) Step 1.4: After the prediction in Step 1.3 is completed, multiple predicted sequences are obtained. ,in The sequence with the highest overall prediction probability is selected as the automatically generated template. ; Step 2 is implemented in the following steps: Step 2.1: Construct a prompt for each entity type. For the named entity recognition task, constructing a prompt involves three elements: the input text sequence, and the input text sequence. X Entity category [entity_type] and entities to be extracted [candidate_span]For a text sequence X And templates generated by the automatic template generation module The constructed prompt is represented as: ; Step 2.2, set a hyperparameter—negative sampling rate. During the training phase, the number of negative samples input into the model is the total number of negative samples multiplied by 1. ; Step 2.3, input sequence After encoding by the Encoder module of the BART model, the hidden layer vector is obtained. As shown in the following formula: (3) In the formula, , The length of the text sequence after adding the prompt information is d, where d is the dimension of the hidden layer vector of the model. Step 2.4, for the token at time t The prediction will be the encoded vector of the Encoder. And the results of previous predictions Simultaneously, the Decoder module of the BART module is input, and an autoregressive method is used for prediction to obtain the last hidden layer vector output by the Decoder. .

[0006] The invention is further characterized by: negative sampling rate .

[0007] Step 3 is implemented in the following steps: Step 3.1, after obtaining Encoding vector Then, select the first l tokens, connect them to the MLP layer, and obtain the vector. ; (4) In the formula, ; Step 3.2, introduce a separator. When multiple entities of the same type exist in the input text, this is used as a separator between entities of the same type. Calculate its token encoding Then , and terminator Hidden layer vectors The sequences are concatenated to obtain the final candidate sequence encoding vector. ; (5) (6) In the formula, This indicates that the vectors are concatenated along the first dimension. ; Step 3.3, Calculation and The token can be obtained by passing the dot product through a softmax layer. Subscript in the input sequence Calculate using the following formula (7) The conditional probability can be obtained after decoding. , as in equation (8); (7) In the formula, ; (8) Step 3.4: Training is performed using a forced teaching method, which involves concatenating the initial word and the original input sequence together as the input to the Decoder module; the loss function during the training phase is the negative log-likelihood loss function, as shown in equation (9); during the prediction phase, decoding is performed sequentially using an autoregressive method, and the Decoder at each time step... t The input is the previous time step. t -1 is used to predict the token, and decoding ends when a stop character is encountered. (9) In the formula, m This is the length of the output sequence of the Decoder module.

[0008] The beneficial effects of this invention are: This invention addresses the problem of scarce labeled data in specialized fields such as cybersecurity by employing an automatic prompt-based few-shot entity recognition method. Based on prompts, it uses a generative approach to remodel the named entity recognition task, reducing its reliance on sample data. It utilizes an automatically generated template based on entity knowledge enhancement, directly predicting the optimal prompt template from the model, eliminating the need for manual template design and testing. This results in better entity recognition performance in few-shot scenarios. Furthermore, this invention only requires constructing a prompt for each entity type, allowing the model to directly predict the entities contained in the input text, without enumerating all text fragments, significantly reducing time complexity. Attached Figure Description

[0009] Figure 1This is a schematic diagram of automatic template generation based on entity knowledge enhancement in the few-shot entity recognition method based on automatic prompt of the present invention; Figure 2 This is a framework diagram of the few-shot named entity recognition model based on automatic prompt in the few-shot entity recognition method based on automatic prompt of the present invention. Detailed Implementation

[0010] The present invention will now be described in detail with reference to the accompanying drawings and specific embodiments.

[0011] Example 1 This embodiment provides a few-shot entity recognition method based on automatic prompt, which is implemented according to the following steps: Step 1, Automatic template generation based on entity knowledge enhancement: Incorporate entity knowledge to provide prompts, fill the template positions with special symbols [mask], construct a prompt, and feed it into the BART model, which then predicts and generates the template in reverse. Step 2, construct and encode the prompt: Use the automatically generated template to construct a prompt for each entity type and send it to the BART model for encoding and decoding; Step 3, predict entities using the interaction detection module: By designing the interaction detection module, the output entities of the model are limited to the input text sequence, and the structure of the output entities is guaranteed.

[0012] Example 2 This embodiment is based on a few-shot entity recognition method using automatic prompting. The specific steps in this embodiment are as follows: Step 1 is implemented in the following steps: Step 1.1 involves constructing a prompt and injecting entity knowledge, specifically knowing the entity type (entity_type) and the entity to be extracted (candidate_span). The template position is filled with the special symbol [mask]. This is for a piece of network security text. X The prompt is constructed as follows: ; Step 1.2, input text sequence The inputs are fed into the model to predict the masked template portion. ,Right now [mask1]、[mask2] and [mask3] The generative model BART is used for prediction, and the search method is beam search. Step 1.3, by constructing N lines sequence The input sequence is fed into the model for prediction. The model prediction is a softmax multi-class classification, performed on each input sequence. The predicted probability is obtained from formula (2). ,at last Predicted probability Take the average of the model's N predictions, and select the top k tokens with the highest probabilities as candidate prediction results; (2) Step 1.4: After the prediction in Step 1.3 is completed, multiple predicted sequences are obtained. ,in The sequence with the highest overall prediction probability is selected as the automatically generated template. ; Step 2 is implemented in the following steps: Step 2.1: Construct a prompt for each entity type. For the named entity recognition task, constructing a prompt involves three elements: the input text sequence, and the input text sequence. X Entity category [entity_type] and entities to be extracted [candidate_span] For a text sequence X And templates generated by the automatic template generation module The constructed prompt is represented as: ; Step 2.2, set a hyperparameter—negative sampling rate. , During the training phase, the number of negative samples input into the model is the total number of negative samples multiplied by 1. ; Step 2.3, input sequence After encoding by the Encoder module of the BART model, the hidden layer vector is obtained. As shown in the following formula: (3) In the formula, , The length of the text sequence after adding the prompt information is d, where d is the dimension of the hidden layer vector of the model. Step 2.4, for the token at time t The prediction will be the encoder's encoded vector. And the results of previous predictions Simultaneously, the Decoder module of the BART module is input, and an autoregressive method is used for prediction to obtain the last hidden layer vector output by the Decoder. ; Step 3 is implemented in the following steps: Step 3.1, after obtaining Encoding vector Then, select the first l tokens, connect them to the MLP layer, and obtain the vector. ; (4) In the formula, ; Step 3.2, introduce a separator. When multiple entities of the same type exist in the input text, this is used as a separator between entities of the same type. Calculate its token encoding Then , and terminator Hidden layer vectors The sequences are concatenated to obtain the final candidate sequence encoding vector. ; (5) (6) In the formula, This indicates that the vectors are concatenated along the first dimension. ; Step 3.3, Calculation and The token can be obtained by passing the dot product through a softmax layer. Subscript in the input sequence Calculate using the following formula (7) The conditional probability can be obtained after decoding. , as in equation (8); (7) In the formula, ; (8) Step 3.4: Training is performed using a forced teaching method, which involves concatenating the initial word and the original input sequence together as the input to the Decoder module; the loss function during the training phase is the negative log-likelihood loss function, as shown in equation (9); during the prediction phase, decoding is performed sequentially using an autoregressive method, and the Decoder at each time step... t The input is the previous time step. t -1 is used to predict the token, and decoding ends when a stop character is encountered. (9) In the formula, m This is the length of the output sequence of the Decoder module.

[0013] Example 3 like Figure 1 The diagram shown is a schematic of automatic template generation based on entity knowledge enhancement in this embodiment. Step 1, the automatic template generation process is as follows: For named entity recognition tasks, constructing a prompt involves three elements: the input text sequence X, the entity category, and the input text sequence X. [entity_type] and entities to be extracted [candidate_span] For a text sequence X, the constructed prompt can be represented as:

[0014] Where [CLS] and [SEP] represent the beginning and end of the text, respectively. The template in the prompt can be manually constructed. Manually constructed templates can be very flexible, but experiments show that different templates have a significant impact on the final entity recognition results, and manually constructed templates are not necessarily the best. Therefore, this embodiment designs an entity knowledge-enhanced automatic template generation method. Entity knowledge is used as a prompt, meaning the entity category (entity_type) and the entity to be extracted (candidate_span) are known. The template position is filled with the special symbol [mask]. The model predicts the template from the back end, thus making the template more closely related to the named entity recognition task. Specifically, for a text sequence X, the automatic template generation module constructs the following prompt: ; Input text sequence The samples are fed into the model to predict the masked template parts. ,Right now [mask1]、[mask2] and [mask3] In this embodiment, the generative model BART is used for prediction, and the search method is beam search; taking [mask1] as an example, let the Decoder be at this position. The last hidden layer vector is The model prediction essentially involves performing a softmax multi-class classification of class v, as shown in formula (1): (1) In the formula, ,express The probability of belonging to each character / word, where v is the vocabulary size of the language model; By constructing N lines sequence The data is fed into the model for prediction, and for each input sequence... The predicted probability is obtained from the formula. ,at last Predicted probability Take the average of the N predictions made by the model, as shown in formula (2): (2) In the formula, The top k tokens with the highest probabilities are selected as candidate prediction results. Then, each token is used as the current decoding result and input into the model to continue prediction. The result of the location, for The location is predicted using the same method, and the prediction ends when the average prediction result reaches the stop sign; finally, multiple predicted sequences are obtained. ,in The sequence with the highest overall prediction probability is used as the automatically generated template. .for [mask2] and [mask3] Obtained in the same way ; The table below shows the top 5 templates with the highest probabilities generated automatically by the model from 200 English text sequences. [entity_type] and [candidate_span] Let's take "organization" and "Huawei" as examples to illustrate; Table 1 Templates automatically generated by the model

[0015] Step 2, BART encoding and decoding process: We use automatically generated templates to build prompts for each entity type, and then feed the input text and prompt information together into the BART model for encoding. Specifically, for each entity type, we fill in... [entity_ type] Location: At this point, the entity type will be included as part of the prompt information; only the entity to be extracted will be represented using a special symbol. [mask] Replacements, for example, for the "Organization" type, include the automatically generated template. Finally, the input text It can be represented as: ; A text typically contains only a few named entities, or even none at all. If, during the training phase, a system is built for each text... The proportion of text that does not contain this entity type is very high; these samples are called negative samples. As the entity category k increases, the proportion of negative samples becomes increasingly higher, causing a serious label imbalance problem. The model tends to predict that the text does not contain any entity. Therefore, this embodiment sets a hyperparameter—negative sampling rate. , During the training phase, the number of negative samples input into the model is the total number of negative samples multiplied by 1. ; Input sequence After encoding by the Encoder module of the BART model, the hidden layer vector is obtained. As shown in the following formula: (3) In the formula, , The length of the text sequence after adding the prompt information is d, where d is the dimension of the hidden layer vector of the model. The process of constructing a prompt and decoding it is as follows: Figure 2 As shown, the output vector is obtained in this way. ; Step 3, the entity prediction process of the interactive detection module: For named entity recognition tasks [candidate_span] It can only come from the input text sequence X However, the Decoder module that directly uses the BART model... The output range is too wide, so in order to limit it... It can only originate from the input text sequence X. This embodiment designs an interactive prediction module. Formula (3) yields Then it will be combined with the input text X The final predicted token is obtained through calculation in the interactive prediction module. Specifically, to obtain Encoding vector Then, select the first l tokens and connect them to the MLP layer (Multilayer Perceptron), as shown in formula (4): (4) In the formula, In addition, this embodiment introduces a spacing symbol. To handle cases where multiple entities of the same type exist in the input text, this is used to represent the entity interval. Calculate its token code g, as shown in formula (5): (5) in ,Will and terminator Hidden layer vectors The sequences are concatenated to obtain the final candidate sequence encoding vector. As shown in formula (6): (6) This indicates that the vectors are concatenated along the first dimension. Subsequently, calculations were performed. and The token can be obtained by passing the dot product through a softmax layer. Subscript in the input sequence , The formula for calculating the conditional probability (7) is shown in Figure 7: (7) In the formula, After decoding, you can get As shown in formula (8): (8) In the Decoder module, a special token is used. <s>"As the starting word of the sequence, that is During training, a teacher-forcing method is adopted, that is, the starting word and the original input sequence are concatenated together as the input of the Decoder module; the loss function during the training phase is the negative log-likelihood loss function, as shown in formula (9): (9) in, m The length of the output sequence of the Decoder module is given. During the prediction phase, an autoregressive approach is used for sequential decoding. At each time step of the Decoder... t The input is the previous time step. t -1 is used to predict the token until a stop character is encountered.< / s> ", decoding ends; for example, for the "Organization" type in the previous example, the Decoder's output should be " <s> A company< / s> If the input text does not contain the entity type, the Decoder will directly output a stop character, and the output will be "". <s>< / s> The input text "A company's system kernel has a buffer overflow vulnerability." does not contain the "location" entity. For cases with multiple entities of the same type, use the special symbol "". <gap>Separate the text by adding a "organization" entity and then adding "apple". The Decoder's output should be "". <s>A company <gap> apple< / gap> < / s> ”; Finally, the Decoder's output is decoded according to the rules. This involves decoding each text sequence input to the Encoder. It can only output the sequence of entities of the corresponding entity type c. Therefore, for the input text... X The result of named entity recognition should be all of its... The sum of the decoding results of the sequence.

[0016] The process of predicting entities using the exchange detection module, such as Figure 2 As shown, this is how the entities contained in the input text are obtained; The BART model used in this invention employs 12 layers of Encoder and Decoder; the specific parameter settings are shown in Table 2. Table 2 Main Parameter Settings of the Model

[0017] The following table illustrates the entity recognition performance of this invention on public datasets in scenarios with sufficient labeled data and few samples, respectively. The three comparison models are BERT. BiLSTM CRF (from CN113204970A), Template NER (from Leyang Cui et al. Template-Based Named Entity Recognition Using BART. "Findings of the Association for Computational Linguistics: ACL-IJCNLP 2021". 2021, pp. 1835-1845.), BART NER (from Hang Yan et al. A Unified Generative Framework forVarious NER Subtasks. "The 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference onNatural Language Processing". 2021, Volume 1, Pages 5808-5822): Table 3 Comparison of entity recognition performance on the well-labeled Conll2003 dataset.

[0018] As shown in Table 3, under scenarios with sufficient labeled data, existing named entity recognition models all perform well, with F1 scores exceeding 90 indicating that the models can be applied in real-world scenarios; the sequence labeling model BERT... BiLSTM CRF shows a good balance between precision and recall; while the Template BART model is geared towards scenarios with few samples for named entity recognition, it also outperforms the sequence labeling model BERT when training data is plentiful. BiLSTM For CRF, Template BART uses a sliding window approach to exhaustively search for all candidate entity spans and entity types during the modeling process, resulting in an excellent recall rate of 94.34%. However, its precision is only 91.89%, far lower than the recall rate and also lower than BERT. BiLSTM The CRF and BART NER models are designed for named entity recognition tasks with sufficient labeled data, thus exhibiting excellent accuracy and recall, and their F1 scores are higher than BERT. BiLSTM CRF and Template BART have been improved; the method proposed in this invention has a relatively balanced performance in terms of precision and recall, with recall slightly higher than precision. The overall performance of the F1 score is also excellent in scenarios with sufficient training data, and is slightly higher than the latest model BART NER. After introducing the automatically generated template used in this invention, the F1 score is further improved, indicating that the automatically generated template of this invention is more effective than manually constructed templates.

[0019] Table 4 Comparison of entity recognition performance on the few-sample dataset MIT Restaurant

[0020] Table 5 Comparison of entity recognition performance on the few-shot dataset MIT Movie

[0021] As can be seen from Tables 4 and 5, in the few-shot named entity recognition task, the sequence labeling model BERT... BiLSTM CRF performs significantly worse than other models, with F1 scores of only around 20 on both the MIT Restaurant and MIT Movie datasets in 10-shot tests. The Template BART model, due to the introduction of a prompt, outperforms BERT on the MIT Restaurant dataset. BiLSTM The CRF performance improvement is more than double, and the BART NER model, due to its generative modeling approach, achieves a 76.1% improvement in F1 score. However, on the MIT Movie dataset, the BART NER model outperforms the Template BART model overall. This may be related to the fact that the MIT Movie dataset contains more entity types (12). The exhaustive approach of the Template BART model becomes time-consuming and its performance deteriorates when there are too many entity types. In contrast, the model proposed in this invention (which directly uses an automatically generated model) significantly outperforms all other models on both datasets. In the 10-shot experiment, its F1 score is significantly higher than BERT. BiLSTM The CRF improvement is nearly 200%, exceeding its F1 score on 200-shot and approaching its performance on 500-shot, demonstrating the excellent recognition capability of the prompt-based few-shot named entity recognition model proposed in this invention in few-shot scenarios. Meanwhile, on the MITMovie dataset, the F1 score of this invention is 93.7% higher than the Template BART model, which also uses a prompt, and 30.7% higher than the BART NER model, which uses a generative method. This indicates that the method of this invention leads other methods in modeling named entity recognition problems, and its performance is not affected by the increase in entity types.< / gap>

Claims

1. A few-shot entity recognition method based on automatic prompt, characterized in that, Specifically, this includes the automatic template generation process, the process of building and decoding the prompt, and the process of predicting entities by the interactive detection module; The specific steps are as follows: Step 1, Automatic template generation process: Entity knowledge is used as a prompt, that is, the entity category entity_type and the entity to be extracted candidate_span are known, the template position is filled with the special symbol [mask], and the model predicts the template in reverse; Step 2, constructing a prompt for decoding: Use the template automatically generated in Step 1 to construct a prompt for each entity type, and then send the input text and the prompt information together into the BART model for encoding; Step 3, Interactive detection module predicts entity process: In the entity prediction stage, an interactive detection module is designed to limit the output entities of the model to come from the input text sequence, so as to achieve controllable and structured entity prediction; Step 1 is implemented in the following steps: Step 1.1 involves constructing a prompt and injecting entity knowledge, specifically knowing the entity type (entity_type) and the entity to be extracted (candidate_span). The template position is filled with the special symbol [mask]. This is for a piece of network security text. X The prompt is constructed as follows: ； [CLS] and [SEP] indicate the beginning and end of the text, respectively; Step 1.2, input text sequence The inputs are fed into the model to predict the masked template portion. ,Right now [mask1]、[mask2] and [mask3] The generative model BART is used for prediction, and the search method is beam search. Step 1.3, by constructing N lines sequence The input sequence is fed into the model for prediction. The model prediction is a softmax multi-class classification, performed on each input sequence. The predicted probability is obtained from formula (2). ,at last Predicted probability Take the average of the model's N predictions, and select the top k tokens with the highest probabilities as candidate prediction results; （2） Step 1.4: After the prediction in Step 1.3 is completed, multiple predicted sequences are obtained. ,in The sequence with the highest overall prediction probability is selected as the automatically generated template. ; Step 2 is implemented in the following steps: Step 2.1: Construct a prompt for each entity type. For the named entity recognition task, constructing a prompt involves three elements: the input text sequence, and the input text sequence. X Entity category [entity_type] and entities to be extracted [candidate_span] For a text sequence X And templates generated by the automatic template generation module The constructed prompt is represented as: ； Step 2.2, set a hyperparameter—negative sampling rate. During the training phase, the number of negative samples input into the model is the total number of negative samples multiplied by 1. ; Step 2.3, input sequence After encoding by the Encoder module of the BART model, the hidden layer vector is obtained. As shown in the following formula: （3） In the formula, , The length of the text sequence after adding the prompt information is d, where d is the dimension of the hidden layer vector of the model. Step 2.4, for the token at time t The prediction will be the encoded vector of the Encoder. And the results of previous predictions Simultaneously, the Decoder module of the BART module is input, and an autoregressive method is used for prediction to obtain the last hidden layer vector output by the Decoder. .

2. The few-shot entity recognition method based on automatic prompting according to claim 1, characterized in that, The negative sampling rate .

3. The few-shot entity recognition method based on automatic prompt according to claim 2, characterized in that, Step 3 is implemented in the following steps: Step 3.1, after obtaining Encoding vector Then, select the first l tokens, connect them to the MLP layer, and obtain the vector. ; （4） In the formula, ; Step 3.2, introduce a separator. When multiple entities of the same type exist in the input text, this is used as a separator between entities of the same type. Calculate its token encoding Then , and terminator Hidden layer vectors The sequences are concatenated to obtain the final candidate sequence encoding vector. ; （5） （6） In the formula, This indicates that the vectors are concatenated along the first dimension. ; Step 3.3, Calculation and The token can be obtained by passing the dot product through a softmax layer. Subscript in the input sequence Calculate using the following formula (7) The conditional probability can be obtained after decoding. , as in equation (8); （7） In the formula, ; （8） Step 3.4: Training is performed using a forced instruction method, which involves concatenating the initial word and the original input sequence together as the input to the Decoder module; The loss function during the training phase is the negative log-likelihood loss function, as shown in equation (9); during the prediction phase, an autoregressive approach is used for sequential decoding, with the Decoder performing decoding at each time step. t The input is the previous time step. t -1 is used to predict the token, and decoding ends when a stop character is encountered. （9） In the formula, m This is the length of the output sequence of the Decoder module.

Images