Small sample scene named entity recognition method and system based on task-level knowledge migration and prompt learning

Abstract

The invention relates to the technical field of entity recognition, in particular to a small sample scene named entity recognition method and system based on task-level knowledge migration and prompt learning. Training a large model by using a soft prompt training data set to obtain named entity recognition task knowledge; the named entity recognition task knowledge serves as prompt information to be migrated to a target domain, to-be-classified entities contained in a target domain text are obtained through domain adaptation, and the target domain is a small sample domain with sparse entity labeling samples; for the to-be-classified entities, entity features are extracted, the to-be-classified entities are classified based on the entity features, and the entity features comprise character-level features, word-level features and statement-level features. According to the method, the advantages of transfer learning and prompt learning are combined, and the named entity recognition effect in a small sample scene can be effectively improved.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of entity recognition, in particular to a small sample scene named entity recognition method and system based on task-level knowledge transfer and prompt learning. BACKGROUND

[0002] With the rapid development of natural language processing technology, named entity recognition (NER) as one of the core tasks of information extraction has received extensive attention. Chinese NER aims to identify named entities (such as names, place names, organization names, etc.) from unstructured Chinese text and classify them into predefined categories. In practical applications, the distribution of NER labeled data is extremely uneven in different fields. A small number of fields have a large number of entity labeled samples, while the entity labeled samples of most fields are relatively scarce. These fields are also called small sample fields. This data imbalance leads to poor performance of NER models in data-scarce fields, making it difficult to accurately identify entities. Currently, the focus of small sample NER research is on how to better utilize existing auxiliary information to complete the NER task, mainly including three methods: 1. Methods based on external word library; 2. Methods based on transfer learning; 3. Methods based on pre-trained language models.

[0003] With the development of pre-trained language models, methods based on transfer learning and methods based on prompt learning have gradually become mainstream. However, due to the different predefined entity types of different domain datasets and significant semantic distribution differences, transfer learning methods have low accuracy on tail data. Due to the fact that the best text prompt for human understanding is not the most effective prompt for LLM, and the learned prompt information is insufficient due to the lack of training data, it is difficult to generate effective prompt information, resulting in inaccurate NER results in small sample fields. SUMMARY

[0004] The purpose of the present application is to solve the problems of difficult domain knowledge transfer and insufficient prompt training, and to provide a small sample scene named entity recognition method and system based on task-level knowledge transfer and prompt learning, which combines the advantages of transfer learning and prompt learning to improve the named entity recognition effect in small sample scenes.

[0005] According to the design scheme provided by the present application, on the one hand, a small sample scene named entity recognition method based on task-level knowledge transfer and prompt learning is provided, which comprises:

[0006] Based on the multi-domain entity labeled data, a soft prompt training dataset is constructed to train a large model using the soft prompt training dataset to obtain named entity recognition task knowledge;

[0007] Migrate the named entity recognition task knowledge as prompt information to a target domain, and obtain the to-be-classified entities contained in the target domain text through domain adaptation, the target domain being a small sample domain with sparse entity annotation samples;

[0008] For the to-be-classified entity, extract entity features and classify the to-be-classified entity based on the entity features, the entity features including character-level features, word-level features and sentence-level features.

[0009] As the small sample scene named entity recognition method based on task-level knowledge migration and prompt learning of the present application, further, a soft prompt training data set is constructed based on multi-domain entity annotation data, containing:

[0010] Obtain a plurality of domain entity type annotation data sets, convert each entity type annotation data into an entity annotation data set irrelevant to the entity type using a specified label, and add a domain number to each annotation sentence, the specified label including a label for identifying that a character is at the beginning of an entity, a label for identifying that a character is at the middle of an entity, and a label for identifying that a character is a non-entity character;

[0011] Mix the converted multi-domain entity annotation data sets to obtain a soft prompt training data set.

[0012] As the small sample scene named entity recognition method based on task-level knowledge migration and prompt learning of the present application, further, the converted multi-domain entity annotation data sets are mixed, containing:

[0013] Set the batch size n bs , and randomly select an integer k as the number of domains contained in one batch according to the number of domains in the data set;

[0014] Randomly select k domain data from the multi-domain entity annotation data set, and respectively extract annotation samples from the first k-1 domain data by non-replacement random sampling and randomly select annotation samples from the kth domain data according to the batch size, so that the annotation data of the selected k domains contained in the same batch training set has the same proportion;

[0015] If one of the domain annotation data is sampled, a domain is randomly selected from the remaining domains for sampling until the batch size is met, so as to obtain a soft prompt training data set by mixed sampling.

[0016] As the small sample scene named entity recognition method based on task-level knowledge migration and prompt learning of the present application, further, a large model is trained using the soft prompt training data set to obtain named entity recognition task knowledge, containing:

[0017] Decompose the soft prompt matrix into a domain shared matrix and a specific task low-rank matrix;

[0018] In the training process, the classification loss is calculated according to the field to which the data belongs, and the sum of the classification losses of each field is minimized as the optimization target, and the basic large model is trained using the soft prompt training data set, so that the field sharing matrix learns the named entity recognition task knowledge in the training process.

[0019] As the small sample scene named entity recognition method based on task-level knowledge transfer and prompt learning of the application, further, the target field text contains the entity to be classified, which is obtained by field adaptation, including:

[0020] Obtain the target field small sample data, and label the small sample data with character named entity start position, character named entity middle position and character not belonging to named entity label;

[0021] The soft prompt matrix is decomposed into a field sharing matrix and a low rank matrix, and the first learning rate and the second learning rate of the field sharing matrix and the low rank matrix are set respectively, and the first learning rate is less than the second learning rate;

[0022] The model is pre-trained using the target field small sample data, so that the field sharing matrix is fine-tuned with the first learning rate and keeps the named entity recognition task knowledge obtained by the soft prompt training data set, and the low rank matrix is fine-tuned with the second learning rate to adapt to the target field data;

[0023] The target field text sequence is taken as input, and the pre-trained field sharing matrix is taken as model prompt information to guide the model to output the entity to be classified contained in the text sequence.

[0024] As the small sample scene named entity recognition method based on task-level knowledge transfer and prompt learning of the application, further, the entity feature is extracted and the entity to be classified is classified based on the entity feature, which further includes:

[0025] The target field small sample data is enhanced by word replacement and sentence conversion, the word replacement is to replace the labeled entity in the text by the same type of entity or symbol, and the sentence conversion is to transform the original text sentence representation form;

[0026] An entity type classification network model is constructed, and a loss function of the entity type classification network model is set, the entity type classification network model includes a feature extraction unit, a feature fusion unit and a feature classification unit, and the loss function includes a contrastive learning loss term and a cross-entropy loss term, wherein in the contrastive learning loss term, two different dropout parameters are used to extract features from the same sentence twice to obtain two different representation vectors of the same entity, and the positive samples contained in the input sentence are processed based on InfoNCELoss;

[0027] Based on the loss function and using the enhanced target field small sample data, the entity type classification network model is trained to obtain an entity classification target model.

[0028] As the small sample scene named entity recognition method based on task-level knowledge transfer and prompt learning of the application, further, the entity features are extracted and the entity classification is performed based on the entity features, which comprises:

[0029] The language model, the bidirectional long short-term memory network and the convolutional neural network are used as the feature extraction units, and the sentence-level feature, the word-level feature and the character-level feature of the entity to be classified are extracted by the networks in the feature extraction units respectively.

[0030] The gated linear unit is used as the feature fusion unit to fuse the sentence-level feature, the word-level feature and the character-level feature by the feature fusion unit.

[0031] The Softmax layer is used as the feature classification unit, the fused features are input into the Softmax layer, and the feature vector is mapped to the judgment probability of the entity type by the Softmax layer, so that the type with the highest probability is output as the entity type of the entity to be classified.

[0032] In still another aspect, the application further provides a small sample scene named entity recognition system based on task-level knowledge transfer and prompt learning, comprising: a knowledge acquisition module, a domain adaptation module and an entity recognition module, wherein,

[0033] The knowledge acquisition module is used for constructing a soft prompt training data set based on multi-domain entity annotation data, so as to train a large model by using the soft prompt training data set to acquire named entity recognition task knowledge.

[0034] The domain adaptation module is used for transferring the named entity recognition task knowledge as prompt information to a target field, and obtaining the entity to be classified contained in the target field text by domain adaptation, wherein the target field is a small sample field with sparse entity annotation samples.

[0035] The entity recognition module is used for extracting entity features and classifying the entity to be classified based on the entity features, wherein the entity features include character-level features, word-level features and sentence-level features.

[0036] The application has the following beneficial effects:

[0037] The present application utilizes multi-field soft prompt fine-tuning technology, trains prompt information based on constructed multiple field-independent entity annotation data, to obtain field-independent NER task-level knowledge containing entity boundaries, thereby improving the quality of LLM prompts while effectively avoiding the impact of semantic differences between different field annotation samples; then, based on the obtained field-independent NER task-level knowledge, small sample field adaptation is completed, thereby improving the accuracy of small sample Chinese entity detection. The entity type discrimination based on multi-level semantic features is different from the existing method of only using sentence-level features for entity type discrimination. The character-level and word-level features of the entity to be classified are added based on the sentence-level features, and different level features are fused through a gating linear unit and a contrast learning mechanism to obtain the final representation of the entity, thereby enhancing the model's ability to discriminate entity types. Comparative experiments are carried out on public data sets, and the experimental data show that the performance of the present application is generally better than that of the existing comparative methods, further proving the effectiveness and stability of the present application in small sample scenarios. BRIEF DESCRIPTION OF DRAWINGS

[0038] Figure 1 A small sample scenario named entity recognition process based on task-level knowledge transfer and prompt learning in the embodiment is shown;

[0039] Figure 2 A small sample scenario named entity recognition algorithm framework in the embodiment is shown, (a) is an entity boundary detection process based on multi-field soft prompt fine-tuning, and (b) is an entity type determination process based on multi-level feature fusion;

[0040] Figure 3 Examples of full convolution and candidate word convolution in the embodiment are shown;

[0041] Figure 4 Weibo dataset F1 distribution changes in the embodiment are shown;

[0042] Figure 5 Resume dataset F1 distribution changes in the embodiment are shown;

[0043] Figure 6 Taobao dataset F1 distribution changes in the embodiment are shown;

[0044] Figure 7 Youku dataset F1 distribution changes in the embodiment are shown. DETAILED DESCRIPTION

[0045] To make the purpose, technical solutions and advantages of the present application clearer and more apparent, the present application will be further described in detail below with reference to the drawings and technical solutions.

[0046] In the Chinese named entity recognition task, the amount of annotation data in different fields differs greatly. When the annotation data is less, the existing methods mainly improve the named entity recognition performance through transfer learning or prompt learning. However, the transfer learning-based method has poor inter-domain knowledge transfer effect, and the prompt learning-based method is difficult to generate effective prompt information, resulting in inaccurate named entity recognition results in small sample fields. Embodiments of the present application provide a small sample scene named entity recognition method based on task-level knowledge transfer and prompt learning, as shown in Figure 1 The specific contents include the following:

[0047] S101, a soft prompt training data set is constructed based on multi-domain entity annotation data, so as to train a large model by using the soft prompt training data set to obtain named entity recognition task knowledge.

[0048] S102, the named entity recognition task knowledge is migrated to a target field as prompt information, and the target field is a small sample field with sparse entity annotation samples, and the target field is a target field.

[0049] S103, for the classified entity, entity features are extracted and the classified entity is classified based on the entity features, and the entity features include character-level features, word-level features and sentence-level features.

[0050] As shown in Figure 2 , in order to solve the problems of difficult domain knowledge transfer and insufficient prompt training, the small sample Chinese named entity recognition ET&HFF (Entity Tagging and Hierarchical Feature Fusion for NER) algorithm framework based on task knowledge transfer in the present case is used to hide the entity types of the annotation data in different field data sets and uniformly identify them as entities, so as to realize the conversion of entity type annotation data to type-independent entity annotation data and alleviate the influence of the semantic distribution difference of annotation data in different fields. Based on the constructed type-independent entity annotation data, the NER task-level related knowledge is obtained through multi-domain soft prompt fine-tuning technology, the task-level knowledge is transferred to the small sample field, the small sample field is adapted, and the classified entity contained in the text in the small sample field is obtained. Then, the entity type is determined based on the character-level, word-level and sentence-level features of the classified entity. The main steps can be summarized as follows:

[0051] Step 1: Multi-domain fine-tuning training data set construction. First, convert the head domain entity type annotation data set into a type-independent entity annotation data set, denoted as D' head ={D'1,…,D' i ,…,D' n}, and then mix the training data of multiple fields to obtain a multi-domain soft prompt fine-tuning training data set D'train .

[0052] wherein, by acquiring a plurality of field entity type annotation data sets, each entity type annotation data is converted into an entity annotation data set irrelevant to the entity type by using a specified label, the specified label including a label for identifying that a character is at the beginning of an entity, a label for identifying that a character is at the middle position, and a label for identifying that a character is a non-entity character; and a field number is added to each annotation sentence, the converted multi-field entity annotation data set is mixed to obtain a soft prompt training data set.

[0053] The commonly used sample mixing strategy can be used to complete the mixing of the NER head multi-field annotation samples. First, the entity type annotation data of the head field is uniformly replaced with three labels of B (Begin), I (Inside) and O (Outside) to represent that the corresponding character is at the beginning, middle and non-entity character of the entity, respectively, and a field number is added to each annotation sentence to distinguish the field to which each belongs, thereby obtaining a type-independent entity annotation data set D' head ={D'1, D'2, D'3, …, D' n} n}, n represents the number of different field types for training, and then the data sets in D' head are mixed according to the batch size.

[0054] Specifically, the batch size batch-size can be set to n bs , and an integer k is randomly selected as the number of fields contained in one batch according to the number of fields in the data set; k field data are randomly selected from the multi-field entity annotation data set, and annotation samples are extracted from the first k-1 field data by non-replacement random sampling and randomly selected from the kth field data according to the batch size, so that the annotation data of the selected k fields contained in the same batch training set has the same proportion; if the annotation data of one field is all sampled, a field is randomly selected from the remaining fields for sampling, until the batch size is met, so as to obtain a soft prompt training data set by mixed sampling.

[0055] An integer k is randomly selected from [2, n] as the number of fields contained in one batch of data, k fields are randomly selected from D' head , and annotation samples are extracted from the first k-1 fields by non-replacement random sampling , and annotated samples, so as to make the proportion of the selected k domains in the same batch of training set as possible. If the annotated samples of a certain domain are all sampled, a domain is randomly selected from the remaining domains for sampling until the batch size is met, thereby obtaining a multi-domain soft prompt fine-tuning training data set D' train .

[0056] Step 2: Multi-domain soft prompt fine-tuning. Based on the training data set D' train , the prompt information of the LLM is trained by using the multi-domain soft prompt fine-tuning technology where ⊙ represents the Hadamard product between matrices, P * represents the prompt matrix shared between multiple domains, W i represents the prompt matrix specific to the i-th domain, W i can be decomposed into the product of two low-rank matrices (rank 1) u i and v i .

[0057] The different domains of the Chinese named entity recognition task are regarded as different task types in multi-task learning, and the domain-independent NER task-level knowledge is learned from D' train by using the multi-task soft prompt fine-tuning technology. In multi-domain soft prompt fine-tuning, the soft prompt matrix is composed of the Hadamard product of two matrices, as shown in formula 1:

[0058]

[0059] where the prompt matrix P * is shared when D' train , and the low-rank matrix W i is specific to the i-th domain. Let the rank of the matrix W i be 1, then W i can be further decomposed into

[0060] In the embodiments of the present case, the named entity recognition task knowledge is obtained by training the large model using the soft prompt training data set, which can be designed to include:

[0061] The soft prompt matrix is decomposed into a domain-shared matrix and a task-specific low-rank matrix;

[0062] During training, the classification loss is calculated according to the domain to which the data belongs, and the sum of the classification losses of each domain is minimized as the optimization goal. The base large model is trained using the soft prompt training data set, so that the domain-shared matrix learns the named entity recognition task knowledge during the training process.

[0063] During training, the multi-domain training data set D' trainShared prompt matrix P * NER field general knowledge can be learned, that is, field-independent NER task-level knowledge, such as which entities are of interest, and which characters imply more entity boundary information. And the low-rank matrix W i specific information for the domain D i can be embedded in the respective low-rank space.

[0064] The idea of low-rank matrix decomposition is derived from intrinsic dimension theory, which means that although the parameter space of a deep neural network is a high-dimensional space with a large scale, there is a small intrinsic dimension that determines a low-dimensional subspace that is core effective. In other words, although the number of parameters of a deep neural network is large, only a small part of the parameters plays a key role in the performance of the model. This theory also explains why large language models with a large number of parameters can exhibit good small sample learning ability and are widely used in pre-training language model fine-tuning related research.

[0065] Multi-domain soft prompt fine-tuning applies the concept of intrinsic dimension to prompt learning. A good performance soft prompt matrix also contains a core effective low-dimensional subspace. By decomposing the soft prompt matrix into a domain-shared matrix and a task-specific low-rank matrix the core effective low-dimensional subspace W i is separated out, and the NER task-level knowledge that is independent of the domain is embedded in the domain-shared matrix P * , which is applied to the tail domain data, which is equivalent to initializing the soft prompt parameters. *

[0066] Step 3: Tail domain adaptation. Apply P * to the tail domain, based on the labeled data of the tail domain, to train the prompt matrix to complete the adaptation of the model in the tail domain. , which is the prompt matrix for entity boundary detection in the tail domain.

[0067] Specifically, the target domain small sample data can be labeled to contain character named entity start position, character named entity middle position, and character not belonging to named entity; the soft prompt matrix is decomposed into a domain-shared matrix and a low-rank matrix, and the first learning rate and the second learning rate of the domain-shared matrix and the low-rank matrix are set respectively, and the first learning rate is less than the second learning rate; the model is pre-trained using the target domain small sample data, the domain-shared matrix is fine-tuned at the first learning rate and keeps the named entity recognition task knowledge obtained from the soft prompt training data set, and the low-rank matrix is fine-tuned at the second learning rate to adapt to the target domain data. ​

[0068] The adaptation process in the tail domain and the fine-tuning process in the head domain are similar, and can be represented by formula 1, which includes the domain-shared matrix P * Applied to the tail domain, the goal is to accurately predict the entity boundary of the tail domain text, and train the prompt matrix For the training set Where y i ∈{B,I,O} represents the label of character x i , B, I, and O represent the beginning of the named entity, the middle of the named entity, and the character x i that does not belong to the named entity, and the loss function of the tail adaptation process is shown in formula 2:

[0069]

[0070] Where θ represents the parameters of the pre-trained language model, which is frozen during the training process, represents the prompt matrix, which can be decomposed into a domain-shared matrix P * and a low-rank matrix W t specific to the tail domain.

[0071] In the adaptation process in the tail domain, the domain-shared matrix P * and the low-rank matrix W t are optimized with different learning rates. P * is fine-tuned with a small learning rate to maintain the domain-independent NER task-level knowledge obtained on the head domain dataset D′ train ; while the low-rank matrix W t is fine-tuned with a large learning rate to quickly adapt to the unique information of the tail domain fine-tuning dataset D tail . Through the tail domain adaptation, the model prompt information suitable for the entity boundary determination in the small sample domain of the tail domain can be obtained

[0072] Step 4: Entity boundary detection in the tail domain. For the input sequence in the tail domain, use the prompt matrix as the prompt information of the LLM to guide the LLM to output the entity t contained in the sentence x Where x is a substring in the sentence x t .

[0073] The target domain text sequence is input, and the domain-shared matrix obtained by pre-training is used as the model prompt information to guide the model to output the classified entity contained in the text sequence.

[0074] Specifically, the cue matrix adapted to the tail domain. As a hint for LLM, for the input sequence of the tail region Guide the LLM output statement x t Entities to be classified included This process is similar to the question-and-answer process in LLM, the difference being that the prompts in this process are provided by a matrix. Provided, rather than natural language text.

[0075] Step 5: Multi-feature extraction. For the input statement x t Entities awaiting classification The sentence-level features h are obtained using PLM, BiLSTM, and CNN respectively. s Word-level features h w and character-level features h c .

[0076] The pre-trained language model (PLM), bidirectional long short-term memory network (BiLSTM), and convolutional neural network (CNN) are used to extract the entities to be classified, respectively. Statement-level features h s Word-level features h w and character-level features h c .

[0077] When extracting word-level and character-level features, the character embeddings in the word embedding lookup table of the language model used when extracting sentence-level features are used as the initial word vectors of the input sentence. This ensures that the extraction of features at different levels is based on the same basic features and maintains the consistency of the expression basis of features at different levels.

[0078] 1. Statement-level feature extraction

[0079] Pre-trained language models, through pre-training on large-scale text data, can obtain distributed representation vectors containing semantic information. Existing research shows that bidirectional language models (such as BERT) of the same scale often outperform unidirectional language models (such as GPT) in solving natural language understanding tasks such as named entity recognition and entity relation extraction. Therefore, in this embodiment, the bidirectional language model BERT is used as a sentence-level feature extractor.

[0080] Each layer of Bert outputs the hidden states of each character. The hidden states of the last layer are directly selected as the sentence-level feature vectors of each character, and the average of the hidden state vectors of the starting and ending characters of the entity to be classified is used as the sentence-level feature vector of the entity to be classified, that is where h t and h e respectively represent the hidden state vectors of the starting and ending characters of the entity to be classified .

[0081] 2. Word-level feature extraction

[0082] LSTM can effectively extract word-level features in the text sequence. Due to the sequential processing characteristics inherent in its design, LSTM can only understand the text from one direction and cannot take into account the influence of the text before and after the word on it. BiLSTM processes from both the forward and backward directions respectively and can capture information in both directions of the text sequence. Therefore, BiLSTM is used as the word-level feature extractor in the embodiments of this case.

[0083] BiLSTM consists of two parts: a forward LSTM and a backward LSTM. The embedding vector of BiLSTM is composed of the embedding vectors of the LSTM in both directions. Let and respectively represent the embedding vectors of the starting character of the entity to be classified in the forward direction and the reverse direction, then the embedding vector of the starting character of the entity to be classified can be expressed as Similarly, the embedding vector of the ending character of the entity to be classified can be expressed as

[0084] The average of the embedding vectors of the starting and ending characters of the entity to be classified is used as its word-level feature vector, that is

[0085] 3. Character-level feature extraction

[0086] A significant difference between Chinese and English is that Chinese is a ideographic language, and a single character can also be an independent semantic unit. Moreover, a single Chinese character often contains multiple semantic information, and the character semantics are significantly affected by the context. For example, "和" forms an adjective in "和谐", acts as a verb in "和面", and serves as a conjunction in "我和你". Therefore, extracting character-level features plays an important role in deeply understanding Chinese semantics.

[0087] To effectively extract the character-level features of the text, firstly, a suitable network result is determined as a feature extractor. Existing researches show that CNN is good at capturing character-level information, and RNN is good at capturing word-level information. Therefore, in the embodiment of the present case, CNN is used as a character-level feature extractor.

[0088] Unlike the traditional full convolution from the first character to the last character of the data sentence, since the boundary information of the entity has been obtained in the entity boundary detection part, when extracting the character-level features, the convolution kernel only needs to cover at least one character within the entity boundary in the sliding convolution process. As shown in Figure 3 , examples of full convolution and convolution for candidate entities are shown, the green characters (C3C4C5) are the detected entities to be classified, the length of the convolution kernel is 3, and the convolution step is 1. When full convolution, the convolution kernel needs to move 7 times, and when convolution for candidate entities, the convolution kernel only needs to move 4 times in the sliding convolution process at least one character in C3C4C5.

[0089] When performing character-level feature extraction, k convolution kernels of different lengths are selected, and a sliding window mechanism is used, with a convolution step of 1, to perform convolution operations around the text sequence to be classified. Using convolution kernels of different lengths can obtain the relationship between characters in different ranges, that is, what different information will be brought by the occurrence of different ranges of words in a sentence. Similar to the statistical language model N-gram algorithm, different N values are set to consider different context lengths.

[0090] For each length of the convolution kernel, t convolution kernels are repeatedly used for convolution operation. Multiple convolution kernels of the same length are used to learn complementary features from the same length of the convolution kernel, similar to the multi-head attention mechanism in the Transformer, each convolution kernel extracts features from different angles, and different convolution kernels can learn information from different perspectives. Since a convolution kernel only obtains a scalar feature value after sliding convolution and pooling operation, as shown in Figure 3 , k convolution kernels of different lengths are repeated t times, which can obtain kt scalar feature values, which are spliced into a one-dimensional vector as the character-level features h of the entity to be classified c .

[0091] Feature 6: Feature fusion. The three-level feature vectors are spliced [h c : h w : h s ] as the feature vector for subsequent entity type determination Based on the feature vector The three-level feature vectors are comprehensively utilized by a Gated Linear Unit (GLU), and a contrast learning mechanism is used to improve the distinguishing ability of the model to different types of entity features, to obtain an entity to be classified an unnormalized category score vector of the entity to be classified

[0092] The character-level feature h c , the word-level feature h w , and the sentence-level feature h s of the entity to be classified are concatenated into a one-dimensional vector as a feature vector for subsequent entity type determination. To further improve the category distinguishing ability of the model in a small sample scenario, in the embodiments of the present case, the Gated Linear Unit (GLU) is used to fuse the three-level feature vectors, and the contrast learning mechanism is used to further enhance the feature distinguishing degree between different types of entities, so that the feature vector can better reflect the differences between different types of entities. The feature representation based on contrast learning and the feature fusion based on Gated Linear Unit are as follows:

[0093] 1. Feature fusion based on contrast learning

[0094] The basic idea of contrast learning is to pull the distance between the same type of samples closer and the distance between different types of samples farther apart in the feature space to enhance the expression ability of the model to sample features. In the NER task, entities of the same type are positive samples to each other, while entities of different types are negative samples to each other. In special cases, when there is only one entity of a certain type in a training batch, contrast learning cannot be performed due to the lack of positive sample pairs. To solve this problem, in the embodiments of the present case, two different dropout parameters are used to extract features from the same sentence twice in the feature extraction layer, thereby obtaining two different representation vectors of the same entity, which are positive samples to each other. An extended form of InfoNCELoss is used as the loss function of contrast learning to handle multiple positive samples that may be contained in the input sentence. For entity type t, one of the entities is randomly selected as an anchor sample, and other samples of the same type as the anchor sample are considered as positive samples, and other samples of different types from the anchor sample are considered as negative samples. The specific form is shown in formula 3:

[0095]

[0096] where N represents the total number of positive and negative samples in the input sentence, M represents the number of positive samples in the input sentence, z i represents the feature vector of the entity to be classified, z j represents the feature vector of the jth entity in the input sentence, the feature vector of the kth positive sample representing the entity to be classified, sim(z i , z j ) represents the cosine similarity between the anchor entity and the jth entity τ is a temperature parameter. The total loss function of the contrastive learning is the sum of all types of loss functions, i.e., Loss cs =∑ t Loss t .

[0097] 2. Feature fusion based on gated linear unit

[0098] GLU is a gating mechanism composed of a gating function and a linear mapping layer, which can effectively fuse information at different levels, filter useless information and only retain key data to improve computational efficiency and expression ability. Its general form is shown in equation 4:

[0099] GLU(x)=σ(Wx+b) ⊙ (Vx+c) (4)

[0100] Wherein, σ() represents an activation function, W and V represent the weight matrices of the two linear mapping layers, b and c represent the bias vectors of the linear mapping, and ⊙ represents the Hadamard product of the matrix.

[0101] Step 7: Type determination. The class score vector of the entity is normalized by using the Softmax function, so that the type determination result of the entity is obtained

[0102]

[0103] Wherein, before classifying the entity to be classified, it also contains:

[0104] Data augmentation is performed on the target field small sample data by word replacement and sentence conversion, the word replacement is to replace the labeled entity in the text by the same type entity or symbol, and the sentence conversion is to transform the original text sentence representation form;

[0105] An entity type classification network model is constructed, and a loss function of the entity type classification network model is set, the entity type classification network model includes a feature extraction unit, a feature fusion unit and a feature classification unit, and the loss function includes a contrastive learning loss term and a cross-entropy loss term, wherein the contrastive learning loss term obtains two different representation vectors of the same entity by using two different dropout parameters to extract features from the same sentence twice, and sets the positive samples contained in the input sentence based on InfoNCELoss processing;

[0106] ​​Train the entity type classification network model based on the loss function and using the enhanced target field small sample data to obtain an entity classification target model.

[0107] In order to more effectively train the classifier parameters, before training the classification model, the labeled data of the small sample is enhanced by adopting word replacement and sentence conversion.

[0108] Since the entity type classification network needs a certain amount of labeled data for training, when the labeled sample is insufficient, the classification model is not fully trained and it is difficult to achieve the best effect. In order to fully train the model, when the labeled sample is too small, the labeled data can be expanded by two simple data enhancement methods of word replacement and sentence conversion. Word replacement is to replace the labeled entity in the original text with entities of the same type or type symbols, and sentence conversion is to change the form of expression of the original text to increase the diversity of sentence patterns.

[0109] Taking the labeled text "He was diagnosed with epilepsy in West China Hospital" in the medical field as an example, "West China Hospital" belongs to the entity of medical institution type (ORG), and "epilepsy" belongs to the entity of disease type (DIS):

[0110] Through word replacement, "West China Hospital" can be replaced with other entities of the same medical institution type, such as "Union Hospital", "Shanghai East Hospital", etc., or the special symbol "[ORG]"; "epilepsy" can be replaced with other entities of the same disease type, such as "gastric ulcer", "dysentery", etc., or the special symbol "[DIS]". The special symbols "[ORG]" and "[DIS]" are special tokens added to enhance the model's ability to distinguish entity types and understand the context of the text, similar to the "[MASK]" symbol used in the Bert training process, which is an indivisible whole. In the hierarchical feature extraction process, its representation vector is initialized as the average value of the representation vectors of the replaced entity characters.

[0111] Through sentence conversion, "West China Hospital diagnosed his epilepsy" or "epilepsy is what he was diagnosed with in West China Hospital" can be obtained, and the text after sentence conversion still contains all the information in the original text.

[0112] The entity type determination is realized by appending a Softmax layer after the gated linear unit, and the output vector of the gated linear unit is mapped to the determination probability z of the entity type, i.e. z = Softmax(GLU(x)), where GLU(x) is determined by formula 4. The loss function of type determination is cross-entropy loss function, and the specific form is shown in formula 5:

[0113]

[0114] where N represents the number of entity types, yi represents the true label of the i-th entity type, represents the predicted probability of the i-th entity type.

[0115] The trained model can predict the probability of the entity type to which the entity to be classified belongs according to the features of the entity to be classified , and take the type with the highest predicted probability as the entity type of the entity to be classified.

[0116] In order to avoid the distribution deviation and error propagation problems that may be caused by the staged training, the model is optimized in an end-to-end manner during the training process, and the total loss function of the classification network model training is the weighted sum of the contrastive learning loss and the cross-entropy loss, as shown in formula 6:

[0117] Loss total = λLoss cs + (1-λ)Loss ce (5)

[0118] Wherein, the hyperparameter λ∈[0,1] is the weight coefficient of the contrastive learning loss and the cross-entropy loss.

[0119] Further, based on the above method, the embodiment of the application also provides a small sample scene named entity recognition system based on task-level knowledge transfer and prompt learning, comprising: a knowledge acquisition module, a domain adaptation module and an entity recognition module, wherein,

[0120] The knowledge acquisition module is used to construct a soft prompt training data set based on multi-domain entity annotation data, so as to train a large model by using the soft prompt training data set to acquire named entity recognition task knowledge;

[0121] The domain adaptation module is used to transfer the named entity recognition task knowledge as prompt information to a target domain, and acquires the entity to be classified contained in the target domain text through domain adaptation, wherein the target domain is a small sample domain with sparse entity annotation samples;

[0122] The entity recognition module is used to extract entity features and classify the entity to be classified based on the entity features, wherein the entity features include character-level features, word-level features and sentence-level features.

[0123] In order to verify the effectiveness of the scheme, the following experimental data are used for further explanation and description:

[0124] ​​​The experiment includes three parts: entity boundary detection, entity type determination, and named entity recognition comparison experiment, which are used to verify the effectiveness of multi-domain soft prompt fine-tuning in entity boundary detection, the effectiveness of entity type determination based on multi-level semantic features, and the overall effectiveness of the proposed scheme in small sample NER task.

[0125] The named entity comparison experiment consists of entity boundary detection and entity type determination, and the experimental settings are the same as the entity boundary detection experiment and the entity type determination experiment. All experiments were performed on an Ubuntu 22.04.4 LTS system equipped with a 32GB memory NVIDIA Quadro GV100.

[0126] In the entity boundary detection experiment, the model DeepSeek-R1-Distill-Qwen-7B was used for multi-domain soft prompt fine-tuning, and the soft prompt vector length was set to 100, i.e. where d represents the word vector dimension. In the adaptation process of the tail domain data, the shared matrix in the multi-domain soft prompt fine-tuning process was reused, and the low-rank matrix specific to the tail domain was initialized as the average of the low-rank matrices specific to each training domain in the multi-domain soft prompt learning stage.

[0127] In the entity type determination experiment, the bert-wwn-ext model was selected as the pre-trained language model to generate sentence-level features, and the character embeddings in the word embedding query table in the bert-wwm-ext model were used as the initialization word vectors of the input sentence. Then, word-level feature extraction and sentence-level feature extraction were performed. Since the experiment randomly selects k samples to simulate a small sample scenario, to reduce the bias that the randomness of sample selection may cause to the experimental results, the experiment repeats the random sampling 5 times for each sample size k, and the average of the 5 results is taken as the final result. When the sample size k < 500, the data augmentation strategy is used to expand the labeled samples to rank 500, and when the sample size k > 500, no data augmentation is performed.

[0128] The entity boundary detection experiment and the entity type determination experiment both use the Adam optimizer for training and optimization. Table 1 lists the related parameter settings in the experiment.

[0129] Table 1 Hyperparameter settings in the experiment

[0130]

[0131]

[0132] The data set includes a data set for multi-domain soft prompt fine-tuning in the entity boundary detection stage, i.e., a head data set with sufficient labeled data, and a test data set for verifying the effectiveness of the method, i.e., a data set for simulating a small sample scenario.

[0133] The training data set used in the entity boundary detection experiment of the multi-domain soft prompt fine-tuning includes four domains: the bank lending domain data set Bank, the Chinese literature domain data set Literature, the Chinese medical domain data set CMeEE-V2, and the science domain data set SciCN. Table 2 shows the relevant statistical information of the data set used for training in the entity boundary detection stage.

[0134] Table 2 Training domain data set statistical information

[0135]

[0136] To verify the effectiveness of entity boundary detection and named entity recognition, the experiment selects four data sets Weibo, Resume, Taobao, and Youku to simulate small sample scenarios by randomly sampling k = 250, k = 500, k = 1000, and k = 1350 labeled samples to verify the effectiveness of the scheme. Table 3 shows the relevant statistical information of the data set for verifying the effectiveness of entity boundary detection and named entity recognition.

[0137] Table 3 Test domain data set statistical information

[0138]

[0139]

[0140] Entity boundary detection is the basis of named entity recognition, and it is important to ensure that the entity boundary detection part has a high recall rate as much as possible to obtain the named entities contained in the sentence. To verify the effectiveness of multi-domain soft prompt fine-tuning in the entity boundary detection stage, the experiment performs entity boundary detection experiments on the Weibo, Resume, Taobo, and Youku four test sets, and calculates the recall rate of the detection results Recall rate The calculation method is as follows:

[0141]

[0142] where N ec represents the number of named entities correctly detected in the entity boundary detection stage, and N ea represents the number of named entities actually contained in the test sentence.

[0143] The experimental results are shown in Tables 4 and 5.

[0144] Table 4 Entity boundary detection statistical results of Weibo and Resume dataset

[0145]

[0146] Table 5 Entity boundary detection statistical results of Taobao and Youku dataset

[0147]

[0148] As can be seen from Table 4 and Table 5, the recall rate of entity boundary detection based on multi-domain soft prompt fine-tuning Except for Weibo training sample size k = 250, which is slightly lower than 80% All are higher than 80%, and the recall rate on the four datasets The average value is 89.17%. The recall rate on the Resume dataset performs best, all above 96%; the recall rate on the Weibo dataset Lower, which is due to the more standardized expression of Resume text from the resumes of listed company executives, and the stronger pattern of entity position. While in the social network platform Weibo, the text expression form and content are more flexible and diverse, making the entity boundary detection more difficult, resulting in a relatively low detection F1 value.

[0149] The experimental results show the effectiveness of multi-domain soft prompt fine-tuning in guiding large language models to complete small sample tasks, which is due to the method making full use of the large amount of labeled data in different domains of the same task, and fully training the prompt vector, making the prompt vector better reflect the task requirements, and more fully tapping the application potential of large language models in vertical tasks.

[0150] The entity type determination experiment includes entity type determination effectiveness experiment and entity type determination ablation experiment. The former is used to verify the effectiveness of the entity type determination method based on hierarchical features, and the latter is used to verify the effectiveness of the character-level, word-level, semantic-level and gated linear unit models in the entity type determination method.

[0151] 1) Entity type determination effectiveness experiment

[0152] To evaluate the accuracy of the entity type determination part, the type determination accuracy of the entity determination results of the method proposed in this case relative to the entity boundary detection results Accuracy The calculation formula is shown in formula 8:

[0153]

[0154] Where, N tcN, the number of named entities of the correct determination type da N, the number of named entities of the correct determination type This is because the core task of the entity type determination part is the same, and the core task is to accurately determine the entity type of the obtained entity to be classified, thereby ensuring the consistency of the accuracy evaluation index and the task requirements.

[0155] The experimental results are shown in Tables 6 and 7.

[0156] Table 6 Type determination accuracy statistics of Weibo and Resume

[0157]

[0158] Table 7 Type determination accuracy statistics of Taobao and Youku

[0159]

[0160] From Tables 6 and 7, it can be seen that:

[0161] (1) The entity type determination accuracy on the Weibo, Resume, Taobao, and Youku test data sets is all above 84%, and the determination accuracy on the Resume data is as high as 97.69%, which shows the effectiveness of the entity type determination method based on hierarchical features.

[0162] (2) When the training sample size k = 250, the determination accuracy on each test data set reaches more than 84%, and with the increase of the training sample size k (from 250 to 1350), the accuracy does not show a stable upward trend, which shows that the model has better adapted to the small sample environment, and the fluctuation of the accuracy may be affected by the entity boundary detection results under different data sets. At the same time, due to the fewer parameters of the entity type determination model, the risk of sample size close to saturation or overfitting may also cause performance fluctuations.

[0163] (3) The entity type determination accuracy of high-structured data (Resume / Youku) is higher than 90%, especially the Resume dataset is more than 97%, which is significantly higher than the low-structured data (Weibo / Taobao). This is mainly because the Resume is from the resumes of listed company executives, the text format is very standard, and the expression form is very unified. Youku performs better than Weibo and Taobao, which may be because the content of Youku is user-generated, but the expression of entities around the video theme (such as film and television names, actors) is relatively single, the format is relatively standard, and the entity type is less. The open topics in Weibo and Taobao fields lead to more diverse text topics and more flexible expression forms, and there are more informal expressions or noises (such as network language, abbreviation, meaningless character), which increases the difficulty of entity type identification.

[0164] 2) Entity type determination ablation experiment

[0165] Ablation experiments are performed by removing specific features to verify the effectiveness of each module in the entity type determination method. "-char" means removing character-level features, "-word" means removing word-level features, "-char&word" means removing both character-level and word-level features, "-sent" means removing sentence-level features, and "-GLU" means removing the gate linear unit, and only through the Softmax network layer to complete the entity type classification task. The F1 value of the type determination result is recorded, as shown in Tables 8 and 9.

[0166] Table 8 Ablation experiment results of entity type determination of Weibo and Resume datasets

[0167]

[0168] Table 9 Ablation experiment results of entity type determination of Taobao and Youku datasets

[0169]

[0170]

[0171] As can be seen from Tables 8 and 9:

[0172] (1) On Weibo, Resume, Taobao, and Youku datasets, the average F1 value decreases by 2.00%, 0.27%, 5.12%, and 5.08% respectively after removing the character-level features (“-char”); the average F1 value decreases by 1.93%, 0.26%, 5.20%, and 5.09% respectively after removing the word-level features (“-word”); the average F1 value decreases by 4.65%, 0.69%, 7.99%, and 8.30% respectively after removing both the character-level features and the word-level features (“-char&word”); the average F1 value decreases by 5.94%, 0.98%, 9.20%, and 9.24% respectively after removing the sentence-level features (“-sent”); and the average F1 value decreases by 5.30%, 0.85%, 8.44%, and 8.89% respectively after removing the gating linear unit (“-GLU”). This indicates the effectiveness of character-level, word-level, sentence-level, and gating linear unit modules in determining entity types.

[0173] (2) The average F1 value decreases by 3.96%, 0.61%, 7.19%, and 7.32% respectively on Weibo, Resume, Taobao, and Youku datasets using each test method. Among them, the average F1 value decreases the least on the Resume dataset, which may be due to the single semantic text, standard expression, and easier entity type inference of the Resume dataset.

[0174] (3) On Weibo, Resume, Taobao, and Youku datasets, removing both the character-level features and the word-level features (“-char&word”) compared to removing only the character-level features (“-char”) or only the word-level features (“-word”) results in a more significant decrease in F1 value. This may be because the initial vector of the character-level features and the word-level features comes from the same embedding layer of the language model, and the initial semantic information is consistent, making the final semantic expression of the two have high similarity. Therefore, removing one of them has little effect on the F1 value, but after removing both, the model cannot obtain more fine-grained semantic information, resulting in a significant decrease in F1 value.

[0175] 3) Comparative experiment of named entity recognition

[0176] To verify the effectiveness of the method proposed in this case in the field of small sample named entity recognition, nine classic named entity recognition methods are selected as comparative methods to verify the effectiveness of the method proposed in this case.

[0177] Since the test data is generated by random sampling, the experimental results of the comparative methods are obtained by re-running the corresponding model. In order to reduce the deviation of the experimental results caused by sampling randomness, in the experiment, for each sample capacity k, 5 times of random sampling are repeated, and the average value of the 5 results is taken as the final result.

[0178] The F1 value commonly used in NER field research is used to evaluate the performance of entity boundary detection and entity type determination. The method proposed in this case is compared with 9 kinds of comparative algorithms on Weibo, Resume, Taobao and Youku four test sets. The F1 value of each test method on each data set is counted respectively, and the calculation of F1 value is shown in formula 9:

[0179]

[0180] Among them, #word gold represents the number of real Chinese named entities; #word method represents the number of named entities recognized by the method, #word gold∩method The meaning of #word

[0181] The results of the comparative experiment of named entity recognition are shown in Tables 10 and 11, Figures 4-7 showing the distribution of F1 values of each method on Weibo, Resume, Taobao and Youku four data sets.

[0182] Table 10: Test results of Weibo and Resume

[0183]

[0184] Table 11: Test results of Taobao and Youku

[0185]

[0186]

[0187] From Tables 10, 11 and Figures 4-7 It can be seen that:

[0188] (1) In the Weibo, Resume, Taobao, and Youku datasets, the F1 value of the method proposed in the case reaches or approaches the optimal level, and the average F1 value is increased by at least 1.05%, 0.59%, 0.86%, and 0.37%, respectively, indicating the effectiveness of the method proposed in the case. This is because the soft prompt is fully fine-tuned using multi-domain type-independent labeled samples, improving the recall rate of entity boundary detection, and accurately determining the entity type based on hierarchical features. In the Weibo dataset, when k = 1350, it is slightly lower than the optimal result by 0.18%; in the Youku dataset, when k = 1000, it is slightly lower than the optimal result by 0.08%, which may be due to when the training data is relatively large (≥1000), the C-BERT module in PCBERT and the span detection module of the proposed model using contrastive and prompt learning can achieve better optimization effect in specific datasets.

[0189] (2) The Resume dataset is more regular in form and has more standardized language, usually following a fixed template (such as "time-position-company"), and the entity boundary and type are more explicit, so under the same test conditions, the F1 value is more than 10% higher than the other three datasets, and the method proposed in the case can achieve good results on all datasets, indicating that multi-domain samples can help the model better generalize to entities with different forms of expression, and can adapt to both standardized texts and texts with relatively free forms of expression.

[0190] (3) As k increases from 250 to 1350, the F1 value of the method proposed in the case increases from 67.81% to 74.64 in the Weibo dataset, with a significant increase (about 6.83%), indicating that when the data format is not standardized, the model is more sensitive to the amount of training data, and more data can bring better fine-tuning effect. In the Resume dataset, the increase of k has limited effect on performance improvement (from 95.07% to 96.12%), indicating that when the data format is standardized, the model can quickly converge on a small amount of data, and the impact of training data quantity on model performance is reduced, so for models with fewer parameters, the quality of training data may be more important than the quantity.

[0191] From the above experimental results, the scheme of the case obtains NER task-level knowledge based on type-independent head domain entity annotation data through multi-domain soft prompt fine-tuning technology, migrates it to small sample domains, and can accurately obtain the classified entities, effectively alleviating the influence of semantic distribution differences between different domains during knowledge migration, and based on character-level, word-level, and sentence-level three-level semantic features, the entity type can be accurately determined, which has good effectiveness and stability in small sample scenarios.

[0192] The relative arrangement of components and steps, numerical expressions, and numerical values set forth in the Examples are not intended to limit the scope of the application unless specifically stated otherwise.

[0193] The various embodiments described in this specification are presented by way of example, and each embodiment is not necessarily composed of all features described with respect to other embodiments. The systems disclosed in the embodiments are not necessarily limited to the methods disclosed in the embodiments to which they are correspondingly described.

[0194] The units and method steps of the examples described in connection with the embodiments disclosed herein can be implemented in electronic hardware, computer software, or a combination of both. The components and steps of the examples have been described in general terms in the above description so that the exemplary implementations can be better understood. Whether such functionality is implemented in hardware or software depends on the particular application and design constraints imposed on the overall system. Skilled artisans can implement the described functionality in varying ways for each particular application, but such implementation decisions should not be interpreted as causing a departure from the scope of the present application.

[0195] Those skilled in the art can understand that all or part of the steps in the above method can be instructed by a program to relevant hardware, and the program can be stored in a computer readable storage medium, such as a read-only memory, a magnetic disk or an optical disk. Alternatively, all or part of the steps of the above embodiments can also be implemented using one or more integrated circuits, and accordingly, each module / unit in the above embodiments can be implemented in the form of hardware or in the form of a software function module. The present application is not limited to any specific form of combination of hardware and software.

[0196] Finally, it should be noted that the above-described embodiments are merely specific implementations of the present application, which are used to illustrate the technical solutions of the present application, and are not limiting. The protection scope of the present application is not limited thereto. Although the present application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that any person skilled in the art can make modifications or easily think of changes to the technical solutions described in the foregoing embodiments within the technical scope disclosed by the present application, or make equivalent replacements to some of the technical features; and these modifications, changes or replacements do not cause the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present application, and should be covered within the protection scope of the present application. Therefore, the protection scope of the present application should be subject to the protection scope of the claims.

Claims

1. A small sample scene named entity recognition method based on task-level knowledge transfer and prompt learning, characterized in that, The application relates to a method for obtaining named entity recognition task knowledge by using a large model. The method comprises the following steps: constructing a soft prompt training data set based on multi-domain entity annotation data, so as to train the large model by using the soft prompt training data set to obtain the named entity recognition task knowledge; migrating the named entity recognition task knowledge as prompt information to a target domain, and obtaining the to-be-classified entities contained in the target domain text through domain adaptation; 2. The method of claim 1, wherein the task-level knowledge transfer and prompt learning based small sample scene named entity recognition method is characterized in that, extracting entity features for the to-be-classified entities and classifying the to-be-classified entities based on the entity features, wherein the entity features comprise character-level features, word-level features and sentence-level features. The method for constructing the soft prompt training data set based on the multi-domain entity annotation data comprises the following steps: obtaining a plurality of domain entity type annotation data sets, converting each entity type annotation data into an entity annotation data set irrelevant to the entity type by using a specified label, and adding a domain number to each annotation sentence, wherein the specified label comprises a label for identifying that a character is located at a start position of an entity, a label for identifying that a character is located at a middle position, and a label for identifying that a character is a non-entity character; 3. The method of claim 2, wherein the task-level knowledge transfer and prompt learning based small sample scene named entity recognition method is characterized in that, mixing the converted multi-domain entity annotation data sets to obtain the soft prompt training data set. Setting batch size n bs and randomly select an integer k as the number of domains contained in a batch according to the number of domains in the data set; The method for mixing the converted multi-domain entity annotation data sets comprises the following steps: randomly selecting k domain data from the multi-domain entity annotation data set, and respectively extracting the annotation samples from the first k-1 domain data by non-replacement random sampling and randomly selecting the annotation samples from the kth domain data according to a batch size, so that the annotation data of the selected k domains contained in the same batch of training set accounts for the same proportion; 4. The method of claim 1, wherein the task-level knowledge transfer and prompt learning based small sample scene named entity recognition method is characterized by, if the annotation data of one domain is sampled completely, a domain is randomly selected from the remaining domains for sampling until the batch size is met, so as to obtain the soft prompt training data set by mixed sampling. The method for training the large model by using the soft prompt training data set to obtain the named entity recognition task knowledge comprises the following steps: decomposing the soft prompt matrix into a domain-shared matrix and a specific task low-rank matrix; 5. The method of claim 1, wherein the task-level knowledge transfer and prompt learning based small sample scene named entity recognition method is characterized by, in the training process, the classification loss is calculated according to the domain to which the data belongs, and the sum of the classification losses of all domains is minimized as the optimization target, the basic large model is trained by using the soft prompt training data set, so that the domain-shared matrix learns the named entity recognition task knowledge in the training process. The method for obtaining the to-be-classified entities contained in the target domain text through domain adaptation comprises the following steps: obtaining target domain small sample data, and labeling the small sample data with labels containing character named entity start positions, character named entity middle positions and character non-named entities; decomposing the soft prompt matrix into a domain-shared matrix and a low-rank matrix, and setting a first learning rate and a second learning rate of the domain-shared matrix and the low-rank matrix respectively, wherein the first learning rate is smaller than the second learning rate; pre-training the model by using the target domain small sample data, so that the domain-shared matrix is fine-tuned at the first learning rate and keeps the named entity recognition task knowledge obtained by the soft prompt training data set, and the low-rank matrix is fine-tuned at the second learning rate to adapt to the target domain data; inputting the target domain text sequence as input, inputting the pre-trained domain-shared matrix as model prompt information, and guiding the model to output the to-be-classified entities contained in the text sequence.

6. The method of claim 1, wherein the task-level knowledge transfer and prompt learning based small sample scene named entity recognition method is characterized by, Before extracting entity features and classifying the entity to be classified based on the entity features, it further comprises: Data augmentation is performed on the target field small sample data through word replacement and sentence transformation, the word replacement is replacing the labeled entity in the text with the same type of entity or symbol, and the sentence transformation is transforming the original text sentence representation form; An entity type classification network model is constructed, and a loss function of the entity type classification network model is set, the entity type classification network model comprises a feature extraction unit, a feature fusion unit and a feature classification unit, and the loss function comprises a contrast learning loss term and a cross-entropy loss term, wherein in the contrast learning loss term, two different dropout parameters are used to extract features from the same sentence twice to obtain two different representation vectors of the same entity, and the positive samples contained in the input sentence are processed based on InfoNCE Loss; The entity type classification network model is trained based on the loss function and using the enhanced target field small sample data to obtain an entity classification target model.

7. The method of small sample scene named entity recognition based on task-level knowledge transfer and prompt learning according to claim 1 or 6, characterized in that, Extracting entity features and classifying the entity to be classified based on the entity features comprises: Using a language model, a bidirectional long short-term memory network and a convolutional neural network as a feature extraction unit, using each network in the feature extraction unit to extract sentence-level features, word-level features and character-level features of the entity to be classified; Using a gated linear unit as a feature fusion unit to fuse the sentence-level features, word-level features and character-level features using the feature fusion unit; Using a Softmax layer as a feature classification unit, inputting the fused features into the Softmax layer, using the Softmax layer to map the feature vector to the judgment probability of the entity type, and outputting the type with the highest probability as the entity type of the entity to be classified.

8. A small sample scene named entity recognition system based on task-level knowledge transfer and prompt learning, characterized in that, It comprises a knowledge acquisition module, a domain adaptation module and an entity recognition module, wherein The knowledge acquisition module is configured to construct a soft prompt training data set based on multi-domain entity annotation data, and to train a large model using the soft prompt training data set to acquire knowledge of a named entity recognition task; The domain adaptation module is configured to migrate the knowledge of the named entity recognition task as prompt information to a target domain, and to obtain an entity to be classified contained in a target domain text through domain adaptation, the target domain being a small sample domain with sparse entity annotation samples; The entity recognition module is configured to extract entity features and classify the entity to be classified based on the entity features, the entity features including character-level features, word-level features and sentence-level features.

9. An electronic device, comprising: It comprises: At least one processor, and a memory coupled to the at least one processor; The memory stores a computer program, which can be executed by the at least one processor to implement the method of any one of claims 1-7.

10. A computer-readable storage medium, characterized in that, The computer readable storage medium stores a computer program, which when executed, can implement the method of any one of claims 1-7.

Images