Small-sample named entity recognition method for joint operation of reservoir groups

Through two-stage model and comparison learning technology, the problem of unstructured sentence processing and labeled data dependence in the joint scheduling of reservoir groups is solved, and high-accurate naming entity recognition is achieved, reducing the cost of manual labeling, and suitable for the field of data scarcity.

CN118569260BActive Publication Date: 2024-11-12HOHAI UNIV +1
View PDF 5 Cites 0 Cited by

Patent Information

Application Number
CN202411036769.9
Authority / Receiving Office
CN · China
Patent Type
Patents(China)
Current Assignee / Owner
Filing Date
2024-07-31
Publication Date
2024-11-12
Estimated Expiration
2044-07-31

AI Technical Summary

Technical Problem

In the field of joint scheduling of reservoir groups, the existing technology is difficult to effectively process unstructured sentences, which leads to difficulty in building a knowledge base and relies on a large amount of labeled data, increasing the cost of manpower and material resources.

Method used

Using a two-stage model, entity span detection and entity type classification, sentence encoding is performed through the pre-trained language model BERT, span boundary matrix and prototype network are constructed, and combined with comparison learning, the dependence on labeled data is reduced and the recognition accuracy is improved.

Benefits of technology

Highly accurate named entity recognition is achieved on small sample data sets, reducing manual tagging costs, improving model generalization capabilities, and suitable for data scarcity.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN118569260B_ABST
    Figure CN118569260B_ABST
Patent Text Reader

Abstract

The present invention discloses a small sample named entity recognition method for joint scheduling of a reservoir group, and constructs a model through two stages: entity span detection and entity type classification. In the span detection stage, a span boundary matrix is ​​used to learn span boundary information and filter potential erroneous spans to improve detection accuracy; in the entity type classification stage, label guidance is used to improve the quality of prototype network construction. In the model training process, contrastive learning is used to improve the loss function, and the parameters are updated through back propagation to make the spatial representation of entities of the same type closer. After the entity prototype is constructed, the distance function is used to calculate the distance between the sample and the category prototype, and it is converted into a similarity probability to classify the sample. The present invention realizes the automatic extraction of entities for joint scheduling of a reservoir group, can complete the named entity recognition task in a data set with fewer samples, has a high recognition effect, and greatly saves the manpower, material resources and time costs required for manual labeling.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The invention relates to a small sample named entity recognition method for joint dispatching of a reservoir group, and belongs to the technical field of information extraction. Background Art

[0002] In the context of smart water conservancy, the joint dispatching of reservoir groups, as an important part of reservoir and water resources management, involves complex decision-making processes and information processing. The joint dispatching of reservoir groups needs to comprehensively consider various information such as water quantity, water quality, ecological needs and socio-economic factors among multiple reservoirs to ensure the efficient use of water resources and flood control safety. With the advancement of science and technology, especially the development of information technology and artificial intelligence, more and more technologies are being applied to reservoir management and dispatching to improve the intelligence and automation level of dispatching. However, most of the information management systems currently built by water conservancy are mainly based on monitoring and management. The core model of intelligent simulation of watersheds is not highly autonomous, and the analysis ability of the model cannot meet the needs of real-time dispatching. The digitization, rule-based, knowledge-based and dispatching object modeling of the scheme have not yet formed knowledge-driven.

[0003] By using artificial intelligence technologies such as deep learning and knowledge graphs, we build models such as joint dispatching procedures and plan corpora for controllable reservoir groups, improve the level of intelligent basin dispatching, and provide diversified, knowledge-based, and intelligent information services for pilot projects. At present, in the field of joint dispatching of reservoir groups, there are corpora such as special dispatching procedures for each reservoir in different regions and reservoirs. However, there are a large number of unstructured sentences in these corpora, which is not conducive to the construction of a knowledge base for joint dispatching of reservoir groups. Summary of the invention

[0004] Purpose of the invention: In view of the problems and shortcomings of the prior art, the present invention provides a small sample named entity recognition method for joint scheduling of reservoir groups.

[0005] Technical solution: A small sample named entity recognition method for joint operation of reservoir groups, including the following steps:

[0006] Step 1: The sentence sequences of the joint operation procedures of a group of reservoirs constitute a joint operation dataset of a group of reservoirs. The dataset is divided into a support set and a query set. The pre-trained language model BERT is used to convert the sentence sequences into vector representations that can be processed by the model.

[0007] Step 2: Construct a small sample named entity recognition model using two stages: entity span detection and entity type classification. In the entity span detection stage, construct a span boundary matrix to convert the output of BERT into a form suitable for entity boundary detection; design a span filtering strategy to filter the potential erroneous spans in the span set output by the entity span detection stage to obtain a candidate span set as the input of the entity classification stage; enumerate the vector representation of each span example through the span boundary matrix, calculate the difference between the entity category label predicted by the model and the actual label, and obtain a span-based loss function. The filtering strategy is to filter the potential erroneous spans, rank them according to the probability score of the entity span, and select the span with a high score. When extracting candidate spans, the filtering strategy adjusts the threshold of the span score based on the predefined confidence threshold to keep the number of extracted spans within the set range.

[0008] Step 3, in the entity type classification stage, the mean vector of the sample is calculated for each span category in the candidate span set as a prototype to construct a prototype network. These prototype vectors represent the center points of different entity categories. The samples specifically refer to the semantic representations of all spans belonging to a certain entity category in the training set. For example, if there are many entities labeled as "reservoirs", the semantic representations of these "reservoir" entities will be collected and the mean will be calculated to form a prototype vector representing the "reservoir" category. The prototype network is a model containing prototype vectors of all categories and is used for the entire classification task. It learns prototype vectors of all categories through training data and uses these vectors for classification. Using the above prototype as a classifier, the entity span in the span detection stage is classified by calculating the distance between the candidate span and the prototype; the contrast loss function is constructed using contrastive learning to measure the difference between samples of different categories by calculating their distance in the feature space. The samples specifically refer to the representation of entity spans of different categories in the feature space.

[0009] Step 4: For the given sentence sequence sample of the joint dispatching regulations of a group of reservoirs in the query set, the possible entity span is extracted through step 2, and the similarity is calculated with the prototype network constructed in step 3 to obtain the distance between them, and then the distance is converted into probability, and the prototype with the highest probability of similarity with the sample is selected as the prediction result of entity recognition.

[0010] In step 1, in the support set, by context encoding the characters in the sentence, the vector representation of all entities can be obtained. The specific implementation process is: using BERT as a pre-trained language model, the words in the sentence sequence of the reservoir group joint scheduling corpus are encoded into computer-understandable vectors. During the encoding process, the contextual semantics of each word in the sentence are considered and these words are converted into vector representations:

[0011] H = [h1,…,h n]=f θ1 ([x1,…,x n ])

[0012] Where H is the output of the last layer of the encoder, x i represents the i-th word in the sentence, h i is through the encoder f θ1 The transformed, semantically enhanced x i Vector representation of , i∈[1,n].

[0013] In step 2, a small sample named entity recognition model is constructed using two stages: entity span detection and entity type classification. The specific steps of the entity span detection stage are as follows:

[0014] (2.1) Based on the label span, generate a labeled span boundary matrix for each support sentence , in this matrix In it, i and j represent the start and end tag intervals [i:j] of the entity span respectively. If this interval is an entity, the elements in the span boundary matrix will correspond to 1, and if this interval is not an entity, the elements will correspond to 0. Support sentences refer to sentences that have already labeled entities in the training set. These sentences are used to train the model to help the model learn to identify the boundaries of entities by generating labeled span boundary matrices. Suppose there is a sentence included in the support set, "Reservoir A and Reservoir B are jointly scheduled during the flood season" is a support sentence. In this sentence, "Reservoir A" and "Reservoir B" are labeled entities. After the BERT model obtains the context vector H of the input sequence, two fully connected layers are introduced to convert the output of the BERT model into a span form suitable for entity boundary detection. Then, by learning the idea of ​​the self-attention mechanism, for the vector representation of each span, the query matrix W is used. q and bond matrix W k Generate query vector q i and key vector k i ; Then calculate q i and k i The dot product between them is used to obtain a matrix representing the similarity between each pair of words, which is used to further calculate the weight of each word in the self-attention mechanism, so that the model can better capture the contextual information in the sentence.

[0015] (2.2) Design a span filtering strategy. First, iterate the probability tensor and index tensor of each input sample. Based on the predefined confidence threshold, for each sample, adjust the threshold of the span score to keep the number of extracted spans within a certain range. The confidence threshold is dynamically adjusted as a filtering condition, with the aim of adapting to the need to find high-quality entity spans by lowering the initial value. Therefore, if the probability of a sample exceeds the threshold, it will be added to the list of predicted entities; otherwise, it will be filtered out. This process will be repeated. If the candidate span does not reach the specified number, the threshold is lowered until enough entities are found or all probabilities are below the threshold. This strategy can accurately extract span information from the model output, screen and filter potential erroneous spans, and obtain a set of candidate spans as the input of the entity classification stage, providing effective support for subsequent entity type classification tasks.

[0016] (2.3) The loss function of the final entity span detection stage is defined as:

[0017]

[0018] Among them, log is the logarithmic function with base e, exp is the exponential function, the element in the i-th row and j-th column in the matrix Ω is used to indicate whether the span [i, j] is an entity, and g(i, j) is the probability function of the entity span boundary.

[0019] In step 3, in the entity type classification stage, the mean vector of the sample is calculated for each span category in the candidate span set as a prototype, a prototype network is constructed, the prototype is used as a classifier, the entity span in the span detection stage is classified, and the contrast loss function is constructed using contrastive learning. The difference between samples of different categories is measured by calculating the distance in the feature space. The specific steps are as follows:

[0020] The type classification stage mainly includes the training of the prototype network. In the type classification stage, the prototype network enhanced by label prompts is mainly used for type classification, and a span-based loss function is used. By using the label information and prompt information in the support set sentences, the prototypes of the entities in the support set are constructed to ensure that each prototype effectively represents the common characteristics of the corresponding category label. Then, the similarity between each sample and each category label is expressed by calculating the distance function between the query set sample and all prototypes. The entity type classification stage uses the prototype network to construct its prototype and generate a class prototype with specific semantic features. Finally, the softmax function is used to convert the similarity into a probability output, and the cross entropy function is used to calculate the classification loss, and the label sequence with the highest probability is output. The detailed process is as follows:

[0021] (3.1) First, calculate the semantic information representation of all words involved from the i-th word to the j-th word :

[0022]

[0023] Among them, h k is the vector representation of the words involved between the i-th word and the j-th word; let Represents the candidate span set contained in the given support set S, and calculates S k The mean vector of each span category in is used to represent each entity class c k Prototype :

[0024]

[0025] in, represents all the semantic information about words from the i-th word to the j-th word, |S k | is the set of all entity spans in the support set that are marked as class k, entity class c k Prototype It is represented as the average of the semantic representations of all k-th entity spans; by calculating the semantic representations of all entity spans belonging to the k-th category The average value of , we get the prototype representation of the kth class;

[0026] (3.2) Use contrastive learning to construct a contrastive loss function and calculate the sentence sequence samples of the joint scheduling procedures of the reservoir group extracted by the BERT encoder and their label context vectors h1, h2. Through the loss function of contrastive learning, the entity representations of different categories in the semantic space can be more effectively distinguished, so that samples with the same label information are closer in the vector space, and samples with different label information are farther away in the vector space. The contrastive loss function for the prototype network is defined as:

[0027]

[0028] Among them, T j Represented as the jth category in the dataset T, (x i ,y i ) belongs to an entity and label in the support set S, is the entity set of the i-th category, where Calculated samples , The exponential value of the similarity between them, log is the natural logarithm function.

[0029] In step 4, the specific steps of using similarity calculation to predict entity categories are as follows:

[0030] Calculate s in step (3.1) [i,j] Belongs to a certain prototypek The probability of , where d(x,y) is used as a similarity measure, representing the Euclidean distance between x and y:

[0031]

[0032] Among them, p(y k ; x [i,j] ) is the entity span x [i,j] The probability of belonging to category k, for each entity span x [i,j] , select the entity category with the highest probability as the prediction result:

[0033]

[0034] is the entity span x [i,j] Finally, the category k with the highest probability is assigned.

[0035] A computer device includes a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the computer program, the small sample named entity recognition method for joint scheduling of a reservoir group as described above is implemented.

[0036] A computer-readable storage medium stores a computer program / instruction, which, when executed by a processor, implements the steps of the small sample named entity recognition method for joint scheduling of a reservoir group.

[0037] Beneficial effects: Compared with the prior art, the present invention has the following advantages:

[0038] (1) In the field of joint operation regulations of reservoir groups, it is difficult to obtain a large amount of labeled data. This method uses a two-stage model to separate entity span detection from entity type classification, reducing the dependence on a large amount of labeled data, thereby enabling effective entity recognition on a smaller sample dataset such as the joint operation regulations of reservoir groups.

[0039] (2) Introducing the span boundary matrix in the span detection stage can more accurately learn span boundary information, filter out potential erroneous spans, and ensure more accurate detection. This design reduces the error of entity span detection and improves the accuracy of named entity recognition. By building a higher-quality prototype network in the entity type classification stage, the prototype network can more accurately express the semantic features of the entity type, which helps the model to be more accurate in classification and reduce the number of misclassifications.

[0040] (3) In the process of model training, adding contrastive learning can enhance the generalization ability of the model. By improving the loss function and using back propagation to update the parameters, the spatial representation of entities of the same type is made closer, which helps the entity prototype to better express the entity characteristics.

[0041] (4) The method of the present invention can perform effective named entity recognition tasks on a small number of sample data sets, thus greatly reducing the manpower, material resources and time costs required for manual data labeling. This is particularly important for professional fields such as joint scheduling regulations for reservoir groups, because manual labeling requires the participation of experts and is costly. In addition, the scope of application of the present invention is wide. The method is not only applicable to the field of joint scheduling regulations for reservoir groups, but can also be applied to other fields where data is scarce and accurate named entity recognition is required. This versatility makes the method have broad potential in practical applications. BRIEF DESCRIPTION OF THE DRAWINGS

[0042] Figure 1 is a flow chart of a method according to an embodiment of the present invention;

[0043] Figure 2 It is a flow chart of the entity span detection phase of the two-stage small sample named entity recognition method in the example implementation of the present invention;

[0044] Figure 3 This is a flow chart of the entity type classification phase of the two-stage small sample named entity recognition method in the example implementation of the present invention. DETAILED DESCRIPTION

[0045] The present invention is further explained below in conjunction with specific embodiments. It should be understood that these embodiments are only used to illustrate the present invention and are not used to limit the scope of the present invention. After reading the present invention, various equivalent forms of modifications to the present invention by those skilled in the art all fall within the scope defined by the claims attached to this application.

[0046] like Figure 1 As shown in the figure, the small sample named entity recognition method for joint operation of reservoir groups includes the following steps:

[0047] Step 1: Construct a dataset of joint dispatching procedures for a group of reservoirs from a sequence of sentences of joint dispatching procedures for a group of reservoirs, perform preprocessing operations such as data cleaning and segmentation on the dataset, and divide the dataset into a support set and a query set. Define the entity categories that need to be identified according to the specific needs of the joint dispatching procedures for a group of reservoirs. For example: reservoir entity category, organizational entity category, event entity category, etc. When dividing the data, ensure that entities of different categories are relatively evenly distributed in the training set and the validation set. Randomly shuffle the data before dividing it to ensure the randomness and uniformity of the data distribution, which also helps prevent overfitting problems in the model during training.

[0048] Use the pre-trained language model BERT to encode the input sentence sequence. The encoding calculation formula is:

[0049] H = [h1,…,h n ]=f θ1 ([x1,…,x n ])

[0050] Among them, x n represents the nth word in the sentence sequence of the joint dispatching regulations of the reservoir group, f θ1 The encoder used converts the nth word of the sentence sequence of the joint dispatching procedures of the reservoir group into a vector representation H with semantic features. For example, the word "reservoir" is encoded as a one-dimensional vector form of [0.4695123, 0.7864234, -0.7965853, ..., 0.2453667, 0.1248615, -0.1256104]. Different words have different vector representations, and similar words have closer vector representations. In the implementation process, the BERT encoder is used for encoding, and its vector dimension size is selected as 768.

[0051] Step 2: The encoded sentence will be input into the two-stage small sample named entity recognition model. The structure of the model mainly includes two stages: entity span detection and entity classification. In the entity span detection stage, by using the span boundary matrix, the feature representation of a single word is converted into span-level metric learning, focusing on the entity span in the text rather than a single word, so that the model can better capture the information within the span. In the span detection stage, only the detection task of entities and boundaries needs to be processed. The constructed span boundary matrix will be input according to the span vector in the sentence to construct a span boundary matrix . Each element of this matrix represents the start and end boundary of a span in the sentence. If this interval is an entity, then the element in the span boundary matrix will correspond to 1, if this interval is not an entity, then the element will correspond to 0. After the output of the span boundary matrix, high-quality candidate spans are screened out according to the score to form a candidate span set. These candidate spans will be used as input for entity type classification in the next stage, and the model also uses the span loss function L span Back propagation trains the model parameters by minimizing the loss function, so that it continuously updates the parameters and learns better and clearer entity boundary information. The span cross entropy loss function for multi-label classification is expressed as:

[0052]

[0053] Entity span detection example Figure 2As shown in Figure 1, for the sentence "Flood control dispatching of large reservoirs should be subject to the dispatching of flood control and drought relief command departments with dispatching authority.", the span detection stage first uses BERT to encode the sentence, and then uses the above-mentioned span loss function L in the span extraction process. span , effectively output the spans of the entities "large reservoir", "flood control dispatch" and "flood control and drought relief command department" in the sentence, among which the span of the entity "large reservoir" is (1, 4), which means it is between the 1st and 4th words; the span of the entity "flood control dispatch" is (5, 8), which means it is between the 5th and 8th words; the span of the entity "flood control and drought relief command department" is (18, 25), which means it is between the 18th and 25th words.

[0054] Step 3: In the entity classification stage, by constructing a prototype network and using the contrast loss function during the training process, the model learns how to continuously reduce the feature differences of similar entities so that similar entities are close to each other in the spatial representation, and the constructed entity class prototype can accurately express the characteristics of this class.

[0055] For example, the word vector representations of the words "Baisha Reservoir" and "Large Reservoir" when the model is just trained are:

[0056] [0.4527876,-0.4532193,-0.948292,…,0.3354558,0.7521344,-0.3540298] and

[0057] [0.467344, 0.76755058,-07684167,…,0.2654797,0.3611071,-0.6812087].

[0058] After model training, the word vector representations become:

[0059] [0.7561294,-0.354472,-0.15662472,…,0.4456172,0.5192837,-0.391827] and

[0060] [0.7456218,-0.3156727,-0.14621638,…,0.4462272,0.5214231,-0.3956281].

[0061] These two words are in the same entity type, so their vector representations are close to each other. Since entities of the same type are quite different from entities of different types, the distance between the words in the given support set sample and the obtained class prototype is calculated to assign categories to the words in the sample according to the distance. Let S k ={x [i,j]} represents the entity span set contained in a given support set S, by calculating the span x [i,j] ∈S k The mean of the sum of all information in represents each entity class c k Prototype The smaller the value, the closer the semantic features of similar samples are, which is more beneficial for accurately constructing prototypes. The calculation formula is as follows:

[0062]

[0063] The loss function in the entity classification stage can more effectively distinguish entity representations of different categories in the semantic space by adding contrastive learning ideas, so that samples with the same label information are closer in the vector space, while samples with different label information are farther away in the vector space. j Represented as the jth category in the dataset T, (x i ,y i ) belongs to an entity and label in the support set S, For the The contrast loss function is calculated as follows:

[0064]

[0065] N represents the total number of samples in the batch, τ is the temperature hyperparameter, which is set to 0.05, and sim(x, y) is the cosine similarity formula. After obtaining the model loss, the parameters are updated through back propagation so that the spatial representations of entities of the same type can be close to each other, and the representations of entities of different types are distant from each other. This is the result of continuously reducing feature differences and losses.

[0066] Examples of entity classification are Figure 3 As shown in the figure, the input candidate span set is "large reservoirs, flood control dispatch, flood control and drought relief command departments". In the entity classification stage, BERT is first used to encode the candidate span set. In the entity type classification process, the contrast loss function L is used. contrastive , three entity categories are assigned to the above entity span: reservoir entity category, event entity category and organization entity category.

[0067] Through the above steps, the support set S is calculated train After the prototypes of all entity classes in the query set Q train Each span x in [i,j] , according to the semantic representation of its span s [i,j] , calculate whether it belongs to a certain prototype y k The probability of , where d(x,y) is used as a similarity measure, representing the Euclidean distance between x and y:

[0068]

[0069] According to the above steps, entities can be effectively classified using semantic representations and compared with prototype networks with similar semantics. Given the words in the support set sample after span detection, the distance between the words and the prototype obtained in step 4 is calculated, and the words in the sample are assigned categories based on the distance calculation results. The sample data in the query set is obtained through the boundary matrix and span filtering strategy of the span detector to obtain a candidate span set C containing all candidate spans. span , these candidate spans will be assigned to the entity type with the closest semantics in the type classification stage. Finally, the candidate span set C is calculated according to the formula span With each prototype y k distance between them and select the closest prototype type y k As a result.

[0070]

[0071] The similarity between each extracted entity span and the prototype vector generated in step 3 can be calculated using distance function or cosine similarity, and the calculated similarity can be expressed as probability. For example, for the span "large reservoir", calculate its similarity with each category prototype vector to obtain a similarity score. The similarity score between "large reservoir" and the reservoir category prototype is 0.85; the similarity score between "large reservoir" and the organization entity category prototype is 0.35; the similarity score between "large reservoir" and the event entity category prototype is 0.50. The category prototype with the highest similarity probability is selected as the prediction result, that is, "large reservoir" is classified as the reservoir entity category. Similarly, the similarity of other extracted entity spans is calculated and their categories are predicted. By calculating the similarity between the entity span and the prototype, the entities in the joint dispatching procedures of the reservoir group can be classified and identified.

[0072] The present invention utilizes limited data for training on a corpus of joint dispatching procedures of reservoir groups with research value, and uses the proposed two-stage small sample named entity recognition method to first preprocess and divide the sentence sequence. After encoding with the pre-trained model BERT, span detection and entity classification are performed using a two-stage small sample named entity recognition model, ultimately achieving the ability to accurately represent the semantic features of the sample, thereby obtaining better classification results.

[0073] The present invention gets rid of the dependence of existing machine learning and deep learning algorithms in the field of joint scheduling regulations of reservoir groups on a large amount of training data, can complete the named entity recognition task in a data set with fewer samples, and has high recognition accuracy and good convergence performance, effectively saving manpower and material resources.

[0074] Obviously, those skilled in the art should understand that the various steps of the small sample named entity recognition method for the joint scheduling procedure of a reservoir group in the above-mentioned embodiment of the present invention can be implemented by a general-purpose computing device. They can be concentrated on a single computing device or distributed on a network composed of multiple computing devices. Optionally, they can be implemented with program codes executable by the computing device, so that they can be stored in a storage device and executed by the computing device, and in some cases, the steps shown or described can be executed in an order different from that shown here. In this way, the embodiments of the present invention are not limited to any specific combination of hardware and software.

Claims

1. A small sample named entity recognition method for joint operation of reservoir groups, characterized in that: The steps include: Step 1: divide the reservoir group joint dispatching dataset consisting of the sentence sequence of the reservoir group joint dispatching regulations into a support set and a query set, and use the pre-trained language model to convert the sentence sequence into a vector representation that can be processed by the model; Step 2: construct a small sample named entity recognition model using two stages: entity span detection and entity type classification. In the entity span detection stage, construct a span boundary matrix to convert the output of the pre-trained language model into a form suitable for entity boundary detection. Design a span filtering strategy to filter out potential erroneous spans from the span set output by the entity span detection phase, and obtain a candidate span set as the input of the entity classification phase; The vector representation of each span example is enumerated through the span boundary matrix, and the difference between the entity category label predicted by the model and the actual label is calculated to obtain the span-based loss function; Step 3: In the entity type classification stage, the mean vector of the sample for each span category in the candidate span set is calculated as a prototype to construct a prototype network. The sample refers to the semantic representation of all spans belonging to a certain entity category in the training set; Step 4: for the sentence sequence sample of the joint dispatching procedures of the reservoir group given in the query set, extract the entity span through step 2, and calculate the similarity between the entity span and the prototype network constructed in step 3 to obtain the distance between them, and then convert the distance into probability, and select the prototype with the highest probability of similarity with the sample as the prediction result of entity recognition; In the entity type classification stage, the mean vector of the sample is calculated for each span category in the candidate span set as a prototype, and a prototype network is constructed. The prototype is used as a classifier to classify the entity span in the span detection stage. The contrast loss function is constructed using contrastive learning, and the difference between samples is measured by calculating the distance between samples of different categories in the feature space. The specific steps are as follows: In the type classification stage, the prototype network is trained, and the prototype network enhanced by label prompts is used for type classification, and a span-based loss function is used; by using the label information and prompt information in the support set sentences, the prototypes of the entities in the support set are constructed; then, the similarity between each sample and each category label is expressed by calculating the distance function between the query set sample and all prototypes. The prototype is constructed using the prototype network in the entity type classification stage to generate a class prototype with specific semantic features; finally, the similarity is converted into a probability output using the softmax function, and the classification loss is calculated using the cross entropy function, and the label sequence with the highest probability is output; the process is as follows: (3.1) First, calculate the semantic information representation of all words from the i-th word to the j-th word : , Among them, h k is the vector representation of the words involved between the i-th word and the j-th word; let Represents the candidate span set contained in the given support set S, and calculates S k The mean vector of each span category in is used to represent each entity class c k Prototype : , in, represents all the semantic information about words from the i-th word to the j-th word, |S k | is the set of all entity spans in the support set that are marked as class k, entity class c k Prototype It is represented as the average of the semantic representations of all k-th entity spans; by calculating the semantic representations of all entity spans belonging to the k-th category The average value of , we get the prototype representation of the kth class; (3.2) Use contrastive learning to construct a contrastive loss function, calculate the sentence sequence samples of the joint scheduling regulations of the reservoir group extracted by the BERT encoder and their label context vectors h1, h2; distinguish the entity representations of different categories in the semantic space through the contrastive learning loss function; the contrastive loss function used for the prototype network is defined as: , in, is the entity set of the i-th category, where Calculated samples , The exponential value of the similarity between them, log is the natural logarithm function.

2. The small sample named entity recognition method for joint dispatching of reservoir groups according to claim 1 is characterized in that: In the step 3, the prototype network is a model containing prototype vectors of all categories, which is used for the entire entity classification task. The prototype network learns prototype vectors of all categories through training data and uses the prototype vectors for classification; the prototype is used as a classifier, and the entity span in the span detection stage is classified by calculating the distance between the candidate span and the prototype; contrastive learning is used to construct a contrastive loss function, and the difference between samples of different categories is measured by calculating the distance between them in the feature space; the samples refer to the representation of entity spans of different categories in the feature space.

3. The small sample named entity recognition method for joint dispatching of reservoir groups according to claim 1 is characterized in that: In the step 1, in the support set, the vector representation of all entities is obtained by context encoding the characters in the sentence. The implementation process is: using BERT as a pre-trained language model, the words in the sentence sequence of the reservoir group joint scheduling corpus are encoded into computer-understandable vectors.

4. The small sample named entity recognition method for joint operation of reservoir groups according to claim 1 is characterized in that: In step 2, the specific steps of the entity span detection phase are as follows: (2.1) Based on the label span, generate a labeled span boundary matrix for each support sentence , i and j represent the start tag and end tag interval [i:j] of the entity span respectively. If the interval is an entity, the element in the span boundary matrix will correspond to 1, and if the interval is not an entity, the element will correspond to 0; the support sentence refers to the sentence that has been marked with entities in the training set; After the BERT model obtains the context vector of the input sequence, two fully connected layers are introduced to convert the output of the BERT model into a span form suitable for entity boundary detection; Then, by learning the idea of ​​the self-attention mechanism, for the vector representation of each span, the query vector and key vector are generated through the query matrix and the key matrix; then, by calculating the dot product between the query vector and the key vector, a matrix representing the similarity between each pair of words is obtained, which is used to calculate the weight of each word in the self-attention mechanism; (2.2) Design a span filtering strategy. First, iterate the probability tensor and index tensor of each input sample. Based on a predefined confidence threshold, for each input entity span sample, adjust the threshold of the span score so that the number of extracted spans is within the set range. The confidence threshold is dynamically adjusted as a filtering condition, with the aim of adapting to the need to find high-quality entity spans by lowering the initial value. If the probability of a sample exceeds the threshold, it will be added to the predicted entity list. Otherwise, it will be filtered out. If the candidate spans do not reach the specified number, the threshold is lowered until the required number of entities are found or all probabilities are below the threshold. The span filtering strategy extracts span information from the model output to obtain a set of candidate spans as input to the entity classification stage. , Where log is the logarithmic function with base e, exp is the exponential function, the element in the i-th row and j-th column of the matrix Ω is used to indicate whether the span [i, j] is an entity, and g(i, j) is the probability function of the entity span boundary.

5. A computer device, characterized in that: The computer device includes a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the computer program, the small sample named entity recognition method for joint scheduling of a reservoir group as described in any one of claims 1 to 4 is implemented.

6. A computer-readable storage medium having a computer program / instruction stored thereon, characterized in that: When the computer program / instructions are executed by a processor, the steps of the small sample named entity recognition method for joint scheduling of a reservoir group are implemented as described in any one of claims 1 to 4.

Citation Information

Patent Citations

  • Drug molecule property prediction method, device and equipment based on comparative learning

    CN114386694A

  • Cross-domain named entity recognition method and device based on span contrast learning, storage medium and product

    CN116644751A

  • Nested named entity recognition method based on span boundary perception

    CN117131869A

  • Named entity recognition model training method and named entity recognition method and device

    CN117556819A

  • Techniques for facilitating the joining of datasets

    US20180075115A1