A method and system for extracting information from a small number of samples

Abstract

This invention discloses a method and system for few-shot information extraction, relating to the field of artificial intelligence technology. The method includes: inputting sentence samples from a sentence support set into an initial named entity recognition model for model training; performing boundary detection on the sentence samples using an entity boundary detector to output entity boundaries, and calculating boundary detection classification loss and boundary-aware contrast loss; classifying entities using sentence samples and entity boundaries using an entity classifier, outputting the probability distribution of predicted entity labels embedded in multiple target entities, and calculating token contrast loss, regularization loss, prototype classification loss, and entity contrast loss; iteratively optimizing the initial named entity recognition model based on each loss, and verifying and determining the target named entity recognition model using a sentence query set; inputting the sentence to be recognized into the target named entity recognition model and outputting the target recognition result. The target named entity recognition model obtained by the above scheme helps improve the accuracy of few-shot named entity recognition.

Description

Technical Field

[0001] This invention relates to the field of artificial intelligence technology, and in particular to a method and system for extracting information from a small number of samples. Background Technology

[0002] Information extraction is one of the core technologies in natural language processing, and named entity recognition, as an entity information extraction in information extraction, aims to locate the boundaries of entities in a given sentence and classify them into predefined categories. Its results are often used for downstream tasks such as entity linking, relation extraction and syntactic analysis.

[0003] Traditional named entity recognition (MID) methods are implemented using deep neural networks trained on large-scale labeled datasets. However, labeling data is time-consuming and labor-intensive, thus necessitating exploration of methods for few-shot MID tasks. Currently, many methods treat few-shot MID as a single-stage sequence labeling task. The core idea is to classify by comparing the semantic distance between the token to be classified and the prototype of each category. However, this often ignores the completeness of entities and the potential relationships between tokens. To address this, some researchers have proposed a two-stage approach, decomposing the few-shot MID task into two independent processes: entity boundary detection and entity classification. In the boundary detection stage, tokens are divided into entity tokens and non-entity tokens to locate the boundaries of MID. Then, in the entity classification stage, detected entities are classified into predefined categories. However, the boundary detection stage may detect some entities that do not belong to the predefined categories, resulting in lower accuracy for few-shot MID. Summary of the Invention

[0004] This invention provides a method and system for extracting information from few samples, which improves the entity recognition accuracy of existing few-sample named entity recognition methods.

[0005] The first aspect of this invention provides a method for extracting information from a small number of samples, comprising:

[0006] Sentence samples from the sentence support set are used as input to train the initial named entity recognition model. The initial named entity recognition model includes an entity boundary detector and an entity classifier.

[0007] The entity boundary detector is used to perform boundary detection on the sentence samples, output the entity boundary, and calculate the boundary detection classification loss and the boundary-aware contrast loss.

[0008] The entity classifier performs entity classification using the sentence samples and entity boundaries, outputs the probability distribution of predicted entity labels embedded in multiple target entities, and calculates token comparison loss, regularization loss, prototype classification loss and entity comparison loss.

[0009] The initial named entity recognition model is iteratively optimized based on the boundary detection classification loss, the boundary-aware contrast loss, the token contrast loss, the regularization loss, the prototype classification loss, and the entity contrast loss, and the model parameters are verified using a sentence query set to determine the target named entity recognition model.

[0010] The sentence to be recognized is input into the target named entity recognition model for boundary detection and entity classification, and the target recognition result is output.

[0011] Further, the step of performing boundary detection on the sentence sample using the entity boundary detector, outputting multiple token-embedded target location labels, and calculating the boundary detection classification loss includes:

[0012] The sentence samples are converted into location token embedding sequences based on the boundary BERT embedding layer;

[0013] A linear layer is used to map the location token embedding sequence to the BIOES tag space, and the predicted location tag probability distribution of each location token embedding is calculated based on the softmax function;

[0014] The target location label embedded in each location token is determined by the conditional random field of Viterbi decoding based on the probability distribution of each predicted location label, and the entity boundary is determined by each target location label.

[0015] The boundary detection classification loss is calculated based on the probability distribution of each predicted label, and the corresponding boundary-aware contrast loss is calculated based on the entity embedding, positive sample representation, and negative sample representation of each entity boundary.

[0016] Further, the entity classifier performs entity classification using the sentence samples and the entity boundaries, outputs the probability distribution of predicted entity labels embedded in multiple target entities, and calculates token comparison loss, regularization loss, prototype classification loss, and entity comparison loss, including:

[0017] The sentence samples are transformed into semantic token embedding sequences through a semantic BERT embedding layer;

[0018] Determine the semantic entity embedding corresponding to the semantic token embedding sequence based on the entity boundary;

[0019] An entity filtering layer is used to remove invalid entity embeddings from each semantic entity embedding to determine the target entity embedding;

[0020] Using a topic prototype network, prototype embeddings are calculated based on entity embedding samples from the sentence support set, and the predicted entity label probability distribution of each target entity embedding is determined using a temperature-scaled cosine similarity function based on each prototype embedding.

[0021] Calculate the token comparison loss based on the semantic token embedding sequence using the KL divergence;

[0022] The regularization loss is calculated using the prototype embeddings described above, the prototype classification loss is calculated based on the predicted entity label probability distribution, and the entity comparison loss is calculated using the target entity embeddings described above based on KL divergence.

[0023] Furthermore, the boundary detection classification loss includes:

[0024] ;

[0025] Boundary-aware contrastive loss includes:

[0026] ;

[0027] ;

[0028] ;

[0029] In the formula, For boundary detection classification loss, For token index, For the total number of tokens, For cross-entropy, For the location label truth value, For the first One token, For the location label probability distribution, For boundary-aware contrast loss, For the sigmoid function, For entity embedding, Characterized by positive samples. For negative sample representation, It is an L2 norm.

[0030] Furthermore, the process of determining the target entity embedding includes:

[0031] ;

[0032] ;

[0033] In the formula, Embedded for the target entity, For entity embedding, A collection of semantic entity embeddings. For the first A collection of entities with entity category tags. For the beginning of entities, For the end of an entity, To and The beginning of different entities, To and The endings of different entities, For dynamic thresholds, To add a semantic token embedding of [CLS], This is the cosine similarity with a scaling factor.

[0034] Furthermore, the process of determining entity embedding includes:

[0035] ;

[0036] In the formula, For entity embedding, For the beginning of entities, For the end of an entity, For linear layers, Embed the entity that is the beginning of the entity. Embedding of entities at the end of entities, This is for concatenating vectors. The first learnable matrix OK.

[0037] A second aspect of the present invention provides a few-sample information extraction system, comprising:

[0038] The data acquisition module is used to input sentence samples from the sentence support set into the initial named entity recognition model for model training. The initial named entity recognition model includes an entity boundary detector and an entity classifier.

[0039] The boundary detection module is used to perform boundary detection on the sentence samples using the entity boundary detector, output the entity boundary, and calculate the boundary detection classification loss and the boundary-aware contrast loss.

[0040] The entity classification module is used to classify entities using the sentence samples and entity boundaries through the entity classifier, output the probability distribution of predicted entity labels embedded in multiple target entities, and calculate token comparison loss, regularization loss, prototype classification loss and entity comparison loss.

[0041] The model optimization module is used to iteratively optimize the model parameters of the initial named entity recognition model based on the boundary detection classification loss, the boundary-aware contrast loss, the token contrast loss, the regularization loss, the prototype classification loss, and the entity contrast loss, and to verify the model using a sentence query set to determine the target named entity recognition model.

[0042] The data recognition module is used to input the sentence to be recognized into the target named entity recognition model for boundary detection and entity classification, and output the target recognition result.

[0043] A computer device provided in a third aspect of the present invention includes a memory and a processor, wherein the memory stores a computer program, and when the computer program is executed by the processor, the processor performs the steps of the few-sample information extraction method as described in any of the preceding claims.

[0044] The fourth aspect of the present invention provides a computer-readable storage medium having a computer program stored thereon, wherein the computer program, when executed, implements the few-sample information extraction method as described in any of the preceding claims.

[0045] The fifth aspect of the present invention provides a computer program product comprising a computer program / instruction, wherein the computer program / instruction, when executed by a processor, implements the few-sample information extraction method as described in any of the preceding claims.

[0046] As can be seen from the above technical solutions, the present invention has the following advantages:

[0047] The above-described scheme of the present invention provides a method for extracting information with few samples, comprising: inputting sentence samples from a sentence support set into an initial named entity recognition model for model training, wherein the initial named entity recognition model includes an entity boundary detector and an entity classifier; performing boundary detection on the sentence samples using the entity boundary detector, outputting entity boundaries, and calculating boundary detection classification loss and boundary-aware contrast loss; performing entity classification using the sentence samples and entity boundaries using the entity classifier, outputting the probability distribution of predicted entity labels embedded in multiple target entities, and calculating token contrast loss, regularization loss, prototype classification loss, and entity contrast loss; iteratively optimizing the model parameters of the initial named entity recognition model based on the boundary detection classification loss, boundary-aware contrast loss, token contrast loss, regularization loss, prototype classification loss, and entity contrast loss, and verifying the model using a sentence query set to determine the target named entity recognition model; inputting the sentence to be recognized into the target named entity recognition model for boundary detection and entity classification, and outputting the target recognition result. Based on the above scheme, boundary-aware contrast loss is introduced to improve the boundary awareness capability of the entity boundary detector. Token contrast loss is used to help the designed entity filtering layer filter invalid entities. Entity contrast loss is also introduced to effectively mine the implicit information between entities. This can better optimize the performance of the topic prototype network and make full use of limited data, which helps to improve the accuracy of few-sample named entity recognition. Attached Figure Description

[0048] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0049] Figure 1 A flowchart illustrating the steps of a few-sample information extraction method provided in this embodiment of the invention;

[0050] Figure 2 This is an overview diagram of the architecture of the named entity recognition model provided in the embodiments of the present invention;

[0051] Figure 3 This is a structural block diagram of a few-sample information extraction system provided in an embodiment of the present invention. Detailed Implementation

[0052] This invention provides a method and system for extracting information from a few samples, which improves the technical problem of low entity recognition accuracy in existing few-sample named entity recognition methods.

[0053] To make the objectives, features, and advantages of this invention more apparent and understandable, the technical solutions of the embodiments of this invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the embodiments described below are only some embodiments of this invention, and not all embodiments. Based on the embodiments of this invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this invention.

[0054] Please see Figure 1 , Figure 1 This is a flowchart illustrating the steps of a method for extracting information from a small number of samples, as provided in an embodiment of the present invention.

[0055] This invention provides a method for extracting information from a small number of samples, comprising:

[0056] Step 101: Input sentence samples from the sentence support set into the initial named entity recognition model for model training. The initial named entity recognition model includes an entity boundary detector and an entity classifier.

[0057] It should be noted that, in order to perform named entity recognition, this embodiment designs a named entity recognition model, such as... Figure 2 As shown, it includes an entity boundary detector and an entity classifier, and performs model training to optimize model parameters. During model training, a training set of sentences is provided. Sampling subset and the initial model parameters ,in, Represents the sentence support set, Represents a sentence query set. This represents the set of entity categories to be classified. The sentence training set contains multiple sentence samples for each entity category. First, the initial named entity recognition model is input with sentence samples from the sentence support set to facilitate internal parameter updates using gradient descent.

[0058]

[0059] In the formula, For internally updated learning rates, To find the gradient, For the model's loss, For the forward propagation process of the model, To support the centralized first A training subset, These are the model parameters after internal updates.

[0060] Step 102: Perform boundary detection on the sentence samples using an entity boundary detector, output the entity boundaries, and calculate the boundary detection classification loss and the boundary-aware contrast loss.

[0061] Step 102 includes the following sub-steps:

[0062] The sentence samples are transformed into position token embedding sequences based on the boundary BERT embedding layer;

[0063] A linear layer is used to map the location token embedding sequence to the BIOES tag space, and the predicted location tag probability distribution of each location token embedding is calculated based on the softmax function.

[0064] By using Viterbi decoding of the conditional random field, the target location label of each location token is determined according to the probability distribution of each predicted location label, and the entity boundary is determined by each target location label;

[0065] The boundary detection classification loss is calculated based on the probability distribution of each predicted location label, and the corresponding boundary-aware contrast loss is calculated based on the location entity embedding, positive sample representation, and negative sample representation of each entity boundary.

[0066] It should be noted that the purpose of the entity boundary detector is to find all named entities in the input sentence by locating entity boundaries. This embodiment treats boundary detection as a sequence labeling problem, using the "BIOES" token scheme to classify each token in the sequence. "B, I, O, E, S" represent the entity's "Begin", "Inside", "Outside", "End", and "Single" tokens, respectively, and the token labels are transformed into entity boundaries. The specific implementation process in the boundary detection stage includes:

[0067] (1) Location embedding learning

[0068] Given a total number of tokens Sentence samples This embodiment uses boundary BERT as the embedding layer to learn the contextual embedding of tokens in a sentence, which can be expressed by the formula:

[0069]

[0070] in, It is the position token embedding sequence output by the last hidden layer of the boundary BERT embedding layer, which consists of multiple position token embeddings. For the boundary BERT embedding layer, For the first A location token is embedded. To add a [CLS] location token embedding;

[0071] (2) Location classification

[0072] After obtaining the location token embedding sequence, this embodiment maps it to the BIOES tag space through a linear layer and uses a softmax function to calculate the location tag probability distribution:

[0073]

[0074] Among them, the probability distribution of location labels Location tag set ;

[0075] To prevent the predicted location labels from violating the BIOES labeling rules, this embodiment introduces a Conditional Random Field (CRF) to calculate the emission and transition probabilities of the labels, and uses the Viterbi decoding algorithm when inferring the location labels. The final target location label can then be represented as:

[0076]

[0077] in, For token index, Indicates the first tokens The corresponding target location labels; to facilitate subsequent calculations, the BIOES location labels are converted into entity boundaries, and each pair of entity boundaries corresponds to one entity;

[0078] (3) Loss function

[0079] To improve the entity boundary detector's ability to perceive entity boundaries, this embodiment proposes using a boundary-aware contrastive learning method to assist in training the entity detector; assuming... and Let A be the beginning and the end of an entity, respectively. Then, the positive sample representation corresponding to the entity boundary can be expressed as: The negative sample representation can be expressed as The entity embedding of a location entity can be calculated using the following formula:

[0080]

[0081] in, For entity embedding, and These represent the beginning and end of the entity, respectively. As a learnable linear layer, This is for concatenating vectors. It is a learnable matrix The OK, For dimensions;

[0082] Then, boundary-aware contrastive loss It can be represented as:

[0083]

[0084]

[0085]

[0086] in, For the sigmoid function, For scaling factor cosine similarity, It is an L2 norm;

[0087] The boundary detection classification loss of an entity boundary detector can be calculated by plotting the predicted location label probability distribution and the ground truth location label. cross-entropy get:

[0088]

[0089] Therefore, the total loss for boundary detection in this stage is:

[0090]

[0091] Step 103: Use the entity classifier to classify entities using sentence samples and entity boundaries, output the probability distribution of predicted entity labels for multiple target entity embeddings, and calculate the token comparison loss, regularization loss, prototype classification loss and entity comparison loss.

[0092] Step 103 includes the following sub-steps:

[0093] Sentence samples are transformed into semantic token embedding sequences through a semantic BERT embedding layer;

[0094] Determine the semantic entity embedding corresponding to the semantic token embedding sequence based on entity boundaries;

[0095] An entity filtering layer is used to remove invalid entity embeddings from each semantic entity embedding to determine the target entity embedding;

[0096] The prototype embeddings are computed based on entity embedding samples from the sentence support set through the topic prototype network, and the predicted entity label probability distribution of each target entity embedding is determined by using a temperature-scaled cosine similarity function based on each prototype embedding.

[0097] Calculate the token contrast loss based on the semantic token embedding sequence using KL divergence;

[0098] Regularization loss is calculated using each prototype embedding, prototype classification loss is calculated based on the predicted entity label probability distribution, and entity contrast loss is calculated using each target entity embedding based on KL divergence.

[0099] It should be noted that the entity classifier aims to filter and classify entities obtained from the entity boundary detector. After obtaining the boundary information of entities based on entity boundary detection, the semantic token embeddings learned by the semantic BERT embedding layer are transformed into semantic entity embeddings. The entity filtering layer designed in this embodiment filters out the boundaries of non-entities or entities that do not belong to predefined categories. Finally, the topic prototype network is used to classify the remaining entities. The specific implementation process in the entity classification stage includes:

[0100] (1) Semantic embedding learning

[0101] Similar to entity boundary detectors, the entity classifier first uses an encoder to learn the contextual representation embedding of the token. However, since this encoder focuses more on the semantic information of the token itself rather than its positional information, a separate BERT layer, namely the semantic BERT embedding layer, is fine-tuned to serve as the encoder for this stage. The contextual embedding of the semantic token can be represented as:

[0102]

[0103] in, It is the semantic token embedding sequence output by the last hidden layer of the semantic BERT embedding layer. For semantic BERT embedding layer, For the first A semantic token embedding, To add a semantic token embedding for [CLS];

[0104] An entity filtering layer is used to remove invalid entity embeddings from each semantic entity embedding to determine the target entity embedding;

[0105] (2) Entity filtering

[0106] Combining the entity boundaries output by the entity boundary detector, the semantic token embedding sequence is transformed into entity embedding according to the process in Equation 5 to obtain the corresponding semantic entity embedding;

[0107] To remove invalid entity boundaries, the entity filtering layer computes the entity embedding as a semantic entity embedding. And the similarity between various entity embedding samples in the support set, if the semantic entity embedding and the support set of entity categories. Each entity category label The maximum similarity between the entity embedding samples is less than the similarity threshold. If the condition is not met, the entity will be considered invalid and removed.

[0108]

[0109] In the formula, The remaining semantic entities after filtering are then embedded, i.e., the target entity embedding. For entity embedding, A collection of semantic entity embeddings. For the first A collection of entities with entity category tags. and To and and The beginnings and ends of different entities, The dynamic threshold is calculated using the following formula:

[0110]

[0111] (3) Prototype classification

[0112] After removing the noisy entities detected in the first stage, a topic prototype network is used to process the remaining entities. Topic Prototype Networks extend the metric-based few-shot learning principle to the field of topic modeling, establishing a framework that can quickly adapt to new topics while maintaining semantic coherence. It operates in a dedicated metric space where topic prototype representations can be effectively compared and classified.

[0113] The prototype representation mechanism implements a non-linear projection function that maps features to a shared embedding space while preserving semantic relationships within that space; for a given set of entity embedding samples in a sentence support set... Entity embedded projection Calculated through a series of transformations:

[0114]

[0115] in, , , and These are the learnable first weight parameter, second weight parameter, first bias parameter, and second bias parameter. For ReLU functions; this projection ensures that entities with similar semantic content are grouped together in the topic embedding space; the prototype embedding for each entity category. Calculate the weighted average of the examples for the support set:

[0116]

[0117] Attention weight Determined by a learned similarity function that considers feature similarity and topic coherence:

[0118]

[0119] in, The balancing factor is used to balance these two factors. As an evaluation factor for assessing topic coherence, it can be obtained by calculating the average of the cosine similarity of the entity representations of the same topic (category) in the support set:

[0120]

[0121] Based on the similarity between the entity and the prototype mentioned above, this embodiment uses a temperature-scaled cosine similarity function to calculate the final entity label probability distribution. :

[0122]

[0123] in, For the softmax function, It is a learnable temperature parameter used to control the sharpness of the probability distribution to ensure that the model can make confident predictions under appropriate conditions, while maintaining uncertainty under ambiguous conditions.

[0124] Understandably, during the reasoning phase, entities embedded with semantic entities can be embedded. The label corresponding to the most similar prototype will be used as the predicted target entity label. :

[0125]

[0126] (4) Loss function

[0127] To help the entity filtering layer filter out erroneously detected entities, this embodiment considers using token-level contrastive learning:

[0128] Assuming the semantic token embeddings obtained from the semantic BERT embedding layer follow a Gaussian distribution, consider a token. and its semantic embedding First, two projection layers are used to generate the parameters of the corresponding Gaussian distribution:

[0129]

[0130] in, and Let these represent the mean and diagonal covariance, respectively. and These are the mean multilayer perceptron and the diagonal covariance multilayer perceptron, respectively. Represents the exponential linear unit. These are constants used for stability calculations;

[0131] Since invalid entities detected by the entity boundary detector are usually caused by non-entity tokens, this embodiment uses contrastive learning loss to separate them in the Gaussian embedding space; given two tokens and and their Gaussian embeddings and Using the token KL divergence between these two Gaussian distributions As a distance metric between two tokens, it is shown below:

[0132]

[0133] The trace of the matrix, Let be the dimension of the Gaussian distribution; since the KL divergence is unidirectional, we calculate the bidirectional token KL divergence in both directions simultaneously. :

[0134]

[0135] In the formula, For the first The token embedding pair of the first One-way token KL divergence of each token embedding, For the first The token embedding pair of the first One-way token KL divergence of each token embedding;

[0136] In training batches In this context, if two tokens have the same label, they are considered a positive sample pair; conversely, two tokens with different labels are considered a negative sample pair. Specifically, for tokens... Its token positive sample set and token negative sample set It can be represented as:

[0137]

[0138]

[0139] Then the token comparison loss for all tokens in this training batch It can be calculated using the following formula:

[0140]

[0141] By using the token comparison loss described above, the entity filtering layer can increase the distance between entity tokens and non-entity tokens, making it easier to separate non-entity tokens.

[0142] In addition, this embodiment employs additional regularization terms to prevent prototype crashes and maintain diversity:

[0143]

[0144] in, For regularization loss, For the first Entity category labels, The square of the norm;

[0145] Prototype classification loss of topic prototype network The following formula is used to calculate:

[0146]

[0147] in, This represents the true value of the entity label probability distribution;

[0148] Furthermore, to optimize the distribution of semantic entity embeddings and make them closer to the corresponding prototypes, this embodiment uses entity-level contrastive loss to train the topic prototype network; similar to token contrastive loss, the Gaussian embeddings of semantic entity embeddings are first calculated. :

[0149]

[0150] in, For the first The mean of token embeddings, For the first Each token is embedded in the diagonal covariance. Embed the mean for the entity. Embed the diagonal covariance for the entity;

[0151] Similar to methods for calculating KL divergence between token Gaussian embeddings, given two entities... and (using respectively) and express, and (All are entity indexes) and their Gaussian embeddings and , No. Entity Embedded With the Entity Embedded Two-way entity KL divergence between The following formula is used to calculate:

[0152]

[0153]

[0154] In the formula, For the first The entity embedding pair of the first entity KL divergence of a single entity embedded in an entity. For the first The entity embedding pair of the first entity KL divergence of one-way entities embedded in an entity;

[0155] Then the target entity embeds the entity positive sample set. and entity negative sample set for:

[0156]

[0157]

[0158] Finally, entity contrast loss The following formula is used to calculate:

[0159]

[0160] Overall loss of entity classification at this stage It can be represented as:

[0161]

[0162] Step 104: Iteratively optimize the model parameters of the initial named entity recognition model based on boundary detection classification loss, boundary-aware contrast loss, token contrast loss, regularization loss, prototype classification loss, and entity contrast loss, and verify the model using a sentence query set to determine the target named entity recognition model.

[0163] It should be noted that, in order to enable the model to learn good parameter initialization methods and quickly adapt to new domains, this embodiment uses a model-independent meta-learning method to train a two-stage model; based on the losses calculated in the boundary detection and entity classification stages, the initial named entity recognition model is iteratively optimized using gradient descent, and then a sentence query set is used. Validate the internally updated model parameters And perform a meta update:

[0164]

[0165] in, It is the learning rate that is updated meta-. It is a sentence query set The first in In Equation (33), parameter tuning via the gradient update step requires calculating the second-order gradient, which is here approximated by its first-order approximation to accelerate the backpropagation process:

[0166]

[0167] Step 105: Input the sentence to be recognized into the target named entity recognition model for boundary detection and entity classification, and output the target recognition result.

[0168] It should be noted that after the target named entity recognition model is trained, when a sentence to be recognized is received, the target named entity recognition model is input to perform boundary detection and entity classification, and the target recognition result is output. It can be understood that the target recognition result may include the target entity label probability distribution shown in Equation (18), and may further include the target entity label shown in Equation (19).

[0169] To verify the effectiveness of the proposed method in this embodiment, experiments were conducted on two commonly used few-shot named entity recognition datasets, Few-NERD and CrossNER. On both datasets, the method in this embodiment outperforms existing few-shot named entity recognition methods.

[0170] In this embodiment of the invention, boundary-aware contrastive loss is introduced to improve the boundary awareness capability of the entity boundary detector. Token contrastive loss is used to help the designed entity filtering layer filter invalid entities. Entity contrastive loss is also introduced to effectively mine the implicit information between entities, which can better optimize the performance of the topic prototype network and make full use of limited data. In addition, the interpretability and statistical rigor of traditional topic models are combined with the representational capability of the prototype network to construct rich prototype representations, enabling the model to make entity recognition decisions based on local syntactic patterns and global topic context. This helps to improve the accuracy of named entity recognition with few samples. The method proposed in this embodiment also has certain reference value for similar tasks in other natural language processing fields.

[0171] Please see Figure 3 , Figure 3 This is a structural block diagram of a few-sample information extraction system provided in an embodiment of the present invention.

[0172] This invention provides a few-sample information extraction system, comprising:

[0173] The data acquisition module 301 is used to input sentence samples from the sentence support set into the initial named entity recognition model for model training. The initial named entity recognition model includes an entity boundary detector and an entity classifier.

[0174] The boundary detection module 302 is used to perform boundary detection on sentence samples using an entity boundary detector, output entity boundaries, and calculate boundary detection classification loss and boundary-aware contrast loss.

[0175] The entity classification module 303 is used to classify entities using sentence samples and entity boundaries through an entity classifier, output the probability distribution of predicted entity labels embedded in multiple target entities, and calculate token contrast loss, regularization loss, prototype classification loss and entity contrast loss.

[0176] The model optimization module 304 is used to iteratively optimize the model parameters of the initial named entity recognition model based on boundary detection classification loss, boundary-aware contrast loss, token contrast loss, regularization loss, prototype classification loss and entity contrast loss, and to verify the model using a sentence query set to determine the target named entity recognition model.

[0177] The data recognition module 305 is used to input the sentence to be recognized into the target named entity recognition model for boundary detection and entity classification, and output the target recognition result.

[0178] Furthermore, the boundary detection module 302 is specifically used for:

[0179] The sentence samples are transformed into position token embedding sequences based on the boundary BERT embedding layer;

[0180] A linear layer is used to map the location token embedding sequence to the BIOES tag space, and the predicted location tag probability distribution of each location token embedding is calculated based on the softmax function.

[0181] By using Viterbi decoding of the conditional random field, the target location label of each location token is determined according to the probability distribution of each predicted location label, and the entity boundary is determined by each target location label;

[0182] The boundary detection classification loss is calculated based on the probability distribution of each predicted label, and the corresponding boundary-aware contrast loss is calculated based on the entity embedding, positive sample representation, and negative sample representation of each entity boundary.

[0183] Furthermore, the entity classification module 303 is specifically used for:

[0184] Sentence samples are transformed into semantic token embedding sequences through a semantic BERT embedding layer;

[0185] Determine the semantic entity embedding corresponding to the semantic token embedding sequence based on entity boundaries;

[0186] An entity filtering layer is used to remove invalid entity embeddings from each semantic entity embedding to determine the target entity embedding;

[0187] The prototype embeddings are computed based on entity embedding samples from the sentence support set through the topic prototype network, and the predicted entity label probability distribution of each target entity embedding is determined by using a temperature-scaled cosine similarity function based on each prototype embedding.

[0188] Calculate the token contrast loss based on the semantic token embedding sequence using KL divergence;

[0189] Regularization loss is calculated using each prototype embedding, prototype classification loss is calculated based on the predicted entity label probability distribution, and entity contrast loss is calculated using each target entity embedding based on KL divergence.

[0190] Furthermore, the boundary detection classification loss includes:

[0191] ;

[0192] Boundary-aware contrastive loss includes:

[0193] ;

[0194] ;

[0195] ;

[0196] In the formula, For boundary detection classification loss, For token index, For the total number of tokens, For cross-entropy, For the location label truth value, For the first One token, For the location label probability distribution, For boundary-aware contrast loss, For the sigmoid function, For entity embedding, Characterized by positive samples. For negative sample representation, It is an L2 norm.

[0197] Furthermore, the process of determining the target entity embedding includes:

[0198] ;

[0199] ;

[0200] In the formula, Embedded for the target entity, For entity embedding, A collection of semantic entity embeddings. For the first A collection of entities with entity category tags. For the beginning of entities, For the end of an entity, To and The beginning of different entities, To and The endings of different entities, For dynamic thresholds, To add a semantic token embedding of [CLS], This is the cosine similarity with a scaling factor.

[0201] Furthermore, the process of determining entity embedding includes:

[0202] ;

[0203] In the formula, For entity embedding, For the beginning of entities, For the end of an entity, For linear layers, Embed the entity that is the beginning of the entity. Embedding of entities at the end of entities, This is for concatenating vectors. The first learnable matrix OK.

[0204] This invention also provides a computer device, including a memory and a processor, wherein the memory stores a computer program; when the computer program is executed by the processor, the processor performs the steps of the few-sample information extraction method as described in any of the above embodiments.

[0205] This invention also provides a computer-readable storage medium storing a computer program / instructions thereon, which, when executed by a processor, implements the steps of the few-sample information extraction method as described in any of the above embodiments.

[0206] This invention also provides a computer program product, including a computer program / instructions, which, when executed by a processor, implement the steps of the few-sample information extraction method as described in any of the above embodiments.

[0207] Those skilled in the art will understand that, for the sake of convenience and brevity, the specific working process of the system and modules described above can be referred to the corresponding process in the foregoing method embodiments, and will not be repeated here.

[0208] In the several embodiments provided in this application, it should be understood that the disclosed systems and methods can be implemented in other ways. For example, the system embodiments described above are merely illustrative; for instance, the division of units is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be an indirect coupling or communication connection through some interfaces, devices, or units, and may be electrical, mechanical, or other forms.

[0209] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0210] Furthermore, the functional units in the various embodiments of the present invention can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit.

[0211] If the integrated unit is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, in essence, or the part that contributes to the prior art, or all or part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of the present invention. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0212] The above-described embodiments are only used to illustrate the technical solutions of the present invention, and are not intended to limit it. Although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A method for extracting information from a small number of samples, characterized in that, include: Sentence samples from the sentence support set are used as input to train the initial named entity recognition model. The initial named entity recognition model includes an entity boundary detector and an entity classifier. The entity boundary detector is used to perform boundary detection on the sentence samples, output the entity boundary, and calculate the boundary detection classification loss and the boundary-aware contrast loss. The entity classifier performs entity classification using the sentence samples and entity boundaries, outputs the probability distribution of predicted entity labels embedded in multiple target entities, and calculates token comparison loss, regularization loss, prototype classification loss and entity comparison loss. The initial named entity recognition model is iteratively optimized based on the boundary detection classification loss, the boundary-aware contrast loss, the token contrast loss, the regularization loss, the prototype classification loss, and the entity contrast loss, and the model parameters are verified using a sentence query set to determine the target named entity recognition model. The sentence to be recognized is input into the target named entity recognition model for boundary detection and entity classification, and the target recognition result is output. Boundary-aware contrastive loss includes: ； ； ； In the formula, For boundary-aware contrast loss, For the sigmoid function, For entity embedding, For the beginning of entities, For the end of an entity, For cosine similarity with scaling factor, Characterized as positive samples For negative sample representation, It is an L2 norm.

2. The method for extracting information from a small number of samples according to claim 1, characterized in that, The step of performing boundary detection on the sentence samples using the entity boundary detector, outputting multiple token-embedded target location labels, and calculating the boundary detection classification loss includes: The sentence samples are converted into location token embedding sequences based on the boundary BERT embedding layer; A linear layer is used to map the location token embedding sequence to the BIOES tag space, and the predicted location tag probability distribution of each location token embedding is calculated based on the softmax function; The target location label embedded in each location token is determined by the conditional random field of Viterbi decoding based on the probability distribution of each predicted location label, and the entity boundary is determined by each target location label. The boundary detection classification loss is calculated based on the probability distribution of each predicted location label, and the corresponding boundary-aware contrast loss is calculated based on the location entity embedding, positive sample representation, and negative sample representation of each entity boundary.

3. The method for extracting information from a small number of samples according to claim 1, characterized in that, The entity classifier performs entity classification using the sentence samples and entity boundaries, outputting a probability distribution of predicted entity labels embedded in multiple target entities, and calculating token comparison loss, regularization loss, prototype classification loss, and entity comparison loss, including: The sentence samples are transformed into semantic token embedding sequences through a semantic BERT embedding layer; Determine the semantic entity embedding corresponding to the semantic token embedding sequence based on the entity boundary; An entity filtering layer is used to remove invalid entity embeddings from each semantic entity embedding to determine the target entity embedding; Through the topic prototype network, prototype embeddings are calculated based on entity embedding samples from the sentence support set, and the predicted entity label probability distribution of each target entity embedding is determined by using a temperature-scaled cosine similarity function according to each prototype embedding; wherein, the topic prototype network extends the metric-based few-shot learning principle to the domain of topic modeling, and establishes a framework that can quickly adapt to new topics while maintaining semantic coherence. It operates in a dedicated metric space in which topic prototype representations can be effectively compared and classified. Calculate the token comparison loss based on the semantic token embedding sequence using the KL divergence; The regularization loss is calculated using the prototype embeddings described above, the prototype classification loss is calculated based on the predicted entity label probability distribution, and the entity comparison loss is calculated using the target entity embeddings described above based on KL divergence.

4. The method for extracting information from a small number of samples according to claim 2, characterized in that, Boundary detection classification loss includes: ； In the formula, For boundary detection classification loss, For token index, For the total number of tokens, For cross-entropy, For the location label truth value, For the first One token, This represents the probability distribution of location labels.

5. The method for extracting information from a small number of samples according to claim 3, characterized in that, The process of determining the target entity embedding includes: ； ； In the formula, Embedded for the target entity, For entity embedding, A collection of semantic entity embeddings. For the first A collection of entities with entity category tags. For the beginning of entities, For the end of an entity, To and The beginning of different entities, To and The endings of different entities, For dynamic thresholds, To add a semantic token embedding of [CLS], This is the cosine similarity with a scaling factor.

6. The method for extracting information from a small number of samples according to claim 2 or 3, characterized in that, The process of determining entity embedding includes: ； In the formula, For entity embedding, For the beginning of entities, For the end of an entity, For linear layers, Embedding of the entity at the beginning of the entity, Embedding of entities at the end of entities, This is for concatenating vectors. The first learnable matrix OK.

7. A few-sample information extraction system, characterized in that, include: The data acquisition module is used to input sentence samples from the sentence support set into the initial named entity recognition model for model training. The initial named entity recognition model includes an entity boundary detector and an entity classifier. The boundary detection module is used to perform boundary detection on the sentence samples using the entity boundary detector, output the entity boundary, and calculate the boundary detection classification loss and the boundary-aware contrast loss. The entity classification module is used to classify entities using the sentence samples and entity boundaries through the entity classifier, output the probability distribution of predicted entity labels embedded in multiple target entities, and calculate token comparison loss, regularization loss, prototype classification loss and entity comparison loss. The model optimization module is used to iteratively optimize the model parameters of the initial named entity recognition model based on the boundary detection classification loss, the boundary-aware contrast loss, the token contrast loss, the regularization loss, the prototype classification loss, and the entity contrast loss, and to verify the model using a sentence query set to determine the target named entity recognition model. The data recognition module is used to input the sentence to be recognized into the target named entity recognition model for boundary detection and entity classification, and output the target recognition result; Boundary-aware contrastive loss includes: ； ； ； In the formula, For boundary-aware contrast loss, For the sigmoid function, For entity embedding, For the beginning of entities, For the end of an entity, For cosine similarity with scaling factor, Characterized as positive samples For negative sample representation, It is an L2 norm.

8. A computer device, characterized in that, The system includes a memory and a processor, wherein the memory stores a computer program, and when the computer program is executed by the processor, the processor causes the processor to perform the steps of the few-sample information extraction method as described in any one of claims 1-6.

9. A computer-readable storage medium having a computer program / instructions stored thereon, characterized in that, When the computer program / instructions are executed by the processor, they implement the steps of the few-sample information extraction method as described in any one of claims 1-6.

10. A computer program product comprising a computer program / instructions, characterized in that, When the computer program / instructions are executed by the processor, they implement the steps of the few-sample information extraction method as described in any one of claims 1-6.

Images