Small sample relationship classification method and system based on data enhanced triplet network

Abstract

A small sample relationship classification method and system based on a data-enhanced triplet network includes sample extraction: sampling boundary samples as a training task to generate samples that are easily confused at the classification boundary; sample data discrimination: using a triplet network to discriminate the sample data and then uniformly training to obtain the correct classification results of small sample relationships. It solves the problems of existing methods that are difficult to handle samples at the classification boundary and long-tail distribution of data, and can be widely used in the field of knowledge extraction in natural language processing.

Description

Technical Field

[0001] The present invention relates to the field of big data processing, and in particular to a small sample relationship classification method and system based on a data enhanced triplet network. Background Art

[0002] Relationship classification is a key task in natural language processing. Its primary purpose is to identify the semantic relationships between target entities in unstructured text, thereby facilitating the construction of large-scale knowledge graphs, intelligent question-answering, and downstream tasks such as semantic search. With the explosive growth of data, new relationships in specific fields are rapidly emerging. While supervised learning models offer powerful performance, they suffer from issues such as a lack of labeled data samples, labeled training samples being limited to a specific field, a narrow scope of model applicability, and a long-tail distribution of relationships between entity pairs. This is where the small-sample relationship classification problem arises. Its goal is to use a small number of labeled training samples to uncover the semantic relationships between target entities in natural language text. This addresses the resource constraints faced by traditional relationship classification methods, allowing them to be better generalized to specific fields with scarce data, such as medicine and finance.

[0003] Currently, research on small-shot relation classification all uses meta-learning training strategies to learn prior knowledge and quickly adapt to new tasks. These approaches can be broadly categorized into three approaches: those based on prototypical networks, those based on pretrained language models, and those based on parameter optimization. Prototypical network-based methods use the sample mean to calculate a prototype vector for a sample and classify it by measuring the similarity between the test sample and the prototype vector. This approach reflects a simpler inductive bias and reduces model overfitting. Pretrained language model-based methods train on large amounts of unlabeled data to learn general linguistic features and semantic information that can be transferred to small-shot tasks. Parameter optimization-based methods learn how to initialize and quickly adjust parameters from a limited number of examples to achieve optimal prediction performance across all tasks with as few gradient descent steps as possible. However, existing methods still suffer from the following issues: Due to the randomness of the sample distribution, prototypical networks struggle to correctly classify samples at the boundary of the classification; pretrained language models focus on enhancing the semantic representation of text during pretraining, but struggle to generate good relational representations; and the implicit statistics based on parameter optimization suffer from unreliable examples and poor interpretability. Summary of the Invention

[0004] In order to solve the above technical problems, the present invention provides a small sample relationship classification method and system based on data enhanced triple network.

[0005] A first aspect of an embodiment of the present application provides a small sample relationship classification method based on a data-enhanced triplet network, which includes the following steps:

[0006] Sample extraction: Sampling boundary samples as training tasks to generate samples that are easily confused at the classification boundary;

[0007] Discrimination of sample data: Use the triplet network to discriminate the sample data and then conduct unified training to obtain the correct classification results of small sample relationships.

[0008] Preferably, the sample extraction is achieved in the following manner:

[0009] Input the training samples into the pre-trained model to obtain vector encoding, and calculate the prototype vector of each class;

[0010] Calculate the distance between each sample and various prototype vectors;

[0011] Calculate the probability that each sample belongs to each type of prototype vector;

[0012] Calculate the cross entropy of samples and sort them;

[0013] Draw a specified number of samples.

[0014] Preferably, the calculation of the prototype vector is achieved by:

[0015]

[0016] Where S k is the sample set of the k-th class sample, |S k | is the number of samples of the kth class, f φ (x i ) is the text vector processed by the pre-training model, c k is the prototype vector of the k-th class sample.

[0017] Preferably, the probability of each sample belonging to each type of prototype vector is calculated by:

[0018]

[0019] Where dis represents the Euclidean distance between the sample and the prototype vector, P i,j is the probability that the jth sample belongs to class i.

[0020] Preferably, the cross entropy of the samples is calculated and sorted by:

[0021]

[0022] Where P(i) represents whether the class to which it belongs is the correct classification of sample j, and its value is 0 or 1.

[0023] Preferably, the discrimination of sample data is achieved in the following manner:

[0024] The triplet network is trained according to the samples, and the triplet network is used as the discriminant network to determine the category of the generated data samples at the boundary. The filtered data is added to the original data set, and re-trained to obtain a new prototype vector to obtain the correct classification result of the small sample relationship.

[0025] Preferably, a data enhancement method is used to generate a portion of sample points at the classification boundary to participate in the training of the triplet network. The specific method is as follows:

[0026]

[0027] In the formula, x represents the characteristics of the generated sample, y represents the class in the target task, Represents the sample statistics of the whole population, μ x is the sample mean of the samples at the classification boundary, μ is the mean of the overall samples of this type, and Σ is the variance of the overall samples of this type.

[0028] Preferably, the training of the triplet network is achieved by:

[0029] The easily confused samples extracted from the sample extraction are used as anchor samples of the triplet network, samples of the same class are used as positive samples of the triplet network, and samples of other classes are used as negative samples to train the triplet network. After inputting positive samples, negative samples and anchor samples, the text encoding is obtained through the encoder, and the triplet network is obtained after triplet loss optimization.

[0030] Preferably, the optimization of triplet loss is achieved by the following triplet network loss function:

[0031]

[0032] Where d(a,p) is the distance between the positive sample and the anchor sample, d(a,n) is the distance between the negative sample and the anchor sample, and m is the set margin.

[0033] The second aspect of the present application provides a small sample relationship classification system based on a data-enhanced triplet network, comprising:

[0034] Sample extraction module: used to sample boundary samples as training tasks and generate samples that are easily confused at the classification boundary;

[0035] Sample discrimination module: used to use the triplet network to discriminate the sample data and then conduct unified training to obtain the correct classification results of small sample relationships.

[0036] The present invention achieves the correct classification of small sample relationships through the information content of the original data set; by extracting difficult-to-classify samples based on cross entropy, using boundary samples to generate data, and then using a triplet network to discriminate the generated data and uniformly train them, thereby obtaining the correct classification results of small sample relationships; the present invention has strong generalization ability, overcoming the defects of existing methods such as excessive dependence on data, unreliable classification instances, and easy overfitting of models. It can be used in downstream fields such as knowledge graphs to improve the quality of knowledge graph construction; even when data annotation is incomplete and the number of annotated samples is small, it can still maintain good performance, thereby reducing the labor cost of data annotation and improving work efficiency. BRIEF DESCRIPTION OF THE DRAWINGS

[0037] Figure 1 A flowchart of a method for determining a data period of time series data provided in one embodiment of the present application;

[0038] Figure 2 A schematic diagram of a flow chart of a sample extraction method provided in one embodiment of the present application;

[0039] Figure 3 A schematic diagram of a triplet network model provided in one embodiment of the present application;

[0040] Figure 4 A schematic diagram of triple loss discrimination provided in an embodiment of the present application;

[0041] Figure 5 A schematic diagram comparing the method provided in one embodiment of the present application with a prototype network. DETAILED DESCRIPTION

[0042] In order to make the technical problems, technical solutions and beneficial effects to be solved by this application more clearly understood, this application is further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain this application and are not intended to limit this application.

[0043] See also Figure 1 , is a flow chart of a method for determining a data period of time series data provided in one embodiment of the present application. For ease of explanation, only the portion related to this embodiment is shown, which is described in detail as follows:

[0044] In one embodiment, a small sample relationship classification method based on a data-augmented triplet network includes the following steps:

[0045] S101. Sample extraction: Sampling boundary samples as training tasks to generate samples that are easily confused at the classification boundary;

[0046] Specifically, if Figure 2 As shown, the sample extraction is achieved by the following steps:

[0047] Since the sample distribution in small sample relationship classification is random, samples at the classification boundary are often difficult to be correctly classified. Therefore, this embodiment proposes a task construction method based on metric distance sample extraction. First, the training sample is input into the BERT pre-training model to obtain vector encoding, and the prototype vector of each class is calculated by the following formula. The prototype vector can be roughly used as the center of the sample distribution of each class in the classification task, and is used for subsequent comparison with the distance of the sample.

[0048]

[0049] Where S k is the sample set of the k-th class sample, |S k | is the number of samples of the kth class, f φ (x i ) is the text vector processed by the pre-training model, c k is the prototype vector of the k-th class sample.

[0050] During the training and testing process of the model, the small sample relationship classification task requires a support set and a query set. Most existing technologies randomly construct the support set and query set of the training task. When selecting samples to construct the support set and query set, this embodiment gives priority to samples that are easy to cause confusion. Specifically, N classes are first randomly selected from the data set as the total number of classes in the training task. When selecting samples for each class, the Euclidean distance of each sample to each class prototype vector is pre-calculated. The size of this distance is negatively correlated with the probability that the sample belongs to this class. The probability of the sample belonging to this class is defined using the following formula.

[0051]

[0052] Where dis represents the Euclidean distance between the sample and the prototype vector, P i,j is the probability that the jth sample belongs to class i.

[0053] Entropy can represent the degree of disorder of a system. To this end, the cross entropy function is used to select samples in the support set and query set. The value of H(P) is between [0,1]. When it is equal to 0, the degree of disorder is the lowest, and when it is equal to 1, the degree of disorder is the highest. That is, the higher the uncertainty of the sample category, the K and L samples with the largest cross entropy are extracted for each category in the support set and query set respectively.

[0054]

[0055] In the formula, P(i) represents whether the class to which each sample belongs is the correct classification of sample j, and its value is 0 or 1.

[0056] In summary, if Figure 2 As shown in the figure, the task construction method based on metric distance sample extraction is divided into the following steps: calculating the prototype vectors of N classes, calculating the Euclidean distance between the sample and the N prototype vectors, calculating the probability that the sample belongs to each class, calculating the cross entropy and sorting, and extracting K and L samples for the support set and query set.

[0057] S102. Sample data discrimination: The sample data is discriminated using a triplet network and then trained uniformly to obtain the correct classification results of small sample relationships.

[0058] Specifically, the triplet network is trained based on the samples, and the triplet network is used as the discriminant network to determine the category of the generated data samples at the boundary. The filtered data is added to the original data set, and retrained to obtain a new prototype vector to obtain the correct classification result of the small sample relationship; the discrimination of the sample data is achieved in the following way:

[0059] The difficulty of small sample relationship classification lies in the insufficient number of labeled samples in the training set, and the samples at the classification boundary mainly affect the classification accuracy. Therefore, the embodiment of the present application adopts a data enhancement method to generate a portion of sample points at the classification boundary to participate in the training of the triple network. For the class in the target task, the overall sample statistics extracted from step S101 are sampled from the Gaussian distribution of this class to generate a set of labeled feature vectors. The specific method is as follows:

[0060]

[0061] In the formula, x represents the characteristics of the generated sample, y represents the class in the target task, Represents the sample statistics of the whole population, μ x is the sample mean of the samples at the classification boundary, μ is the mean of the overall samples of this type, and Σ is the variance of the overall samples of this type.

[0062] Since the generation of sample data is random, it is necessary to screen the generated data. The triplet network is an extension of the twin neural network and is suitable for solving the problem of a large number of sample categories or uncertain samples, while the number of samples in the training data set is small. The small sample relationship classification task just meets the above conditions. The easily confused samples extracted in step S101 are used as anchor samples of the triplet network, samples of the same class are used as positive samples of the triplet network, and samples of other classes are used as negative samples to train the triplet network. Figure 3 The figure shows a schematic diagram of the triplet network model. After inputting positive samples, negative samples and anchor samples, the text encoding is obtained through the encoder, and the triplet network is obtained after triplet loss optimization.

[0063] The optimization of triplet loss is achieved through the following loss function of the triplet network:

[0064]

[0065] Where a is the sample located at the classification boundary extracted in step S101, p is the positive example sample of the same class as a, and n is the negative example sample of a different class from a; d(a, p) is the distance between the positive sample and the anchor sample, d(a, n) is the distance between the negative sample and the anchor sample, and m is the set margin, whose initial value is set according to experience, such as Figure 4 shown, when d(a, p) > d(a, n) - m, At this time, there are two cases that need to be optimized, such as Figure 4 (a), when d(a, p) > d(a, n), optimization is required, such as Figure 4 (b), when d(a, p) < d(a, n), and d(a, p) > d(a, n) - m, so optimization is also required, such as Figure 4 (c), when d(a, p) < d(a, n) - m, no optimization is required. Optimizing the result of this loss function will make the distance between samples with the same label in the vector space as small as possible, which is more suitable for the task objective.

[0066] Such as Figure 5 shown, the trained triplet network will be used as a discriminative network to determine the class of the generated data samples at the boundary, and the filtered data will be added to the original data set for retraining.

[0067] Such as Figure 5 (a), the X region represents the prototype vector calculated from the support set. Since the given query instance with the true label in the Y region is closest to the estimated prototype vector in the Z region, the prototype network misclassifies it as the Z region, which is essentially attributed to the randomness of sampling the support set, such as Figure 5 (b), generating data at the classification boundary in the Y region, and obtaining the square generated data through the discrimination of the triplet network, such as Figure 5 (c), training the generated data and the original class data together to obtain a new prototype, which can correctly classify the sample points at the boundary.

[0068] The second aspect of this application provides a few-shot relation classification system based on a data-augmented triplet network, including:

[0069] The second aspect of this application provides a few-shot relation classification system based on a data-augmented triplet network, including a sample extraction module and a sample discrimination module.

[0070] Sample extraction module: used to sample boundary samples as training tasks to generate easily confused samples at the classification boundary;

[0071] Sample discrimination module: used to use the triplet network to discriminate the sample data and then conduct unified training to obtain the correct classification results of small sample relationships.

[0072] It should be noted that the small sample relationship classification system based on data-enhanced triplet network in this embodiment is an embodiment of the system corresponding to the small sample relationship classification based on data-enhanced triplet network mentioned above. Therefore, the specific implementation of the software method in each module of the system can be referred to. Figure 1-Figure 5 The embodiments are not described in detail here.

[0073] The present invention provides a small sample relationship classification method and system based on a data-enhanced triplet network. It can achieve correct classification of small sample relationships only through the information content of the original data set without the help of external information. By extracting difficult-to-classify samples based on cross entropy, generating data using boundary samples, and then using the triplet network to discriminate the generated data and uniformly train them, the correct classification results of small sample relationships are obtained. The method provided by this application has strong generalization ability, overcomes the defects of existing methods such as excessive reliance on data, unreliable classification instances, and easy overfitting of models. It can be used in downstream fields such as knowledge graphs to improve the quality of knowledge graph construction. In the case of incomplete data annotation and a small number of annotated samples, it can still maintain good performance, thereby reducing the human cost of data annotation and improving work efficiency.

[0074] Those skilled in the art will appreciate that the units and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this application.

[0075] The above-described embodiments are only used to illustrate the technical solutions of the present application, rather than to limit them. Although the present application has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. These modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the various embodiments of the present application, and should all be included in the scope of protection of the present application.

Claims

1. A small sample relationship classification method based on data-enhanced triplet networks, characterized by: The following steps are involved: Sample extraction: Sampling boundary samples as training tasks to generate samples that are easily confused at the classification boundary; Discrimination of sample data: Use triplet networks to discriminate sample data and then conduct unified training to obtain the correct classification results of small sample relationships; The extraction of the sample is specifically achieved by the following method: Input the training samples into the pre-trained model to obtain vector encoding, and calculate the prototype vector of each class; Calculate the distance between each sample and various prototype vectors; Calculate the probability that each sample belongs to each type of prototype vector; Calculate the cross entropy of samples and sort them; Taking a prescribed number of samples; The calculation of the prototype vector is achieved in the following way: Where S k is the sample set of the k-th class sample, |S k | is the number of samples of the kth class, f φ (x i ) is the text vector processed by the pre-training model, c k is the prototype vector of the k-th class sample; Calculate the probability that each sample belongs to each type of prototype vector, which is achieved by: Where dis represents the Euclidean distance between the sample and the prototype vector, P i,j is the probability that the jth sample belongs to class i; Calculate the cross entropy of the samples and sort them, which can be achieved by: Where P(i) represents whether the class to which it belongs is the correct classification of sample j, and its value is 0 or 1.

2. The small sample relationship classification method based on data-enhanced triplet network according to claim 1 is characterized in that: The discrimination of sample data is achieved through the following methods: The triplet network is trained according to the samples, and the triplet network is used as the discriminant network to determine the category of the generated data samples at the boundary. The filtered data is added to the original data set, and re-trained to obtain a new prototype vector to obtain the correct classification result of the small sample relationship.

3. The small sample relationship classification method based on data-enhanced triplet network according to claim 2 is characterized in that: The data enhancement method is used to generate a part of the sample points at the classification boundary to participate in the training of the triplet network. The specific method is as follows: In the formula, x represents the characteristics of the generated sample, y represents the class in the target task, Represents the sample statistics of the whole population, μ x is the sample mean of the samples at the classification boundary, μ is the mean of the overall samples of this type, and Σ is the variance of the overall samples of this type.

4. The small sample relationship classification method based on data-enhanced triplet network according to claim 3 is characterized in that: The training of the triplet network is achieved by: The easily confused samples extracted from the sample extraction are used as anchor samples of the triplet network, samples of the same class are used as positive samples of the triplet network, and samples of other classes are used as negative samples to train the triplet network. After inputting positive samples, negative samples and anchor samples, the text encoding is obtained through the encoder, and the triplet network is obtained after triplet loss optimization.

5. The small sample relationship classification method based on data-enhanced triplet network according to claim 4 is characterized in that: The optimization of triplet loss is achieved through the following loss function of the triplet network: Where d(a,p) is the distance between the positive sample and the anchor sample, d(a,n) is the distance between the negative sample and the anchor sample, and m is the set margin.

6. A small sample relationship classification system based on data-enhanced triplet networks, characterized by: A method for implementing a small sample relationship classification method based on a data-enhanced triplet network according to any one of claims 1 to 5, comprising: Sample extraction module: used to sample boundary samples as training tasks and generate samples that are easily confused at the classification boundary; Sample discrimination module: used to use the triplet network to discriminate the sample data and then conduct unified training to obtain small sample relationships.

Images