A method, apparatus and equipment for classifying and filtering relationships in small samples

Abstract

This invention provides a method, apparatus, and device for small-sample relation classification filtering. The method includes: constructing a coarse-grained filter to filter an unlabeled corpus to obtain candidate instances with the same entity and similar semantics as a seed instance; constructing a fine-grained filter to filter the candidate instances to obtain candidate instances with the same relational concept as the seed instance; defining the candidate instances as a positive instance set and the candidate instances with different relational concepts as a negative instance set; constructing a false positive instance correction module to adjust and control the proportion of the negative instance set used by the classifier during training; and training the classifier based on a small number of labeled instances belonging to newly emerging relations, as well as the positive instance set and negative instance set adjusted by the false positive instance correction module, so that it can classify relations in small-sample data. The method of this invention can extract high-quality instances to train the classifier and improve the accuracy of the classifier.

Description

Technical Field

[0001] This invention relates to the field of data filtering and classification technology, and in particular to a method, apparatus and device for small sample relationship classification and filtering. Background Technology

[0002] In recent years, information technology and the internet have experienced explosive growth, resulting in a vast amount of unstructured text online containing information. Information extraction (IE) involves extracting valuable knowledge from this text, including named entity recognition, event detection (ED), and relation classification (RC). Among these tasks, relation classification, which aims to detect the relationship between a pair of entities contained in a sentence, has significant practical value in reality, providing input for many downstream applications (knowledge graphs or question-answering systems). Figure 1 As shown, sentence S1 contains two entities: the head entity "highway" and the tail entity "river". The purpose of relation classification is to determine whether there is an "intersecting" relationship between these two entities. Due to its considerable commercial value, relation classification has become one of the most prominent research topics in information extraction. However, the scarcity of annotated instances limits the effectiveness of data-driven relation classification methods, especially for emerging relations with only a few instances.

[0003] Therefore, existing methods mainly improve the performance of few-shot data classification models from two aspects: increasing the quantity and quality of annotations. To increase the number of instances, many researchers have chosen to use semi-supervised learning and distant supervision. Distant supervision uses entity alignment to automatically and continuously label unlabeled instances from external databases. However, the strong relation assumption of distant supervision, "if two sentences contain the same instance, then they represent the same relation," often introduces a lot of noise into the model. Therefore, existing schemes propose a neural snowball algorithm, which is a distant supervision method. To address the noise problem, it uses a relational Siamese network (RSN) to determine whether they have the same relation by measuring the distance between query instances and supporting seed instances. In addition, cue learning has gained widespread attention for improving the quality of instances. Templates in cue learning can store rich semantics and stimulate the common sense of pre-trained language models (PLMs), making them excellent performers in few-shot learning tasks. Many cue-based methods have been proposed, such as PRBOOST, PTR, and PAIE. However, these methods only focus on individual sentences and local-level features, such as entity pairs, while ignoring the rich information of global-level conceptual semantics. Over-reliance on single pieces of information is likely to lead to semantic bias, which will hinder a comprehensive and accurate overall understanding of the semantics of relations, especially in the context of few-shot learning. Summary of the Invention

[0004] The problem to be solved by the present invention is to provide a method, apparatus and equipment for small sample relationship classification and filtering that can solve the problem of scarce small sample training data and improve the recognition accuracy of small samples.

[0005] To address the above problems, this invention discloses a small sample relationship classification and filtering method, comprising:

[0006] A coarse-grained filter is constructed to filter the unlabeled corpus to obtain a first candidate instance with the same entity as the seed instance and a second candidate instance with similar semantics.

[0007] A fine-grained filter is constructed to filter the first candidate instance and the second candidate instance to obtain a third candidate instance with the same relational concept as the seed instance;

[0008] The third candidate instance is defined as the positive instance set, and the candidate instances in the first and second candidate instances whose relationship concept is different from that of the seed instance are defined as the negative sample set. The relationship concept is used to describe the relationship between different instances.

[0009] A false positive instance correction module is constructed, which is used to adjust and control the proportion of the negative sample set used by the classifier during training;

[0010] The classifier is trained based on a small number of labeled instances belonging to newly emerging relationships, as well as the set of positive instances and the set of negative samples adjusted by the false positive instance correction module.

[0011] Relationship classification is performed on small sample data based on the trained classifier.

[0012] As an optional embodiment, constructing the coarse-grained filter includes:

[0013] An entity alignment module is set up to identify the first candidate instance in the corpus that has the same entity as the seed instance;

[0014] A relational twin network is set up to filter out second candidate instances with similar semantics to the seed instance by measuring the distance between the word vectors of instances in the corpus and the seed instance;

[0015] The coarse-grained filter is formed based on the entity alignment module and the relational twin network.

[0016] As an optional embodiment, the seed instances include multiple instances, and the step of filtering out second candidate instances with similar semantics to the seed instances by measuring the distance between the instances in the corpus and the word vectors in the seed instances includes:

[0017] The relational twin network determines the similarity score between two instances by measuring the Euclidean distance between word vectors in the instances in the corpus and the seed instance:

[0018]

[0019] The average similarity score is calculated based on the similarity scores between each instance in the corpus and each seed instance.

[0020] Second candidate instances with similar semantics to each of the seed instances are filtered out based on multiple average similarity scores;

[0021] Where σ(·) is the sigmoid activation function, f p (·) is used to encode the sentence vector, S RSN The range of (·) is 0 to 1, and the weight w p and deviation b p These are trainable parameters. It is a seed instance. It is an instance in the corpus.

[0022] As an optional embodiment, the construction of the fine-grained filter includes:

[0023] Set a prompt template generation function to convert the first candidate instance and the second candidate instance into new text that meets the requirements of the input template. The input template includes the text description, relational concepts and tag bar of the candidate instance. The tag bar is used to add tags to indicate whether the relational concepts of the candidate instance and the seed instance are consistent.

[0024] A pre-trained language model is used to process new text corresponding to the first candidate instance and the second candidate instance based on the relational concept of the seed instance, so as to add the tags matching each new text to the tag bar based on the processing result, and to filter the first candidate instance and the second candidate instance based on the tags to obtain the third candidate instance.

[0025] As an optional embodiment, it also includes:

[0026] Based on the obtained labeled instances that satisfy frequently occurring relationships, the coarse-grained filter, fine-grained filter, and classifier are pre-trained respectively.

[0027] The positive instance set and negative sample set are obtained by filtering the corpus based on the pre-trained coarse-grained filter and fine-grained filter.

[0028] The pre-trained classifier is trained based on the positive instance set, the negative sample set, and a small number of labeled instances belonging to newly emerging relationships.

[0029] As an optional embodiment, the pre-training of the coarse-grained filter, fine-grained filter, and classifier based on the obtained labeled instances satisfying frequently occurring relationships includes:

[0030] Based on any two instances from the labeled instances that satisfy the frequently occurring relationship, the coarse-grained filter is pre-trained using cross-entropy loss;

[0031] Different input templates are constructed, and the fine-grained filter is pre-trained based on the different input templates and labeled instances that satisfy common relationships. The input template includes template format content, text descriptions in candidate instances, relationship concepts, and a tag bar. The tag bar is used to add tags to indicate whether the relationship concepts between the candidate instance and the seed instance are consistent.

[0032] The classifier is pre-trained based on a few-shot learning model and labeled instances that satisfy frequently occurring relationships.

[0033] As an optional embodiment, constructing different input templates includes:

[0034] The system constructs input templates for text descriptions and relational concepts in different positions, input templates lacking some or all of the template format content, and input templates that form negative templates by changing relational concepts.

[0035] As an optional embodiment, it also includes:

[0036] Based on the false positive instance correction module, the parameters and loss function of the classifier are corrected as follows:

[0037]

[0038] Among them, B pos B is a batch of positive cases. neg These are negative instances within a set of negative samples. For the classifier, μ is the negative sampling loss coefficient, used to control the proportion of negative instances used by the classifier during training.

[0039] Another embodiment of the present invention also provides a small sample relationship classification filtering device, comprising:

[0040] The first construction module is used to construct a coarse-grained filter, which is used to filter the unlabeled corpus to obtain a first candidate instance with the same entity as the seed instance and a second candidate instance with similar semantics;

[0041] The second construction module is used to construct a fine-grained filter, which is used to filter the first candidate instance and the second candidate instance to obtain a third candidate instance with the same relational concept as the seed instance.

[0042] The definition module is used to define the third candidate instance as a positive instance set, and to define the candidate instances in the first and second candidate instances whose relationship concept is different from that of the seed instance as a negative sample set, wherein the relationship concept is used to describe the relationship between different instances;

[0043] The third construction module is used to construct a false positive instance correction module, which is used to adjust and control the proportion of the negative sample set used by the classifier during training.

[0044] The training module is used to train the classifier based on a small number of labeled instances belonging to newly emerging relationships, as well as the set of positive instances and the set of negative samples adjusted by the false positive instance correction module.

[0045] The classification module is used to classify relationships in small sample data based on the trained classifier.

[0046] Another embodiment of the present invention also provides an electronic device, comprising:

[0047] At least one processor; and,

[0048] A memory communicatively connected to the at least one processor; wherein,

[0049] The memory stores instructions that can be executed by the at least one processor to implement the small sample filtering classification method as described in any of the embodiments above.

[0050] Based on the disclosure of the above embodiments, it can be understood that the beneficial effects of the embodiments of the present invention include capturing high-quality instances from the corpus by employing coarse-grained and fine-grained dual-layer filters, expanding the training data, improving the training accuracy of the classifier, and avoiding the noise problem that occurs in existing semi-supervised learning. Furthermore, the method of this embodiment utilizes changing the input text to new text that conforms to the requirements of the input template to help the classifier understand the conceptual connection between the input text and the seed instances, thereby improving the accuracy of the classifier in classifying small sample relationships. Moreover, in order to improve the ability of the input template to identify non-attributive relationships between instances, the method of this embodiment also introduces different input templates during the pre-training stage of the fine-grained filter, including negative templates and positive templates with different content. In addition, to alleviate the overfitting problem during the classifier training process, the method of this embodiment also adds a false positive instance correction module to dynamically adjust the proportion of negative samples participating in training, thereby reducing the damage of positive instances to the classifier performance caused by overfitting.

[0051] Other features and advantages of the invention will be set forth in the description which follows, and will be apparent in part from the description, or may be learned by practicing the invention. The objects and other advantages of the invention may be realized and obtained by means of the structures particularly pointed out in the written description, claims, and drawings.

[0052] The technical solution of the present invention will be further described in detail below with reference to the accompanying drawings and embodiments. Attached Figure Description

[0053] The accompanying drawings are provided to further illustrate the invention and form part of the specification. They are used in conjunction with embodiments of the invention to explain the invention and do not constitute a limitation thereof. In the drawings:

[0054] Figure 1 This is a schematic diagram of an example of relation classification.

[0055] Figure 2 This is a flowchart of the small sample filtering and classification method in an embodiment of the present invention.

[0056] Figure 3 This is the data distribution obtained from an unlabeled corpus in an embodiment of the present invention.

[0057] Figure 4 This is a flowchart illustrating the process of the coarse-grained to fine-grained filter model in an embodiment of the present invention.

[0058] Figure 5 This is a diagram illustrating the working process of the fine-particle filter in an embodiment of the present invention.

[0059] Figure 6 A diagram illustrating the process of filtering out false positive instances using a fine-grained filter (CHP).

[0060] Figure 7 This is a structural block diagram of the small sample filtering and classification device in an embodiment of the present invention. Detailed Implementation

[0061] The specific embodiments of the present invention will now be described in detail with reference to the accompanying drawings, but these are not intended to limit the scope of the invention.

[0062] It should be understood that various modifications can be made to the embodiments disclosed herein. Therefore, the following description should not be considered as limiting, but merely as an example of embodiments. Other modifications within the scope and spirit of this disclosure will be apparent to those skilled in the art.

[0063] The accompanying drawings, which are included in and form part of this specification, illustrate embodiments of the present disclosure and, together with the general description of the disclosure given above and the detailed description of the embodiments given below, serve to explain the principles of the disclosure.

[0064] These and other features of the invention will become apparent from the following description of preferred forms of embodiments given as non-limiting examples, with reference to the accompanying drawings.

[0065] It should also be understood that although the invention has been described with reference to some specific examples, those skilled in the art can certainly implement many other equivalent forms of the invention, which have the features described in the claims and are therefore all within the scope of protection defined herein.

[0066] The above and other aspects, features and advantages of this disclosure will become more apparent when taken in conjunction with the accompanying drawings and in view of the following detailed description.

[0067] Specific embodiments of the present disclosure are described thereafter with reference to the accompanying drawings; however, it should be understood that the disclosed embodiments are merely examples of the present disclosure, which may be implemented in various ways. Well-known and / or repeated functions and structures are not described in detail to avoid unnecessary or redundant details that could obscure the present disclosure. Therefore, the specific structural and functional details disclosed herein are not intended to be limiting, but merely to serve as the basis and representative basis for the claims to teach those skilled in the art to use the present disclosure in a variety of substantially any suitable detailed structures.

[0068] This specification may use the phrases “in one embodiment,” “in another embodiment,” “in yet another embodiment,” or “in still another embodiment,” all of which may refer to one or more of the same or different embodiments according to this disclosure.

[0069] The embodiments of the present invention will now be described in detail with reference to the accompanying drawings.

[0070] like Figure 1As shown, this embodiment of the invention provides a small sample relationship classification and filtering method, including:

[0071] S100: Construct a coarse-grained filter. The coarse-grained filter is used to filter the unlabeled corpus to obtain the first candidate instance with the same entity as the seed instance and the second candidate instance with similar semantics.

[0072] S101: Construct a fine-grained filter. The fine-grained filter is used to filter the first candidate instance and the second candidate instance to obtain a third candidate instance with the same relationship concept as the seed instance.

[0073] S102: Define the third candidate instance as the positive instance set, and define the first candidate instance and the candidate instances in the second candidate instance whose relationship concept with the seed instance is different as the negative sample set. The relationship concept is used to describe the relationship between different instances.

[0074] S103: Construct a false positive instance correction module, which is used to adjust and control the proportion of negative samples used by the classifier during training;

[0075] S104: Train the classifier based on the small number of labeled instances belonging to newly emerging relationships that have been obtained, as well as the set of positive instances and negative samples adjusted by the false positive instance correction module;

[0076] S105: Classify relationships based on a trained classifier for small sample data.

[0077] Based on the disclosure of the above embodiments, it can be understood that the method disclosed in this embodiment has the following beneficial effects: by employing a coarse-grained and fine-grained two-layer filter to capture high-quality instances from the corpus, it expands the training data, improves the training accuracy of the classifier, and avoids the noise problem that occurs in existing semi-supervised learning. In addition, in order to alleviate the overfitting problem during the classifier training process, the method of this embodiment also adds a false positive instance correction module to dynamically adjust the proportion of negative samples participating in training, thereby reducing the damage of positive instances to the overfitting of the classifier performance.

[0078] The method disclosed in this embodiment enables the classifier to achieve high classification accuracy even with low resources. The recall and F1 score (F1 score is a measure of classification problems, which is the harmonic mean of precision and recall) are also significantly improved relative to the baseline. It truly enables the classifier to think like a human, overcomes the limitations related to local information in the input data, and achieves high-precision relation classification regardless of the length of the input text sequence, the number of seed instances, or the interference of false positive instances.

[0079] Furthermore, the main distribution characteristics of existing relation classification data are as follows: 1) A certain number of labeled instances D belonging to frequently occurring relations. pre ;2) A small number of annotation instances D that belong to newly emerging relationships. n 3) A large number of unlabeled corpora / sets u The method in this embodiment combines semi-supervised few-shot learning to fully utilize the three types of data mentioned above to train the classifier. In general, the method in this embodiment includes two stages; the first stage is data augmentation. For example, from D... n Extract a series of seed instances x s , including its entity pairs [e h e t ], Relationship label r s The concept of relationship c s Next, all unlabeled instances x u ∈D u The unlabeled instances will be compared with seed instances using predefined matching rules. If these unlabeled instances meet the criteria, they will be selected and fed into the model to train the classifier. (symbols in other positions) (Both are classifiers). In addition, this embodiment also includes a specially designed prompt template generation function, as shown below:

[0080] f T (x u c s [Mask])→T(x u )

[0081] Wherein, template T(x) u The concept of inclusion relation c s Input text x u (Text descriptions in candidate instances) and special markers [Mask] (labels). The labels in [Mask] that represent the relationships between instances are the criteria for determining whether an instance should be labeled, that is, whether the candidate instance has the same relationship concept as the seed instance. The second stage is the testing process for new relationships. Unlike traditional few-sample relationship classification, the method in this embodiment tests the relationships from D n D u and D pre The input text x and its corresponding entity pair e h and e t The model aims to classify an unlabeled instance into a new relation label, as follows:

[0082]

[0083] in This represents the set of all related relationship labels.

[0084] like Figure 3 As shown, the rectangular boxes represent the unlabeled corpus; the left circle represents the positive instances identified by the coarse-grained filter – the first candidate instances; the right circle marks the positive instances identified by the coarse-grained filter – the second candidate instances; and the overlapping shaded area passing through the two circles represents the truly positive instances identified by the fine-grained filter – the third candidate instances. The goal of the classifier model in this embodiment is to obtain as many instances as possible labeled by the overlapping shaded areas from the unlabeled corpus.

[0085] Specifically, the classifier in this embodiment uses a snowball structure as its model framework to identify matching relation instances from an external, unlabeled corpus. When executing the method flow in this embodiment, a coarse-grained filter first needs to be constructed, including:

[0086] S106: Set up an entity alignment module, which is used to identify the first candidate instance in the corpus that has the same entity as the seed instance;

[0087] S107: Set up a relational twin network, which is used to filter out second candidate instances with similar semantics to the seed instance by measuring the distance between the instance in the corpus and the word vector in the seed instance;

[0088] S108: A coarse-grained filter is formed based on the entity alignment module and relational twin network.

[0089] For example, such as Figure 4 As shown, the coarse-grained filter performs a coarse-grained snowballing workflow, which includes an entity alignment module and a relational twin network (RSN module). The coarse-grained filter can process data from D... n Randomly select several seed instances Form a seed instance set D s Each instance corresponds to a relation r. s ∈R n and a relation concept c s ∈C n At the same time, a set of entity pairs E is formed. s Then D u Each input instance (input sentence) in the dataset. In the entity alignment module and RSN module respectively Compare them.

[0090] Specifically, the coarse-grained filter uses an entity alignment module to identify seed instances. Input instances sharing the same entity This process can be described as follows:

[0091]

[0092] Where e h and e t yes Entity pairs. E align (·) decided and Whether the entity pairs match. ε pos_ent (·) refers to the set of matched entity pairs. Then, satisfying E... align Examples Will be added to candidate instance set D pos-ent In this case, the first candidate instance is formed.

[0093] However, the entity alignment module, by its very nature, acts as a coarse-grained filter, often extracting only elements that are... Instances that share entities but express different relationships will filter out noisy data. For example, sentence #1 "Bill Gates founded Microsoft" and sentence #2 "Bill Gates announced his departure from Microsoft" share the same entity pair, "Bill Gates" and "Microsoft." However, the two entities in the different sentences have different relationships: "founded" and "departed," respectively. Therefore, the positive instances (first candidate instances) obtained at this stage will be sent to the CHP (fine-grained filter) for further filtering.

[0094] Through filtering by the entity alignment module, some instances with new relationships have been collected from the corpus. However, the above process still misses entity pairs and... These are different instances that represent the same relationship, such as "Bill Gates founded Microsoft" and "Steve Jobs founded Apple." In fact, there are many such instances in practical applications.

[0095] Therefore, this embodiment employs a Relational Siamese Network (hereinafter referred to as RSN) to capture this type of missed instances. As can be seen above, there are multiple seed instances. In this embodiment, the RSN filters out second candidate instances with similar semantics to the seed instances by measuring the distance between instances in the corpus and word vectors in the seed instances. These include:

[0096] S109: Relational Siamese Networks determine the similarity score between two instances by measuring the Euclidean distance between word vectors in the corpus and the seed instance.

[0097]

[0098] S110: Calculate the average similarity score based on the similarity scores between each instance and various sub-instances in the corpus;

[0099] S111: Filter out second candidate instances with similar semantics to various sub-instances based on multiple average similarity scores;

[0100] Where σ(·) is the sigmoid activation function, f p (·) is used to encode the sentence vector, S RSN The range of (·) is 0 to 1, and the weight w p and deviation b p These are trainable parameters. It is a seed instance. It is an instance in the corpus.

[0101] Specifically, in this embodiment, RSN is equivalent to calculating a seed instance. and query examples The distance between instances in the corpus is used to determine the probability that they share the same relation:

[0102]

[0103] A higher similarity score (probability) indicates a greater likelihood that the two instances express the same relationship. After obtaining the similarity score, the average score of the query instance against various seed instances can be calculated:

[0104]

[0105] Where n is the number of seed instances. Optionally, in this embodiment, the top K instances ranked from highest to lowest score are selected. As a positive instance dataset D pos-RSN This is the second candidate instance. Alternatively, a threshold for the RSN can be set, such as 0.5, to filter and obtain second candidate instances based on this threshold and multiple average similarity scores. These second candidate instances are not all accurate positive instances; they must simultaneously meet the criteria of having the same entity pairs and similar semantics. Therefore, they also need to be filtered again using a fine-grained filter.

[0106] Furthermore, the fine-grained filter in this embodiment is based on the concept of concept heuristic cues (CHP), inspired by the logical reasoning humans use to understand new concepts. For example, for the sentence "Nick founded a welfare school to teach children science," people tend to focus on the word "found," rather than entities (vocabulary) such as "Nick" and "welfare school." Therefore, the meaning of the relationship "found" is crucial. Inspired by this concept, this embodiment constructs a fine-grained filter, including:

[0107] S112: Set the prompt template generation function to convert the first candidate instance and the second candidate instance into new text that meets the requirements of the input template. The input template includes the text description, relational concepts and label bar of the candidate instance. The label bar is used to add labels to indicate whether the relational concepts of the candidate instance and the seed instance are consistent.

[0108] S113: A pre-trained language model used to process new text corresponding to the first and second candidate instances based on the relational concepts of seed instances. Based on the processing results, the model adds the labels of each new text to the label bar and filters the first and second candidate instances based on the labels to obtain the third candidate instance.

[0109] Specifically, the function f that generates the prompt template is set. T Candidate instances extracted by the coarse-grained filter can be transformed into new text that meets the requirements of the input template, namely, text descriptions of candidate instances, relational concepts, and a tag bar. Then, a pre-trained language model (PLM) is used to process the new text to obtain tags that characterize whether the current candidate instance and the seed instance have the same relational concept. For details, please refer to [link / reference]. Figure 5 As shown, Figure 5 The text describes the template generation function f given an input x. T This is converted into new text that includes relational concepts, and then the new text is sent to the PLM to obtain the hidden embedding of the [Mask] (tag) markup.

[0110] For D pos-ent The first candidate instance derived by the entity alignment module Given the concept of seed relation and seed tags The input template (new text) is constructed as follows:

[0111]

[0112] The new text is then entered into PLM to populate the answer for [Mask], and then confirmed. Does it signify relation r? s For instances that are considered to constitute this relationship, they can be labeled with a weak positive label and then appended to the exact set of positive instances D. pos In this context, the above examples must meet the following conditions:

[0113]

[0114]

[0115]

[0116] The above equation shows that [Mask] labeled h[Mask] can be mapped to a seed relation. The label. Regarding false positive instances, they can be appended to the negative sample set D. neg In the middle, as shown below:

[0117]

[0118] in This indicates that the filtering conditions of the fine-grained filter CHP were not met. Through the above process, reliable, high-quality positive instances (accurate positive instances) can be collected from external unlabeled corpora to increase the annotation data (training data). Simultaneously, identified inducing instances with high error rates are clustered to form a negative sample set. Using this negative sample set for classifier training enhances the model's robustness, achieving fine-grained filtering.

[0119] Furthermore, similar to the process described above, the filtered second candidate instance D pos-RSN The data is fed into the CHP filter and ultimately assigned a weak label. Then, based on the filtering results, the second candidate instance is divided into a positive instance set D. pos (Third candidate instance) and a negative sample set D neg As shown below:

[0120]

[0121]

[0122]

[0123]

[0124] The first two equations represent the filtering conditions for RSN and CHP, respectively. Following the above procedure, the maximum amount of data can be collected. Figure 3 The data in the shaded overlapping area. Since the above input template is based on the relational concept heuristic, it allows fine-grained filters to fully learn the semantic relationship between labels and input text under low-resource conditions, effectively identify false positive instances, and effectively stimulate the rich knowledge hidden in PLM.

[0125] For example, see reference. Figure 6The left figure illustrates a case where an instance conforms to the entity alignment criteria but not the concept heuristic criteria. As shown in the figure, the seed instance and the unlabeled instance have the same entity pair "MyFreedom, Make some noise," but represent different relations. In the seed instance, the relation of the entity pair is "partof," indicating that "My Freedom" is a song in the album "Make some noise." Conversely, in the unlabeled instance, their relation is "follows," indicating that "My Freedom" follows the song "Make some noise."

[0126] Figure 6 The right figure shows that RSN misclassified two instances belonging to different relations. According to the principle of RSN, this misclassification is because the two sentences are structurally comparable; they both express a person's occupational attributes, while the input template based on concept heuristics helps the filter classify them correctly. Therefore, the use of concept heuristics not only helps the filter identify true positive instances but also detects false positive instances, which is crucial for correcting the model's classification ability.

[0127] Furthermore, in another embodiment, the method further includes:

[0128] S114: Based on the obtained labeled instances that satisfy frequently occurring relationships, pre-train the coarse-grained filter, fine-grained filter, and classifier respectively;

[0129] S115: The corpus is filtered using pre-trained coarse-grained and fine-grained filters to obtain a positive instance set and a negative sample set;

[0130] S116: Train the pre-trained classifier based on the positive instance set, the negative sample set, and a small number of labeled instances belonging to newly emerging relationships.

[0131] Specifically, based on the labeled instances that satisfy frequently occurring relationships, the coarse-grained filter, fine-grained filter, and classifier are pre-trained, including:

[0132] S117: Based on any two instances in the labeled instances that satisfy the frequently occurring relationship, pre-train the coarse-grained filter by combining cross-entropy loss;

[0133] S118: Construct different input templates and pre-train the fine-grained filter based on different input templates and labeled instances that satisfy common relationships. The input template includes template format content, text descriptions in candidate instances, relationship concepts, and a label bar. The label bar is used to add labels to indicate whether the relationship concepts between the candidate instance and the seed instance are consistent.

[0134] S119: Pre-train the classifier based on the few-shot learning pattern and labeled instances that satisfy frequently occurring relationships.

[0135] When constructing different input templates, including:

[0136] S120: Construct input templates for text descriptions and relational concepts in different locations, input templates lacking some or all of the template format content, and input templates that form negative templates by changing relational concepts.

[0137] For example, in order to enable the classifier to better transfer instances to newly emerging relations, this embodiment proposes to apply this to a large number of existing relations D. pre Pre-trained RSN module and classifier and CHP filters. For This embodiment pre-trains it using the traditional N-way K-shot method. For RSN, this embodiment starts from D. pre Two sentences are randomly selected from the dataset, and then trained using cross-entropy loss. For the CHP filter, this embodiment improves its performance by setting three positive templates and one negative template during training.

[0138] Specifically, this embodiment sets up three positive prompt templates T i (·), i = 1, 2, 3, and in a large number of existing relations R pre Training. For example, given an input text. and its relationship and using relational concepts The following is a template for establishing a relationship concept:

[0139] The template includes a description of the input text, relational concepts, a label bar, and template formatting content, as shown in the table below:

[0140]

[0141]

[0142] The phrase "This sentence [] contains this relationship []. This statement is []" is the template format content. The first three templates have different format content, and the positions of the input text and relational concepts in T1 and T2 are also different. This setting is to explore the impact of the input text and relational concepts in different positions on the fine-grained filter. The T3 template is set to explore whether the template needs additional vocabulary beyond the key semantics, such as whether template format content is required. Continuing to combine... Figure 5 As shown, by using the template By inputting PLM, we can obtain the hidden vector h[Mask] of v. Given the embedding of token v∈V, we can calculate the probability of v filling the [Mask] position:

[0143]

[0144] The goal of the fine-grained filter is to respond to v[mask] with words like "yes," "correct," or "right," indicating that the input sentence contains the seed relation, or is considered to have the same relation concept as the seed instance. Conversely, if v[mask] is "no," "incorrect," or "false," it means there is no relation. Then, the vectors of these words can be mapped to the corresponding relations:

[0145]

[0146] in It is a label mapping function Y→V, where Y is the true value. However, constructing only positive instances (positive instances) for each relation introduces semantic bias. These instances provide little benefit and harm the model. To improve the robustness of the model, this embodiment also performs pre-training of fine-grained filters for each relation r. i Positive and negative templates were created, namely the last two templates in the table above, to correct the semantic skew caused by an excessive number of positive templates. In the negative templates, the relational concept is... It has been changed to The negative template is primarily used to determine whether an instance does not belong to a specified relation. This capability is crucial for later porting this fine-grained filter to a filter network, as most unlabeled samples do not belong to the relation being evaluated. As previously noted, without a negative template, the overall performance of the model will be significantly reduced. Through training with the different templates described above, the fine-grained filter can achieve its final learning objective—minimizing the loss function.

[0147]

[0148] Where L is the cross-entropy loss; N x This refers to the number of input instances. Additionally, during training, the classification of each relation can be set as a binary classification task to improve model efficiency and facilitate the addition of new relation types.

[0149] Preferably, the coarse-to-fine filters described above generate a large number of high-quality weakly labeled instances, namely, a set of positive instances and a set of negative samples. However, if the classifier is trained solely on positive instances, it will result in a severe imbalance between the number of positive and negative samples. This imbalance will cause the classifier to have a high recall but low precision. To address this issue, this embodiment proposes adding a false positive instance correction module to the model. False positive instances are used to correct the skewness of the classifier during training, thereby alleviating the overfitting problem caused by an excessive number of positive instances.

[0150] For example, for a seed relationship A batch of positive cases B can be extracted. pos And a set of negative instances B neg Then, connect these instances with D. n Combined, the classifier is trained. Based on this false positive instance correction module, the classifier's parameters and loss function can be corrected as follows:

[0151]

[0152] in, For the classifier, μ is the negative sampling loss coefficient, which is a hyperparameter that can take the value 0.5 or other values. It represents the input size of false positive instances and is used to control the proportion of negative instances used by the classifier during training.

[0153] Optionally, in this embodiment, the devices and applications involved in executing the above method include: small sample instance data encoded using BERT (a language representation model); the experiment uses a Windows 11 system, PyCharm software (integrated development environment), Torch 3.8 (an open-source machine learning library), and an NVIDIA GTX 3060 GPU. Text data in the corpus can be obtained through various means, such as online collection, targeted manual collection, etc.

[0154] like Figure 7 As shown, another embodiment of the present invention also provides a small sample relationship classification and filtering device 100, comprising:

[0155] The first construction module is used to construct a coarse-grained filter, which is used to filter the unlabeled corpus to obtain a first candidate instance with the same entity as the seed instance and a second candidate instance with similar semantics;

[0156] The second construction module is used to construct a fine-grained filter, which is used to filter the first candidate instance and the second candidate instance to obtain a third candidate instance with the same relational concept as the seed instance.

[0157] The definition module is used to define the third candidate instance as a positive instance set, and to define the candidate instances in the first and second candidate instances whose relationship concept is different from that of the seed instance as a negative sample set, wherein the relationship concept is used to describe the relationship between different instances;

[0158] The third construction module is used to construct a false positive instance correction module, which is used to adjust and control the proportion of the negative sample set used by the classifier during training.

[0159] The training module is used to train the classifier based on a small number of labeled instances belonging to newly emerging relationships, as well as the set of positive instances and the set of negative samples adjusted by the false positive instance correction module.

[0160] The classification module is used to classify relationships in small sample data based on the trained classifier.

[0161] As an optional embodiment, the construction of the coarse-grained filter includes:

[0162] An entity alignment module is set up to identify the first candidate instance in the corpus that has the same entity as the seed instance;

[0163] A relational twin network is set up to filter out second candidate instances with similar semantics to the seed instance by measuring the distance between the word vectors of instances in the corpus and the seed instance;

[0164] The coarse-grained filter is formed based on the entity alignment module and the relational twin network.

[0165] As an optional embodiment, the seed instances include multiple instances, and the step of filtering out second candidate instances with similar semantics to the seed instances by measuring the distance between the instances in the corpus and the word vectors in the seed instances includes:

[0166] The relational twin network determines the similarity score between two instances by measuring the Euclidean distance between word vectors in the instances in the corpus and the seed instance:

[0167]

[0168] The average similarity score is calculated based on the similarity scores between each instance in the corpus and each seed instance.

[0169] Second candidate instances with similar semantics to each of the seed instances are filtered out based on multiple average similarity scores;

[0170] Where σ(·) is the sigmoid activation function, f p(·) is used to encode the sentence vector, S RSN The range of (·) is 0 to 1, and the weight w p and deviation b p These are trainable parameters. It is a seed instance. It is an instance in the corpus.

[0171] As an optional embodiment, the construction of the fine-grained filter includes:

[0172] Set a prompt template generation function to convert the first candidate instance and the second candidate instance into new text that meets the requirements of the input template. The input template includes the text description, relational concepts and tag bar of the candidate instance. The tag bar is used to add tags to indicate whether the relational concepts of the candidate instance and the seed instance are consistent.

[0173] A pre-trained language model is used to process new text corresponding to the first candidate instance and the second candidate instance based on the relational concept of the seed instance, so as to add the tags matching each new text to the tag bar based on the processing result, and to filter the first candidate instance and the second candidate instance based on the tags to obtain the third candidate instance.

[0174] As an optional embodiment, it also includes:

[0175] The pre-training module is used to pre-train the coarse-grained filter, fine-grained filter, and classifier based on the obtained labeled instances that satisfy frequently occurring relationships.

[0176] The filtering module is used to filter the corpus according to the pre-trained coarse-grained filter and fine-grained filter to obtain the positive instance set and negative sample set;

[0177] The training module trains the pre-trained classifier based on the positive instance set, the negative sample set, and a small number of labeled instances belonging to newly emerging relationships.

[0178] As an optional embodiment, the pre-training of the coarse-grained filter, fine-grained filter, and classifier based on the obtained labeled instances satisfying frequently occurring relationships includes:

[0179] Based on any two instances from the labeled instances that satisfy the frequently occurring relationship, the coarse-grained filter is pre-trained using cross-entropy loss;

[0180] Different input templates are constructed, and the fine-grained filter is pre-trained based on the different input templates and labeled instances that satisfy common relationships. The input template includes template format content, text descriptions in candidate instances, relationship concepts, and a tag bar. The tag bar is used to add tags to indicate whether the relationship concepts between the candidate instance and the seed instance are consistent.

[0181] The classifier is pre-trained based on a few-shot learning model and labeled instances that satisfy frequently occurring relationships.

[0182] As an optional embodiment, constructing different input templates includes:

[0183] The system constructs input templates for text descriptions and relational concepts in different positions, input templates lacking some or all of the template format content, and input templates that form negative templates by changing relational concepts.

[0184] As an optional embodiment, it also includes:

[0185] The correction module is used to correct the parameters and loss function of the classifier based on the false positive instances.

[0186]

[0187] Among them, B pos A batch of positive cases, Bx eg These are negative instances within a set of negative samples. For the classifier, μ is the negative sampling loss coefficient, used to control the proportion of negative instances used by the classifier during training.

[0188] Another embodiment of the present invention also provides an electronic device, comprising:

[0189] At least one processor; and,

[0190] A memory communicatively connected to the at least one processor; wherein,

[0191] The memory stores instructions that can be executed by the at least one processor to implement the small sample relation classification and filtering method as described in any of the embodiments above.

[0192] Furthermore, one embodiment of the present invention also provides a storage medium storing a computer program, which, when executed by a processor, implements the small sample relationship classification and filtering method described above. It should be understood that the various solutions in this embodiment have the corresponding technical effects in the above-described method embodiments, and will not be repeated here.

[0193] Furthermore, embodiments of the present invention also provide a computer program product, which is tangibly stored on a computer-readable medium and includes computer-readable instructions that, when executed, cause at least one processor to perform a small sample relation classification filtering method as described in the embodiments above.

[0194] It should be noted that the computer storage medium of the present invention can be a computer-readable signal medium or a computer-readable storage medium, or any combination thereof. A computer-readable medium can be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, system, or device, or any combination thereof. More specific examples of a computer-readable storage medium may include, but are not limited to: an electrical connection having one or more wires, a portable computer disk, a hard disk, a random access storage medium (RAM), a read-only storage medium (ROM), an erasable programmable read-only storage medium (EPROM or flash memory), an optical fiber, a portable compact disk read-only storage medium (CD-ROM), an optical storage medium, a magnetic storage medium, or any suitable combination thereof. In the present invention, a computer-readable storage medium can be any tangible medium containing or storing a program that can be used by or in conjunction with an instruction execution system, system, or device. In the present invention, a computer-readable signal medium can include a data signal propagated in baseband or as part of a carrier wave, carrying computer-readable program code. Such propagated data signals can take various forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination thereof. Computer-readable signal media can also be any computer-readable medium other than computer-readable storage media, which can send, propagate, or transmit a program configured for use by or in connection with an instruction execution system, system, or device. The program code contained on the computer-readable medium can be transmitted using any suitable medium, including but not limited to: wireless, antenna, optical fiber, RF, etc., or any suitable combination thereof.

[0195] Furthermore, those skilled in the art will understand that embodiments of the present invention can be provided as methods, systems, or computer program products. Therefore, the present invention can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Moreover, the present invention can take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage and optical storage) containing computer-usable program code.

[0196] This invention is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of the invention. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, generate instructions for implementing the flowchart illustrations and / or block diagrams. Figure 1 One or more processes and / or boxes Figure 1 A system that specifies functions in one or more boxes.

[0197] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including an instruction set implemented in a process. Figure 1 One or more processes and / or boxes Figure 1 The function specified in one or more boxes.

[0198] These computer program instructions may also be loaded onto a computer or other programmable data processing equipment to cause a series of operational steps to be performed on the computer or other programmable equipment to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable equipment for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.

[0199] Obviously, those skilled in the art can make various modifications and variations to this invention without departing from its spirit and scope. Therefore, if these modifications and variations fall within the scope of the claims of this invention and their equivalents, this invention also intends to include these modifications and variations.

[0200] The above embodiments are merely exemplary embodiments of the present invention and are not intended to limit the present invention. The scope of protection of the present invention is defined by the claims. Those skilled in the art can make various modifications or equivalent substitutions to the present invention within its spirit and scope of protection, and such modifications or equivalent substitutions should also be considered to fall within the scope of protection of the present invention.

Claims

1. A small sample relationship classification and filtering method, characterized in that, include: A coarse-grained filter is constructed to filter the unlabeled corpus to obtain a first candidate instance with the same entity as the seed instance and a second candidate instance with similar semantics. A fine-grained filter is constructed to filter the first candidate instance and the second candidate instance to obtain a third candidate instance with the same relational concept as the seed instance; The third candidate instance is defined as the positive instance set, and the candidate instances in the first and second candidate instances whose relationship concept is different from that of the seed instance are defined as the negative sample set. The relationship concept is used to describe the relationship between different instances. A false positive instance correction module is constructed, which is used to adjust and control the proportion of the negative sample set used by the classifier during training; The classifier is trained based on a small number of labeled instances belonging to newly emerging relationships, as well as the set of positive instances and the set of negative samples adjusted by the false positive instance correction module. Relationship classification is performed on small sample data based on the trained classifier.

2. The small sample relationship classification and filtering method according to claim 1, characterized in that, The construction of the coarse-grained filter includes: An entity alignment module is set up to identify the first candidate instance in the corpus that has the same entity as the seed instance; A relational twin network is set up to filter out second candidate instances with similar semantics to the seed instance by measuring the distance between the word vectors of instances in the corpus and the seed instance; The coarse-grained filter is formed based on the entity alignment module and the relational twin network.

3. The small sample relationship classification and filtering method according to claim 2, characterized in that, The seed instances include multiple instances, and the step of filtering out second candidate instances with similar semantics to the seed instances by measuring the distance between the instances in the corpus and the word vectors in the seed instances includes: The relational twin network determines the similarity score between two instances by measuring the Euclidean distance between word vectors in the instances in the corpus and the seed instance: The average similarity score is calculated based on the similarity scores between each instance in the corpus and each seed instance. Second candidate instances with similar semantics to each of the seed instances are filtered out based on multiple average similarity scores; Where σ(·) is the sigmoid activation function, f p (·) is used to encode the sentence vector, S RSN The range of (·) is 0 to 1, and the weight w p and deviation b p These are trainable parameters. It is a seed instance. It is an instance in the corpus.

4. The small sample relationship classification and filtering method according to claim 1, characterized in that, The construction of the fine-grained filter includes: Set a prompt template generation function to convert the first candidate instance and the second candidate instance into new text that meets the requirements of the input template. The input template includes the text description, relational concepts and tag bar of the candidate instance. The tag bar is used to add tags to indicate whether the relational concepts of the candidate instance and the seed instance are consistent. A pre-trained language model is used to process new text corresponding to the first candidate instance and the second candidate instance based on the relational concept of the seed instance, so as to add the tags matching each new text to the tag bar based on the processing result, and to filter the first candidate instance and the second candidate instance based on the tags to obtain the third candidate instance.

5. The small sample relationship classification and filtering method according to claim 1, characterized in that, Also includes: Based on the obtained labeled instances that satisfy frequently occurring relationships, the coarse-grained filter, fine-grained filter, and classifier are pre-trained respectively. The positive instance set and negative sample set are obtained by filtering the corpus based on the pre-trained coarse-grained filter and fine-grained filter. The pre-trained classifier is trained based on the positive instance set, the negative sample set, and a small number of labeled instances belonging to newly emerging relationships.

6. The small sample relationship classification and filtering method according to claim 5, characterized in that, The pre-training of the coarse-grained filter, fine-grained filter, and classifier based on the obtained labeled instances satisfying frequently occurring relationships includes: Based on any two instances from the labeled instances that satisfy the frequently occurring relationship, the coarse-grained filter is pre-trained using cross-entropy loss; Different input templates are constructed, and the fine-grained filter is pre-trained based on the different input templates and labeled instances that satisfy common relationships. The input template includes template format content, text descriptions in candidate instances, relationship concepts, and a tag bar. The tag bar is used to add tags to indicate whether the relationship concepts between the candidate instance and the seed instance are consistent. The classifier is pre-trained based on a few-shot learning model and labeled instances that satisfy frequently occurring relationships.

7. The small sample relationship classification and filtering method according to claim 6, characterized in that, The construction of different input templates includes: The system constructs input templates for text descriptions and relational concepts in different positions, input templates lacking some or all of the template format content, and input templates that form negative templates by changing relational concepts.

8. The small sample relationship classification and filtering method according to claim 1, characterized in that, Also includes: Based on the false positive instance correction module, the parameters and loss function of the classifier are corrected as follows: Among them, B pos B is a batch of positive cases. neg This refers to a set of negative instances within a negative sample set. For the classifier, μ is the negative sampling loss coefficient, used to control the proportion of negative instances used by the classifier during training.

9. A small sample relationship classification and filtering device, characterized in that, include: The first construction module is used to construct a coarse-grained filter, which is used to filter the unlabeled corpus to obtain a first candidate instance with the same entity as the seed instance and a second candidate instance with similar semantics; The second construction module is used to construct a fine-grained filter, which is used to filter the first candidate instance and the second candidate instance to obtain a third candidate instance with the same relational concept as the seed instance. The definition module is used to define the third candidate instance as a positive instance set, and to define the candidate instances in the first and second candidate instances whose relationship concept is different from that of the seed instance as a negative sample set, wherein the relationship concept is used to describe the relationship between different instances; The third construction module is used to construct a false positive instance correction module, which is used to adjust and control the proportion of the negative sample set used by the classifier during training. The training module is used to train the classifier based on a small number of labeled instances belonging to newly emerging relationships, as well as the set of positive instances and the set of negative samples adjusted by the false positive instance correction module. The classification module is used to classify relationships in small sample data based on the trained classifier.

10. An electronic device, characterized in that, include: At least one processor; as well as, A memory communicatively connected to the at least one processor; wherein, The memory stores instructions that can be executed by the at least one processor to implement the small sample relation classification filtering method as described in any one of claims 1-8.

Images