Weak supervision relation extraction method and device for collaborative rule discovery and prompt learning
By combining collaborative rule discovery and prompting learning with rule mining and pre-trained language models, the relation extraction process is optimized, which solves the problem of errors and missing data caused by insufficient labeled data in weakly supervised environments and improves the precision and recall of relation extraction.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- WUHAN UNIV
- Filing Date
- 2024-11-07
- Publication Date
- 2026-05-08
AI Technical Summary
In a weakly supervised environment, existing technologies struggle to avoid errors and omissions in relation extraction results, mainly due to insufficient labeled data and inadequate prior knowledge, resulting in poor integration of rules and pre-trained language models.
By employing collaborative rule discovery and prompting learning, the corpus data is divided into labeled and unlabeled sets. Rule sets are mined and initially labeled. Relationships are extracted using an active learning strategy and a graph-based rule matching method, combined with a pre-trained language model. The labeled data is then optimized through manual verification, achieving bidirectional data and knowledge empowerment between rules and the pre-trained language model.
It improves the precision and recall of relation extraction, reduces erroneous and missing triples, and achieves high-quality data annotation in a weakly supervised environment.
Smart Images
Figure CN121997930A_ABST
Abstract
Description
Technical Field
[0001] This application relates to the field of relation extraction technology, and in particular to a weakly supervised relation extraction method and apparatus based on collaborative rule discovery and prompting learning. Background Technology
[0002] Relation extraction, which extracts structured information from unstructured and semi-structured corpora, is fundamental to text analysis and mining. It helps build domain knowledge bases and supports intelligent applications such as information retrieval and knowledge recommendation. However, due to domain specialization, high costs of manual annotation, and data updates, relation extraction methods are often affected by sparse training data and insufficient prior knowledge.
[0003] Rules are commonly used for automatic data labeling. However, due to their fixed pattern matching methods, rules may produce missing triples, meaning they may miss labeling triples representing target relations, thus limiting the coverage of rule discovery. Currently, pre-trained language models have achieved strong scalability in zero-shot scenarios of natural language processing. They can generate usable training data for small, vertically specific models.
[0004] In related technologies, attempts have been made to combine rules and pre-trained language model bins to achieve relation extraction. However, it is still difficult to avoid the situation where the amount of labeled data is small in weakly supervised environments, resulting in errors and omissions in the relation extraction results. For example, when the pre-trained language model lacks relevant domain knowledge, it may generate incorrect triples that are inconsistent with the real labels; weakly related and unrelated information contained in the rules may mislead the pre-trained language model; and based on limited prior knowledge, it is difficult to verify the relation extraction results of the rules and the pre-trained language model one by one, which urgently needs improvement. Summary of the Invention
[0005] This application provides a weakly supervised relation extraction method and apparatus for collaborative rule discovery and prompting learning, in order to solve the technical problem in related technologies that it is difficult to avoid errors and omissions in relation extraction results due to the small amount of labeled data in a weakly supervised environment.
[0006] The first aspect of this application provides a weakly supervised relation extraction method based on collaborative rule discovery and cue learning, comprising the following steps: dividing the data in a corpus to obtain an labeled dataset and an unlabeled dataset; mining the labeled dataset based on cue learning to obtain a mining rule set; determining whether the unlabeled dataset is an empty set; if the unlabeled dataset is not an empty set, extracting unlabeled data from the unlabeled dataset for labeling to obtain preliminary labeled data, and performing relation extraction on the preliminary labeled data based on the mining rule set and the target relation extraction method to obtain corresponding relation extraction results; determining whether the preliminary labeled data meets a preset deletion condition based on the relation extraction results; if the preset deletion condition is met, deleting the preliminary labeled data from the unlabeled dataset; otherwise, adding the preliminary labeled data to a manual validation set.
[0007] Optionally, in one embodiment of this application, the step of mining the labeled dataset based on prompting learning to obtain a mining rule set includes: preprocessing the labeled dataset to obtain a text dataset; comparing multiple text sequences in the text dataset to mine common subsequences that satisfy preset mining conditions as rule subjects to obtain mining results; and mining common subsequences that satisfy the preset mining conditions from the remaining text sequences based on the mining results until a preset stopping condition is reached to obtain the mining rule set.
[0008] Optionally, in one embodiment of this application, the step of extracting relations from the preliminary labeled data based on the mining rule set and the target relation extraction method to obtain the corresponding relation extraction result includes: constructing a rule semantic graph and a sentence semantic graph based on the mining rule set and the structural and semantic information of the sentences in the preliminary labeled data, respectively; calculating the similarity between the rule semantic graph and the sentence semantic graph; and obtaining the relation extraction result based on the similarity.
[0009] Optionally, in one embodiment of this application, the step of extracting relations from the preliminary labeled data based on the mining rule set and the target relation extraction method to obtain the corresponding relation extraction result includes: extracting the rule with the highest relevance to the sentence in the preliminary labeled data from the mining rule set, and integrating the rule into the prompt template of the pre-trained language model to construct the input of the pre-trained language model; and constraining the output of the pre-trained language model using a prototype-based mapper method to obtain the relation extraction result.
[0010] Optionally, in one embodiment of this application, determining whether the preliminary labeled data meets the preset deletion conditions based on the relation extraction results includes: obtaining the previous round of relation extraction results; determining whether the relation extraction results are consistent with the previous round of relation extraction results; if the relation extraction results are consistent with the previous round of relation extraction results, then determining that the preliminary labeled data meets the preset deletion conditions.
[0011] A second aspect of this application provides a weakly supervised relation extraction device for collaborative rule discovery and prompting learning, comprising: a partitioning module for partitioning data in a corpus to obtain a labeled dataset and an unlabeled dataset; a mining module for mining the labeled dataset based on prompting learning to obtain a mining rule set; an extraction module for determining whether the unlabeled dataset is empty, and if the unlabeled dataset is not empty, extracting unlabeled data from the unlabeled dataset for labeling to obtain preliminary labeled data, and performing relation extraction on the preliminary labeled data based on the mining rule set and the target relation extraction method to obtain a corresponding relation extraction result; and a judgment module for determining whether the preliminary labeled data meets a preset deletion condition based on the relation extraction result, and if the preset deletion condition is met, deleting the preliminary labeled data from the unlabeled dataset; otherwise, adding the preliminary labeled data to a manual validation set.
[0012] Optionally, in one embodiment of this application, the mining module includes: a processing unit for preprocessing the labeled dataset to obtain a text dataset; a first mining unit for comparing multiple text sequences in the text dataset and mining common subsequences that satisfy preset mining conditions as rule subjects to obtain mining results; and a second mining unit for mining common subsequences that satisfy the preset mining conditions from the remaining text sequences based on the mining results until a preset stopping condition is reached to obtain the mining rule set.
[0013] Optionally, in one embodiment of this application, the extraction module includes: a first construction unit, configured to construct a rule semantic graph and a sentence semantic graph based on the mining rule set and the structural and semantic information of sentences in the preliminary labeled data, respectively; a calculation unit, configured to calculate the similarity between the rule semantic graph and the sentence semantic graph; and a first acquisition unit, configured to obtain the relation extraction result based on the similarity.
[0014] Optionally, in one embodiment of this application, the extraction module includes: a second construction unit, configured to extract the rule with the highest relevance to the sentence in the preliminary labeled data from the mining rule set, and integrate the rule into the prompt template of the pre-trained language model to construct the input of the pre-trained language model; and a constraint unit, configured to constrain the output of the pre-trained language model using a prototype-based mapper method to obtain the relation extraction result.
[0015] Optionally, in one embodiment of this application, the judgment module includes: a second acquisition unit, configured to acquire the previous round of relation extraction results; a judgment unit, configured to determine whether the relation extraction results are consistent with the previous round of relation extraction results; and a determination unit, configured to determine that the preliminary labeled data meets the preset deletion conditions if the relation extraction results are consistent with the previous round of relation extraction results.
[0016] A third aspect of this application provides an electronic device, including: a memory, a processor, and a computer program stored in the memory and executable on the processor. The processor executes the program to implement the weakly supervised relation extraction method for collaborative rule discovery and prompting learning as described in the above embodiments.
[0017] A fourth aspect of this application provides a computer-readable storage medium storing computer instructions for causing the computer to execute the weakly supervised relation extraction method for collaborative rule discovery and prompting learning as described in the above embodiments.
[0018] A fifth aspect of this application provides a computer program product, including a computer program, which, when executed, is used to implement the weakly supervised relation extraction method for collaborative rule discovery and prompting learning as described above.
[0019] This application embodiment can acquire labeled and unlabeled datasets from a corpus, obtain a mining rule set by mining the labeled dataset, and, provided the unlabeled dataset is not empty, extract unlabeled data from the unlabeled dataset for labeling. Based on the mining rule set and the target relation extraction method, relation extraction is performed on the labeled data to obtain corresponding relation extraction results. It then determines whether the relation extraction results meet preset deletion conditions. If they do, the labeled data is deleted from the unlabeled dataset, and the next round of labeling begins. If not, the labeled data is manually labeled. This allows for relation extraction in a weakly supervised environment, collaboratively using rule discovery and prompting learning, achieving bidirectional empowerment of data and knowledge between rules and pre-trained language models, thereby improving the precision and recall of relation extraction. This solves the technical problem in related technologies where errors and omissions in relation extraction results are unavoidable due to the limited amount of labeled data in a weakly supervised environment.
[0020] Additional aspects and advantages of this application will be set forth in part in the description which follows, and in part will be obvious from the description, or may be learned by practice of this application. Attached Figure Description
[0021] The above and / or additional aspects and advantages of this application will become apparent and readily understood from the following description of the embodiments taken in conjunction with the accompanying drawings, wherein:
[0022] Figure 1 This is a flowchart of a weakly supervised relation extraction method based on collaborative rule discovery and prompting learning, according to an embodiment of this application.
[0023] Figure 2 This is a schematic diagram illustrating the principle of a weakly supervised relation extraction method based on collaborative rule discovery and prompting learning according to an embodiment of this application;
[0024] Figure 3 This is a schematic diagram illustrating the principle of a graph-based rule matching mechanism according to an embodiment of this application;
[0025] Figure 4 This is a schematic diagram illustrating the principle of a prototype-based mapper mechanism according to an embodiment of this application;
[0026] Figure 5 This is a schematic diagram of a weakly supervised relation extraction device for collaborative rule discovery and prompting learning provided in an embodiment of this application;
[0027] Figure 6 This is a schematic diagram of the structure of an electronic device provided according to an embodiment of this application. Detailed Implementation
[0028] The embodiments of this application are described in detail below. Examples of these embodiments are shown in the accompanying drawings, wherein the same or similar reference numerals denote the same or similar elements or elements having the same or similar functions throughout. The embodiments described below with reference to the accompanying drawings are exemplary and intended to explain this application, and should not be construed as limiting this application.
[0029] The following describes a weakly supervised relation extraction method and apparatus based on collaborative rule discovery and prompting learning, according to embodiments of this application, with reference to the accompanying drawings. Addressing the technical problem mentioned in the background art that errors and omissions in relation extraction results are difficult to avoid in weakly supervised environments due to limited labeled data, this application provides a weakly supervised relation extraction method based on collaborative rule discovery and prompting learning. In this method, labeled and unlabeled datasets from a corpus are acquired. A mining rule set is obtained by mining the labeled dataset. If the unlabeled dataset is not empty, unlabeled data is extracted from the unlabeled dataset for labeling. Relations are extracted from the labeled data based on the mining rule set and the target relation extraction method to obtain corresponding relation extraction results. The method then determines whether the relation extraction results meet preset deletion conditions. If they do, the labeled data is deleted from the unlabeled dataset, and the next round of labeling begins. If not, the labeled data is manually labeled. This allows for relation extraction through collaborative rule discovery and prompting learning in a weakly supervised environment, achieving bidirectional empowerment of data and knowledge between rules and pre-trained language models, thereby improving the precision and recall of relation extraction. This solves the technical problem in related technologies where, under weak supervision, the limited amount of labeled data inevitably leads to errors and omissions in relation extraction results.
[0030] Specifically, Figure 1 This is a flowchart illustrating a weakly supervised relation extraction method for collaborative rule discovery and prompting learning provided in an embodiment of this application.
[0031] like Figure 1 As shown, the weakly supervised relation extraction method for collaborative rule discovery and prompting learning includes the following steps:
[0032] In step S101, the data in the corpus is divided to obtain labeled datasets and unlabeled datasets.
[0033] In actual implementation, the embodiments of this application can divide the data in the corpus into an labeled dataset C. l and unlabeled dataset C u Among them, the labeled dataset C l The data in the data can be a combination of pre-annotated data through manual methods and other means, and the annotated data that has been verified in the subsequent process.
[0034] In step S102, the labeled dataset is mined based on prompts to obtain a mining rule set.
[0035] This application embodiment can be implemented on an already labeled dataset C. l Based on this, a rule set R is mined using a rule mining method based on prompting learning.
[0036] Optionally, in one embodiment of this application, learning based on prompts to mine labeled datasets to obtain a mining rule set includes: preprocessing the labeled dataset to obtain a text dataset; comparing multiple text sequences in the text dataset to mine common subsequences that meet preset mining conditions as rule subjects to obtain mining results; and mining common subsequences that meet preset mining conditions from the remaining text sequences based on the mining results until a preset stopping condition is reached to obtain a mining rule set.
[0037] As one possible approach, firstly, this embodiment of the application can perform preprocessing operations such as removing stop words and punctuation marks from the labeled dataset, and then compare multiple text sequences to mine the longest common subsequence containing the subject and object, which serves as the main body of the rules. Based on the mining results, the mining of the longest common subsequence continues until no new sequences can be mined. These longest common subsequences are the structural patterns. Next, a prompt template is designed based on the labeled dataset and its structural patterns. Contextual words in the structural patterns are masked for prediction by the pre-trained language model. Specifically, the prompt template design is as follows:
[0038] T(seed k = {[CLS], seed k, [SEP], Pattern(seed) k .w s SUB, Seed k .w o ,OBJ,[Mask])}(1)
[0039] Among them, seed k Seed indicates the seed clause. k .w s and seed k .w o Represents seed k The subject and object in the sentence. SUB and OBJ represent seed. k .w s and seed k .w o The entity type. Pattern() represents seed. k The structural pattern. [Mask] refers to the context words that are being masked.
[0040] Next, a pre-trained language model is used to predict [Mask], and the top k most likely words are selected to replace [Mask], thus generating scalable rules. To learn the masked context words, the P-Tuning method is used to fine-tune the pre-trained language model based on an labeled dataset.
[0041] Finally, based on the results of rule mining, the accuracy of the rules is evaluated to filter out low-quality rules:
[0042]
[0043] in, Represented by r i Extracted correct and incorrect triples.
[0044] In step S103, it is determined whether the unlabeled dataset is empty. If the unlabeled dataset is not empty, unlabeled data is extracted from the unlabeled dataset for labeling to obtain preliminary labeled data. Relationships are then extracted from the preliminary labeled data based on the mining rule set and the target relation extraction method to obtain the corresponding relation extraction results.
[0045] Furthermore, embodiments of this application can determine the unlabeled dataset C. u Is it empty? If dataset C is unlabeled. u If the dataset is empty, the annotation process ends; otherwise, the dataset C is not empty. i Unlabeled data is extracted and labeled to obtain preliminary labeled data. Then, relation extraction is performed according to the mining rule set R and the target relation extraction method (e.g., relation extraction using rules or relation extraction using a pre-trained language model) to obtain relation extraction results.
[0046] The target relation extraction method can be determined based on an active learning strategy based on slot machines.
[0047] The slot machine-based active learning strategy specifically addresses this: considering potential errors in rule discovery and pre-trained language model prompts, optimizing the synergy between rules and the pre-trained language model by allocating and validating relation extraction tasks based on limited prior knowledge is crucial. Therefore, a slot machine-based active learning strategy is adopted, dynamically allocating and validating relation extraction tasks between rules and the pre-trained language model based on historical performance, leveraging limited human resources. The multi-armed slot machine is a classic strategy in reinforcement learning, aiming to balance exploration and utilization based on the historical performance of multiple choices to maximize rewards.
[0048] In this strategy, embodiments of this application can consider rules and pre-trained language models as arms a0 and a1, with the set of arms being A = {a0, a1}. Sentences to be labeled are considered as relation extraction tasks, with each task s... k Reward X k The following can be calculated:
[0049]
[0050] Considering the limited prior knowledge, the true label of the task is determined by extracting the results of multiple iterations of relations. When the arm completes task s in round t-th... j If s in the t-th round j If the label is the same as the label in the (t-1)-th round, then the label is... This can be considered a confidence label. Therefore, s k The reward for the upper arm is 1, otherwise the reward is 0.
[0051] Subsequently, based on the confidence score Con for each arm... i,k Determine the next task k+1 Arm a should be selected i :
[0052]
[0053] Among them, Con i,k It consists of two parts: exploring fractions and utilizing fractions. N i,k Indicates a i Number of completed relation extraction tasks. In Con i,k In the calculation formula, the term to the left of the plus sign represents the exploration score, which is used to calculate arm a. i The average reward. A higher average reward indicates that the task allocation is more inclined to utilize that arm. The item to the right of the plus sign represents the utilization score, which is calculated as a... i The confidence interval width is determined by the arm's confidence score. When the total number of tasks is large but the arm completes few tasks, the confidence interval width is large, leading to a greater tendency to allocate tasks to explore that arm. In relation extraction task allocation and validation, the arm's confidence score is used to complete the task allocation. Then, by considering the results of multiple iterations of relation extraction, the task with the lowest reward is assigned to manual validation.
[0054] Optionally, in one embodiment of this application, relation extraction is performed on the preliminary labeled data based on the mining rule set and the target relation extraction method to obtain the corresponding relation extraction results, including: constructing a rule semantic graph and a sentence semantic graph based on the structure and semantic information of sentences in the mining rule set and the preliminary labeled data, respectively; calculating the similarity between the rule semantic graph and the sentence semantic graph; and obtaining the relation extraction results based on the similarity.
[0055] This section describes the method of relation extraction based on graph-based rule matching.
[0056] To expand the pattern matching method of rules, embodiments of this application can use semantic graphs to model the structure and semantic information of rules and sentences. Then, the similarity between the semantic graphs of rules and sentences is used as a matching score. To construct the semantic graphs of rules and sentences, embodiments of this application can define a semantic graph G = (V, E). V is a set of nodes representing words in the rules and sentences. Word embeddings of nodes are obtained based on a pre-trained language model to represent the semantic information of rules and sentences. E is a set of edges. The co-occurrence relationships of words within a fixed-size window are used to capture the word order information of rules and sentences.
[0057] In the rule semantic graph and sentence semantic graph Walking on can be formally defined as including
[0058] {v 0, ,v1},{v 1, ,v2},…,{v i, ,v i+1 A finite sequence of edges}, where v i ∈V,{v i, ,v i+1}∈E. The length of the walk is the number of edges in the sequence. A path can be viewed as a special walk where all nodes are distinct. v i and v j Path length p between ij It can be viewed as the distance between these two nodes.
[0059] To capture deep structural information of rules and sentences, the shortest path of a node is used to represent the dependency relationship between a word and its neighboring words. Specifically, in the rule semantic graph... and sentence semantic graph A depth-first search is used to calculate the distance between nodes. If node v i No connection to v j And the distance from p ij If it is less than the length d, then v i and v j A new edge will be added between them as the shortest path, with edge weight W(e). m ) = 1 / p ij The strength of the relationship between words can be assessed by the weight of the edges.
[0060] Next, according to Calculation rule r i and sentence s k The graph similarity is used as the matching score:
[0061]
[0062] Among them, Knode () is used for comparison Node similarity, Intended to assess Edge similarity. N is the normalization factor. If the matching score exceeds the predefined parameter λ, then sentence s k It can be done through rule r i Perform matching. Specifically, in calculating K... node In parentheses, the first evaluation was conducted. The semantic similarity of words. A common word set I = {(v...} is constructed based on word pairs whose semantic similarity exceeds a predefined parameter τ. a ,v b Based on a common vocabulary set, K is used. node () to measure and The similarity between nodes is defined as follows:
[0063]
[0064] Used for measurement The similarity of edges is defined as follows:
[0065]
[0066]
[0067] in, express From the node The edge. express From the node The edge of W(e) m ) and W(e n ) represents edge e m and e n The weights. The normalization factor can be calculated as:
[0068]
[0069] in, express The adjacency matrix, express A diagonal matrix. ∥·∥ F It is the F-norm of the matrix. By comprehensively comparing the nodes and edges of the rule and sentence semantic graph, the structural and semantic similarity between the rule and the sentence can be evaluated, and finally the rule-based relation extraction result can be obtained.
[0070] Optionally, in one embodiment of this application, relation extraction is performed on the initially labeled data based on the mining rule set and the target relation extraction method to obtain the corresponding relation extraction result. This includes: extracting the rule with the highest relevance to the sentence in the initially labeled data from the mining rule set, and integrating the rule into the prompt template of the pre-trained language model to construct the input of the pre-trained language model; and constraining the output of the pre-trained language model using a prototype-based mapper method to obtain the relation extraction result S. k,t .
[0071] Here, we will explain the extraction method for relation extraction using pre-trained language models.
[0072] Understandably, existing work in this field has attempted to combine rules with pre-trained language models. On one hand, to reduce missing triples in rule discovery, some works empower rules using pre-trained language models, generating rules from extracted triples. Then, the matching results of these rules are used to optimize the pre-trained language model. However, erroneous triples in the pre-trained language model can affect rule generation. On the other hand, to further reduce erroneous triples in the pre-trained language model, some works empower it with rules, designing rule-based prompt templates to inject domain knowledge into the pre-trained language model. However, since the rules are designed by experts, this requires significant annotation costs. Furthermore, missing triples in the rules may lead to incomplete domain knowledge, potentially misleading the pre-trained language model.
[0073] In summary, related technologies struggle to simultaneously reduce missing and incorrect triples generated by rules and pre-trained language models in weakly supervised relation extraction. Therefore, this application aims to achieve high-quality data annotation by coordinating rules and pre-trained language models for weakly supervised relation extraction. The challenges are as follows: (1) Pre-trained language models can capture semantic information of sentences to discover rules. However, when pre-trained language models lack relevant domain knowledge, it is difficult to explicitly capture the diverse semantic and structural patterns of rules. (2) Rules can help pre-trained language models understand the context of the corpus. However, weakly related and unrelated information contained in the rules may mislead the pre-trained language model. (3) Based on limited prior knowledge, it is difficult to verify the relation extraction results of rules and pre-trained language models one by one.
[0074] In practical implementation, embodiments of this application can utilize a rule-integrated prompting ensemble method to construct the input of a pre-trained language model, and use a prototype-based mapper method to constrain the output of the pre-trained language model to obtain the relation extraction result S. k,t .
[0075] Considering the sensitivity of pre-trained language models to prompts, the rules with the highest relevance to sentences are integrated into the prompt templates of the pre-trained language model. Specifically, the labeled dataset is divided into a support set and a query set. The support set is used to provide prompts to the pre-trained language model, and the query set is used to provide feedback and optimize the pre-trained language model. For each sentence s in the support set and query set... k Based on its matching rules, the following prompt template can be constructed.
[0076]
[0077] Among them, s k .w s and s k .w o It is s k The subject and object. s k The rule with the highest matching score.
[0078] Furthermore, a prompting integration strategy is utilized to help the pre-trained language model learn semantic information from inputs that support set rules and sentences. This is achieved by modifying the basic prompting template. The location of [Mask] is used to generate multiple hints:
[0079]
[0080] To reduce the impact of irrelevant rules and sentences on the pre-trained language model, embodiments of this application can construct an attention-based relation label prototype network by aggregating labeled sentences from a support set. Then, by comparing the semantic similarity between the task and the relation prototype in the query set, the pre-trained language model predictions can be mapped to relation labels. Specifically, by incorporating a rule-based prompting ensemble method, sentence s can be obtained. k ∈C s The prompt template is then used to predict [Mask] in the prompt template based on a pre-trained language model, in order to obtain s k Vector representation of:
[0081] h k =M(T(s) k ))(15)
[0082] Next, the relation prototype y is constructed based on the vector representations of sentences with relation labels in the support set. n To filter h in prototype building k Weakly relevant and irrelevant semantic information is extracted from the support set using an instance-level attention mechanism, focusing on semantic information relevant to the query set.
[0083]
[0084] Among them, Proto n Indicates label y n The prototype. C n Indicates a label y n The sentence. Indicates the construction of Proto n hour The weight.
[0085]
[0086]
[0087] Among them, W p It is a learnable weight matrix, ⊙ represents the element-wise dot product, θ() is the activation function, and sum() represents the sum of all elements of the vector.
[0088] Next, the embodiments of this application can obtain sentence s in the query set. i The probability of a relation label can be calculated by s. i To tag y n Generate by distance from the prototype:
[0089]
[0090] Here, d() is a distance function. Finally, the loss is calculated using the cross-entropy loss function, and the pre-trained language model is optimized based on the query set.
[0091]
[0092] In step S104, based on the relation extraction results, it is determined whether the preliminary labeled data meets the preset deletion conditions. If the preset deletion conditions are met, the preliminary labeled data is deleted from the unlabeled dataset; otherwise, the preliminary labeled data is added to the manual validation set.
[0093] Furthermore, in this embodiment, the deletion conditions can be determined based on the relation extraction results. For example, if the current relation extraction result is consistent with the previous relation extraction result, it is determined that the preset deletion conditions are met, the initial annotation data is correctly labeled, and can be deleted from the unlabeled dataset and added to the labeled dataset. Alternatively, if the current relation extraction result is inconsistent with the previous relation extraction result, and the current extraction has reached the maximum number of extraction iterations, in this embodiment, the initial annotation data can be added to the manual validation set for subsequent manual annotation.
[0094] Optionally, in one embodiment of this application, determining whether the preliminary labeled data meets the preset deletion conditions based on the relation extraction results includes: obtaining the relation extraction results of the previous round; determining whether the relation extraction results are consistent with the relation extraction results of the previous round; if the relation extraction results are consistent with the relation extraction results of the previous round, then determining that the preliminary labeled data meets the preset deletion conditions.
[0095] For example, embodiments of this application can determine the annotation result L of the t-th round. t The annotation result L of the (t-1)th round t-1 If the two annotation results are identical, the labeled data is directly deleted from the unlabeled dataset, i.e., C. u =C u / S k,t If the annotation results are different, they will be assigned to manual annotation.
[0096] Combination Figures 2 to 4 As shown, an embodiment is used to illustrate in detail the working principle of the weakly supervised relation extraction method for collaborative rule discovery and prompting learning in this application.
[0097] The overall algorithm involved in the embodiments of this application can be as follows:
[0098]
[0099]
[0100] As shown in the algorithm, during initialization, this embodiment constructs an arm set based on rules and a pre-trained language model (lines 1-3). Then, in the relation extraction task assignment, labeled sentences are first assigned to the rule discovery method and rule-induced prompting learning method modules enhanced by the pre-trained language model to evaluate the performance of the rule module and the pre-trained language model module (lines 4-9). Based on the historical rewards of the rule module and the pre-trained language model module, unlabeled sentences are assigned tasks to obtain initial relation extraction results (lines 10-12). Next, in the relation extraction task verification phase, arms with higher confidence scores are selected to complete the task. Rewards are obtained based on the arm's predictions to update its confidence score (lines 13-16). If the prediction result in the current round is inconsistent with the previous round... This indicates that the task has failed. The task will be reassigned to manual annotation for verification until the manual annotation budget is exhausted (lines 17-18). If the prediction result in the current round is consistent with the previous round, the relation extraction result is added to the labeled sentence set C. l (Lines 19-20). Through multiple task validations, the rule module and the pre-trained language model module will be based on the labeled sentence set C. lPerform iterative optimization until all unlabeled sentences are labeled or the maximum number of iterations is reached. Then return the final rule set, pre-trained language model, and relation extraction results (lines 21-23).
[0101] In actual implementation, such as Figure 2 As shown, embodiments of this application may include the following steps:
[0102] Step S1: Select a small amount of data from the corpus for annotation, and divide the corpus into an annotated dataset C. l and unlabeled dataset C u .
[0103] Step S2: Set the iteration count t = 0.
[0104] Step S3: The labeled dataset C from step S1 l Based on this, a rule set R is mined using a rule mining method based on prompting learning.
[0105] First, in this embodiment, after preprocessing the labeled dataset by removing stop words and punctuation marks, the longest common subsequence containing the subject and object is mined by comparing multiple text sequences, and this subsequence serves as the rule body. The mining of the longest common subsequence continues until no new sequences can be mined. These longest common subsequences are the structural patterns. Next, a prompt template is designed based on the labeled dataset and its structural patterns. Contextual words in the structural patterns are masked for prediction by the pre-trained language model. Specifically, the prompt template is designed as follows:
[0106] T(seed k = {[CLS], seed k, [SEP], Pattern(seed) k .w s SUB, Seed k .w o ,OBJ,[Mask])}(1)
[0107] Among them, seed k Seed indicates the seed clause. k .w s and seed k .w o Represents seed k The subject and object in the sentence. SUB and OBJ represent seed. k .w s and seed k .w o The entity type. Pattern() represents seed. kThe structural pattern. [Mask] refers to the context words that are being masked.
[0108] Next, a pre-trained language model is used to predict [Mask], and the top k most likely words are selected to replace [Mask], thus generating scalable rules. To learn the masked context words, the P-Tuning method is used to fine-tune the pre-trained language model based on an labeled dataset.
[0109] Finally, based on the results of rule mining, the accuracy of the rules is evaluated to filter out low-quality rules:
[0110]
[0111] in, Represented by r i Extracted correct and incorrect triples.
[0112] Step S4: Determine the unlabeled dataset C u Is it empty? If dataset C is unlabeled. u If the annotation is empty, then end the annotation process; otherwise, proceed to step S5.
[0113] Step S5: Determine whether to use rules or pre-trained language models for relation extraction based on the slot machine-based active learning strategy.
[0114] Considering the potential errors in rule discovery and pre-trained language model prompts, how to allocate and validate relation extraction tasks based on limited prior knowledge is crucial for optimizing the synergy between rules and the pre-trained language model. Therefore, a slot machine-based active learning strategy is adopted, dynamically allocating and validating relation extraction tasks between rules and the pre-trained language model based on historical performance, leveraging limited human resources. The multi-armed slot machine is a classic strategy in reinforcement learning, aiming to balance exploration and utilization based on the historical performance of multiple choices to maximize the obtained reward.
[0115] In this strategy, embodiments of this application can consider rules and pre-trained language models as arms a0 and a1, with the set of arms being A = {a0, a1}. Sentences to be labeled are considered as relation extraction tasks, with each task s... k Reward X k The following can be calculated:
[0116]
[0117] Considering the limited prior knowledge, the true label of the task is determined by extracting the results of multiple iterations of relations. When the arm completes task s in round t-th... j If s in the t-th round jIf the label is the same as the label in the (t-1)-th round, then the label is... This can be considered a confidence label. Therefore, s k The reward for the upper arm is 1, otherwise the reward is 0.
[0118] Subsequently, based on the confidence score Con for each arm... i,k Determine the next task k+1 Arm a should be selected i :
[0119]
[0120] Among them, Con i,k It consists of two parts: exploring fractions and utilizing fractions. N i,k Indicates a i Number of completed relation extraction tasks. In Con i,k In the calculation formula, the term to the left of the plus sign represents the exploration score, which is used to calculate arm a. i The average reward. A higher average reward indicates that the task allocation is more inclined to utilize that arm. The item to the right of the plus sign represents the utilization score, which is calculated as a... i The confidence interval width is determined by the arm's confidence score. When the total number of tasks is large but the arm completes few tasks, the confidence interval width is large, leading to a greater tendency to allocate tasks to explore that arm. In relation extraction task allocation and validation, the arm's confidence score is used to complete the task allocation. Then, by considering the results of multiple iterations of relation extraction, the task with the lowest reward is assigned to manual validation.
[0121] Step S6: If a rule was selected for relation extraction in step S5, then the relation extraction result S is obtained using a graph-based rule matching method. k,t .
[0122] like Figure 3 As shown, to expand the pattern matching method of rules, this embodiment can use semantic graphs to model the structure and semantic information of rules and sentences. Then, the similarity between the semantic graphs of rules and sentences is used as the matching score. To construct the semantic graphs of rules and sentences, this embodiment can set the semantic graph G = (V, E). V is a set of nodes, representing words in rules and sentences. Word embeddings of nodes are obtained based on a pre-trained language model to represent the semantic information of rules and sentences. E is a set of edges. The co-occurrence relationship of words within a fixed-size window is used to capture the word order information of rules and sentences.
[0123] In the rule semantic graph and sentence semantic graph Walking on can be formally defined as including
[0124] {v 0,,V1},{v 1, ,v2},…,{v i, ,v i+1 A finite sequence of edges}, where v i ∈V,{v i, ,v i+1}∈E. The length of the walk is the number of edges in the sequence. A path can be viewed as a special walk where all nodes are distinct. v i and v j Path length p between ij It can be viewed as the distance between these two nodes.
[0125] To capture deep structural information of rules and sentences, the shortest path of a node is used to represent the dependency relationship between a word and its neighboring words. Specifically, in the rule semantic graph... and sentence semantic graph A depth-first search is used to calculate the distance between nodes. If node v i No connection to v j And the distance from p ij If it is less than the length d, then v i and v j A new edge will be added between them as the shortest path, with edge weight W(e). m ) = 1 / p ij The strength of the relationship between words can be assessed by the weight of the edges.
[0126] Next, according to Calculation rule r i and sentence s k The graph similarity is used as the matching score:
[0127]
[0128] Among them, K node () is used for comparison Node similarity, Intended to assess Edge similarity. N is the normalization factor. If the matching score exceeds the predefined parameter λ, then sentence s k It can be done through rule r i Perform matching. Specifically, in calculating K... node In parentheses, the first evaluation was conducted. The semantic similarity of words. A common word set I = {(v...} is constructed based on word pairs whose semantic similarity exceeds a predefined parameter τ. a ,v b Based on a common vocabulary set, K is used. node () to measure and The similarity between nodes is defined as follows:
[0129]
[0130] Used for measurement The similarity of edges is defined as follows:
[0131]
[0132]
[0133] in, express From the node The edge. express From the node The edge of W(e) m ) and W(e n ) represents edge e m and e n The weights. The normalization factor can be calculated as:
[0134]
[0135] in, express The adjacency matrix, express A diagonal matrix. ∥·∥ F It is the F-norm of the matrix. By comprehensively comparing the nodes and edges of the rule and sentence semantic graph, the structural and semantic similarity between the rule and the sentence can be evaluated, and finally the rule-based relation extraction result can be obtained.
[0136] Step S7: If, in step S5, a pre-trained language model is selected for relation extraction, then the input to the pre-trained language model is constructed using a rule-integrated hint ensemble method, and the output of the pre-trained language model is constrained using a prototype-based mapper method to obtain the relation extraction result S. k,t .
[0137] The rule-based suggestion ensemble method specifically refers to integrating the rules with the highest relevance to sentences into the suggestion templates of the pre-trained language model, taking into account the sensitivity of the pre-trained language model to suggestions. Specifically, the labeled dataset is divided into a support set and a query set. The support set is used to provide suggestions to the pre-trained language model, while the query set is used to provide feedback and optimize the pre-trained language model. For each sentence s in the support set and query set... k Based on its matching rules, the following prompt template can be constructed.
[0138]
[0139] Among them, s k .w s and s k .w o It is s k The subject and object. s k The rule with the highest matching score.
[0140] Furthermore, a prompting integration strategy is utilized to help the pre-trained language model learn semantic information from inputs that support set rules and sentences. This is achieved by modifying the basic prompting template. The location of [Mask] is used to generate multiple hints:
[0141]
[0142] like Figure 4 As shown, the prototype-based mapper method in step S7 specifically refers to the following: To reduce the impact of irrelevant information between rules and sentences on the pre-trained language model, this embodiment of the application can construct a relation label prototype network based on an attention mechanism by aggregating labeled sentences from the support set. Then, by comparing the semantic similarity between the task and the relation prototype in the query set, the pre-trained language model can predict and map to relation labels. Specifically, by incorporating a rule-based prompting ensemble method, sentence s can be obtained. k ∈C s The prompt template is then used to predict [Mask] in the prompt template based on a pre-trained language model, in order to obtain s k Vector representation of:
[0143] h k =M(T(s) k ))(15)
[0144] Next, the relation prototype y is constructed based on the vector representations of sentences with relation labels in the support set. n To filter h in prototype building k Weakly relevant and irrelevant semantic information is extracted from the support set using an instance-level attention mechanism, focusing on semantic information relevant to the query set.
[0145]
[0146] Among them, Proto n Indicates label y n The prototype. C n Indicates a label y n The sentence. Indicates the construction of Proto n hour The weight.
[0147]
[0148]
[0149] Among them, W p It is a learnable weight matrix, ⊙ represents the element-wise dot product, θ() is the activation function, and sum() represents the sum of all elements of the vector.
[0150] Next, through the method in step S7, this embodiment of the application can obtain the sentence s in the query set. i The probability of a relation label can be calculated by s. i To tag y n Generate by distance from the prototype:
[0151]
[0152] Here, d() is a distance function. Finally, the loss is calculated using the cross-entropy loss function, and the pre-trained language model is optimized based on the query set.
[0153]
[0154] Step S8: Determine the annotation result L of round t. t The annotation result L of the (t-1)th round t-1 If the two annotation results are identical, the labeled data is directly deleted from the unlabeled dataset, i.e., C. u =C u / S k,t If the annotation results are different, they will be assigned to manual annotation.
[0155] Step S9: Repeat steps S3-S8 until the unlabeled dataset C is reached. u Stop iterating when the value is empty or when the maximum number of iterations is reached.
[0156] In summary, the embodiments of this application can solve the problems of erroneous triples and missing triples in weakly supervised relation extraction: To reduce missing triples, the embodiments of this application employ a semantically enhanced rule discovery method, considering the semantic information of rule mining and matching based on a pre-trained language model. To reduce erroneous triples, the embodiments of this application employ a structure-aware pre-trained language model prompting method to adaptively incorporate rules into the prompts and lexical mapper of the pre-trained language model. To improve the overall quality of relation extraction, the embodiments of this application utilize an active learning strategy based on a slot machine, dynamically validating the relation extraction results of the rules and the pre-trained language model based on the historical performance of the rules and the pre-trained language model with limited human resources.
[0157] This application's embodiments are based on a rule discovery method enhanced by a pre-trained language model. This method considers both pattern and semantic information in rule generation and matching, expanding the rule coverage. It utilizes a rule-induced structured pre-trained language model hint method. Rules are integrated into the hint set mapper of the pre-trained language model, helping it learn domain knowledge. Finally, to improve the overall quality of relation extraction, a slot machine-based active learning strategy is proposed. This strategy enables dynamic verification of relation extraction results with limited human resources, ultimately generating a high-quality data annotation set.
[0158] The weakly supervised relation extraction method based on collaborative rule discovery and prompting learning proposed in this application can acquire labeled and unlabeled datasets from a corpus. A mining rule set is obtained by mining the labeled dataset. If the unlabeled dataset is not empty, unlabeled data is extracted from the unlabeled dataset for labeling. Relations are then extracted from the labeled data based on the mining rule set and the target relation extraction method to obtain the corresponding relation extraction results. The method then determines whether the relation extraction results meet preset deletion conditions. If they do, the labeled data is deleted from the unlabeled dataset, and the next round of labeling begins. If not, the labeled data is manually labeled. This allows for relation extraction through collaborative rule discovery and prompting learning in a weakly supervised environment, achieving bidirectional empowerment of data and knowledge between rules and pre-trained language models, thereby improving the precision and recall of relation extraction. This solves the technical problem in related technologies where errors and omissions in relation extraction results are unavoidable due to the limited amount of labeled data in a weakly supervised environment.
[0159] Next, referring to the accompanying drawings, a weakly supervised relationship extraction device for collaborative rule discovery and prompting learning proposed according to an embodiment of this application is described.
[0160] Figure 5 This is a block diagram of a weakly supervised relationship extraction device for collaborative rule discovery and prompting learning according to an embodiment of this application.
[0161] like Figure 5 As shown, the weakly supervised relation extraction device 10 for collaborative rule discovery and prompting learning includes: a partitioning module 100, a mining module 200, an extraction module 300, and a judgment module 400.
[0162] Specifically, the segmentation module 100 is used to segment the data in the corpus to obtain labeled datasets and unlabeled datasets.
[0163] Mining module 200 is used to learn from prompts and mine labeled datasets to obtain a set of mining rules.
[0164] The extraction module 300 is used to determine whether the unlabeled dataset is empty. If the unlabeled dataset is not empty, it extracts unlabeled data from the unlabeled dataset for labeling to obtain preliminary labeled data. Then, it extracts relations from the preliminary labeled data based on the mining rule set and the target relation extraction method to obtain the corresponding relation extraction results.
[0165] The judgment module 400 is used to determine whether the preliminary labeled data meets the preset deletion conditions based on the relation extraction results. If the preset deletion conditions are met, the preliminary labeled data is deleted from the unlabeled dataset; otherwise, the preliminary labeled data is added to the manual validation set.
[0166] Optionally, in one embodiment of this application, the mining module 200 includes: a processing unit, a first mining unit, and a second mining unit.
[0167] The processing unit is used to preprocess the labeled dataset to obtain a text dataset.
[0168] The first mining unit is used to compare multiple text sequences in the text dataset, mine common subsequences that meet preset mining conditions as the main rules, and obtain the mining results.
[0169] The second mining unit is used to mine common subsequences that meet preset mining conditions from the remaining text sequence based on the mining results, until the preset stopping condition is reached, so as to obtain a mining rule set.
[0170] Optionally, in one embodiment of this application, the extraction module 300 includes: a first construction unit, a calculation unit, and a first acquisition unit.
[0171] The first construction unit is used to construct a rule semantic graph and a sentence semantic graph based on the structure and semantic information of sentences in the mining rule set and the preliminary labeled data, respectively.
[0172] The computational unit is used to calculate the similarity between the rule semantic graph and the sentence semantic graph.
[0173] The first acquisition unit is used to obtain the relationship extraction results based on similarity.
[0174] Optionally, in one embodiment of this application, the extraction module 300 includes a second construction unit and a constraint unit.
[0175] The second construction unit is used to extract the rules with the highest relevance to the sentences in the preliminary labeled data from the mining rule set, and integrate the rules into the prompt template of the pre-trained language model to construct the input of the pre-trained language model.
[0176] Constraint units are used to constrain the output of a pre-trained language model using a prototype-based mapper method to obtain relation extraction results.
[0177] Optionally, in one embodiment of this application, the judgment module 400 includes: a second acquisition unit, a judgment unit, and a determination unit.
[0178] The second acquisition unit is used to acquire the relationship extraction results from the previous round.
[0179] The judgment unit is used to determine whether the relation extraction result is consistent with the relation extraction result of the previous round.
[0180] The determination unit is used to determine whether the preliminary labeled data meets the preset deletion conditions if the relation extraction result is consistent with the relation extraction result of the previous round.
[0181] It should be noted that the explanation of the aforementioned embodiment of the weakly supervised relation extraction method for collaborative rule discovery and prompting learning also applies to the weakly supervised relation extraction device for collaborative rule discovery and prompting learning in this embodiment, and will not be repeated here.
[0182] The weakly supervised relation extraction device based on collaborative rule discovery and prompting learning proposed in this application can acquire labeled and unlabeled datasets from a corpus. It obtains a mining rule set by mining the labeled dataset, and, provided the unlabeled dataset is not empty, extracts and labels unlabeled data from it. Relations are then extracted from the labeled data based on the mining rule set and the target relation extraction method to obtain the corresponding relation extraction results. The device then determines whether the relation extraction results meet preset deletion conditions. If they do, the labeled data is deleted from the unlabeled dataset, and the next round of labeling begins. If not, the labeled data is manually labeled. This allows for relation extraction through collaborative rule discovery and prompting learning in a weakly supervised environment, achieving bidirectional empowerment of data and knowledge between rules and pre-trained language models, thereby improving the precision and recall of relation extraction. This solves the technical problem in related technologies where errors and omissions in relation extraction results are unavoidable due to the limited amount of labeled data in a weakly supervised environment.
[0183] Figure 6 A schematic diagram of the structure of an electronic device provided in an embodiment of this application. The electronic device may include:
[0184] The memory 601, the processor 602, and the computer program stored on the memory 601 and capable of running on the processor 602.
[0185] When the processor 602 executes the program, it implements the weakly supervised relationship extraction method for collaborative rule discovery and prompting learning provided in the above embodiments.
[0186] Furthermore, electronic devices also include:
[0187] Communication interface 603 is used for communication between memory 601 and processor 602.
[0188] The memory 601 is used to store computer programs that can run on the processor 602.
[0189] The memory 601 may include high-speed RAM memory, and may also include non-volatile memory, such as at least one disk storage device.
[0190] If the memory 601, processor 602, and communication interface 603 are implemented independently, then the communication interface 603, memory 601, and processor 602 can be interconnected via a bus to complete communication between them. The bus can be an Industry Standard Architecture (ISA) bus, a Peripheral Component Interconnect (PCI) bus, or an Extended Industry Standard Architecture (EISA) bus, etc. Buses can be categorized as address buses, data buses, control buses, etc. For ease of representation, Figure 6 The bus is represented by a single thick line, but this does not mean that there is only one bus or one type of bus.
[0191] Optionally, in a specific implementation, if the memory 601, processor 602, and communication interface 603 are integrated on a single chip, then the memory 601, processor 602, and communication interface 603 can communicate with each other through an internal interface.
[0192] The processor 602 may be a central processing unit (CPU), an application specific integrated circuit (ASIC), or one or more integrated circuits configured to implement the embodiments of this application.
[0193] This embodiment also provides a computer-readable storage medium storing a computer program that, when executed by a processor, implements the weakly supervised relation extraction method for collaborative rule discovery and prompting learning as described above.
[0194] This application also provides a computer program product, including a computer program that, when executed by a processor, implements the weakly supervised relation extraction method for collaborative rule discovery and prompting learning provided in this embodiment of the invention.
[0195] In the description of this specification, the references to terms such as "one embodiment," "some embodiments," "example," "specific example," or "some examples," etc., indicate that a specific feature, structure, material, or characteristic described in connection with that embodiment or example is included in at least one embodiment or example of this application. In this specification, the illustrative expressions of the above terms do not necessarily refer to the same embodiment or example. Furthermore, the specific features, structures, materials, or characteristics described may be combined in any suitable manner in one or more embodiments or examples. Moreover, without contradiction, those skilled in the art can combine and integrate the different embodiments or examples described in this specification, as well as the features of different embodiments or examples.
[0196] Furthermore, the terms "first" and "second" are used for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the number of technical features indicated. Thus, a feature defined as "first" or "second" may explicitly or implicitly include at least one of that feature. In the description of this application, "N" means at least two, such as two, three, etc., unless otherwise explicitly specified.
[0197] Any process or method described in the flowchart or otherwise herein can be understood as representing a module, segment, or portion of code comprising one or N executable instructions for implementing custom logic functions or processes, and the scope of the preferred embodiments of this application includes additional implementations in which functions may be performed not in the order shown or discussed, including substantially simultaneously or in reverse order depending on the functions involved, as should be understood by those skilled in the art to which embodiments of this application pertain.
[0198] The logic and / or steps represented in the flowchart or otherwise described herein, for example, can be considered as a sequenced list of executable instructions for implementing logical functions, and can be embodied in any computer-readable medium for use by, or in conjunction with, an instruction execution system, apparatus, or device (such as a computer-based system, a processor-included system, or other system that can fetch and execute instructions from, an instruction execution system, apparatus, or device). For the purposes of this specification, "computer-readable medium" can be any means that can contain, store, communicate, propagate, or transmit programs for use by, or in conjunction with, an instruction execution system, apparatus, or device. More specific examples (a non-exhaustive list) of computer-readable media include: an electrical connection having one or more wires (electronic device), a portable computer disk drive (magnetic device), random access memory (RAM), read-only memory (ROM), erasable and editable read-only memory (EPROM or flash memory), fiber optic devices, and portable optical disc read-only memory (CDROM). Alternatively, the computer-readable medium may be paper or other suitable media on which the program can be printed, since the program can be obtained electronically by optically scanning the paper or other medium, followed by editing, interpreting, or otherwise processing as necessary, and then stored in a computer memory.
[0199] It should be understood that the various parts of this application can be implemented using hardware, software, firmware, or a combination thereof. In the above embodiments, the N steps or methods can be implemented using software or firmware stored in memory and executed by a suitable instruction execution system. For example, if implemented in hardware as in another embodiment, it can be implemented using any one or a combination of the following techniques known in the art: discrete logic circuits having logic gates for implementing logical functions on data signals, application-specific integrated circuits (ASICs) having suitable combinational logic gates, programmable gate arrays (PGAs), field-programmable gate arrays (FPGAs), etc.
[0200] Those skilled in the art will understand that all or part of the steps of the methods in the above embodiments can be implemented by a program instructing related hardware. The program can be stored in a computer-readable storage medium, and when executed, the program includes one or a combination of the steps of the method embodiments.
[0201] Furthermore, the functional units in the various embodiments of this application can be integrated into a processing module, or each unit can exist physically separately, or two or more units can be integrated into a module. The integrated module can be implemented in hardware or as a software functional module. If the integrated module is implemented as a software functional module and sold or used as an independent product, it can also be stored in a computer-readable storage medium.
[0202] The storage medium mentioned above can be a read-only memory, a disk, or an optical disk, etc. Although embodiments of this application have been shown and described above, it is understood that the above embodiments are exemplary and should not be construed as limiting this application. Those skilled in the art can make changes, modifications, substitutions, and variations to the above embodiments within the scope of this application.
Claims
1. A weakly supervised relation extraction method for collaborative rule discovery and prompting learning, characterized in that, Includes the following steps: The data in the corpus is divided into labeled and unlabeled datasets; Based on prompts, the labeled dataset is mined to obtain a set of mining rules; Determine whether the unlabeled dataset is an empty set. If the unlabeled dataset is not an empty set, extract unlabeled data from the unlabeled dataset for labeling to obtain preliminary labeled data. Then, extract relations from the preliminary labeled data based on the mining rule set and target relation extraction method to obtain the corresponding relation extraction results. Based on the relation extraction results, it is determined whether the preliminary labeled data meets the preset deletion conditions. If the preset deletion conditions are met, the preliminary labeled data is deleted from the unlabeled dataset; otherwise, the preliminary labeled data is added to the manual validation set.
2. The method according to claim 1, characterized in that, The step of learning from prompts to mine the labeled dataset to obtain a set of mining rules includes: The labeled dataset is preprocessed to obtain a text dataset; By comparing multiple text sequences in the text dataset, common subsequences that meet preset mining conditions are extracted as the main rules to obtain the mining results; Based on the mining results, common subsequences that satisfy the preset mining conditions are mined from the remaining text sequences until the preset stopping condition is reached, so as to obtain the mining rule set.
3. The method according to claim 1, characterized in that, The step of extracting relations from the initially labeled data based on the mining rule set and the target relation extraction method to obtain the corresponding relation extraction results includes: Based on the mining rule set and the structural and semantic information of sentences in the preliminary labeled data, rule semantic graphs and sentence semantic graphs are constructed respectively. Calculate the similarity between the rule semantic graph and the sentence semantic graph; The relationship extraction result is obtained based on the similarity.
4. The method according to claim 1, characterized in that, The step of extracting relations from the initially labeled data based on the mining rule set and the target relation extraction method to obtain the corresponding relation extraction results includes: The rules with the highest relevance to the sentences in the preliminary labeled data are extracted from the mining rule set, and the rules are integrated into the prompt template of the pre-trained language model to construct the input of the pre-trained language model; The output of the pre-trained language model is constrained using a prototype-based mapper method to obtain the relation extraction result.
5. The method according to claim 1, characterized in that, The step of determining whether the preliminary labeled data meets the preset deletion conditions based on the relation extraction results includes: Retrieve the results of the previous round of relation extraction; Determine whether the relationship extraction result is consistent with the relationship extraction result of the previous round; If the relationship extraction result is consistent with the previous round of relationship extraction result, then the preliminary labeled data is determined to meet the preset deletion condition.
6. A weakly supervised relation extraction device for collaborative rule discovery and prompting learning, characterized in that, include: The partitioning module is used to partition the data in the corpus, resulting in labeled and unlabeled datasets. The mining module is used to learn and mine the labeled dataset based on prompts to obtain a set of mining rules; The extraction module is used to determine whether the unlabeled dataset is an empty set. If the unlabeled dataset is not an empty set, unlabeled data is extracted from the unlabeled dataset for labeling to obtain preliminary labeled data. Relationships are extracted from the preliminary labeled data based on the mining rule set and the target relation extraction method to obtain the corresponding relation extraction results. The judgment module is used to determine whether the preliminary labeled data meets the preset deletion conditions based on the relation extraction results. If the preset deletion conditions are met, the preliminary labeled data is deleted from the unlabeled dataset; otherwise, the preliminary labeled data is added to the manual validation set.
7. The apparatus according to claim 6, characterized in that, The mining module includes: The processing unit is used to preprocess the labeled dataset to obtain a text dataset; The first mining unit is used to compare multiple text sequences in the text dataset, mine common subsequences that meet preset mining conditions as the rule body, and obtain the mining results. The second mining unit is used to mine common subsequences that satisfy the preset mining conditions from the remaining text sequence based on the mining results, until the preset stopping condition is reached, so as to obtain the mining rule set.
8. The apparatus according to claim 6, characterized in that, The extraction module includes: The first construction unit is used to construct a rule semantic graph and a sentence semantic graph based on the mining rule set and the structural and semantic information of sentences in the preliminary labeled data, respectively. A computing unit is used to calculate the similarity between the rule semantic graph and the sentence semantic graph; The acquisition unit is used to obtain the relationship extraction result based on the similarity.
9. An electronic device, characterized in that, include: The memory, the processor, and the computer program stored in the memory and executable on the processor, the processor executing the program to implement the weakly supervised relation extraction method for collaborative rule discovery and prompting learning as described in any one of claims 1-5.
10. A computer-readable storage medium having a computer program stored thereon, characterized in that, The program is executed by the processor to implement the weakly supervised relation extraction method for collaborative rule discovery and prompting learning as described in any one of claims 1-5.