Parallel knowledge extraction method and device based on large model

Abstract

This invention provides a parallel knowledge extraction method and apparatus based on a large model. The method includes: inputting target text into a knowledge extraction model, and using three extraction modules in the knowledge extraction model to execute their respective extraction tasks in parallel and synchronously, outputting candidate sets of relations, entities, and facts respectively. The knowledge extraction model includes a basic large model and three independent extraction modules: relation extraction module, entity extraction module, and fact extraction module. The Cartesian product operation is performed on the candidate sets of relations, entities, and facts to generate a candidate triplet set. Semantic consistency verification is performed on each candidate triplet in the candidate triplet set, and its semantic similarity is calculated. The semantic similarity is compared with a first preset threshold, and the target triplet is determined based on the comparison result. This invention significantly improves the efficiency, recall, and accuracy of knowledge extraction through parallel extraction, full combination, and semantic consistency verification.

Description

Technical Field

[0001] This invention relates to the field of artificial intelligence technology, and in particular to a method and apparatus for parallel knowledge extraction based on a large model. Background Technology

[0002] In the fields of natural language processing and knowledge graph construction, knowledge extraction is a core technical step. Traditional methods are mostly based on pre-trained models such as BERT for sequence labeling or classification. However, these methods have weak generalization ability, their performance drops significantly after cross-domain transfer, and the extraction process and results lack interpretability, making it difficult to guarantee the reliability of the results.

[0003] In recent years, knowledge extraction schemes based on large language models have gradually emerged, but existing schemes still have significant drawbacks: First, the extraction process generally adopts a serial pipeline mode of "relation-entity-fact", with high coupling between modules. Subsequent tasks can only be started after the output of the preceding module, resulting in low overall processing efficiency, and errors in a single step will prevent subsequent tasks from executing. Second, candidate element combinations mostly rely on fixed templates, which limit the types of relations and combination patterns that can be covered, making it easy to miss potentially valid knowledge triples and resulting in low recall. Third, the extraction results lack an automated verification mechanism, and erroneous triples cannot be effectively identified and filtered, leading to a high false positive rate. Fourth, the model is difficult to support independent optimization and iteration of a single module, and the cost of later expansion is high.

[0004] Therefore, there is a lack of knowledge extraction methods and devices in the existing technology that can achieve efficient parallel extraction, full candidate combination and semantic consistency verification, so as to systematically improve extraction efficiency, recall and accuracy. Summary of the Invention

[0005] Therefore, it is necessary to provide a parallel knowledge extraction method and apparatus based on a large model to address the aforementioned technical problems.

[0006] In a first aspect, the present invention provides a parallel knowledge extraction method based on a large model, comprising:

[0007] The target text is input into the knowledge extraction model, and the three extraction modules in the knowledge extraction model are used to execute their respective extraction tasks in parallel and synchronously, and output the candidate set of relations, the candidate set of entities, and the candidate set of facts respectively. The knowledge extraction model includes a basic large model and three independent extraction modules, namely the relation extraction module, the entity extraction module, and the fact extraction module.

[0008] Perform a Cartesian product operation on the candidate sets of relations, entities, and facts to generate a set of candidate triples;

[0009] Perform semantic consistency verification on each candidate triplet in the candidate triplet set and calculate its semantic similarity;

[0010] The semantic similarity is compared with a first preset threshold, and the target triplet is determined based on the comparison result.

[0011] Optionally, after performing semantic consistency verification on each candidate triplet in the candidate triplet set and calculating its semantic similarity, and before comparing the semantic similarity with a first preset threshold and determining the target triplet based on the comparison result, the method further includes:

[0012] Candidate triples with semantic similarity less than a second preset threshold are marked as failed candidates;

[0013] Calculate the failure rate of each relation, entity, and fact in the candidate triples in which it participates, where the failure rate of any relation, entity, or fact is equal to the number of candidate triples in which it participates that are marked as failure candidates divided by the total number of all candidate triples in which the relation, entity, or fact participates.

[0014] If the failure rate of a single relation, entity, or fact reaches a first preset proportion, the relation, entity, or fact will be removed from the corresponding candidate set of relations, entity, or fact.

[0015] Based on the deleted candidate sets of relations, entities, and facts, perform the Cartesian product operation again to generate a new set of candidate triples.

[0016] Semantic consistency verification is performed again for each candidate triple in the new set of candidate triples, and their semantic similarity is calculated.

[0017] Optionally, the step of performing semantic consistency verification on each candidate triplet in the candidate triplet set and calculating its semantic similarity includes:

[0018] For each candidate triple in the candidate triple set, concatenate the relation and entity into the first text, and use the fact as the second text;

[0019] The first text and the second text are encoded using a pre-trained dual encoder semantic similarity model to obtain the first semantic vector and the second semantic vector.

[0020] Calculate the cosine similarity between the first semantic vector and the second semantic vector, and use the cosine similarity as the semantic similarity of the candidate triplet.

[0021] Optionally, comparing semantic similarity with a first preset threshold and determining the target triplet based on the comparison result includes:

[0022] The semantic similarity is compared with a first preset threshold, and the candidate triples with semantic similarity greater than or equal to the first preset threshold are taken as the first candidate set;

[0023] Candidate triples whose semantic similarity is less than the first preset threshold and greater than or equal to the second preset threshold are used as the second candidate set, wherein the second preset threshold is less than the first preset threshold;

[0024] Candidate triples with semantic similarity less than the second preset threshold are discarded;

[0025] Each candidate triplet from the first and second candidate sets is input into the target text and the large language model. The large language model then determines whether the candidate triplet is correct.

[0026] The candidate triples that are judged as correct by the large language model are used as the target triples.

[0027] Optionally, the method further includes:

[0028] Obtain candidate triples that are judged as incorrect by the large language model, and calculate the failure rate caused by entity error, relation error, and fact error.

[0029] If the failure rate caused by entity errors, relationship errors, or factual errors exceeds a second preset rate, a targeted re-extraction of the corresponding extraction module is triggered. The targeted re-extraction includes adjusting the temperature parameters of the extraction module and / or adding constraint prompts.

[0030] Optionally, if the failure rate due to entity errors, relationship errors, or factual errors exceeds a second preset rate, a targeted re-extraction of the corresponding extraction module is triggered, including:

[0031] If the corresponding extraction module is an entity extraction module, generate a first constraint prompt word and adjust the temperature parameter of the extraction module to 0.2;

[0032] If the corresponding extraction module is a relation extraction module, generate a second constraint prompt word and adjust the temperature parameter of the extraction module to 0.3;

[0033] If the corresponding extraction module is a fact extraction module, generate a third constraint prompt word and adjust the temperature parameter of the extraction module to 0.2;

[0034] After concatenating the corresponding constraint prompts with the target text, re-enter the corresponding extraction module to extract the new candidate set.

[0035] Optionally, after concatenating the corresponding constraint prompts with the target text and re-entering them into the corresponding extraction module to obtain a new candidate set, the process further includes:

[0036] Replace the original candidate set with the new candidate set, and perform the Cartesian product operation again based on the new candidate set to generate an updated candidate triplet set;

[0037] For each candidate triplet in the updated candidate triplet set, semantic consistency verification is performed again, and its semantic similarity is recalculated.

[0038] The recalculated semantic similarity is compared with the first preset threshold, and the target triplet is re-determined based on the comparison result.

[0039] Optionally, the first preset threshold value ranges from [0.75, 0.85], and the second preset threshold value ranges from [0.2, 0.4].

[0040] Optionally, the basic large model is the Vicuna-7b-v1.5 large language model, and the relation extraction module, entity extraction module and fact extraction module are all low-rank adaptive modules.

[0041] Secondly, the present invention provides a parallel knowledge extraction device based on a large model, comprising:

[0042] A knowledge extraction model is used to receive target text and use three independent extraction modules embedded within it to execute their respective extraction tasks in parallel and synchronously, outputting candidate sets of relations, entity, and facts respectively. The knowledge extraction model includes a basic large model and three independent extraction modules, namely a relation extraction module, an entity extraction module, and a fact extraction module.

[0043] The combination module, connected to the knowledge extraction model, is used to perform Cartesian product operations on the candidate sets of relations, entities, and facts to generate a set of candidate triples.

[0044] The semantic verification module, connected to the combination module, is used to perform semantic consistency verification on each candidate triplet in the candidate triplet set and calculate its semantic similarity.

[0045] The determination module, connected to the semantic verification module, is used to compare semantic similarity with a first preset threshold and determine the target triplet based on the comparison result.

[0046] The parallel knowledge extraction method and apparatus based on a large model provided by this invention executes relation, entity, and fact extraction tasks in parallel through three independent extraction modules, breaking the module dependence of traditional serial pipelines and significantly improving knowledge extraction efficiency. By performing a full combination of Cartesian products on the candidate sets of relations, entities, and facts to generate candidate triples covering all potential matches, it effectively avoids knowledge omissions caused by template limitations or stepwise screening, improving recall from the source. Through a multi-level semantic consistency verification mechanism combining initial screening based on semantic similarity and secondary verification based on a large language model, it accurately filters out erroneous triples and outputs judgment reasons, greatly reducing the false positive rate and giving the extraction results good interpretability. By statistically analyzing the failure rate of each element to automatically identify and delete noisy elements, combined with an iterative feedback targeted re-extraction module, it further enhances the robustness and adaptability of the system. Attached Figure Description

[0047] Figure 1a A flowchart illustrating a parallel knowledge extraction method based on a large model provided in an embodiment of the present invention;

[0048] Figure 1b Another flowchart illustrating the parallel knowledge extraction method based on a large model provided in this embodiment of the invention;

[0049] Figure 1c This is another flowchart illustrating the parallel knowledge extraction method based on a large model provided in this embodiment of the invention.

[0050] Figure 2 A schematic diagram of the circuit module structure of the parallel knowledge extraction device based on a large model provided in an embodiment of the present invention;

[0051] Figure 3 This is an internal structural diagram of a computer device according to one embodiment of the present invention. Detailed Implementation

[0052] To make the objectives, technical solutions, and advantages of this invention clearer, the invention will be further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the invention.

[0053] like Figure 1a As shown, this invention provides a parallel knowledge extraction method based on a large model, including:

[0054] Step S10: Input the target text into the knowledge extraction model, and use the three extraction modules in the knowledge extraction model to execute their respective extraction tasks in parallel and synchronously, and output the relation candidate set, entity candidate set and fact candidate set respectively. The knowledge extraction model includes a basic large model and three independent extraction modules, namely the relation extraction module, entity extraction module and fact extraction module.

[0055] The basic large model and the three extraction modules used in this invention are all pre-trained models and modules using existing technologies, requiring no further training and can be directly used for knowledge extraction tasks. Specifically, the basic large model can be Vicuna-7b-v1.5, which has been pre-trained on a large-scale general corpus and possesses mature text semantic understanding capabilities. The relation extraction module, entity extraction module, and fact extraction module are all pre-trained low-rank adaptive modules (i.e., LoRA models), and their parameters have been efficiently fine-tuned using document-level knowledge extraction datasets (such as the Redocred dataset), each mastering the extraction knowledge for its corresponding task (relation classification, entity recognition, and fact generation).

[0056] Taking the target text "Einstein proposed the theory of relativity. He was born in Germany" as an example, the target text is input into a basic large-scale model (such as Vicuna-7b-v1.5). The basic large-scale model outputs a deep semantic representation of the target text. This deep semantic representation is simultaneously fed into three pre-trained LoRA modules: a relation extraction module, an entity extraction module, and a fact extraction module. The three extraction modules execute in parallel, outputting a relation candidate set R = {proposed, birthplace}, an entity candidate set E = {Einstein, relativity, Germany}, and a fact candidate set F = {proposed the theory of relativity, born in Germany, was a scientist}. It should be noted that the fact "was a scientist" in the fact candidate set F is a hallucinatory fact, not mentioned in the target text.

[0057] Step S20: Perform a Cartesian product operation on the candidate relation set, the candidate entity set, and the candidate fact set to generate a candidate triplet set;

[0058] Continuing the previous example, a Cartesian product is performed on the candidate relation set R = {proposed, birthplace}, the candidate entity set E = {Einstein, relativity, Germany}, and the candidate fact set F = {proposed relativity, born in Germany, a scientist}. This generates a candidate triplet set containing 2 × 3 × 3 = 18 candidate triplets, as follows: 1. (proposed, Einstein, proposed relativity); 2. (proposed, Einstein, born in Germany); 3. (proposed, Einstein, a scientist); 4. (proposed, relativity, proposed relativity); 5. (proposed, relativity, born in Germany); 6. (proposed, relativity, a scientist); 7. 1. (Proposed by Germany, proposed the theory of relativity); 8. (Proposed by Germany, born in Germany); 9. (Proposed by Germany, a scientist); 10. (Birthplace of Einstein, proposed the theory of relativity); 11. (Birthplace of Einstein, born in Germany); 12. (Birthplace of Einstein, a scientist); 13. (Birthplace of relativity, proposed the theory of relativity); 14. (Birthplace of relativity, born in Germany); 15. (Birthplace of relativity, a scientist); 16. (Birthplace of Germany, proposed the theory of relativity); 17. (Birthplace of Germany, born in Germany); 18. (Birthplace of Germany, a scientist). This set of candidate triples contains all possible combinations of relation-entity-fact, providing a complete candidate space for subsequent semantic consistency verification.

[0059] Step S30: Perform semantic consistency verification on each candidate triplet in the candidate triplet set and calculate its semantic similarity;

[0060] In one optional embodiment of the present invention, such as Figure 1b As shown, step S30 specifically includes:

[0061] Step S301: For each candidate triple in the candidate triple set, concatenate the relation and entity into the first text, and use the fact as the second text;

[0062] Step S302: Encode the first text and the second text using a pre-trained dual encoder semantic similarity model to obtain the first semantic vector and the second semantic vector respectively;

[0063] The pre-trained dual encoder semantic similarity model can adopt the Sentence-BERT model in the existing technology. Of course, those skilled in the art can flexibly choose other dual encoder semantic similarity models in the existing technology according to actual needs, which is not limited here.

[0064] Step S303: Calculate the cosine similarity between the first semantic vector and the second semantic vector, and use the cosine similarity as the semantic similarity of the candidate triplet.

[0065] Continuing the previous example, semantic consistency is verified for each of the 18 candidate triples generated above. For example, for the candidate triple (proposed, Einstein, proposed relativity), the relation "proposed" and the entity "Einstein" are concatenated to form the first text, i.e., "proposed Einstein," and the fact "proposed relativity" is used as the second text. A pre-trained dual-encoder semantic similarity model (such as the Sentence-BERT model) is used to encode the corresponding first and second semantic vectors respectively, and the cosine similarity is calculated to be 0.98, which is taken as the semantic similarity of this candidate triple. The semantic similarity of the remaining candidate triples is calculated in the same way, and will not be elaborated here.

[0066] Step S40: Compare the semantic similarity with the first preset threshold, and determine the target triplet based on the comparison result.

[0067] In an optional embodiment of the present invention, step S40 specifically includes:

[0068] Compare semantic similarity with a first preset threshold;

[0069] If the semantic similarity is greater than or equal to the first preset threshold, the candidate triplet corresponding to the semantic similarity is taken as the target triplet;

[0070] If the semantic similarity is less than the first preset threshold, the candidate triplet corresponding to the semantic similarity is discarded.

[0071] In another alternative embodiment of the invention, such as Figure 1c As shown, step S40 specifically includes:

[0072] Step S401: Compare the semantic similarity with the first preset threshold, and select the candidate triples whose semantic similarity is greater than or equal to the first preset threshold as the first candidate set;

[0073] Step S402: Select candidate triples whose semantic similarity is less than the first preset threshold and greater than or equal to the second preset threshold as the second candidate set, wherein the second preset threshold is less than the first preset threshold;

[0074] Step S403: Discard candidate triples whose semantic similarity is less than the second preset threshold;

[0075] Step S404: Input each candidate triple from the first candidate set and the second candidate set along with the target text into the large language model, and let the large language model determine whether the candidate triple is correct;

[0076] Step S405: The candidate triplet that the large language model determines to be correct is taken as the target triplet.

[0077] In this invention, the value range of the first preset threshold can be [0.75, 0.85], and the value range of the second preset threshold can be [0.2, 0.4].

[0078] The large language model used in this invention can be a pre-trained general-purpose large language model in the prior art, such as GPT-3.5 or GPT-4. Taking GPT-3.5 as an example, for each candidate triplet, it is concatenated with the target text according to the following prompt word template and then input into the model to determine the correctness of the candidate triplet and obtain the reason for the error:

[0079] Prompt template:

[0080] "Please determine whether the following knowledge triples are correct relative to the given original text."

[0081] If correct, simply output 'Correct'.

[0082] If an error occurs, please output 'Error, Reason:' followed by a brief explanation of the error (e.g., incorrect entity, mismatched relation, or fact does not exist).

[0083] Original text: {target text}

[0084] Candidate triples to be judged: ({entity}, {relation}, {fact})

[0085] After the large language model outputs its output, its correctness is determined by whether it begins with "Correct". If it begins with "Incorrect", the content after "Reason:" is extracted as the basis for error type classification (e.g., containing "entity" is classified as an entity error, containing "relation" as a relation error, and containing "fact" as a fact error). Those skilled in the art can also select other large language models with instruction understanding capabilities according to actual needs and adjust the prompt word template accordingly; no specific limitations are made here.

[0086] Continuing the previous example, assuming the first preset threshold is 0.75 and the second preset threshold is 0.4, the semantic similarity of each candidate triplet calculated in step S30 is as follows: 1. (Proposed by Einstein, who proposed the theory of relativity) has a semantic similarity of 0.98; 2. (Proposed by Einstein, born in Germany) has a semantic similarity of 0.15; 3. (Proposed by Einstein, a scientist) has a semantic similarity of 0.12; 4. (Proposed by Einstein, who proposed the theory of relativity) has a semantic similarity of 0.91; 5. (Proposed by Einstein, born in Germany) has a semantic similarity of 0.08; 6. (Proposed by Einstein, a scientist) has a semantic similarity of 0.06; 7. (Proposed by Germany, who proposed the theory of relativity) has a semantic similarity of 0.10; 8. (Proposed by Germany, born in Germany) has a semantic similarity of 0.89; 9. (Proposed by Germany) has a semantic similarity of 0.89; 10. (Proposed by Germany) has a semantic similarity of 0.91; 11. (Proposed by Einstein, born in Germany) has a semantic similarity of 0.91; 12. (Proposed by Einstein, a scientist) has a semantic similarity of 0.91; 13. (Proposed by Einstein, who proposed the theory of relativity) has a semantic similarity of 0.91; 14. (Proposed by Einstein, born in Germany) has a semantic similarity of 0.91; 15. (Proposed by Einstein, born in Germany) has a semantic similarity of 0.91; 16. (Proposed by Einstein, born in Germany) has a semantic similarity of 0.91; 17. (Proposed by Einstein, born in Germany) has a semantic similarity of The semantic similarity of the following pairs is as follows: 10. (Birthplace: Einstein, who proposed the theory of relativity) has a semantic similarity of 0.07; 11. (Birthplace: Einstein, born in Germany) has a semantic similarity of 0.97; 12. (Birthplace: Einstein, a scientist) has a semantic similarity of 0.11; 13. (Birthplace: Relativity, who proposed the theory of relativity) has a semantic similarity of 0.09; 14. (Birthplace: Relativity, born in Germany) has a semantic similarity of 0.05; 15. (Birthplace: Relativity, a scientist) has a semantic similarity of 0.04; 16. (Birthplace: Germany, who proposed the theory of relativity) has a semantic similarity of 0.13; 17. (Birthplace: Germany, born in Germany) has a semantic similarity of 0.93; 18. (Birthplace: Germany, a scientist) has a semantic similarity of 0.06.

[0087] The 18 semantic similarities mentioned above are compared with the first preset threshold.

[0088] Candidate triples with a semantic similarity ≥ 0.75 (first preset threshold) are selected as the first candidate set. The following triples meet the criteria: No. 1 (0.98), No. 4 (0.91), No. 8 (0.89), No. 11 (0.97), and No. 17 (0.93), for a total of 5 candidate triples.

[0089] Candidate triples with semantic similarity <0.75 (first preset threshold) and ≥0.4 (second preset threshold) are selected as the second candidate set. Since the semantic similarity of all remaining candidate triples is less than 0.4, the second candidate set is empty.

[0090] Candidates with a semantic similarity of <0.4 (second preset threshold) (numbers 2, 3, 5, 6, 7, 9, 10, 12, 13, 14, 15, 16, 18) are directly discarded.

[0091] The five candidate triples from the first candidate set are concatenated with the target text "Einstein proposed the theory of relativity. He was born in Germany." and then input into a large language model (such as GPT-3.5). The large language model then determines whether each candidate triple is correct. The results are as follows:

[0092] (Proposed by Einstein, who proposed the theory of relativity) → Correct;

[0093] (Proposed, the theory of relativity, proposed the theory of relativity) → Error, reason: the entity should be Einstein;

[0094] (Birthplace, Einstein, born in Germany) → Correct;

[0095] (Place of birth, Germany, born in Germany) → Error, reason: The entity should be a person;

[0096] (Proposed, Germany, Born in Germany) → Error, reason: The entity “Germany” does not match the relation “Proposed” and the fact “Born in Germany”.

[0097] The candidate triplet that the large model judges to be correct is taken as the target triplet, namely (proposed by Einstein, who proposed the theory of relativity) and (birthplace of Einstein, who was born in Germany).

[0098] Optionally, after step S404, the method of the present invention further includes:

[0099] Step S406: Obtain candidate triples that are judged as incorrect by the large language model, and count the failure rate caused by entity error, relation error, and fact error;

[0100] Step S407: If the failure rate caused by entity error, relationship error, or fact error is greater than the second preset rate, a targeted re-extraction of the corresponding extraction module is triggered, wherein the targeted re-extraction includes: adjusting the temperature parameter of the extraction module and / or adding constraint prompt words.

[0101] The second preset ratio can be flexibly set by those skilled in the art according to actual needs, and is not limited here. Preferably, the second preset ratio is 50%.

[0102] Optionally, in step S407, if the failure rate due to entity errors, relationship errors, or factual errors is greater than a second preset rate, targeted re-extraction of the corresponding extraction module is triggered, including:

[0103] Step S4071: If the corresponding extraction module is a solid extraction module, generate a first constraint prompt word and adjust the temperature parameter of the extraction module to 0.2;

[0104] The first constraint prompt can be "Please only output the names of specific people, organizations or works that appear in the target text, and only output the most standard name for the same entity". Those skilled in the art can set it flexibly according to actual needs, and there is no limitation here.

[0105] Step S4072: If the corresponding extraction module is a relation extraction module, generate a second constraint prompt word and adjust the temperature parameter of the extraction module to 0.3;

[0106] The second constraint prompt can be "Please only output the relationship types that are clearly present in the text". Those skilled in the art can set it flexibly according to actual needs, and there is no limitation here.

[0107] Step S4073: If the corresponding extraction module is a fact extraction module, generate a third constraint prompt word and adjust the temperature parameter of the extraction module to 0.2;

[0108] The third constraint prompt can be "Please extract only the facts explicitly stated in the text, and do not infer or add external knowledge." Those skilled in the art can set it flexibly according to actual needs, and there is no limitation here.

[0109] During targeted resampling, the temperature parameter controls the sharpness of the probability distribution output by the extraction module: the higher the temperature, the smoother the probability distribution, and the extraction module tends to generate more diverse candidate results, but it may also introduce more illusions or noise; the lower the temperature, the sharper the probability distribution, and the extraction module tends to select the output with the highest probability, resulting in a more conservative and certain outcome.

[0110] When a certain extraction module (such as the entity extraction module) produces too many irrelevant entities, causing the failure rate to exceed the second preset ratio, it indicates that the current generation strategy of the extraction module is too divergent, generating a large amount of noise. In this case, reducing the temperature parameter (e.g., from the default value of 1.0 to 0.2 or 0.3) can make the output of the extraction module more focused on high-confidence candidate results and suppress low-probability irrelevant outputs, thereby obtaining a more accurate and reliable candidate set in subsequent re-extraction.

[0111] Conversely, if the extraction module experiences a high failure rate due to overly simplistic output and missing potentially correct candidate results, the temperature can be appropriately increased to enhance diversity. By dynamically adjusting the temperature parameter, this invention can quickly adjust the behavior of the extraction module without retraining it, achieving lightweight and low-cost targeted optimization.

[0112] Step S4074: After concatenating the corresponding constraint prompts with the target text, re-enter the corresponding extraction module to extract the new candidate set.

[0113] Optionally, after step S4074, the method of the present invention further includes:

[0114] Step S4075: Replace the original candidate set with the new candidate set, and perform the Cartesian product operation again based on the new candidate set to generate an updated candidate triplet set;

[0115] Step S4076: Perform semantic consistency verification again for each candidate triple in the updated candidate triple set and recalculate its semantic similarity;

[0116] Step S4077: Compare the recalculated semantic similarity with the first preset threshold, and redetermine the target triplet based on the comparison result.

[0117] In this invention, the specific processes of steps S4076 and S4077 can be referred to the specific description of steps S30 and S40, and will not be repeated here.

[0118] Continuing the previous example, let's retrieve the candidate triples that the large language model judged as incorrect. Based on the judgment results of the previous example, 3 out of the 5 candidate triples in the first candidate set were judged as incorrect, as follows:

[0119] (Proposed, Relativity, Proposed Relativity): Error, due to entity error (entity should be Einstein); (Birthplace, Germany, Born in Germany): Error, due to entity error (entity should be person); (Proposed, Germany, Born in Germany): Error, due to entity error (entity "Germany" does not match the relation "proposed" and the fact "born in Germany").

[0120] Error types: Entity errors account for 100% (3 / 3) of the failure rate, while relation and fact errors are both 0%. A second preset percentage (e.g., 50%) is set. If the entity error rate of 100% exceeds this preset percentage, a targeted re-extraction of the entity extraction module is triggered.

[0121] Targeted re-extraction was performed on the entity extraction module: the temperature parameter of the entity extraction module was reduced from the default value of 1.0 to 0.2, making its output more conservative and reducing illusions; at the same time, a first constraint prompt was added: "Please only output the names of specific people, organizations, or works that appear in the text, and output only one most standard name for the same entity, such as 'Einstein', do not output 'relativity', 'Germany', etc." The first constraint prompt was concatenated with the target text "Einstein proposed the theory of relativity. He was born in Germany." (e.g., write the first constraint prompt first, then write the target text on a new line) and then re-entered into the entity extraction module for extraction. After re-extraction, the new entity candidate set only contained {Einstein}. Subsequently, based on the new entity candidate set, the Cartesian product operation, semantic consistency verification, and secondary verification of the large language model were re-executed, finally obtaining the correct target triples: (Einstein, proposed, relativity) and (Einstein, birthplace, Germany). Through targeted re-extraction, the noise output of the entity extraction module was effectively reduced, significantly improving the accuracy of subsequent knowledge extraction.

[0122] It should be noted that in the example above, the failure rate due to entity errors alone exceeded the second preset threshold, therefore, targeted re-extraction was only performed on the entity extraction module. If the failure rate due to relation errors or factual errors exceeds the second preset threshold, the specific targeted re-extraction process can be referred to the targeted re-extraction process for the entity extraction module, which will not be repeated here.

[0123] In an optional embodiment of the present invention, after step S30 and before step S40, the method of the present invention further includes:

[0124] Step S31: Mark candidate triples with semantic similarity less than the second preset threshold as failed candidates;

[0125] Step S32: Calculate the failure rate of each relation, each entity, and each fact in the candidate triples in which it participates, where the failure rate of any relation, entity, or fact is equal to the number of candidates marked as failure candidates in the candidate triples in which it participates divided by the total number of all candidate triples in which the relation, entity, or fact participates.

[0126] Step S33: If the failure rate of a single relation, entity, or fact reaches a first preset ratio, the relation, entity, or fact is removed from the corresponding relation candidate set, entity candidate set, or fact candidate set.

[0127] The first preset ratio can be flexibly set by those skilled in the art according to actual needs, and is not limited here. Preferably, the first preset ratio is 100%.

[0128] Step S34: Perform the Cartesian product operation again based on the deleted candidate sets of relations, entities, and facts to generate a new set of candidate triples;

[0129] Step S35: Perform semantic consistency verification on each candidate triple in the new candidate triple set and calculate its semantic similarity.

[0130] Continuing with the previous example, candidates with a semantic similarity of less than 0.4 (the second preset threshold) are marked as failed candidates. As seen in the example, the similarity of 13 candidate triples (numbered 2, 3, 5, 6, 7, 9, 10, 12, 13, 14, 15, 16, and 18) is all <0.4, and they are marked as failed candidates; the similarity of triples numbered 1, 4, 8, 11, and 17 is ≥0.4, and they are not marked.

[0131] The failure rate of each relation, entity, and fact in its candidate triples is calculated separately, as follows:

[0132] Relationship "Proposal": There are 9 candidate triples (numbered 1-9), of which 6 failed candidates are numbered 2, 3, 5, 6, 7, and 9. The failure rate is approximately 66.7% (6 / 9).

[0133] Relationship "birthplace": Participated in 9 candidate triples (numbered 10-18), with failed candidates numbered 10, 12, 13, 14, 15, 16, and 18 (7 in total), resulting in a failure rate of 7 / 9 ≈ 77.8%.

[0134] Entity "Einstein": Participated in 6 candidate triples (1,2,3,10,11,12), with 4 failed candidates numbered 2,3,10,12, resulting in a failure rate of ≈66.7%.

[0135] Entity "Relativity": Participated in 6 candidate triplets (4,5,6,13,14,15), with 5 candidates (numbered 5,6,13,14,15) failing, resulting in a failure rate of ≈83.3%.

[0136] Entity "Germany": Participated in 6 candidate triples (7,8,9,16,17,18), with 4 failed candidates numbered 7,9,16,18, resulting in a failure rate of ≈66.7%.

[0137] The fact “proposed the theory of relativity”: among the 6 candidate triplets (1, 4, 7, 10, 13, 16), the failed candidates were numbered 7, 10, 13, 16 (4), with a failure rate of ≈66.7%.

[0138] The fact that the candidate was "born in Germany" is as follows: 6 candidate triples were participated in (2, 5, 8, 11, 14, 17), and 2, 5, and 14 (3 candidates) failed, resulting in a failure rate of 50%.

[0139] The fact is that "a scientist" participated in 6 candidate triples (3, 6, 9, 12, 15, 18), and the similarity of all candidates was <0.4 (0.12, 0.06, 0.07, 0.11, 0.04, 0.06 respectively), all of which were failed candidates, with a failure rate of 100%.

[0140] The failure rate for the fact "is a scientist" is 100% (first preset percentage), therefore this fact is removed from the fact candidate set. The failure rates for the remaining relations and entities are not 100% and remain unchanged.

[0141] Based on the removed candidate fact set F' = {Proposed the theory of relativity, born in Germany}, the Cartesian product operation is performed again. The new candidate triple set is R × E × F', with a total of 2 × 3 × 2 = 12 candidate triples, that is, all candidates containing "is a scientist" (numbered 3, 6, 9, 12, 15, 18) are removed.

[0142] Semantic consistency was re-validated for the new 12 candidate triples, and their semantic similarity was calculated. The results were the same as before (because the text pairs remained unchanged), as follows: (Proposed by Einstein, who proposed the theory of relativity): 0.98; (Proposed by Einstein, born in Germany): 0.15; (Proposed by Einstein, who proposed the theory of relativity): 0.91; (Proposed by Einstein, who proposed the theory of relativity): 0.08; (Proposed by Einstein, who proposed the theory of relativity): 0.10; (Proposed by Einstein, who proposed the theory of relativity): 0.89; (Birthplace of Einstein, who proposed the theory of relativity): 0.14; (Birthplace of Einstein, who proposed the theory of relativity): 0.97; (Birthplace of Einstein, who proposed the theory of relativity): 0.09; (Birthplace of Einstein, who proposed the theory of relativity): 0.05; (Birthplace of Einstein, who proposed the theory of relativity): 0.13; (Birthplace of Einstein, who proposed the theory of relativity): 0.93.

[0143] After statistical pre-filtering, 6 incorrect candidate triples related to the illusory fact "is a scientist" were successfully eliminated, reducing the total number of candidate triples from 18 to 12, effectively reducing the burden of subsequent large language model validation.

[0144] It should be noted that steps S406 and S407, steps S4071 and S4077, and steps S31 to S35 are for ease of description only and are not shown in the figure.

[0145] The parallel knowledge extraction method based on a large model provided by this invention executes relation, entity, and fact extraction tasks in parallel through three independent extraction modules, breaking the module dependence of traditional serial pipelines and significantly improving knowledge extraction efficiency. By performing a full combination of Cartesian products on the candidate sets of relations, entities, and facts to generate candidate triples covering all potential matches, it effectively avoids knowledge omissions caused by template limitations or stepwise screening, improving recall from the source. Through a multi-level semantic consistency verification mechanism combining initial screening based on semantic similarity and secondary verification using a large language model, it accurately filters out erroneous triples and outputs judgment reasons, greatly reducing the false positive rate and giving the extraction results good interpretability. By statistically analyzing the failure rate of each element to automatically identify and delete noisy elements, combined with an iterative feedback targeted re-extraction module, it further enhances the robustness and adaptability of the system.

[0146] Based on the same inventive concept, embodiments of the present invention also provide a large-model-based parallel knowledge extraction device for implementing the large-model-based parallel knowledge extraction method described above. The solution provided by this device is similar to the implementation scheme described in the above method; therefore, the specific limitations of one or more embodiments of the large-model-based parallel knowledge extraction device provided below can be found in the limitations of the large-model-based parallel knowledge extraction method described above, and will not be repeated here.

[0147] like Figure 2 As shown, this invention provides a parallel knowledge extraction device based on a large model, characterized by comprising: a knowledge extraction model 21, a combination module 22, a semantic verification module 23, and a determination module 24; wherein, the knowledge extraction model 21 is used to receive target text and utilize its three internally embedded, mutually independent extraction modules to execute their respective extraction tasks in parallel and synchronously, outputting a set of candidate relations, a set of candidate entities, and a set of candidate facts, respectively; wherein, the knowledge extraction model includes a basic large model and three mutually independent extraction modules, namely, a relation extraction module, an entity extraction module, and a fact extraction module; the combination module 22, connected to the knowledge extraction model 21, is used to perform Cartesian product operations on the set of candidate relations, the set of candidate entities, and the set of candidate facts to generate a set of candidate triples; the semantic verification module 23, connected to the combination module 22, is used to perform semantic consistency verification on each candidate triple in the set of candidate triples and calculate its semantic similarity; the determination module 24, connected to the semantic verification module 23, is used to compare the semantic similarity with a first preset threshold and determine the target triple based on the comparison result.

[0148] Optionally, the device further includes: a primary screening module (not shown in the figure), connected to the semantic verification module 23 and the determination module 24 respectively, used to mark candidate triples with semantic similarity less than a second preset threshold as failed candidates; calculate the failure rate of each relation, each entity, and each fact in the candidate triples in which it participates, wherein the failure rate of any relation, entity, or fact is equal to the number of candidate triples in which it participates that are marked as failed candidates divided by the total number of all candidate triples in which the relation, entity, or fact participates; if the failure rate of a single relation, entity, or fact reaches a first preset proportion, delete the relation, entity, or fact from the corresponding relation candidate set, entity candidate set, or fact candidate set; perform Cartesian product operation again based on the deleted relation candidate set, entity candidate set, and fact candidate set to generate a new candidate triple set; and re-verify the semantic consistency of each candidate triple in the new candidate triple set and calculate its semantic similarity.

[0149] Optionally, the semantic verification module 23 is specifically used for: for each candidate triplet in the candidate triplet set, concatenating the relation and entity into a first text and the fact as a second text; using a pre-trained dual encoder semantic similarity model to encode the first text and the second text respectively to obtain a first semantic vector and a second semantic vector; calculating the cosine similarity between the first semantic vector and the second semantic vector, and using the cosine similarity as the semantic similarity of the candidate triplet.

[0150] Optionally, the determining module 24 is specifically used to: compare the semantic similarity with a first preset threshold, and take candidate triples with semantic similarity greater than or equal to the first preset threshold as a first candidate set; take candidate triples with semantic similarity less than the first preset threshold but greater than or equal to a second preset threshold as a second candidate set, wherein the second preset threshold is less than the first preset threshold; discard candidate triples with semantic similarity less than the second preset threshold; input each candidate triple in the first and second candidate sets and the target text into a large language model, and have the large language model determine whether the candidate triple is correct; and take the candidate triple determined as correct by the large language model as the target triple.

[0151] Optionally, the determining module 24 is further configured to: obtain candidate triples that are judged as incorrect by the large language model, and count the failure ratio caused by entity errors, relation errors, and fact errors; if the failure ratio caused by entity errors, relation errors, or fact errors is greater than a second preset ratio, trigger targeted re-extraction of the corresponding extraction module, wherein the targeted re-extraction includes: adjusting the temperature parameters of the extraction module and / or adding constraint prompt words.

[0152] Optionally, if the failure rate due to entity errors, relation errors, or fact errors exceeds a second preset rate, a targeted re-extraction of the corresponding extraction module is triggered. This includes: if the corresponding extraction module is an entity extraction module, generating a first constraint prompt and adjusting the temperature parameter of the extraction module to 0.2; if the corresponding extraction module is a relation extraction module, generating a second constraint prompt and adjusting the temperature parameter of the extraction module to 0.3; if the corresponding extraction module is a fact extraction module, generating a third constraint prompt and adjusting the temperature parameter of the extraction module to 0.2; concatenating the corresponding constraint prompt with the target text and re-entering it into the corresponding extraction module for extraction to obtain a new candidate set.

[0153] Optionally, after concatenating the corresponding constraint prompts with the target text and re-inputting them into the corresponding extraction module to obtain a new candidate set, the method further includes: replacing the original candidate set with the new candidate set, and performing the Cartesian product operation again based on the new candidate set to generate an updated candidate triplet set; re-verifying the semantic consistency of each candidate triplet in the updated candidate triplet set and recalculating its semantic similarity; comparing the recalculated semantic similarity with a first preset threshold, and redetermining the target triplet based on the comparison result.

[0154] Optionally, the first preset threshold value ranges from [0.75, 0.85], and the second preset threshold value ranges from [0.2, 0.4].

[0155] Optionally, the basic large model is the Vicuna-7b-v1.5 large language model, and the relation extraction module, entity extraction module and fact extraction module are all low-rank adaptive modules.

[0156] The parallel knowledge extraction device based on a large model provided by this invention executes relation, entity, and fact extraction tasks in parallel through three independent extraction modules, breaking the module dependence of traditional serial pipelines and significantly improving knowledge extraction efficiency. By performing a full combination of Cartesian products on the candidate sets of relations, entities, and facts, candidate triples covering all potential matches are generated, effectively avoiding knowledge omissions caused by template limitations or stepwise screening, thus improving recall from the source. Through a multi-level semantic consistency verification mechanism combining initial screening based on semantic similarity and secondary verification based on a large language model, erroneous triples are accurately filtered and judgment reasons are output, greatly reducing the false positive rate and giving the extraction results good interpretability. By statistically analyzing the failure rate of each element, noisy elements are automatically identified and deleted, combined with an iterative feedback targeted re-extraction module, further enhancing the robustness and adaptability of the system.

[0157] It should be noted that "multiple" in this invention includes two or more.

[0158] It should be understood that although the steps in the flowcharts of the embodiments described above are shown sequentially according to the arrows, these steps are not necessarily executed in the order indicated by the arrows. Unless explicitly stated herein, there is no strict order restriction on the execution of these steps, and they can be executed in other orders. Moreover, at least some steps in the flowcharts of the embodiments described above may include multiple steps or multiple stages. These steps or stages are not necessarily completed at the same time, but can be executed at different times. The execution order of these steps or stages is not necessarily sequential, but can be performed alternately or in turn with other steps or at least some of the steps or stages of other steps.

[0159] Each module in the devices of this invention can be implemented entirely or partially through software, hardware, or a combination thereof. These modules can be embedded in or independent of the processor in a computer device in hardware form, or stored in the memory of a computer device in software form, so that the processor can call and execute the operations corresponding to each module.

[0160] In one embodiment, a computer device is provided, which may be a server, and its internal structure diagram may be as follows: Figure 3 As shown, the computer device includes a processor, memory, and a network interface connected via a system bus. The processor provides computational and control capabilities. The memory includes non-volatile storage media and internal memory. The non-volatile storage media stores the operating system, computer programs, and a database. The internal memory provides an environment for the operation of the operating system and computer programs in the non-volatile storage media. The database stores data required for or generated by the large-model-based parallel knowledge extraction method described above. The network interface communicates with external terminals via a network connection. When the computer program is executed by the processor, it implements a large-model-based parallel knowledge extraction method.

[0161] In one embodiment, a computer device is provided, which may be a terminal, and its internal structure diagram may be as follows: Figure 3As shown, the computer device includes a processor, memory, communication interface, display screen, and input devices connected via a system bus. The processor provides computational and control capabilities. The memory includes non-volatile storage media and internal memory. The non-volatile storage media stores the operating system and computer programs. The internal memory provides an environment for the operation of the operating system and computer programs stored in the non-volatile storage media. The communication interface is used for wired or wireless communication with external terminals; wireless communication can be achieved through Wi-Fi, mobile cellular networks, NFC (Near Field Communication), or other technologies. When executed by the processor, the computer program implements a parallel knowledge extraction method based on a large model. The display screen can be an LCD screen or an e-ink display. The input devices can be a touch layer covering the display screen, buttons, a trackball, or a touchpad on the computer device's casing, or an external keyboard, touchpad, or mouse.

[0162] Those skilled in the art will understand that Figure 3 The structure shown is merely a block diagram of a portion of the structure related to the present invention and does not constitute a limitation on the computer device to which the present invention is applied. A specific computer device may include more or fewer components than those shown in the figure, or combine certain components, or have different component arrangements.

[0163] In one embodiment, a computer device is provided, including a memory and a processor, wherein the memory stores a computer program, and the processor executes the computer program to implement the steps in the above-described method embodiments.

[0164] In one embodiment, a computer-readable storage medium is provided having a computer program stored thereon, which, when executed by a processor, implements the steps in the above method embodiments.

[0165] In one embodiment, a computer program product is provided, including a computer program that, when executed by a processor, implements the steps in the above method embodiments.

[0166] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, stored data, displayed data, etc.) involved in this invention are all information and data authorized by the user or fully authorized by all parties.

[0167] Those skilled in the art will understand that all or part of the processes in the above embodiments can be implemented by a computer program instructing related hardware. The computer program can be stored in a non-volatile computer-readable storage medium, and when executed, it can include the processes of the embodiments of the above methods. Any references to memory, databases, or other media used in the embodiments provided by this invention can include at least one of non-volatile and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetic random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory can include random access memory (RAM) or external cache memory, etc. By way of illustration and not limitation, RAM can take many forms, such as Static Random Access Memory (SRAM) or Dynamic Random Access Memory (DRAM). The databases involved in the embodiments provided by this invention may include at least one type of relational database and non-relational database. Non-relational databases may include, but are not limited to, blockchain-based distributed databases. The processors involved in the embodiments provided by this invention may be general-purpose processors, central processing units, graphics processing units, digital signal processors, programmable logic devices, quantum computing-based data processing logic devices, etc., and are not limited to these.

[0168] The technical features of the above embodiments can be combined in any way. For the sake of brevity, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0169] The embodiments described above are merely examples of several implementations of the present invention, and while the descriptions are relatively specific and detailed, they should not be construed as limiting the scope of the present invention. It should be noted that those skilled in the art can make various modifications and improvements without departing from the concept of the present invention, and these modifications and improvements all fall within the scope of protection of the present invention.

Claims

1. A parallel knowledge extraction method based on a large model, characterized in that, include: The target text is input into the knowledge extraction model, and the three extraction modules in the knowledge extraction model are used to execute their respective extraction tasks in parallel and synchronously, and output the candidate set of relations, the candidate set of entities, and the candidate set of facts respectively. The knowledge extraction model includes a basic large model and three independent extraction modules, namely the relation extraction module, the entity extraction module, and the fact extraction module. Perform a Cartesian product operation on the candidate sets of relations, entities, and facts to generate a set of candidate triples; Perform semantic consistency verification on each candidate triplet in the candidate triplet set and calculate its semantic similarity; The semantic similarity is compared with a first preset threshold, and the target triplet is determined based on the comparison result.

2. The method according to claim 1, characterized in that, After performing semantic consistency verification on each candidate triplet in the candidate triplet set and calculating its semantic similarity, and before comparing the semantic similarity with a first preset threshold and determining the target triplet based on the comparison result, the method further includes: Candidate triples with semantic similarity less than a second preset threshold are marked as failed candidates; Calculate the failure rate of each relation, entity, and fact in the candidate triples in which it participates, where the failure rate of any relation, entity, or fact is equal to the number of candidate triples in which it participates that are marked as failure candidates divided by the total number of all candidate triples in which the relation, entity, or fact participates. If the failure rate of a single relation, entity, or fact reaches a first preset proportion, the relation, entity, or fact will be removed from the corresponding candidate set of relations, entity, or fact. Based on the deleted candidate sets of relations, entities, and facts, perform the Cartesian product operation again to generate a new set of candidate triples. Semantic consistency verification is performed again for each candidate triple in the new set of candidate triples, and their semantic similarity is calculated.

3. The method according to claim 1, characterized in that, The step of performing semantic consistency verification on each candidate triplet in the candidate triplet set and calculating its semantic similarity includes: For each candidate triple in the candidate triple set, concatenate the relation and entity into the first text, and use the fact as the second text; The first text and the second text are encoded using a pre-trained dual encoder semantic similarity model to obtain the first semantic vector and the second semantic vector. Calculate the cosine similarity between the first semantic vector and the second semantic vector, and use the cosine similarity as the semantic similarity of the candidate triple.

4. The method according to any one of claims 1-3, characterized in that, The step of comparing semantic similarity with a first preset threshold and determining the target triplet based on the comparison result includes: The semantic similarity is compared with a first preset threshold, and the candidate triples with semantic similarity greater than or equal to the first preset threshold are taken as the first candidate set; Candidate triples whose semantic similarity is less than the first preset threshold and greater than or equal to the second preset threshold are used as the second candidate set, wherein the second preset threshold is less than the first preset threshold; Candidate triples with semantic similarity less than the second preset threshold are discarded; Each candidate triplet from the first and second candidate sets is input into the target text and the large language model. The large language model then determines whether the candidate triplet is correct. The candidate triples that are judged as correct by the large language model are used as the target triples.

5. The method according to claim 4, characterized in that, Also includes: Obtain candidate triples that are judged as incorrect by the large language model, and calculate the failure rate caused by entity error, relation error, and fact error. If the failure rate caused by entity errors, relationship errors, or factual errors exceeds a second preset rate, a targeted re-extraction of the corresponding extraction module is triggered. The targeted re-extraction includes adjusting the temperature parameters of the extraction module and / or adding constraint prompts.

6. The method according to claim 5, characterized in that, If the failure rate due to entity errors, relationship errors, or factual errors exceeds a second preset rate, a targeted re-extraction of the corresponding extraction module is triggered, including: If the corresponding extraction module is an entity extraction module, generate a first constraint prompt word and adjust the temperature parameter of the extraction module to 0.2; If the corresponding extraction module is a relation extraction module, generate a second constraint prompt word and adjust the temperature parameter of the extraction module to 0.3; If the corresponding extraction module is a fact extraction module, generate a third constraint prompt word and adjust the temperature parameter of the extraction module to 0.2; After concatenating the corresponding constraint prompts with the target text, re-enter the corresponding extraction module to extract the new candidate set.

7. The method according to claim 6, characterized in that, After concatenating the corresponding constraint prompts with the target text and re-entering them into the corresponding extraction module to obtain a new candidate set, the process further includes: Replace the original candidate set with the new candidate set, and perform the Cartesian product operation again based on the new candidate set to generate an updated candidate triplet set; For each candidate triplet in the updated candidate triplet set, semantic consistency verification is performed again, and its semantic similarity is recalculated. The recalculated semantic similarity is compared with the first preset threshold, and the target triplet is re-determined based on the comparison result.

8. The method according to claim 4, characterized in that, The first preset threshold value ranges from [0.75, 0.85], and the second preset threshold value ranges from [0.2, 0.4].

9. The method according to claim 1, characterized in that, The underlying large model is the Vicuna-7b-v1.5 large language model, and the relation extraction module, entity extraction module, and fact extraction module are all low-rank adaptive modules.

10. A parallel knowledge extraction device based on a large model, characterized in that, include: A knowledge extraction model is used to receive target text and use three independent extraction modules embedded within it to execute their respective extraction tasks in parallel and synchronously, outputting candidate sets of relations, entity, and facts respectively. The knowledge extraction model includes a basic large model and three independent extraction modules, namely a relation extraction module, an entity extraction module, and a fact extraction module. The combination module, connected to the knowledge extraction model, is used to perform Cartesian product operations on the candidate sets of relations, entities, and facts to generate a set of candidate triples. The semantic verification module, connected to the combination module, is used to perform semantic consistency verification on each candidate triplet in the candidate triplet set and calculate its semantic similarity. The determination module, connected to the semantic verification module, is used to compare semantic similarity with a first preset threshold and determine the target triplet based on the comparison result.

Images