Ffact exclusion determination method, system and equipment and storage medium

Abstract

The invention discloses a fact exclusion determination method, system and device and a storage medium, and the method comprises the steps: firstly obtaining and analyzing a to-be-verified assertion to extract triple information, then generating an exclusion query set comprising a negative expression, a mutual exclusion relation, a time boundary and a counter-example query, then executing the query to collect exclusion evidence, and finally determining the fact exclusion according to the exclusion evidence. And finally, carrying out intensity scoring on the evidence and outputting an exclusive evidence set with scores. Multi-dimensional generation exclusion query is achieved, the situation that assertion is not established can be comprehensively covered, and verification is more detailed; the evidence is subjected to strength scoring, high-strength evidence can be preferentially referred to in subsequent judgment, and the accuracy and reliability of an assertion verification result are improved.

Description

[0001] Technology Neighborhood

[0002] This invention relates to the fields of artificial intelligence and natural language processing technology, and in particular to a method, system, device and storage medium for determining fact exclusion. Background Technology

[0003] In fields such as knowledge verification and information retrieval, traditional assertion verification methods often rely on single queries or simple evidence matching, lacking a systematic approach to verify assertions from multiple dimensions. For example, when verifying assertions, they often only directly retrieve relevant basic information without comprehensively collecting exclusionary evidence from multiple perspectives such as negation, the existence of mutually exclusive relationships, time conflicts, and counterexamples. This can lead to one-sided verification results, making it impossible to accurately determine the veracity of assertions, resulting in so-called "illusions" of relationships and confirmation bias. Summary of the Invention

[0004] The purpose of this invention is to address the technical problems existing in the prior art by proposing a method, system, device, and storage medium for determining fact exclusion.

[0005] To achieve the above-mentioned technical objectives, the technical solution adopted by the present invention is as follows:

[0006] A first implementation of the first aspect of the present invention provides a method for determining fact exclusion, comprising:

[0007] S101. Obtain and parse the assertion to be verified to extract triple information;

[0008] S102. Based on the triple information, generate the corresponding exclusion query set, which includes negative queries, mutual exclusion queries, time boundary queries, and counterexample queries.

[0009] S103. Based on the exclusion query set, execute the corresponding exclusion query to collect multiple exclusionary pieces of evidence;

[0010] S104. Assess the strength of each piece of exclusionary evidence and output the exclusionary evidence set, which includes multiple pieces of exclusionary evidence with strength scores.

[0011] Optionally, in a second implementation of the first aspect of the present invention, the negation query includes multiple negation types and multiple query templates corresponding to the negation types, wherein each negation type has a preset weight value.

[0012] Optionally, in a third implementation of the first aspect of the present invention, the mutual exclusion relationship query includes multiple relationship types and multiple mutual exclusion relationships corresponding to the relationship types, wherein each interest type has a preset mutual exclusion strength value.

[0013] Optionally, in the fourth implementation of the first aspect of the present invention, the time boundary query includes timing logic conflict verification, state change tracking, and future plan identification.

[0014] Optionally, in the fifth implementation of the first aspect of the present invention, the strength scoring formula is:

[0015] ES = Σ(wi×si×di×ci);

[0016] Where ES is the strength score, wi is the evidence type weight, si is the semantic matching degree, di is the time-decrease factor, and ci is the consistency coefficient.

[0017] A first implementation of the second aspect of the present invention provides a fact exclusion determination system, comprising:

[0018] The parsing module is used to obtain and parse the assertions to be verified in order to extract triple information;

[0019] The query set module is used to generate corresponding exclusion query sets based on triple information. The exclusion query sets include negative queries, mutual exclusion queries, time boundary queries, and counterexample queries.

[0020] The evidence module is used to execute corresponding exclusion queries based on the exclusion query set in order to collect multiple exclusionary pieces of evidence.

[0021] The evaluation module is used to score the strength of each piece of exclusionary evidence and output a set of exclusionary evidence, which includes multiple pieces of exclusionary evidence with strength scores.

[0022] A first implementation of the third aspect of the present invention provides a fact exclusion determination device, the fact exclusion determination device comprising: a memory and at least one processor, the memory storing instructions, the memory and the at least one processor being interconnected via a line;

[0023] The at least one processor invokes the instructions in the memory to cause the fact exclusion determining device to perform the fact exclusion determining method as described in any one of the first aspects of the invention.

[0024] A first implementation of the fourth aspect of the present invention provides a computer-readable storage medium storing a computer program that, when executed by a processor, implements the fact exclusion determination method as described in any one of the first aspects of the present invention.

[0025] Compared with the prior art, the present invention has the following beneficial technical effects:

[0026] This process involves first acquiring and parsing the assertion to be verified to extract triple information, then generating an exclusion query set that includes negations, mutual exclusion relations, time boundaries, and counterexample queries. Next, the queries are executed to collect exclusionary evidence. Finally, the evidence is scored for strength, and a scored set of exclusionary evidence is output. This multi-dimensional generation of exclusion queries comprehensively covers situations where the assertion is false, making verification more detailed. The strength scoring of evidence allows for prioritizing high-strength evidence in subsequent judgments, improving the accuracy and reliability of assertion verification results. Attached Figure Description

[0027] Figure 1 This is a schematic diagram of the first embodiment of the fact exclusion determination method in this invention;

[0028] Figure 2 This is a schematic diagram of one embodiment of the fact exclusion determination system in this invention;

[0029] Figure 3 This is a schematic diagram of one embodiment of the fact exclusion determination device in the present invention;

[0030] Figure 4 This is a schematic diagram of the execution flow of the fact exclusion determination method in an embodiment of the present invention;

[0031] Figure 5 This is a schematic diagram illustrating the execution flow of the fusion mechanism for excluding and confirming evidence in an embodiment of the present invention.

[0032] Figure 6 This is a schematic diagram illustrating the execution flow of the collaborative mechanism in practical application according to an embodiment of the present invention. Detailed Implementation

[0033] It should be noted that, unless otherwise specified, the embodiments and features described in the present invention can be combined with each other.

[0034] For ease of understanding, the specific process of the embodiments of the present invention is described below. Please refer to [link / reference]. Figure 1 The fact exclusion determination method in this embodiment of the invention includes:

[0035] S101. Obtain and parse the assertion to be verified to extract triple information;

[0036] In this embodiment, in scenarios such as knowledge graph construction and semantic understanding, the assertion to be verified is usually a statement expressing a fact or relationship, such as Company A acquiring Company B. After obtaining this assertion, it is parsed using natural language processing technology to extract triples. Triples generally contain a subject, a relation, and an object. In this example, the subject is "Company A," the relation is "acquisition," and the object is "Company B." This step provides the core information unit for subsequent operations such as fact verification and relation deduction. It is a prerequisite for subsequent processes such as evidence retrieval and contradiction detection. Only by clarifying the core entities and relationships in the assertion can the veracity of the assertion be verified in a targeted manner.

[0037] Extracting triple information can be achieved through rule parsing or a combination of rule parsing and machine learning. Rule parsing involves pre-defining a large number of grammatical rules and pattern matching templates. For example, rules can be written to identify the subject, relation, and object in common sentence structures like "[subject][relational verb][object]". When an assertion is input for verification, triples are extracted according to the predefined rules using regular expressions, string matching, and other techniques. A machine learning approach can utilize named entity recognition models to identify the subject and object in the assertion, and then a relation extraction model to determine the relationship between them. Models with architectures such as recurrent neural networks and Transformers can be trained on large amounts of labeled corpora to learn the features of entities and relations, thereby achieving automatic triple extraction.

[0038] S102. Based on the triple information, generate the corresponding exclusion query set, which includes negative queries, mutual exclusion queries, time boundary queries, and counterexample queries.

[0039] In this embodiment, generating an exclusion query set starting from triples is to verify whether the assertion is true. Various targeted queries are constructed to find evidence contradicting the assertion. Negation queries directly negate the triple relationship; for example, if the assertion is that A acquired B, the negation query could be "A did not acquire B." Mutually exclusive relationship queries are generated based on mutually exclusive relationships in the knowledge system. If acquisition and competition are mutually exclusive, queries such as "A and B are competitors" will be generated. Time boundary queries revolve around the time dimension, checking for time conflicts or timeliness issues. Counterexample queries introduce counter-evidence from other entities or official clarifications. These queries mine counter-evidence from different dimensions, providing a comprehensive search direction for subsequent assertion verification.

[0040] Specifically, negative queries include multiple negative types and multiple query templates corresponding to each negative type. Each negative type has a preset weight value, as shown in the table below:

[0041] Negation pattern table:

[0042]

[0043] The mutual exclusion query includes multiple relation types and multiple mutual exclusion relations corresponding to each relation type. Each type of interest has a preset mutual exclusion strength value, as shown in the table below:

[0044] Mutual Exclusion Relationship Knowledge Base Table:

[0045]

[0046]

[0047] Time boundary queries include time-series logic conflict verification, state change tracking, and future plan identification. Time-series logic conflict verification is used to detect the logical consistency of the timeline, such as Company A being established later than the claimed acquisition time. State change tracking is used to track historical changes in the state of a relationship, such as an acquisition that was previously made but has since been withdrawn. Future plan identification is used to distinguish between existing facts and future plans, such as the two states of planned acquisition and completed acquisition.

[0048] Counterexample queries are used to construct counterexample queries, involving other entities, official clarifications, and other counterexample situations. For example, Company C acquired Company B, but Company B officially stated that it was not acquired.

[0049] S103. Based on the exclusion query set, execute the corresponding exclusion query to collect multiple exclusionary pieces of evidence;

[0050] In this embodiment, the exclusion query set includes various types of queries such as negation, mutual exclusion, time boundary, and counterexamples. Executing these queries utilizes information retrieval technology to search for content that contradicts the assertion to be verified in various data sources. For example, if the exclusion query is "Company A did not acquire Company B," then texts, reports, etc., containing this statement or related negative information will be retrieved. For the mutual exclusion query, "Company A and Company B are competitors," then materials that can prove their competitive relationship will be retrieved. By executing these queries, evidence that weakens or negates the assertion to be verified is collected from different dimensions, providing material for subsequent evidence fusion and assertion verification. This is the core data acquisition step for verifying the accuracy of the assertion.

[0051] S104. Assess the strength of each piece of exclusionary evidence and output the exclusionary evidence set, which includes multiple pieces of exclusionary evidence with strength scores.

[0052] In this embodiment, the intensity scoring formula is:

[0053] ES = Σ(wi×si×di×ci);

[0054] Among them, ES is the strength score, wi is the weight of evidence type, si is the semantic matching degree, di is the time decay factor, and ci is the consistency coefficient;

[0055] The determination of the exclusion level includes:

[0056] Strong exclusion (ES > 0.8): It is highly likely that the assertion does not hold, and strong positive evidence is required to overturn it;

[0057] Medium exclusion (0.5 < ES ≤ 0.8): There are major doubts and further verification is needed;

[0058] Weak exclusion (0.2 < ES ≤ 0.5): It is possible that it does not hold, and it is recommended to be treated with caution;

[0059] Unable to exclude (ES ≤ 0.2): Lack of exclusion evidence, which does not affect the judgment.

[0060] In this embodiment, by first obtaining and parsing the assertion to be verified to extract triple information, then generating an exclusion query set including negation, mutual exclusion relationship, time boundary, and counterexample query based on this, then executing the query to collect exclusion evidence, and finally scoring the strength of the evidence and outputting the exclusion evidence set with scores. It realizes multi-dimensional generation of exclusion queries, can comprehensively cover the situation where the assertion does not hold, and makes the verification more meticulous; scoring the strength of the evidence can give priority to high-strength evidence in subsequent judgments, improving the accuracy and reliability of the assertion verification result.

[0061] In this embodiment, as shown in the appendix Figure 4 The fusion mechanism of the exclusion evidence and the confirmation evidence includes:

[0062] Classify the preset original fact set to obtain confirmatory facts, exclusionary facts, and neutral facts;

[0063] Based on the preset strength scoring algorithm, calculate the confirmatory facts and the exclusionary facts to obtain the confirmation score and the exclusion score;

[0064] Based on the preset evidence fusion algorithm, fuse the relevant information of the confirmatory facts and the exclusionary facts, construct an evidence matrix, and perform contradiction analysis to correct the conflicting priority rules;

[0065] Form the final text and output it in a structured form.

[0066] As shown in the appendix Figure 5 The collaborative mechanism of the fact exclusion determination method in practical applications is specifically implemented as:

[0067] Obtain the user's query information, and through strategy selection, there are three strategy situations here: insufficient information, unclear relationship, and both are available;

[0068] If the information is insufficient, a recursive query is executed. During the recursive query, a contradiction detection is performed. If a contradiction exists, the exclusion mechanism in the fact exclusion determination method is called to collect exclusion evidence. If there is no contradiction, the recursive query continues and the final evidence is output.

[0069] If the relationship is unclear, the fact exclusion determination method is initiated. This involves determining if the required information is insufficient. If insufficient, a recursive query is invoked. If sufficient, exclusion evidence is collected.

[0070] If both conditions are met, the execution flow for both scenarios will be executed simultaneously.

[0071] Finally, the collected exclusionary evidence and final evidence are fused together using an evidence fusion algorithm to construct an enhanced context. An answer is generated using a pre-set LLM model and then verified. If the verification passes, the answer is output directly. If the verification fails, the deep proof algorithm is called and the process returns to re-execute the strategy selection.

[0072] The above describes the fact exclusion determination method in the embodiments of the present invention. The following describes the fact exclusion determination system in the embodiments of the present invention. Please refer to [link / reference]. Figure 2 The fact exclusion determination system includes:

[0073] Parsing module 201 is used to obtain and parse the assertions to be verified in order to extract triple information;

[0074] The query set module 202 is used to generate corresponding exclusion query sets based on the triple information. The exclusion query sets include negative queries, mutual exclusion queries, time boundary queries, and counterexample queries.

[0075] Evidence module 203 is used to execute corresponding exclusion queries based on the exclusion query set in order to collect multiple exclusionary evidences;

[0076] The evaluation module 204 is used to score the strength of each piece of exclusionary evidence and output a set of exclusionary evidence, which includes multiple pieces of exclusionary evidence with strength scores.

[0077] Among them, the "negation query" in the query set module 202 includes multiple negation types and multiple query templates corresponding to the negation types, and each negation type has a preset weight value.

[0078] The "mutual exclusion relationship query" in the query set module 202 includes multiple relationship types and multiple mutual exclusion relationships corresponding to the relationship types. Each type of interest has a preset mutual exclusion strength value.

[0079] Among them, the "Time Boundary Query" in the query set module 202 includes temporal logic conflict verification, state change tracking, and future plan identification.

[0080] The formula for "Intensity Score" in assessment module 204 includes:

[0081] ES = Σ(wi×si×di×ci);

[0082] Where ES is the strength score, wi is the evidence type weight, si is the semantic matching degree, di is the time-decrease factor, and ci is the consistency coefficient.

[0083] In this embodiment, the assertion to be verified is first acquired and parsed to extract triple information. Then, an exclusion query set containing negation statements, mutual exclusion relations, time boundaries, and counterexample queries is generated. Next, the queries are executed to collect exclusionary evidence. Finally, the evidence is scored for strength, and a scored exclusionary evidence set is output. This multi-dimensional generation of exclusion queries comprehensively covers situations where the assertion is false, making the verification more detailed. Scorering the evidence for strength allows for prioritizing high-strength evidence in subsequent judgments, improving the accuracy and reliability of the assertion verification results.

[0084] The above is attached Figure 2 The fact exclusion determination method in the embodiments of the present invention will be described in detail from the perspective of unitized functional entities. The fact exclusion determination device in the embodiments of the present invention will be described in detail from the perspective of hardware processing.

[0085] Figure 3 This is a schematic diagram of a fact exclusion determination device 300 provided in an embodiment of the present invention. The fact exclusion determination device 300 can vary significantly due to different configurations or performance. It may include one or more central processing units (CPUs) 310 (e.g., one or more processors) and a memory 320, and one or more storage media 330 (e.g., one or more mass storage devices) storing application programs 333 or data 332. The memory 320 and storage media 330 can be temporary or persistent storage. The program stored in the storage media 330 may include one or more units (not shown in the diagram), each unit may include a series of instruction operations on the fact exclusion determination device 300. Furthermore, the processor 310 may be configured to communicate with the storage media 330 and execute the series of instruction operations in the storage media 330 on the fact exclusion determination device 300.

[0086] Based on factual exclusion, it is determined that device 300 may also include one or more power supplies 340, one or more wired or wireless network interfaces 350, one or more input / output interfaces 360, and / or one or more operating systems 331, such as Windows Server, Mac OS X, Unix, Linux, FreeBSD, etc. Those skilled in the art will understand that... Figure 3 The facts shown exclude the determination that the device structure does not constitute a limitation on communication protocol devices based on LAN projection, which may include more or fewer components than shown, or combine certain components, or have different component arrangements.

[0087] The present invention also provides a computer-readable storage medium, which may be a non-volatile computer-readable storage medium or a volatile computer-readable storage medium, wherein the computer-readable storage medium stores instructions that, when the instructions are executed on a computer, cause the computer to perform the steps of the fact exclusion determination method.

[0088] If the integrated unit is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, in essence, or the part that contributes to the prior art, or all or part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of the present invention. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0089] The above are examples of methods or implementations for determining facts by exclusion, provided in conjunction with specific content. It is not intended that the specific implementation of this invention is limited to these descriptions. Any methods or structures similar to or identical to those of this invention, or any technical deductions or substitutions made based on the concept of this invention, should be considered within the scope of protection of this invention.

Claims

1. A method for determining fact exclusion, characterized in that, include: S101. Obtain and parse the assertion to be verified to extract triple information; S102. Based on the triplet information, generate a corresponding exclusion query set, wherein the exclusion query set includes negative queries, mutual exclusion relationship queries, time boundary queries, and counterexample queries; S103. Based on the exclusion query set, execute the corresponding exclusion query to collect multiple exclusionary pieces of evidence; S104. Calculate the strength score of each of the excluded pieces of evidence and output an excluded evidence set, wherein the excluded evidence set includes multiple excluded pieces of evidence with strength scores.

2. The method for determining fact exclusion according to claim 1, characterized in that, The negation query includes multiple negation types and multiple query templates corresponding to the negation types, wherein each negation type has a preset weight value.

3. The method for determining fact exclusion according to claim 2, characterized in that, The mutual exclusion relationship query includes multiple relationship types and multiple mutual exclusion relationships corresponding to the relationship types, wherein each of the interest types has a preset mutual exclusion strength value.

4. The method for determining fact exclusion according to claim 3, characterized in that, The time boundary query includes time-series logic conflict verification, state change tracking, and future plan identification.

5. The method for determining fact exclusion according to claim 4, characterized in that, The strength scoring formula is as follows: ES = Σ(wi×si×di×ci); Wherein, ES is the strength score, wi is the evidence type weight, si is the semantic matching degree, di is the time-decrease factor, and ci is the consistency coefficient.

6. A fact exclusion determination system, characterized in that, include: The parsing module is used to obtain and parse the assertions to be verified in order to extract triple information; The query set module is used to generate a corresponding exclusion query set based on the triple information, wherein the exclusion query set includes negative queries, mutual exclusion queries, time boundary queries, and counterexample queries; The evidence module is used to execute corresponding exclusion queries based on the exclusion query set in order to collect multiple exclusionary pieces of evidence; An evaluation module is used to score the strength of each piece of exclusionary evidence and output a set of exclusionary evidence, wherein the set of exclusionary evidence includes multiple pieces of exclusionary evidence with strength scores.

7. A fact exclusion determination device, characterized in that, The fact exclusion determination device includes: a memory and at least one processor, wherein the memory stores instructions and the memory and the at least one processor are interconnected via a line; The at least one processor invokes the instructions in the memory to cause the fact exclusion determining device to perform the fact exclusion determining method as described in any one of claims 1-5.

8. A computer-readable storage medium storing a computer program thereon, characterized in that, When the computer program is executed by the processor, it implements the fact exclusion determination method as described in any one of claims 1-5.

Images