Text verification method, system and equipment for language generation model and medium

Abstract

The invention discloses a text verification method, system and device for a language generation model and a medium, and belongs to the technical field of computers.The method comprises the steps that a to-be-verified text is constructed to obtain a to-be-verified triad, and the to-be-verified triad comprises time-space joint fields corresponding to time information and space information; performing multi-dimensional matching on the to-be-verified triad and a standard triad in a preset knowledge base to obtain a weighted matching degree; performing secondary verification on the consistency matching degree and the weighted matching degree in sequence, if verification fails, iteratively generating a plurality of pieces of prompt information according to a preset knowledge base, inputting each piece of prompt information into the language generation model to obtain a corrected triple until the corrected matching degree of the corrected triple and the standard triple is greater than or equal to a preset threshold value, and outputting the corrected triple. When the number of times of iteration is larger than or equal to the preset number of times, iteration is stopped, and the target text is obtained, so that the verification precision of the model output text can be improved.

Description

Technical Field

[0001] This invention relates to the field of computer technology, and in particular to a text verification method, system, device, and medium for language generation models. Background Technology

[0002] As language generation models are increasingly applied in historical research, digitization of ancient books, and cultural dissemination, ensuring the accuracy of their output in terms of historical facts has become crucial. Because the generation mechanism of language generation models is inherently probabilistic, they are highly susceptible to "structural illusions" when dealing with complex historical knowledge, including temporal misalignment, spatial confusion, misattribution of events, and even fabricated document citations. Therefore, developing a technology capable of automating and accurately verifying the output of language generation models has become a key prerequisite for promoting the reliable application of AI in knowledge-sensitive fields.

[0003] Currently, existing technologies rely on knowledge graph matching and verification methods based on simple triples (such as subject-verb-object structures). These methods extract entities and relationships from the text to be verified, forming discrete fact units, and then perform string or semantic matching in a structured knowledge base. This approach can only perform shallow, isolated entity relationship verification and is the mainstream technical path for automated fact verification.

[0004] However, this existing technology processes each attribute independently and in parallel. Therefore, it is significantly inadequate in discerning structural illusions in typical historical events such as "same name in different places" (e.g., "Zhenhai Tower" in different cities) and "time sequence misplacement" (misplacement of Ming Dynasty events in Qing Dynasty), resulting in a sharp decline in verification accuracy. Summary of the Invention

[0005] This invention provides a text verification method, system, device, and medium for language generation models, which can solve the problem of poor identification and correction of spatiotemporally sensitive fact errors in automated fact verification tasks for text generated by language generation models, resulting in low verification accuracy.

[0006] This invention provides a text verification method for language generation models, comprising: Obtain the text to be verified output by the language generation model, and construct a composite triple to be verified based on the text to be verified. The composite triple to be verified includes an event subject field, a documentary evidence field, and a spatiotemporal joint field corresponding to time information and spatial information. The composite triple to be verified is matched with the standard triple in the preset knowledge base in a multidimensional way to obtain a weighted matching degree. The weighted matching degree is obtained by weighting the semantic difference degree corresponding to the event subject field, the spatiotemporal deviation value corresponding to the spatiotemporal joint field, and the consistency matching degree corresponding to the document evidence field. The consistency matching degree and the weighted matching degree are verified in sequence. If the verification fails, multiple prompt messages are generated iteratively according to the preset knowledge base. Each prompt message is input into the language generation model to obtain a corrected triplet. The iteration stops when the corrected matching degree of the corrected triplet and the standard triplet is greater than or equal to the preset threshold, or when the number of iterations is greater than or equal to the preset number, and the target text is obtained.

[0007] This invention, through obtaining the text to be verified from the output of a language generation model, constructs a composite triplet of event subject, documentary evidence, and spatiotemporal connection, transforming free text into structured semantic units and providing a unified data view for subsequent accurate comparison. By performing multi-dimensional matching between the composite triplet to be verified and a preset knowledge base standard triplet, a weighted matching degree is obtained. This is comprehensively evaluated from three dimensions: semantics, spatiotemporal connection, and evidence, avoiding misjudgment from a single dimension and improving verification accuracy. A weighted matching degree ≥ a preset threshold is set to pass verification; otherwise, iterative correction begins, providing a clear pass / correction binary branch logic. This allows the system to quickly release correct text while automatically correcting errors. During iterative correction, prompts are generated using the knowledge base, guiding the model to correct the output text until the matching degree reaches the standard or the maximum number of iterations is reached, completing a self-correction loop. Simultaneously, an upper limit on the number of iterations prevents infinite loops and ensures system termination. Overall, this embodiment ensures consistent verification of the language generation model's output across semantics, spatiotemporal connection, and evidence, significantly reducing the risk of illusion and providing automatic correction capabilities, thus improving the verification accuracy of the language generation model's output text.

[0008] Furthermore, multiple prompt messages are iteratively generated based on the preset knowledge base, specifically: Calculate the weighted similarity between each standard triplet in the preset knowledge base and the composite triplet to be verified. Sort the weighted similarities in descending order and select a preset number of standard triplets in sequence to obtain multiple candidate triplets. The weighted similarity is obtained by weighting the semantic similarity corresponding to the event subject field and the spatiotemporal similarity corresponding to the spatiotemporal joint field. For each candidate triple, the candidate triple and the corresponding original literature evidence field are used as the prompt information, and the prompt information is sequentially input into the language generation model to guide the language generation model to regenerate the corrected triple.

[0009] By compressing the semantic and spatiotemporal dimensions into a single quantitative indicator, the candidate set is highly relevant to the original events and possesses temporal and spatial consistency, avoiding the illusion of identical names in different locations or temporal misalignments caused by simple keyword matching. Using high-credibility original texts directly prompts the model, enabling evidence-driven rewriting, reducing the model's room for improvisation, and enhancing the authority and traceability of the revised facts. Overall, this embodiment proposes an iterative correction process of weighted ranking → evidence-based original text prompts → line-by-line rewriting. This allows the large model to maintain linguistic fluency while automatically absorbing authoritative facts from the knowledge base, significantly reducing the structural illusion of the generated text and improving its accuracy.

[0010] Furthermore, the spatiotemporal deviation value is obtained by weighted fusion of the temporal distance metric and the spatial distance metric, specifically as follows: The distance between the time information to be verified in the spatiotemporal joint field and the standard time information in the standard triple is calculated to obtain the time distance metric. The spatial distance metric is also calculated based on the distance between the spatial information to be verified in the spatiotemporal joint field and the standard spatial information in the standard triple. The time distance metric and the spatial distance metric are weighted and fused according to a preset weighting coefficient to obtain the spatiotemporal deviation value.

[0011] By calculating temporal and spatial distance metrics separately, "when" and "where" are separated into two independently calculable quantities, providing a flexible weighting basis for different business scenarios (historical events vs. geographical events). A pre-set weighted coefficient is used to weight and fuse these two metrics to obtain a spatiotemporal deviation value. The importance of time / space is adjusted according to business characteristics; for example, historical events focus more on time, while logistics events focus more on space. Overall, this embodiment, through its "separable and combinable" spatiotemporal distance design, allows the verification system to tolerate both reasonable time errors and geographical radius errors, avoiding overly stringent requirements that could lead to misjudgments of correct text and improving verification recall.

[0012] Furthermore, the time distance metric is obtained by calculating the absolute difference between the time information to be verified and the standard time information, and normalizing the absolute difference according to a preset time tolerance parameter; the spatial distance metric is obtained by calculating the geographical distance between the spatial information to be verified and the standard spatial information, and normalizing the geographical distance according to a preset spatial tolerance parameter.

[0013] This approach calculates the absolute difference between the time information to be verified and the standard time information to obtain an intuitive measure of time deviation. The absolute difference is then normalized using a preset time tolerance parameter, mapping the original time difference to the [0,1] interval to eliminate the influence of dimensions. The actual physical distance is obtained by calculating the geographical distance (e.g., spherical distance) between the time information to be verified and the standard spatial information. This distance is then normalized using a preset spatial tolerance parameter, mapping kilometers to [0,1] to eliminate the influence of dimensions. Overall, this embodiment provides configurable tolerance for time and distance differences, improving the universality and robustness of the verification process.

[0014] Furthermore, the semantic difference is obtained by performing semantic vector embedding and cosine similarity calculation on the event subject field and the standard event field using a preset semantic matching model.

[0015] This approach uses a semantic matching model to embed vectors into the event subject fields, upgrading words / phrases into high-dimensional semantic vectors to capture synonyms, near-synonyms, and hyponyms / hypernyms. Cosine similarity is calculated as the semantic dissimilarity score; since the range of cosine values ​​is a normalized result, the calculation is fast and the comparability is good, making it suitable for real-time comparison with large-scale knowledge bases. Overall, this embodiment uses mature semantic vector technology to solve the problem of "multiple expressions for the same event," significantly improving the recall and accuracy of subject matching.

[0016] Furthermore, the text verification method for the language generation model further includes: When the document evidence field does not meet the preset evidence conditions with the standard evidence field, the preset level weight corresponding to the first evidence level is determined according to the preset evidence level mapping table, and the consistency matching degree is determined according to the preset level weight. The preset evidence conditions refer to the document evidence field being the same as the standard evidence field, or the first evidence level corresponding to the document evidence field being greater than or equal to the second evidence level corresponding to the standard evidence field.

[0017] In this way, when the document evidence field does not meet the preset evidence conditions with the standard evidence field, the level weight is obtained by consulting the preset evidence classification mapping table. The first values ​​corresponding to the same values ​​are then weighted to obtain the final consistency matching degree. This ensures that evidence from high-level documents (such as SCI, national standards) receives a higher score, while low-level documents (such as self-media) receive a lower score, preventing pseudo-documents from artificially inflating the matching degree. Overall, this embodiment further distinguishes the credibility of evidence based on evidence consistency, thereby improving the factual accuracy of the verification text.

[0018] Further, the step of sequentially performing secondary verification on the consistency matching degree and the weighted matching degree includes: If the document evidence field meets the preset evidence conditions with the standard evidence field in the standard triplet, and the weighted matching degree is greater than or equal to the preset threshold, then the verification is successful. If the document evidence field does not meet the preset evidence conditions with the standard evidence field, the verification fails. If the document evidence field and the standard evidence field meet the preset evidence conditions, but the weighted matching degree is less than the preset threshold, then the verification fails.

[0019] This approach sets a hard threshold for the authority of evidence, preventing highly similar but low-authority texts from passing verification and avoiding citations from pseudo-documents such as those from self-media. Verification is only considered successful if the evidence conditions and weighted matching degree are both met and the threshold is reached. Verification logic that fails if any discrepancy occurs ensures that the output facts are both correct and accurate, effectively suppressing false citations and advanced illusions.

[0020] Another embodiment of the present invention provides a text verification system for a language generation model, comprising: a construction module, a matching module, and a verification module; The construction module is used to obtain the text to be verified output by the language generation model, and construct a composite triple to be verified based on the text to be verified. The composite triple to be verified includes an event subject field, a documentary evidence field, and a spatiotemporal joint field corresponding to time information and spatial information. The matching module is used to perform multidimensional matching between the composite triple to be verified and the standard triple in the preset knowledge base to obtain a weighted matching degree. The weighted matching degree is obtained by weighting the semantic difference degree corresponding to the event subject field, the spatiotemporal deviation value corresponding to the spatiotemporal joint field, and the consistency matching degree corresponding to the document evidence field. The verification module is used to perform secondary verification on the consistency matching degree and the weighted matching degree in sequence. If the verification fails, multiple prompt messages are generated iteratively according to the preset knowledge base. Each prompt message is input into the language generation model to obtain a corrected triplet. The iteration stops when the corrected matching degree of the corrected triplet and the standard triplet is greater than or equal to the preset threshold, or when the number of iterations is greater than or equal to the preset number, and the target text is obtained.

[0021] Another embodiment of the present invention provides a terminal device, including: a processor, a memory, and a computer program stored in the memory and configured to be executed by the processor, wherein when the processor executes the computer program, it implements the steps of the text verification method for language generation models of the present invention.

[0022] Another embodiment of the present invention provides a computer-readable storage medium item, including: a stored computer program, which, when the computer program is running, controls the device where the computer-readable storage medium is located to perform steps such as the text verification method for language generation models of the present invention. Attached Figure Description

[0023] To more clearly illustrate the technical solution of this application, the drawings used in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained from these drawings without creative effort.

[0024] Figure 1 This is a flowchart illustrating a text verification method for a language generation model provided in an embodiment of the present invention; Figure 2 This is a schematic diagram of a composite ternary structure provided in an embodiment of the present invention; Figure 3 This is a schematic diagram of a text verification process provided in an embodiment of the present invention; Figure 4 This is a schematic diagram of a correction process provided in an embodiment of the present invention; Figure 5 This is a schematic diagram of the structure of a text verification system for a language generation model provided in an embodiment of the present invention. Detailed Implementation

[0025] To make the objectives, technical solutions, and advantages of this application clearer, the technical solutions of this application will be clearly and completely described below with reference to the accompanying drawings of the embodiments. Obviously, the described embodiments are only some embodiments of this application, not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0026] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this application pertains; the terminology used herein is for the purpose of describing particular embodiments only and is not intended to limit the application; the terms “comprising” and “having”, and any variations thereof, in the specification, claims, and foregoing description of the drawings are intended to cover non-exclusive inclusion.

[0027] In the description of the embodiments of this application, technical terms such as "first" and "second" are used only to distinguish different objects and should not be construed as indicating or implying relative importance or implicitly specifying the number, specific order, or primary and secondary relationship of the indicated technical features. In the description of the embodiments of this application, "multiple" means two or more, unless otherwise explicitly defined.

[0028] In this document, the term "embodiment" means that a particular feature, structure, or characteristic described in connection with an embodiment may be included in at least one embodiment of this application. The appearance of this phrase in various places throughout the specification does not necessarily refer to the same embodiment, nor is it a separate or alternative embodiment mutually exclusive with other embodiments. It will be explicitly and implicitly understood by those skilled in the art that the embodiments described herein can be combined with other embodiments.

[0029] In the description of the embodiments in this application, the term "and / or" is merely a description of the relationship between related objects, indicating that three relationships can exist. For example, A and / or B can represent: A existing alone, A and B existing simultaneously, and B existing alone. Additionally, the character " / " in this document generally indicates that the preceding and following related objects have an "or" relationship.

[0030] In the description of the embodiments of this application, the term "multiple" refers to two or more (including two), similarly, "multiple sets" refers to two or more (including two sets), and "multiple pieces" refers to two or more (including two pieces).

[0031] In the description of the embodiments of this application, unless otherwise expressly specified and limited, technical terms such as "installation," "connection," "joining," and "fixing" should be interpreted broadly. For example, they can refer to a fixed connection, a detachable connection, or an integral part; they can refer to a mechanical connection or an electrical connection; they can refer to a direct connection or an indirect connection through an intermediate medium; they can refer to the internal communication of two components or the interaction between two components. For those skilled in the art, the specific meaning of the above terms in the embodiments of this application can be understood according to the specific circumstances.

[0032] See Figure 1 To address the problem of poor identification and correction of spatiotemporally sensitive fact errors in existing automated fact verification tasks for text generated by language generation models, resulting in low verification accuracy, an embodiment of the present invention provides a text verification method for language generation models, comprising: Step S101: Obtain the text to be verified output by the language generation model, and construct a composite triple to be verified based on the text to be verified. The composite triple to be verified includes an event subject field, a documentary evidence field, and a spatiotemporal joint field corresponding to time information and spatial information.

[0033] In this embodiment, natural language text output from a language generation model is received as input. Automated information extraction is performed on this text to construct a structured composite triple to be verified. This triple serves as the core data carrier for all subsequent verification operations. Specifically, as shown... Figure 2 The diagram shows a composite triplet structure. Natural language processing techniques are used for entity recognition (NER) and relation extraction to extract the event subject from the text to be verified. (i.e., core historical activities or entity behaviors), time information Spatial information (Geographical location) and documentary evidence (Source cited). The key point is that the system jointly models the extracted temporal and spatial information into a single "spatiotemporal joint field." This data structure design ensures that time and space are considered as a holistic and inseparable context in subsequent processing, rather than as two independent attributes. This yields the composite triple to be verified. Its basic expression is: ; in, This is the event body field, i.e., the event semantics, which supports natural language embedding. This is for standardizing the time (year / month / year number to Gregorian calendar) in the spatiotemporal joint field. These are the coordinate values ​​of the place names resolved from the spatiotemporal joint field; The document evidence field includes cited paragraphs, bibliography, and evidence level. Although the above composite triples contain four dimensions in their expression, time and space are jointly modeled as a spatiotemporal nested field, maintaining structural unity and algorithmic compatibility, and possessing advantages over traditional triples.

[0034] Step S102: Perform multidimensional matching between the composite triple to be verified and the standard triple in the preset knowledge base to obtain a weighted matching degree. The weighted matching degree is obtained by weighting the semantic difference degree corresponding to the event subject field, the spatiotemporal deviation value corresponding to the spatiotemporal joint field, and the consistency matching degree corresponding to the document evidence field.

[0035] In this embodiment, as Figure 3The diagram illustrates a text verification process according to an embodiment of the present invention. The construction of standard triples is one of the fundamental steps of this invention, ensuring that the generated historical events can be accurately matched with facts in the knowledge base. These standard triples are standardized through manual annotation, document comparison, and spatiotemporal anchor calibration. Each fact unit obtains a unique event-time-space-evidence combination, which not only possesses historical uniqueness but also allows for verification of its authority through document tracing. The construction of standard triples requires standardization of historical time (e.g., mapping year names to the Gregorian calendar), precise positioning of spatial coordinates (e.g., mapping place names using GIS technology), and grading of preset documentary evidence (official histories, local chronicles, archaeological reports, etc.) according to a preset evidence grading mapping table. (Standard triples in the knowledge base) The formula is as follows: ; in, For the standard event main field, T is the standard Gregorian calendar year / interval, which can be obtained by mapping from the year; X is the GIS coordinate or the standard coordinate value after parsing the resolvable place name; D is the standard documentary evidence field, including the cited paragraphs, bibliography and evidence level.

[0036] The composite triples to be verified constructed in step 101 are compared with standard triples in a pre-defined knowledge base using a multi-dimensional weighted matching function to calculate their matching degree. This function is used to determine the overall consistency between the triples to be verified corresponding to the model's output text and a certain standard triple in four dimensions: event semantics, chronological time, geographical location, and documentary evidence. It is the final criterion for determining whether the triples to be verified exhibit structural illusion. The function form is as follows: ; Where, d e f represents the semantic difference of the event. t For the time difference normalization result, f s For the spatial distance normalization result, δ d w is an indicator of consistency between documentary evidence and written evidence. e、 w t、 w s and w d These are the weight parameters for the corresponding dimensions, with default values ​​of 0.25, 0.15, 0.15, and 0.15 respectively, and w e +w t +w s +w d=1, and the semantic difference is used to evaluate the semantic similarity between the event subject field of the triple to be verified and the corresponding field of the standard triple. The spatiotemporal deviation value is used as a whole to evaluate the comprehensive temporal and spatial differences between the spatiotemporal joint field of the triple to be verified and the corresponding field of the standard triple. The consistency matching degree is used to evaluate whether the literature evidence field of the triple to be verified is consistent with the corresponding field of the standard triple.

[0037] For example, document consistency δ d For binary variables: when the cited references are consistent with or at least equal to the standard references in the knowledge base, δ d = 0; otherwise δ d = 1. The document ranking score L∈{0.9, 0.7, 0.6, 0.3} is used to adjust the ranking during the initial screening stage, prioritizing higher-ranked documents in the Top-K. The final match uses δ. d Instead of L, it ensures hard control over permission levels.

[0038] Step S103: Perform secondary verification on the consistency matching degree and the weighted matching degree in sequence. If the verification fails, generate multiple prompt messages according to the preset knowledge base. Input each prompt message into the language generation model to obtain a corrected triplet. Stop the iteration when the corrected matching degree of the corrected triplet and the standard triplet is greater than or equal to the preset threshold, or when the number of iterations is greater than or equal to the preset number of iterations, and obtain the target text.

[0039] In this embodiment, the calculated weighted matching degree `match` is compared with a preset threshold `θ`. m (e.g., 0.85) are compared, and if match ≥ θ m If the text to be verified passes factual verification, it indicates that the output of the language generation model maintains a high degree of consistency with the authoritative knowledge base across multiple dimensions, including events, spatiotemporal context, and documentary evidence, and the process ends. Wherein, θ m The preset threshold is a technical parameter, such as 0.85, that is determined through testing based on core objectives, weight configuration, and domain knowledge, and is used to achieve the best balance between accuracy and usability.

[0040] If match < θ mIf verification fails, this method doesn't simply report an error; instead, it proactively initiates a correction process. Specifically, when verification fails, the failed composite triplet to be verified can be marked as an illusion triplet, and iterative correction is performed on the composite triplet to be verified. In a single iteration, a preset number of standard triplets are selected from a preset knowledge base to generate a prompt message. This prompt message is then input into the language generation model, guiding the model to correct the output content, resulting in corrected text and the corresponding corrected triplet. After this, the process doesn't end; instead, it returns to step 102 to re-match and judge this corrected triplet. This "generation-update-re-verification" loop continues until one of the following three termination conditions is met, and the corrected text of the current round is used as the target text: 1) Forced termination mechanism: Set the maximum number of iterations L_max (default 3). If the maximum number of iterations is exceeded, the process will terminate, regardless of whether the result is satisfactory.

[0041] 2) State change mechanism: Each iteration uses a new round of Top-K candidates to construct a new evidence enhancement Prompt. Therefore, the event semantics, time anchors, spatial coordinates and document fragments all change, making it impossible for the system to repeat execution in the same state.

[0042] 3) Early termination mechanism: If δ is satisfied in any iteration d =0 and match≥θ m If the error occurs, the process will terminate immediately and output the corrected result.

[0043] Through the above three mechanisms, the present invention theoretically eliminates the risk of infinite loops and ensures that the system can complete the error correction task within a finite time.

[0044] It should be noted that traditional verification methods based on knowledge graph matching or RAG can only detect contradictions, but cannot provide automatic error correction capabilities after errors occur. The embodiments of this invention achieve an integrated function of error detection and correction through a closed-loop structure of "multi-dimensional matching → Top-K recall → evidence hint generation → re-verification". This allows the model output to gradually converge to standard facts under evidence-driven constraints. In fact verification scenarios involving historical and ancient texts and other time- and space-sensitive texts, it can accurately suppress structural illusions, enhance the interpretability and credibility of the model output, and improve factual accuracy.

[0045] As an example of an embodiment of the present invention, the semantic difference is obtained by semantic vector embedding and cosine similarity calculation of the event subject field and the standard event field through a semantic matching model.

[0046] In this embodiment, a sentence-level pre-trained semantic matching model (such as SBERT or SimCSE) is used to match the event subject field (e) in the composite triple to be verified.i The standard event field (E) of the standard triple in the knowledge base is converted into a numerical vector (i.e., an embedding vector) in a high-dimensional space. For example, if the event text (such as "Rebuild Zhenhai Tower" and "Reconstruct Zhenhai Tower") is input into the model, the model outputs two corresponding fixed-dimensional floating-point vectors Vec(e). i The semantic vectors Vec(e) and Vec(E) are calculated using the cosine similarity formula. i The cosine similarity between Vec(E) and Vec(E) is used to obtain the final semantic dissimilarity. .

[0047] As an example of an embodiment of the present invention, the spatiotemporal deviation value is obtained by weighted fusion of a time distance metric and a spatial distance metric. Specifically, the distance between the time information to be verified in the spatiotemporal joint field and the standard time information in the standard triple is calculated to obtain the time distance metric, and the distance between the spatial information to be verified in the spatiotemporal joint field and the standard spatial information in the standard triple is calculated to obtain the spatial distance metric. The time distance metric and the spatial distance metric are weighted and fused according to a preset weighting coefficient to obtain the spatiotemporal deviation value.

[0048] As an example of an embodiment of the present invention, the time distance metric is obtained by calculating the absolute difference between the time information to be verified and the standard time information, and normalizing the absolute difference according to a preset time tolerance parameter; the spatial distance metric is obtained by calculating the geographical distance between the spatial information to be verified and the standard spatial information, and normalizing the geographical distance according to a preset spatial tolerance parameter.

[0049] In this embodiment, although time and space are logically bound as a spatiotemporal joint field, when calculating their distance, they need to be decomposed into independent time and space information. Specifically, regarding the time distance metric, the scalar difference between two time points or time periods is calculated, which is usually achieved by calculating the absolute difference (e.g., the time to be verified t). i The difference from the standard time T is |t i- T|year); Spatial distance metric, which calculates the geographical length between two geographic locations, requires a Geographic Information System (GIS) algorithm (such as the great circle distance algorithm) to calculate the shortest path length between two points on the Earth's surface. Since the original time difference unit is year (or month, day, etc.), and the original spatial distance unit is kilometer (or mile, etc.), these two values ​​are completely different in dimensions and magnitude, and cannot be directly weighted and summed. Therefore, a tolerance parameter can be introduced to normalize them. After normalization, both the time distance metric and the spatial distance metric become dimensionless scalars with values ​​ranging from [0,1], making the data from two completely different dimensions comparable and fusionable. Then, based on preset weighting coefficients, time and space are fused into a single spatiotemporal deviation value, which then participates in higher-level weighted matching degree calculations.

[0050] As an example of an embodiment of the present invention, the text verification method for a language generation model further includes: when the document evidence field and the standard evidence field do not meet preset evidence conditions, determining a preset level weight corresponding to the first evidence level according to a preset evidence level mapping table, and determining the consistency matching degree according to the preset level weight, wherein the preset evidence conditions refer to the document evidence field being the same as the standard evidence field, or the first evidence level corresponding to the document evidence field being greater than or equal to the second evidence level corresponding to the standard evidence field.

[0051] In this embodiment, when the cited literature in the literature evidence field of the language generation model is authentic and consistent with the standard, the consistency matching degree δ is [value missing]. d =0; When the cited literature in the literature evidence field is pseudo-literacy, of too low a level, or without citations, the initial consistency matching degree δ is 0. d =1, and by querying the preset evidence classification mapping table (as shown in Table 1 below), the corresponding preset level weights (e.g., 0.9, 0.7, 0.6, 0.3) are determined based on the evidence level (e.g., I, II, III, IV) in the document evidence field. The initial consistency matching score is then weighted using these preset level weights. For example, if the first value is 1, and the cited evidence is Level I (weight 0.9), then the final consistency matching score is... = 1 * 0.9 = 0.9.

[0052] Table 1 Evidence Classification Mapping Table As an example of an embodiment of the present invention, the step of performing secondary verification on the consistency matching degree and the weighted matching degree sequentially includes: if the document evidence field and the standard evidence field in the standard triplet meet preset evidence conditions, and the weighted matching degree is greater than or equal to a preset threshold, then the verification passes; if the document evidence field and the standard evidence field do not meet the preset evidence conditions, then the verification fails; if the document evidence field and the standard evidence field meet the preset evidence conditions, but the weighted matching degree is less than a preset threshold, then the verification fails.

[0053] In this embodiment, firstly, based on the composite triplet T to be verified i Document evidence field d i Extract the standard evidence field D corresponding to the standard ternary T* from the preset knowledge base, and determine whether it meets the preset evidence conditions. If it does, then δ d If the value is 0, the first-level verification passes and proceeds to the second-level verification; otherwise, the verification fails, and the original output is deemed to lack verifiable evidence of α+β+γ=1. If the first-level verification passes, the weighted matching degree `match` is calculated. If the `match` value is less than or equal to a preset threshold, it is considered a structural illusion triple, and the verification fails, requiring a correction process. If the `match` value is greater than the preset threshold, the verification passes, and the process terminates.

[0054] As an example of an embodiment of the present invention, the step of iteratively generating multiple prompt messages based on the preset knowledge base specifically involves: calculating the weighted similarity between each standard triplet in the preset knowledge base and the composite triplet to be verified; sorting the weighted similarities in descending order; and sequentially selecting a preset number of standard triplets to obtain multiple candidate triplets. The weighted similarity is obtained by weighting the semantic similarity corresponding to the event subject field and the spatiotemporal similarity corresponding to the spatiotemporal joint field. For each candidate triplet, the candidate triplet and the corresponding original document evidence field are used as the prompt message, and the prompt message is sequentially input into the language generation model to guide the language generation model to regenerate the corrected triplet.

[0055] In this embodiment, as Figure 4 The diagram illustrates a correction process according to an embodiment of the present invention. For each standard triplet in the preset knowledge base, the weighted similarity between each standard triplet and the composite triplet to be verified is calculated. The calculation of the weighted similarity reuses the idea of ​​multidimensional matching, but only evaluates from two dimensions: event (semantic similarity) and spatiotemporal similarity, and performs weighted fusion. The formula for calculating the weighted similarity is as follows: ; Where α, β, and γ are preset weighting coefficients, with default values ​​of 0.6, 0.2, and 0.2 respectively, satisfying α + β + γ = 1, s e s t and s s These represent the semantic similarity, temporal proximity, and spatial proximity of the events, respectively.

[0056] All standard triples are sorted according to weighted similarity. A preset number (Top-K, e.g., 5) of candidate triples are recalled in order, and their corresponding original document evidence fields are extracted. The original document evidence fields and candidate triples, namely original document paragraphs, bibliographies, coordinates, and dates, are used as prompts to input the language generation model to guide the language generation model to perform restricted error correction generation, thereby realizing evidence-driven structured regeneration.

[0057] In this invention, after candidate recall, only the Top-K items (between three and ten, with K=5 by default) are selected from the M candidates as the basis for evidence enhancement. This parameter is determined based on the following principles: K that is too small (e.g., 1–2) may miss key evidence, which is not conducive to fact convergence; K that is too large (e.g., >10) will lead to a lengthy Prompt, increased information noise, and a significant increase in generation cost; experiments show that K≈5 is the optimal balance between "improved error correction success rate" and "controllable generation burden". Therefore, the Top-K design adopted in this embodiment avoids wasting computational resources while maintaining sufficient fact coverage, and can effectively guide the large language model to correct erroneous facts.

[0058] It should be noted that this invention explicitly distinguishes between the weighted similarity score `pre_score` used for coarse ranking and the weighted matching score `match` used for final review. Specifically, the weighted similarity score `pre_score` is a coarse ranking, relying only on three dimensions, including the event semantic similarity score `s`. e Time proximity s t Spatial proximity s s This is used for quickly ranking dozens to hundreds of candidate triples. It does not participate in fact-checking; it is only used for recall ranking and does not overlap with `match`. Compared with the weighted similarity score `pre_score`, the weighted matching score `match` also includes literature consistency δ as a calculation parameter. d Dimensions, belonging to the four-dimensional comprehensive judgment, are the sole basis for the final factual verification. Therefore, the weighted similarity pre_score and match are completely different in dimensional composition, purpose, and position of action, and do not constitute duplicate calculation or functional overlap.

[0059] To further explain, this invention significantly reduces computational burden by decoupling candidate selection from final judgment. Specifically, the ranking of weighted similarity pre_score involves only simple weighting operations, and the Top-K stage reduces the number of candidates from N to K (typically 3-5), avoiding the model's exposure to a large fact space and further reducing generation costs. The final verification of weighted matching only requires calculating a single triple, with extremely low complexity, which can be easily handled by the server. The overall computational cost is far lower than the traditional method of matching each item in the entire knowledge base, making it suitable for knowledge bases of millions in scale.

[0060] like Figure 3 As shown, this invention proposes an empirical case, assuming that the LLM model outputs the following: "In the 26th year of the Kangxi Emperor's reign (1687), during the Rebellion of the Three Feudatories, the building was damaged by war. The Kangxi Emperor ordered its reconstruction to maintain its original appearance and restore its lookout and symbolic functions, supported by Zhang Yue's 'Record of Zhenhai Tower'." The text verification process of this method is executed, and the specific steps are as follows: S1. Extract the triplet structure as shown in Table 2 from the original text output by the LLM model to obtain the triplet to be verified: Table 2. Triples extracted by LLM S2. Match the set of candidate triples and perform a first-level decision: The standard truth values ​​that are similar to the semantics, time, location and evidence are retrieved from the preset standard database, and multiple candidate triples are obtained. For example, in 1685, the 24th year of Kangxi's reign, the tower was rebuilt and restored to its original function. The location was Yuexiu Mountain in Guangzhou. The documents are Li Shizhen's "Record of the Reconstruction of Zhenhai Tower" and Qu Dajun's "New Tales of Guangdong". Determine whether the literature evidence field of the triple to be verified and the standard evidence field of the matching triple meet the preset evidence conditions. If they do, then δ d =0, proceed to the secondary judgment in step S3; if it does not meet the requirement, then δ d =1, directly classifying it as a structural hallucination, marking it as an anomaly, and then applying it to δ according to a pre-defined evidence grading mapping table. d =1 is processed, and then proceeds to the secondary judgment in step S3; S3. Consistency calculation, perform level two judgment: Calculate the weighted matching degree using the consistency matching function: ; Semantic differences (d) e The generated event contains "Kangxi issued an order", which is not included in the standard event → Cosine similarity 0.75, d e =0.25; Time deviation (f t ): |1687-1685| / 30 = 0.0667; Spatial consistency (fs ):fs=0; Pseudo-citations of literature (δ) d ):δ d =1 (Zhang Yue's inscription has an incorrect date). Substituting the default weights: match = 1 − (0.25 × 0.25 + 0.15 × 0.0667 + 0.15 × 0 + 0.45 × 1) = 0.4775; Because comparing the weighted matching score to a preset threshold, match=0.4775 is lower than the threshold θ. m =0.85, and δ d =1, so it is determined to be a structural hallucination; S4. Top-K candidate recall mechanism activated: The Top-K candidate recall mechanism does not rely on the original output of forged documents. Instead, it uses the following triple structure vector calculation and existing truth set to perform vector-based multidimensional retrieval. The structure of the resulting candidate triples is shown in Table 3. Among them, semantic embedding vector (event E): using sentence vector or keyword aggregation, it matches descriptions such as "reconstructing Zhenhai Tower" and "Shang Kexi's expansion"; timestamp matching (time T): if the input is "Kangxi 26th year", candidates such as "reconstructed in Kangxi 24th year" will be recalled due to their close time; spatial coordinates or place name similarity (location X): such as "Guangzhou Yuexiu Mountain" and "the highest point of the North City Wall", those with spatial overlap will be prioritized for matching; Table 3. Candidate events recalled from the truth set by Top-K S5. Generate prompt text after enhancing evidence: Extract the evidence fragment with the highest weighted similarity C1, and use it as a contextual cue to input into the large language model to generate the cue text: "In the 24th year of Kangxi (1685), Zhenhai Tower was destroyed after the Rebellion of the Three Feudatories. Li Shizhen, the governor of Guangdong, presided over the reconstruction of Zhenhai Tower to restore its functions as a lookout and a symbol of the city, as evidenced by 'Record of the Reconstruction of Zhenhai Tower' and 'New Guangdong Sayings'". Input this cue text into the original language generation model to obtain the updated output text. S6. Re-verification and Loop Closure Termination: The system re-extracts new triples from the updated output text and performs a match check: match=1−(0.25×0.05+0.15×0+0.15×0+0.45×0)=1−0.0125=0.9875; The matching value match=0.9875 is higher than the set threshold θ m =0.85; Result: The new triplet passes the S6 verification step, the closed loop terminates, and it will no longer enter the candidate recall or regeneration stage.

[0061] like Figure 5 As shown, based on the above method embodiments, an embodiment of the present invention provides a text verification system 500 for a language generation model, including: a construction module 501, a matching module 502 and a verification module 503; The construction module 501 is used to obtain the text to be verified output by the language generation model, and construct a composite triple to be verified based on the text to be verified. The composite triple to be verified includes an event subject field, a documentary evidence field, and a spatiotemporal joint field corresponding to time information and spatial information. The matching module 502 is used to perform multidimensional matching between the composite triple to be verified and the standard triple in the preset knowledge base to obtain a weighted matching degree. The weighted matching degree is obtained by weighting the semantic difference degree corresponding to the event subject field, the spatiotemporal deviation value corresponding to the spatiotemporal joint field, and the consistency matching degree corresponding to the document evidence field. The verification module 503 is used to perform secondary verification on the consistency matching degree and the weighted matching degree in sequence. If the verification fails, multiple prompt messages are generated iteratively according to the preset knowledge base. Each prompt message is input into the language generation model to obtain a corrected triplet. The iteration stops when the corrected matching degree of the corrected triplet and the standard triplet is greater than or equal to the preset threshold, or when the number of iterations is greater than or equal to the preset number, and the target text is obtained.

[0062] It is understood that the above system item embodiments correspond to the method item embodiments of the present invention, and can implement the text verification method for language generation model provided by any of the above method item embodiments of the present invention.

[0063] It should be noted that the system embodiments described above are merely illustrative, and some or all of the modules can be selected to achieve the purpose of this embodiment according to actual needs. Furthermore, in the accompanying drawings of the system embodiments provided by this invention, the connection relationships between modules indicate that they have communication connections, which can be specifically implemented as one or more communication buses or signal lines. Those skilled in the art can understand and implement this without any creative effort.

[0064] For ease of description and brevity, the system embodiments of the present invention include all the implementation methods described in the above embodiments of the text verification method for language generation models, and will not be repeated here.

[0065] Based on the above embodiments of the text verification method for language generation models, another embodiment of the present invention provides a terminal device, which includes a processor, a memory, and a computer program stored in the memory and configured to be executed by the processor. When the processor executes the computer program, it implements the text verification method for language generation models according to any embodiment of the present invention.

[0066] For example, in this embodiment, the computer program can be divided into one or more modules, which are stored in the memory and executed by the processor to complete the present invention. The one or more modules may be a series of computer program instruction segments capable of performing a specific function, which describe the execution process of the computer program in the terminal device.

[0067] The terminal device may be a desktop computer, laptop, handheld computer, or cloud server, etc. The terminal device may include, but is not limited to, a processor and a memory.

[0068] The processor can be a Central Processing Unit (CPU), or other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. A general-purpose processor can be a microprocessor or any conventional processor. The processor is the control center of the terminal device, connecting all parts of the terminal device via various interfaces and lines.

[0069] Based on the above-described method embodiments, another embodiment of the present invention provides a computer-readable storage medium including a stored computer program, wherein, when the computer program is executed, it controls the device where the computer-readable storage medium is located to execute the text verification method for language generation model described in any of the above-described method embodiments of the present invention.

[0070] The modules / units integrated in the device / terminal equipment, if implemented as software functional units and sold or used as independent products, can be stored in a computer-readable storage medium. Based on this understanding, all or part of the processes in the above embodiments of the present invention can also be implemented by a computer program instructing related hardware. The computer program can be stored in a computer-readable storage medium, and when executed by a processor, it can implement the steps of the various method embodiments described above. The computer program includes computer program code, which can be in the form of source code, object code, executable files, or certain intermediate forms. The computer-readable medium can include: any entity or device capable of carrying the computer program code, a recording medium, a USB flash drive, a portable hard drive, a magnetic disk, an optical disk, a computer memory, a read-only memory (ROM), a random access memory (RAM), an electrical carrier signal, a telecommunication signal, and a software distribution medium, etc.

[0071] The above description represents the preferred embodiments of the present invention. It should be noted that those skilled in the art can make various improvements and modifications without departing from the principles of the present invention, and these improvements and modifications are also considered to be within the scope of protection of the present invention.

Claims

1. A text verification method for a language generation model, characterized in that, include: Obtain the text to be verified output by the language generation model, and construct a composite triple to be verified based on the text to be verified. The composite triple to be verified includes an event subject field, a documentary evidence field, and a spatiotemporal joint field corresponding to time information and spatial information. The composite triple to be verified is matched with the standard triple in the preset knowledge base in a multidimensional way to obtain a weighted matching degree. The weighted matching degree is obtained by weighting the semantic difference degree corresponding to the event subject field, the spatiotemporal deviation value corresponding to the spatiotemporal joint field, and the consistency matching degree corresponding to the document evidence field. The consistency matching degree and the weighted matching degree are verified in sequence. If the verification fails, multiple prompt messages are generated iteratively according to the preset knowledge base. Each prompt message is input into the language generation model to obtain a corrected triplet. The iteration stops when the corrected matching degree of the corrected triplet and the standard triplet is greater than or equal to the preset threshold, or when the number of iterations is greater than or equal to the preset number, and the target text is obtained.

2. The text verification method for language generation models as described in claim 1, characterized in that, The step of iteratively generating multiple prompt messages based on the preset knowledge base specifically includes: Calculate the weighted similarity between each standard triplet in the preset knowledge base and the composite triplet to be verified. Sort the weighted similarities in descending order and select a preset number of standard triplets in sequence to obtain multiple candidate triplets. The weighted similarity is obtained by weighting the semantic similarity corresponding to the event subject field and the spatiotemporal similarity corresponding to the spatiotemporal joint field. For each candidate triple, the candidate triple and the corresponding original literature evidence field are used as the prompt information, and the prompt information is sequentially input into the language generation model to guide the language generation model to regenerate the corrected triple.

3. The text verification method for language generation models as described in claim 1, characterized in that, The spatiotemporal deviation value is obtained by weighted fusion of temporal distance and spatial distance metrics, specifically: The distance between the time information to be verified in the spatiotemporal joint field and the standard time information in the standard triple is calculated to obtain the time distance metric. The spatial distance metric is also calculated based on the distance between the spatial information to be verified in the spatiotemporal joint field and the standard spatial information in the standard triple. The time distance metric and the spatial distance metric are weighted and fused according to a preset weighting coefficient to obtain the spatiotemporal deviation value.

4. The text verification method for language generation models as described in claim 3, characterized in that, The time distance metric is obtained by calculating the absolute difference between the time information to be verified and the standard time information, and normalizing the absolute difference according to a preset time tolerance parameter; the spatial distance metric is obtained by calculating the geographical distance between the spatial information to be verified and the standard spatial information, and normalizing the geographical distance according to a preset spatial tolerance parameter.

5. The text verification method for language generation models as described in claim 1, characterized in that, The semantic difference is obtained by performing semantic vector embedding and cosine similarity calculation on the event subject field and the standard event field using a preset semantic matching model.

6. The text verification method for language generation models as described in claim 1, characterized in that, The text verification method for the language generation model further includes: When the document evidence field does not meet the preset evidence conditions with the standard evidence field, the preset level weight corresponding to the first evidence level is determined according to the preset evidence level mapping table, and the consistency matching degree is determined according to the preset level weight. The preset evidence conditions refer to the document evidence field being the same as the standard evidence field, or the first evidence level corresponding to the document evidence field being greater than or equal to the second evidence level corresponding to the standard evidence field.

7. The text verification method for language generation models as described in claim 6, characterized in that, The step of performing secondary verification on the consistency matching degree and the weighted matching degree sequentially includes: If the document evidence field meets the preset evidence conditions with the standard evidence field in the standard triplet, and the weighted matching degree is greater than or equal to the preset threshold, then the verification is successful. If the document evidence field does not meet the preset evidence conditions with the standard evidence field, the verification fails. If the document evidence field and the standard evidence field meet the preset evidence conditions, but the weighted matching degree is less than the preset threshold, then the verification fails.

8. A text verification system for a language generation model, characterized in that, include: Build module, match module, and validation module; The construction module is used to obtain the text to be verified output by the language generation model, and construct a composite triple to be verified based on the text to be verified. The composite triple to be verified includes an event subject field, a documentary evidence field, and a spatiotemporal joint field corresponding to time information and spatial information. The matching module is used to perform multidimensional matching between the composite triple to be verified and the standard triple in the preset knowledge base to obtain a weighted matching degree. The weighted matching degree is obtained by weighting the semantic difference degree corresponding to the event subject field, the spatiotemporal deviation value corresponding to the spatiotemporal joint field, and the consistency matching degree corresponding to the document evidence field. The verification module is used to perform secondary verification on the consistency matching degree and the weighted matching degree in sequence. If the verification fails, multiple prompt messages are generated iteratively according to the preset knowledge base. Each prompt message is input into the language generation model to obtain a corrected triplet. The iteration stops when the corrected matching degree of the corrected triplet and the standard triplet is greater than or equal to the preset threshold, or when the number of iterations is greater than or equal to the preset number, and the target text is obtained.

9. A terminal device, characterized in that, The method includes a processor, a memory, and a computer program stored in the memory and configured to be executed by the processor, wherein, when the processor executes the computer program, it implements the text verification method for a language generation model as described in any one of claims 1-7.

10. A computer-readable storage medium, characterized in that, include: A stored computer program, wherein, when the computer program is executed, it controls the device containing the computer-readable storage medium to perform the text verification method for a language generation model as described in any one of claims 1-7.

Images