A processing method and apparatus

Abstract

The application discloses a processing method and device, the method comprises the following steps: obtaining a target expression text; the target expression text is obtained based on an information expression text generated by an intelligent agent; based on the target expression text, determining a matching value of each target reference text corresponding to the information expression text; the matching value is obtained based on comparison between the target expression text and the target reference text; the target reference text is a reference of the information expression text generated by the intelligent agent; displaying the matching value of each target reference text corresponding to the information expression text and the corresponding reference identifier.

Description

Technical Field

[0001] This application relates to the field of computer technology, and in particular to a processing method and apparatus. Background Technology

[0002] Currently, reports can be automatically generated based on intelligent agents. Although the reports indicate the sources cited for each conclusion, the intelligent agents are prone to discrepancies between the conclusions and the corresponding cited content during the integration of research materials, which leads to insufficient credibility of the reports. Summary of the Invention

[0003] The technical solution provided in this application is as follows:

[0004] The first aspect of this application provides a processing method, including:

[0005] Obtain the target expression text; the target expression text is obtained based on the information expression text generated by the intelligent agent;

[0006] Based on the target expression text, the matching value of each target reference text corresponding to the information expression text is determined; the matching value is obtained by comparing the target expression text and the target reference text; the target reference text is a reference for the agent to generate the information expression text;

[0007] Displays the matching value and corresponding reference identifier of each target reference text corresponding to the information expression text.

[0008] In one possible implementation, obtaining the target expression text includes:

[0009] Obtain the information expression text generated by the intelligent agent;

[0010] Identify at least one of a first factual factor and a second factual factor in the information expression text; the first factual factor characterizes the frequency of occurrence and semantic certainty of assertive words in the information expression text; the second factual factor characterizes the completeness of the subject-verb-object structure in the information expression text.

[0011] If the first factual factor and the second factual factor satisfy the first set condition, extract the noun entity that matches the demonstrative pronoun in the information expression text from the preceding text of the information expression text, replace the demonstrative pronoun with the noun entity, and obtain the target expression text.

[0012] In one possible implementation, determining the matching value of each target reference text corresponding to the target expression text based on the target expression text includes:

[0013] Select multiple candidate reference texts that match the target expression text from the reference source;

[0014] Determine at least one of a first verification factor and a second verification factor for each of the candidate cited texts; the first verification factor characterizes the degree of semantic consistency between the candidate cited text and the target expressed text; the second verification factor characterizes the degree of credibility of the numerical matching between the candidate cited text and the target expressed text.

[0015] Based on at least one of the first verification factor and the second verification factor, at least one target reference text is selected from the plurality of candidate reference texts;

[0016] The matching value of the target reference text is determined based on at least one of the first verification factor and the second verification factor of the target reference text.

[0017] In one possible implementation, determining the first verification factor for each of the candidate referenced texts includes:

[0018] Extract evidence text fragments related to the target expression text from the candidate cited text;

[0019] The evidence text fragment and the target expression text are concatenated to obtain the input text;

[0020] The input text is processed based on the target model to obtain confidence values ​​corresponding to various logical relationships; the various logical relationships include implication, neutrality, and contradiction.

[0021] The confidence value corresponding to the implied relationship is determined as the first verification factor.

[0022] In one possible implementation, determining the second verification factor for each of the candidate referenced texts includes:

[0023] Extract the predicted values ​​corresponding to the quantitative indicators from the target text;

[0024] Extract the true value corresponding to the quantitative indicator from the first source of reference; the credibility of the first source of reference meets the set credibility threshold.

[0025] Determine the relative error rate between the predicted value and the true value; the relative error rate characterizes the degree of deviation between the predicted value and the true value;

[0026] Obtain the target coefficient corresponding to the target business scenario to which the target expression text belongs; the target coefficient represents the reasonable error range between the predicted value and the actual value allowed under the target business scenario;

[0027] The second verification factor is determined based on the target coefficient and the relative error rate.

[0028] In one possible implementation, the plurality of candidate reference texts includes: a first type of candidate reference text and a second type of candidate reference text; the authenticity and credibility of the first type of candidate reference text is higher than that of the second type of candidate reference text.

[0029] The step of displaying the matching values ​​and corresponding reference identifiers of each target reference text corresponding to the information expression text includes:

[0030] Select a comparison reference text from the plurality of candidate reference texts; the comparison reference text is of a different type from the target reference text, and at least one of the first verification factor and the second verification factor satisfies the second set condition;

[0031] A first weighting factor is determined based on the target reference text and the reference texts in the comparison reference texts that belong to the first type; the first weighting factor represents the importance of the target reference text and the reference texts in the comparison reference texts that belong to the first type.

[0032] Based on the first weighting factor, at least one of the first verification factor and the second verification factor of the target reference text and the comparison reference text is processed to obtain the third verification factor of the target reference text.

[0033] If the third verification factor of the target reference text meets the set threshold, the third verification factor is used as the matching value, and the third verification factor and the corresponding reference identifier of each target reference text corresponding to the information expression text are displayed.

[0034] In one possible implementation, the processing method further includes:

[0035] If the third verification factor of the target reference text does not meet the set threshold, select a core reference text from the plurality of candidate reference texts that meets at least one of the first verification factor and the second verification factor;

[0036] The information expression text is revised based on the core reference text.

[0037] In one possible implementation, the processing method further includes at least one of the following:

[0038] If the third verification factor of the target reference text does not meet the set threshold, select conflicting reference text pairs with mismatched information from the plurality of candidate reference texts;

[0039] Generate conflict annotation information for the conflicting reference text pairs; the conflict annotation information is used to identify at least one of the following: mismatched information in the conflicting reference text pairs, text in the conflicting reference text pairs that is referenced by the target expression text, and recommended reference text in the conflicting reference text pairs that corresponds to the target expression text.

[0040] Displays the reference identifier, matching value, and conflict annotation information of each referenced text in the conflicting reference text pair.

[0041] In one possible implementation, displaying the matching values ​​and corresponding reference identifiers of each target reference text corresponding to the information expression text includes:

[0042] Sort the target reference texts corresponding to the information expression text according to the matching value to generate a sorted reference text list;

[0043] The sorted list of referenced texts is displayed, and the reference identifier and matching value of each target referenced text are displayed in the sorted list of referenced texts.

[0044] In another aspect, this application provides a processing apparatus, comprising:

[0045] The acquisition module is used to acquire the target expression text; the target expression text is obtained based on the information expression text generated by the intelligent agent;

[0046] The determining module is used to determine the matching value of each target reference text corresponding to the information expression text based on the target expression text; the matching value is obtained by comparing the target expression text and the target reference text; the target reference text is a reference for the agent to generate the information expression text;

[0047] The display module is used to display the matching value and corresponding reference identifier of each target reference text corresponding to the information expression text. Attached Figure Description

[0048] The above and other features, advantages, and aspects of the embodiments of this disclosure will become more apparent from the accompanying drawings and the following detailed description. Throughout the drawings, the same or similar reference numerals denote the same or similar elements. It should be understood that the drawings are schematic, and the originals and elements are not necessarily drawn to scale.

[0049] Figure 1 A flowchart illustrating a processing method provided in Embodiment 1 of this application;

[0050] Figure 2 This is a flowchart illustrating a processing method provided in Embodiment 2 of this application;

[0051] Figure 3This is a flowchart illustrating a processing method provided in Embodiment 3 of this application;

[0052] Figure 4 This is a flowchart illustrating a processing method provided in Embodiment 6 of this application;

[0053] Figure 5 This is a flowchart illustrating a processing method provided in Embodiment 7 of this application. Detailed Implementation

[0054] The embodiments of this application are described below with reference to the accompanying drawings. The terminology used in the implementation section of this application is for explaining specific embodiments only and is not intended to limit the scope of this application.

[0055] The embodiments of this application will now be described with reference to the accompanying drawings. Those skilled in the art will recognize that, with technological advancements and the emergence of new scenarios, the technical solutions provided in the embodiments of this application are equally applicable to similar technical problems.

[0056] The terms "first," "second," etc., used in this application are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such terms can be used interchangeably where appropriate; this is merely a way of distinguishing objects with the same attributes in the embodiments of this application. Furthermore, the terms "comprising" and "having," and any variations thereof, are intended to cover non-exclusive inclusion, so that a process, method, system, product, or apparatus that comprises a series of units is not necessarily limited to those units, but may include other units not explicitly listed or inherent to those processes, methods, products, or apparatuses.

[0057] Reference Figure 1 This is a flowchart illustrating a processing method provided in Embodiment 1 of this application, as shown below. Figure 1 As shown, the method may include, but is not limited to, the following steps:

[0058] Step S101: Obtain the target expression text; the target expression text is obtained based on the information expression text generated by the intelligent agent.

[0059] In this embodiment, the intelligent agent can be a single intelligent agent or multiple intelligent agents working collaboratively. A single intelligent agent can independently generate, verify, and optimize the information expression text.

[0060] In multi-agent collaborative scenarios, there may be, but are not limited to, text generation agents, credibility scoring agents, text correction agents, citation verification agents, etc. Different agents perform their respective duties and work together to complete the entire process of information expression text processing.

[0061] For example, a text generation agent is used to generate original information expression text based on multi-source materials; a credibility scoring agent is used to score the credibility of conclusions and assertions in the information expression text; a text correction agent is used to correct and optimize the text content with deviations based on the scoring results; and a citation verification agent is used to verify and quantify the matching degree between the cited text and the target expression text. The agents can interact with each other and pass instructions to jointly improve the quality and credibility of the generated information expression text.

[0062] Intelligent agents can generate information expression text through methods such as network retrieval, reading from enterprise internal knowledge bases, integrating multi-source heterogeneous materials, and automatic writing.

[0063] In this embodiment, the content of the information expression text may include, but is not limited to: a description of the research background, the data and phenomenon analysis process, text transition sentences, and the core key conclusions and data assertions of the report. The form of the information expression text may include: reports, research minutes, data summaries, analysis summaries, decision summaries, or thematic reviews, etc.

[0064] In this embodiment, the target text can be extracted from the information text generated by the agent. The target text may include content in the information text that has a clear conclusion attribute, a factual assertion attribute, or a quantitative data attribute. For example, the information text generated by the agent may be "This survey focuses on the core business performance of Company B in 2025. After multi-source data integration and analysis, the revenue of Company B's core business reached 860 million yuan in 2025, a year-on-year increase of 15.3%, which is mainly due to the market launch of new product lines." The phrase "the revenue of Company B's core business reached 860 million yuan in 2025, a year-on-year increase of 15.3%" can be considered the target text.

[0065] It should be noted that the target expression text can be flexibly determined according to actual verification needs. When the information expression text generated by the agent consists entirely of conclusive content without redundant expressions, the target expression text can be directly equivalent to the information expression text generated by the agent. When the information expression text contains redundant content, the target expression text can select a single sentence conclusion, or it can select a conclusion fragment with complete semantics that can independently represent a conclusion or a factual assertion.

[0066] Step S102: Based on the target expression text, determine the matching value of each target reference text corresponding to the information expression text; the matching value is obtained by comparing the target expression text and the target reference text.

[0067] The target reference text serves as a reference for the agent in generating the information expression text. Its form may include, but is not limited to, text paragraphs, data tables, chart descriptions, chart annotations, text transcription records of audio and video content, and original record summaries. In other words, any content that can be presented in a readable form and can be used for semantic or numerical comparison and verification with the target expression text belongs to the target reference text described in this step. The target reference text can prove the authenticity and accuracy of the target expression text, and it can be directly read and used for comparison and verification.

[0068] Based on the public nature of the source, the target cited text can be divided into two categories: publicly available cited text and privately available cited text. Publicly available cited text can come from public channels, while privately available cited text can come from internal corporate channels. Both types of text can serve as core support for verifying the authenticity of the target text.

[0069] The target reference text can take many forms, including but not limited to the following, illustrated with examples of public / non-public types: Publicly available target reference text may include corresponding text fragments from publicly available internet web pages (such as original revenue data published on industry news websites, or fragments of company news reported by authoritative media), original search results from publicly available databases (such as fragments of industry statistical data published by the National Bureau of Statistics), and original entries from publicly available knowledge bases (such as complete descriptions of relevant technologies or data in encyclopedia entries). Non-publicly available target reference text may include relevant chapters from confidential internal documents (such as original annual revenue reports retained internally, or fragments of business data from internal meeting minutes), original entries from the company's proprietary knowledge base (such as product parameters and customer cooperation data stored internally), and original records from authoritative internal databases (such as fragments of cost accounting data retained in the company's financial system).

[0070] In this embodiment, a matching value can be obtained by comparing the target text and the target cited text in multiple dimensions. For example, one or more of the following can be performed: literal content comparison, deep semantic comparison, and logical support comparison. Literal content comparison can be used to verify the consistency of the core vocabulary, data, and sentence expression of the two; deep semantic comparison can be used to verify the consistency of the core meaning, viewpoint, and assertion of the two (avoiding situations where the literal meaning is different but the semantic meaning is the same, or the literal meaning is similar but the semantic meaning is different); logical support comparison can be used to verify whether the target cited text can effectively support the conclusion of the target text (avoiding situations where the citation and the conclusion have no logical connection). Through the above comparison process, a specific matching value (such as a numerical range of 0-100) is calculated.

[0071] When an intelligent agent generates informational text, simply labeling citation sources cannot determine whether the conclusion and the citation truly match, or to what extent they match. This leaves users unsure whether the report's conclusion genuinely originates from the citation, or whether there is citation bias (such as discrepancies between conclusion data and citations, contradictions between conclusion viewpoints and citations, or omissions of conclusion content in the citation), thus hindering their ability to assess the conclusion's credibility. However, by comparing and determining the matching value, this fuzzy judgment of whether the conclusion and citation match can be transformed into an objective, quantifiable numerical value. This provides a clear and reliable quantitative standard for subsequent automatic credibility assessments by the system and manual verification by users, enabling accurate identification of citation bias issues and providing a foundation for improving the overall credibility of informational texts.

[0072] Step S103: Display the matching value and corresponding reference identifier of each target reference text corresponding to the information expression text.

[0073] Reference identifiers can be used to uniquely identify and quickly locate each target referenced text. The specific form of the reference identifier can be flexibly set according to the type of the target referenced text. For example, if the target referenced text is a public webpage, the reference identifier can be the complete link to the webpage and the location identifier of the core segment; if it is an internal enterprise document, the reference identifier can be the document number, storage path, and corresponding chapter number; if it is a knowledge base entry, the reference identifier can be the entry ID and entry name, ensuring that users can quickly and accurately find the original content of the target referenced text through this reference identifier and complete manual review.

[0074] In this embodiment, by displaying the citation identifier and the matching value in association, users can intuitively judge the degree of matching between the information expression text and the corresponding target citation text without manually verifying the original text (e.g., the higher the matching value, the more consistent the conclusion is with the citation and the higher the credibility; the lower the matching value, the greater the probability of citation deviation). If a user has doubts about the credibility of a certain conclusion, they can quickly locate the original target citation text through the citation identifier and complete manual verification. This not only improves the verification efficiency but also makes the credibility of the conclusion visible and traceable, effectively solving the technical problem of insufficient credibility of the information expression text caused by deviations in the citation content.

[0075] For example, when displaying the conclusion in the information text that "the company's product line A revenue increased by 12% year-on-year in 2025", the following should be displayed simultaneously: "Citation Identifier: Internal Document Number DOC-2025001 + Storage Path / Revenue Report / 2025 / Product Line A; Match Value: 96" and "Citation Identifier: Public Web Link https: / / xxx.com / 2025 Revenue Data + Location Segment 3; Match Value: 92". Users can quickly determine that the conclusion matches both citations highly and has high credibility through the match value. If the match value is 30, it can quickly identify citation deviations and locate the original text for verification through the citation identifier.

[0076] In this embodiment, by determining the matching values ​​of each target cited text (which can be considered as a citation) corresponding to the target expressed text (which can be regarded as a conclusion), the fuzzy judgment of whether the conclusion and the citation match is transformed into an objective and quantifiable numerical value. This enables accurate identification of citation deviations (data inconsistency, contradictory viewpoints, citations not mentioning the conclusion, etc.), providing a clear quantitative basis for judging the credibility of the conclusion. Furthermore, by displaying the citation identifiers and matching values ​​of the target expressed text and the target cited text in association, rapid tracing of the conclusion (target expressed text) - matching value - original citation text (target cited text) is achieved. Users do not need to manually check the original citation text one by one; they can intuitively judge the degree of matching between the conclusion and the citation, improving verification efficiency. Simultaneously, it achieves visualization and traceability of the credibility of the conclusion, effectively improving the overall credibility of the information expressed text, reducing manual verification costs, and further leveraging the high efficiency advantage of the intelligent agent automatically generating information expressed text.

[0077] As another optional embodiment of this application, refer to Figure 2 This is a flowchart illustrating a processing method provided in Embodiment 2 of this application. This embodiment is mainly an implementation of step S101 in Embodiment 1, such as... Figure 2 As shown, step S101 may include, but is not limited to:

[0078] Step S11: Obtain the information expression text generated by the intelligent agent.

[0079] For a detailed description of step S11, please refer to the relevant introduction on generating information expression text in Example 1, which will not be repeated here.

[0080] Step S12: Determine at least one of the first factual factor and the second factual factor in the information expression text; the first factual factor characterizes the frequency of occurrence and semantic certainty of assertive words in the information expression text; the second factual factor characterizes the completeness of the subject-verb-object structure in the information expression text.

[0081] In this embodiment, assertive terms may include words that can clearly express facts, conclusions, judgments, or definitive viewpoints, such as "reached," "increased," "percentage," "equal to," "certain," "confirmed," "indicated," etc.

[0082] The higher the value of the First Fact Factor, the stronger the factual and conclusive nature of the text fragment in the corresponding information expression text, and the more likely it is to become the target expression text. Conversely, the lower the value, the more the text fragment in the information expression text tends to be descriptive, speculative, or transitional, and the higher the probability of it being non-conclusive. For example, in the text fragment "Company B's core business revenue reached 860 million yuan in 2025," the word "reached" is an assertive word, so the First Fact Factor value of this fragment will be relatively high; while in the text fragment "This survey will focus on Company B's core business performance in 2025," there are no explicit assertive words, so the First Fact Factor value of this fragment will be relatively low.

[0083] Because target texts need to possess clear conclusion and factual assertion attributes, and a complete subject-verb-object structure is the foundation for a statement to clearly express facts and conclusions, incomplete subject-verb-object structures often fail to clearly express specific conclusions or facts (e.g., simply stating "increased by 15.3%" without the subject "who increased," thus failing to constitute a valid conclusion). Therefore, the higher the value of the second factual factor, the more complete the subject-verb-object structure of the corresponding statement, the clearer the expressed facts and conclusions, and the more it meets the requirements of the target text; conversely, the lower the value, the more vague and incomplete the statement is, making it unsuitable as a target text.

[0084] In this embodiment, the sliding window method can be used to accurately determine the factors. Specifically, it can include: moving the sliding window in the acquired information expression text according to a set step size (such as every 2 characters or every 1 sentence as a step size, the step size can be flexibly adjusted according to the form of the information expression text), and obtaining multiple continuous text segments by truncating the sliding window; then, determining at least one of the first factual factor and the second factual factor in each truncated text segment (only one factor can be calculated, or both factors can be calculated simultaneously, flexibly selected according to the actual verification requirements).

[0085] For example, setting the sliding window step size to 1 statement, the information text is: "This survey focuses on the core business performance of Company B in 2025. Through multi-source data integration and analysis, Company B's core business revenue reached 860 million yuan in 2025, a year-on-year increase of 15.3%, mainly due to the market launch of new product lines." Four text fragments are captured through the sliding window, and two factual factors are calculated for each text fragment: Text fragment 1 "This survey focuses on Company B..." 2025 Core Business Performance (no assertive words, complete subject-verb-object structure, low first factual factor, high second factual factor); Text fragment 2 "Based on multi-source data integration and analysis" (no assertive words, incomplete subject-verb-object structure, both factual factors are low); Text fragment 3 "In 2025, the company's core business revenue reached 860 million yuan, a year-on-year increase of 15.3%" (contains two assertive words, "reached" and "increased", complete subject-verb-object structure, both factual factors are high); Text fragment 4 "This growth is mainly due to the market launch of new product lines" (no explicit assertive words, complete subject-verb-object structure, low first factual factor, high second factual factor).

[0086] Step S13: If the first factual factor and the second factual factor satisfy the first set condition, extract the noun entity that matches the indicator pronoun in the information expression text from the preceding text of the information expression text, replace the indicator pronoun with the noun entity, and obtain the target expression text.

[0087] In this embodiment, the factual total factor can be determined according to the following relationship:

[0088]

[0089] Indicates the factual total factor; Indicates the first factual factor; Indicates the second factual factor; and These represent the weighting coefficients of the first factual factor and the second factual factor, respectively, and can be flexibly set according to the actual verification accuracy requirements. It can represent a normalization function, used to normalize the calculation results to a preset range (such as 0-100).

[0090] The first factual factor and the second factual factor satisfy the first predefined condition, which may include, but is not limited to:

[0091] The total factual factor is not less than a first preset threshold. The first preset threshold can be flexibly set according to the type of information text and the actual verification accuracy requirements, as long as it can filter out text fragments with clear conclusion attributes and factual assertion attributes. No specific limitation is made in this application.

[0092] In this embodiment, when the agent generates information expression text, to ensure the fluency of the sentences, it often uses indicator pronouns (such as "this," "its," "this," "the above," etc.) to refer to the noun entities mentioned above (such as company names, business names, data indicators, etc.). However, such indicator pronouns will lead to incomplete semantics in the text fragments. If directly used as the target expression text, semantic ambiguity and matching deviation may occur when comparing it with the target reference text later (for example, the fragment "This growth is mainly due to the market launch of the new product line," if extracted alone, it is impossible to clearly identify the specific content referred to by "this growth"). Therefore, when two factual factors of a certain text fragment meet the first set condition, it can be first determined whether the fragment contains an indicator pronoun; if it does, then the noun entity that uniquely matches the indicator pronoun is extracted from the preceding text of the information expression text (i.e., the text content before the fragment), and the indicator pronoun in the fragment is replaced with the noun entity, finally obtaining the semantically complete and clearly expressed target expression text; if the fragment does not contain an indicator pronoun, then the fragment is directly used as the target expression text.

[0093] Specifically, extracting noun entities that match the demonstrative pronouns in the information expression text may include: in the preceding text of the current information expression text containing the demonstrative pronoun, searching for all candidate noun entities that may be referred to by the demonstrative pronoun, and calculating the matching score M(P,E) between the current demonstrative pronoun P and each candidate noun entity E. The matching score can be calculated using the following relation:

[0094]

[0095] in The distance score represents the positional distance between the demonstrative pronoun P and the candidate noun entity E in the text (the closer the distance, the higher the score). The type matching score represents the degree of match between the referential type of the demonstrative pronoun P and the entity type of the candidate noun entity E (the more closely the types match, the higher the score). and These are the weighting coefficients for the two scores, which can be flexibly set according to the actual requirements for referential matching accuracy.

[0096] After the calculation is completed, the candidate noun entity with the highest matching score M(P,E) can be selected, and the demonstrative pronoun in the original sentence can be directly replaced with the noun entity to obtain the target expression text that is semantically complete and clearly expressed; if the segment does not contain demonstrative pronouns, the segment is directly used as the target expression text.

[0097] In this embodiment, by obtaining the information expression text generated by the agent, an original benchmark is provided for the subsequent extraction of the target expression text, ensuring that the extraction work conforms to the actual scenario of the text generated by the agent and continuing the coherence of the overall solution. By determining at least one of the first factual factor and the second factual factor in the information expression text, it is possible to accurately screen out text fragments with conclusion attributes and factual assertion attributes, effectively excluding redundant and invalid non-conclusive texts, and avoiding the problems of low efficiency and insufficient accuracy in subsequent matching degree verification caused by blind extraction, providing an objective judgment basis for the accurate extraction of the target expression text.

[0098] When the two factual factors meet the first set condition, by extracting the noun entity that matches the demonstrative pronoun from the previous text of the information expression text and performing replacement, the problem of incomplete semantics and ambiguous expression of the target expression text caused by the use of demonstrative pronouns in the text generated by the agent is effectively solved, ensuring that the finally obtained target expression text has clear semantics, is independent and complete, and can be directly used for subsequent matching degree verification, avoiding citation matching deviation caused by semantic ambiguity, and further improving the accuracy and efficiency of subsequent verification work, and further enhancing the overall credibility of the information expression text.

[0099] As another optional embodiment of this application, refer to Figure 3 , which is a schematic flowchart of a processing method provided for Embodiment 3 of this application. This embodiment is mainly an implementation manner of step S102 in Embodiment 1. As Figure 3 shown, step S102 may include but is not limited to:

[0100] Step S21: Select multiple candidate reference texts that match the target expression text from the reference source.

[0101] In this embodiment, the target expression text can be tokenized, disassembling the text into independent lexical units, and at the same time removing meaningless stop words (such as words like "of", "already", "and", etc. that do not have actual semantics and do not affect the core meaning of the conclusion), retaining the keywords that can represent the core meaning of the target expression text (such as data, subject, conclusion view, etc.), to obtain the preprocessed query Q (that is, the structured conclusion object, only containing core semantic information), ensuring the accuracy of subsequent retrieval.

[0102] To balance the accuracy and semantic relevance of the retrieval, and to avoid the problems of missed or false detections caused by a single retrieval method, this embodiment can use two retrieval methods to be executed in parallel. The two methods retrieve reference documents related to the preprocessed query Q from all reference sources. The reference sources include, but are not limited to, the publicly available channels (public web pages, public databases, etc.) and non-public channels (internal enterprise databases, internal documents, etc.) mentioned in Embodiment 1. All traceable original materials belong to the reference sources in this step, namely reference documents D (if there are 10,000 internal databases and 10,000 external databases, then there are a total of 20,000 reference documents D, and the relevance between query Q and each reference document D needs to be calculated separately).

[0103] The specific implementation methods of the two parallel retrieval methods may include, but are not limited to:

[0104] Keyword retrieval: The core is based on exact keyword matching, calculating the relevance score between query Q and each cited document D. This score characterizes the degree of matching between the two at the keyword level. A higher score indicates a greater overlap of core keywords between the cited document D and the target text, signifying a stronger correlation. The calculation can be performed using the following formula:

[0105]

[0106] n represents the number of keywords in the preprocessed query Q;

[0107] This indicates a query for the i-th keyword to be calculated in Q;

[0108] Indicates word frequency, i.e., keywords The frequency of occurrence of the keyword in the current referenced document D. The higher the frequency, the more important the keyword is in document D.

[0109] Inverse frequency is used to characterize keywords. The scarcity of the keyword, that is, the lower the frequency of the keyword in all cited documents D (all 20,000 documents), the higher the IDF value, indicating that the keyword has a stronger distinguishing effect on the search matching and can better reflect the relevance between query Q and cited document D;

[0110] |D| represents the length of the currently referenced document D (quantitative metrics such as word count or character count can be used, and must be consistent with the quantification standard of avgdl);

[0111] avgdl represents the average length of all referenced documents D, which is the sum of the lengths of all referenced documents D divided by the total number of referenced documents;

[0112] This represents the first adjustment parameter, used to adjust word frequency. Relevance score The degree of influence should be considered to avoid score bias caused by excessively high or low word frequencies. The determination method can be flexible, based on the type of the cited source, the average length of the cited document, and the keyword density of the target text. For example, when the cited document is mainly short text (such as data fragments or short conclusions), a setting can be made. ∈[1.2,2.0]; When the referenced document is mainly long text (such as a complete report or chapter content), it can be set to [1.2,2.0]. ∈[2.0,3.0]; Alternatively, the value that optimizes the retrieval accuracy can be selected through multiple trials and calibrations. This application does not impose specific limitations, as long as it can achieve accurate matching at the keyword level.

[0113] b represents the second adjustment parameter, used to adjust the impact of the length of the cited document D on the relevance score. The degree of influence should be considered to avoid excessive impact of long or short documents on the score.

[0114] Semantic retrieval: The core is based on deep semantic association, which makes up for the shortcomings of key information retrieval that only focuses on literal matching and ignores semantic consistency (such as "revenue increased by 12% year-on-year" and "revenue increased by 12% year-on-year", which have different literal keywords but the same semantics), and ensures the comprehensiveness of retrieval results.

[0115] The specific implementation method can be: using pre-trained embedding models (such as BERT, RoBERTa, etc., which have semantic understanding capabilities), the preprocessed query Q and each cited document D can be transformed into high-dimensional embedding vectors respectively. This represents the embedding vector of the preprocessed query Q. (Representing the embedding vector of the cited document D), the semantic relevance score is obtained by calculating the cosine similarity between the two embedding vectors. This score characterizes the degree of consistency between the two at a deep semantic level. A higher score indicates a stronger semantic connection between the cited document D and the target text. The calculation can be performed using the following formula:

[0116]

[0117] Represents the embedding vector of the preprocessed query Q. Embedding vector of referenced document D The cosine similarity of the two vectors ranges from -1 to 1. The closer the value is to 1, the more consistent the directions of the two vectors are, meaning that the semantic relationship between query Q and referenced document D is stronger. The closer the value is to -1, the weaker the semantic relationship between the two. When the value is 0, it means that there is no semantic relationship between the two.

[0118] Represents the embedding vector and The dot product is used to quantify the degree of overlap between two vectors. The larger the dot product, the higher the degree of overlap and the stronger the semantic association.

[0119] Represents the embedding vector The magnitude (length of the vector). Represents the embedding vector The modulus length is normalized to avoid the influence of vector dimension and numerical size on similarity calculation, thus ensuring the objectivity and comparability of the scores.

[0120] After the two retrieval methods are executed, the top k documents with the highest relevance scores are selected from the key information retrieval results and semantic retrieval results respectively (k is a preset value, which can be flexibly set according to the actual retrieval accuracy and efficiency requirements, such as k=10, k=20, etc.), resulting in two Top-k document lists.

[0121] In this embodiment, all referenced documents D in the two Top-k document lists (documents in the two lists may be duplicates or different) can be integrated into a unified candidate document set, without distinguishing whether they come from key information retrieval or semantic retrieval. Then, a final score Score(D) is calculated for each referenced document D in the candidate document set to comprehensively characterize its relevance to query Q.

[0122] In this embodiment, the final score can be calculated using the following formula:

[0123]

[0124] This indicates the ranking of cited document D in the Top-k document list for key information retrieval. It is determined by scoring all cited documents D according to their relevance after the key information retrieval is completed. Sort in descending order, the document ranked first is The highest level document, corresponding to =1, the document ranked 2nd corresponds to =2, and so on, until the document ranked k ( (the lowest among the first k), corresponding to =k; If a referenced document D only exists in the semantic search Top-k list and is not included in the key information search Top-k list, then the document has no... Take the value, at this time you can... Setting it to k+1 avoids calculation errors caused by no possible values ​​and accurately reflects the fact that the document is not included in the Top-k list of key information retrieval and has weak relevance. (The higher the ranking, the smaller the value; for example, if ranked 1st, then...) =1, then the kth rank is... =k);

[0125] This indicates the rank of cited document D in the semantic search Top-k document list (ranking rules are the same as those in the previous section). Completely consistent), meaning that after semantic retrieval, all referenced documents D are scored according to their relevance. Sort in descending order and assign rankings based on the sorting results. the highest =1, decreasing sequentially).

[0126] It should be clarified that the documents in the Top-k document list for key information retrieval and the Top-k document list for semantic retrieval are not necessarily the same. This is because the matching logic of the two retrieval methods is different. Key information retrieval focuses on precise keyword matching, while semantic retrieval focuses on deep semantic association. Therefore, it is possible for a document to enter the Top-k list for key information retrieval due to high keyword overlap, but not enter the Top-k list for semantic retrieval due to weak semantic association, and vice versa. It is also possible for some documents to enter both lists at the same time, all of which are normal search results.

[0127] k represents a smoothing parameter used to prevent a single search result from ranking too high (or too low) and thus causing an excessively high final score, unduly affecting the overall document ranking and avoiding bias from a single search result. Its determination method is similar to that used in key information retrieval. Similarly, it can be flexibly set according to the actual scenario. It is usually set to be consistent with the value of k in the Top-k document list to ensure a balanced smoothing effect.

[0128] Finally, all candidate documents are sorted in descending order based on the final score (Score(D)). The top K documents with the highest scores (K is a preset value that can be set according to the accuracy requirements of subsequent validation, such as K=5, K=10, etc.) are selected as multiple candidate reference texts (i.e., Top-K documents) that match the target text. It should be noted that these candidate reference texts may include documents from internal databases and external web sources. Each candidate reference text itself has a preset tag, which is natively provided by the citation source and is used to identify whether its source type is an internal database or an external web source. This tag is not generated by the model and ensures the authenticity and traceability of the source identification.

[0129] Step S22: Determine at least one of the first verification factor and the second verification factor for each of the candidate reference texts; the first verification factor characterizes the degree of semantic consistency between the candidate reference text and the target expression text; the second verification factor characterizes the degree of credibility of the numerical matching between the candidate reference text and the target expression text.

[0130] In this embodiment, the first verification factor can be used to determine whether the core meaning, viewpoint, and assertion between the candidate quoted text and the target expressed text are consistent, thus avoiding situations where there is a literal match but a semantic deviation, or a literal mismatch but a semantic consistency (for example, the target expressed text "revenue increased by 12% year-on-year" and the candidate quoted text "revenue increased by 12% compared to the previous year" are semantically consistent and have a higher first verification factor).

[0131] The higher the first verification factor, the stronger the semantic consistency between the candidate cited text and the target expressed text; the lower the first verification factor, the weaker the semantic correlation between the two.

[0132] The second verification factor can be used to verify the consistency and accuracy of the core data between the candidate cited text and the target expressed text, thereby avoiding the problem of invalid citations caused by data deviation.

[0133] The higher the second verification factor, the higher the numerical matching degree between the candidate cited text and the target expressed text; the lower the second verification factor, the greater the numerical deviation between the two.

[0134] If the target text does not contain quantitative data (only qualitative conclusions, such as "Company B's core business ranks among the top in the industry"), then there is no need to determine the second verification factor; the judgment can be completed solely through the first verification factor.

[0135] For each candidate cited text, the verification factors to be calculated can be flexibly selected according to the type of the target text, without the need to uniformly calculate two factors. For example, if the target text is a qualitative conclusion (without quantitative data), only the first verification factor can be determined, and the supporting validity of the candidate cited text can be determined by the degree of semantic consistency; if the target text is a quantitative conclusion (including quantitative data), both the first and second verification factors can be determined simultaneously, or only the second verification factor can be determined. The specific setting can be determined according to the actual verification accuracy requirements, and this application does not impose specific limitations.

[0136] Step S23: Select at least one target reference text from the plurality of candidate reference texts based on at least one of the first verification factor and the second verification factor.

[0137] In this embodiment, a preset verification factor judgment threshold (which can be flexibly set according to the actual verification accuracy and the type of information expression text) can be used to judge the verification factor of each candidate cited text:

[0138] If only the first verification factor is determined, a first verification factor judgment threshold can be preset. When the first verification factor is not less than the first verification factor judgment threshold, the candidate reference text is determined to effectively support the target expression text and can be used as the target reference text.

[0139] If only the second verification factor is determined, a second verification factor judgment threshold can be preset. When the second verification factor is not less than the second verification factor judgment threshold, the candidate reference text is determined to effectively support the target expression text and can be used as the target reference text.

[0140] In this embodiment, a threshold different from the first verification factor judgment threshold can be set according to the accuracy requirements of numerical verification. For example, if numerical verification has higher accuracy requirements, the second verification factor judgment threshold can be set to 0.8, which is higher than the 0.7 for semantic verification (i.e., an implementation of the first verification factor judgment threshold).

[0141] If both the first and second verification factors are determined simultaneously, independent thresholds can be preset for each factor (e.g., the first verification factor threshold is set to 0.7, and the second verification factor threshold is set to 0.8). In this embodiment, the judgment logic can be set according to actual needs. For example, both verification factors must be no less than their respective verification factor judgment thresholds (semantic consistency and numerical matching must be satisfied simultaneously for more rigorous support); or, any verification factor must be no less than its corresponding verification factor judgment threshold (semantic consistency or numerical matching is sufficient, suitable for scenarios with relatively relaxed support requirements). The specific judgment logic can be flexibly set.

[0142] From all candidate cited texts that meet the judgment threshold, select at least one as the target cited text. If multiple candidate cited texts meet the judgment threshold, all can be selected (multiple citations jointly support a conclusion), or they can be sorted in descending order of the validation factor score and the top few with the highest scores can be selected. The specific settings can be flexibly configured.

[0143] In this embodiment, for the scenario where a single target expression text is associated with multiple candidate cited texts, the set of verification factors for all candidate cited texts that meet the judgment threshold can be set as follows: Where m is the number of candidate cited texts that meet the threshold.

[0144] In this embodiment, the target referenced text can be determined using the following relation:

[0145]

[0146] This represents the verification factor of the j-th candidate cited text. If two verification factors are calculated simultaneously, This is the result of a weighted fusion of two factors.

[0147] This indicates that the candidate text with the highest score is selected as the target text. Indicates the target referenced text.

[0148] Step S24: Determine the matching value of the target reference text based on at least one of the first verification factor and the second verification factor of the target reference text.

[0149] If only the first verification factor is determined, it can be directly used as the matching value for the corresponding target reference text.

[0150] If only the second verification factor is determined, it can be directly used as the matching value for the corresponding target reference text.

[0151] If the first verification factor and the second verification factor are determined simultaneously, the two verification factors can be weighted using the weight factor corresponding to the first verification factor and the weight factor corresponding to the second verification factor to obtain the final matching value.

[0152] The weighting factors for the first and second validation factors can be flexibly set according to actual needs. For example, in quantitative conclusions, the weighting factor for the second validation factor can be set to be greater than the weighting factor for the first validation factor to emphasize the importance of numerical matching; in qualitative conclusions, the weighting factor for the first validation factor can be set to be greater than the weighting factor for the second validation factor to emphasize the importance of semantic consistency.

[0153] In this embodiment, by selecting Top-K candidate citations that match the target text from the citation sources, the initial accurate screening of citation texts is completed. This avoids the inefficiency caused by indiscriminate processing of all citation sources while retaining candidate texts from different sources, thus balancing screening efficiency and comprehensiveness. Secondly, by determining at least one of the first and second verification factors for each candidate citation text, and selecting at least one target citation text from multiple candidate citation texts based on these verification factors, misjudgments can be effectively avoided, such as literal matching with semantic deviation or literal mismatch with semantic consistency. This ensures the semantic consistency between the citation and the conclusion, and accurately verifies data deviations in quantitative conclusions, guaranteeing the data validity of the citation. This achieves dual accurate verification of semantics and numerical values, improving the accuracy of the target citation text.

[0154] Finally, the matching value is determined based on the corresponding verification factor of the target cited text. The dual verification results of semantic consistency and numerical matching degree are transformed into objective and quantifiable values. When the matching value is low, it can quickly locate whether the error is caused by semantic deviation or numerical deviation, realize efficient investigation of citation deviation problems, further improve the overall credibility of the information expression text generated by the agent, reduce the cost of manual verification, and give full play to the high efficiency advantage of the agent in automatically generating text.

[0155] As another optional embodiment of this application, a processing method provided in Embodiment 4 of this application, this embodiment is mainly an implementation of determining the first verification factor of each candidate cited text in Embodiment 3, and may specifically include, but is not limited to:

[0156] Step S31: Extract evidence text fragments related to the target expression text from the candidate reference text.

[0157] In this embodiment, based on the keywords of the target text (e.g., subject, conclusion, core description, etc.), continuous text fragments that can form a semantic association with the target text can be screened and extracted from the candidate cited text as evidence text fragments (which can be represented as k_Evidence(D), where D represents the candidate cited text being processed, and k_Evidence(D) is the evidence text fragment corresponding to the candidate cited text D).

[0158] Evidence text fragments can independently express a viewpoint, a fact, or a description related to the target text, avoiding semantic verification biases caused by incomplete fragments. Furthermore, evidence text fragments can contain content related to keywords in the target text, ensuring that the fragment has a foundation to support semantic consistency verification.

[0159] For example, if the target text is "Company B's core business revenue reached 860 million yuan in 2025, a year-on-year increase of 15.3%", and a candidate cited text is "In 2025, the overall industry revenue showed a steady growth trend. Among them, Company B, with the strength of its new product line, performed outstandingly in its core business, with annual core business revenue reaching 860 million yuan, a year-on-year increase of 15.3%, setting a new high in nearly three years", then the evidence text fragment k_Evidence(D) extracted from the candidate cited text can include "Company B's annual core business revenue reached 860 million yuan, a year-on-year increase of 15.3%". This fragment removes irrelevant and redundant content such as the overall industry situation and the strength of the new product line, and only retains the core information directly related to the target text, thus meeting the requirements of semantic integrity and relevance.

[0160] Step S32: Concatenate the evidence text fragment and the target expression text to obtain the input text.

[0161] In this embodiment, the target text and evidence text fragments can be concatenated into the input text using the following method:

[0162] X=["CLS"]⊕k_Evidence(D)⊕["SEP"]⊕Gen_Claim(Q)⊕["SEP"]

[0163] Here, ⊕ represents a concatenation operation, which connects the various parts sequentially to form a complete sequence; ["CLS"] and ["SEP"] are both preset special marker symbols used to distinguish text boundaries and identify text types.

[0164] X represents the input text;

[0165] k_Evidence(D) represents the evidence text fragment; Gen_Claim(Q) represents the target text.

[0166] The ["CLS"] marker can be used as the sentence beginning identifier of the input text X, informing the target model that the input sequence is for classifying semantic relationships between text pairs. It also provides the target model with an aggregation node for global semantic representation. When processing the input text, the target model can first identify the ["CLS"] marker, thus clarifying that the purpose of the input sequence is to determine the semantic relationships between text pairs. Based on the semantic vector corresponding to the ["CLS"] marker, and integrating the information from the entire input text, it outputs a confidence value for the logical relationship between the two text segments (evidence text fragment and target expression text).

[0167] The ["SEP"] marker can be used to distinguish different text segments in the input sequence, clearly defining the boundary between the evidence text segment k_Evidence(D) and the target expression text Gen_Claim(Q), thus avoiding confusion between the two texts. In this embodiment, two ["SEP"] markers are used. The first ["SEP"] is placed after the evidence text segment k_Evidence(D) to indicate the end of the evidence text segment; the second ["SEP"] is placed after the target expression text Gen_Claim(Q) to indicate the end of the target expression text, ensuring that the target model can clearly distinguish the scope of the two texts and accurately focus on the semantic association analysis between them.

[0168] It should be clarified that ["CLS"] and ["SEP"] are special markers that are supported by the target model before and after. They can be recognized by the model without additional training. Their format is fixed (with square brackets before and after) to avoid confusion with ordinary characters in the text. The concatenated input text X must remain continuous and have no extra spaces or redundant characters to ensure that the target model can parse and process it normally.

[0169] For example, combining the example from step S31, the evidence text fragment k_Evidence(D) is “Company B’s core business revenue for the whole year reached 860 million yuan, an increase of 15.3% compared with the same period of the previous year”, and the target expression text Gen_Claim(Q) is “Company B’s core business revenue in 2025 reached 860 million yuan, an increase of 15.3% year-on-year”. Then the concatenated input text X is: ["CLS"]⊕“Company B’s core business revenue for the whole year reached 860 million yuan, an increase of 15.3% compared with the same period of the previous year”⊕["SEP"]⊕“Company B’s core business revenue in 2025 reached 860 million yuan, an increase of 15.3% year-on-year”⊕["SEP"]. That is, the complete input text is: ["CLS"]Company B’s core business revenue for the whole year reached 860 million yuan, an increase of 15.3% compared with the same period of the previous year["SEP"]Company B’s core business revenue in 2025 reached 860 million yuan, an increase of 15.3% year-on-year["SEP"].

[0170] Step S33: Process the input text based on the target model to obtain confidence values ​​corresponding to various logical relationships; the various logical relationships include implication relationships, neutral relationships, and contradictory relationships.

[0171] In this embodiment, the target model may include, but is not limited to, a pre-trained Natural Language Inference (NLI) model, which can be fine-tuned based on existing mature pre-trained semantic models (such as BERT, RoBERTa, etc., which have deep semantic understanding capabilities). The fine-tuning process is adapted to the semantic relationship determination scenario of evidence text fragments and target expression text in this application, ensuring that the model can accurately identify the deep semantic relationship between the two texts and avoid the problem of literal matching deviation.

[0172] Implication can be understood as the core semantics of the evidence text fragment k_Evidence(D) fully supporting and proving the core viewpoint, fact, or assertion of the target expression text Gen_Claim(Q), meaning that the two are semantically identical (including cases where the literal meanings are different but the semantics are the same). For example, if the target expression text is "revenue increased by 12% year-on-year" and the evidence text fragment is "revenue increased by 12% compared to the previous year," the two are semantically identical and belong to an implication relationship.

[0173] A neutral relationship can be understood as one where the core semantics of the evidence text fragment k_Evidence(D) are not directly related to the core semantics of the target expression text Gen_Claim(Q). It neither supports nor refutes the viewpoints, facts, or assertions of the target expression text; their semantics have no overlap. For example, if the target expression text is "Company B's core business revenue increased by 15.3%" and the evidence text fragment is "Company B's new product line achieved a market share of 20%", their semantics are unrelated, thus constituting a neutral relationship.

[0174] A contradictory relationship can be understood as one where the core semantics of the evidence text fragment k_Evidence(D) are completely opposite to and conflict with the core semantics of the target expression text Gen_Claim(Q). In other words, the evidence text fragment can refute the viewpoint, fact, or assertion of the target expression text. For example, if the target expression text is "Company B's core business revenue reached 860 million yuan," and the evidence text fragment is "Company B's core business revenue was only 680 million yuan," the two are semantically conflicting and thus constitute a contradictory relationship.

[0175] After processing the input text X, the target model outputs a three-dimensional probability distribution. The three values ​​in this distribution correspond to the confidence levels of the three logical relationships (implication, neutrality, and contradiction) mentioned above, and the sum of the three confidence levels is 1 (i.e., it meets the basic requirements of a probability distribution). A higher confidence level for a given logical relationship indicates a greater probability that the evidence text fragment and the target text belong to that logical relationship; conversely, a lower confidence level indicates a smaller probability that they belong to that logical relationship.

[0176] For example, combining the input text X from step S32, the three-dimensional probability distribution output by the target model after processing is [0.96, 0.03, 0.01]. The first value of 0.96 corresponds to the confidence value of the implication relationship, the second value of 0.03 corresponds to the confidence value of the neutral relationship, and the third value of 0.01 corresponds to the confidence value of the contradictory relationship. This result shows that the probability that the evidence text fragment and the target expression text belong to the implication relationship is 96%, and the probability of belonging to the neutral relationship and the contradictory relationship is extremely low. The two are highly consistent in meaning.

[0177] Step S34: Determine the confidence value corresponding to the implication relationship as the first verification factor.

[0178] In this embodiment, the confidence value corresponding to the implication relation is extracted from the three-dimensional probability distribution output in step S33, and this confidence value is directly determined as the first verification factor corresponding to the current candidate cited text D, i.e. ,in This represents the result of the target model's processing of the input text X (i.e., the confidence value of the implication relation). This represents the first verification factor.

[0179] The confidence value of implication relations can accurately quantify semantic consistency. The higher the confidence value of implication relations, the stronger the semantic consistency between the two, the higher the first verification factor, and the higher the degree of semantic support of the candidate reference text to the target text. Conversely, the lower the confidence value of implication relations, the weaker the semantic consistency between the two, the lower the first verification factor, and the lower the degree of semantic support of the candidate reference text to the target text.

[0180] In this embodiment, relevant evidence text fragments are extracted from candidate cited texts, redundant and irrelevant content is removed, and interference from non-core information on semantic verification is reduced, thereby improving the efficiency and accuracy of subsequent model processing. By splicing the evidence text fragments with the target expression text to form the input text, it is ensured that the target model can clearly identify text boundaries, laying the foundation for accurate determination of semantic relationships.

[0181] Building upon this foundation, the target model processes the input text and outputs confidence scores for three logical relationships: implication, neutrality, and contradiction. This enables an objective quantitative assessment of the deep semantic connections between the two texts, effectively avoiding misjudgments such as literal matching with semantic deviation or literal mismatch with semantic consistency. Ultimately, the confidence score of the implication relationship is determined as the first validation factor, achieving precise quantification of semantic consistency. This provides a clear quantitative basis for semantic-level citation validity verification, further improving the accuracy of target text selection, ensuring accurate identification of citation bias from a semantic perspective, and guaranteeing the credibility of the reports generated by the intelligent agent.

[0182] As another optional embodiment of this application, a processing method provided in Embodiment 5 of this application, this embodiment is mainly an implementation of determining the second verification factor of each candidate cited text in Embodiment 3, and may specifically include, but is not limited to:

[0183] Step S41: Extract the predicted values ​​corresponding to the quantitative indicators from the target text.

[0184] In this embodiment, the target text can be structured and parsed to identify indicators with quantitative attributes (such as "Company B's core business revenue in 2025" and "year-on-year growth rate"). The specific values ​​corresponding to these indicators can then be extracted, and it is necessary to ensure that the indicators and values ​​correspond one-to-one to avoid confusing values ​​of different dimensions.

[0185] For example, if the target text is "Company B's core business revenue reached 860 million yuan in 2025, a year-on-year increase of 15.3%", the parsing process will first identify two quantitative indicators: "Company B's core business revenue in 2025" and "year-on-year growth rate", and then extract the corresponding predicted values: revenue forecast value. =860 million yuan, growth rate forecast =15.3%.

[0186] Step S42: Extract the real value corresponding to the quantitative indicator from the first reference source.

[0187] The authenticity and credibility of the first source of reference meet the set credibility threshold. The first source of reference (such as internal financial ledgers, original records of business systems, core database archives, etc.) is more authoritative and more in line with the actual business of the enterprise than the data from publicly available search channels, and can avoid the problems of lag or distortion of publicly available data.

[0188] In this embodiment, based on quantitative indicators, the original values ​​of the same dimension can be retrieved from the first reference source (e.g., "Company B's core business revenue in 2025" needs to match the revenue data of the same dimension in the internal financial statements, rather than the revenue of other business lines).

[0189] For example, regarding the quantitative indicator of "Company B's core business revenue in 2025," the actual value was retrieved from the company's internal financial ledger for 2025. =850 million yuan; Regarding the "year-on-year growth rate", the actual value was retrieved from the internal business analysis system. =15.1%.

[0190] Step S43: Determine the relative error rate between the predicted value and the true value; the relative error rate characterizes the degree of deviation between the predicted value and the true value.

[0191] In this embodiment, the relative error rate between the predicted value and the true value can be determined by the following relationship:

[0192]

[0193] This represents the absolute difference between the predicted value and the actual value.

[0194] Represents the absolute value of the true value, ensuring that the denominator is positive; This represents a very small division-to-zero constant (usually 0.0001), avoiding division by zero when the actual value is zero. When the value is 0 (e.g., "the number of customer churns is 0"), the denominator is 0, which makes the formula unable to calculate;

[0195] δ represents the relative error rate. The value of δ is ≥0. The larger the value, the greater the deviation between the predicted value and the true value.

[0196] Step S44: Obtain the target coefficient corresponding to the target business scenario to which the target expression text belongs; the target coefficient represents the reasonable error range between the predicted value and the actual value allowed under the target business scenario.

[0197] In this embodiment, multiple reasonable error thresholds (i.e. multiple alternative σ values) can be set according to actual business needs, for different business scenarios and different types of quantitative indicators.

[0198] For example, in business scenarios with extremely high numerical accuracy requirements and extremely low error tolerance (such as financial core data accounting, audit data verification, compliance data reporting, etc.), a small alternative σ value (such as 1% or 2%) can be set. This means that in such scenarios, as long as the relative error rate exceeds the threshold, it is judged as numerical mismatch and unacceptable.

[0199] For business scenarios where numerical precision requirements are less stringent and are significantly affected by objective factors (such as market fluctuations and differences in data statistical methods) (such as market size estimation, industry trend analysis, and preliminary survey data verification), a larger alternative σ value (such as 5% or 8%) can be set to allow for a higher relative error range and avoid misjudgment due to minor fluctuations.

[0200] Meanwhile, in the same business scenario, different types of quantitative indicators may correspond to different alternative σ values. For example, in the financial scenario, the alternative σ value for core indicators such as "core business revenue" and "net profit" is set at 2%, while "year-on-year growth rate" and "month-on-month growth rate" are more affected by market environment and seasonal factors, so the alternative σ value can be set at 3%, further refining the threshold adaptability.

[0201] After setting multiple alternative σ values, a mapping relationship between business scenarios, quantitative indicator types, and reasonable error thresholds (alternative σ values) can be established and stored. Alternatively, it can be simplified to a mapping relationship between business scenarios and reasonable error thresholds according to actual needs.

[0202] In this embodiment, the target business scenario to which the target expression text belongs can be determined by using keywords, preset tags, etc., in the target expression text. For example, by using keywords, preset tags, etc., the target expression text can be identified as belonging to a financial scenario, a marketing scenario, or an audit scenario.

[0203] From the mapping relationship of multiple alternative σ values, a reasonable error threshold corresponding to the target business scenario can be retrieved as the target coefficient.

[0204] In this embodiment, the specific type of quantitative indicator in the target text can also be determined (e.g., identifying core business revenue, year-on-year growth rate, or market size). After identification, a pre-stored mapping relationship is invoked to accurately match a reasonable error threshold corresponding to the target business scenario and quantitative indicator type from multiple candidate σ values. This threshold is then determined as the target coefficient used for this numerical verification.

[0205] Step S45: Determine the second verification factor based on the target coefficient and the relative error rate.

[0206] In this embodiment, a second verification factor can be obtained through nonlinear mapping using the Gaussian radial basis function (RBF). Specifically, the second verification factor can be determined using the following relationship:

[0207]

[0208] Indicates the second verification factor;

[0209] This represents the natural exponential function, with its base e approximately equal to 2.71828. This function can convert input real numbers into values ​​between 0 and positive infinity. Combined with the negative sign of the exponent in the above formula, this ensures the final output... The second verification factor always falls within the 0-1 range, which facilitates the calculation and determination of subsequent matching values.

[0210] This is used to normalize the relative error rate δ relative to the business tolerance coefficient σ, eliminating the impact of differences in σ values ​​under different scenarios on error measurement. This ensures that regardless of the value of σ, the deviation of δ from σ can be uniformly measured through this calculation item. The results, combined with the characteristics of the natural exponential function, can be divided into three core cases, clearly defining the numerical matching confidence levels corresponding to different error levels:

[0211] In the first scenario, when the relative error rate δ is less than or equal to the business tolerance coefficient σ, meaning the error is within a reasonable range acceptable to the business scenario, then... The calculated result will be between 0 and 0.5, a relatively small value. Since the natural exponential function exp(-x) outputs a result close to 1 when x is a small positive number, the second validation factor is used in this case. It will approach 1, and the closer δ is to 0 (that is, the closer the predicted value is to the actual value). The closer the value is to 1, the higher the numerical matching degree between the target text and the target cited text, and the stronger the numerical credibility.

[0212] In the second scenario, when the relative error rate δ equals the business tolerance coefficient σ, that is, when the error just reaches the maximum acceptable limit for the business scenario, then... The calculated result is exactly 0.5. After substituting into the natural exponential function, the result of exp(-0.5) is approximately 0.607. This score is the acceptable boundary benchmark value of the credibility of numerical matching, which indicates that the deviation between the predicted value and the true value has just reached the upper limit of business tolerance, and the credibility of numerical matching is in an acceptable critical state.

[0213] The third scenario occurs when the relative error rate δ exceeds the business tolerance coefficient σ, meaning the error exceeds the reasonable range acceptable to the business scenario. The calculated result will be greater than 0.5, and the value of this calculated term will increase rapidly as the difference between δ and σ increases. Since the natural exponential function exp(-x) decreases exponentially when x is a large positive number, the second verification factor... It will decrease rapidly, and the more δ exceeds σ, The faster the rate of decline, the closer it will eventually be to 0, indicating a severe mismatch between the numerical values ​​of the target text and the target referenced text. There is a logical contradiction between the two at the numerical level, and the target referenced text cannot support the numerical assertions of the target text.

[0214] In this embodiment, the nonlinear characteristics of the Gaussian radial basis function are more in line with the actual business decision-making logic of enterprises. Within an acceptable error range, small errors are allowed, and the second verification factor decreases gradually, so as not to overly negate the credibility of numerical matching due to minor numerical fluctuations. However, once the error exceeds the acceptable range, the second verification factor will drop rapidly, which can accurately identify unacceptable numerical deviations and highlight the accuracy requirements of core data. This ensures that the calculated second verification factor can truly and objectively reflect the credibility of numerical matching, providing accurate and business-compliant quantitative basis for determining the subsequent target reference text matching value.

[0215] As another optional embodiment of this application, refer to Figure 4 This is a flowchart illustrating a processing method provided in Embodiment 6 of this application. This embodiment is mainly an implementation of step S103 in Embodiment 3. In this embodiment, the plurality of candidate reference texts may include: candidate reference texts of a first type and candidate reference texts of a second type.

[0216] The credibility of the first type of candidate reference text is higher than that of the second type of candidate reference text.

[0217] The sources of the first type of candidate cited text may include, but are not limited to:

[0218] Internal enterprise databases, internal access control documents, internal original vouchers, internal real data sources, and verified internal statistical data, etc.

[0219] This type of text usually requires access permissions, is supported by original data, and its authenticity can be directly verified, thus its authenticity and credibility are higher.

[0220] The sources of the second type of candidate cited text may include, but are not limited to:

[0221] Texts that are publicly available through search channels, such as web pages, databases, news articles, reports, and knowledge bases.

[0222] This type of text can be accessed without permission, but its authenticity is usually lower than that of the first type of candidate cited text.

[0223] The candidate cited text of the first type is not publicly disclosed. This type of candidate cited text has the technical characteristics of closed source, original data and no risk of public tampering. It is the core authoritative basis for verifying the authenticity and accuracy of the target expression text, and its credibility is significantly higher than that of externally published text.

[0224] The second type of candidate cited texts are publicly available. These candidate cited texts are easily affected by factors such as the publisher's subjective factors, differences in data statistics, information lag, and distortion in the dissemination process. Their authority and credibility are lower than those of the first type of candidate cited texts.

[0225] like Figure 4 As shown, step S103 may include, but is not limited to:

[0226] Step S51: Select a comparison reference text from the plurality of candidate reference texts; the comparison reference text is of a different type from the target reference text, and at least one of the first verification factor and the second verification factor satisfies the second set condition.

[0227] If the target reference text is a candidate reference text of the first type, then the reference text to be compared should be selected from the candidate reference text of the second type.

[0228] If the target reference text is a candidate reference text of the second type, then the reference text to be compared should be selected from the candidate reference text of the first type.

[0229] At least one of the first verification factor and the second verification factor satisfies the second set condition, which may include, but is not limited to, at least one of the following:

[0230] The first and / or second validation factors of the compared cited texts are the largest among multiple candidate cited texts;

[0231] The first and / or second verification factors of the compared reference text are not less than a preset minimum valid threshold. This preset minimum valid threshold can be flexibly set according to the actual business scenario and verification accuracy requirements. It is used to ensure that the compared reference text is substantially related to the target text and to exclude text that is completely irrelevant, semantically contradictory, or numerically unrelated.

[0232] Step S52: Based on the target reference text and the reference text in the comparison reference text that belong to the first type, determine a first weighting factor; the first weighting factor represents the importance of the reference text in the target reference text and the reference text in the comparison reference text that belong to the first type.

[0233] From the text set consisting of the target cited text and the comparison cited text, all cited texts belonging to the first type (internal and non-public) are selected, and two core features of the text are extracted to construct a feature vector. The expression for the eigenvector is as follows:

[0234]

[0235] This represents the source identification factor, used to determine whether the text is a first-type cited text. Its value is a fixed constant of 1, with no other possible values. This method of value selection avoids ambiguity in source determination and ensures the consistency of feature vectors.

[0236] The information timeliness factor is used to characterize the degree of match between the update time of the first type of cited text and the time dimension pointed to by the target text.

[0237] The above feature vectors The pre-trained weight model is input and non-linearly mapped using the Sigmoid function. Specifically, the first weight factor can be obtained through the following relationship:

[0238]

[0239] Indicates the first weighting factor;

[0240] and This represents the pre-trained parameters of the weighted model, which do not require real-time training in this step and can be directly accessed. These parameters are trained using historical enterprise validation data (including historical target text, corresponding multi-source cited text, and manually annotated credibility results). When there is a conflict between the validation results of the first type of cited text and the second type of cited text, the weight of the first type of cited text can be automatically increased to ensure that authoritative evidence takes precedence.

[0241] The Sigmoid function is used to convert the result of linear calculations. Mapping to the (0,1) interval avoids weight overflow and ensures the rationality of the first weight factor value.

[0242] Step S53: Based on the first weighting factor, process at least one of the first verification factor and the second verification factor of the target reference text and the comparison reference text to obtain the third verification factor of the target reference text.

[0243] In this embodiment, a linear weighted fusion algorithm can be used, combined with a first weight factor, to fuse the verification factors of the two types of cited texts. Specifically, the calculation can be performed using the following relational formula:

[0244]

[0245] This represents the third verification factor, which is the final comprehensive matching value of the target cited text. The value ranges from (0,1). The higher the value, the higher the overall matching degree between the target text and the multi-source cited text, and the stronger the credibility of its authenticity and accuracy.

[0246] This represents the first weight factor, with a value range of (0,1), which indicates the weight percentage of the first type of cited text.

[0247] This represents the overall score of the first type of cited text. Its determination is based on the type of the target text, and specific methods of determination may include:

[0248] If the target text is a qualitative conclusion (without quantitative data), then =The first validation factor for the first type of referenced text;

[0249] If the target text expresses a quantitative conclusion (containing quantitative data), then =The second verification factor for the first type of referenced text;

[0250] If the target text is a mixed conclusion (containing both qualitative description and quantitative data), then = w1×first verification factor+ w2×second verification factor, where w1 and w2 are weight coefficients, w1+w2 = 1, which can be set according to actual verification needs, preferably w1=0.4 and w2=0.6 (prioritizing the accuracy of numerical matching).

[0251] This represents the overall score of the second type of cited text, and its determination method is the same as... Completely consistent, except that the data source is the second type of reference text (comparison reference text), to avoid distortion of weighted results due to differences in calculation methods.

[0252] This indicates the weight percentage of the second type of cited text, which complements the first weight factor, ensuring that the sum of the weights of the two types of cited text is 1. This conforms to the mathematical logic of linear weighted fusion and ensures the rationality of the calculation results.

[0253] Step S54: If the third verification factor of the target reference text meets the set threshold, the third verification factor is used as the matching value, and the matching value and corresponding reference identifier of each target reference text corresponding to the information expression text are displayed.

[0254] The threshold value is set according to the business scenario to which the target text belongs. Different business scenarios have different requirements for credibility, and therefore different threshold values, to ensure the rationality and adaptability of the threshold setting.

[0255] If the third verification factor of the target referenced text is not less than the set threshold, it can be determined that the overall credibility of the target expression text and the corresponding target referenced text meets the standard, and the display operation is performed.

[0256] In this embodiment, by filtering comparative reference texts that are different from the target reference text and whose verification factors meet the second set conditions, complementary verification of internal authoritative evidence and external supplementary evidence can be achieved, avoiding the one-sidedness of verification of a single type of evidence and interference from invalid texts.

[0257] The first weighting factor, which represents the importance of the first type of text in the two types of cited texts, can highlight the authority of internal non-public texts. The third verification factor is obtained by processing at least one verification factor of the target cited text and the comparison cited text in combination with the first weighting factor. This can solve the verification bias problem caused by the conflict of authority of texts from different sources and improve the objectivity and accuracy of the verification results.

[0258] The reference identifier of the target reference text and the third verification factor (i.e., an implementation of a matching value) are only displayed when the third verification factor meets the set threshold. This further enhances the credibility and traceability of the target reference text generated by the agent, and can prevent users from mistakenly taking the target expression text that has not been fully verified as a credible conclusion, thereby avoiding misleading users' decisions.

[0259] As another optional embodiment of this application, a processing method provided in embodiment 7 of this application may include, but is not limited to, the following steps:

[0260] Step S201: Obtain the target expression text; the target expression text is obtained based on the information expression text generated by the intelligent agent.

[0261] For a detailed description of step S201, please refer to the relevant description of step S101 in Example 1, which will not be repeated here.

[0262] Step S202: Select multiple candidate reference texts from the reference sources that match the target expression text.

[0263] The candidate cited texts include: a first type of candidate cited text and a second type of candidate cited text; the credibility of the first type of candidate cited text is higher than that of the second type of candidate cited text.

[0264] Step S203: Determine at least one of the first verification factor and the second verification factor for each of the candidate reference texts; the first verification factor characterizes the degree of semantic consistency between the candidate reference text and the target expression text; the second verification factor characterizes the degree of credibility of the numerical matching between the candidate reference text and the target expression text.

[0265] Step S204: Select at least one target reference text from the plurality of candidate reference texts based on at least one of the first verification factor and the second verification factor.

[0266] Step S205: Determine the matching value of the target reference text based on at least one of the first verification factor and the second verification factor of the target reference text.

[0267] For a detailed description of steps S202-S205, please refer to the relevant description of steps S21-S24 in Example 3, which will not be repeated here.

[0268] Step S206: Select a comparison reference text from the plurality of candidate reference texts; the comparison reference text is of a different type from the target reference text, and at least one of the first verification factor and the second verification factor satisfies the second set condition.

[0269] Step S207: Based on the target reference text and the reference text belonging to the first type in the comparison reference text, determine a first weighting factor; the first weighting factor characterizes the importance of the reference text belonging to the first type in the target reference text and the comparison reference text.

[0270] Step S208: Based on the first weighting factor, process at least one of the first verification factor and the second verification factor of the target reference text and the comparison reference text to obtain the third verification factor of the target reference text.

[0271] Step S209: If the third verification factor of the target reference text meets the set threshold, the third verification factor is used as the matching value, and the third verification factor and the corresponding reference identifier of each target reference text corresponding to the information expression text are displayed.

[0272] For a detailed description of steps S206-S209, please refer to the relevant description of steps S51-S54 in Example 6, which will not be repeated here.

[0273] Step S210: If the third verification factor of the target reference text does not meet the set threshold, select the core reference text from the plurality of candidate reference texts that meets at least one of the first verification factor and the second verification factor.

[0274] The fact that at least one of the first verification factor and the second verification factor satisfies the third setting condition may include, but is not limited to, at least one of the following:

[0275] Optimal validation factor: Among all candidate reference texts, the candidate reference text has the highest first validation factor (semantic consistency) and / or second validation factor (numerical matching degree), that is, it has the highest matching degree with the target expression text compared with other candidate texts;

[0276] Validation factor meets the standard: The first and / or second validation factors of the candidate referenced text meet the preset minimum valid threshold.

[0277] One or more core reference texts can be selected, and the specific selection logic may include:

[0278] Candidate cited texts that simultaneously meet the criteria of optimal validation factor and validation factor compliance are selected as core cited texts (a single optimal text is preferred to ensure a consistent direction of correction).

[0279] If multiple candidate reference texts meet the third set condition (e.g., multiple text verification factors meet the standard and the values ​​are close), multiple can be selected as core reference texts and used together as the basis for correction.

[0280] The core cited text is still divided into the first type (internal non-public) and the second type (external public). The first type of text is preferred (as it is more authoritative). If only the second type of text exists, the text that meets the conditions in the second type can be selected.

[0281] Step S211: Correct the information expression text based on the core reference text.

[0282] In this embodiment, constraint instructions can be constructed based on the core reference text to clarify the direction of model correction. That is, the corrected information expression text must be consistent with the core information of the core reference text and not deviate from the core meaning.

[0283] The completed constraint instructions are input into the constraint instruction model. The model then uses these constraints to calibrate the information expression text, correcting any inconsistencies with the core reference text (such as semantic deviations, numerical errors, or ambiguous expressions), while preserving the main structure of the information expression text and avoiding excessive corrections.

[0284] After the correction is completed, a secondary verification process can be triggered, and the verification process of steps S202-S209 in Example 7 can be repeated to recalculate the third verification factor of the corrected target expression text and determine whether it meets the set threshold.

[0285] If the third verification factor meets the set threshold after the second verification, the correction is deemed qualified, and the corrected information expression text, the corresponding third verification factor (which has met the set threshold after the second verification), and the citation identifier can be output normally. If it still does not meet the requirements, the core citation text can be selected again (such as selecting multiple core texts for joint correction), and the correction process in this step can be repeated until the target is met.

[0286] In this embodiment, if the third verification factor of the target reference text does not meet the set threshold, at least one of the first verification factor and the second verification factor that meets the third set condition is selected from the plurality of candidate reference texts. This avoids deviations caused by unfounded corrections. At the same time, by specifically correcting the inconsistencies between the information expression text and the core reference text, the deficiency of insufficient credibility of the original information expression text is effectively compensated. This ensures that the semantics and numerical values ​​(if any) of the corrected information expression text are consistent with those of the core reference text, and triggers secondary verification to form a closed loop. This not only ensures the effectiveness and accuracy of the correction effect, but also eliminates the need for complex operation procedures. This further improves the verification and optimization system of information expression text generated by the intelligent agent, enhances the credibility and practicality of the information expression text, and ensures that the output content can meet the usage requirements of business scenarios.

[0287] The following combination Figure 5 The processing method provided in this application is described in complete and accurate detail. The technical details of each step are consistent with the previous embodiments, and may specifically include:

[0288] I. Information Entropy Extraction (Target Text Acquisition Stage)

[0289] The core objective of this step is to accurately extract target text with conclusion and factual assertion attributes from the information text generated by the agent. The specific steps are as follows:

[0290] Obtain the original information expression text generated by the intelligent agent. This text can be generated through network retrieval, reading from the enterprise's internal knowledge base, and integration of multi-source heterogeneous materials. The forms include reports, survey minutes, data briefs, etc.

[0291] The acquired information text is processed by text segmentation and context preservation using a sliding window method: the sliding window moves in the information text according to a preset step size (e.g., one step size for each sentence, which can be flexibly adjusted according to the text format) to capture multiple consecutive text segments, ensuring that each segment retains complete context information and avoiding semantic breaks.

[0292] Evaluate the factual features of each intercepted text segment, and specifically calculate two core indicators: one is the assertive word score (i.e., the first factual factor), which represents the frequency and semantic certainty of assertive words (such as "reach", "increase", "determine", etc.) in the text segment. The higher the score, the stronger the factuality and conclusiveness of the text segment; the other is the subject-predicate-object integrity score (i.e., the second factual factor), which represents the degree of integrity of the subject-predicate-object structure in the text segment. The higher the score, the clearer the facts and conclusions expressed in the text segment.

[0293] Perform the operation of replacing the execution entity and the pronouns: If the first factual factor and the second factual factor of a certain text segment meet the first set condition (i.e., the total factual factor is not less than the first preset threshold), then extract the noun entity (such as enterprise name, business name, data indicator, etc.) that uniquely matches the demonstrative pronoun (such as "this", "its", "this", etc.) in the segment from the previous text of the information expression text (the content before this segment), and replace the demonstrative pronoun with this noun entity to finally obtain the target expression text with complete semantics and clear expression (also called the structured conclusion object); if there is no demonstrative pronoun in the segment, directly use this segment as the target expression text.

[0294] II. Selection of evidence texts based on keyword and semantic matching (candidate citation text screening session)

[0295] The core of this session is to screen out candidate citation texts that are highly relevant to the target expression text from all citation sources to provide evidence support for subsequent verification. The specific implementation steps are as follows:

[0296] Keyword retrieval (TF-IDF): Perform word segmentation on the target expression text,剔除 meaningless stop words (such as "of", "already", "by", etc.), and retain the keywords representing the core meaning (such as data, subject, conclusion view, etc.). Based on these keywords, perform an accurate search on all citation sources (including internal non-public channels and external public channels), calculate the keyword relevance score of each citation document and the target expression text, and screen out the documents with higher relevance;

[0297] Semantic retrieval: Use pre-trained embedding models (such as BERT, RoBERTa, etc.) to convert the target expression text and each citation document into high-dimensional embedding vectors (Embedding) respectively. By calculating the cosine similarity of the two vectors, obtain the semantic relevance score between the two, and make up for the defect that keyword retrieval only focuses on literal matching and ignores semantic consistency, and screen out the documents with higher semantic relevance;

[0298] Citation document reordering: The two Top-k document lists obtained from keyword retrieval and semantic retrieval (k is a preset value that can be adjusted flexibly) are integrated to form a unified candidate document set, without distinguishing the retrieval source of each document; then, for each cited document D in the set, its final score Score(D) is calculated to comprehensively characterize the degree of relevance between the document and the target text, and finally, the documents are sorted in descending order of Score(D) to obtain the sorted candidate cited text documents.

[0299] For example, text A has a final score of 0.92, ranking first in the candidate document set; text B has a final score of 0.88, ranking second, meaning that text A is more relevant to the target text than text B.

[0300] III. Logical Consistency Verification (Matching Value Calculation)

[0301] The core of this step is to verify the consistency between the candidate cited text and the target expression text, quantifying the degree of matching between the two. The specific steps are as follows:

[0302] Determine the discrimination method: Select the corresponding consistency discrimination method according to the type of target text (qualitative text, quantitative numerical, mixed) to ensure the targeting of the verification;

[0303] Semantic implication calculation (first validation factor): Based on the Natural Language Inference (NLI) model, semantic association analysis is performed on the evidence fragments of the candidate cited text and the target expression text to obtain the semantic implication confidence value, which is the first validation factor, representing the degree of semantic consistency between the two. For example, if the input text format is [cls] Zhang Gong completed the project in 2023 [SEP] Zhang Gong completed the project in 2024… [SEP] Tax increase of 20%, the model outputs the semantic implication confidence value of the two as the first validation factor.

[0304] Numerical bias calculation (second validation factor): For target text containing quantitative data, a second validation factor is calculated using RBF (Gaussian radial basis function) combined with the relative error method to characterize the credibility of the numerical match between the two. For example, if a certain value in the target text (report) is 100, and the corresponding value in the core cited text (real data) is 96, the relative error between the two is first calculated, and then the relative error is mapped to a credibility score in the 0-1 range using the RBF function. This score is the second validation factor.

[0305] Weighted fusion score: Based on the type of the target text, the first verification factor and the second verification factor are assigned corresponding weights and weighted calculations are performed to obtain the comprehensive matching value between the candidate reference text and the target text. The target reference text is then selected based on this value.

[0306] For example, the target texts obtained through filtering or selection methods include "It is predicted that if AI demand continues, memory prices may fluctuate significantly in 2026", with a final overall matching score of 0.44; and "In the first quarter of 2025, xx Electronics raised the contract price of enterprise-grade DRAM by 15%", with a final overall matching score of 0.92 (the higher the score, the higher the matching degree and the stronger the credibility).

[0307] IV. Conflict and Optimization Feedback (Weight Integration and Credibility Determination)

[0308] The core of this step is to resolve the conflict of authority between different types of cited texts (internal non-public and external public) and optimize the credibility determination results. The specific steps are as follows:

[0309] Trust weight (first weight factor) calculation: Based on the target cited text and the comparison cited text (different from the target cited text) that belong to the first type (internal non-public), extract their feature vector (including source identifier and information timeliness factor), and map the feature vector to the first weight factor α_auth (value range 0-1) through the Sigmoid function. This factor represents the importance of internal non-public cited text, ensuring that internal authoritative evidence receives higher weight in conflict scenarios;

[0310] Weighted fusion: Based on the calculated first weight factor, the verification factors (first verification factor and second verification factor) of the target cited text and the comparison cited text are linearly weighted and fused to obtain the third verification factor S of the target expressed text. final This will serve as the final basis for determining credibility.

[0311] V. Automation and Correction Strategies (Final Verification and Optimization Closed-Loop)

[0312] The core of this step is to automatically verify and correct the target text based on the results of the third verification factor, forming a complete closed loop. The specific rules are as follows:

[0313] If the third verification factor S final ≥ Set threshold T pass If the target text is deemed authentic and reliable, a citation index (associated with the corresponding target citation text identifier) ​​is added to it and retained in the final report.

[0314] If the third verification factor S final <Set threshold T> passThen, high-confidence evidence (i.e., core reference text that meets the third set condition) is extracted from the candidate reference text. Constraint instructions are constructed based on the core reference text. The constraint instructions are input into the constraint instruction model, and the model performs semantic calibration on the target expression text. After calibration, a secondary verification (Rewrite) process is triggered. The entire process of selecting evidence text, verifying logical consistency, and fusion weights is repeated until the third verification factor meets the set threshold.

[0315] As another optional embodiment of this application, a processing method provided in embodiment 8 of this application may include, but is not limited to, the following steps:

[0316] Step S301: Obtain the target expression text; the target expression text is obtained based on the information expression text generated by the intelligent agent.

[0317] For a detailed description of step S301, please refer to the relevant description of step S101 in Example 1, which will not be repeated here.

[0318] Step S302: Select multiple candidate reference texts from the reference sources that match the target expression text.

[0319] The candidate citation text is of type 1 and type 2; the candidate citation text of type 1 is not publicly disclosed; the candidate citation text of type 2 is publicly disclosed.

[0320] Step S303: Determine at least one of the first verification factor and the second verification factor for each of the candidate reference texts; the first verification factor characterizes the degree of semantic consistency between the candidate reference text and the target expression text; the second verification factor characterizes the degree of credibility of the numerical matching between the candidate reference text and the target expression text.

[0321] Step S304: Select at least one target reference text from the plurality of candidate reference texts based on at least one of the first verification factor and the second verification factor.

[0322] Step S305: Determine the matching value of the target reference text based on at least one of the first verification factor and the second verification factor of the target reference text.

[0323] For a detailed description of steps S302-S305, please refer to the relevant description of steps S21-S24 in Example 3, which will not be repeated here.

[0324] Step S306: Select a comparison reference text from the plurality of candidate reference texts; the comparison reference text is of a different type from the target reference text, and at least one of the first verification factor and the second verification factor satisfies the second set condition.

[0325] Step S307: Based on the target reference text and the reference text belonging to the first type in the comparison reference text, determine a first weighting factor; the first weighting factor characterizes the importance of the reference text belonging to the first type in the target reference text and the comparison reference text.

[0326] Step S308: Based on the first weighting factor, process at least one of the first verification factor and the second verification factor of the target reference text and the comparison reference text to obtain the third verification factor of the target reference text.

[0327] Step S309: If the third verification factor of the target reference text meets the set threshold, the third verification factor is used as the matching value, and the matching value and corresponding reference identifier of each target reference text corresponding to the information expression text are displayed.

[0328] For a detailed description of steps S306-S309, please refer to the relevant description of steps S51-S54 in Example 6, which will not be repeated here.

[0329] Step S310: If the third verification factor of the target reference text does not meet the set threshold, select conflicting reference text pairs with mismatched information from the plurality of candidate reference texts.

[0330] In this embodiment, different types of cited text pairs (i.e., one is of the first type (i.e., internally non-public) and the other is of the second type (i.e., externally public)) can be selected from the plurality of candidate cited texts. Such conflicts are more valuable for reference and can reflect the information bias caused by the difference in the authority of cited texts from different sources.

[0331] If there are no conflicting text pairs of different types, you can choose text pairs of the same type but with different core information (such as two internal, non-publicly cited texts that describe the same data differently).

[0332] Step S311: Generate conflict annotation information for the conflicting reference text pair; the conflict annotation information is used to identify at least one of the following: mismatched information in the conflicting reference text pair, text in the conflicting reference text pair that is referenced by the target expression text, and recommended reference text in the conflicting reference text pair corresponding to the target expression text.

[0333] In this embodiment, the mismatched information in the conflicting reference text pair may include, but is not limited to, at least one of the following:

[0334] Numerical conflicts. For example, a label might read, "Conflict point: Text A (reference D1) states 'DRAM contract prices will increase by 15% in the first quarter of 2025,' while text B (reference D2) states 'DRAM contract prices will increase by 10% in the first quarter of 2025,' a numerical discrepancy of 5%, exceeding the reasonable range." Conflict labeling information clearly identifies the references of the two texts, the specific mismatched numerical values, and the magnitude of the discrepancy.

[0335] Semantic conflict. For example, annotate "Conflict point: Text A (citation identifier D3) states 'AI demand continues to rise,' while text B (citation identifier D4) states 'AI demand is gradually declining,' the semantics are completely opposite." Conflict annotation information can clearly identify the citation identifiers of two texts and the specific mismatched semantic viewpoints.

[0336] By identifying the texts referenced by the target text in the conflicting text pair, it is possible to identify which one (or two) of the referenced texts in the conflicting text pair is directly referenced by the current target text. This clarifies the direct cause of the target text's insufficient credibility (failure of the third verification factor). That is, the relevant statements in the target text may originate from one of the conflicting text pairs, and this text conflicts with the other text, resulting in a low credibility of the target text.

[0337] For example, the annotation could be: "Text referenced by the target text: Reference identifier D1 (text A), the phrase 'DRAM contract price increased by 15% in the first quarter of 2025' in the target text originates from this text." Through conflict annotation information, the specific text (reference identifier + text abbreviation) referenced by the target text in the conflicting text pair is clearly identified, along with the specific content of the reference.

[0338] Both texts in a conflicting text pair have a certain degree of credibility, but there is information bias. By identifying the recommended citation text corresponding to the target expression text in the conflicting citation text pair, it is possible to clearly recommend which text is more suitable as the citation basis for the target expression text, avoiding the subjectivity of manual judgment and improving optimization efficiency.

[0339] For example, the annotation could read, "Recommended cited text: Citation identifier D1 (text A), Recommendation reason: It belongs to the first type of internal non-public text, with a first verification factor of 0.92, a second verification factor of 0.89, and an overall credibility higher than text B (D2)." Conflict annotation information clearly identifies the recommended cited text (identifier + abbreviation) and the reason for the recommendation.

[0340] Step S312: Display the reference identifier, matching value, and conflict annotation information of each referenced text in the conflicting reference text pair.

[0341] In this embodiment, the conflict annotation information clearly presents the core of the conflict, the source of the target text's citation, and the recommended citation direction. This eliminates the need to compare and filter conflict information one by one and trace citation associations, significantly reducing verification costs and improving verification efficiency and accuracy. The intuitive display method can quickly present conflict details, related citations, and recommendations, clarifying the subsequent optimization direction of the target text. This not only makes up for the limitations of automated correction scenarios but also provides clear and practical guidance for the optimization of the target text, further improving the verification system for information expression text generated by the intelligent agent. It takes into account both the relevance and practicality of the verification, ensuring that the output information expression text can meet the credibility requirements of the business scenario.

[0342] As another optional embodiment of this application, this is a flowchart illustrating a processing method provided in Embodiment 9 of this application. This embodiment is mainly an implementation of step S103 in Embodiment 1, and may specifically include, but is not limited to, the following steps:

[0343] Step S61: Sort the target reference texts corresponding to the information expression text according to the matching value to generate a sorted reference text list.

[0344] In this embodiment, a descending sorting method can be used to sort the target reference texts corresponding to the information expression text from high to low according to the matching value; if there are two or more target reference texts with completely identical matching values, a secondary sorting can be performed according to the retrieval source priority (first type, second type) of the reference texts; if the source types are also consistent, the sorting can be performed according to the timeliness of the referenced documents (newest release priority). The specific secondary sorting rules can be flexibly set according to actual business needs, and this application does not impose specific limitations.

[0345] After sorting, all target cited texts are organized according to the sorting results to form a sorted list of cited texts. This list can contain the core related information of each target cited text (at least including the citation identifier and matching value), and can be supplemented with auxiliary information such as the source type and publication time of the cited text as needed to ensure that the list information is complete and well-organized.

[0346] Step S62: Display the sorted list of referenced texts, and display the reference identifier and matching value of each target referenced text in the sorted list of referenced texts.

[0347] In this embodiment, a clear display format such as a table or ordered list can be used to clearly distinguish the information of each target reference text.

[0348] Additional information such as the source type, publication time, and core summary of the cited text can be added to the list as needed, further enhancing its usability.

[0349] In this embodiment, the strength of support for the target expression text by each target reference text can be clearly distinguished through matching value sorting, realizing the structured sorting of target reference texts and avoiding confusion of relevance caused by disordered reference texts. At the same time, by intuitively displaying the sorted list and corresponding reference identifiers and matching values, the core relevance information of each target reference text can be quickly presented, clarifying the traceability basis and supporting capabilities of each reference text. This ensures the traceability and regularity of reference texts and improves the efficiency of obtaining relevant information of target reference texts. It provides orderly and clear guidance for the credibility verification, subsequent optimization and final output of target expression texts, further improving the display and verification system of information expression texts generated by intelligent agents, and meeting the needs of standardized and visualized use of reference texts in actual business scenarios.

[0350] The processing apparatus provided in this application will be described below. The processing apparatus described below can be referred to in correspondence with the processing method described above.

[0351] The processing device includes:

[0352] The acquisition module is used to acquire the target expression text; the target expression text is obtained based on the information expression text generated by the intelligent agent.

[0353] The determining module is used to determine the matching value of each target reference text corresponding to the information expression text based on the target expression text; the matching value is obtained by comparing the target expression text and the target reference text; the target reference text is a reference for the agent to generate the information expression text.

[0354] The first display module is used to display the matching value and corresponding reference identifier of each target reference text corresponding to the information expression text.

[0355] The acquisition module can be used specifically for:

[0356] Obtain the information expression text generated by the intelligent agent;

[0357] Identify at least one of a first factual factor and a second factual factor in the information expression text; the first factual factor characterizes the frequency of occurrence and semantic certainty of assertive words in the information expression text; the second factual factor characterizes the completeness of the subject-verb-object structure in the information expression text.

[0358] If the first factual factor and the second factual factor satisfy the first set condition, extract the noun entity that matches the demonstrative pronoun in the information expression text from the preceding text of the information expression text, replace the demonstrative pronoun with the noun entity, and obtain the target expression text.

[0359] The module can be used specifically for:

[0360] Select multiple candidate reference texts that match the target expression text from the reference source;

[0361] Determine at least one of a first verification factor and a second verification factor for each of the candidate cited texts; the first verification factor characterizes the degree of semantic consistency between the candidate cited text and the target expressed text; the second verification factor characterizes the degree of credibility of the numerical matching between the candidate cited text and the target expressed text.

[0362] Based on at least one of the first verification factor and the second verification factor, at least one target reference text is selected from the plurality of candidate reference texts;

[0363] The matching value of the target reference text is determined based on at least one of the first verification factor and the second verification factor of the target reference text.

[0364] The determining module determines the first verification factor for each of the candidate reference texts, which may specifically include:

[0365] Extract evidence text fragments related to the target expression text from the candidate cited text;

[0366] The evidence text fragment and the target expression text are concatenated to obtain the input text;

[0367] The input text is processed based on the target model to obtain confidence values ​​corresponding to various logical relationships; the various logical relationships include implication, neutrality, and contradiction.

[0368] The confidence value corresponding to the implied relationship is determined as the first verification factor.

[0369] The determining module determines a second verification factor for each of the candidate reference texts, which may specifically include:

[0370] Extract the predicted values ​​corresponding to the quantitative indicators from the target text;

[0371] Extract the true value corresponding to the quantitative indicator from the first source of reference; the credibility of the first source of reference meets the set credibility threshold.

[0372] Determine the relative error rate between the predicted value and the true value; the relative error rate characterizes the degree of deviation between the predicted value and the true value;

[0373] Obtain the target coefficient corresponding to the target business scenario to which the target expression text belongs; the target coefficient represents the reasonable error range between the predicted value and the actual value allowed under the target business scenario;

[0374] The second verification factor is determined based on the target coefficient and the relative error rate.

[0375] The plurality of candidate reference texts may include: a first type of candidate reference text and a second type of candidate reference text; the authenticity and credibility of the first type of candidate reference text is higher than that of the second type of candidate reference text.

[0376] The first display module can be specifically used for:

[0377] Select a comparison reference text from the plurality of candidate reference texts; the comparison reference text is of a different type from the target reference text, and at least one of the first verification factor and the second verification factor satisfies the second set condition;

[0378] A first weighting factor is determined based on the target reference text and the reference texts in the comparison reference texts that belong to the first type; the first weighting factor represents the importance of the target reference text and the reference texts in the comparison reference texts that belong to the first type.

[0379] Based on the first weighting factor, at least one of the first verification factor and the second verification factor of the target reference text and the comparison reference text is processed to obtain the third verification factor of the target reference text.

[0380] If the third verification factor of the target reference text meets the set threshold, the third verification factor is used as the matching value, and the matching value and corresponding reference identifier of each target reference text corresponding to the information expression text are displayed.

[0381] The processing apparatus may also include:

[0382] The first selection module is used to select, from the plurality of candidate reference texts, at least one of the first verification factor and the second verification factor that satisfies the third set condition if the third verification factor of the target reference text does not meet the set threshold.

[0383] The correction module is used to correct the information expression text based on the core reference text.

[0384] The processing apparatus may also include at least one of the following:

[0385] The second selection module is used to select conflicting reference text pairs with mismatched information from the plurality of candidate reference texts if the third verification factor of the target reference text does not meet the set threshold.

[0386] A generation module is used to generate conflict annotation information for the conflicting reference text pairs; the conflict annotation information is used to identify at least one of the following: mismatched information in the conflicting reference text pairs, text in the conflicting reference text pairs that is referenced by the target expression text, and recommended reference text in the conflicting reference text pairs that corresponds to the target expression text.

[0387] The second display module is used to display the reference identifier, matching value, and conflict annotation information of each referenced text in the conflicted reference text pair.

[0388] The first display module can be specifically used for:

[0389] Sort the target reference texts corresponding to the information expression text according to the matching value to generate a sorted reference text list;

[0390] The sorted list of referenced texts is displayed, and the reference identifier and matching value of each target referenced text are displayed in the sorted list of referenced texts.

[0391] It should also be noted that the device embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate, and the components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to achieve the purpose of this embodiment according to actual needs. In addition, in the device embodiment drawings provided in this application, the connection relationship between modules indicates that they have a communication connection, which can be implemented as one or more communication buses or signal lines.

[0392] Through the above description of the embodiments, those skilled in the art can clearly understand that this application can be implemented by means of software plus necessary general-purpose hardware, or it can be implemented by special-purpose hardware including application-specific integrated circuits, special-purpose CPUs, special-purpose memory, special-purpose components, etc. Generally, any function performed by a computer program can be easily implemented by corresponding hardware, and the specific hardware structure used to implement the same function can also be diverse, such as analog circuits, digital circuits, or special-purpose circuits. However, for this application, software program implementation is more often the preferred implementation method. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product is stored in a readable storage medium, such as a computer floppy disk, USB flash drive, mobile hard disk, ROM, RAM, magnetic disk, or optical disk, etc., and includes several instructions to cause a computer device (which may be a personal computer, training equipment, or network device, etc.) to execute the methods described in the various embodiments of this application.

[0393] In the above embodiments, implementation can be achieved, in whole or in part, through software, hardware, firmware, or any combination thereof. When implemented in software, it can be implemented, in whole or in part, as a computer program product.

[0394] The computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, all or part of the processes or functions described in the embodiments of this application are generated. The computer may be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions may be stored in a computer-readable storage medium or transmitted from one computer-readable storage medium to another. For example, the computer instructions may be transmitted from one website, computer, training device, or data center to another website, computer, training device, or data center via wired (e.g., coaxial cable, fiber optic, digital subscriber line (DSL)) or wireless (e.g., infrared, wireless, microwave, etc.) means. The computer-readable storage medium may be any available medium that a computer can store or a data storage device such as a training device or data center that integrates one or more available media. The available media may be magnetic media (e.g., floppy disks, hard disks, magnetic tapes), optical media (e.g., DVDs), or semiconductor media (e.g., solid-state drives (SSDs)).

Claims

1. A processing method, comprising: Obtain the target text; The target expression text is obtained based on the information expression text generated by the intelligent agent; Based on the target expression text, the matching value of each target reference text corresponding to the information expression text is determined; the matching value is obtained by comparing the target expression text and the target reference text; the target reference text is a reference for the agent to generate the information expression text; Displays the matching value and corresponding reference identifier of each target reference text corresponding to the information expression text.

2. The processing method according to claim 1, wherein obtaining the target expression text includes: Obtain the information expression text generated by the intelligent agent; Determine at least one of a first factual factor and a second factual factor in the information expression text; The first factual factor characterizes the frequency of occurrence and semantic certainty of assertive words in the information expression text; The second factual factor characterizes the completeness of the subject-verb-object structure in the information expression text; If the first factual factor and the second factual factor satisfy the first set condition, extract the noun entity that matches the demonstrative pronoun in the information expression text from the preceding text of the information expression text, replace the demonstrative pronoun with the noun entity, and obtain the target expression text.

3. The processing method according to claim 1, wherein determining the matching value of each target reference text corresponding to the information expression text based on the target expression text includes: Select multiple candidate reference texts that match the target expression text from the reference source; Determine at least one of a first verification factor and a second verification factor for each of the candidate cited texts; the first verification factor characterizes the degree of semantic consistency between the candidate cited text and the target expressed text; the second verification factor characterizes the degree of credibility of the numerical matching between the candidate cited text and the target expressed text. Based on at least one of the first verification factor and the second verification factor, at least one target reference text is selected from the plurality of candidate reference texts; The matching value of the target reference text is determined based on at least one of the first verification factor and the second verification factor of the target reference text.

4. The processing method according to claim 3, wherein determining the first verification factor for each of the candidate cited texts includes: Extract evidence text fragments related to the target expression text from the candidate cited text; The evidence text fragment and the target expression text are concatenated to obtain the input text; The input text is processed based on the target model to obtain confidence values ​​corresponding to various logical relationships; The various logical relationships include implication, neutrality, and contradiction. The confidence value corresponding to the implied relationship is determined as the first verification factor.

5. The processing method according to claim 3, wherein determining the second verification factor for each of the candidate cited texts includes: Extract the predicted values ​​corresponding to the quantitative indicators from the target text; Extract the true value corresponding to the quantitative indicator from the first source of reference; The credibility of the first source of reference meets the set credibility threshold. Determine the relative error rate between the predicted value and the true value; the relative error rate characterizes the degree of deviation between the predicted value and the true value; Obtain the target coefficient corresponding to the target business scenario to which the target expression text belongs; The target coefficient represents the reasonable error range between the predicted value and the actual value allowed under the target business scenario; The second verification factor is determined based on the target coefficient and the relative error rate.

6. The processing method according to claim 3, wherein the plurality of candidate reference texts includes: First type of candidate cited text and second type of candidate cited text; the authenticity and credibility of the first type of candidate cited text is higher than that of the second type of candidate cited text; The step of displaying the matching values ​​and corresponding reference identifiers of each target reference text corresponding to the information expression text includes: Select a comparison reference text from the plurality of candidate reference texts; the comparison reference text is of a different type from the target reference text, and at least one of the first verification factor and the second verification factor satisfies the second set condition; A first weighting factor is determined based on the target reference text and the reference texts in the comparison reference texts that belong to the first type; the first weighting factor represents the importance of the target reference text and the reference texts in the comparison reference texts that belong to the first type. Based on the first weighting factor, at least one of the first verification factor and the second verification factor of the target reference text and the comparison reference text is processed to obtain the third verification factor of the target reference text. If the third verification factor of the target reference text meets the set threshold, the third verification factor is used as the matching value, and the third verification factor and the corresponding reference identifier of each target reference text corresponding to the information expression text are displayed.

7. The processing method according to claim 6, further comprising, after processing at least one of the first verification factor and the second verification factor of the target cited text and the comparison cited text based on the first weighting factor to obtain the third verification factor of the target cited text: If the third verification factor of the target reference text does not meet the set threshold, select a core reference text from the plurality of candidate reference texts that meets at least one of the first verification factor and the second verification factor; The information expression text is revised based on the core reference text.

8. The processing method according to claim 6, further comprising at least one of the following: If the third verification factor of the target reference text does not meet the set threshold, select conflicting reference text pairs with mismatched information from the plurality of candidate reference texts; Generate conflict annotation information for the conflicting reference text pairs; the conflict annotation information is used to identify at least one of the following: mismatched information in the conflicting reference text pairs, text in the conflicting reference text pairs that is referenced by the target expression text, and recommended reference text in the conflicting reference text pairs that corresponds to the target expression text. Displays the reference identifier, matching value, and conflict annotation information of each referenced text in the conflicting reference text pair.

9. The processing method according to claim 1, wherein displaying the matching value and corresponding citation identifier of each target reference text corresponding to the information expression text includes: Sort the target reference texts corresponding to the information expression text according to the matching value to generate a sorted reference text list; The sorted list of referenced texts is displayed, and the reference identifier and matching value of each target referenced text are displayed in the sorted list of referenced texts.

10. A processing apparatus, comprising: The acquisition module is used to acquire the target text. The target expression text is obtained based on the information expression text generated by the intelligent agent; The determining module is used to determine the matching value of each target reference text corresponding to the information expression text based on the target expression text; the matching value is obtained by comparing the target expression text and the target reference text; the target reference text is a reference for the agent to generate the information expression text; The display module is used to display the matching value and corresponding reference identifier of each target reference text corresponding to the information expression text.

Images