Rag anti-illusion method and system based on credibility grading and traceability checking

Abstract

The application discloses a RAG anti-illusion method and system based on credibility grading and traceability verification, which comprises the following steps: obtaining a query text of a target enterprise personnel; performing semantic analysis on the query text to extract a plurality of candidate knowledge segments from an enterprise policy knowledge base, and extracting multi-dimensional credible data from each candidate knowledge segment; determining the credibility level and state mark of each candidate knowledge segment based on the multi-dimensional credible data; generating the content hash corresponding to each candidate knowledge segment, and associating the content hash corresponding to each candidate knowledge segment with the evidence timestamp to generate the verifiable credential corresponding to each candidate knowledge segment; based on each verifiable credential, performing majority verification on the corresponding candidate knowledge segment, and selecting the candidate knowledge segment which passes the majority verification, has a credibility level meeting a preset level and has a state mark in a valid state as a credible knowledge segment; performing retrieval enhancement generation processing on the credible knowledge segment and the query text to generate an interpretation answer.

Description

Technical Field

[0001] This application relates to the field of text processing technology, and more specifically, to a RAG anti-hallucination method and system based on credibility grading and source tracing verification. Background Technology

[0002] In enterprise management scenarios, compliance personnel frequently need to consult various policy documents to obtain accurate interpretations and implementation guidelines. With the widespread adoption of large language models and retrieval-enhanced generation technologies, enterprises are gradually introducing automated question-and-answer tools to assist in policy interpretation, thereby improving response efficiency and reducing labor costs. However, policy interpretation demands extremely high levels of accuracy, timeliness, and traceability of information. Any answer inconsistent with the original text may lead to compliance risks, thus creating an urgent need for credibility verification and source tracing of generated content.

[0003] Existing solutions typically employ semantic retrieval-based augmentation methods. These methods create vector indexes on policy texts, retrieve relevant paragraphs based on semantic similarity, and then use this as contextual input to a language model to generate interpretive responses. However, this approach relies solely on semantic matching to retrieve text, lacking assessment of the consistency, version timeliness, and historical feedback of knowledge fragments within the policy framework. This can lead to outdated or contradictory policy provisions being adopted. Furthermore, the generated responses do not provide reliable source documentation for cited provisions, making it difficult for users to quickly verify the authenticity and completeness of the content, thus failing to meet the rigor requirements of policy interpretation scenarios. Summary of the Invention

[0004] The main purpose of this application is to provide a RAG anti-hallucination method and system based on credibility grading and source tracing verification, so as to solve the problem in the prior art that the lack of knowledge timeliness verification and source tracing leads to outdated or contradictory terms in the generated answers and the inability to verify the authenticity of the content.

[0005] To achieve the above objectives, the first aspect of this application proposes a RAG anti-hallucination method based on credibility grading and source tracing verification, including:

[0006] Obtain the query text of personnel from the target company;

[0007] Semantic analysis is performed on the query text to extract multiple candidate knowledge fragments from the enterprise policy knowledge base, and multidimensional credible data is extracted from each candidate knowledge fragment.

[0008] Based on the knowledge association data, semantic annotation data, and feedback correction data in the multidimensional trusted data, the trust level of each candidate knowledge fragment is determined, and a status label for each candidate knowledge fragment is generated.

[0009] Generate the content hash corresponding to each candidate knowledge fragment, and associate the content hash corresponding to each candidate knowledge fragment with the evidence storage timestamp to generate a verifiable credential corresponding to each candidate knowledge fragment.

[0010] Based on each verifiable credential, the corresponding candidate knowledge fragments are subjected to majority verification, and the candidate knowledge fragments that pass the majority verification, meet the preset credibility level, and are marked as valid are selected as credible knowledge fragments.

[0011] The credible knowledge fragment and the query text are subjected to retrieval enhancement and generation processes to generate an interpretation answer.

[0012] Secondly, this application provides a RAG anti-hallucination system based on credibility grading and source tracing verification, including:

[0013] The acquisition module is used to acquire the query text of personnel from the target company;

[0014] The analysis module is used to perform semantic analysis on the query text to extract multiple candidate knowledge fragments from the enterprise policy knowledge base and extract multidimensional credible data from each candidate knowledge fragment.

[0015] The determination module is used to determine the credibility level of each candidate knowledge fragment based on the knowledge association data, semantic annotation data and feedback correction data in the multidimensional trusted data, and to generate a status label for each candidate knowledge fragment.

[0016] The association module is used to generate the content hash corresponding to each candidate knowledge fragment, and associate the content hash corresponding to each candidate knowledge fragment with the evidence storage timestamp to generate a verifiable credential corresponding to each candidate knowledge fragment.

[0017] The verification module is used to perform majority verification on the corresponding candidate knowledge fragments based on each verifiable credential, and select the candidate knowledge fragments that pass the majority verification, meet the preset credibility level, and are marked as valid as credible knowledge fragments.

[0018] The generation module is used to perform retrieval enhancement generation processing on the credible knowledge fragment and the query text to generate an interpretation answer.

[0019] Thirdly, this application provides a computing device, including a processing component and a storage component; the storage component stores one or more computer instructions; the one or more computer instructions are invoked and executed by the processing component to implement the RAG anti-hallucination method based on credibility grading and traceability verification.

[0020] Fourthly, this application provides a computer-readable storage medium storing a computer program, which, when executed by a computer, implements the RAG anti-hallucination method based on credibility grading and traceability verification.

[0021] The technical solutions provided by the embodiments of this application may include the following beneficial effects:

[0022] In this application, query texts from personnel of the target enterprise are obtained, and semantic analysis is performed on the query texts to obtain multiple search conditions. Based on each search condition, multiple candidate knowledge fragments are extracted from the enterprise's policy knowledge base, and multidimensional credible data is extracted from each candidate knowledge fragment. This ensures that subsequent credibility assessment and source tracing verification are only conducted on knowledge fragments relevant to the query. Based on knowledge association data, semantic annotation data, and feedback correction data in the multidimensional credible data, the credibility level of each candidate knowledge fragment is determined, and a status marker for each candidate knowledge fragment is generated. This comprehensively assesses the structural consistency, content standardization, and version timeliness of the knowledge fragments within the policy system, effectively distinguishing between currently valid and outdated provisions. A content hash corresponding to each candidate knowledge fragment is generated, and the content hash corresponding to each candidate knowledge fragment is associated with a storage timestamp to generate a verifiable credential for each candidate knowledge fragment, assigning each knowledge fragment an independently verifiable integrity certificate. Based on verifiable credentials, the corresponding candidate knowledge fragments are validated to obtain the validated candidate knowledge fragments. Candidate knowledge fragments that meet the preset credibility level and are marked as valid are selected as credible knowledge fragments, ensuring that the knowledge fragments entering the generation stage undergo double filtering in terms of content authenticity, timeliness, and authority. The credible knowledge fragments and query text are then subjected to retrieval enhancement processing to generate interpretive responses. The final output interpretive content strictly adheres to the validated, highly credible knowledge fragments, and the respondent includes the issuing authority and issuance date of the cited provisions, facilitating direct tracing of the policy's source and version for users. Attached Figure Description

[0023] The accompanying drawings, which form part of this application, are used to provide a further understanding of the application and to make other features, objects, and advantages of the application more apparent. The illustrative embodiments and descriptions of this application are used to explain the application and do not constitute an undue limitation of the application. In the drawings:

[0024] Figure 1 A flowchart of the RAG anti-hallucination method based on credibility grading and source tracing verification provided in this application;

[0025] Figure 2 A schematic diagram of the RAG anti-hallucination system based on credibility grading and traceability verification provided in this application. Detailed Implementation

[0026] To enable those skilled in the art to better understand the present application, the technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present application, and not all embodiments. Based on the embodiments in the present application, all other embodiments obtained by those of ordinary skill in the art without creative effort should fall within the scope of protection of the present application.

[0027] It should be noted that the terms "first," "second," etc., in the specification, claims, and accompanying drawings of this application are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate for the embodiments of this application described herein. Furthermore, the terms "comprising" and "having," and any variations thereof, are intended to cover non-exclusive inclusion; for example, a process, method, system, product, or apparatus that comprises a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to such processes, methods, products, or apparatus.

[0028] In this application, the terms "upper," "lower," "left," "right," "front," "rear," "top," "bottom," "inner," "outer," "middle," "vertical," "horizontal," "lateral," and "longitudinal" indicate the orientation or positional relationship based on the orientation or positional relationship shown in the accompanying drawings. These terms are primarily for the purpose of better describing this application and its embodiments, and are not intended to limit the indicated device, element, or component to having a specific orientation, or to be constructed and operated in a specific orientation.

[0029] Furthermore, in addition to indicating location or positional relationship, some of the aforementioned terms may also have other meanings. For example, the term "above" may also be used in some cases to indicate a certain dependency or connection relationship. Those skilled in the art can understand the specific meaning of these terms in this application based on the specific circumstances.

[0030] Furthermore, the terms "installation," "setup," "equipped with," "connection," "linked," and "socketing" should be interpreted broadly. For example, "connection" can be a fixed connection, a detachable connection, or an integral structure; it can be a mechanical connection or an electrical connection; it can be a direct connection or an indirect connection through an intermediate medium, or an internal connection between two devices, components, or parts. Those skilled in the art can understand the specific meaning of the above terms in this application according to the specific circumstances.

[0031] The core of this application is to provide a RAG anti-hallucination method based on credibility grading and source tracing verification. A flowchart of one specific implementation is shown below. Figure 1 As shown, the method includes:

[0032] Step 101: Obtain the query text of the target company's personnel.

[0033] Step 102: Perform semantic analysis on the query text to extract multiple candidate knowledge fragments from the enterprise policy knowledge base, and extract multidimensional credible data from each candidate knowledge fragment.

[0034] Step 103: Based on the knowledge association data, semantic annotation data and feedback correction data in the multidimensional trusted data, determine the credibility level of each candidate knowledge fragment and generate the status label of each candidate knowledge fragment.

[0035] Step 104: Generate the content hash corresponding to each candidate knowledge fragment, and associate the content hash corresponding to each candidate knowledge fragment with the evidence storage timestamp to generate a verifiable credential corresponding to each candidate knowledge fragment.

[0036] Step 105: Based on each verifiable credential, perform majority validation on the corresponding candidate knowledge fragments, and select the candidate knowledge fragments that pass the majority validation, meet the preset credibility level, and are marked as valid as credible knowledge fragments.

[0037] Step 106: Perform retrieval enhancement processing on the credible knowledge fragments and query text to generate interpretive answers.

[0038] In step 101 above, the target company personnel refers to personnel within the company who are responsible for policy compliance review, legal consultation, or business approval. For example, the target company personnel may be reviewers, legal specialists, or managers responsible for approval of specific business lines in the company's compliance department.

[0039] The query text refers to the text content submitted by the target company's personnel in natural language to express their need for policy interpretation. It can be obtained by receiving characters entered by keyboard through the text input box on the terminal device, receiving speech and converting it into text through an audio acquisition device, extracting text messages from the conversation records of instant messaging tools, or extracting policy consultation remarks fields from the approval documents circulating in the business system.

[0040] In this embodiment of the application, one way to obtain the query text of the target enterprise personnel is to obtain the policy questions that the target enterprise personnel need to interpret, which are entered by the target enterprise personnel in the input interface of the enterprise's internal policy Q&A platform. For example, the obtained query text is "What is the R&D expense deduction ratio entered by the target enterprise personnel in the search box"; or, the query text is obtained by obtaining the policy consultation needs filled in by the target enterprise personnel in the remarks column of the approval system when they encounter policy application questions in the approval process.

[0041] In step 102 above, firstly, semantic analysis is performed on the query text to extract multiple candidate knowledge fragments from the enterprise policy knowledge base. One implementation method for extracting multidimensional credible data from each candidate knowledge fragment is as follows: read the reference relationship records and hierarchical relationship records between each candidate knowledge fragment from the metadata of the enterprise policy knowledge base, and integrate the reference relationship records and hierarchical relationship records into knowledge association data; obtain the semantic annotation text corresponding to each candidate knowledge fragment from the pre-stored semantic annotation library as semantic annotation data; read the previous correction records corresponding to each candidate knowledge fragment from the historical operation log, extract the correction operation time, operation type and correction number of each correction record, and obtain feedback correction data.

[0042] The enterprise policy knowledge base refers to a database that stores all policy documents involved in the enterprise and the policy knowledge fragments formed after structured decomposition. Each policy document is pre-divided into multiple independently searchable policy knowledge fragments according to the clause hierarchy. Each policy knowledge fragment is associated with a fragment identifier, issuing authority identifier, issuing time identifier, and document number. The fragment identifier is a unique number for each policy knowledge fragment in the enterprise policy knowledge base.

[0043] Candidate knowledge fragments refer to knowledge fragments that are semantically related to the query text and are selected from the enterprise policy knowledge base through semantic matching.

[0044] Multidimensional credible data refers to the basic data collected for each candidate knowledge fragment based on multiple credibility assessment dimensions. This includes knowledge association data, semantic annotation data, and feedback correction data for each candidate knowledge fragment. Knowledge association data reflects the structural relationships between different candidate knowledge fragments, including records of citation relationships and hierarchical relationships in policy documents. Semantic annotation data records the annotation information recorded after semantic analysis of each candidate knowledge fragment, including the semantic annotation text corresponding to each fragment. This text is a summary description of the policy points and scope of application involved in the corresponding knowledge fragment. Feedback correction data records the corrections made to each candidate knowledge fragment during historical interpretation, including the number of corrections, correction time, and correction type for each fragment.

[0045] The step "perform semantic analysis on the query text to extract multiple candidate knowledge fragments from the enterprise policy knowledge base" specifically includes the following steps:

[0046] Step 201: Perform word segmentation and dependency parsing on the query text to identify multiple target entity names with dependency relationships.

[0047] Step 202: Based on the preset name mapping table and the preset synonym mapping table, determine the standard name and synonym corresponding to each target entity name.

[0048] Step 203: Convert each target entity name and its corresponding standard name and synonym into a semantic vector. Calculate the similarity between each semantic vector and the semantic vector corresponding to each policy knowledge fragment in the enterprise policy knowledge base to obtain the comprehensive similarity of each policy knowledge fragment. Select policy knowledge fragments with a comprehensive similarity exceeding a preset threshold as candidate knowledge fragments.

[0049] In step 201 above, the target entity name refers to the entity words in the query text that represent specific objects, behaviors, or attributes in the policy field, including enterprise type names, preferential type names, industry names, etc.

[0050] Dependency relations refer to the syntactic relationships, such as modification, domination, or coordination, between two words in a query text, where the semantic understanding of one word depends on the other.

[0051] In this embodiment of the application, the query text is segmented into words to divide the continuous string of the query text into multiple independent words to obtain a word sequence; the syntactic relationship of each adjacent word pair in the word sequence is determined to determine whether there is a modification relationship, subject-predicate relationship or verb-object relationship between each word pair; two words that have any of the above syntactic relationships are used as target entity names, and the dependency relationship between the two target entity names is recorded.

[0052] For example, when the query text is "the additional deduction ratio for R&D expenses of high-tech enterprises", word segmentation yields "high-tech enterprises", "R&D expenses", "additional deduction", and "ratio". It is determined that there is a modification relationship between "high-tech enterprises" and "R&D expenses", a subject-predicate relationship between "R&D expenses" and "additional deduction", and a modification relationship between "additional deduction" and "ratio". Therefore, "high-tech enterprises", "R&D expenses", "additional deduction", and "ratio" are all used as target entity names, and it is recorded that there is a dependency relationship between "high-tech enterprises" and "R&D expenses", a dependency relationship between "R&D expenses" and "additional deduction", and a dependency relationship between "additional deduction" and "ratio".

[0053] In step 202 above, the preset name mapping table refers to a table that maps various expressions of entity names to a unified standard name. The preset name mapping table is constructed based on the standard names associated with each policy knowledge fragment in the enterprise policy knowledge base and the different expressions appearing in the policy text. The standard name refers to the unified naming format determined after standardization of each entity name.

[0054] The pre-built synonym mapping table refers to a pre-constructed mapping table that stores the correspondence between standard names and synonyms. The pre-built synonym mapping table is constructed based on the standard names in the enterprise policy knowledge base and the synonyms that appear in the policy text and interpretation text.

[0055] Synonyms refer to other names that have the same meaning as the standard name but are expressed differently in words, including various alternative names, abbreviations, or former expressions of the standard name.

[0056] In this embodiment, each target entity name is compared one by one with each entity name entry in a preset name mapping table. When a target entity name matches an entity name in a certain entry in the preset name mapping table, the standard name corresponding to that entry is obtained as the standard name corresponding to the target entity name. For each standard name, all synonyms associated with the standard name are searched in a preset synonym mapping table. For example, the target entity name "High-tech Enterprise" is mapped to the standard name "High-tech Enterprise", and synonyms include "Technology-based Enterprise" and "High-tech Certified Enterprise", etc.

[0057] In step 203 above, a semantic vector refers to a set of ordered numerical values ​​representing the meaning of words. Each numerical value corresponds to the measurement value of a word in a certain semantic feature dimension. By comparing the degree of difference between two sets of semantic vectors, the degree of similarity between two words in meaning can be measured.

[0058] The overall similarity score refers to the overall matching degree value obtained by combining the similarity scores of each semantic vector and the corresponding semantic vector in the knowledge segment after calculating the similarity between each semantic vector and the corresponding semantic vector.

[0059] The preset threshold refers to a pre-set value used to determine whether the overall similarity meets the matching requirements. The preset threshold can be set according to the size of the enterprise policy knowledge base, the average length of policy knowledge fragments, or the similarity statistics of historical queries. This application does not make specific limitations on this.

[0060] In this embodiment of the application, the target entity name, the corresponding standard name, and each synonym are input into a pre-trained semantic coding model to obtain the entity semantic vector corresponding to each target entity name, the standard semantic vector corresponding to each standard name, and the synonym semantic vector corresponding to each synonym.

[0061] The similarity between the entity semantic vector and the entity semantic vectors pre-stored in the enterprise policy knowledge base is calculated to obtain a similarity score for the entity semantic vector. The similarity between the standard semantic vector and the standard semantic vectors pre-stored in each policy knowledge fragment is also calculated to obtain a similarity score for the standard semantic vector. The similarity between the synonymous semantic vector and the synonymous semantic vectors pre-stored in each policy knowledge fragment is further calculated to obtain a similarity score for the synonymous semantic vector. The similarity can be calculated using a spatial distance metric, specifically by dividing the dot product of the two semantic vectors by the product of their magnitudes. The quotient is used as the similarity score, ranging from 0 to 1, with values ​​closer to 1 indicating higher similarity. Other algorithms can also be used. The similarity scores of the three semantic vectors are weighted and summed according to preset weights to obtain a comprehensive similarity score for each policy knowledge fragment. These preset weights can be set according to the importance of the word type corresponding to each semantic vector in the query intent; this embodiment does not specifically limit this. Policy knowledge fragments with a comprehensive similarity exceeding a preset threshold are selected as candidate knowledge fragments.

[0062] The embodiments of this application improve the structured matching degree between candidate knowledge fragments and query intent, achieve comprehensive coverage of different expressions of the same policy concept, improve the coverage of candidate knowledge fragments and the retrieval anti-interference ability; and improve the accuracy of candidate knowledge fragment selection by comprehensively evaluating the overall semantic matching degree.

[0063] Step 103 above specifically includes the following steps:

[0064] Step 301: Calculate the semantic similarity between each semantically annotated text and the corresponding candidate knowledge fragment in the semantically annotated data of the multidimensional trusted data. Using each candidate knowledge fragment as a node, establish the association edges between each node based on the knowledge association data in the multidimensional trusted data. Based on each node and its corresponding association edges and similarity, construct a confidence propagation network. The confidence of each node in the confidence propagation network is the semantic similarity.

[0065] Step 302: Perform confidence propagation in the confidence propagation network to obtain the basic confidence level corresponding to each node.

[0066] Step 303: Based on the number of corrections, correction time and correction type of each candidate knowledge fragment in the feedback correction data, determine the credibility correction amount corresponding to each candidate knowledge fragment. Based on the credibility correction amount corresponding to each candidate knowledge fragment, adjust the corresponding basic credibility level to obtain the credibility level of each candidate knowledge fragment.

[0067] Step 304: Based on the publication time identifier, effective time, expiration time, and obsolescence mark in the knowledge association data of each candidate knowledge fragment, generate a status mark for each candidate knowledge fragment. The status mark can be either valid, outdated, obsolescence, pending effectiveness, or expired.

[0068] In step 301 above, a node refers to mapping each candidate knowledge fragment to an independent processing unit. An association edge refers to the connection line established between two corresponding nodes when there is a reference relationship or hierarchical relationship between two candidate knowledge fragments in the knowledge association data.

[0069] Confidence level refers to the initial value of a node's credibility in a confidence propagation network. A confidence propagation network is an association structure composed of nodes and associated edges used to propagate and update confidence levels among nodes.

[0070] In this embodiment, a pre-trained semantic coding model is used to convert each semantically labeled text and its corresponding candidate knowledge fragment in the semantically labeled data into semantic vectors. The same method as in step 203 is used to calculate the semantic similarity between the semantic vector corresponding to each semantically labeled text and the semantic vector corresponding to the corresponding candidate knowledge fragment. Taking each candidate knowledge fragment as a node, an association edge is established between two nodes with a reference relationship or a hierarchical relationship based on the reference relationship record and hierarchical relationship record in the knowledge association data. The semantic similarity of each node is used as the confidence of that node, and a confidence propagation network is constructed based on each node, the association edge, and the confidence of each node.

[0071] For example, if there is a reference relationship between candidate knowledge fragment P1 and fragment P2, and a hierarchical relationship between candidate knowledge fragment P1 and fragment P3, then fragment P1, fragment P2, and fragment P3 are used as nodes, and association edges are established between P1 and P2 and between P1 and P3 respectively. The semantic similarity of P1 is used as the confidence of node P1, the semantic similarity of P2 is used as the confidence of node P2, and the semantic similarity of P3 is used as the confidence of node P3.

[0072] In step 302 above, the basic confidence level refers to the confidence level corresponding to the confidence value finally reached by each node after the confidence propagation iteration has stabilized.

[0073] In this embodiment, in the confidence propagation network, each node transmits its current confidence to neighboring nodes along each associated edge, and simultaneously receives the confidence from each neighboring node. According to preset node weights, each node calculates a weighted average of the confidences received from all neighboring nodes. Based on a decay coefficient, this weighted average is summed with the node's current confidence, and the sum is used as the updated confidence. That is, updated confidence = (weighted average × decay coefficient) + each node's current confidence × (1 - decay coefficient). The preset node weights can be determined based on the candidate knowledge corresponding to each neighboring node. The semantic similarity of the segments is set accordingly, and the attenuation coefficient can be set according to the convergence speed requirement of the confidence propagation network. The value of the attenuation coefficient is between 0 and 1, and this embodiment does not limit it. The above process of transmitting and updating confidence is repeated until the change amplitude of the confidence of each node is less than the preset stability judgment value or the preset iteration number limit is reached, so as to obtain the basic confidence level corresponding to each node. The above preset stability judgment value can be set according to the confidence level division accuracy requirement, and the preset iteration number can be set according to the total number of nodes in the confidence propagation network or the convergence speed, and this embodiment does not limit it.

[0074] For example, node P1 transmits its current confidence level to its neighboring nodes P2 and P3, and simultaneously receives the confidence levels transmitted by nodes P2 and P3. It then performs a weighted average of the confidence levels of nodes P2 and P3 according to preset node weights. Based on a set decay coefficient, it sums the weighted average with node P1's own current confidence level. The weighted sum is used as the updated confidence level of node P1. After multiple iterations, the confidence levels of each node tend to stabilize, and finally, the stabilized confidence levels are mapped to the basic confidence level of the corresponding nodes.

[0075] In step 303 above, the number of corrections refers to the cumulative number of times a candidate knowledge fragment has been corrected during the historical interpretation process. The correction time refers to the time when the candidate knowledge fragment was most recently corrected. The correction type refers to the nature of the correction operation on the candidate knowledge fragment, including content updates, clause replacements, and error corrections.

[0076] Credibility level refers to the level of credibility obtained by comprehensively evaluating the candidate knowledge fragment in terms of content accuracy, version timeliness, and source authority.

[0077] The credibility correction amount refers to the correction value used to adjust the basic credibility level, calculated based on the statistical characteristics of the correction behavior.

[0078] In this embodiment, the number of corrections, correction time, and correction type corresponding to each candidate knowledge fragment are obtained from the feedback correction data. Based on the number of corrections and a preset benchmark number, a first correction amount is determined. One way to determine the first correction amount is: when the number of corrections is less than or equal to the preset benchmark number, the first correction amount is zero; when the number of corrections is greater than the preset benchmark number, the difference between the number of corrections and the preset benchmark number is used as the first correction amount. The preset benchmark number can be set according to the average number of corrections or the statistical results of the distribution of corrections among all knowledge fragments in the knowledge base; this embodiment does not specifically limit this.

[0079] Based on the type of correction, a correction coefficient is determined. The correction coefficients corresponding to different correction types can be set according to the sensitivity requirements of different policy knowledge fragments to the correction type. This application does not impose specific limitations on this. For example, for policy knowledge fragments involving tax incentives or fiscal subsidies that are sensitive to monetary amounts, the accuracy of their content has a significant impact on compliance. The correction coefficient corresponding to error correction can be set to a higher value, such as a first preset coefficient of 5, a second preset coefficient of 3 for clause replacement, and a third preset coefficient of 2 for content update. For policy knowledge fragments involving process guidance or application procedures, the correction coefficient corresponding to error correction can be set to a lower value, such as a first preset coefficient of 2, a second preset coefficient of 1 for clause replacement, and a third preset coefficient of 1 for content update.

[0080] Based on the correction time and the current operation time, a second correction amount is determined. One method for determining the second correction amount is to calculate the time interval between the current operation time and the correction time. When the time interval is less than or equal to a preset time threshold, the second correction amount is a first preset value; when the time interval is greater than the preset time threshold, the second correction amount is a second preset value, and the second preset value is greater than the first preset value. The preset time threshold can be set according to the average revision cycle of policy documents, and the aforementioned first and second preset values ​​can be set according to the sensitivity to the timeliness of knowledge. This application embodiment does not specifically limit this.

[0081] For example, by setting the preset time threshold to 6 months, for policy knowledge fragments with high time sensitivity, such as tax incentives or fiscal subsidies, the first preset value can be set to 2 and the second preset value can be set to 5. That is, the further back the correction time is from the current operation time, the greater the correction of the credibility of the knowledge fragment, so as to reflect the time sensitivity risk of policy clauses that have not been updated for a long time. For policy knowledge fragments with low time sensitivity, such as enterprise registration procedures or filing matters, the first preset value can be set to 1 and the second preset value can be set to 2, so as to correct knowledge fragments that have not been updated for a long time but whose content is relatively stable to a smaller extent.

[0082] Multiply the first correction amount, the second correction amount, and the correction coefficient to obtain the credibility correction amount for each candidate knowledge fragment. Map the base credibility level of each candidate knowledge fragment to the corresponding level value. Subtract the credibility correction amount from the level value, and map the difference to the corresponding credibility level to obtain the adjusted credibility level for each candidate knowledge fragment. One credibility level mapping rule can be: divide the credibility levels from high to low into first level, second level, and third level, corresponding to the numerical ranges of the first level, second level, and third level, respectively; after subtracting the credibility correction amount from the level value corresponding to the base credibility level to obtain the difference, determine the numerical range in which the difference falls, and use the credibility level corresponding to the difference as the adjusted credibility level. The numerical ranges corresponding to the different levels can be set according to the granularity requirements of the credibility level division.

[0083] In step 304 above, the effective date refers to the date on which the policy clause corresponding to the candidate knowledge fragment officially takes effect. The expiration date refers to the date on which the policy clause corresponding to the candidate knowledge fragment loses its validity. The repeal mark refers to the mark recorded in the knowledge association data used to identify that the candidate knowledge fragment has been explicitly repealed by the policy document.

[0084] Status markers refer to the marking information used to identify the current applicability status of candidate knowledge fragments. The types of status markers include valid status, outdated status, repealed status, pending effectiveness status, and expired status. Among them, the valid status indicates that the candidate knowledge fragment is the latest applicable valid clause; the outdated status indicates that the candidate knowledge fragment has been replaced by a clause with an updated publication time in the same group; the repealed status indicates that the candidate knowledge fragment has been explicitly repealed; the pending effectiveness status indicates that the effective time of the candidate knowledge fragment is later than the current time; and the expired status indicates that the expiration time of the candidate knowledge fragment is earlier than the current time.

[0085] In this embodiment, the document publication time identifier, effective time, and expiration time corresponding to each candidate knowledge fragment are extracted from the enterprise policy knowledge base. Based on the reference relationship and hierarchical relationship in the knowledge association data, multiple candidate knowledge fragments with reference relationship or belonging to the same level node are divided into the same association group. The candidate knowledge fragments in each association group are sorted according to the document publication time identifier. The status mark of the candidate knowledge fragment whose document publication time identifier is earlier than the latest document publication time identifier in the group is set to the outdated state. The status mark of the candidate knowledge fragment whose effective time is later than the current operation time is set to the pending effective state. The status mark of the candidate knowledge fragment whose expiration time is earlier than the current operation time is set to the expired state. The status mark of the candidate knowledge fragment associated with the repeal mark in the knowledge association data in the association group is set to the repealed state. The status mark of the candidate knowledge fragment with the latest document publication time identifier in the group and not set to other states is set to the valid state.

[0086] This application's embodiments construct candidate knowledge fragments into a confidence propagation network and iteratively propagate them using semantic similarity as the initial confidence level. This enables the credibility assessment to comprehensively consider the structural relevance and semantic consistency of knowledge fragments within the policy system, thereby improving the structural rationality of the basic credibility level assessment. By dynamically adjusting the credibility level in conjunction with feedback correction data, the credibility assessment can reflect the historical evolution of the knowledge fragments' effectiveness in actual use. It also enables the differentiation of knowledge in different time states, clearly distinguishing between valid clauses and outdated, repealed, or not yet effective clauses.

[0087] In step 104 above, the step "generating the content hash value corresponding to each candidate knowledge fragment" specifically includes the following steps:

[0088] Step 401: Perform content normalization processing on each candidate knowledge fragment to obtain the corresponding normalized text, and perform paragraph analysis on each normalized text to obtain the corresponding paragraph information;

[0089] Step 402: Obtain the issuing authority identifier, issuing time identifier, fragment identifier, and document number of each candidate knowledge fragment from the enterprise policy knowledge base. Concatenate the standardized text, issuing authority identifier, issuing time identifier, fragment identifier, document number, and paragraph information corresponding to each candidate knowledge fragment to obtain multiple strings to be hashed.

[0090] Step 403: Perform hash calculation on each string to be hashed to obtain the content hash corresponding to each candidate knowledge fragment.

[0091] In step 401 above, standardized text refers to text content with a unified format obtained after content standardization processing.

[0092] Paragraph information refers to the paragraph number and the starting character position of each paragraph obtained after paragraph analysis.

[0093] In this embodiment of the application, content normalization processing is performed on the text content of each candidate knowledge fragment. One implementation method for performing content normalization on the text content of candidate knowledge fragments is to replace full-width punctuation marks in the text with corresponding half-width punctuation marks, replace line breaks and tabs with space characters, and compress multiple consecutive spaces in the text content into a single space to obtain the corresponding normalized text.

[0094] For each normalized text, paragraph analysis is performed. Specifically, starting from the first character of the normalized text, the line break positions in each normalized text are detected, and the text content between two adjacent line break positions is divided into a paragraph. The paragraphs are assigned paragraph numbers in the order they appear, and the position of the first character of each paragraph in the entire normalized text is recorded as the starting character position of the paragraph. The paragraph number and starting character position of each paragraph are used as the paragraph information corresponding to the normalized text.

[0095] For example, after content normalization, the text content of a candidate knowledge fragment becomes normalized text, with a total length of 800 characters. Scanning this normalized text, line breaks are detected at the 255th and 511th characters, respectively, indicating that the text is divided into three paragraphs: the first paragraph is from the 1st to the 255th character, the second paragraph is from the 256th to the 511th character, and the third paragraph is from the 512th to the 800th character. The first paragraph is assigned a paragraph number of 1 and a starting character position of 1; the second paragraph is assigned a paragraph number of 2 and a starting character position of 256; and the third paragraph is assigned a paragraph number of 3 and a starting character position of 512. These paragraph numbers and starting character positions are used as the paragraph information of the normalized text.

[0096] In step 402 above, the string to be hashed refers to the string formed by concatenating the standardized text, issuing authority identifier, issuing time identifier, segment identifier, document number, and paragraph information in a preset concatenation order.

[0097] In this embodiment, the issuing authority identifier, issuing time identifier, segment identifier, and document number of each candidate knowledge segment are obtained from the enterprise policy knowledge base; the paragraph information, standardized text, issuing authority identifier, issuing time identifier, segment identifier, and document number corresponding to each candidate knowledge segment are connected sequentially in chronological order, and a preset separator is inserted between adjacent items to obtain the string to be hashed corresponding to each candidate knowledge segment. The preset separator can be a newline character or other preset specific characters, and this application does not limit it.

[0098] In step 403 above, the content hash value refers to the fixed-length output digest value obtained after hashing the string to be hashed, which is used to uniquely identify the content status of the candidate knowledge fragment.

[0099] In this embodiment, a hash function can be used to perform hash calculations on each string to be hashed, obtaining the content hash value corresponding to each candidate knowledge fragment. The hash function can be implemented using other algorithms. By performing hash calculations on each string to be hashed, the calculated content hash value uniquely corresponds to the current content state of the string to be hashed. When any character in the string to be hashed changes, the recalculated hash value is different from the original hash value.

[0100] The step of "associating the content hash value and the notarization timestamp corresponding to each candidate knowledge fragment to generate a verifiable credential for each candidate knowledge fragment" specifically includes the following steps:

[0101] Step 411: In a preset order, concatenate the content hash and storage timestamp corresponding to each candidate knowledge fragment to obtain multiple voucher strings. Perform hash calculation on each voucher string to obtain the voucher hash value corresponding to each candidate knowledge fragment.

[0102] Step 412: Send the voucher string and voucher hash value corresponding to each candidate knowledge fragment to multiple evidence storage nodes to obtain the storage information of each candidate knowledge fragment corresponding to each evidence storage node;

[0103] Step 413: Combine the voucher string, voucher hash value and storage information corresponding to each candidate knowledge fragment to generate a verifiable voucher corresponding to each candidate knowledge fragment.

[0104] In step 411 above, the preset order refers to the order in which the content hash value and the evidence storage timestamp are concatenated. This preset order can be set according to the data storage format requirements of the evidence storage node, and this application does not make any specific limitation on it.

[0105] The evidence storage timestamp refers to the current system time obtained when the content hash value is written to the evidence storage node, and is used to record the generation time of the content hash value.

[0106] The credential string refers to the string formed by concatenating the content hash value and the evidence storage timestamp in a preset order.

[0107] A voucher hash value is a fixed-length digest value obtained by hashing a voucher string. It is used to uniquely identify the voucher string and reflect its integrity status.

[0108] In this embodiment of the application, the current system time is obtained as the evidence storage timestamp. The content hash value and evidence storage timestamp corresponding to each candidate knowledge fragment are concatenated in a preset order, and adjacent items are separated by a preset separator to obtain the voucher string corresponding to each candidate knowledge fragment. The preset separator can be a newline character or other preset specific characters, and this application does not limit it. The hash calculation is performed on each voucher string to obtain the voucher hash value corresponding to each candidate knowledge fragment.

[0109] In step 412 above, the evidence storage node refers to an independent computing unit in the distributed storage architecture that undertakes data storage and verification tasks. Each evidence storage node has independent storage space and network address.

[0110] Storage information refers to the path information returned by each evidence storage node after storing the credential string and credential hash value, which is used to locate the stored data. This includes the network address of the evidence storage node and the data index within the node.

[0111] In this embodiment of the application, the credential string and credential hash value corresponding to each candidate knowledge fragment are sent to multiple evidence storage nodes. After receiving the credential string and credential hash value, each evidence storage node stores them in its own storage space and returns the storage information corresponding to that evidence storage node. The storage information returned by each evidence storage node is collected to obtain the storage information of each evidence storage node corresponding to each candidate knowledge fragment.

[0112] In step 413 above, the verifiable voucher refers to the structured data unit formed by combining the voucher string, voucher hash value and storage information.

[0113] In this embodiment of the application, the credential string, credential hash value and storage information returned by each evidence storage node corresponding to each candidate knowledge fragment are combined to generate a verifiable credential corresponding to each candidate knowledge fragment.

[0114] The content hash value of this application embodiment can fully reflect the content status and metadata attributes of the knowledge fragment, and any change in any element can be detected by the change in hash value; the verifiable credential has time anchoring characteristics and integrity self-verification capability. By sending the credential string and credential hash value to multiple evidence storage nodes for storage and obtaining storage information, the verification dependency of the verifiable credential is distributed across multiple independent nodes, avoiding the risk of single point of failure or single point of tampering, and providing distributed storage guarantee for the integrity verification of the knowledge fragment.

[0115] In step 105 above, the preset level refers to the minimum credibility level set in advance, which is used to further filter out fragments with a certain credibility level from the candidate knowledge fragments that have passed the verification. The preset level can be set according to the accuracy requirements of the enterprise for the policy interpretation content. This application does not make specific limitations on the preset level.

[0116] A credible knowledge fragment refers to a candidate knowledge fragment that has passed verification by a majority of verifiable credentials, meets the preset credibility level, and is marked as valid. This fragment has met the quality requirements for generating interpretation answers in terms of content completeness, version timeliness, and source credibility.

[0117] In this embodiment, a majority vote verification is performed on the verifiable credentials corresponding to each candidate knowledge fragment. One implementation of the majority vote verification can be as follows: extract the storage information of each evidence storage node from the verifiable credentials; send information retrieval requests to each evidence storage node according to the network address and data index within the node in the storage information; receive the evidence storage credential string and evidence storage hash value returned by each evidence storage node; perform hash calculation on the evidence storage credential string returned by each evidence storage node to obtain the verification credential hash value corresponding to each evidence storage node; compare the verification credential hash value corresponding to each evidence storage node with the evidence storage credential hash value returned by each node one by one; count the number of evidence storage nodes that match the comparison, and calculate the proportion of the number of evidence storage nodes that match the comparison to the total number of evidence storage nodes that sent the information retrieval request; compare this proportion with a preset proportion value. If the proportion exceeds the preset proportion value, it is determined that the majority vote verification of the candidate knowledge fragment has passed. The preset proportion value can be set according to the total number of evidence storage nodes and the fault tolerance requirements.

[0118] For example, the verifiable credentials of candidate knowledge fragment P1 contain the storage information of three evidence storage nodes. After sending information retrieval requests to the three evidence storage nodes respectively, the hash value of the stored credentials returned by the first evidence storage node is consistent with the hash value of the verification credentials, the comparison result returned by the second evidence storage node is consistent, and the comparison result returned by the third evidence storage node is inconsistent. The number of evidence storage nodes that are consistent is 2, which accounts for two-thirds of the total number of evidence storage nodes. This exceeds the preset proportion of one-half, so it is determined that P1 passes the majority verification.

[0119] From the candidate knowledge fragments that pass the majority verification, fragments that meet the preset credibility level and are marked as valid are selected as credible knowledge fragments. For example, if the preset credibility level is set to level two, and candidate knowledge fragment P1 that passes verification has a credibility level of level two and is marked as valid, and candidate knowledge fragment P2 that passes verification has a credibility level of level three and is marked as valid, then P1 is selected as a credible knowledge fragment, and P2 is discarded because its credibility level is lower than the preset credibility level.

[0120] This application embodiment uses distributed storage with multiple evidence storage nodes and a majority verification mechanism to ensure that the integrity verification of candidate knowledge fragments does not depend on a single node, thereby improving the reliability of the verification results. By combining content integrity verification with credibility level screening and status label screening, the knowledge fragments entering the generation stage are constrained from three dimensions: content authenticity, source credibility, and timeliness validity, providing multi-layered protection for the content quality of the final interpretation and answer.

[0121] Step 106 above specifically includes the following steps:

[0122] Step 601: Based on the credibility level and publication time identifier of the credible knowledge fragment, add a fragment number to each credible knowledge fragment, concatenate the standardized text of each credible knowledge fragment with the preset connecting words, and attach the corresponding fragment number to obtain the context text;

[0123] Step 602: Combine the context text and query text to obtain the prompt text, and perform word-by-word prediction on the prompt text to generate the answer text.

[0124] Step 603: Combine the issuing authority identifier and issuing time identifier of the credible knowledge fragment corresponding to the target fragment number, and the response text to obtain the interpretation response. The target fragment number is the number of the credible knowledge fragment in the response text that matches each output word.

[0125] In step 601 above, the fragment number refers to the sequential identifier temporarily assigned to each trusted knowledge fragment, which is used to mark the source location of the knowledge fragment in the context text.

[0126] Preset conjunctions refer to transitional words used to naturally connect standardized texts with different credible knowledge fragments, such as in addition, at the same time, and another.

[0127] In this embodiment, trusted knowledge fragments are sorted from high to low according to their trustworthiness level, and trusted knowledge fragments with the same trustworthiness level are sorted from newest to oldest according to their publication time identifier to obtain an ordered sequence of trusted knowledge fragments; starting from the serial number 1, each trusted knowledge fragment in the sequence is assigned an increasing fragment number, and the additional form of the fragment number can be marked in square brackets at the end of the normalized text, for example [1]; the normalized text of each trusted knowledge fragment is extracted, and each normalized text is concatenated in sequence according to the sequence order, a preset connecting word is inserted between two adjacent normalized texts, and the fragment number corresponding to the fragment is appended to the end of each normalized text to obtain the context text.

[0128] For example, trusted knowledge fragment P1 has a credibility level of Level 1 and a publication date of June 2025; trusted knowledge fragment P2 has a credibility level of Level 1 and a publication date of March 2025; and trusted knowledge fragment P3 has a credibility level of Level 2 and a publication date of January 2025. After sorting by credibility level, P1 and P2 have the same credibility level. After sorting by publication date from newest to oldest, P1 is located before P2, and the final sequence is P1, P2, P3. Fragment numbers 1, 2, and 3 are assigned to P1, P2, and P3 respectively. Extract the normalized text of P1, P2 and P3, insert the preset conjunction "in addition" between the normalized text of P1 and P2, insert the preset conjunction "at the same time" between the normalized text of P2 and P3, and append [1] to the end of the normalized text of P1, [2] to the end of the normalized text of P2 and [3] to the end of the normalized text of P3. After splicing, the context text is obtained.

[0129] In step 602 above, the context text is placed before the query text, and a preset separator is inserted between the context text and the query text to obtain the prompt text. The preset separator can be a newline character or a specific marker. Then, word-by-word prediction is performed on the prompt text to generate the answer text. This step specifically includes the following steps:

[0130] Step 611: Match and calculate the probability score of each candidate word in the prompt text and the preset word list.

[0131] Step 612: Based on the normalized text in the context text, adjust the preliminary probability score corresponding to each candidate word to obtain the probability score, and select the candidate word with the highest probability score as the current output word.

[0132] Step 613: Add the current output word to the end of the prompt text to obtain the updated prompt text. Select the next output word based on the updated prompt text until the number of selected output words meets the preset limit. Concatenate all output words to obtain the answer text.

[0133] In step 611 above, the preset vocabulary refers to a pre-constructed vocabulary set that stores all candidate words that can be selected as output words and their corresponding semantic vectors.

[0134] The preliminary probability score refers to the initial probability measure of each candidate word in the preset vocabulary being selected as the next output word in the current context after matching calculation.

[0135] In this embodiment, the prompt text and each candidate word in a preset vocabulary are input into a pre-trained semantic encoding model for semantic encoding to obtain a global semantic vector corresponding to the prompt text. The global semantic vector is a numerical representation of the overall semantics of the prompt text. The spatial distance between the global semantic vector and the semantic vectors corresponding to each candidate word is calculated one by one. The spatial distance calculation can be performed by calculating the cosine angle between the two vectors, or by other available algorithms.

[0136] The cosine angle value corresponding to each candidate word is converted into a corresponding preliminary probability score through a preset mapping relationship. The closer the cosine angle value is to 1, the better the semantic match between the candidate word and the prompt text, and the higher the corresponding preliminary probability score. The preset mapping relationship can be set according to the conversion rules between the semantic matching degree between the candidate word and the prompt text and the preliminary probability score. This application embodiment does not limit this. For example, one way to set the preset mapping relationship is to divide the range of cosine angle values ​​into multiple intervals, each interval corresponding to a preliminary probability score. The closer the cosine angle value is to 1, the higher the corresponding preliminary probability score. For example, a cosine angle value between 0.9 and 1.0 corresponds to a preliminary probability score of 0.9, a cosine angle value between 0.8 and 0.9 corresponds to a preliminary probability score of 0.7, a cosine angle value between 0.7 and 0.8 corresponds to a preliminary probability score of 0.5, and a cosine angle value below 0.7 corresponds to a preliminary probability score of 0.3. The number of intervals and the corresponding scores can be adjusted according to actual application needs. This application embodiment does not limit this. The above spatial distance metric calculation and score conversion process ensures that the initial probability score of each candidate word not only reflects its linguistic coherence with the generated text, but also implicitly incorporates the semantic fit of the candidate word under the current query intent and the generated context.

[0137] In step 612 above, each candidate word in the preset vocabulary is compared with each normalized text in the context text. If a candidate word matches a word in a normalized text, the preliminary probability score corresponding to the candidate word is multiplied by a preset enhancement coefficient to obtain the probability score of the candidate word. If the candidate word does not match any word in any normalized text, the preliminary probability score of the candidate word is directly used as the probability score. The preset enhancement coefficient is a value greater than 1. The preset enhancement coefficient can be set according to the constraint requirements on the consistency between the generated content and the knowledge fragment. For example, for policy areas with high compliance risks, a higher enhancement coefficient can be set to force the generated content to be consistent with the original text of the knowledge fragment; for policy areas with low compliance risks, a lower enhancement coefficient can be set to take into account the natural fluency of the generated text. This application embodiment does not limit the specific value of the preset enhancement coefficient. The candidate word with the highest probability score is selected from all candidate words as the current output word.

[0138] In step 613 above, the preset quantity limit refers to the upper limit of the number of output words generated in advance, which is used to control the maximum length of the response text.

[0139] In this embodiment, the current output word is appended to the end of the prompt text, and the appended text is used as the updated prompt text. Based on the updated prompt text, the matching calculation in step 611, the probability score adjustment and output word selection in step 612, and the prompt text update operation in step 613 are repeated, generating a new output word in each round of the loop. When the total number of generated output words reaches a preset limit, the loop is terminated. All output words are concatenated in the order of their generation to obtain the answer text.

[0140] In step 603 above, the target fragment number is the number of the credible knowledge fragment in the response text that matches each output word.

[0141] In this embodiment, during the word-by-word prediction process to generate the response text, the segment number corresponding to each output word when it is matched with each normalized text in the context text in step 612 is recorded. After deduplicating all recorded segment numbers, the target segment number is obtained. The issuing authority identifier and issuance time identifier of the credible knowledge segment corresponding to each target segment number are obtained from the enterprise policy knowledge base. The response text, the issuing authority identifiers, and the issuance time identifiers are combined to obtain the interpretation response. In the interpretation response, the response text is the main content, and the issuing authority identifiers and issuance time identifiers are appended to the response text as source annotations for the policy basis cited in the response text.

[0142] For example, during the response text generation process, it was recorded that the output word matched the normalized text corresponding to segment number 1 5 times and the normalized text corresponding to segment number 3 2 times. After deduplication, the target segment numbers were obtained as 1 and 3. The issuing authority identifier for segment number 1 was obtained as "Ministry of Finance" and the issuing time identifier as "June 2025", and the issuing authority identifier for segment number 3 was obtained as "State Taxation Administration" and the issuing time identifier as "January 2025". The response text was combined with the above two issuing authority identifiers and issuing time identifiers to obtain the interpretation response.

[0143] The context text generated in this application conforms to the principles of prioritizing high credibility and new version; it avoids expressions inconsistent with the retrieved knowledge by predicting words one by one; and it allows users to directly trace the policy source and version validity of the answer by recording the segment number of the output word match and adding the corresponding issuing authority and issuing time identifier to the interpretation answer.

[0144] Figure 2This is a schematic diagram of a specific implementation of the RAG anti-hallucination system based on credibility grading and traceability verification provided in this application embodiment, with reference to... Figure 2 The system may include:

[0145] The acquisition module 21 is used to acquire the query text of the target company's personnel;

[0146] Analysis module 22 is used to perform semantic analysis on the query text to extract multiple candidate knowledge fragments from the enterprise policy knowledge base and extract multidimensional credible data from each candidate knowledge fragment;

[0147] The determination module 23 is used to determine the credibility level of each candidate knowledge fragment based on the knowledge association data, semantic annotation data and feedback correction data in the multidimensional trusted data, and to generate a status label for each candidate knowledge fragment.

[0148] The association module 24 is used to generate the content hash corresponding to each candidate knowledge fragment, and associate the content hash corresponding to each candidate knowledge fragment with the evidence storage timestamp to generate a verifiable credential corresponding to each candidate knowledge fragment.

[0149] The verification module 25 is used to perform majority verification on the corresponding candidate knowledge fragments based on each verifiable credential, and select the candidate knowledge fragments that pass the majority verification, meet the preset credibility level, and are marked as valid as credible knowledge fragments.

[0150] The generation module 26 is used to perform retrieval enhancement generation processing on the credible knowledge fragment and the query text to generate an interpretation answer.

[0151] The RAG anti-hallucination system based on credibility grading and source tracing verification in this application embodiment is used to implement the aforementioned RAG anti-hallucination method based on credibility grading and source tracing verification. Therefore, the specific implementation of the RAG anti-hallucination system based on credibility grading and source tracing verification can be found in the embodiment section of the RAG anti-hallucination method based on credibility grading and source tracing verification above. The specific implementation can be referred to the description of the corresponding embodiments, and will not be repeated here.

[0152] This application also provides a computing device, including a processing component and a storage component; the storage component stores one or more computer instructions; the one or more computer instructions are to be invoked and executed by the processing component to implement any of the above-described RAG anti-hallucination methods based on credibility grading and traceability verification.

[0153] This application also provides a computer-readable storage medium storing a computer program, which, when executed by a computer, implements any of the aforementioned RAG anti-hallucination methods based on credibility grading and traceability verification.

[0154] In one exemplary embodiment, the aforementioned computer-readable storage medium may include, but is not limited to, various media capable of storing computer programs, such as USB flash drives, read-only memory, random access memory, portable hard drives, magnetic disks, or optical disks.

[0155] Embodiments of this application also provide a computer program product, which includes a computer program that, when executed by a processor, implements the steps in any of the embodiments of the RAG anti-hallucination method based on credibility grading and traceability verification described above.

[0156] It should be noted that the steps shown in the flowchart in the accompanying drawings can be executed in a computer system such as a set of computer-executable instructions, and although a logical order is shown in the flowchart, in some cases the steps shown or described may be executed in a different order than that shown here.

[0157] Obviously, those skilled in the art should understand that the various units or steps of this application described above can be implemented using general-purpose computing devices. They can be centralized on a single computing device or distributed across a network of multiple computing devices. Optionally, they can be implemented using computer-executable program code, thereby storing them in a storage device for execution by a computing device, or fabricating them separately as individual integrated circuit modules, or fabricating multiple modules or steps into a single integrated circuit module. Thus, this application is not limited to any particular combination of hardware and software.

[0158] The above description is merely a preferred embodiment of this application and is not intended to limit this application. Various modifications and variations can be made to this application by those skilled in the art. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of this application should be included within the protection scope of this application.

Claims

1. A RAG anti-hallucination method based on credibility grading and source tracing verification, characterized in that, include: Obtain the query text of personnel from the target company; Semantic analysis is performed on the query text to extract multiple candidate knowledge fragments from the enterprise policy knowledge base, and multidimensional credible data is extracted from each candidate knowledge fragment. Based on the knowledge association data, semantic annotation data, and feedback correction data in the multidimensional trusted data, the credibility level of each candidate knowledge fragment is determined, and a status label for each candidate knowledge fragment is generated. Generate the content hash corresponding to each candidate knowledge fragment, and associate the content hash corresponding to each candidate knowledge fragment with the evidence storage timestamp to generate a verifiable credential corresponding to each candidate knowledge fragment. Based on each verifiable credential, the corresponding candidate knowledge fragments are subjected to majority verification, and the candidate knowledge fragments that pass the majority verification, meet the preset credibility level, and are marked as valid are selected as credible knowledge fragments. The credible knowledge fragment and the query text are subjected to retrieval enhancement and generation processes to generate an interpretation answer.

2. The method according to claim 1, characterized in that, Semantic analysis is performed on the query text to extract multiple candidate knowledge fragments from the enterprise policy knowledge base, including: The query text is segmented and subjected to dependency parsing to identify multiple target entity names with dependency relationships; Based on the preset name mapping table and the preset synonym mapping table, determine the standard name and synonym corresponding to each target entity name; Each target entity name and its corresponding standard name and synonym are converted into a semantic vector. The similarity between each semantic vector and the semantic vector corresponding to each policy knowledge fragment in the enterprise policy knowledge base is calculated to obtain the comprehensive similarity of each policy knowledge fragment. Policy knowledge fragments with a comprehensive similarity exceeding a preset threshold are selected as candidate knowledge fragments.

3. The method according to claim 1, characterized in that, Based on the knowledge association data, semantic annotation data, and feedback correction data in the multidimensional trusted data, the trustworthiness level of each candidate knowledge fragment is determined, and a status label for each candidate knowledge fragment is generated, including: Calculate the semantic similarity between each semantically annotated text and the corresponding candidate knowledge fragment in the semantically annotated data of the multidimensional trusted data. Using each candidate knowledge fragment as a node, establish the association edge between each node based on the knowledge association data in the multidimensional trusted data. Based on each node and the corresponding association edge and similarity, construct a confidence propagation network. The confidence of each node in the confidence propagation network is the semantic similarity. Confidence propagation is performed in the confidence propagation network to obtain the basic confidence level corresponding to each node; Based on the number of corrections, correction time, and correction type corresponding to each candidate knowledge fragment in the feedback correction data, the credibility correction amount corresponding to each candidate knowledge fragment is determined. Based on the credibility correction amount corresponding to each candidate knowledge fragment, the corresponding basic credibility level is adjusted to obtain the credibility level of each candidate knowledge fragment. Based on the publication time identifier, effective time, expiration time, and obsolescence flag in the knowledge association data of each candidate knowledge fragment, a status flag is generated for each candidate knowledge fragment. The status flag can be a valid status, an outdated status, an obsolescence status, a pending status, or an expired status.

4. The method according to claim 1, characterized in that, Generate the content hashes corresponding to each candidate knowledge fragment, including: Each candidate knowledge fragment is processed for content standardization to obtain the corresponding standardized text. Each standardized text is then analyzed to obtain the corresponding paragraph information. The issuing authority identifier, issuing time identifier, fragment identifier, and document number of each candidate knowledge fragment are obtained from the enterprise policy knowledge base. The standardized text, issuing authority identifier, issuing time, fragment identifier, document number, and paragraph information corresponding to each candidate knowledge fragment are concatenated to obtain multiple strings to be hashed. Perform hash calculations on each string to be hashed to obtain the content hash corresponding to each candidate knowledge fragment.

5. The method according to claim 1, characterized in that, The content hash and storage timestamp corresponding to each candidate knowledge fragment are associated to generate verifiable credentials for each candidate knowledge fragment, including: According to the preset order, the content hash and storage timestamp corresponding to each candidate knowledge fragment are concatenated to obtain multiple voucher strings. The hash calculation is performed on each voucher string to obtain the voucher hash value corresponding to each candidate knowledge fragment. Send the credential string and credential hash value corresponding to each candidate knowledge fragment to multiple evidence storage nodes to obtain the storage information of each candidate knowledge fragment corresponding to each evidence storage node; The credential string, credential hash value, and storage information corresponding to each candidate knowledge fragment are combined to generate a verifiable credential for each candidate knowledge fragment.

6. The method according to claim 1, characterized in that, The credible knowledge fragment and the query text are subjected to retrieval enhancement and generation processing to generate an interpretation answer, including: Based on the credibility level and publication time identifier of the credible knowledge fragment, a fragment number is added to each credible knowledge fragment. The standardized text of each credible knowledge fragment and the preset connecting words are concatenated and the corresponding fragment number is added to obtain the context text. The context text and the query text are combined to obtain the prompt text, and word-by-word prediction is performed on the prompt text to generate the answer text; The issuing authority identifier and issuing time identifier of the credible knowledge fragment corresponding to the target fragment number are combined with the answer text to obtain the interpretation answer. The target fragment number is the number of the credible knowledge fragment in the answer text that matches each output word.

7. The method according to claim 6, characterized in that, The prompt text is predicted word by word to generate the response text, including: The prompt text is matched and calculated with each candidate word in the preset word list to obtain the preliminary probability score corresponding to each candidate word; Based on the normalized text in the context text, the preliminary probability scores of each candidate word are adjusted to obtain the probability scores, and the candidate word with the highest probability score is selected as the current output word. The current output word is added to the end of the prompt text to obtain the updated prompt text. The next output word is selected based on the updated prompt text until the number of selected output words meets the preset limit. All output words are then concatenated to obtain the answer text.

8. A RAG anti-hallucination system based on credibility grading and traceability verification, characterized in that, include: The acquisition module is used to acquire the query text of personnel from the target company; The analysis module is used to perform semantic analysis on the query text to extract multiple candidate knowledge fragments from the enterprise policy knowledge base and extract multidimensional credible data from each candidate knowledge fragment. The determination module is used to determine the credibility level of each candidate knowledge fragment based on the knowledge association data, semantic annotation data and feedback correction data in the multidimensional trusted data, and to generate a status label for each candidate knowledge fragment. The association module is used to generate the content hash corresponding to each candidate knowledge fragment, and associate the content hash corresponding to each candidate knowledge fragment with the evidence storage timestamp to generate a verifiable credential corresponding to each candidate knowledge fragment. The verification module is used to perform majority verification on the corresponding candidate knowledge fragments based on each verifiable credential, and select the candidate knowledge fragments that pass the majority verification, meet the preset credibility level, and are marked as valid as credible knowledge fragments. The generation module is used to perform retrieval enhancement generation processing on the credible knowledge fragment and the query text to generate an interpretation answer.

9. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores computer instructions for causing the computer to execute the RAG anti-hallucination method based on credibility grading and traceability verification as described in any one of claims 1-7.

10. A computing device, characterized in that, The computing device includes: at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores a computer program executable by the at least one processor, the computer program being executed by the at least one processor to cause the at least one processor to perform the RAG anti-hallucination method based on credibility grading and traceability verification as described in any one of claims 1-7.

Images