A trusted answer generation method, device, equipment and storage medium

Abstract

The application provides a trusted answer generation method and device, equipment and a storage medium, comprising: obtaining a user query, identifying the query intention of the user query; configuring a retrieval strategy according to the query intention and performing retrieval to obtain a candidate document; performing hierarchical relevance analysis on the candidate document based on the retrieval strategy to obtain a plurality of relevant documents; cross-document association of evidence in the plurality of relevant documents to form an evidence chain; and generating a final answer according to the hierarchical relevance analysis and / or the evidence chain. The above method can effectively improve the accuracy and credibility of answer generation.

Description

Technical Field

[0001] This application relates to the field of natural language processing, and more specifically, to a method, apparatus, device, and storage medium for generating reliable answers. Background Technology

[0002] Retrieval-Augmented Generation (RAG) is a key supporting technology for the practical application of Large Language Models (LLM). By accessing external entity document resources, this technology compensates for the shortcomings of large models, such as limited parameter storage and the inability to update the knowledge base in real time. It is currently widely used in industrial scenarios such as internal document Q&A, professional information retrieval, and scenario-based intelligent question answering. It is also a common solution in the industry to alleviate the problem of distortion in model-generated content.

[0003] Currently, mainstream traditional retrieval enhancement generation adopts a standardized fixed processing pipeline. In the early stage, all documents are uniformly segmented with a fixed character length. The segmented text is uniformly converted into high-dimensional vectors and stored in a vector library using an embedding model. In the user request response stage, regardless of the content of the user's query, the query text is uniformly converted into a vector. The top-K document fragments are fixedly selected based on the cosine similarity between the vectors. The selected content is directly concatenated to form context and then input into a large model to generate the corresponding answer. The retrieval rules and configuration parameters of the entire process remain fixed throughout.

[0004] This technical solution lacks the logic to differentiate and process query intent. It uses the same set of search parameters for queries with different needs, such as factual inquiries, comparative analysis, and summary, which makes the search strategy unsuitable for the actual query requirements. This easily leads to the omission of high-value relevant documents and the incorrect recall of a large number of irrelevant documents, ultimately resulting in low-quality reference materials used to generate answers. Summary of the Invention

[0005] In view of this, the purpose of this application is to provide a reliable answer generation method, apparatus, device and storage medium that can effectively improve the accuracy and reliability of answer generation.

[0006] In a first aspect, embodiments of this application provide a method for generating a reliable answer, the method comprising: Obtain user queries and identify the query intent of the user queries; Configure the retrieval strategy according to the query intent and execute the retrieval to obtain candidate documents; Based on the retrieval strategy, a hierarchical relevance analysis is performed on the candidate documents to obtain several relevant documents; The evidence in the aforementioned related documents is linked across documents to form a chain of evidence; The final answer is generated based on the hierarchical correlation analysis and / or the chain of evidence.

[0007] Optionally, identifying the user's query intent includes: Multidimensional feature extraction is performed on the user query to obtain the target features; Based on the target features, the user query is classified to obtain the query intent, wherein the query intent includes factual, comparative analysis, causal reasoning, and comprehensive summary types.

[0008] Optionally, configuring a retrieval strategy based on the query intent and performing the retrieval to obtain candidate documents includes: A retrieval strategy configuration table is generated based on the query intent, wherein the retrieval strategy configuration table includes retrieval channel weight allocation, inference depth parameters, and whether cross-document inference is enabled; Multiple retrieval channels are scheduled to perform retrieval in parallel according to the retrieval strategy configuration table; The search results from each channel are merged and sorted to obtain the candidate documents.

[0009] Optionally, the step of performing hierarchical relevance analysis on the candidate documents based on the retrieval strategy to obtain a number of relevant documents, including: Based on the reasoning depth parameter in the retrieval strategy configuration table, a relevance judgment involving at least two reasoning stages is performed on the candidate documents, wherein the subsequent stages have more granular document analysis than the previous stages, and the reasoning stages include at least two of outline-level judgment, evidence extraction, and paragraph-level close reading. After each phase is completed, assess the number of directly relevant documents obtained and their average confidence level. When both the quantity and the average confidence level reach a preset threshold, the execution of subsequent stages is terminated early, and the aforementioned related documents are obtained.

[0010] Optionally, the step of cross-document association of evidence in the plurality of related documents to form a chain of evidence includes: Evidence fragments from the aforementioned related documents are used as nodes, and edges are established based on the relationships between nodes to construct an evidence graph. The edge types include shared entity edges, semantically similar edges, and edges representing logical relationships. Consistency mediation is performed on the evidence nodes in the evidence graph that have contradictory relationships, and the confidence level of each node in the evidence graph is updated according to the mediation result; In the updated evidence graph, starting from the seed node that matches the query intent, a sequence of evidence nodes that can form a complete reasoning path is searched, and the sequence of evidence nodes is taken as the evidence chain.

[0011] Optionally, generating the final answer based on the hierarchical correlation analysis and / or the chain of evidence includes: Based on the hierarchical correlation analysis and / or the chain of evidence, a comprehensive confidence level is calculated, which includes at least the dimensions of sufficiency of evidence and consistency of sources. The final answer is generated based on the comprehensive confidence level constraint answer generation process.

[0012] Optionally, it also includes: A structured citation list is appended after the final answer. The citation list includes the identification information of the cited documents, the cited chapters, the confidence level, and a summary of the reasoning path of the chain of evidence.

[0013] Secondly, embodiments of this application provide a reliable answer generation apparatus, the apparatus comprising: The intent recognition module is used to acquire user queries and recognize the query intent of the user queries; The retrieval module is used to configure a retrieval strategy according to the query intent and execute the retrieval to obtain candidate documents; The hierarchical analysis module is used to perform hierarchical relevance analysis on the candidate documents based on the retrieval strategy to obtain several related documents; The cross-document association module is used to associate evidence from the aforementioned related documents across documents to form a chain of evidence; The answer generation module is used to generate a final answer based on the hierarchical correlation analysis and / or the chain of evidence.

[0014] Optionally, the intent recognition module specifically includes: The feature extraction submodule is used to perform multi-dimensional feature extraction on the user query to obtain the target features; The intent classification submodule is used to classify the user query based on the target features to obtain the query intent, wherein the query intent includes factual, comparative analysis, causal reasoning and comprehensive summary.

[0015] Optionally, the retrieval module specifically includes: The configuration table generation submodule is used to generate a retrieval strategy configuration table based on the query intent. The retrieval strategy configuration table includes retrieval channel weight allocation, inference depth parameters, and whether to enable cross-document inference. The multi-channel retrieval submodule is used to schedule multiple retrieval channels to perform retrieval in parallel according to the retrieval strategy configuration table; The result merging submodule is used to merge and sort the search results of each channel to obtain the candidate documents.

[0016] Optionally, the hierarchical analysis module specifically includes: The phased reasoning submodule is used to perform relevance judgment on the candidate document, which includes at least two reasoning stages, according to the reasoning depth parameter in the retrieval strategy configuration table. The subsequent stages have more granular document analysis than the previous stages. The reasoning stages include at least two of the following: outline-level judgment, evidence extraction, and paragraph-level close reading. The threshold verification submodule is used to evaluate the number of directly relevant documents obtained and their average confidence level after each stage is completed; The early termination submodule is used to terminate the execution of subsequent stages in advance when the quantity and average confidence level both reach a preset threshold, thereby obtaining the aforementioned related documents.

[0017] Optionally, the cross-document association module specifically includes: The graph construction submodule is used to construct an evidence graph by taking evidence fragments from the several related documents as nodes and establishing edges based on the relationships between nodes. The edge types include shared entity edges, semantically similar edges, and edges representing logical relationships. The conflict mediation submodule is used to mediate the consistency of evidence nodes with contradictory relationships in the evidence graph, and update the confidence level of each node in the evidence graph according to the mediation result. The evidence chain retrieval submodule is used to search for a sequence of evidence nodes that can form a complete reasoning path in the updated evidence graph, starting from the seed node that matches the query intent, and to use the sequence of evidence nodes as the evidence chain.

[0018] Optionally, the answer generation module specifically includes: The confidence calculation submodule is used to calculate the overall confidence level based on the hierarchical correlation analysis and / or the chain of evidence, wherein the overall confidence level includes at least the sufficiency of evidence dimension and the consistency of source dimension; The constraint generation submodule is used to generate the final answer based on the comprehensive confidence level constraint answer generation process.

[0019] Optionally, the answer generation module is further configured to: A structured citation list is appended after the final answer. The citation list includes the identification information of the cited documents, the cited chapters, the confidence level, and a summary of the reasoning path of the chain of evidence.

[0020] Thirdly, embodiments of this application provide a computer device, including: a processor, a memory, and a bus. The memory stores machine-readable instructions executable by the processor. When the computer device is running, the processor communicates with the memory via the bus. When the machine-readable instructions are executed by the processor, the steps of the reliable answer generation method described in any of the optional embodiments of the first aspect are performed.

[0021] Fourthly, embodiments of this application provide a computer-readable storage medium storing a computer program that, when executed by a processor, performs the steps of the reliable answer generation method described in any of the optional embodiments of the first aspect.

[0022] The technical solution provided in this application includes, but is not limited to, the following beneficial effects: By acquiring user queries and identifying query intent, we can categorize queries based on their specific needs, providing a basis for subsequent differentiated retrieval rule configuration. This avoids the problem of using the same processing logic for all queries from the outset, and facilitates the on-demand allocation of retrieval resources.

[0023] Based on the identified query intent, a targeted retrieval strategy is configured and a search is conducted. Candidate documents are filtered based on the appropriate retrieval rules. This can retrieve documents that match the current query information requirements, reduce the probability of irrelevant documents being included in the candidate set, and improve the overall relevance of candidate documents.

[0024] Based on the configured search strategy, hierarchical relevance analysis is performed on candidate documents. Invalid documents are gradually eliminated by relying on the hierarchical screening mode. Valid and relevant documents are obtained through refined screening. The reference materials used in subsequent answer generation are simplified, and the interference of invalid content on the generated results is reduced.

[0025] By linking the internal evidence of the selected relevant documents across documents and constructing an evidence chain, information barriers between different documents can be broken down, scattered information from multiple documents can be integrated to form a complete basis for information reasoning, and the data support for the answer can be improved.

[0026] The final answer is generated by combining the results of hierarchical correlation analysis and the constructed chain of evidence. The answer is generated based on the selected reliable documents and the complete chain of evidence, which effectively ensures that the information source of the output content is verifiable and improves the credibility of the answer content.

[0027] In summary, the entire method, consisting of intent recognition, differentiated retrieval, hierarchical screening, evidence concatenation, and credible generation, works in sequence to optimize document screening and information integration at each level. While improving the quality of document retrieval, it also enhances evidence support, effectively improving the accuracy and credibility of answer generation.

[0028] To make the above-mentioned objectives, features and advantages of this application more apparent and understandable, preferred embodiments are described below in detail with reference to the accompanying drawings. Attached Figure Description

[0029] To more clearly illustrate the technical solutions of the embodiments of this application, the accompanying drawings used in the embodiments will be briefly introduced below. It should be understood that the following drawings only show some embodiments of this application and should not be regarded as a limitation of the scope. For those skilled in the art, other related drawings can be obtained based on these drawings without creative effort.

[0030] Figure 1 A flowchart of a reliable answer generation method provided in Embodiment 1 of this application is shown; Figure 2 This paper presents an overall system architecture diagram of a reliable answer generation method provided in Embodiment 1 of this application; Figure 3 A flowchart of a query intent recognition method provided in Embodiment 1 of this application is shown; Figure 4 This paper illustrates a full-link processing logic flowchart provided in Embodiment 1 of this application; Figure 5 A flowchart of a candidate document determination method provided in Embodiment 1 of this application is shown; Figure 6 This document shows a flowchart of a query intent analysis and dynamic strategy selection method provided in Embodiment 1 of this application. Figure 7 A flowchart of a method for obtaining related documents provided in Embodiment 1 of this application is shown; Figure 8 The flowchart of an adaptive progressive relevance inference module provided in Embodiment 1 of this application is shown. Figure 9 A flowchart of a chain of evidence formation method provided in Embodiment 1 of this application is shown; Figure 10 This paper shows a flowchart of a cross-document evidence chain reasoning module provided in Embodiment 1 of this application; Figure 11 A flowchart of a final answer generation method provided in Embodiment 1 of this application is shown; Figure 12 This paper shows a flowchart of an answer generation module for confidence calibration provided in Embodiment 1 of this application; Figure 13 This illustration shows a schematic diagram of a reliable answer generation device provided in Embodiment 2 of this application; Figure 14A schematic diagram of the structure of a computer device provided in Embodiment 3 of this application is shown. Detailed Implementation

[0031] To make the objectives, technical solutions, and advantages of the embodiments of this application clearer, the technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments. The components of the embodiments of this application described and shown in the accompanying drawings can generally be arranged and designed in various different configurations. Therefore, the following detailed description of the embodiments of this application provided in the accompanying drawings is not intended to limit the scope of the claimed application, but merely represents selected embodiments of this application. All other embodiments obtained by those skilled in the art based on the embodiments of this application without inventive effort are within the scope of protection of this application.

[0032] Example 1 To facilitate understanding of this application, the following is combined with... Figure 1 The flowchart illustrating a reliable answer generation method provided in Embodiment 1 of this application will be described in detail for Embodiment 1 of this application.

[0033] See Figure 1 As shown, Figure 1 A flowchart of a reliable answer generation method provided in Embodiment 1 of this application is shown, wherein the method includes steps S101 to S105: S101: Obtain user query and identify the query intent of the user query.

[0034] Specifically, this step extracts and fuses features from the query text across three dimensions: lexical, structural, and entity, to form a query feature vector. At the lexical level, the system statistically analyzes interrogative words, comparative words, causal words, and time- and quantity-related expressions within the query, while also filtering out stop words.

[0035] At the structural level, the system distinguishes between simple, compound, and parallel sentence structures, counts the number of sub-questions and the length of the full-text query, and at the entity level, it extracts proper nouns and technical terms through entity recognition and counts entity types.

[0036] Based on the generated feature vectors, query classification is performed, dividing queries into four categories: factual, comparative analysis, causal reasoning, and comprehensive summary. Then, mapping rules are applied. Complete parameter matching.

[0037] in Represents the weights of three types of search channels. This represents the number of documents recalled. Represents the maximum depth of the reasoning stage. Used to indicate whether cross-document reasoning is enabled.

[0038] The four types of queries correspond to fixed parameter configurations, while fact queries are configured with weights. Disable cross-document inference; perform comparative analysis of configuration weights. Enable cross-document reasoning; configure weights for causal reasoning. Enable cross-document reasoning; comprehensively review and configure weights. Enable cross-document reasoning.

[0039] S102: Configure the retrieval strategy according to the query intent and execute the retrieval to obtain candidate documents.

[0040] Specifically, this step simultaneously runs three independent retrieval channels and calculates scores according to preset weights. In the structured metadata retrieval channel, the base score for title / summary fuzzy matching is 0.6, for entity exact matching is 0.5, and for topic matching is 0.4. The individual scores are correlated with the corresponding channel weights. Multiply.

[0041] The dual-embedded semantic retrieval channel retrieves two sets of vector indexes for similarity calculation. When both indexes for the same document are matched, the average similarity is taken and multiplied by a 1.1 additive coefficient. If only one index is matched, the similarity score is used directly, and the result is multiplied by... .

[0042] The graph retrieval channel selects high-scoring documents from the first two channels as starting nodes to perform a breadth-first search (BFS) traversal. The maximum traversal depth is two hops. The score for a single document is calculated using the formula: Score = The number of jumps: 0.3 points for the first jump, 0.15 points for the second jump, and the score is multiplied by... When a circular link occurs during the retrieval process, duplicate documents are deduplicated.

[0043] In the multi-channel result merging stage, when the same document is retrieved by multiple channels, the highest weighted score is selected and an additional fixed score of 0.1 is added. All documents are sorted in descending order of comprehensive score and a specified number of documents are truncated according to parameter K. At the same time, full metadata information of the documents is supplemented from the structured index to form a candidate document set.

[0044] S103: Perform hierarchical relevance analysis on the candidate documents based on the retrieval strategy to obtain several related documents.

[0045] Specifically, this step is based on the parameters. Divide into three inference execution paths, Using the fast path, only the outline content is evaluated. The standard approach was used to complete the outline assessment and evidence extraction. The deep path is used to sequentially perform outline judgment, evidence extraction, and paragraph in-depth reading.

[0046] The document outline consists of the document title, a list of headings at all levels, subject tags, and the first 200 characters of the abstract. After the outline is input into the large model, the model outputs a structured JSON (JavaScript Object Notation, a lightweight data exchange format) result, which includes relevance (direct / indirect / none), confidence value, likely sections (potential evidence sections), and reasoning.

[0047] After each reasoning stage, the formula is used. Calculate the average confidence level of directly relevant documents at the current stage, where... Representing the Average confidence level of directly relevant documents for each stage. This is the collection of directly relevant documents selected at the current stage. It is the total number of documents contained in the collection. Refers to the first The relevance confidence score of the document.

[0048] When satisfied and The subsequent reasoning steps can be terminated in advance. The default value is 3. The default value is 0.85.

[0049] In the evidence extraction stage, the standard path has a document processing character limit of 8000 characters, and the deep path has a limit of 15000 characters. These character limits are determined by the `max_token` parameter of the mainstream embedding model. Close reading of paragraphs only applies to cases where the confidence level in the previous stage is below a threshold. The non-irrelevant documents are broken down into paragraphs for detailed analysis. After filtering out documents marked as "none" and removing them as irrelevant, the remaining documents are sorted and summarized according to their confidence level.

[0050] S104: Cross-document association of evidence in the aforementioned related documents to form an evidence chain.

[0051] Specifically, this operation only occurs when... When marked as enabled, the first step is to build an evidence graph. , A node represents the set of nodes consisting of all pieces of evidence; a single node... Record source documents and self-confidence level and related tags, Represents the edges connecting nodes.

[0052] Related edges are divided into three categories: cross-document evidence containing the same entity constructs a shared entity edge, with the edge weight equal to the number of shared entities; evidence fragment vector similarity exceeds a threshold. When the value is 0.75, semantically similar edges are generated; logically related edges are generated by distinguishing four logical relationships: support, contradiction, supplement, and deduction based on semantic analysis.

[0053] For nodes with conflicting connections, the conflict content is mediated based on the timeliness of information, the authority of document sources, and the amount of supporting evidence, and the confidence level of the corresponding node is updated according to the mediation results.

[0054] Subsequent reliance on formula Calculate the overall confidence level of a single chain of evidence, where... Represents the overall confidence level of the chain of evidence. The confidence level of a single evidence node within the link. This represents the weight of the edge between adjacent nodes.

[0055] Only retain those with a confidence level higher than [missing information] The valid evidence chain is determined by discarding the link with lower confidence when the overlap of nodes in two evidence chains exceeds 80%, and finally, standardized evidence chain information is obtained.

[0056] S105: Generate the final answer based on the hierarchical correlation analysis and / or the chain of evidence.

[0057] Specifically, the generation process relies on a four-dimensional evaluation system to complete the credibility calculation and construct a confidence vector. , Characterizing the sufficiency of evidence, Characterizes the reliability of document sources. Characterizing the consistency of content from multiple sources of evidence, It represents the overall logical completeness.

[0058] Then through the calculation formula To obtain the overall confidence level of a single fact, the formula is as follows: For the final confidence level, These are the engineering experience weighting coefficients corresponding to the four evaluation dimensions.

[0059] During the generation process, low-confidence evidence is restricted from participating in content writing, and content of different confidence levels is displayed separately; if no valid relevant documents are found throughout the entire process, a prompt text indicating that no relevant documents were found is returned directly, and the model is no longer called to generate content, thereby reducing model illusions from the source.

[0060] This solution abandons the traditional RAG (Retrieval-Augmented Generation) approach, which uses a fixed two-stage execution process: document segmentation, vector input, single-round vector retrieval, and unified generation of a large LLM (Large Language Model). Instead, it is divided into five processing units that sequentially complete query intent parsing, dynamic retrieval filtering, hierarchical relevance reasoning, cross-document evidence networking, and answer generation based on confidence constraints.

[0061] The entire method is equipped with three types of collaborative storage resources. The first type relies on relational databases to build structured indexes, which are used to store structured information such as document titles, summaries, categories, entities, heading levels, and file hashes, and support SQL (Structured Query Language) conditional retrieval.

[0062] The second type relies on a vector database to construct two sets of vector indexes, one based on document summary embedding (vector embedding generated from document summary) and the other based on the content of concatenated text of all levels of headings in the entire document (vector embedding generated from concatenated text of all levels of headings in the document). The two sets of indexes are stored separately and retrieved independently.

[0063] The third type relies on graph databases to build document relationship graphs, which record document associations in the form of triples of subject document, relation predicate, and object document. The relationship types include reference, dependency, comparison, supplement, and association. An additional evidence association edge, evidence_link (evidence_link, a graph edge identifier used to record the logical relationship between different document evidence fragments), is set to record the logical relationship between different document evidence fragments.

[0064] The entire processing logic can improve several shortcomings of conventional search enhancement generation technology, such as mismatch between search strategies and query needs, rigid reasoning processes, isolated document information, reliance on vector similarity to filter documents, and lack of systematic credibility control.

[0065] See Figure 2 As shown, Figure 2The diagram illustrates the overall system architecture of a reliable answer generation method provided in Embodiment 1 of this application. The diagram shows a five-stage, four-layer architecture: query intent analysis → intent-driven dynamic retrieval → adaptive progressive reasoning → cross-document evidence chain reasoning → confidence calibration generation. The architecture is divided into four dimensions: overall process, logical hierarchy, data and knowledge support layer, and input / output. The overall process is divided into five stages: Stage 1: Query Intent Analysis (Intent Understanding); Stage 2: Intent-Driven Dynamic Retrieval (Resource Completion); Stage 3: Adaptive Progressive Reasoning (Resource Filtering); Stage 4: Cross-Document Evidence Chain Reasoning (Association Integration); and Stage 5: Confidence Calibration Generation (Reliable Output). These five stages correspond to the five core steps of the method in this application. The process involves: acquiring user queries and identifying query intent; configuring and executing retrieval strategies based on query intent to obtain candidate documents; performing hierarchical relevance analysis on the candidate documents based on the retrieval strategy to obtain several related documents; linking evidence from related documents across documents to form an evidence chain; and generating the final answer based on the hierarchical relevance analysis results and the evidence chain. The logic is divided into four layers: Layer 1 determines the retrieval strategy and parameters; Layer 2 multi-source retrieval and candidate recall; Layer 2.0 deep screening of document relevance; Layer 2.5 cross-document evidence association and integration; and Layer 3 confidence constraint-based answer generation. These layers correspond to the hierarchical execution logic of each step, from parameter configuration to reliable output. The knowledge support layer is a triplet storage engine group, serving as a unified document knowledge foundation. It consists of three parts: a 151 structured metadata index (relational database), a 152 dual-embedded semantic directory (vector database), and a 153 document relational knowledge graph (graph storage engine). The 151 structured metadata index (relational database) stores document titles, summaries, topic categories, entity lists, title level lists, file types, content hashes, etc., and supports SQL queries and filtering. The 152 dual-embedded semantic directory (vector database) sets up summary embedding (vector embeddings generated based on document summaries) and structural embedding (vector embeddings). Embedding (vector embedding based on concatenated text of document headings at all levels) and two sets of vector indexes enable joint retrieval, weighting, and similarity calculation. The 153-document relationship knowledge graph (graph storage engine) adopts document relationship triples in subject-predicate-object format, with relationship types including reference, dependency, comparison, supplement, and association. Evidence-link edges (graph edge identifiers used to record the logical relationships between different document evidence fragments) are set up, and relationship confidence and source are recorded. These three storage structures provide underlying data source support for multi-channel parallel retrieval and cross-document evidence association in this application. The right side of the architecture sets up a system general capability module, including a unified identification system (document / evidence / entity), a unified storage format (metadata / vector / graph), and a unified query interface (API (Application Programming Interface)). The system possesses four capabilities: Interface (Application Programming Interface), SQL (Graph Query), and unified security management (permissions, auditing, and data masking), ensuring the standardized implementation and operation of the entire system. On the input and output side, user queries in natural language form serve as the architecture input. After five stages of processing, the final answer with confidence and citation information is output. At the same time, the final answer can be fed back to achieve result feedback and iterative optimization, further optimizing the processing effect of the entire application.

[0066] In an optional implementation, see Figure 3 As shown, Figure 3The flowchart of a query intent recognition method provided in Embodiment 1 of this application is shown, wherein recognizing the query intent of the user query includes steps S301-S302: Specifically, the intention to differentiate steps can be replaced by a variety of implementation methods. The original mode of classification based on multi-dimensional features combined with rules can be replaced by zero-shot classification of large models, supervised classification based on labeled data training of models such as BERT (Bidirectional Encoder Representations from Transformers) and RoBERTa (Robustly Optimized BERT Pretraining Approach), or automatic grouping based on historical query vector clustering.

[0067] The mapping rules used to match parameters can also be replaced with dynamic parameter tuning based on reinforcement learning and adaptive parameter adjustment based on historical retrieval data statistics.

[0068] S301: Perform multidimensional feature extraction on the user query to obtain the target features.

[0069] Specifically, the target feature is the feature vector formed after fusing multi-dimensional information. Content is extracted from three aspects: lexical, sentence structure, and entity information. All features are quantized and then fused to serve as input data for subsequent classification operations.

[0070] S302: Based on the target features, classify the user query to obtain the query intent, wherein the query intent includes factual, comparative analysis, causal reasoning, and comprehensive summary types.

[0071] Specifically, fact-based queries are mostly used for inquiries about a single entity attribute, comparative analysis queries focus on comparing the similarities and differences of multiple types of objects, causal reasoning queries focus on analyzing the causes and effects of events, and comprehensive summary queries require integrating multiple sources to complete a summary and generalization.

[0072] The classification results are directly linked to four configuration parameters: retrieval weight, recall, inference depth, and cross-document switch.

[0073] See Figure 4 As shown, Figure 4This paper illustrates a full-link processing logic flowchart provided in Embodiment 1 of this application. The flowchart fully demonstrates the entire processing logic from the user's original query input to the output retrieval strategy configuration parameters. The overall flow is as follows: S001 Multidimensional Feature Extraction → S002 Intent Classification → S003 Strategy Configuration Table Generation. S001 Multidimensional Feature Extraction is used to extract features from the query text Q, extracting features from three dimensions: lexical, structural, and entity. Lexical features include interrogative words, comparison words, causal words, time words, quantifiers, keywords, and stop words. Structural features include sentence structure (simple / compound / parallel), number of subquestions, and query length. Entity features include named entity recognition, technical terminology recognition, and entity type statistics. After the multidimensional features are fused, a feature vector is obtained. S002 Intent Classification Based on Feature Vectors Determine query intent category I, classifying queries into four categories: Factual: Querying specific factual information, typically in simple sentences containing clearly defined entities and attributes, e.g., "What is the registered capital of Company XX?"; Comparative: Querying the similarities and differences between two or more objects, e.g., "Differences between Option A and Option B"; Causal: Querying causal relationships or impact analysis, e.g., "The impact of this policy change on the market"; and Comprehensive: Requires integrating information from multiple documents for a comprehensive summary, e.g., "Summarizing industry development trends over the past three years." The S003 strategy configuration table generates retrieval strategy configuration parameters based on intent category I, recording the channel weight vectors corresponding to different query intents through the RetrievalStrategy table. Candidate number K, inference depth D (maximum number of stages), cross-document inference flags The parameters corresponding to the fact query are: (Judgment based on outline only) =No, the corresponding parameters are compared and analyzed. (Outline + Evidence Extraction) =Yes, the parameters corresponding to causal reasoning are (Outline + Evidence Extraction) =Yes, the corresponding parameter for the comprehensive review is: (Outline + Evidence + Close Reading of Paragraphs) =Yes, it is marked in the parameter description. Weighting of the structured metadata retrieval channel. For dual-embedded semantic retrieval channel weights, Here, K represents the weight of the retrieval channel during knowledge graph traversal, K is the number of candidate documents to be recalled (Top-K), and D is the maximum number of stages for adaptive inference (1 / 2 / 3). Whether to enable cross-document evidence chain reasoning; the process relies on mapping functions. The final output is the retrieval strategy configuration parameters. This is for use by the dynamic retrieval module in the next stage; the flowchart is attached: 1. S001: Perform multi-dimensional feature extraction on query Q to form a feature vector. 2.S002: Based on 3. Determine the query intent category I using an intent classifier; S003: Use a mapping function. 4. Generate retrieval configuration parameters; these parameters will drive the subsequent dynamic retrieval and adaptive reasoning process; at the same time, a feedback optimization link is set up, which can optimize intent classification and parameter mapping based on the result feedback (continuous learning and adaptation). The configuration parameters output in this step are the key input basis for the subsequent step of "configuring the retrieval strategy according to the query intent and executing the retrieval to obtain candidate documents", realizing the differentiated configuration of retrieval rules based on query intent.

[0074] In an optional implementation, see Figure 5 As shown, Figure 5 The flowchart of a candidate document determination method provided in Embodiment 1 of this application is shown, wherein the step of configuring a retrieval strategy according to the query intent and performing a retrieval to obtain candidate documents includes steps S501 to S503: Specifically, the storage media on which the three types of search channels rely can be flexibly replaced. Structured search can be replaced with PostgreSQL, Elasticsearch, and OpenSearch, which are equipped with full-text indexes. Vector search can be replaced with vector engines such as FAISS (Facebook AI Similarity Search), Milvus, Qdrant, and Weaviate. Graph search can be replaced with graph databases such as Neo4j and JanusGraph.

[0075] Multi-channel parallel retrieval can compensate for the shortcomings of single-vector retrieval, such as the tendency to produce spurious relevance and miss key documents.

[0076] S501: Generate a retrieval strategy configuration table based on the query intent, wherein the retrieval strategy configuration table includes retrieval channel weight allocation, inference depth parameters, and whether to enable cross-document inference flags.

[0077] Specifically, the core parameters of the configuration table are: Through differentiated weight allocation, fact queries focus on accurately matching entities using metadata retrieval channels, causal queries focus on mining document associations using graph channels, and comparison and summary queries focus on semantic retrieval to achieve broad recall.

[0078] S502: Schedule multiple retrieval channels to perform retrieval in parallel according to the retrieval strategy configuration table.

[0079] Specifically, the three retrieval channels operate independently and synchronously. The scoring rules within each channel, the 1.1x bonus for dual-embedded retrieval, the score conversion based on the number of hops for graph retrieval, and the deduplication logic for loop nodes are all executed according to predetermined rules.

[0080] S503: Merge and sort the search results of each channel to obtain the candidate documents.

[0081] Specifically, in addition to the fusion scheme of selecting the highest score from multiple channels and adding a fixed score of 0.1, other result fusion methods such as weighted average, learning ranking model, and large model meta-inference scoring can also be used.

[0082] After merging and sorting, the document is truncated according to parameter K, and the complete attribute information of the document is supplemented from the structured repository.

[0083] See Figure 6 As shown, Figure 6 This diagram illustrates a flowchart of query intent analysis and dynamic strategy selection provided in Embodiment 1 of this application. The flowchart corresponds to the implementation flow of this application: "Configure a retrieval strategy based on the query intent and execute the retrieval to obtain candidate documents." The diagram shows that the inputs to this module are query Q and retrieval strategy configuration parameters. The output is a list of location results, LocateResult (Top-K candidate documents and their sources); the left side of the module sets the input area for strategy parameters, and the input content includes... , With query Q, and weight explanation: Dynamic allocation is based on the query intent. This module sets up three independent retrieval channels in parallel. Channel one is for structured metadata query (locate_by_metadata), with corresponding weights. The following steps are executed sequentially: S101a Full-text Search: Fuzzy matching in fields such as title / summary / topic, with a score of 0.6 for a successful match; S102a Query Terminology Extraction: Extracting candidate words such as entity words / abbreviations / quoted terms; S103a Entity Matching Query: Matching document entities based on entity words, with a score of 0.5 for a successful match; S104a Topic Matching Query: Matching topic categories based on entity words, with a score of 0.4 for a successful match. Finally, a candidate result set is generated. Channel 2 is a dual-embedding semantic retrieval (locate_by_semantic), with corresponding weights. The process proceeds sequentially: S101b Query Vectorization: Vectorizing Q; S102b Dual-Path Similarity Retrieval: Retrieving Top-N results from both the abstract embedding index and the structure embedding index; S103b Merging and Weighting: Dual-path matching within the same document. Single-path hit: Calculate the similarity score to generate a candidate result set. Channel 3 is knowledge graph traversal retrieval (locate_by_graph), with corresponding weights. The following steps are executed sequentially: S101c Seed Node Selection: Select the top M document IDs from channels one and two as seed nodes; S102c Graph Traversal (BFS): Perform a breadth-first search in the document relationship graph with a depth ≤ 2 hops; S103c Hop Count Decay Scoring: Neighbor document score = 0.3 / hop count (1 hop 0.3, 2 hops 0.15), generating a candidate result set. After the candidate results from the three channels are aggregated, they enter the multi-source merge sorting (_merge_candidates) stage on the right. This involves sequentially executing S105 multi-source merge and bonus: merging by document ID and taking the highest weighted score from each channel; if multiple channels hit, an additional +0.1 multi-source hit bonus is added, and the hit source is recorded; S106 sorting and truncation: sorting by comprehensive score in descending order and taking the Top-K (K is given by the strategy parameter); and S107 supplementing metadata: supplementing complete metadata and source information from the structured metadata index, finally outputting the LocateResult, which is the Top-K candidate document list (document information + comprehensive score + source). The bottom of the figure indicates the channel score calculation method: final score. The channel weight is: The bottom right corner includes a note: Three channels execute in parallel, with weight vectors. Dynamically adjust the influence of each channel; multi-source hit bonus encourages consistent high-quality documents across channels. The final number of candidate documents is determined, impacting the cost and effectiveness of subsequent inference. This module leverages the configuration parameters output by the preceding intent recognition to achieve multi-dimensional differentiated retrieval, accurately recalling candidate documents and providing a document data source for the subsequent hierarchical relevance analysis steps in this application.

[0084] In an optional implementation, see Figure 7 As shown, Figure 7 The flowchart of a method for obtaining relevant documents provided in Embodiment 1 of this application is shown, wherein the step of performing hierarchical relevance analysis on the candidate documents based on the retrieval strategy to obtain a number of relevant documents includes steps S701 to S703: Specifically, the large model used in the inference stage can replace the GPT, Claude, Llama, Gemini and other series, while the embedded model can be a mainstream model such as BGE, E5, Cohere Embed.

[0085] By relying on dynamic reasoning depth and early termination mechanism, simple fact-based queries can reduce the amount of token (the basic unit of text segmentation) processing by more than 80%, thereby reducing the computational cost of model calls.

[0086] S701: Based on the reasoning depth parameter in the retrieval strategy configuration table, perform a relevance judgment on the candidate document that includes at least two reasoning stages, wherein the subsequent stages have more granular document analysis than the previous stages, and the reasoning stages include at least two of outline-level judgment, evidence extraction, and paragraph-level close reading.

[0087] Specifically, The document outline is analyzed in detail. It consists of a title, subheadings at all levels, topic tags, and the first 200 characters of the abstract. The model output includes four pieces of information: relevance labels, confidence scores, potential evidence sections, and reasons for judgment.

[0088] Added a new evidence extraction step and limited the text size of a single document to 8000 characters. Additional paragraph in-depth reading is enabled, only for those with a confidence level of less than [value missing]. The document is broken down into paragraphs and analyzed in detail.

[0089] S702: After each phase is completed, assess the number of directly relevant documents obtained and their average confidence level.

[0090] Specifically, continue to use The average confidence score of documents in each round is calculated, and the definitions of each parameter are kept consistent to quantify the overall quality of documents selected in this round.

[0091] S703: When the quantity and average confidence level both reach a preset threshold, the execution of subsequent stages is terminated in advance to obtain the aforementioned related documents.

[0092] Specifically, the quantity threshold is fixed at... The confidence threshold is fixed at 1. Once both conditions are met, the remaining reasoning process will terminate. Irrelevant documents will be removed, and the reasoning results will be sorted from high to low confidence.

[0093] See Figure 8 As shown, Figure 8This document illustrates a flowchart of an adaptive progressive relevance reasoning module provided in Embodiment 1 of this application. The flowchart corresponds to the step in this application of "performing hierarchical relevance analysis on the candidate documents based on the retrieval strategy to obtain several relevant documents." The module's inputs are a set of candidate documents and a reasoning depth control parameter D, defined as D=1: fast, D=2: standard, D=3: depth. The module internally sets up a progressively refined three-stage reasoning process: Stage 1 is coarse-grained outline reasoning (coarse-grained relevance check), outputting: relevance judgment + preliminary confidence; Stage 2 is fine-grained evidence reasoning (content matching / evidence extraction), outputting: evidence fragments + support score; Stage 3 is evidence-enhancing reasoning (refined reasoning analysis), outputting: precise evidence + confidence enhancement. After Stage 1 processing is completed, a coarse-screening termination checkpoint is entered, with the check judgment condition being... and When the judgment result is yes, the process proceeds to stage 2; when the judgment result is no, the overall reasoning process can be terminated in stage 1 for the D=1 (fast) configuration scenario. After stage 2 processing is completed, the process enters the fine screening termination checkpoint, and the judgment condition is also the same. and If the determination is yes, proceed to stage 3; if the determination is no, the entire reasoning process can be terminated in stages 1 and 2 for the D=2 (standard) configuration scenario; the D=3 (depth) configuration scenario requires the complete execution of all three stages; after the entire reasoning process is completed, the module outputs the sorted evidence set. + Confidence information. This module relies on the depth parameter D determined in the previous steps to achieve adaptive hierarchical filtering. When the filtering threshold is met, redundant inference is terminated early, irrelevant documents are finely removed, and finally, several relevant documents required for this application are selected.

[0094] In an optional implementation, see Figure 9 As shown, Figure 9 The flowchart illustrates a method for forming a chain of evidence according to Embodiment 1 of this application, wherein the step of cross-document association of evidence in several related documents to form a chain of evidence includes steps S901-S903: Specifically, there are multiple alternative solutions for evidence association and contradiction handling. The identification of logically related edges can be achieved using Natural Language Inference (NLI), Chain-of-Thought (CoT), or fixed rule templates.

[0095] The handling of contradictory content can be replaced by methods such as prioritizing the latest document, prioritizing authoritative sources, majority voting, and Bayesian probabilistic fusion. Relying on cross-document evidence processing, it is possible to achieve information complementarity among multiple documents, identification of conflicting content, and cross-document logical deduction.

[0096] S901: Take the evidence fragments in the aforementioned related documents as nodes, and establish edges according to the association between nodes to construct an evidence graph, wherein the edge types include shared entity edges, semantically similar edges, and edges representing logical associations.

[0097] Specifically, according to An evidence graph is constructed, strictly adhering to three types of rules—entity matching, vector similarity, and logical judgment—to generate corresponding related edges. The similarity threshold for semantic edges is fixed at a certain value. =0.75, the logical edge distinguishes four types of logical relations: support, contradiction, supplement, and deduction.

[0098] S902: Perform consistency mediation on the evidence nodes with contradictory relationships in the evidence graph, and update the confidence level of each node in the evidence graph according to the mediation result.

[0099] Specifically, the mediation process sequentially filters valid information based on three dimensions: publication time, source authority, and the amount of supporting evidence. The confidence level of the nodes corresponding to the conflicting content is lowered according to the final mediation conclusion to prevent contradictory information from being mixed into subsequent answers.

[0100] S903: In the updated evidence graph, starting from the seed node that matches the query intent, search for a sequence of evidence nodes that can form a complete reasoning path, and use the sequence of evidence nodes as the evidence chain.

[0101] Specifically, by means of Calculate the confidence level of a single link, and only retain those with a confidence level higher than [the specified value]. Links with an overlap of over 80% are selected and retained based on the threshold, and the final output is a complete chain of evidence data including source, confidence level, and conclusion on handling contradictions.

[0102] See Figure 10 As shown, Figure 10 This diagram illustrates a flowchart of a cross-document evidence chain reasoning module provided in Embodiment 1 of this application. This diagram corresponds to the implementation steps of this application: "Associating evidence from several related documents across documents to form an evidence chain." The input to this module is a set of evidence fragments (from step 3). The process first executes step 1: evidence graph construction. The constructed evidence graph contains three different types of association edges. The legend specifies that solid blue lines represent entity association edges (same entity), and dashed green lines represent semantic similarity edges (similarity > 1). The red dashed lines represent logical implications (contradictions / support / guidance). After completing the evidence graph construction, execute the following steps in sequence: Step 2: Conflicting evidence identification; Step 3: Conflict mediation (timeliness / authority / evidence support assessment); Step 4: Evidence chain search (path discovery and scoring); Step 5: Link confidence calculation, where the formula for calculating link confidence is... After the entire process, the module finally outputs... +Confidence information. This process builds an evidence map through multi-dimensional correlation, identifies and reconciles contradictory evidence, retrieves valid reasoning paths and quantifies the credibility of the links, breaks down information barriers between different document evidence, completes cross-document evidence linkage and evidence chain generation, and provides complete and credible evidence for the subsequent answer generation stage of this application.

[0103] In an optional implementation, see Figure 11 As shown, Figure 11 The flowchart of a final answer generation method provided in Embodiment 1 of this application is shown, wherein generating the final answer based on the hierarchical correlation analysis and / or the evidence chain includes steps S1101~S1102: Specifically, the four-dimensional confidence assessment framework can be flexibly expanded to include two new assessment dimensions: information timeliness and evidence independence. Alternatively, it can eliminate the sub-item calculations and directly output the overall confidence score from the large model in one go.

[0104] By relying on the dual constraints of multi-dimensional confidence constraints and original evidence tracing, the illusion problem of unfounded content fabrication by the model can be effectively reduced.

[0105] S1101: Based on the hierarchical correlation analysis and / or the chain of evidence, calculate the overall confidence level, which includes at least the sufficiency of evidence dimension and the consistency of sources dimension.

[0106] Specifically, complete confidence level includes four sub-items: sufficiency of evidence, authority of the source, consistency of information, and logical completeness. The weighted calculation yields the overall confidence level, and the four weighting coefficients are fixed empirical values ​​determined through engineering debugging.

[0107] S1102: Generate the final answer according to the comprehensive confidence constraint answer generation process.

[0108] Specifically, the system controls the output of low-credibility content during generation. All expressions must rely on the original text evidence obtained from the search. If no valid information is found, the system will directly return a prompt and will not initiate the generation logic.

[0109] See Figure 12 As shown, Figure 12This diagram illustrates a flowchart of a confidence calibration answer generation module provided in Embodiment 1 of this application. This diagram corresponds to the implementation steps of this application: "generating the final answer based on the hierarchical correlation analysis and / or the evidence chain." The inputs to this module are the evidence chain input and evidence fragments. The first step of the process is Step 1: Four-dimensional confidence assessment, considering the sufficiency of evidence... Reliability of source Information consistency Integrity of reasoning The indicators are evaluated across four dimensions; then step 2 is performed: the comprehensive confidence score is calculated using the formula. The module calculates the overall confidence level of each individual piece of content. Based on the overall confidence level result, step 4 is executed: assertion confidence level annotation. An example annotation includes assertion 1 confidence level: 0.92, assertion 2 confidence level: 0.78, assertion 3 confidence level: 0.94, and several other assertions. Then, step 3, the LLM evidence constraint generation stage, is initiated. This stage generates the final answer (in segments / sections), assertion confidence levels, evidence citation list (document ID + fragment position), and endpoint descriptions (confidence level hints). After the entire process described above, the module finally outputs a structured output. This module relies on the relevant documents and complete evidence chain obtained in the preceding steps, quantifies content credibility through four-dimensional scoring, and uses confidence level values ​​to constrain the answer generation logic, effectively constraining the model from fabricating content out of thin air and improving the credibility of the final answer.

[0110] In an optional implementation, it also includes: A structured citation list is appended after the final answer. The citation list includes the identification information of the cited documents, the cited chapters, the confidence level, and a summary of the reasoning path of the chain of evidence.

[0111] Specifically, the citation list marks each document name, specific citation location, corresponding content confidence level, and reasoning process of cross-document evidence, making it convenient for users to verify the source and reliability of answer information and achieve full-chain traceability of answers.

[0112] Example 2 See Figure 13 As shown, Figure 13 This illustration shows a schematic diagram of a reliable answer generation device provided in Embodiment 2 of this application, wherein the device includes: The intent recognition module 1301 is used to acquire user queries and recognize the query intent of the user queries; The retrieval module 1302 is used to configure a retrieval strategy according to the query intent and perform a retrieval to obtain candidate documents; The hierarchical analysis module 1303 is used to perform hierarchical relevance analysis on the candidate documents based on the retrieval strategy to obtain several related documents; The cross-document association module 1304 is used to associate evidence in the several related documents across documents to form an evidence chain; The answer generation module 1305 is used to generate a final answer based on the hierarchical correlation analysis and / or the chain of evidence.

[0113] In an optional implementation, the intent recognition module specifically includes: The feature extraction submodule is used to perform multi-dimensional feature extraction on the user query to obtain the target features; The intent classification submodule is used to classify the user query based on the target features to obtain the query intent, wherein the query intent includes factual, comparative analysis, causal reasoning and comprehensive summary.

[0114] In an optional implementation, the retrieval module specifically includes: The configuration table generation submodule is used to generate a retrieval strategy configuration table based on the query intent. The retrieval strategy configuration table includes retrieval channel weight allocation, inference depth parameters, and whether to enable cross-document inference. The multi-channel retrieval submodule is used to schedule multiple retrieval channels to perform retrieval in parallel according to the retrieval strategy configuration table; The result merging submodule is used to merge and sort the search results of each channel to obtain the candidate documents.

[0115] In an optional implementation, the hierarchical analysis module specifically includes: The phased reasoning submodule is used to perform relevance judgment on the candidate document, which includes at least two reasoning stages, according to the reasoning depth parameter in the retrieval strategy configuration table. The subsequent stages have more granular document analysis than the previous stages. The reasoning stages include at least two of the following: outline-level judgment, evidence extraction, and paragraph-level close reading. The threshold verification submodule is used to evaluate the number of directly relevant documents obtained and their average confidence level after each stage is completed; The early termination submodule is used to terminate the execution of subsequent stages in advance when the quantity and average confidence level both reach a preset threshold, thereby obtaining the aforementioned related documents.

[0116] In an optional implementation, the cross-document association module specifically includes: The graph construction submodule is used to construct an evidence graph by taking evidence fragments from the several related documents as nodes and establishing edges based on the relationships between nodes. The edge types include shared entity edges, semantically similar edges, and edges representing logical relationships. The conflict mediation submodule is used to mediate the consistency of evidence nodes with contradictory relationships in the evidence graph, and update the confidence level of each node in the evidence graph according to the mediation result. The evidence chain retrieval submodule is used to search for a sequence of evidence nodes that can form a complete reasoning path in the updated evidence graph, starting from the seed node that matches the query intent, and to use the sequence of evidence nodes as the evidence chain.

[0117] In an optional implementation, the answer generation module specifically includes: The confidence calculation submodule is used to calculate the overall confidence level based on the hierarchical correlation analysis and / or the chain of evidence, wherein the overall confidence level includes at least the sufficiency of evidence dimension and the consistency of source dimension; The constraint generation submodule is used to generate the final answer based on the comprehensive confidence level constraint answer generation process.

[0118] In an optional implementation, the answer generation module is further configured to: A structured citation list is appended after the final answer. The citation list includes the identification information of the cited documents, the cited chapters, the confidence level, and a summary of the reasoning path of the chain of evidence.

[0119] Example 3 Based on the same application concept, see [link / reference] Figure 14 As shown, Figure 14 This illustration shows a structural schematic diagram of a computer device provided in Embodiment 3 of this application, wherein, as shown... Figure 14 As shown, the computer device 1400 provided in Embodiment 3 of this application includes: The computer device 1400 includes a processor 1401, a memory 1402, and a bus 1403. The memory 1402 stores machine-readable instructions executable by the processor 1401. When the computer device 1400 is running, the processor 1401 communicates with the memory 1402 via the bus 1403. The machine-readable instructions are executed by the processor 1401 to perform the steps of the reliable answer generation method shown in Embodiment 1 above.

[0120] Example 4 Based on the same concept, embodiments of this application also provide a computer-readable storage medium storing a computer program, which, when executed by a processor, performs the steps of the reliable answer generation method described in any of the above embodiments.

[0121] Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the specific working process of the system and apparatus described above can be referred to the corresponding process in the foregoing method embodiments, and will not be repeated here.

[0122] The computer program product for generating reliable answers provided in this application includes a computer-readable storage medium storing program code. The instructions included in the program code can be used to execute the methods described in the preceding method embodiments. For specific implementation details, please refer to the method embodiments, which will not be repeated here.

[0123] The reliable answer generation device provided in this application embodiment can be specific hardware on a device or software or firmware installed on the device. The implementation principle and technical effects of the device provided in this application embodiment are the same as those in the foregoing method embodiments. For the sake of brevity, any parts not mentioned in the device embodiment can be referred to the corresponding content in the foregoing method embodiments. Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the specific working processes of the systems, devices, and units described above can all be referred to the corresponding processes in the above method embodiments, and will not be repeated here.

[0124] In the embodiments provided in this application, it should be understood that the disclosed apparatus and methods can be implemented in other ways. The apparatus embodiments described above are merely illustrative. For example, the division of units is only a logical functional division, and in actual implementation, there may be other division methods. Furthermore, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Additionally, the displayed or discussed mutual couplings, direct couplings, or communication connections may be through some communication interfaces; indirect couplings or communication connections between devices or units may be electrical, mechanical, or other forms.

[0125] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0126] In addition, the functional units in the embodiments provided in this application can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit.

[0127] If the aforementioned functions are implemented as software functional units and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, or a portion of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of this application. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0128] It should be noted that similar labels and letters in the following figures indicate similar items. Therefore, once an item is defined in one figure, it does not need to be further defined and explained in subsequent figures. In addition, the terms "first", "second", "third", etc. are used only to distinguish descriptions and should not be construed as indicating or implying relative importance.

[0129] Finally, it should be noted that the above-described embodiments are merely specific implementations of this application, used to illustrate the technical solutions of this application, and not to limit them. The protection scope of this application is not limited thereto. Although this application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that any person skilled in the art can still modify or easily conceive of changes to the technical solutions described in the foregoing embodiments, or make equivalent substitutions for some of the technical features, within the scope of the technology disclosed in this application; and these modifications, changes, or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of this application. All should be covered within the protection scope of this application. Therefore, the protection scope of this application should be determined by the protection scope of the claims.

Claims

1. A method for generating reliable answers, characterized in that, include: Obtain user queries and identify the query intent of the user queries; Configure the retrieval strategy according to the query intent and execute the retrieval to obtain candidate documents; Based on the retrieval strategy, a hierarchical relevance analysis is performed on the candidate documents to obtain several relevant documents; The evidence in the aforementioned related documents is linked across documents to form a chain of evidence; The final answer is generated based on the hierarchical correlation analysis and / or the chain of evidence.

2. The method according to claim 1, characterized in that, The process of identifying the user's query intent includes: Multidimensional feature extraction is performed on the user query to obtain the target features; Based on the target features, the user query is classified to obtain the query intent, wherein the query intent includes factual, comparative analysis, causal reasoning, and comprehensive summary types.

3. The method according to claim 1, characterized in that, The step of configuring a retrieval strategy based on the query intent and executing the retrieval to obtain candidate documents includes: A retrieval strategy configuration table is generated based on the query intent, wherein the retrieval strategy configuration table includes retrieval channel weight allocation, inference depth parameters, and whether cross-document inference is enabled; Multiple retrieval channels are scheduled to perform retrieval in parallel according to the retrieval strategy configuration table; The search results from each channel are merged and sorted to obtain the candidate documents.

4. The method according to claim 3, characterized in that, The step of performing a hierarchical relevance analysis on the candidate documents based on the retrieval strategy yields several relevant documents, including: Based on the reasoning depth parameter in the retrieval strategy configuration table, a relevance judgment involving at least two reasoning stages is performed on the candidate documents, wherein the subsequent stages have more granular document analysis than the previous stages, and the reasoning stages include at least two of outline-level judgment, evidence extraction, and paragraph-level close reading. After each phase is completed, assess the number of directly relevant documents obtained and their average confidence level. When both the quantity and the average confidence level reach a preset threshold, the execution of subsequent stages is terminated early, and the aforementioned related documents are obtained.

5. The method according to claim 1, characterized in that, The step of linking evidence from the aforementioned related documents across documents to form a chain of evidence includes: Evidence fragments from the aforementioned related documents are used as nodes, and edges are established based on the relationships between nodes to construct an evidence graph. The edge types include shared entity edges, semantically similar edges, and edges representing logical relationships. Consistency mediation is performed on the evidence nodes in the evidence graph that have contradictory relationships, and the confidence level of each node in the evidence graph is updated according to the mediation result; In the updated evidence graph, starting from the seed node that matches the query intent, a sequence of evidence nodes that can form a complete reasoning path is searched, and the sequence of evidence nodes is taken as the evidence chain.

6. The method according to claim 1, characterized in that, The step of generating the final answer based on the hierarchical correlation analysis and / or the chain of evidence includes: Based on the hierarchical correlation analysis and / or the chain of evidence, a comprehensive confidence level is calculated, which includes at least the dimensions of sufficiency of evidence and consistency of sources. The final answer is generated based on the comprehensive confidence level constraint answer generation process.

7. The method according to claim 1, characterized in that, Also includes: A structured citation list is appended after the final answer. The citation list includes the identification information of the cited documents, the cited chapters, the confidence level, and a summary of the reasoning path of the chain of evidence.

8. A reliable answer generation device, characterized in that, include: The intent recognition module is used to acquire user queries and recognize the query intent of the user queries; The retrieval module is used to configure a retrieval strategy according to the query intent and execute the retrieval to obtain candidate documents; The hierarchical analysis module is used to perform hierarchical relevance analysis on the candidate documents based on the retrieval strategy to obtain several related documents; The cross-document association module is used to associate evidence from the aforementioned related documents across documents to form a chain of evidence; The answer generation module is used to generate a final answer based on the hierarchical correlation analysis and / or the chain of evidence.

9. A computer device, characterized in that, include: The computer device includes a processor, a memory, and a bus. The memory stores machine-readable instructions executable by the processor. When the computer device is running, the processor communicates with the memory via the bus. When the machine-readable instructions are executed by the processor, they perform the steps of the reliable answer generation method as described in any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a computer program that, when executed by a processor, performs the steps of the reliable answer generation method as described in any one of claims 1 to 7.

Images