Question answering method based on small model retrieval enhancement
By combining knowledge graph layer and vector layer hybrid retrieval, and employing Markov causal iteration and Markov blanket reordering methods, the problem of insufficient reasoning ability and illusion in reasoning question answering tasks of small-scale language models is solved, and efficient and accurate question answer generation is achieved in resource-constrained environments.
Patent Information
- Application Number
- CN202611588490.0
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2026-10-14
- Publication Date
- 2026-11-10
AI Technical Summary
Small-scale language models suffer from insufficient reasoning ability and are prone to illusion problems in reasoning question answering tasks. They are also difficult to deploy in resource-constrained environments, and existing improvement methods have limited effectiveness.
By combining knowledge graph layer and vector layer retrieval, and employing Markov causal iteration, Markov blanket reordering, and a small number of large models for assistance, this approach reduces illusions in generated responses and improves the accuracy and logical completeness of answers through multi-dimensional information fusion and closed-loop optimization.
It effectively reduces the illusion phenomenon in the response generated by small language models, improves the factual accuracy and logical completeness of the generated content, and is suitable for deployment in resource-constrained environments.
Smart Images

Figure SMS_5 
Figure SMS_6 
Figure SMS_7
Abstract
Description
Technical Field
[0001] This invention relates to the field of knowledge question answering technology, as well as the field of natural language processing and artificial intelligence technology, specifically a question answering method based on small model retrieval enhancement generation. Background Technology
[0002] Traditional retrieval-enhanced generative systems typically rely on large-scale language models as the core of generation. In recent years, with the significant improvement in the reasoning capabilities of large models, researchers have begun to incorporate reasoning abilities into retrieval-enhanced generative frameworks, enabling models to perform multi-step reasoning when faced with complex problems. For example, in multi-hop question answering tasks, models need to integrate and reason about multiple pieces of evidence from different documents to answer complex questions.
[0003] Despite the outstanding performance of large-scale models in reasoning and question answering tasks, their widespread application faces serious challenges: high inference costs, as the reasoning process of large-scale language models requires a large amount of computing resources, resulting in extremely high operating costs when deployed in real-world systems; limited deployability, as large-scale models have a huge number of parameters (usually reaching billions or even hundreds of billions), placing extremely high demands on hardware resources, making them difficult to implement in resource-constrained environments (such as edge devices and mobile devices) or security-sensitive environments (such as private clouds and local deployments); and significant latency issues, as the complete forward propagation of large-scale models in high-concurrency industrial retrieval systems can lead to severe computational redundancy and response latency.
[0004] To address these issues, researchers have begun exploring the use of small-scale language models to replace large models in retrieval enhancement and reasoning question answering. However, small-scale language models often exhibit relatively weak reasoning abilities, frequently resulting in reasoning illusions or answering illusions. Existing technologies primarily improve upon these issues in the following ways: First, intelligent fine-tuning. Novel fine-tuning methods no longer rely solely on a large number of standard answers but instead allow the model to learn "problem-solving guidance." This helps small models better understand the essence of the problem, improves generalization ability, and remains effective even with limited data. Second, reinforcement learning. By setting reward mechanisms, models autonomously learn the optimal reasoning path in a "trial-and-error" cycle. Experiments have shown that this significantly improves the reasoning ability of small models. Third, decoupling understanding and reasoning. Some studies propose separating the processes of "understanding the problem" and "reasoning based on understanding." A mapper transforms complex natural language problems into a standardized "problem space," allowing small models with limited parameters to focus on the core reasoning task, thereby improving performance.
[0005] However, the above methods still have shortcomings, resulting in poor or unstable answer quality. For example, after fine-tuning, the reasoning ability of small models is still limited, tending towards shallow pattern matching rather than true understanding. At the same time, they are prone to overfitting the training data, leading to a significant drop in performance in unseen scenarios. Another example is that the exploration ability of small models in reinforcement learning is very limited. If supervised fine-tuning is performed using data distilled from large models before reinforcement learning, the improvement will be very small. Furthermore, "decoupling" itself is a complex system engineering project. For example, iteratively training the decoupled understanding and reasoning framework requires a three-step iterative algorithm to collaboratively train the "mapper" and "inferencer." This complexity not only increases the training difficulty but may also introduce new unstable factors. Summary of the Invention
[0006] To address the aforementioned issues, the present invention aims to propose a question-answering method based on small-model retrieval enhancement. This invention combines a comprehensive technical solution that integrates knowledge graph layer and vector layer hybrid retrieval, Markov causal iteration, Markov blanket reordering, sub-problem decomposition causal thought chain, causal adversarial verification, and the use of a small number of large models for assistance. Through multi-dimensional information fusion and closed-loop optimization mechanisms, it effectively reduces the illusion phenomenon in the response generated by small language models and improves the factual accuracy and logical completeness of the generated content.
[0007] To achieve the above objectives, the present invention provides the following technical solution: A question-answering method based on small model retrieval enhancement is characterized by the following steps: S1. Determine the retrieval documents and questions based on the collected dataset, and transform the retrieval documents into a vector layer and a knowledge graph layer; S2. The question is initially retrieved in the vector layer, and the retrieved statements are multi-hop diffused in the knowledge graph layer to obtain the first candidate statement pool. S3. Construct a probability matrix based on the first candidate statements to represent the probability that each statement in the first candidate statement pool points to another statement. Then, perform Markov causal iteration to obtain the final causal influence value between each sentence in the first candidate statement pool and the question. Select the top sentences with the highest final causal influence values. The sentences are used to construct a second candidate statement pool; S4. Select statements in the second candidate statement pool that have high final causal impact values and contain keywords from the question as target statements. Determine the Markov blanket set of each target statement in the directed acyclic graph constructed based on the second candidate statement pool. Classify the target statements and statements in their Markov blanket sets as first-class statements, and the rest as second-class statements. Calculate the final retrieval score for each statement in the second candidate statement pool, and then re-rank them according to the final retrieval scores, selecting the top-ranked statements. The statements are sorted by score and used as the initial context; S5. Break down the problem into multiple sub-problems with sequential logic and causal evolution relationships; S6. The small model combines each sub-problem and the initial context to generate a causal thought chain, and then combines this causal thought chain with the question to generate the initial answer; S7. Perform adversarial testing on the initial answer. If the test passes, the initial answer is taken as the final answer. If it fails, the initial context is adjusted and updated. Then, return to step S6 to generate the initial answer again and perform the test. If it still fails, treat each sub-problem as a problem and repeat steps S2-S4 to obtain its corresponding initial context. Merge the initial contexts corresponding to each sub-problem to update the initial context. Then, return to step S6, regenerate the initial answer, and perform adversarial testing again. If the test still fails, return to step S5 to regenerate several causal sub-problems and continue iterating.
[0008] In one specific embodiment of the present invention, step S1, which transforms the retrieved document into a vector layer and a knowledge graph layer, specifically includes the following steps: S11. Divide the text in the retrieved document into blocks according to sentences to obtain a set of sentence text blocks; S12. Using a pre-trained text encoding model, each statement in the set of statement text blocks is mapped to a low-dimensional dense vector to obtain the vector layer of the retrieved document, thereby forming a "dense space" representation. S13. Extract semantic information from each statement in the set of statement text blocks and construct triples; score the causal relationships of the entity relations in the triples and construct weighted quadruples; combine all weighted quadruples into a knowledge graph layer with causal influence to form a "sparse space" expression.
[0009] In one specific embodiment of the present invention, step S2 includes the following steps: S21. Convert the problem into a vector, and query statements in the vector layer whose similarity to the vector is greater than a certain preset value as an initial statement set; S22. Extract entities from all sentences in the initial statement set, and expand these entities with adjacent nodes using the knowledge graph layer to obtain a batch of adjacent entities; extract the original statements of each of these adjacent entities to form an extended statement set; merge the initial statement set and the extended statement set to obtain the first candidate statement pool.
[0010] In one specific embodiment of the present invention, step S3, constructing the probability matrix includes the following steps: constructing a dual-track adjacency matrix based on the first candidate statement pool, wherein the track-one matrix is a dynamic context matrix used to characterize the first candidate statement pool. The continuity of different statements at the physical and semantic levels; the orbital matrix is a knowledge graph matrix used to represent the first candidate statement pool. The causal value of different statements in the discrete space of the knowledge graph layer is determined; the dual-track adjacency matrix is weighted, fused, and normalized to obtain the probability matrix.
[0011] As one specific embodiment of the present invention. The expression for the orbital matrix is as follows: In the formula, Represents the orbital matrix; for The elements in the text represent statements. and statement Continuity at both the physical location and semantic levels; This indicates the number of statements in the first candidate statement pool; It is the fusion coefficient, with a value range of [0.4, 0.6]; Statement and semantic similarity; Statement and Cosine similarity; and Statements and ; Indicates a preset threshold; Indicates taking Norm; Statement and The physical location distance value of the retrieved document; Represents an exponential function; Indicates the position attenuation coefficient; This indicates the physical location number of the statement within the retrieved document. Indicates the paragraph number to which the sentence belongs; The expression for the orbital matrix is as follows: In the formula, Represents the orbital matrix; for The elements in the text represent statements. and statement Causal value at the discrete space level; Indicates satisfaction , and All triples of these three relations The average causal relationship score.
[0012] The expression for the probability matrix is as follows: In the formula, Indicates the dual-track fusion factor; Probability matrix The middle element represents the statement Once identified as the starting point of the iteration, the Markov causal iteration points to the next statement. The probability of; and All are comprehensive adjacency matrices The elements in the text represent statements. With statement Causal influence values and statements between them With statement The causal influence value between them.
[0013] In one specific embodiment of the present invention, the calculation formula for the Markov causal iteration in step S3 is as follows: In the formula, The damping coefficient is a constant. and They represent the first Wheel and First The causal vector after iteration is In the vector, the values of each element in the causal vector represent the first candidate statement pool. The causal impact values of each statement in the question; in, Let be the initial vector of the causal vector, and its calculation formula is as follows: In the formula, Indicate the problem The initial causal influence value with a statement in the first candidate statement pool; This indicates the first candidate statement pool. , , , and All of them are statements from the first candidate statement pool; Indicate the problem ; This represents the temperature coefficient.
[0014] In one specific embodiment of the present invention, the formula for calculating the final retrieval score in step S5 is as follows: In the formula, Statement The final search score; This indicates the statement after Markov causal iteration. The causal impact value between the problem and the value of the cause. Table Statements With a certain bullseye statement Cosine similarity; Statement and various bullseye statements The maximum value of the cosine similarity; Represents the affinity weight within the blanket; It is an external punishment factor; A set of statements representing the bullseye; This represents a set of Markov blankets.
[0015] In a specific embodiment of the present invention, in step S6, when constructing the causal thinking chain, the small model solves each sub-problem sequentially based on the input data including the initial context. Starting from solving the second sub-problem, the causal transmission evidence of the previous sub-problem is introduced as input. The causal transmission evidence of the sub-problem refers to the statement that can state the factual cause of the problem obtained by combining and transforming the sub-problem and its corresponding answer by the small model. Finally, the causal transmission evidence corresponding to each sub-problem is linked together to form a causal thinking chain.
[0016] In one specific embodiment of the present invention, step S7, the adversarial testing, includes the following steps: S71. Extract the key entities from the initial answer, and randomly replace the content in the initial context corresponding to the key entity with perturbation entities of the same part of speech but different semantics to form a perturbation context. The small model first combines each sub-question and the perturbation context to generate a causal thinking chain, and then combines the causal thinking chain and the question to generate an adversarial answer. S72. Extract the key entities of the initial answer and the opposing answer to verify the rationality of the reasoning. If the key entities of both change synchronously with the perturbation entity, the test is passed; otherwise, the test is not passed.
[0017] In a specific embodiment of the present invention, step S7 involves adjusting and updating the initial context, including the following steps: selecting the statement that performs entity perturbation in the adversarial test; deleting the statement if it does not belong to the first type of statement; otherwise, finding the target statement based on the directed acyclic graph, and including the target statement, its parent node statement, child node statement, and spouse node statement into the penalty statement set; penalizing and reducing the final retrieval score of the statements in the penalty statement set, and then reordering the statements in the second candidate statement pool according to the final retrieval score to generate the initial context; The formula for calculating the penalty points deduction is as follows: in: Represents a statement in the set of penalty statements. The updated final search score, It is the penalty coefficient; It is a statement Related to the original problem similarity, It is a statement The bullseye statement.
[0018] Beneficial effects: This invention constructs a dual-track adjacency matrix, heterogeneously fusing a sparse knowledge graph matrix expressing long-distance symbolic causal dependencies with a dense context matrix capturing local semantic affinity and positional constraints, thus achieving alignment between symbolic topology and dense semantic representation. This mechanism not only overcomes the bottleneck of sliding window retrieval in capturing cross-document common sense but also compensates for the structural sparsity of graph relation extraction by utilizing contextual smoothness, ultimately homogenizing heterogeneous association logic into a unified Markov transition probability matrix.
[0019] In this invention, the synergistic effect of Markov causal iteration and Markov blanket achieves the unification of "global dynamic diffusion" and "local causal convergence." At the global level, the system drives steady-state energy to propagate across texts in the fusion space, laying the topological foundation for multi-hop implicit reasoning. At the local level, the mechanism introduces Markov blanket boundaries, eliminates spurious correlation noise caused by topological centrality bias through conditional independence pruning, and simultaneously enriches the set of direct causal neighbors of the target variable. This architecture blocks semantic drift through probabilistic reconstruction of "divergence before convergence," delivering a highly deterministic minimum complete knowledge subset to the causal thought chain.
[0020] In this invention, adversarial testing serves as a "perturbation source," using semantic perturbation or counterfactual attacks to accurately detect cognitive vulnerabilities in the model at extreme boundaries. This verifies the resilience of the causal reasoning architecture against perturbation noise and ultimately provides a two-way closed-loop evaluation paradigm for constructing a secure large-scale model that is immune to illusions and logically rigorous. Detailed Implementation
[0021] The present invention will be further described in detail below with reference to the embodiments, but the implementation of the present invention is not limited thereto.
[0022] A question-answering method based on small model retrieval enhancement includes the following steps: S1. Extract text data from the dataset, separate the retrieval documents and questions from the text data, and transform the retrieval documents into vector layers and knowledge graph layers as a two-layer decoupled representation of "dense space" and "sparse space".
[0023] In this step, a vector layer and a knowledge graph layer are constructed based on the information in the retrieved documents. This includes the following steps: S11. Divide the text in the retrieved document into blocks according to sentences to obtain a set of sentence text blocks; In the formula, A collection of statement text blocks; To retrieve a specific statement in a document, the index is the statement number; S12. Using the pre-trained text encoding model mxbai-embed-large-v1, the set of sentence text blocks... Each statement in the document is mapped to a low-dimensional dense vector, resulting in a vector layer for the retrieved document. This creates a "dense space" expression; This represents the total dimension of the vector layer. Represents a collection of statement text blocks S The number of statements in the text This represents the dimension of a statement as a low-dimensional dense vector. S13, By analyzing the set of sentence text blocks S Semantic information is extracted from each statement to obtain entity and relation information, and triples are constructed. Using the large language model deepseek-v4-flash, causal relationships in the triples are scored according to prompting rules to obtain weighted quadruples. Finally, all weighted quadruples are combined into a knowledge graph layer with causal influence. This creates an expression of "sparse space"; In the formula, This indicates the constraints added when extracting entity information; Indicates from statement The set of triples extracted from; For each triple, the subscript in parentheses indicates the triple's index. For head entity, For tail entities, For relationship; It is a causal relationship score between the head entity and the tail entity, which is obtained by the model evaluating the logical relationship between the head entity and the tail entity. The stronger the causal relationship, the higher the score. For example, if words such as "cause" or "affect" appear in the relation words, the score will be higher. Statement The corresponding set of weighted quadruples, with the index being the number of the statement in the retrieved document.
[0024] S2. Perform a preliminary search on the question at the vector layer, and then transfer the retrieved statements to the knowledge graph layer. The first candidate statement pool is obtained by performing multi-hop diffusion. .
[0025] S21. Convert the question into a question vector using a pre-trained text encoding model, and query statements in the vector layer that have a similarity greater than a preset threshold (0.6) to the question vector as the initial statement set; In the formula, A vector representing the problem; Indicates the problem; This represents the text encoding function in the pre-trained text encoding model, used to map text to vectors; The dimension is ; S22. Extract entities from all sentences in the initial sentence set, and utilize the knowledge graph layer. KG These entities are expanded by neighboring nodes to obtain a batch of neighboring entities; the original statements containing each of these neighboring entities are extracted through reverse mapping to form an expanded statement set; the initial statement set and the expanded statement set are merged to obtain the first candidate statement pool. S' .
[0026] S3. Construct a probability matrix based on the first candidate statements to represent the probability that each statement in the first candidate statement pool points to another statement. Then, perform Markov causal iteration to obtain the final causal influence value between each sentence in the first candidate statement pool and the question. Select the top sentences with the highest final causal influence values. The sentences are used to construct a second candidate statement pool; In this step, constructing the probability matrix includes the following steps: S311, Based on the first candidate statement pool Construct a two-track adjacency matrix, where the track-one matrix is the dynamic context matrix. Used to characterize the first candidate statement pool The continuity of different statements at the physical and semantic levels; the orbital matrix is a knowledge graph matrix. Used to characterize the first candidate statement pool The causal value of different statements in the discrete space of the knowledge graph layer; This step constructs a dual-track adjacency matrix, transforming two originally incompatible relational logics into a unified mathematical language within the discrete "symbolic common sense space" and the continuous "semantic text space".
[0027] The expression for the orbital matrix is as follows: In the formula, Denotes the orbital matrix, which is: The matrix, Indicates the first candidate statement pool Number of statements in the text; for The elements in the text represent statements. and statement Continuity at both the physical location and semantic levels; It is the fusion coefficient, with a value range of [0.4, 0.6]; Statement and semantic similarity; Statement and Cosine similarity; and Statements and ; This represents the preset threshold. During calculation, the first step is to calculate... If it is less than this threshold, then The value is set to 0, thereby removing noise smaller than the threshold. Indicates taking Norm; Statement and The physical location distance value of the retrieved document is used to protect the linear logic of the original text; Represents an exponential function; Indicates the position attenuation coefficient; This indicates the physical location number of the statement within the retrieved document. Indicates the paragraph number to which the sentence belongs; The expression for the orbital matrix is as follows: In the formula, This represents the orbital matrix, which is also... The matrix, for The elements in the text represent statements. and statement Causal value at the discrete space level; Indicates satisfaction , and All triples of these three relations The average causal relationship score.
[0028] S312. Weight and normalize the dual-track adjacency matrix established in S311 to obtain the probability matrix. To characterize the first candidate statement pool The probability that each statement in the text points to another statement; probability matrix The calculation formula is as follows; In the formula, This represents the dual-track fusion factor, with a value range of (0,1). Probability matrix The element in represents the statement Once identified as the starting point of the iteration, the turning statement in the Markov causal iteration. The probability of; and All are comprehensive adjacency matrices The elements in the text represent statements. With statement Causal influence values and statements between them With statement The causal influence value between them; In this step, the calculation formula for the Markov causal iteration is as follows: In the formula, The damping coefficient is a constant. and They represent the first Wheel and First The causal vector after iteration is In the vector, the values of each element in the causal vector represent the first candidate statement pool. The causal impact values of each statement in the question; in, This is the initial vector for the causal vector, where each element represents a problem. With the first candidate statement pool The initial causal impact value of each statement in the formula is calculated as follows: In the formula, Indicate the problem ; Indicates the temperature coefficient. ; Indicate the problem Statement The similarity between the two vectors divided by a temperature coefficient Used to expand the problem With statement The initial causal influence value between them.
[0029] After iteration, the final causal vector is obtained, representing the first candidate statement pool. The final causal impact value between each statement and the question is determined; then, based on the final causal vector after iteration, the first candidate statement pool is... Sort the causal impact values of each statement in the question and filter out the statements with the highest values. The first few statements are used to obtain the second candidate statement pool. .
[0030] S4, Based on the second candidate statement pool Construct a directed acyclic graph; pool the second candidate statements. Statements with high final causal impact values and containing keywords from the question are designated as target statements. Their Markov blanket sets are determined using the directed acyclic graph (DAG). The target statements and statements within their Markov blanket sets are classified as first-class statements, and the rest as second-class statements. The final retrieval scores for each statement in the second candidate statement pool are calculated, and then the statements are re-ranked based on their final retrieval scores, with the top-ranked statements selected. The statements are sorted by score and used as the initial context for the model to generate answers.
[0031] In step S4, the text underwent initial causal-oriented screening using a Markov causal iterative algorithm. However, the resulting text set may still suffer from information redundancy, insufficient coverage, or loose internal structure. These issues weaken the efficiency of the small model in utilizing context, thereby inducing inference bias. To address this, this step introduces a Markov blanket causal ranking mechanism. This mechanism aims to identify and rank key texts that are conditionally independent of the target variable and complementary in information, starting from the causal graph structure. This yields a subset of information that is statistically independent and complete, providing the small model with clearly structured and semantically refined inference support. The specific operation is as follows: S41. Extract key information about the problem using the small model llama3:8b, such as keywords related to location, people, and specific events, and then use the second candidate statement pool. The three statements with the highest causal impact values and containing at least one keyword are used as the bullseye statements of the Markov blanket, and a set of bullseye statements is constructed. S42. Utilize a large language model to analyze the second candidate statement pool. Information is extracted from the sentences to determine the logical relationships between them, and a directed acyclic graph is constructed. This ensures that no text is mutually causal during subsequent sorting; S43. Based on the target statement determined in step S41 and the directed acyclic graph determined in step S52. By classifying and filtering, a set of Markov blankets is obtained. Specifically, Markov blanket collection It includes three types of statement sets, namely , and , It is the set of parent nodes of the bullseye statement. It is the set of child nodes of the bullseye statement, that is, the bullseye statement in the acyclic graph. The pointer in the text points to the next statement, and the relationship is causal. It is an acyclic graph The set of other statements that share children with the target statement; S44. Move the second candidate statement pool The statements within the second candidate statement pool are divided into two categories: the target statement and statements in its Markov blanket set constitute the first category, while the remaining statements constitute the second category. For each statement in the second candidate statement pool, the appropriate model is selected based on the statement type to calculate its final retrieval score. The statements are then re-ranked based on their final retrieval scores, and the top-ranked statements are selected. The statement serves as the initial context for the model to generate the answer; The final retrieval score is calculated using the following model: In the formula, Statement The final search score; This indicates the statement after Markov causal iteration. The causal influence value between the problem and the solution is obtained from the final result of the S43 iteration; Statement With a certain bullseye statement Cosine similarity; Statement and various bullseye statements The maximum value of the cosine similarity; This represents the affinity weight within the blanket, adjusting the affinity of dense spatial semantics for closely related sentences within the blanket; It is an external penalty factor that isolates statements not included in the blanket to reduce noise interference.
[0032] S5. Break down the problem into multiple sub-problems with sequential logic and causal evolution relationships; In practice, the prompt word project can be used to call the large language model DeepSeek-V4-Flash to process the input question. By logically decomposing the problem, multiple sub-problems with sequential logical relationships and causal evolution are generated. , Denotes the set of subproblems. This indicates a subproblem, with the subproblem number in parentheses.
[0033] S6. The small model combines each sub-problem and the initial context to generate a causal thought chain, and then combines this causal thought chain with the question to generate the initial answer; In practical implementation, the small model can adopt llama3:8b. When constructing the causal thought chain, the small model solves each sub-problem sequentially based on the input data including the initial context. Starting from solving the second sub-problem, the causal transmission evidence of the previous sub-problem is introduced as input (i.e., the input includes the initial context and the causal transmission evidence of the previous sub-problem). The causal transmission evidence of the sub-problem refers to the statement that can state the factual cause of the problem obtained by the small model by combining and transforming the sub-problem and its corresponding answer. Finally, the causal transmission evidence corresponding to each sub-problem is linked together to form a causal thought chain.
[0034] S7. Perform adversarial testing on the initial answer. If the test passes, the initial answer is taken as the final answer. If it fails, the initial context is adjusted and updated. Then, return to step S6 to generate the initial answer again and perform the test. If it still fails, treat each sub-problem as a problem and repeat steps S2-S4 to obtain its corresponding initial context. Merge the initial contexts corresponding to each sub-problem to update the initial context. Then, return to step S6, regenerate the initial answer, and perform adversarial testing again. If the test still fails, return to step S5 to regenerate several causal sub-problems and continue iterating.
[0035] In this step, the adversarial testing includes the following steps: S71. Extract the key entities in the initial answer, and randomly replace the content in the initial context corresponding to the key entity with perturbation entities of the same part of speech but different semantics to form a perturbation context. The small model first combines each sub-question and the perturbation context to generate a causal thinking chain, and then combines the causal thinking chain and the question to generate an adversarial answer. Both this step and S6 involve the operation of generating answers by combining the small model with various sub-questions and context. The main difference between the two lies in the different contexts used. This step uses a perturbed context after perturbing the initial context. Specifically, entity keywords can be replaced as needed during perturbation. For example, if an answer is London, the word "London" in the initial context statement can be changed to "Paris" during replacement.
[0036] S72. Extract the key entities of the initial answer and the adversarial answer to verify the rationality of the reasoning. If the key entities of both change synchronously with the perturbation entity (the perturbation entity appears in the key entities of the adversarial answer), it means that modifying the entities of the original context affects the answer, proving that the answer is obtained by correct reasoning. This answer can be used as the final generated answer output and passes the test. Conversely, if modifying the entities of the original context has no effect on the answer, it is determined that the initial answer has information noise interference and cannot pass the test.
[0037] In this step, the initial context is adjusted and updated, including the following steps: Clean or penalize the statements containing the information that has been perturbed by S71. Select the statement that was perturbed in step S71. If the statement does not belong to the first type of statement (the bullseye statement and the statement in its Markov blanket set), delete the statement. Otherwise, find the bullseye statement of the statement based on the directed acyclic graph (if the statement is itself a bullseye statement, it is directly used as the bullseye statement, or its corresponding bullseye statement is found through the Markov blanket set / directed acyclic graph). Include the bullseye statement and its parent node statement, child node statement and spouse node statement in the penalty statement set. Penalize and reduce the final retrieval score of the statements in the penalty statement set. Then, reorder the statements in the second candidate statement pool according to the final retrieval score to generate the initial context. The formula for calculating the penalty points deduction is as follows: in: Represents a statement in the set of penalty statements. The updated final search score, It is the penalty coefficient; It is a statement Related to the original problem similarity, It is a statement The bullseye statement.
[0038] To further illustrate the superiority of the method in the embodiments of the present invention, specific test examples are given below.
[0039] To verify the effectiveness of the method of this invention in reducing the reasoning illusion problem in small language models, this test uses the HotpotQA dataset for experimental evaluation. HotpotQA is a large-scale question-answering dataset for multi-hop reasoning. Its core feature is that questions must be answered through reasoning from multiple supporting documents, and it provides sentence-level supporting facts as reference answers. This dataset is particularly suitable for testing the method of this invention because it requires the model to have multi-step reasoning capabilities and provides fine-grained supporting cues to verify the factual accuracy of the answers.
[0040] Taking the question "In which city was the lead singer of a certain band born?" as an example, the standard RAG method only retrieved documents related to "a certain band" and identified the lead singer as A, but failed to retrieve further information about his birthplace, resulting in a missing or incomplete answer. When using the method of this invention, steps S1-S2 retrieve information about "lead singer A of a certain band" through vector layer retrieval, expand the associated entity "A" in the knowledge graph layer through multiple hops, find the statement containing the associated entity "A", and then merge all statements to obtain the first candidate statement pool; after constructing a dual-track adjacency matrix and performing Markov causal iteration sorting in step S3, the statement containing the band and A information has a higher final causal influence value and is retained in the second candidate statement pool; in step S4, the bullseye statement and its Markov blanket statement are found. For example, the bullseye statement is "A becomes the lead singer of a certain band", and its Markov blanket statement is A was born in New York (parent node), A holds a concert in his hometown (child node), and B accompanies A to hold a concert in his hometown (spouse node). In step S44, statements that are ranked lower and are irrelevant to the bullseye statement are removed to obtain the initial context; in step S5, the problem is decomposed into two related sub-problems. The question is, "Who is the lead singer of a certain band?" and "Which city was he born in?". Step S6 initializes the context to find that the lead singer is A, and A is in New York, resulting in the initial answer "New York". Step S7 performs an adversarial test on the initial answer "New York". The "New York" in the initial context is changed to "Paris", and the answer is generated again to obtain an adversarial answer. If the adversarial answer is "Paris", the test passes. If it is New York, the initial context is adjusted, and then the process returns to step S6 to generate the initial answer again and test it. If it still fails, steps S2-S4 are repeated for each sub-problem to obtain its corresponding initial context. The initial contexts corresponding to each sub-problem are merged to update the initial context, and then the process returns to step S6 to regenerate the initial answer and perform the adversarial test again. If the test still fails, the process returns to step S5 to regenerate several causally related sub-problems and continues iterating.
[0041] To verify the effectiveness of the present invention, this embodiment compares the performance of different enhancement generation methods. The main operations of each method are as follows: The basic Retrieval Enhancement Generation (RAG) process consists of two main stages: offline indexing and online querying. First, in the offline stage, large-scale external knowledge base documents are segmented into semantically complete text blocks. An embedding model is then used to encode these text blocks into dense vectors, which are then stored in a vector database to build an external knowledge index. When a user asks a question, the system enters the online stage. The user's question is also encoded into a query vector, and distances are calculated in the vector database using algorithms such as cosine similarity. The top-K text blocks most semantically relevant to the question are retrieved. Then, the original user question and these retrieved candidate text blocks are concatenated according to a specific template to form structured enhanced prompts. Finally, these prompts, incorporating external corroborating information, are input into a large language model, which generates accurate, specific, and traceable natural language answers based on the retrieved context.
[0042] Hybrid retrieval enhancement generation builds upon standard RAG by parallelizing sparse retrieval (keyword exact matching) and dense retrieval (semantic vector recall) through dual channels. The process involves three steps: First, offline dual-index construction is implemented, including building a BM25 inverted index and encoding document blocks into dense vectors stored in a vector library. Second, during online retrieval, user questions simultaneously trigger keyword and semantic searches, retrieving candidate results separately. Finally, a fusion ranking is introduced, employing the Reverse Rank Fusion (RRF) algorithm to normalize, weight, and reorder the dual-channel recall results, selecting the Top-K text blocks with the highest overall relevance. These Top-K text blocks are then concatenated with the question to create an enhanced prompt input to the large language model, generating the final answer.
[0043] The lightweight knowledge graph retrieval enhancement generation process is divided into two main stages: index building and online retrieval. Index building stage: First, the document is segmented into text blocks and stored in a vector database. Then, a large language model is used to extract entities and relations from each text block to build a lightweight knowledge graph. Finally, the entities and relations are simultaneously stored in both the graph database and the vector database, forming a two-layer index. Online retrieval stage: When a user asks a question, the system simultaneously performs vector retrieval (semantic matching) and graph retrieval (relational query based on the knowledge graph). Entity-level and relation-level information are integrated through a two-layer retrieval paradigm. The retrieved relevant knowledge graph subgraphs are concatenated with the text blocks to create enhanced hints, which are then input into the large language model to generate the final answer.
[0044] The comparative experimental data of different enhancement generation methods are shown in Table 1 below.
[0045] Table 1 Performance Comparison of Different Methods As shown in Table 1, the method of the present invention has high accuracy and reduces the problem of reasoning illusion in small language models.
[0046] The above description is only a preferred embodiment of the present invention, but the scope of protection of the present invention is not limited thereto. Any equivalent substitutions or modifications made by those skilled in the art within the scope of the technology disclosed in the present invention, based on the technical solution and inventive concept of the present invention, should be covered within the scope of protection of the present invention.
Claims
1. A question-answering method based on small model retrieval enhancement generation, characterized in that, Includes the following steps: S1. Determine the retrieval documents and questions based on the collected dataset, and transform the retrieval documents into a vector layer and a knowledge graph layer; S2. The question is initially retrieved in the vector layer, and the retrieved statements are multi-hop diffused in the knowledge graph layer to obtain the first candidate statement pool. S3. Construct a probability matrix based on the first candidate statements to represent the probability that each statement in the first candidate statement pool points to another statement. Then, perform Markov causal iteration to obtain the final causal influence value between each sentence in the first candidate statement pool and the question. Select the top sentences with the highest final causal influence values. The sentences are used to construct a second candidate statement pool; S4. Select the statements in the second candidate statement pool that have high final causal influence values and contain the keywords contained in the question as the target statements, and determine the Markov blanket set of each target statement in the directed acyclic graph constructed based on the second candidate statement pool. Statements in the target statement and its Markov blanket set are classified as Class I statements, and the rest as Class II statements. The final retrieval score is calculated for each statement in the Class II candidate statement pool, and then the statements are re-ranked based on their final retrieval scores, with the top-ranked statements selected. The statements are sorted by score and used as the initial context; S5. Break down the problem into multiple sub-problems with sequential logic and causal evolution relationships; S6. The small model combines each sub-problem and the initial context to generate a causal thought chain, and then combines this causal thought chain with the question to generate the initial answer; S7. Perform adversarial testing on the initial answer. If the test passes, the initial answer is taken as the final answer. If the test fails, the initial context is adjusted and updated. Then, return to step S6 to generate the initial answer again and test it. If the test still fails, take each sub-problem as a problem and repeat steps S2-S4 to obtain its corresponding initial context. Merge the initial contexts corresponding to each sub-problem to update the initial context. Then, return to step S6, regenerate the initial answer, and perform adversarial testing again. If the test still fails, return to step S5 to regenerate several causal subproblems and continue iterating.
2. The question-answering method based on small model retrieval enhancement according to claim 1, characterized in that, In step S1, the retrieved documents are transformed into vector layers and knowledge graph layers, specifically including the following steps: S11. Divide the text in the retrieved document into blocks according to sentences to obtain a set of sentence text blocks; S12. Using a pre-trained text encoding model, each statement in the set of statement text blocks is mapped to a low-dimensional dense vector to obtain the vector layer of the retrieved document, thereby forming a "dense space" representation. S13. Extract semantic information from each statement in the set of statement text blocks and construct triples; score the causal relationships of the entity relations in the triples and construct weighted quadruples; combine all weighted quadruples into a knowledge graph layer with causal influence to form a "sparse space" expression.
3. The question-answering method based on small model retrieval enhancement generation according to claim 1, characterized in that, Step S2 includes the following steps: S21. Convert the problem into a vector, and query statements in the vector layer whose similarity to the vector is greater than a certain preset value as an initial statement set; S22. Extract the entities of all sentences in the initial statement set, and use the knowledge graph layer to expand these entities with adjacent nodes to obtain a batch of adjacent entities; extract the original statements in which each of these adjacent entities belongs to form an extended statement set. The initial statement set and the expanded statement set are merged to obtain the first candidate statement pool.
4. The question-answering method based on small model retrieval enhancement generation according to claim 1, characterized in that, In step S3, constructing the probability matrix includes the following steps: constructing a dual-track adjacency matrix based on the first candidate statement pool, wherein the track-one matrix is a dynamic context matrix used to represent the first candidate statement pool. The continuity of different statements at the physical and semantic levels; the orbital matrix is a knowledge graph matrix used to represent the first candidate statement pool. The causal value of different statements in the discrete space of the knowledge graph layer is determined; the dual-track adjacency matrix is weighted, fused, and normalized to obtain the probability matrix.
5. The question-answering method based on small model retrieval enhancement according to claim 4, characterized in that, In step S3, The expression for the orbital matrix is as follows: In the formula, Represents the orbital matrix; for The elements in represent statements and statement Continuity at both the physical location and semantic levels; This indicates the number of statements in the first candidate statement pool; It is the fusion coefficient, with a value range of [0.4, 0.6]; Statement and semantic similarity; Statement and Cosine similarity; and Statements and ; Indicates a preset threshold; Indicates taking Norm; Statement and The physical location distance value of the retrieved document; Represents an exponential function; Indicates the position attenuation coefficient; This indicates the physical location number of the statement within the retrieved document. Indicates the paragraph number to which the sentence belongs; The expression for the orbital matrix is as follows: In the formula, Represents the orbital matrix; for The elements in the text represent statements. and statement Causal value at the discrete space level; Indicates satisfaction , and All triples of these three relations The average causal relationship score; The expression for the probability matrix is as follows: In the formula, Indicates the dual-track fusion factor; Probability matrix The middle element represents the statement Once identified as the starting point of the iteration, the Markov causal iteration points to the next statement. The probability of; and All are comprehensive adjacency matrices The elements in the text represent statements. With statement Causal influence values and statements between them With statement The causal influence value between them.
6. The question-answering method based on small model retrieval enhancement according to claim 5, characterized in that, In step S3, the calculation formula for the Markov causal iteration is as follows: In the formula, The damping coefficient is a constant. and They represent the first Wheel and the first The causal vector after iteration is In the vector, the values of each element in the causal vector represent the first candidate statement pool. The causal impact values of each statement in the question; in, Let be the initial vector of the causal vector, and its calculation formula is as follows: In the formula, Indicate the problem The initial causal influence value with a statement in the first candidate statement pool; This indicates the first candidate statement pool. , , , and All of them are statements from the first candidate statement pool; Indicate the problem ; This represents the temperature coefficient.
7. The question-answering method based on small model retrieval enhancement according to claim 1, characterized in that, In step S5, the final retrieval score is calculated as follows: In the formula, Statement The final search score; This indicates the statement after Markov causal iteration. The causal impact value between the problem and the value of the cause. Table Statements With a certain bullseye statement Cosine similarity; Statement and various bullseye statements The maximum value of the cosine similarity; Represents the affinity weight within the blanket; It is an external punishment factor; A set of statements representing the bullseye; This represents a set of Markov blankets.
8. The question-answering method based on small model retrieval enhancement according to claim 1, characterized in that, In step S6, when constructing the causal thought chain, the small model solves each sub-problem sequentially based on the input data including the initial context. Starting from solving the second sub-problem, the causal transmission evidence of the previous sub-problem is introduced as input. The causal transmission evidence of the sub-problem refers to the statement that can state the factual cause of the problem obtained by combining and transforming the sub-problem and its corresponding answer. Finally, the causal transmission evidence corresponding to each sub-problem is linked together to form the causal thought chain.
9. The question-answering method based on small model retrieval enhancement generation according to claim 1, characterized in that, In step S7, the adversarial test includes the following steps: S71. Extract the key entities in the initial answer, and randomly replace the content in the initial context corresponding to the key entity with perturbation entities of the same part of speech but different semantics to form a perturbation context. The small model first combines each sub-question and the perturbation context to generate a causal thinking chain, and then combines the causal thinking chain and the question to generate an adversarial answer. S72. Extract the key entities of the initial answer and the opposing answer to verify the rationality of the reasoning. If the key entities of both change synchronously with the perturbation entity, the test is passed; otherwise, the test is not passed.
10. The question-answering method based on small model retrieval enhancement generation according to claim 1, characterized in that, In step S7, the initial context is adjusted and updated, including the following steps: Select the statement that performs entity perturbation in the adversarial test. If the statement does not belong to the first type of statement, delete the statement. Otherwise, find the target statement of the statement based on the directed acyclic graph, and include the target statement and its parent node statement, child node statement and spouse node statement into the penalty statement set. The final retrieval scores of the statements in the penalty statement set are penalized and reduced. Then, the statements in the second candidate statement pool are reordered according to their final retrieval scores to generate the initial context. The formula for calculating the penalty points deduction is as follows: in: Represents a statement in the set of penalty statements. The updated final search score, It is the penalty coefficient; It is a statement Related to the original problem similarity, It is a statement The bullseye statement.
