Large language model reasoning enhancement system and method based on reliability evaluation
By introducing a large language model reasoning enhancement system with reliability assessment, the reliability of information pairs is optimized and iterative reasoning is performed, which solves the problems of interpretability and reliability of large language models in complex tasks and improves the accuracy and interpretability of the model in complex reasoning tasks.
Patent Information
- Application Number
- CN202511396137.8
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2025-09-28
- Publication Date
- 2026-01-06
AI Technical Summary
Large language models struggle to guarantee the interpretability of the reasoning process and the reliability of the results when faced with complex tasks. In particular, their generalization ability is weak in cross-domain tasks, which can easily lead to the breakage of the reasoning chain and the accumulation of errors, making it difficult to meet the needs of real-time decision-making.
A large language model reasoning enhancement system based on reliability assessment is introduced. Through information pair extraction module, reliability scoring module, information pair summarization module and iterative execution module, the reliability of information pairs is gradually optimized to form unique information pairs and reasoning is performed through the thinking chain reasoning module.
It significantly improves the accuracy and reliability of large language models in complex reasoning tasks, enhances logical completeness and interpretability, and meets the application requirements of high reliability fields.
Smart Images
Figure CN121279445A_ABST
Abstract
Description
Technical Field
[0001] This invention relates to the fields of natural language processing and large model optimization technology, specifically to a large language model reasoning enhancement system and method based on reliability assessment. Background Technology
[0002] In recent years, artificial intelligence technology has developed rapidly, and large language models have shown great application potential in tasks such as natural language processing, intelligent question answering, text generation, and complex reasoning. These models, pre-trained on massive amounts of text data, learn the deep structure and semantic rules of language, enabling them to generate coherent and natural text, as well as possess a certain degree of logical reasoning and contextual understanding. However, when faced with complex tasks requiring multi-step logical deduction, such as solving mathematical word problems and causal inference, large language models, relying on end-to-end direct output mechanisms, often struggle to guarantee the interpretability of the reasoning process and the reliability of the results. To overcome this bottleneck, the "thinking chain" technology has been introduced. This technology guides the model to deconstruct complex problems into logically coherent intermediate reasoning steps through carefully designed prompts, explicitly generating a step-by-step derivation process. This step-by-step thinking mechanism not only significantly improves the model's accuracy in tasks such as complex mathematical reasoning and symbolic computation but also makes the reasoning path traceable. Large language models, leveraging the step-by-step reasoning mechanism of the thinking chain, have significantly improved their reasoning accuracy and interpretability in professional scenarios such as medical diagnostic assistance and financial risk analysis.
[0003] However, the MindChain model heavily relies on manually designed prompt templates, resulting in weak generalization ability in cross-domain tasks. This easily leads to breaks in the reasoning chain and error accumulation, making it difficult to meet real-time decision-making needs. Furthermore, the inherent logical inconsistencies and factual illusions inherent in the probabilistic nature of model generation severely restrict the reliability of the results. Future research needs to overcome the bottleneck of reliance on manual prompts, develop automated reasoning path construction technology to improve efficiency, and integrate symbolic logic verification mechanisms to enhance robustness. This will enable the MindChain model to achieve safe and reliable large-scale applications in high-precision fields such as autonomous driving and precision medicine. Summary of the Invention
[0004] In view of the deficiencies in the prior art, the purpose of this invention is to provide a large language model reasoning enhancement system and method based on reliability assessment. This invention can improve the understanding ability of large language models and enhance the reasoning reliability of large language models in critical decision-making tasks.
[0005] To solve the above problems, the technical solution of the present invention is as follows:
[0006] A large language model reasoning enhancement system based on reliability assessment includes:
[0007] The information pair extraction module is used to obtain preliminary information. It extracts entities from the current problem using the large language model and generates descriptions of the corresponding entities under the conditions of the problem, forming preliminary information pairs to complete the initial information acquisition and processing for problem solving.
[0008] The reliability scoring module is used to evaluate the reliability of the information pair set. The score of each information pair is scored independently by the large language model in the problem scenario. The average score of all descriptions of the information pair is the reliability score of the information pair.
[0009] The information pair summary module is used to combine information pairs to improve the density of information in the problem. It selects the two information pairs with the lowest reliability scores from the information pair set and merges them through a summary method.
[0010] The iterative execution module, based on the reliability scoring module and the information pair summary module, iteratively executes the two modules until only one unique information pair remains in the information pair list;
[0011] The thought chain reasoning module is used to solve problems and obtain the final result. It inputs the unique information pair along with the original problem into the large language model for reasoning. The information pair is used as a pre-provided hint to break down the problem step by step and arrive at the reasoning explanation and the final answer.
[0012] Furthermore, the present invention also provides a method for enhancing reasoning in large language models based on reliability assessment, comprising the following steps:
[0013] The current problem is extracted from the large language model to generate the description of the corresponding entity under the condition of the problem, forming a preliminary information pair, and completing the initial information acquisition and processing for problem solving;
[0014] To assess the reliability of the information pair set, each information pair is scored independently by the large language model in the problem scenario. The average score of all descriptions of the information pair is the reliability score of that information pair.
[0015] Select the two information pairs with the lowest reliability scores from the list of all information pairs, merge them by summarizing, and iterate to obtain a unique information pair;
[0016] The unique information pair is input into the large language model along with the original question for reasoning. The information pair is used as a pre-condition to break down the question step by step and arrive at the reasoning explanation and the final answer.
[0017] Preferably, the step of extracting entities from the current problem using a large language model and generating descriptions of the corresponding entities under the conditions of the problem to form preliminary information pairs, thereby completing the initial information acquisition and processing for problem solving, specifically includes: extracting multiple entities from the current problem using a large language model; for each entity, the large language model also needs to generate a structured information description; and the entity and its corresponding information description are organized into information pairs.
[0018] Preferably, in the step of evaluating the reliability of the set of information pairs, where each information pair is scored independently by the large language model in the problem scenario, and the average score of all descriptions of the information pair is taken as the reliability score of that information pair, the reliability score is defined as the expected value of the model's probability distribution, as shown in the following formula:
[0019] V(IP t ) = E v～pLLM (v|T,Q,IP {1:n} [v]
[0020] In the formula, T represents the instruction, Q represents the question, and IP represents the problem. {1:n} pLLM(v|T,Q,IP) represents a set of information pairs. {1:n} ) represents a large language model given T, Q, and IP. {1:n} Under these conditions, the conditional probability distribution of information on reliability is considered; when the expected value cannot be precisely calculated, multiple sampling is used for approximation: from the conditional probability distribution pLLM(v|T,Q,IP) {1:n} k samples v are independently drawn from ) (i) The mean value is used as the reliability score, as shown in the following formula:
[0021]
[0022] In the formula, k is weighted according to the system accuracy and computational cost.
[0023] Preferably, in the step of selecting the two information pairs with the lowest reliability scores from all information pair lists, merging them through a summarization method, and iteratively executing to obtain a unique information pair, the summarization operation follows the probability maximization principle, as shown in the following formula:
[0024]
[0025] Where, p k IP represents the context probability of the k-th knowledge element given context x. a and IP b This is represented as the original information pair.
[0026] Preferably, the step of inputting a unique information pair along with the original question into the large language model for reasoning, using the information pair as a pre-provided hint, and breaking down the question step by step to arrive at the reasoning explanation and final answer, specifically includes: the input context x consists of two parts: question Q and hint T. Given the context, the model needs to generate the final result y. To generate an accurate result, the large language model needs to use a thought chain and perform correct reasoning through intermediate steps z. The prediction probability formula is defined as follows:
[0027] p(y|x=(T,Q))=p(y|x,z)·p(z|x)
[0028] Where p(y|x,z) represents the probability of generating the final result y given the input context x and intermediate inference step z, and p(z|x) represents the probability of generating the intermediate inference step z given the input context x. Given x, y and z are conditionally independent, and the intermediate inference step z can be considered as being generated independently of the final result y given the input x. The final inference process combines the unique information pair IP with the question, and its formal representation is as follows:
[0029] p(y|x=(T,Q,IP))=p(y|x,z)·p(z|x)
[0030] IP represents a unique information pair generated through iterative optimization.
[0031] Compared with the prior art, the beneficial effects of the present invention are as follows:
[0032] 1. This invention introduces a reasoning chain generation mechanism based on structured information pairs, employing an iterative summarization strategy to display the logical dependencies between modeling premises and conclusions at each stage of reasoning, effectively avoiding error accumulation and the propagation of illusions. By introducing a learnable logical consistency module, the intermediate reasoning steps of subsequent thought chain generation are verified multiple times, enhancing the model's ability to grasp semantic relationships. The final output is a logically complete and clearly defined reasoning chain, significantly improving the accuracy and reliability in complex reasoning tasks.
[0033] 2. This invention addresses the prominent problems of existing large language models in reasoning chains, such as logical inconsistencies, lack of interpretability, and uncontrollable intermediate states. Through a structured design of prompts, it proposes a complete method covering key aspects such as information pair construction, reasoning chain reinforcement, and interpretability enhancement. These improvements not only significantly enhance the model's performance in various complex reasoning tasks but also improve the stability and acceptability of the output results, providing a solid technical foundation for the application of artificial intelligence systems in security-sensitive fields.
[0034] 3. The thought chain reasoning method constructed in this invention can provide a logically clear and highly interpretable multi-step reasoning process for fields with high reliability requirements, such as finance, healthcare, and law. By introducing a structured reasoning chain generation and consistency verification mechanism, the model can effectively decompose complex problems, reveal the logical dependencies between modeling and reasoning steps, significantly improve generalization ability and decision stability in complex tasks, and meet the key requirements for interpretability and credibility of artificial intelligence systems in industrial scenarios. Attached Figure Description
[0035] Other features, objects, and advantages of the present invention will become more apparent from the following detailed description of non-limiting embodiments with reference to the accompanying drawings:
[0036] Figure 1 This is a module architecture diagram of the large language model reasoning enhancement system based on reliability assessment according to the present invention;
[0037] Figure 2 This is a flowchart of the large language model reasoning enhancement method based on reliability assessment of the present invention;
[0038] Figure 3 This is a schematic diagram of information extraction in this invention;
[0039] Figure 4 This is a schematic diagram of the reliability scoring in this invention;
[0040] Figure 5 This is a schematic diagram of iterative execution and thought chain reasoning in this invention. Detailed Implementation
[0041] The present invention will now be described in detail with reference to specific embodiments. These embodiments will help those skilled in the art to further understand the present invention, but do not limit the invention in any way. It should be noted that those skilled in the art can make several changes and improvements without departing from the concept of the present invention. These all fall within the protection scope of the present invention.
[0042] Specifically, this invention provides a large language model reasoning enhancement system based on reliability assessment, such as... Figure 1 As shown, the system includes an information pair extraction module 1, a reliability scoring module 2, an information pair summarization module 3, an iterative execution module 4, and a thought chain reasoning module 5.
[0043] The information extraction module 1 is used to obtain preliminary information. Based on the design prompts, the large language model extracts entities for the current problem and generates descriptions of the corresponding entities under the conditions of the problem, forming preliminary information pairs to complete the initial information acquisition and processing for problem solving.
[0044] Specifically, the information extraction logic is as follows: a large language model extracts multiple entities from the current problem; to ensure the relevance of the extraction, prompts are added during extraction for guidance. These prompts employ example-driven learning, providing two templated extraction cases. These templated instructions include entity extraction examples from related domain problems, enabling the large language model to learn relevant extraction criteria, such as semantic proximity and causal influence, through these examples; finally, the large language model imitates the template examples to extract the entity set. For each entity, the large language model also needs to generate a structured information description. These descriptions are typically specific to the problem for that entity, including defining attributes (such as physical laws), contextual constraints (such as applicable domains), and operational relationships (such as mathematical dependencies). The description of each entity is limited to a maximum of four information items to avoid computational redundancy; entities and their corresponding information descriptions are organized into information pairs.
[0045] The reliability scoring module 2 is used to evaluate the reliability of the information pair set. The score of each information pair is scored independently by the large language model in the problem scenario. Then, the average score of all descriptions of the information pair is taken as the reliability score of the information pair.
[0046] Specifically, the processing logic of the reliability score is as follows: the reliability score is used to quantify the quality and reliability of information pairs, and to assess their problem-solving potential and completeness. This mechanism evaluates the current information pair using a large language model, given all previous information pairs. The reliability score is defined as the expected value of the model's probability distribution, as shown in the following formula:
[0047] V(IP t ) = E v～pLLM (v|T,Q,IP }1:n} [v]
[0048] In the formula, T represents the instruction, Q represents the question, and IP represents the problem. {1:n} pLLM(v|T,Q,IP) represents a set of information pairs. {1:n} ) represents a large language model given T, Q, and IP. {1:n} Under these conditions, the conditional probability distribution of information on reliability is considered. Reliability scoring incorporates contextual dependencies into the evaluation process, ensuring that the score reflects not only a single deterministic outcome but also a holistic assessment of information quality. This expectation-based evaluation method integrates multiple factors influencing reliability (such as fluency, relevance, and completeness) into a unified score. When the expected value cannot be precisely calculated, multiple sampling is used for approximation: from the conditional probability distribution pLLM(v|T,Q,IP) {1:n} k samples v are independently drawn from ) (i) The mean value is used as the reliability score, as shown in the following formula:
[0049]
[0050] In the formula, k is weighted according to the system's accuracy and computational cost. The prompts T used to evaluate information pairs are divided into two categories: scalar-valued prompts: directly prompting the large language model to output a scalar value v between 0 and 1; and opinion-based prompts: prompting the large language model to generate opinion-based judgments (such as "completely reliable", "somewhat reliable", "somewhat unreliable", "unreliable") and convert them into corresponding numerical values (1, 0.67, 0.33, 0). The system will use different prompts depending on the type of question. Generally, mathematical logic reasoning questions commonly use scalar-valued prompts to select priorities with more precise numerical values, while common sense questions use opinion-based prompts to analyze relevance and determine the order.
[0051] The information pair summarization module 3 is used to combine information pairs to improve the density of information in the problem. It selects two information pairs with the lowest reliability scores from the information pair set and merges them through a summarization method. During the summarization process, carefully designed prompts are used to constrain the large language model to integrate the two information pairs into an integrated description. After the summary, the information pair format is still retained and the content is not limited to the original two information pairs.
[0052] Specifically, the logic for obtaining the information pair summary is as follows: given two information pairs, the large language model performs the summary. Through a customized prompt, the large language model is guided to generate a new information pair, in which the summarized information pair replaces the original two information pairs, because the new information pair contains both the original entity information and the descriptive content. The summary operation follows the probability maximization principle, as shown in the following formula:
[0053]
[0054] Where, p k IP represents the context probability of the k-th knowledge element given context x. a and IP b This is represented as the original information pair. In customized prompts, the large language model is required to generate new information pairs not only by copying and summarizing the descriptions of the two entities, but also by proposing new descriptions, which are represented as the associations between the original information pairs.
[0055] The iterative execution module 4, based on the reliability scoring module 2 and the information pair summary module 3, iteratively executes the two modules for several rounds until only a unique information pair remains in the information pair list.
[0056] The processing logic of the iterative execution is as follows: Iterative execution is a comprehensive operation combining the above-mentioned reliability assessment and information pair summarization: selecting the two information pairs with the lowest reliability scores, generatively synthesizing composite knowledge, and rating the reliability of the new information pairs. The core of the iterative process lies in achieving knowledge summarization by merging two low-reliability information pairs. In a given set of information pairs, the two with the lowest reliability scores are selected for merging, rather than selecting high-scoring pairs. This shift from low to high scores reflects the evolution of knowledge from incomplete information to complete information, embodying the process of deepening understanding. After merging, the new information pair replaces the original information pair, and the reliability score of the entire list is recalculated. This process continues to iterate until the condition is met: the information list converges to a single information pair (regardless of its reliability score) and merging operations can no longer be performed.
[0057] The thought chain reasoning module 5 is used to solve problems and obtain the final result. It inputs the unique information pair along with the original problem into the large language model for reasoning, uses the information pair as a pre-provided hint, and decomposes the problem step by step to obtain the reasoning explanation and the final answer to the problem.
[0058] The logic for obtaining the thought chain reasoning is as follows: the reasoning process of the large language model is regarded as an autoregressive generation task. Typically, the input context x consists of two parts: a question Q and a hint T. Given the context, the model needs to generate the final result y. To generate an accurate result, the large language model needs to use the thought chain and perform correct reasoning through intermediate steps z. The prediction probability formula is defined as follows:
[0059] p(y|x=(T,Q))=p(y|x,z)·p(z|x)
[0060] Where p(y|x,z) represents the probability of generating the final result y given the input context x and intermediate inference step z, and p(z|x) represents the probability of generating the intermediate inference step z given the input context x. Given x, y and z are conditionally independent. The intermediate inference step z can be considered as being generated independently of the final result y given the input x. In practice, information pair extraction, reliability assessment, information pair summarization, and iterative execution all contribute to the overall generation and do not directly depend on subsequent steps. Iterative summarization before inference, gradually optimizing information pairs based on the given problem, not only extracts relevant information but also deepens the understanding of the current problem's context and nuances. This allows the large language model to generate more accurate and context-aware information pairs for the final inference step by integrating information extraction and cognitive summarization. Finally, the unique information pair IP is combined with the problem for the final inference process, formally represented as:
[0061] p(y|x=(T,Q,IP))=p(y|x,z)·p(z|x)
[0062] Where IP represents the final information pair generated through iteration.
[0063] Furthermore, this invention also provides a method for enhancing reasoning in large language models based on reliability assessment, such as... Figure 2 As shown, the method includes the following steps:
[0064] S1: Extract entities from the current problem using the large language model and generate descriptions of the corresponding entities under the conditions of the problem, forming preliminary information pairs, and completing the initial information acquisition and processing for problem solving;
[0065] Specifically, Figure 3 This is a schematic diagram of information extraction in an embodiment of the present invention, such as... Figure 3 As shown, a large language model extracts multiple entities from the current problem. To ensure the relevance of the extraction, prompts are added during the extraction process. The prompts adopt example-driven learning and provide two templated extraction cases. These templated instructions are accompanied by entity extraction demonstrations of related domain problems, enabling the large language model to learn relevant criteria for extraction through examples, such as semantic proximity and causal influence. Finally, the large language model imitates the template examples to extract the entity set.
[0066] For each entity, the large language model also needs to generate a structured information description, which typically includes defining attributes, contextual constraints, and operational relationships. The description of each entity is limited to a maximum of four information items to avoid computational redundancy. Entities and their corresponding information descriptions are organized into information pairs. This step requires calling the large language model twice.
[0067] S2: Evaluate the reliability of the information pair set. The score of each information pair is scored independently by the large language model in the problem scenario. The average score of all descriptions of the information pair is the reliability score of the information pair.
[0068] Specifically, Figure 4 This is a schematic diagram of reliability scoring in an embodiment of the present invention, such as... Figure 4 As shown, the reliability score is used to quantify the quality and reliability of information pairs, assessing their problem-solving potential and completeness. This mechanism evaluates the current information pair using a large language model, given all previous information pairs. The reliability score is defined as the expected value of the model's probability distribution, as shown in the following formula:
[0069] V(IP t ) = E v～pLLM (v|T,Q,IP {1:n} [v]
[0070] In the formula, T represents the instruction, Q represents the question, and IP represents the problem. {1:n}pLLM(v|T,Q,IP) represents a set of information pairs. {1:n} ) represents a large language model given T, Q, and IP. {1:n} Under certain conditions, the conditional probability distribution of information on reliability is considered. Reliability scoring incorporates contextual dependencies into the evaluation process, ensuring that the score reflects not only a single deterministic outcome but also a holistic assessment of information quality. This expectation-based evaluation method integrates multiple factors affecting reliability (such as fluency, relevance, and completeness) into a unified score.
[0071] When the expected value cannot be precisely calculated, an approximation is made using multiple sampling: from the conditional probability distribution pLLM(v|T,Q,IP) {1:n} k samples v are independently drawn from ) (i) The mean value is used as the reliability score, as shown in the following formula:
[0072]
[0073] In the formula, k is weighted according to the system accuracy and computational cost. The prompts T used to evaluate the information pairs are divided into two categories: scalar value prompts: directly prompting the large language model to output a scalar value v between 0 and 1; opinion-based judgment prompts: prompting the large language model to generate opinion-based judgments (such as "completely reliable", "somewhat reliable", "somewhat unreliable", "unreliable") and convert them into corresponding numerical values (1, 0.67, 0.33, 0).
[0074] S3: Select the two information pairs with the lowest reliability scores from the list of all information pairs, merge them by summarizing, and iterate to obtain a unique information pair;
[0075] Specifically, the summary operation follows the principle of maximizing probability, as shown in the following formula:
[0076]
[0077] Where, p k IP represents the context probability of the k-th knowledge element given context x. a and IP b This is represented as the original information pair. In customized prompts, the large language model is required to generate new information pairs not only by copying and summarizing the descriptions of the two entities, but also by proposing new descriptions, which are represented as the associations between the original information pairs.
[0078] like Figure 5As shown, the reliability scoring and information pair summarization are iteratively executed several times, iterating until only a single information pair remains in the information pair list. The core of the iterative process lies in achieving knowledge aggregation by merging two low-reliability information pairs. In a given set of information pairs, the two with the lowest reliability scores are selected for merging, rather than selecting high-scoring pairs. This shift from low to high scores reflects the evolution of knowledge from incomplete to complete information, embodying the process of deepening understanding. After merging, the new information pair replaces the original information pair, and the reliability score of the entire list is recalculated. This process continues iterating until the condition is met: the information list converges to a single information pair (regardless of its reliability score) and no further merging operations can be performed.
[0079] S4: Input the unique information pair along with the original question into the large language model for reasoning. Use the information pair as a pre-provided hint to break down the question step by step and arrive at the reasoning explanation and the final answer.
[0080] Specifically, such as Figure 5 As shown, a unique information pair is input into the large language model along with the original question for reasoning. The information pair serves as a pre-provided cue, breaking down the question step by step to arrive at the reasoning explanation and the final answer. The reasoning process of the large language model is considered an autoregressive generative task. Typically, the input context x consists of two parts: the question Q and the cue T. Given the context, the model needs to generate the final result y. To generate accurate results, the large language model needs to utilize a thought chain and perform correct reasoning through intermediate steps z. The prediction probability formula is defined as follows:
[0081] p(y|x=(T,Q))=p(y|x,z)·p(z|x)
[0082] Where p(y|x,z) represents the probability of generating the final result y given the input context x and intermediate inference steps z, and p(z|x) represents the probability of generating the intermediate inference step z given the input context x. Given x, y and z are conditionally independent. The intermediate inference step z can be considered as being generated independently of the final result y given the input x.
[0083] Information pair extraction, reliability assessment, information pair summarization, and iterative execution all contribute to the overall generation without directly depending on subsequent steps. Iterative summarization before inference, progressively optimizing information pairs based on the given problem, not only extracts relevant information but also deepens the understanding of the current problem's context and nuances. This allows the large language model to generate more accurate and context-aware information pairs for the final inference step by integrating information extraction and cognitive summarization. Finally, the unique information pair IP is combined with the problem for the final inference process. Its formal representation is:
[0084] p(y|x=(T,Q,IP))=p(y|x,z)·p(z|x)
[0085] IP represents a unique information pair generated through iterative optimization.
[0086] In summary, the large language model reasoning enhancement system and method based on reliability assessment proposed in this invention can achieve the following core functions and technical advantages:
[0087] 1. Structured Information Pair Extraction: Based on the current problem input, entity recognition and attribute description generation are performed using a large language model. The model is guided to accurately extract key entities through an example-driven prompting mechanism. A structured description containing definitional attributes, contextual constraints, and operational relationships is generated for each entity, forming an initial set of information pairs, thereby achieving refined modeling and preliminary organization of problem knowledge elements.
[0088] 2. Context-aware reliability scoring: Given a global context, a large language model assesses the quality of each information pair and calculates its reliability score using the expected value modeling method. It supports two prompting modes: scalar value output and opinion-based judgment. The scoring can be approximated through multiple sampling, ensuring that the evaluation results are both accurate and robust, and providing a priority basis for subsequent knowledge fusion.
[0089] 3. Information Pair Summarization and Knowledge Fusion: Through customized prompts, the large language model is guided to semantically integrate two specified information pairs. This not only preserves the original entities and their descriptions, but also actively infers the potential relationships between them, generating a new, more informative composite information pair. This process follows the principle of maximizing probability, ensuring that the newly generated information generates more detailed content based on the original individuals in terms of semantic consistency and reasoning support capabilities.
[0090] 4. Low reliability priority iterative aggregation mechanism: The lowest score priority merging strategy is adopted. The reliability score and information pair summary operation are performed in a loop. Each time, the two information pairs with the lowest reliability in the current set are selected for merging. Gradually, the evolution from fragmented and incomplete knowledge to a highly condensed and semantically complete unified representation is realized. The iterative process continues until the information pair set converges to a unique item, and the knowledge condensation of the problem context is completed.
[0091] 5. Enhanced Reasoning Based on Optimized Thinking Chain: The unique information pairs generated iteratively are used as enhanced information, which together with the original question and reasoning instructions constitute the input context, guiding the large language model to gradually deduce the final answer through multi-step intermediate reasoning. This mechanism significantly improves the model's understanding of complex problems and logical coherence, and shows stronger accuracy and interpretability in tasks such as mathematical reasoning and causal analysis.
[0092] Specific embodiments of the present invention have been described above. It should be understood that the present invention is not limited to the specific embodiments described above, and those skilled in the art can make various changes or modifications within the scope of the claims, which do not affect the essence of the present invention. Unless otherwise specified, the embodiments and features described in this application can be arbitrarily combined with each other.
Claims
1. A large language model inference enhancement system based on reliability evaluation, characterized in that, The system comprises: An information pair extraction module for obtaining preliminary information, extracting entities from the current question by a large language model, generating descriptions of the corresponding entities under the condition of the question, forming preliminary information pairs, and completing the initial information acquisition and processing for solving the question; A reliability scoring module for evaluating the reliability of the information pair set, each information pair being scored independently by the large language model under the question scenario, and the average score of all descriptions of the information pair being the reliability score of the information pair; An information pair summarization module for improving the information density in the question by combining information pairs, selecting the two information pairs with the lowest reliability scores from the information pair set, and merging them by summarization; An iterative execution module based on the reliability scoring module and the information pair summarization module, iteratively executing the two modules until only one unique information pair is left in the information pair list; A thought chain reasoning module for solving the question to obtain the final result, inputting the unique information pair and the original question into the large language model for reasoning, using the information pair as a pre-prompt, and step-by-step reasoning of the question to obtain the reasoning explanation and the final answer of the question.
2. A large language model inference enhancement method based on reliability evaluation, characterized in that, The method comprises the following steps: Extracting entities from the current question by a large language model and generating descriptions of the corresponding entities under the condition of the question, forming preliminary information pairs, and completing the initial information acquisition and processing for solving the question; Evaluating the reliability of the information pair set, each information pair being scored independently by the large language model under the question scenario, and the average score of all descriptions of the information pair being the reliability score of the information pair; Selecting the two information pairs with the lowest reliability scores from the information pair list, merging them by summarization, and iteratively executing to obtain a unique information pair; Inputting the unique information pair and the original question into the large language model for reasoning, using the information pair as a pre-prompt, and step-by-step reasoning of the question to obtain the reasoning explanation and the final answer of the question.
3. The method of claim 2, wherein the method further comprises: The step of extracting entities from the current question by a large language model and generating descriptions of the corresponding entities under the condition of the question, forming preliminary information pairs, and completing the initial information acquisition and processing for solving the question, specifically comprises: extracting multiple entities from the current question by a large language model, for each entity, the large language model also needs to generate a structured information description, and the entity and its corresponding information description are organized as an information pair.
4. The large language model inference enhancement method based on reliability evaluation according to claim 2, characterized in that, In the step of evaluating the reliability of the information pair set, each information pair being scored independently by the large language model under the question scenario, and the average score of all descriptions of the information pair being the reliability score of the information pair, the reliability score is defined as the expected value of the model probability distribution, and the formula is as follows: V(IP t ) = E v～pLLM (v | T, Q, IP {1:n} )[v] where T represents instruction, Q represents question, IP {1:n} represents the set of information pairs, pLLM(v|T,Q,IP {1:n} ) represents the conditional probability distribution of the large language model on the reliability of the information pair given T, Q, IP {1:n} ; In the case where the expected value cannot be accurately calculated, multiple sampling is used for approximation: k samples v {1:n} are independently extracted from the conditional probability distribution pLLM(v|T,Q,IP (i) ), and the mean value is taken as the reliability score, as follows: In the formula, k is balanced according to the system accuracy and the calculation cost.
5. The method of claim 2, wherein the method further comprises: In the step of selecting the two information pairs with the lowest reliability scores from the information pair list, merging them by summarization, and iteratively executing to obtain a unique information pair, the summarization operation follows the principle of maximum probability, and the formula is as follows: where p k represents the context probability of the kth knowledge element given the context x, IP a and IP b are represented as a pair of original information.
6. The method of claim 2, wherein the method further comprises: The step of inputting the unique information pair together with the original question into the large language model for reasoning, taking the information pair as a pre-prompt, and step by step decomposing and reasoning the question to obtain the reasoning explanation of the question and the final answer, specifically includes: inputting the context x consisting of two parts: question Q and prompt T, after giving the context, the model needs to generate the final result y, in order to generate accurate results, the large language model needs to rely on the thinking chain and pass through the intermediate step z to correctly reason, and the prediction probability formula is defined as follows: p(y|x=(T,Q))=p(y|x,z)·p(z|x) Where p(y|x,z) represents the probability of generating the final result y under the condition of given input context x and intermediate reasoning step z, p(z|x) represents the probability of generating the intermediate reasoning step z under the condition of given input context x, y and z have conditional independence under the condition of given x, and the intermediate reasoning step z can be regarded as independent of the final result y generated under the condition of input x; the unique information pair IP is combined with the question to perform the final reasoning process, which is formally represented as: p(y|x=(T,Q,IP))=p(y|x,z)·p(z|x) Where IP represents the unique information pair generated by iterative optimization.