Adaptive reasoning method, device and equipment based on large and small model collaboration and medium
By introducing an adaptive reasoning method that combines small and large models into a large language model, the method automatically selects a verification strategy, utilizes the preliminary answer from the small model, and calls the large model for deep reasoning when necessary. This solves the problem of excessive resource consumption of the large language model and improves economy and scalability.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- PING AN TECH (SHENZHEN) CO LTD
- Filing Date
- 2026-01-09
- Publication Date
- 2026-04-17
AI Technical Summary
Large language models are inefficient in allocating computational resources for complex reasoning tasks, leading to excessive token consumption for simple problems, which affects economic efficiency and scalability. In particular, computational cost becomes a limiting factor in high-frequency commercial scenarios.
An adaptive reasoning method based on collaboration between small and large models is adopted. By inputting the question into the small model, a preliminary answer is obtained. The system automatically selects an immediate judgment strategy or a stepwise judgment strategy for verification based on the question. If the verification is successful, the answer from the small model is used; otherwise, the large model is used for deep reasoning. This ensures the accuracy of the final answer while reducing resource consumption.
This approach reduces resource consumption while maintaining answer accuracy, improves the economy and scalability of large language models, and significantly reduces computational costs, especially in high-frequency commercial scenarios.
Smart Images
Figure CN121882274A_ABST
Abstract
Description
Technical Field
[0001] This invention relates to the field of intelligent decision-making technology, and in particular to an adaptive reasoning method, apparatus, device, and medium based on the collaboration of large and small models. Background Technology
[0002] Large language models are currently used in various fields to answer user-input questions. For example, in the medical field, users input questions such as "headache, chest pain, need further treatment" in intelligent consultations. In the fintech field, users input questions such as "recommend car insurance," and large language models can then be used to perform complex reasoning.
[0003] However, current large language models face a core bottleneck in complex reasoning tasks. Inefficient allocation of computational resources leads to excessive consumption of tokens for simple problems. Simple arithmetic problems that could directly output the answer are forced to undergo multiple steps of reasoning, resulting in additional token consumption. Research shows that large models (such as Gemini-2.5-Pro) consume approximately 1300 more tokens when processing simple problems than small models (Qwen2.5-3B) at the same accuracy, clearly indicating significant room for optimization in resource allocation. This severely restricts the economy and scalability of large language models in practical applications, especially in commercial scenarios requiring high-frequency calls, where computational cost becomes a major constraint. Therefore, how to ensure model output accuracy while reducing resource consumption has become a pressing technical problem for those skilled in the art. Summary of the Invention
[0004] This invention provides an adaptive inference method, apparatus, device, and medium based on large and small model collaboration to solve the technical problem of high model output accuracy but excessive resource consumption.
[0005] Firstly, an adaptive inference method based on size-model collaboration is provided, including: Input the problem into the small model, and output the answer from the small model; Based on the problem described, the target verification strategy for the large model is determined; the target verification strategy is either an immediate judgment strategy or a stepwise judgment strategy. If the target verification strategy is a stepwise judgment strategy, then the large model is used to decompose the problem into multiple reasoning steps based on the question and the answer of the small model; from the multiple reasoning steps, one reasoning step is selected as the target reasoning step in a preset order. The target reasoning steps are verified to obtain a second verification result; If the second verification result is unsuccessful, then the question is input into the large model, and the large model's answer is output. If the second verification result is passed, then determine whether the current target reasoning step is the last reasoning step; If it is not the last reasoning step, then the next reasoning step is obtained as the target reasoning step; the step of verifying the target reasoning step to obtain the second verification result is executed. If it is the last reasoning step, then the answer from the smaller model is determined as the final answer.
[0006] Secondly, an adaptive inference device based on size-model collaboration is provided, comprising: The first output unit is used to input the question into the small model and output the answer from the small model; The first determining unit is used to determine the target verification strategy to be used by the large model based on the problem; the target verification strategy is an immediate judgment strategy or a stepwise judgment strategy. The decomposition unit is used to decompose the target verification strategy into multiple reasoning steps based on the question and the answer of the small model using the large model if the target verification strategy is a stepwise judgment strategy; and to select one of the reasoning steps as the target reasoning step from the multiple reasoning steps in a preset order. The first verification unit is used to verify the target reasoning steps and obtain a second verification result; The second output unit is used to input the question into the large model and output the answer of the large model if the second verification result is unsuccessful. The judgment unit is used to determine whether the current target reasoning step is the last reasoning step if the second verification result is passed. The acquisition unit is used to acquire the next reasoning step as the target reasoning step if it is not the last reasoning step; and to execute the step of verifying the target reasoning step to obtain a second verification result. The second determining unit is used to determine the small model answer as the final answer if it is the last reasoning step.
[0007] Thirdly, a computer device is provided, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor, when executing the computer program, implements the steps of the adaptive inference method based on size-model collaboration as described above.
[0008] Fourthly, a computer-readable storage medium is provided, the computer-readable storage medium storing a computer program, which, when executed by a processor, implements the steps of the adaptive inference method based on size model collaboration as described above.
[0009] In the above-described adaptive reasoning method, apparatus, device, and medium based on large and small model collaboration, a problem can be input into a small model, and the small model's answer can be output. Based on the problem, a target verification strategy for the large model is determined. The target verification strategy can be an immediate judgment strategy or a step-by-step judgment strategy. If the target verification strategy is a step-by-step judgment strategy, the large model is used to decompose the problem into multiple reasoning steps based on the problem and the small model's answer. From these multiple reasoning steps, one reasoning step is selected as the target reasoning step in a preset order. The target reasoning step is verified to obtain a second verification result. If the second verification result is a failure, the problem is input into the large model, and the large model's answer is output. If the second verification result is a success, it is determined whether the current target reasoning step is the last reasoning step. If it is not the last reasoning step, the next reasoning step is selected as the target reasoning step. The step of verifying the target reasoning step to obtain a second verification result is executed. If it is the last reasoning step, the small model's answer is determined as the final answer. In this invention, the problem is first input into a small model to obtain the small model's answer. Then, a target verification strategy is automatically selected based on the problem. This target verification strategy includes an immediate judgment strategy or a step-by-step judgment strategy. The immediate judgment strategy is suitable for simple problems and can reduce verification overhead; the step-by-step judgment strategy can ensure the reliability of verification for complex problems. Verification is performed using the target verification strategy. If the verification passes, the small model's answer is directly used as the final answer. If the verification fails, reasoning is performed using a large model to obtain the large model's answer, which is then used as the final answer. This ensures the accuracy of the final answer while reducing resource consumption. Attached Figure Description
[0010] To more clearly illustrate the technical solutions of the embodiments of the present invention, the drawings used in the description of the embodiments of the present invention will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.
[0011] Figure 1 This is a schematic diagram of an application environment for an adaptive inference method that combines large and small models in one embodiment of the present invention; Figure 2 A flowchart illustrating the adaptive inference method for big-small model collaboration provided in an embodiment of the present invention; Figure 3 An exemplary schematic diagram of an adaptive inference device for size model collaboration is shown according to some embodiments; Figure 4 This is a schematic diagram of the structure of a computer device according to an embodiment of the present invention; Figure 5 This is another structural schematic diagram of a computer device according to one embodiment of the present invention. Detailed Implementation
[0012] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of the present invention. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.
[0013] Large language models are currently used in various fields to answer user-input questions. For example, in the medical field, users input questions such as "headache, chest pain, need further treatment" in intelligent consultations. In the fintech field, users input questions such as "recommend car insurance," and large language models can then be used to perform complex reasoning.
[0014] However, current large language models face a core bottleneck in complex reasoning tasks. Inefficient allocation of computational resources leads to excessive consumption of tokens for simple problems. Simple arithmetic problems that could directly output the answer are forced to undergo multiple steps of reasoning, resulting in additional token consumption. Research shows that large models (such as Gemini-2.5-Pro) consume approximately 1300 more tokens when processing simple problems than small models (Qwen2.5-3B) at the same accuracy, clearly indicating significant room for optimization in resource allocation. This severely restricts the economy and scalability of large language models in practical applications, especially in commercial scenarios requiring high-frequency calls, where computational cost becomes a major constraint. Therefore, how to ensure model output accuracy while reducing resource consumption has become a pressing technical problem for those skilled in the art.
[0015] The adaptive inference method based on size model collaboration provided in this invention can be applied to, for example... Figure 1In this application environment, the client writes a question on the client side. The server inputs the question into a small model and outputs the answer from the small model. Based on the question, it determines the target verification strategy to be used by the large model. The target verification strategy is either an immediate judgment strategy or a step-by-step judgment strategy. If the target verification strategy is a step-by-step judgment strategy, the large model is decomposed into multiple reasoning steps based on the question and the answer from the small model. From the multiple reasoning steps, one reasoning step is selected as the target reasoning step in a preset order. The target reasoning step is verified to obtain a second verification result. If the second verification result is a failure, the question is input into the large model, and the large model's answer is output. If the second verification result is a success, it is determined whether the current target reasoning step is the last reasoning step. If it is not the last reasoning step, the next reasoning step is selected as the target reasoning step. The step of verifying the target reasoning step to obtain a second verification result is executed. If it is the last reasoning step, the answer from the small model is determined as the final answer. In this invention, the problem is first input into a small model to obtain the small model's answer. Then, a target verification strategy is automatically selected based on the problem. This target verification strategy includes an immediate judgment strategy or a step-by-step judgment strategy. The immediate judgment strategy is suitable for simple problems and can reduce verification overhead; the step-by-step judgment strategy can ensure the reliability of verification for complex problems. Verification is performed using the target verification strategy. If the verification passes, the small model's answer is directly used as the final answer. If the verification fails, reasoning is performed using a large model to obtain the large model's answer, which is then used as the final answer. This ensures the accuracy of the final answer while reducing resource consumption.
[0016] The client can be, but is not limited to, various personal computers, laptops, smartphones, tablets, and portable wearable devices. The server can be implemented using a standalone server or a server cluster consisting of multiple servers. The invention will now be described in detail through specific embodiments.
[0017] Please see Figure 2 As shown, Figure 2 A flowchart illustrating an adaptive inference method based on big-small model collaboration provided in an embodiment of the present invention includes the following steps S100-S800.
[0018] S100. Input the problem into the small model and output the answer of the small model.
[0019] In this embodiment, the small model can be a model with 3B parameters. The small model acts as a front-end processor to generate the corresponding small model answer based on the question. This lightweight inference model avoids complex calculations and focuses on fast response.
[0020] In the financial field, the question "How do I apply for car insurance?" can be input into a small model to output the corresponding answer. In the medical field, the question "Which department should I go to at the hospital for a headache?" can be input into a small model to output the corresponding answer.
[0021] S200. Based on the problem, determine the target verification strategy to be used in the large model; the target verification strategy is an immediate judgment strategy or a stepwise judgment strategy.
[0022] In this embodiment, to avoid directly using the small model's answer as the final answer, which would lead to inaccurate results, the large model uses a target verification strategy to verify the small model's answer, combining the small model's answer with the large model's verification capabilities. This embodiment introduces a dual-path verification mechanism: an immediate judgment strategy and a step-by-step judgment strategy.
[0023] In this embodiment, an immediate judgment strategy is used as the target verification strategy for simple problems, and a step-by-step judgment strategy is used as the target verification strategy for complex problems.
[0024] In one example, the specific implementation and hint design of the target validation strategy used for the large model are determined as follows: Prompt template: { Select an appropriate target verification strategy based on the characteristics of the problem: Question: [Question Text] Question types: [Mathematics / Logic / Common Sense, etc.] Complexity metrics: [Length / Number of concepts / Number of reasoning steps] Strategy Selection Guide: - Short questions with clear answers: Use an immediate judgment strategy - Multi-step reasoning problems: using a step-by-step judgment strategy - Ambiguity or complexity issues exist: default step-by-step judgment strategy. Output: [Immediate decision strategy / Step-by-step decision strategy] } In some embodiments, the step of determining the target verification strategy to be used for the large model based on the problem includes S201-S203.
[0025] S201. Determine the first type to which the problem belongs, and determine the first complexity index of the problem.
[0026] In this embodiment, the first type may include mathematical problems, logical problems, or common-sense problems. The first complexity index is used to characterize the complexity of the problem.
[0027] In some embodiments, the steps of determining the first type to which the problem belongs and determining the first complexity index of the problem include: Analyze the keywords in the problem; based on the keywords, determine the first type; the first type includes mathematical problems, logic problems, or common sense problems.
[0028] In some embodiments, a classifier may be used to analyze keywords in the question and determine a first type based on the keywords.
[0029] For example, keywords including square, equation, calculation, and / or percentage can identify a problem as a first type of mathematical problem. Alternatively, keywords involving mathematical formulas and numerical calculations can also identify a problem as a first type of mathematical problem.
[0030] Keywords including logical connectives such as reasoning, if, then, all and / or some can identify the first type as a logic problem.
[0031] Keywords related to daily life, physical phenomena, and historical events can be identified as common sense questions. In one example, the question is "Why is the sky blue?", which can be classified as a common sense question.
[0032] Determine the text length, number of concepts, and number of reasoning steps of the problem; and define the text length, number of concepts, and number of reasoning steps of the problem as the first complexity index.
[0033] In some embodiments, the text length of a question can be determined by counting the number of characters or words in the text of the question.
[0034] The conceptual data of a problem can be identified using natural language processing techniques to determine the number of technical terms or entity names appearing in the problem. For example, if the problem compares the roles of photosynthesis and cellular respiration, then photosynthesis and cellular respiration are two technical terms, and the conceptual data of the problem can be determined to be two.
[0035] The number of reasoning steps in a problem can be estimated based on its structure. For example, the question "What is the capital of China?" is a single-step problem, requiring one reasoning step. Single-step problems typically involve direct lookup or simple calculation. Another example is a multi-step problem: "Xiaoming is 10 years old this year, and his father's age is four times his age. How many years older will his father be than Xiaoming in five years?" This multi-step problem requires three steps: "Calculate the father's current age" -> "Calculate their ages in five years" -> "Calculate the age difference." In this case, the number of reasoning steps is determined to be three. Multi-step problems contain clear sequential, conditional, or causal relationships.
[0036] S202. Find a second type that is the same as the first type, and a second complexity index that is the same as the first complexity index, and the corresponding advantage verification strategy.
[0037] In this embodiment of the application, advantage verification strategies corresponding to different second types and different second complexity indicators are pre-determined based on a historical database. For example, advantage verification strategies corresponding to second type a and second complexity indicator A, second type a and second complexity indicator B, second type b and second complexity indicator B, and second type c and second complexity indicator C are pre-determined, etc.
[0038] S203. The advantage verification strategy is determined as the target verification strategy.
[0039] In this embodiment, the subsequent large model uses this objective verification strategy to verify the answer of the small model.
[0040] In some embodiments, the advantage verification strategy corresponding to different second types and different second complexity indicators can be determined in the following manner. The method further includes S900-S1100.
[0041] S900. Obtain a historical database, which includes multiple historical data. The historical data includes the second type to which the historical problem belongs, the second complexity index of the historical problem, the historical target verification strategy used by the large model when outputting the historical final answer corresponding to the historical problem, and historical result indicators. The historical result indicators include the total historical token consumption of the large model and the small model, and the historical confidence of the historical final answer.
[0042] For example, historical data includes a second type (e.g., mathematical type), a second complexity index, historical goal verification strategies (e.g., step-by-step judgment strategies), and historical index results (e.g., historical token consumption total of 800, historical confidence level of 0.92).
[0043] In this embodiment, the determination process for the second type, the second complexity index, and the historical target verification strategy is the same as that for the first type, the first complexity index, and the target verification strategy mentioned above. The difference is that the first type, the first complexity index, and the target verification strategy are obtained by executing the method in this embodiment based on the current problem, while the second type, the second complexity index, and the historical target verification strategy are obtained by executing the method in this embodiment based on historical problems.
[0044] In this embodiment of the application, the method further includes calculating the sum of the token consumption of the small model and the token consumption of the large model, and determining the sum as the total token consumption.
[0045] In this embodiment, both the small model and the large model consume tokens during the execution of the method in this application embodiment. The sum of the token consumption of the small model and the token consumption of the large model is the total token consumption. Correspondingly, during the execution of the method in this application embodiment based on historical problems, the sum of the token consumption of the small model and the token consumption of the large model is also calculated, and this sum is the historical total token consumption.
[0046] In this embodiment of the application, the confidence level of the final answer is also determined. Accordingly, during the execution of the method in this embodiment of the application based on historical questions, the confidence level of the final answer is also determined, which is the historical confidence level of the historical final answer.
[0047] In this embodiment, when the final answer is the answer from the small model, the confidence level of the final answer is the same as the confidence level when the small model outputs the answer from the small model. When the final answer is the answer from the large model, the confidence level of the final answer is the same as the confidence level when the large model outputs the answer from the large model.
[0048] In this embodiment of the application, the total amount of tokens consumed and the confidence level are used as the result indicators.
[0049] S1000. Based on the historical database, determine the cost-effectiveness of the historical target verification strategy corresponding to the second type and the second complexity index.
[0050] In some embodiments, the step of determining the cost-effectiveness of the historical target verification strategy corresponding to the second type and the second complexity index based on the historical database includes S1001-S1004.
[0051] S1001. Based on the historical database, determine the historical target verification strategy corresponding to the second type and the second complexity index.
[0052] In this embodiment, the historical database includes multiple historical data sets. Different historical data sets may have the same second type and the same second complexity index, but the historical target verification strategies may be different. The historical target verification strategies corresponding to the second type and the second complexity index may include an immediate judgment strategy and a step-by-step judgment strategy, or they may only include an immediate judgment strategy or only a step-by-step judgment strategy.
[0053] In this embodiment of the application, the historical database is continuously updated. The method further includes: Generate new historical data, which includes a first type, a first complexity index, a target verification strategy, and a result index; update the historical database using the new historical data.
[0054] It is understood that after each execution of steps S100-S800 in this embodiment, a first type, a first complexity index, a target verification strategy, and a result index are generated. The first type, the first complexity index, the target verification strategy, and the result index are then used as the second type, the second complexity index, the historical target verification strategy, and the historical result index in the new historical data. The historical database can then be updated using the new historical data (e.g., new historical data can be added to the historical database). Then, the cost-effectiveness of the historical target verification strategy corresponding to the second type and the second complexity index is reassessed, and a new, advantageous verification strategy is subsequently determined.
[0055] In some embodiments, after collecting a preset number of new historical data points, the preset number of new historical data points can be used as a historical database. This historical database can then be used to determine historical target verification strategies corresponding to the second type and the second complexity index, and subsequently, new advantageous verification strategies can be determined. For example, the preset number can be 100.
[0056] In this embodiment of the application, the advantage verification strategy can be continuously optimized by utilizing a constantly updated historical database, making the determined advantage verification strategy more accurate.
[0057] S1002. If the historical target verification strategy corresponding to the second type and the second complexity index includes an immediate judgment strategy and a step-by-step judgment strategy, then calculate the first average historical token consumption and the first average confidence level of the immediate judgment strategy corresponding to the second type and the second complexity index.
[0058] In some embodiments, the steps of calculating the first average historical token consumption and the first average confidence level of the instant judgment strategy corresponding to the second type and the second complexity index include: The statistics include the number of historical data where the historical target verification strategy is an instant judgment strategy, the sum of the total historical token consumption in the historical data where the historical target verification strategy is an instant judgment strategy, and the sum of the historical confidence levels in the historical data where the historical target verification strategy is an instant judgment strategy. The sum of the total historical token consumption is divided by the number to obtain the first average total historical token consumption. The sum of the historical confidence levels is divided by the number to obtain the first average confidence level.
[0059] For example, the second type is 'a', and the second complexity index is 'A'. The number of historical data points including the second type 'a' and the second complexity index 'A' is N. Among these N historical data points, the number of historical data points with the historical target verification strategy of immediate judgment is M, and the number of historical data points with the historical target verification strategy of step-by-step judgment is L, where M+L=N.
[0060] The sum of the total historical token consumption in the M historical data points for the historical target verification strategy (which is an immediate judgment strategy) is Q. The sum of the historical confidence scores in the M historical data points for the historical target verification strategy (which is an immediate judgment strategy) is W. Dividing the sum of the total historical token consumption Q by M yields the first average total historical token consumption. Dividing the historical confidence scores W by M yields the first average confidence score.
[0061] S1003. Based on the historical database, calculate the second average historical token consumption and the second average confidence level of the step-by-step judgment strategy corresponding to the second type and the second complexity index.
[0062] In this embodiment, the steps of calculating the second average historical token consumption and the second average confidence level of the stepwise judgment strategy corresponding to the second type and the second complexity index are similar to those of calculating the first average historical token consumption and the first average confidence level of the instantaneous judgment strategy corresponding to the second type and the second complexity index. Only the instantaneous judgment strategy needs to be changed to the stepwise judgment strategy.
[0063] S1004. Based on the first average historical token consumption total and the first average confidence level, and the second average historical token consumption total and the second average confidence level, determine the cost-effectiveness of the immediate judgment strategy and the cost-effectiveness of the step-by-step judgment strategy corresponding to the second type and the second complexity index.
[0064] In some embodiments, the step of determining the cost-effectiveness of the immediate judgment strategy and the cost-effectiveness of the stepwise judgment strategy corresponding to the second type and the second complexity index based on the first average historical token consumption total and the first average confidence level, and the second average historical token consumption total and the second average confidence level includes S10041-S10044.
[0065] S10041. Determine whether the first average confidence level and the second average confidence level are higher than the preset confidence level.
[0066] For example, the default reliability is set to 0.91.
[0067] S10042. If both the first average confidence level and the second average confidence level are higher than the preset confidence level, then compare the size of the first average historical token consumption total and the second average historical token consumption total; if the first average historical token consumption total is not greater than the second average historical token consumption total, then determine that the cost-effectiveness of the immediate judgment strategy is high; if the first average historical token consumption total is greater than the second average historical token consumption total, then determine that the cost-effectiveness of the step-by-step judgment strategy is high.
[0068] In this embodiment, if both the instantaneous judgment strategy and the step-by-step judgment strategy exceed the preset confidence level, it indicates that the accuracy of the final answer obtained using both strategies is high. At this point, to reduce resource consumption, the system continues to compare the first average historical token consumption and the second average historical token consumption to determine the strategy with lower token consumption when obtaining the final answer. The strategy with lower consumption is considered to have higher cost-effectiveness.
[0069] S10043. If the historical target verification strategy corresponding to the second type and the second complexity index only includes the immediate judgment strategy, then the cost-effectiveness of the immediate judgment strategy is determined to be high.
[0070] S10044. If the historical target verification strategy corresponding to the second type and the second complexity index only includes the step-by-step judgment strategy, then the cost-effectiveness of the step-by-step judgment strategy is determined to be high.
[0071] S1100. The cost-effective instant judgment strategy or the cost-effective step-by-step judgment strategy is determined as the advantage verification strategy corresponding to the second type and the second complexity index.
[0072] In this embodiment, the advantage verification strategy corresponding to the second type and the second complexity index is determined and stored.
[0073] S300. If the target verification strategy is a step-by-step judgment strategy, then the large model is used to decompose the problem and the small model answer into multiple reasoning steps; from the multiple reasoning steps, one reasoning step is obtained as the target reasoning step in a preset order.
[0074] In this embodiment, the large model can deduce multiple reasoning steps based on the question and the answer of the small model, following the logic of the small model.
[0075] S400. Verify the target reasoning step to obtain a second verification result.
[0076] In this embodiment, the large model verifies the target inference step to determine whether there are any illogical errors in the target inference step. If an error exists, the second verification result is determined to be unsuccessful. If no error exists, the second verification result is determined to be successful.
[0077] S500. If the second verification result is unsuccessful, the question is input into the large model, and the large model answer is output.
[0078] In the embodiments of this application, the large model performs step-by-step reasoning to obtain the large model answer, including: 1. Identify specific errors in the small model answer. The core purpose and subsequent applications of identifying specific errors in the small model answer are as follows: Core purpose: Precise positioning to avoid repeated errors: Purpose 1: Ensure the pertinence of corrections. Just as a doctor needs to accurately diagnose the cause of a disease before treating it, the large model must first clarify exactly where the answer of the small model is wrong, whether it is a factual error, a logical error, or a calculation error. This can ensure that subsequent reasoning does not waste time in the wrong direction but directly focuses on making corrections. Purpose 2: Improve the efficiency and quality of subsequent reasoning. By identifying specific errors, the large model can provide a clear "pit avoidance guide" and focus for generating the correct answer, making the newly generated reasoning chain more rigorous and efficient. Subsequent usage: The identified error information will be used as a key input to directly guide the subsequent steps: Guide the "development of the correct reasoning path": The large model will use the identified errors as "road signs" to deliberately avoid wrong reasoning branches. For example, if it is identified that the error of the small model is "incorrect use of the addition formula", then when developing a new path, the large model will pay special attention and correctly apply this formula. Use as a counterexample to strengthen the argument: When generating the final answer, the large model may mention the error of the initial answer during the reasoning process and explain why it is wrong and why the correct approach is reasonable. This is equivalent to providing a positive and negative comparison, making the final answer more persuasive and complete. 2. Develop the correct reasoning path. 3. Generate the final answer with detailed reasoning. 4. Ensure that the solution covers all aspects of the problem.
[0079] In one example, the specific implementation and prompt design of the large model outputting the large model answer are as follows: Prompt template: { After verifying that the small model answer is incorrect, perform comprehensive reasoning: Question: [Question text] Incorrect answer: [Small model answer] Perform step-by-step reasoning: 1. Identify specific errors in the small model answer 2. Develop the correct reasoning path 3. Generate the final answer with detailed reasoning 4. Ensure that the solution covers all aspects of the problem Output format: <Reasoning> [Detailed reasoning process] < / Reasoning> <Answer> [Final answer] < / Answer> For example, the question: "The length of a rectangle is 10 cm and the width is 5 cm. If the length is increased by 2 cm and the width is decreased by 1 cm, what are the perimeter and area of the new rectangle respectively?" The wrong answer of the small model: "The perimeter is 30 cm and the area is 40 square cm." (This answer may only calculate the perimeter or area, or there may be calculation errors) The large model's inspection process to ensure "covering all aspects": Decompose the question requirements: The large model will first list all the "aspects" required by the question: Aspect 1: Calculate the perimeter of the new rectangle.
[0080] Aspect 2: Calculate the area of the new rectangle.
[0081] Check the solution: Before generating the final answer, the large model will check its reasoning draft: "Is the new perimeter clearly calculated and given in my solution? (√ Covered)" "Is the new area clearly calculated and given in my solution? (√ Covered)" "Have I stated that the new length = 10 + 2 = 12 cm and the new width = 5 - 1 = 4 cm? (√ Covered, this is the calculation basis)" Format the output: After confirming that all aspects are covered, the large model will organize the final answer according to the required format to ensure clarity and accuracy.
[0082] Finally, the answer output by the large model will be like this, clearly covering all aspects of the question: <Reasoning> First, calculate the dimensions of the new rectangle: New length = original length + increase = 10 cm + 2 cm = 12 cm New width = original width - decrease = 5 cm - 1 cm = 4 cm Then, calculate the new perimeter: Perimeter = 2×(length + width) = 2×(12 cm + 4 cm) = 2×16 cm = 32 cm Finally, calculate the new area: Area = length × width = 12 cm × 4 cm = 48 square cm<00002S700 If it is not the last reasoning step, then obtain the next reasoning step as the target reasoning step; execute the step of verifying the target reasoning step to obtain the second verification result.
[0084] S800. If this is the last reasoning step, then determine the answer of the small model as the final answer.
[0085] In this embodiment, the large model decomposes the answer from the small model into reasoning steps and verifies the correctness of each step. The verification result determines the subsequent processing path. If the verification passes (i.e., the second verification result of all reasoning steps passes), the answer from the small model is adopted; if the verification fails, deep reasoning of the large model is initiated.
[0086] In one example, the step-by-step decision-making strategy implementation and prompt design are as follows: Prompt template: { Analyze the following answer step by step: Question: [Question Text] Answer: [Small model answer] Break down the answer into logical steps and evaluate each step: 1. Extract each reasoning step from the answer. 2. Determine the appropriate action for each step: - [Correct]: Logically sound and factually accurate. - [Error]: Contains an error or logical flaw. 3. Stop evaluating at the first [error] step. 4. If all steps are correct, output "Completely Correct". Output format: Step 1: [Step Details] - [Correct / Incorrect] … Final: [Completely correct / Terminated at step x] } In some embodiments, the method further includes: if the target verification strategy is an immediate judgment strategy, then using the large model to directly verify the question and the answer of the small model to obtain a first verification result.
[0087] In this embodiment, the large model evaluates the quality of the answer by verifying the answer from the small model in the following ways: content completeness: checking whether it covers all the requirements of the question; logical correctness: verifying whether the reasoning process is reasonable; factual accuracy: checking the factual information in the answer. For the immediate judgment strategy, the large model makes an overall evaluation directly based on the knowledge gained during training.
[0088] If the first verification result is passed, then the answer of the small model is determined to be the final answer; If the first verification result is unsuccessful, the question is input into the large model, and the large model answer is output; the large model answer is determined as the final answer.
[0089] In one example, the implementation and prompt design of the immediate judgment strategy are as follows: Prompt template: { Assess the following questions - answers are correct: Question: [Question Text] Suggested answer: [Small model answer] Perform binary verification: - If the answer is completely correct and covers all aspects of the question, output "Pass" (i.e., the first verification result is Pass). - If the answer contains any errors, omissions, or inaccuracies, output "Fail" (i.e., the first verification result is "Fail"). No explanation or further reasoning is required.
[0090] } In some embodiments, the step of directly verifying the question and the answer of the small model using the large model to obtain a first verification result includes: The question and the answer from the small model are concatenated into a prompt text, which is then input into the large model so that the large model outputs the first verification result.
[0091] The method in this application embodiment uses solving a mathematical problem as an example to illustrate the specific workflow: For example: Small model processing: Qwen2.5-3B generates the preliminary answer "√16=4"; Real-time judgment strategy: Gemini-2.5-Pro outputs "Pass" after verification; Result adoption: Return the answer directly, consuming approximately 50 tokens; Compared to the traditional method (direct inference of large models), which consumes approximately 200 tokens, this method saves 75%.
[0092] For example, for complex problems (such as calculus problems): Small model processing: Generates solutions that include incorrect steps; Step-by-step assessment: The large model detected the error "[Error]" in the third step; Deep reasoning: Initiate a large model for complete reasoning and generate the correct solution; Results returned: Ensure accuracy while avoiding complete reliance on large models.
[0093] The method in this embodiment establishes a hierarchical reasoning mechanism, allowing the system to fully leverage the respective advantages of large and small models: the small model handles efficient preprocessing, while the large model focuses on quality control and complex reasoning. The dual-path verification mechanism provides a trade-off between accuracy and efficiency, making immediate judgments suitable for simple problems to minimize overhead, and progressively determining the reliability for complex problems. In this embodiment, a superior verification strategy is automatically selected based on problem characteristics, and this strategy is continuously optimized through learning. These innovations make the application of large language models more economical and efficient, providing a feasible technical path for practical deployment.
[0094] As can be seen, in the above scheme, the problem is first input into a small model to obtain the small model's answer. Then, a target verification strategy is automatically selected based on the problem. The target verification strategy includes an immediate judgment strategy or a step-by-step judgment strategy. The immediate judgment strategy is suitable for simple problems and can reduce verification overhead; the step-by-step judgment strategy can ensure the verification reliability of complex problems. Verification is performed using the target verification strategy. If the verification passes, the small model's answer is directly used as the final answer. If the verification fails, reasoning is performed using a large model to obtain the large model's answer, which is then used as the final answer. This approach ensures the accuracy of the final answer while reducing resource consumption.
[0095] It should be understood that the sequence number of each step in the above embodiments does not imply the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of the present invention.
[0096] In one embodiment, an adaptive inference device based on big-small model collaboration is provided, which corresponds one-to-one with the adaptive inference method based on big-small model collaboration described in the above embodiments. For example... Figure 3 As shown, the adaptive inference device based on size model collaboration includes a first output unit 301, a first determination unit 302, a decomposition unit 303, a first verification unit 304, a second output unit 305, a judgment unit 306, an acquisition unit 307, and a second determination unit 308. Detailed descriptions of each functional module are as follows: The first output unit 301 is used to input the problem into the small model and output the answer of the small model; The first determining unit 302 is used to determine the target verification strategy used by the large model based on the problem; the target verification strategy is an immediate judgment strategy or a stepwise judgment strategy. The decomposition unit 303 is used to decompose the target verification strategy into multiple reasoning steps based on the question and the answer of the small model using the large model if the target verification strategy is a stepwise judgment strategy; and to obtain one of the reasoning steps as the target reasoning step from the multiple reasoning steps in a preset order. The first verification unit 304 is used to verify the target reasoning step and obtain a second verification result; The second output unit 305 is used to input the question into the large model and output the answer of the large model if the second verification result is unsuccessful. The judgment unit 306 is used to determine whether the current target reasoning step is the last reasoning step if the second verification result is passed. The acquisition unit 307 is used to acquire the next reasoning step as the target reasoning step if it is not the last reasoning step; and to execute the step of verifying the target reasoning step to obtain a second verification result. The second determining unit 308 is used to determine the small model answer as the final answer if it is the last reasoning step.
[0097] In some embodiments, the apparatus further includes: The second verification unit is used to directly verify the question and the answer of the small model using the large model if the target verification strategy is an instant judgment strategy, and to obtain the first verification result. The third determining unit is used to determine the small model answer as the final answer if the first verification result is passed; The fourth determining unit is used to input the question into the large model and output the large model answer if the first verification result is unsuccessful; and to determine the large model answer as the final answer.
[0098] In some embodiments, the second verification unit is specifically used for: The question and the answer from the small model are concatenated into a prompt text, which is then input into the large model so that the large model outputs the first verification result.
[0099] In some embodiments, the first determining unit includes: The fifth determining unit is used to determine the first type to which the problem belongs, and to determine the first complexity index of the problem; The search unit is used to find the second type that is the same as the first type, and the second complexity index that is the same as the first complexity index, and the corresponding advantage verification strategy; The sixth determining unit is used to determine the advantage verification strategy as the target verification strategy.
[0100] In some embodiments, the fifth determining unit is specifically used for: Analyze the keywords in the problem; based on the keywords, determine the first type; Determine the text length, number of concepts, and number of reasoning steps of the problem; and define the text length, number of concepts, and number of reasoning steps of the problem as the first complexity index.
[0101] In some embodiments, the apparatus further includes: An acquisition unit is used to acquire a historical database, which includes multiple historical data. The historical data includes the second type to which the historical problem belongs, the second complexity index of the historical problem, the historical target verification strategy used by the large model when outputting the historical final answer corresponding to the historical problem, and historical result indicators. The historical result indicators include the total historical token consumption of the large model and the small model, and the historical confidence of the historical final answer. The seventh determining unit is used to determine the cost-effectiveness of the historical target verification strategy corresponding to the second type and the second complexity index based on the historical database. The eighth determining unit is used to determine the cost-effective instant judgment strategy or the cost-effective step-by-step judgment strategy as the advantage verification strategy corresponding to the second type and the second complexity index.
[0102] In some embodiments, the seventh determining unit specifically performs the following: Based on the historical database, determine the historical target verification strategy corresponding to the second type and the second complexity index; If the historical target verification strategy corresponding to the second type and the second complexity index includes an immediate judgment strategy and a step-by-step judgment strategy, then calculate the first average historical token consumption and the first average confidence level of the immediate judgment strategy corresponding to the second type and the second complexity index. Based on the historical database, calculate the second average historical token consumption and the second average confidence level of the step-by-step judgment strategy corresponding to the second type and the second complexity index. Based on the first average historical token consumption total and the first average confidence level, and the second average historical token consumption total and the second average confidence level, determine the cost-effectiveness of the immediate judgment strategy and the cost-effectiveness of the step-by-step judgment strategy corresponding to the second type and the second complexity index.
[0103] This invention provides an adaptive inference device based on large and small model collaboration. First, the problem is input into the small model to obtain its answer. Then, a target verification strategy is automatically selected based on the problem. This target verification strategy includes an immediate judgment strategy or a step-by-step judgment strategy. The immediate judgment strategy is suitable for simple problems and can reduce verification overhead; the step-by-step judgment strategy can ensure the reliability of verification for complex problems. Verification is performed using the target verification strategy. If verification passes, the small model's answer is directly used as the final answer. If verification fails, inference is performed using the large model to obtain its answer, which is then used as the final answer. This approach ensures the accuracy of the final answer while reducing resource consumption.
[0104] Specific limitations regarding the adaptive inference device based on big-small model collaboration can be found in the limitations of the adaptive inference method based on big-small model collaboration mentioned above, and will not be repeated here. Each module in the aforementioned adaptive inference device based on big-small model collaboration can be implemented entirely or partially through software, hardware, or a combination thereof. These modules can be embedded in or independent of the processor in a computer device in hardware form, or stored in the memory of a computer device in software form, so that the processor can call and execute the operations corresponding to each module.
[0105] In one embodiment, a computer device is provided, which may be a server, and its internal structure diagram may be as follows: Figure 4 As shown, the computer device includes a processor, memory, network interface, and database connected via a system bus. The processor provides computational and control capabilities. The memory includes non-volatile and / or volatile storage media and internal memory. The non-volatile storage media stores the operating system, computer programs, and database. The internal memory provides an environment for the operation of the operating system and computer programs stored in the non-volatile storage media. The network interface is used to communicate with external clients via a network connection. When the computer program is executed by the processor, it implements server-side functions or steps of a size-model-based adaptive inference method.
[0106] In one embodiment, a computer device is provided, which may be a client, and its internal structure diagram may be as follows: Figure 5As shown, the computer device includes a processor, memory, network interface, display screen, and input devices connected via a system bus. The processor provides computational and control capabilities. The memory includes non-volatile storage media and internal memory. The non-volatile storage media stores the operating system and computer programs. The internal memory provides an environment for the operation of the operating system and computer programs stored in the non-volatile storage media. The network interface is used to communicate with an external server via a network connection. When executed by the processor, the computer program implements client-side functions or steps of a size-model-based adaptive inference method.
[0107] In one embodiment, a computer device is provided, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to perform the following steps: Input the problem into the small model, and output the answer from the small model; Based on the problem described, the target verification strategy for the large model is determined; the target verification strategy is either an immediate judgment strategy or a stepwise judgment strategy. If the target verification strategy is a stepwise judgment strategy, then the large model is used to decompose the problem into multiple reasoning steps based on the question and the answer of the small model; from the multiple reasoning steps, one reasoning step is selected as the target reasoning step in a preset order. The target reasoning steps are verified to obtain a second verification result; If the second verification result is unsuccessful, then the question is input into the large model, and the large model's answer is output. If the second verification result is passed, then determine whether the current target reasoning step is the last reasoning step; If it is not the last reasoning step, then the next reasoning step is obtained as the target reasoning step; the step of verifying the target reasoning step to obtain the second verification result is executed. If it is the last reasoning step, then the answer from the smaller model is determined as the final answer.
[0108] In one embodiment, a computer-readable storage medium is provided having a computer program stored thereon, the computer program performing the following steps when executed by a processor: Input the problem into the small model, and output the answer from the small model; Based on the problem described, the target verification strategy for the large model is determined; the target verification strategy is either an immediate judgment strategy or a stepwise judgment strategy. If the target verification strategy is a stepwise judgment strategy, then the large model is used to decompose the problem into multiple reasoning steps based on the question and the answer of the small model; from the multiple reasoning steps, one reasoning step is selected as the target reasoning step in a preset order. The target reasoning steps are verified to obtain a second verification result; If the second verification result is unsuccessful, then the question is input into the large model, and the large model's answer is output. If the second verification result is passed, then determine whether the current target reasoning step is the last reasoning step; If it is not the last reasoning step, then the next reasoning step is obtained as the target reasoning step; the step of verifying the target reasoning step to obtain the second verification result is executed. If it is the last reasoning step, then the answer from the smaller model is determined as the final answer.
[0109] It should be noted that the functions or steps that can be implemented by the computer-readable storage medium or computer device described above can be referred to the relevant descriptions on the server side and client side in the foregoing method embodiments. To avoid repetition, they will not be described one by one here.
[0110] Those skilled in the art will understand that all or part of the processes in the methods of the above embodiments can be implemented by a computer program instructing related hardware. The computer program can be stored in a non-volatile computer-readable storage medium, and when executed, it can include the processes of the embodiments of the above methods. Any references to memory, storage, databases, or other media used in the embodiments provided in this application can include non-volatile and / or volatile memory. Non-volatile memory can include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM), or flash memory. Volatile memory can include random access memory (RAM) or external cache memory. By way of illustration and not limitation, RAM is available in various forms, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), dual data rate SDRAM (DDRSDRAM), enhanced SDRAM (ESDRAM), synchronous link DRAM (SLDRAM), Rambus direct RAM (RDRAM), direct memory bus dynamic RAM (DRDRAM), and memory bus dynamic RAM (RDRAM), etc.
[0111] Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the above-described division of functional units and modules is used as an example. In practical applications, the above functions can be assigned to different functional units and modules as needed, that is, the internal structure of the device can be divided into different functional units or modules to complete all or part of the functions described above.
[0112] The software tools or components not belonging to our company that appear in the embodiments of this application are merely examples and do not represent actual use.
[0113] The above-described embodiments are only used to illustrate the technical solutions of the present invention, and are not intended to limit it. Although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present invention, and should all be included within the protection scope of the present invention.
Claims
1. An adaptive inference method based on size-model collaboration, characterized in that, include: Input the problem into the small model, and output the answer from the small model; Based on the problem described, determine the target validation strategy to be used for the large model; The target verification strategy can be an immediate judgment strategy or a step-by-step judgment strategy. If the target verification strategy is a stepwise judgment strategy, then the large model is used to decompose the problem into multiple reasoning steps based on the question and the answer of the small model; from the multiple reasoning steps, one reasoning step is selected as the target reasoning step in a preset order. The target reasoning steps are verified to obtain a second verification result; If the second verification result is unsuccessful, then the question is input into the large model, and the large model's answer is output. If the second verification result is passed, then determine whether the current target reasoning step is the last reasoning step; If it is not the last reasoning step, then the next reasoning step is obtained as the target reasoning step; the step of verifying the target reasoning step to obtain the second verification result is executed. If it is the last reasoning step, then the answer from the smaller model is determined as the final answer.
2. The method according to claim 1, characterized in that, Also includes: If the target verification strategy is an immediate judgment strategy, then the large model is used to directly verify the question and the answer of the small model to obtain the first verification result; If the first verification result is passed, then the answer of the small model is determined to be the final answer; If the first verification result is unsuccessful, then the question is input into the large model, and the large model's answer is output. The answer from the large model is determined as the final answer.
3. The method according to claim 2, characterized in that, The step of directly verifying the question and the answer of the small model using the large model to obtain the first verification result includes: The question and the answer from the small model are concatenated into a prompt text, which is then input into the large model so that the large model outputs the first verification result.
4. The method according to any one of claims 1-3, characterized in that, The steps for determining the target validation strategy for the large model based on the problem include: Determine the first type to which the problem belongs, and determine the first complexity index of the problem; Find a second type that is the same as the first type, and a second complexity index that is the same as the first complexity index, and the corresponding advantage verification strategy; The aforementioned advantage verification strategy is determined as the target verification strategy.
5. The method according to claim 4, characterized in that, The steps of determining the first type to which the problem belongs and determining the first complexity index of the problem include: Analyze the keywords in the problem; based on the keywords, determine the first type; Determine the text length, number of concepts, and number of reasoning steps of the problem; and define the text length, number of concepts, and number of reasoning steps of the problem as the first complexity index.
6. The method according to claim 4, characterized in that, Also includes: Obtain a historical database, which includes multiple historical data sets. The historical data sets include the second type to which the historical problem belongs, the second complexity index of the historical problem, the historical target verification strategy used by the large model when outputting the historical final answer corresponding to the historical problem, and historical result indicators. The historical result indicators include the total historical token consumption of the large model and the small model, and the historical confidence of the historical final answer. Based on the historical database, determine the cost-effectiveness of the historical target verification strategy corresponding to the second type and the second complexity index; The cost-effective instant judgment strategy or the cost-effective step-by-step judgment strategy is determined as the advantage verification strategy corresponding to the second type and the second complexity index.
7. The method according to claim 6, characterized in that, The step of determining the cost-effectiveness of historical target verification strategies corresponding to the second type and the second complexity index based on the historical database includes: Based on the historical database, determine the historical target verification strategy corresponding to the second type and the second complexity index; If the historical target verification strategy corresponding to the second type and the second complexity index includes an immediate judgment strategy and a step-by-step judgment strategy, then calculate the first average historical token consumption and the first average confidence level of the immediate judgment strategy corresponding to the second type and the second complexity index. Based on the historical database, calculate the second average historical token consumption and the second average confidence level of the step-by-step judgment strategy corresponding to the second type and the second complexity index. Based on the first average historical token consumption total and the first average confidence level, and the second average historical token consumption total and the second average confidence level, determine the cost-effectiveness of the immediate judgment strategy and the cost-effectiveness of the step-by-step judgment strategy corresponding to the second type and the second complexity index.
8. An adaptive inference device based on size-model collaboration, characterized in that, include: The first output unit is used to input the question into the small model and output the answer from the small model; The first determining unit is used to determine the target verification strategy to be used for the large model based on the problem. The target verification strategy can be an immediate judgment strategy or a step-by-step judgment strategy. The decomposition unit is used to decompose the target verification strategy into multiple reasoning steps based on the question and the answer of the small model using the large model if the target verification strategy is a stepwise judgment strategy; and to select one of the reasoning steps as the target reasoning step from the multiple reasoning steps in a preset order. The first verification unit is used to verify the target reasoning steps and obtain a second verification result; The second output unit is used to input the question into the large model and output the answer of the large model if the second verification result is unsuccessful. The judgment unit is used to determine whether the current target reasoning step is the last reasoning step if the second verification result is passed. The acquisition unit is used to acquire the next reasoning step as the target reasoning step if it is not the last reasoning step. The step of performing the target reasoning step to obtain a second verification result; The second determining unit is used to determine the small model answer as the final answer if it is the last reasoning step.
9. A computer device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the computer program, it implements the steps of the adaptive inference method based on size model collaboration as described in any one of claims 1 to 7.
10. A computer-readable storage medium storing a computer program, characterized in that, When the computer program is executed by a processor, it implements the steps of the adaptive inference method based on size model collaboration as described in any one of claims 1 to 7.