Parallel inference time extension method and device, electronic equipment and storage medium
By using multiple heterogeneous large language models for parallel reasoning and dynamically adjusting step quotas based on collective consensus scores and historical performance information, this approach solves the problems of high verification costs and poor reliability in complex multi-step reasoning tasks in existing technologies, and achieves efficient and robust reasoning path search.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- INST OF AUTOMATION CHINESE ACAD OF SCI
- Filing Date
- 2026-01-26
- Publication Date
- 2026-06-05
AI Technical Summary
Existing technologies suffer from high verification costs, poor reliability, and limited universality in complex multi-step reasoning tasks, especially in methods that rely on external verifiers or complex hints.
We employ parallel reasoning using multiple heterogeneous large language models, evaluate candidate reasoning steps through collective consensus scores, dynamically adjust step quotas by combining historical performance information, utilize the collective wisdom among models for internal verification, and dynamically allocate computing resources.
It enriches the diversity of candidate reasoning steps, broadens the search space, increases the probability of high-quality reasoning paths, provides stable and robust reward signals, achieves a dynamic balance between diversity and high quality, and improves overall search efficiency.
Smart Images
Figure CN122154907A_ABST
Abstract
Description
Technical Field
[0001] This invention relates to the field of artificial intelligence technology, and in particular to a method, apparatus, electronic device, and storage medium for extending parallel inference time. Background Technology
[0002] With the development of artificial intelligence technology, natural language processing (NLP) technology has gradually integrated into people's production and daily lives. Large language models have demonstrated impressive performance in various NLP tasks. However, handling complex multi-step reasoning tasks remains challenging. One feasible strategy to address this challenge is parallel reasoning time extension, which improves the robustness and accuracy of the reasoning process of large language models by exploring a vast space of potential reasoning paths.
[0003] Existing methods for extending reasoning time primarily fall into two categories during the validation phase: external validators and prompt-based self-evaluation. External validator-based validation requires additional training of the process reward model, typically relying on expensive manual annotation or automated annotation via Monte Carlo tree search. Therefore, resources are only available in well-defined and widely explored domains, such as mathematical reasoning tasks. Furthermore, even in domains where resources are available, external validators exhibit limited generalization ability on more challenging tasks. Prompt-based self-evaluation generates reward scores on the output through a prompt model, thus relying on carefully crafted prompts, limiting the general applicability of this approach. Summary of the Invention
[0004] This invention provides a method, apparatus, electronic device, and storage medium for extending parallel inference time, in order to overcome the deficiencies in the prior art.
[0005] This invention provides a method for extending parallel inference time, comprising: Acquire the data to be inferred and multiple inference models; Based on the data to be reasoned, candidate reasoning steps corresponding to the step quota of each reasoning model are generated through each reasoning model; The collective consensus score of each candidate reasoning step is determined by each of the reasoning models, the target candidate reasoning step is determined based on the collective consensus score, and the reasoning result is determined based on the target candidate reasoning step. Based on the target candidate reasoning steps, determine the step quota corresponding to each of the reasoning models in the next round of reasoning, jump to the step of generating candidate reasoning steps with the step quota corresponding to each of the reasoning models based on the data to be reasoned, until the reasoning ends and the target reasoning result is obtained.
[0006] According to a parallel inference time extension method provided by the present invention, the step of determining the collective consensus score of each candidate inference step through each inference model includes: The perplexity of each candidate reasoning step is calculated using each of the aforementioned reasoning models. Based on the perplexity of each candidate reasoning step, the collective consensus score of each candidate reasoning step is calculated.
[0007] According to a parallel inference time extension method provided by the present invention, the step of generating candidate inference steps with corresponding step quotas for each inference model based on the data to be inferred includes: Determine the reasoning-related data required for the current reasoning process; wherein, in the first round of reasoning, the reasoning-related data is the data to be reasoned, and in other reasoning processes besides the first round of reasoning, the reasoning-related data is the data to be reasoned and the reasoning result generated in the previous round of reasoning; Based on the inference association data, candidate inference steps for the step quota corresponding to each inference model are generated through each inference model.
[0008] According to a parallel inference time extension method provided by the present invention, the step of generating candidate inference steps with corresponding step quotas for each inference model based on the data to be inferred includes: If the current reasoning process is the first round of reasoning, obtain the target number of steps; Based on the number of inference models and the number of target steps, determine the initial step quota for each inference model; The candidate inference steps are generated by generating the corresponding initial step quota through each of the inference models.
[0009] According to a parallel inference time extension method provided by the present invention, the step quota corresponding to each inference model in the next round of inference process is determined based on the target candidate inference steps, including: Determine the historical performance information of each of the aforementioned inference models; wherein, the historical performance information is the number of candidate inference steps generated by each of the aforementioned inference models during the historical inference process that are determined to be the target candidate inference steps; Based on the historical performance information, the step quota corresponding to each of the inference models in the next round of inference is determined.
[0010] According to a parallel inference time extension method provided by the present invention, determining the step quota corresponding to each inference model in the next round of inference process based on the historical performance information includes: Based on the historical performance information, a temperature-based sampling strategy is used to determine the step quota corresponding to each inference model in the next round of inference.
[0011] According to a parallel inference time extension method provided by the present invention, the step of determining the target candidate inference based on the collective consensus score includes: Based on the collective consensus score, the candidate reasoning steps are sorted in descending order; The target candidate reasoning result is determined based on the descending order of the results.
[0012] The present invention also provides a parallel inference time extension device, comprising: The acquisition module is configured to acquire the data to be inferred and multiple inference models. The generation module is configured to generate candidate inference steps corresponding to the step quota of each inference model based on the data to be inferred and through each inference model; The determination module is configured to determine the collective consensus score of each candidate reasoning step through each of the reasoning models, determine the target candidate reasoning step based on the collective consensus score, and determine the reasoning result based on the target candidate reasoning step; The jump module is configured to determine the step quota corresponding to each of the inference models in the next round of inference based on the target candidate inference steps, jump to the step that generates candidate inference steps corresponding to the step quota of each inference model based on the data to be inferred, until the inference ends and the target inference result is obtained.
[0013] The present invention also provides an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor, when executing the computer program, implements the parallel inference time extension method as described above.
[0014] The present invention also provides a non-transitory computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the parallel inference time extension method as described above.
[0015] The present invention also provides a computer program product, including a computer program that, when executed by a processor, implements any of the parallel inference time extension methods described above.
[0016] The parallel inference time extension method, apparatus, electronic device, and storage medium provided by this invention integrate multiple inference models into the parallel inference time extension. In the generation phase, multiple inference models are used to generate candidate inference steps, greatly enriching the diversity of candidate inference steps, broadening the search space, and significantly improving the probability of finding high-quality inference paths. Furthermore, a collective consensus score for each candidate inference step is determined based on multiple inference models, eliminating reliance on external validators or complex hints. Utilizing the collective wisdom among models as an intrinsic verification standard, it has strong universality, integrates the evaluation biases of multiple inference models, and provides a more stable and robust reward signal than self-evaluation. Simultaneously, this invention introduces a strategy for dynamic step quota allocation based on determined target candidate inference steps. This allows computational resources to be tilted towards inference models with a higher probability of generating high-quality candidate inference steps during iteration, achieving a dynamic balance between ensuring diversity and pursuing high quality, thus improving overall search efficiency. Attached Figure Description
[0017] To more clearly illustrate the technical solutions in this invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of this invention. For those skilled in the art, other drawings can be obtained from these drawings without creative effort.
[0018] Figure 1 This is a flowchart illustrating the parallel inference time extension method provided by the present invention.
[0019] Figure 2 This is a schematic diagram of the parallel inference time extension method provided by the present invention.
[0020] Figure 3 This is a schematic diagram of the parallel inference time extension device provided by the present invention.
[0021] Figure 4 This is a schematic diagram of the structure of the electronic device provided by the present invention. Detailed Implementation
[0022] To make the objectives, technical solutions, and advantages of this invention clearer, the technical solutions of this invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of this invention. All other embodiments obtained by those skilled in the art based on the embodiments of this invention without creative effort are within the scope of protection of this invention.
[0023] Figure 1 This is a flowchart illustrating a parallel inference time extension method according to an exemplary embodiment. For example... Figure 1 As shown in an exemplary embodiment, the parallel inference time extension method includes steps 110 to 140, which are described in detail below.
[0024] Step 110: Obtain the data to be inferred and multiple inference models.
[0025] In this embodiment of the invention, the data to be reasoned refers to the input information that the reasoning model needs to reason and solve. It can take many forms, such as a piece of natural language text, like a math word problem, a logical reasoning problem, or a document that needs to be summarized; or it can be other forms of data, such as code snippets, structured data tables, etc.
[0026] Multiple inference models refer to a large collection of language models used to perform inference tasks. These inference models can be heterogeneous, meaning they differ in model architecture, parameter size, training dataset, training methods, or development institutions. For example, the collection could include large commercial models from different companies or open-source models from different open-source communities, such as... Figure 2 The models shown are Models 1-3. This heterogeneity ensures that the model set is diverse and complementary in terms of knowledge coverage, reasoning ability, and thinking patterns, which is the basis for generating diverse candidate reasoning steps.
[0027] Step 120: Based on the data to be inferred, generate candidate inference steps for the step quota corresponding to each inference model through each inference model.
[0028] In this embodiment of the invention, the step quota refers to the number of candidate inference steps that each inference model needs to generate in the current iteration. This step quota can be a preset fixed value or a value dynamically determined according to a certain strategy.
[0029] Candidate reasoning steps refer to intermediate reasoning links or logical segments generated to solve the initial data to be reasoned. They are not the final answer, but rather a step that constitutes a complete reasoning chain.
[0030] In a specific scenario, for example, if the data to be reasoned is: "A has 5 books, and B has 1 less book than A, how many books do they have in total?", a valid candidate reasoning step could be: "B has 5 - 1 = 4 books." However, "A likes to read" might be a relevant but invalid candidate reasoning step.
[0031] The generation of candidate inference steps can be executed independently and in parallel by each inference model. Each inference model generates candidate inference steps according to its allocated step quota. During the generation of candidate inference steps, sampling strategies such as greedy search, cluster search, or temperature-based random sampling can be used. The candidate inference steps generated by all inference models together constitute a diverse set of candidate steps.
[0032] Step 130: Determine the collective consensus score of each candidate reasoning step through each reasoning model, determine the target candidate reasoning step based on the collective consensus score, and determine the reasoning result based on the target candidate reasoning step.
[0033] In this embodiment of the invention, numerous candidate inference steps are evaluated and screened based on a collective consensus score. The collective consensus score is a quantitative indicator that reflects the degree of consensus or acceptance of the quality of a particular candidate inference step across the entire model set. There are various ways to determine the collective consensus score. For example, it can be calculated using a pre-defined fusion algorithm based on scores from all inference models for the candidate inference step, such as perplexity or probability values. This algorithm could involve averaging the scores, using a weighted average, or voting to obtain the collective consensus score.
[0034] Candidate inference steps with high collective consensus scores are generally considered to be more logically sound, more relevant to the context, and accepted by multiple inference models. After determining the collective consensus score for each candidate inference step, target candidate inference steps are determined based on these scores. The determination method may include, but is not limited to, selecting one or more candidate inference steps with the highest scores. These selected target candidate inference steps are considered the optimal inference direction in the current round.
[0035] Based on these target candidate reasoning steps, the reasoning result of this round can be determined. The reasoning result can refer to the data concatenated by the selected target candidate reasoning steps in each round, which will serve as the known conditions or contextual prefix for the next round of reasoning.
[0036] Step 140: Based on the target candidate reasoning steps, determine the step quota corresponding to each of the reasoning models in the next round of reasoning, and jump to the step of generating candidate reasoning steps with the step quota corresponding to each of the reasoning models based on the data to be reasoned, until the reasoning ends and the target reasoning result is obtained.
[0037] In this embodiment of the invention, a dynamic feedback and adaptive adjustment mechanism is provided, which analyzes the source of the selected candidate inference steps in the current round, i.e., which inference model generated them. Based on this information, the performance of each inference model in the current round can be evaluated. Then, based on this performance, the step quota for each inference model in the next round of inference is dynamically determined.
[0038] After determining the step quota for each inference model in the next round of inference, the process jumps back to the step of generating candidate inference steps to begin a new round of iteration. This iterative process continues until the preset inference termination conditions are met, such as generating an end marker indicating task completion or reaching the preset maximum number of inference steps. The inference results determined in all iteration rounds are chained together to form the target inference result for solving the initial data to be inferred.
[0039] This invention integrates multiple inference models into a parallel inference time extension. During the generation phase, multiple inference models are used to generate candidate inference steps, greatly enriching the diversity of candidate inference steps, broadening the search space, and significantly increasing the probability of finding high-quality inference paths. Furthermore, a collective consensus score for each candidate inference step is determined based on multiple inference models, eliminating reliance on external validators or complex hints. Utilizing the collective wisdom among models as an intrinsic verification standard provides strong universality, integrates the evaluation biases of multiple inference models, and offers a more stable and robust reward signal than self-evaluation. Simultaneously, this invention introduces a strategy for dynamic step quota allocation based on determined target candidate inference steps. This strategy can allocate computational resources to inference models with a higher probability of generating high-quality candidate inference steps during iteration, achieving a dynamic balance between ensuring diversity and pursuing high quality, thus improving overall search efficiency.
[0040] In an exemplary embodiment of the present invention, determining the collective consensus score of each candidate inference step through each of the inference models includes: The perplexity of each candidate reasoning step is calculated using each of the aforementioned reasoning models. Based on the perplexity of each candidate reasoning step, the collective consensus score of each candidate reasoning step is calculated.
[0041] In this embodiment of the invention, the perplexity of each candidate inference step is calculated using each inference model. Perplexity is a standard intrinsic evaluation metric in the field of language modeling, used to measure the quality of a model's prediction of a sequence. The lower the perplexity value of a candidate inference step, the higher the probability that the inference model believes that the candidate inference step will occur, meaning that the candidate inference step is more natural and logical in the current context.
[0042] The perplexity PPL is calculated by the following formula: ; where, represents the k-th candidate inference step generated in the t-th round, represents the candidate inference step calculated by the i-th inference model of perplexity, represents the candidate inference step the number of tokens in, represents the candidate inference step the j-th token in, represents the generation of the context prefix, represents the given context prefix and the current candidate inference step with the already generated part x<j, the conditional probability that the inference model predicts the next token to be exactly of.
[0043] The collective consensus score c is calculated by the following formula: ; where, N represents the number of inference models, M represents the model set, represents the inference model i.
[0044] In the embodiments of the present invention, for each candidate inference step in the candidate step set, all inference models will independently calculate their own perplexity values for the candidate inference step. For example, as Figure 2 shown, for the candidate inference step Cand1 (C1), the inference models Model 1-3 will calculate their perplexities respectively, and take the negative values of the calculated perplexities to obtain three independent perplexity scores, such as 0.9, 0.8, 0.7.
[0045] After calculating the perplexities of all inference models for each candidate inference step, the perplexity scores of the same candidate inference step are arithmetically averaged, and the calculated average value is defined as the collective consensus score of the candidate inference step. For example, for the above C1, its collective consensus score is (0.9 + 0.8 + 0.7) / 3 = 0.8. A truly high-quality inference step should be rated as having a low perplexity by most capable inference models, thus obtaining a low collective consensus score.
[0046] This embodiment avoids evaluation bias due to reliance on a single inference model by concretizing the collective consensus score as the average of the perplexity scores of all inference models. By aggregating collective judgments, the scoring results are more stable and reliable. It provides a verification signal that requires no external annotation, is computationally efficient, and has strong universality, effectively solving the problems of high cost and poor reliability in the verification stage of existing technologies.
[0047] In an exemplary embodiment of the present invention, the step of generating candidate inference steps corresponding to the step quotas of each inference model based on the data to be inferred includes: Determine the reasoning-related data required for the current reasoning process; wherein, in the first round of reasoning, the reasoning-related data is the data to be reasoned, and in other reasoning processes besides the first round of reasoning, the reasoning-related data is the data to be reasoned and the reasoning result generated in the previous round of reasoning; Based on the inference association data, candidate inference steps for the step quota corresponding to each inference model are generated through each inference model.
[0048] In this embodiment of the invention, inference-related data refers to all contextual information relied upon by the inference model when generating the current candidate inference step. In a specific embodiment, the composition of this inference-related data is related to the iteration round. In the first round of inference, since no intermediate inference results have been generated, the inference-related data at this time is only the initially acquired data to be inferred. In other inference rounds besides the first round, i.e., the second round and all subsequent rounds, the inference process has made some progress. At this time, the inference-related data should include the initial data to be inferred and the inference results generated up to the previous round of inference.
[0049] Based on the inference association data, candidate inference steps corresponding to the step quotas of each inference model are generated. Each inference model uses the complete inference association data determined in the above manner as an input prefix, and generates candidate inference steps for this round based on this prefix.
[0050] For example, let's continue with the question, "A has 5 books, and B has 1 less book than A. How many books do they have in total?" In the first round, the reasoning context is the question itself. The reasoning model might generate "B has 4 books" as a target candidate reasoning step. In the second round, the reasoning context becomes: "A has 5 books, and B has 1 less book than A. How many books do they have in total? B has 4 books." Each reasoning model will then continue to generate candidate reasoning steps based on this richer context, such as "They have a total of 5 + 4 = 9 books."
[0051] This embodiment ensures the coherence and logical progression of the reasoning process by clearly defining and dynamically updating the reasoning-related data for each round. This allows the reasoning model to continue exploring along the confirmed correct reasoning path, thereby gradually building a complete solution.
[0052] In an exemplary embodiment of the present invention, the step of generating candidate inference steps corresponding to the step quotas of each inference model based on the data to be inferred includes: If the current reasoning process is the first round of reasoning, obtain the target number of steps; Based on the number of inference models and the number of target steps, determine the initial step quota for each inference model; The candidate inference steps are generated by generating the corresponding initial step quota through each of the inference models.
[0053] In this embodiment of the invention, the target number of steps refers to the total number of candidate inference steps that all inference models are expected to generate in the first iteration. This number can be preset by the user.
[0054] Based on the number of inference models and the target number of steps, the initial step quota for each inference model is determined. In the first round of inference, since there is no historical information about the performance of each inference model, the fairest and most reasonable approach is to distribute the step quota equally. Therefore, the initial step quota for each inference model can be determined by dividing the target number of steps by the number of inference models.
[0055] Each inference model generates a corresponding number of candidate inference steps based on its equally allocated initial step quota. For example, such as Figure 2 Models 1-3 in the model generate 6 candidate inference steps respectively.
[0056] This embodiment ensures that all inference models have an equal opportunity to contribute in the initial exploration phase by equally distributing the initial step quota during the first round of inference. This helps to maximize the diversity of the initial candidate step set and lays a broad search foundation for subsequent screening and inference.
[0057] In an exemplary embodiment of the present invention, determining the step quota corresponding to each of the inference models in the next round of inference based on the target candidate inference steps includes: Determine the historical performance information of each of the aforementioned inference models; wherein, the historical performance information is the number of candidate inference steps generated by each of the aforementioned inference models during the historical inference process that are determined to be the target candidate inference steps; Based on the historical performance information, the step quota corresponding to each of the inference models in the next round of inference is determined.
[0058] In this embodiment of the invention, historical performance information is an indicator that quantifies the contribution of each inference model in past inference iterations. In one specific embodiment, this historical performance information is defined as the number of candidate inference steps generated by each inference model during the historical inference process that are identified as target candidate inference steps. Specifically, after each round of inference, the target candidate inference steps are tracked. For example, if two target candidate inference steps are selected in this round, one generated by Model1 and the other by Model2, then the count for this item for Model1 and Model2 is incremented by one respectively. This count is accumulated from the beginning of inference.
[0059] Based on historical performance information, the step quota for each inference model in the next round of inference is determined. That is, the historical performance information collected above is used to determine how the step quota is allocated in the next round. Inference models that perform better, i.e., those that generate more adopted candidate inference steps, should receive more step quotas in the next round, and vice versa. There can be various algorithms for determining step quotas; for example, they can be allocated directly based on the proportion of historical performance counts.
[0060] This embodiment establishes a clear feedback loop from target candidate inference steps to historical performance information and future step quotas, enabling the entire scheme to automatically identify and reward high-performing inference models, thus achieving dynamic optimization and intelligent allocation of inference resources.
[0061] In an exemplary embodiment of the present invention, determining the step quota corresponding to each of the inference models in the next round of inference process based on the historical performance information includes: Based on the historical performance information, a temperature-based sampling strategy is used to determine the step quota corresponding to each inference model in the next round of inference.
[0062] In this embodiment of the invention, if step quotas are allocated directly proportionally, high-performing inference models may quickly occupy all quotas, while lower-performing models with unique advantages in certain aspects may have no chance at all, thus impairing the diversity of generated candidate inference steps. To address this issue, this embodiment employs a smoother allocation method: a temperature-based sampling strategy. The historical performance information of each inference model can be input as a vector into the Softmax function. The Softmax function converts this into a probability distribution with a sum of 1. This probability distribution represents the proportion of step quotas each inference model will receive in the next round. The temperature parameter in the Softmax function can adjust the sharpness of this distribution. High temperatures make the distribution smoother, tending towards a more even distribution and encouraging exploration; low temperatures make the distribution sharper.
[0063] like Figure 2 The "Samples per beam" can be understood as the number of candidate inference steps to be generated every two rounds of inference path expansion, assumed to be 9. In the first round of inference, candidate inference steps Cand1 and Cand9 are identified as target candidate inference steps. Based on either Cand1 or Cand9, the three inference models generate 9 candidate inference steps each. The number of candidate inference steps generated by the three inference models is determined based on historical performance information and a temperature-based sampling strategy. Specifically, when Cand1 is used as the context prefix, a step quota is added to the inference model Model1 corresponding to Cand1; when Cand9 is used as the context prefix, a step quota is added to the inference model Model2 corresponding to Cand1.
[0064] This embodiment cleverly balances leveraging high-performing inference models with exploring the potential of other inference models by employing a temperature-based sampling strategy to allocate step quotas. This makes resource allocation more flexible and robust, concentrating strengths while preserving diversity.
[0065] In an exemplary embodiment of the present invention, the step of determining the target candidate inference based on the collective consensus score includes: Based on the collective consensus score, the candidate reasoning steps are sorted in descending order; The target candidate reasoning result is determined based on the descending order of the results.
[0066] In this embodiment of the invention, the collective consensus score is determined by the average of the perplexity scores. A lower perplexity value indicates better quality; taking a negative perplexity value results in a higher perplexity score, which in turn improves the overall quality. Consequently, a higher collective consensus score is also preferred. The candidate reasoning steps are then sorted in descending order of their collective consensus scores, with the highest-scoring step appearing at the top.
[0067] After sorting, starting from the top of the list, a predetermined number of candidate inference steps are selected as the target candidate inference steps for this round. This predetermined number is typically defined by the hyperparameter of beam width. For example, in Figure 2 In this process, the bundle width is 2, and the top two sorted items are selected as the target candidate reasoning steps.
[0068] This embodiment ensures that each iteration can deterministically and efficiently select the optimal inference path from numerous candidates for expansion by defining a clear, sorting- and selection-based screening process.
[0069] The parallel inference time extension device provided by the present invention will be described below. The parallel inference time extension device described below can be referred to in correspondence with the parallel inference time extension method described above. It should be noted that the device provided in the following embodiments belongs to the same concept as the method provided in the above embodiments, and the specific way in which each module and unit performs operations has been described in detail in the method embodiments, and will not be repeated here.
[0070] In one exemplary embodiment of the present invention, please refer to Figure 3 , Figure 3 This is a parallel inference time extension device according to an exemplary embodiment, comprising the following modules.
[0071] Module 310 is configured to acquire the data to be inferred and multiple inference models. The generation module 320 is configured to generate candidate inference steps corresponding to the step quota of each inference model based on the data to be inferred and through each inference model; The determination module 330 is configured to determine the collective consensus score of each candidate reasoning step through each of the reasoning models, determine the target candidate reasoning step based on the collective consensus score, and determine the reasoning result based on the target candidate reasoning step; The jump module 340 is configured to determine the step quota corresponding to each of the inference models in the next round of inference based on the target candidate inference steps, jump to the step based on the data to be inferred, generate candidate inference steps corresponding to the step quota of each of the inference models, until the inference ends and the target inference result is obtained.
[0072] In an exemplary embodiment of the present invention, the determining module 330 includes: The first calculation submodule is configured to calculate the perplexity of each candidate reasoning step using each of the aforementioned reasoning models. The second calculation submodule is configured to calculate the collective consensus score of each candidate reasoning step based on the perplexity of each candidate reasoning step.
[0073] In an exemplary embodiment of the present invention, the generation module 320 includes: The first determining submodule is configured to determine the reasoning-related data required for the current reasoning process; wherein, in the first round of reasoning, the reasoning-related data is the data to be reasoned, and in other reasoning processes besides the first round of reasoning, the reasoning-related data is the data to be reasoned and the reasoning result generated in the previous round of reasoning. The first generation submodule is configured to generate candidate inference steps for each inference model's corresponding step quota based on the inference association data and through each inference model.
[0074] In an exemplary embodiment of the present invention, the generation module 320 includes: The submodule is configured to retrieve the target number of steps if the current inference process is the first round of inference. The second determining submodule is configured to determine the initial step quota for each of the inference models based on the number of inference models and the number of target steps; The second generation submodule is configured to generate the candidate inference steps corresponding to the initial step quota through each of the inference models.
[0075] In an exemplary embodiment of the present invention, the jump module 340 includes: The third determining submodule is configured to determine the historical performance information of each of the inference models; wherein, the historical performance information is the number of candidate inference steps generated by each of the inference models during the historical inference process that are determined to be the target candidate inference steps; The third determining submodule is configured to determine the step quota corresponding to each of the inference models in the next round of inference based on the historical performance information.
[0076] In one exemplary embodiment of the present invention, the third determining submodule includes: The determining unit is configured to determine the step quota corresponding to each of the inference models in the next round of inference process based on the historical performance information and through a temperature-based sampling strategy.
[0077] In an exemplary embodiment of the present invention, the determining module 330 includes: The descending order sorting submodule is configured to sort the candidate reasoning steps in descending order based on the collective consensus score; The fourth determination submodule is configured to determine the target candidate inference result based on the result of descending order.
[0078] Figure 4 An example is a schematic diagram of the physical structure of an electronic device, such as... Figure 4 As shown, the electronic device may include a processor 410, a communications interface 420, a memory 430, and a communication bus 440, wherein the processor 410, the communications interface 420, and the memory 430 communicate with each other via the communication bus 440. The processor 410 can call logical instructions in the memory 430 to execute a parallel inference time extension method, which includes: acquiring data to be inferred and multiple inference models; Based on the data to be reasoned, candidate reasoning steps corresponding to the step quota of each reasoning model are generated through each reasoning model; The collective consensus score of each candidate reasoning step is determined by each of the reasoning models, the target candidate reasoning step is determined based on the collective consensus score, and the reasoning result is determined based on the target candidate reasoning step. Based on the target candidate reasoning steps, determine the step quota corresponding to each of the reasoning models in the next round of reasoning, jump to the step of generating candidate reasoning steps with the step quota corresponding to each of the reasoning models based on the data to be reasoned, until the reasoning ends and the target reasoning result is obtained.
[0079] Furthermore, the logical instructions in the aforementioned memory 430 can be implemented as software functional units and, when sold or used as independent products, can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, or the part that contributes to the prior art, or a part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of the present invention. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.
[0080] On the other hand, the present invention also provides a computer program product, the computer program product including a computer program, the computer program being able to be stored on a non-transitory computer-readable storage medium, and when the computer program is executed by a processor, the computer is able to execute the parallel inference time extension method provided by the above methods, the method including: acquiring data to be inferred and multiple inference models; Based on the data to be reasoned, candidate reasoning steps corresponding to the step quota of each reasoning model are generated through each reasoning model; The collective consensus score of each candidate reasoning step is determined by each of the reasoning models, the target candidate reasoning step is determined based on the collective consensus score, and the reasoning result is determined based on the target candidate reasoning step. Based on the target candidate reasoning steps, determine the step quota corresponding to each of the reasoning models in the next round of reasoning, jump to the step of generating candidate reasoning steps with the step quota corresponding to each of the reasoning models based on the data to be reasoned, until the reasoning ends and the target reasoning result is obtained.
[0081] In another aspect, the present invention also provides a non-transitory computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, is implemented to perform the parallel inference time extension method provided by the above methods, the method comprising: acquiring data to be inferred and multiple inference models; Based on the data to be reasoned, candidate reasoning steps corresponding to the step quota of each reasoning model are generated through each reasoning model; The collective consensus score of each candidate reasoning step is determined by each of the reasoning models, the target candidate reasoning step is determined based on the collective consensus score, and the reasoning result is determined based on the target candidate reasoning step. Based on the target candidate reasoning steps, determine the step quota corresponding to each of the reasoning models in the next round of reasoning, jump to the step of generating candidate reasoning steps with the step quota corresponding to each of the reasoning models based on the data to be reasoned, until the reasoning ends and the target reasoning result is obtained.
[0082] The device embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to achieve the purpose of this embodiment according to actual needs. Those skilled in the art can understand and implement this without any creative effort.
[0083] Through the above description of the embodiments, those skilled in the art can clearly understand that each embodiment can be implemented by means of software plus necessary general-purpose hardware platforms, and of course, it can also be implemented by hardware. Based on this understanding, the above technical solutions, in essence or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, magnetic disk, optical disk, etc., and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute the methods described in the various embodiments or some parts of the embodiments.
[0084] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, and not to limit them; although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features; and these modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present invention.
Claims
1. A method for extending the time of parallel inference, characterized in that, include: Acquire the data to be inferred and multiple inference models; Based on the data to be reasoned, candidate reasoning steps corresponding to the step quota of each reasoning model are generated through each reasoning model; The collective consensus score of each candidate reasoning step is determined by each of the reasoning models, the target candidate reasoning step is determined based on the collective consensus score, and the reasoning result is determined based on the target candidate reasoning step. Based on the target candidate reasoning steps, determine the step quota corresponding to each of the reasoning models in the next round of reasoning, jump to the step of generating candidate reasoning steps with the step quota corresponding to each of the reasoning models based on the data to be reasoned, until the reasoning ends and the target reasoning result is obtained.
2. The parallel inference time extension method according to claim 1, characterized in that, The determination of the collective consensus score for each candidate reasoning step through each of the aforementioned reasoning models includes: The perplexity of each candidate reasoning step is calculated using each of the aforementioned reasoning models. Based on the perplexity of each candidate reasoning step, the collective consensus score of each candidate reasoning step is calculated.
3. The parallel inference time extension method according to claim 1, characterized in that, The step of generating candidate inference steps for each inference model's corresponding step quota based on the data to be inferred includes: Determine the reasoning-related data required for the current reasoning process; wherein, in the first round of reasoning, the reasoning-related data is the data to be reasoned, and in other reasoning processes besides the first round of reasoning, the reasoning-related data is the data to be reasoned and the reasoning result generated in the previous round of reasoning; Based on the inference association data, candidate inference steps for the step quota corresponding to each inference model are generated through each inference model.
4. The parallel inference time extension method according to claim 1, characterized in that, The step of generating candidate inference steps for each inference model's corresponding step quota based on the data to be inferred includes: If the current reasoning process is the first round of reasoning, obtain the target number of steps; Based on the number of inference models and the number of target steps, determine the initial step quota for each inference model; The candidate inference steps are generated by generating the corresponding initial step quota through each of the inference models.
5. The parallel inference time extension method according to claim 1, characterized in that, The step quota determination for each inference model in the next round of inference based on the target candidate inference steps includes: Determine the historical performance information of each of the aforementioned inference models; wherein, the historical performance information is the number of candidate inference steps generated by each of the aforementioned inference models during the historical inference process that are determined to be the target candidate inference steps; Based on the historical performance information, the step quota corresponding to each of the inference models in the next round of inference is determined.
6. The parallel inference time extension method according to claim 5, characterized in that, The step quota determination based on the historical performance information for each inference model in the next round of inference includes: Based on the historical performance information, a temperature-based sampling strategy is used to determine the step quota corresponding to each inference model in the next round of inference.
7. The parallel inference time extension method according to claim 1, characterized in that, The step of determining the target candidate inference based on the collective consensus score includes: Based on the collective consensus score, the candidate reasoning steps are sorted in descending order; The target candidate reasoning result is determined based on the descending order of the results.
8. A parallel inference time extension device, characterized in that, include: The acquisition module is configured to acquire the data to be inferred and multiple inference models. The generation module is configured to generate candidate inference steps corresponding to the step quota of each inference model based on the data to be inferred and through each inference model; The determination module is configured to determine the collective consensus score of each candidate reasoning step through each of the reasoning models, determine the target candidate reasoning step based on the collective consensus score, and determine the reasoning result based on the target candidate reasoning step; The jump module is configured to determine the step quota corresponding to each of the inference models in the next round of inference based on the target candidate inference steps, jump to the step that generates candidate inference steps corresponding to the step quota of each inference model based on the data to be inferred, until the inference ends and the target inference result is obtained.
9. An electronic device comprising a memory, a processor, and a computer program stored in the memory and running on the processor, characterized in that, When the processor executes the computer program, it implements the parallel inference time extension method as described in any one of claims 1 to 7.
10. A non-transitory computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by the processor, it implements the parallel inference time extension method as described in any one of claims 1 to 7.