An inference optimization method, device and equipment of a code generation large model and a medium

Abstract

The application discloses a code generation large model inference optimization method and device, equipment and medium, and relates to the technical field of model inference. The method comprises the following steps: in the inference process of a target code generation large model performing a code generation task, a new Token is generated every time, and a multi-granularity uncertainty quantization step is executed in parallel to obtain a multi-granularity uncertainty score; a state space vector is constructed, and a preset reinforcement learning strategy network is used to evaluate the selection probability of a plurality of preset inference optimization strategies based on a preset smoothing decision mechanism to determine a target inference optimization strategy; if the preset inference optimization strategy is a preset inference acceleration strategy, optimization processing is performed through speculative decoding; if the preset inference optimization strategy is a preset exploration optimization strategy, optimization processing is performed by using a preset multi-path sampling technology and a preset knowledge enhancement technology; if the preset inference optimization strategy is a preset fuzzy processing strategy, a plurality of candidate outputs are used as target inference outputs for optimization processing; and the decision effect is evaluated according to the inference result to optimize the preset reinforcement learning strategy network.

Description

Technical Field

[0001] This invention relates to the field of model reasoning technology, and in particular to a reasoning optimization method, apparatus, device, and medium for generating large models of code. Background Technology

[0002] As model size continues to increase and application scenarios become increasingly complex, the autoregressive nature of LLM (Large Language Model) leads to computationally intensive and time-consuming inference processes, severely limiting inference speed. Furthermore, with the increase in output sequence length, the contextual information the model needs to process grows linearly, further exacerbating the efficiency bottleneck. Meanwhile, the inherent uncertainty in the LLM generation process leads to reliability issues with its output. This uncertainty mainly stems from the incompleteness of the model's own knowledge and the inaccuracy of parameter estimation (cognitive uncertainty), as well as the ambiguity and polysemy of the input data itself (data uncertainty). These problems are particularly prominent in code generation tasks.

[0003] To address the inefficiency of LLM inference, existing speculative decoding techniques significantly improve inference speed by using small draft models to predict multiple tokens (the smallest units of text, the basic units used for model input and output processing) and then verifying them in parallel with a large model. However, this acceleration is highly dependent on the prediction accuracy of the draft models; when the draft models' prediction quality is poor, frequent rollback operations can negate the acceleration effect. Model quantization techniques reduce computational and storage overhead by decreasing the precision of model parameters. However, quantization often comes with a loss of precision, and the output quality of the model may significantly decrease. On the other hand, research on the reliability problem of LLM mainly focuses on uncertainty quantization. However, most uncertainty quantization methods focus on the granularity at the token level or the entire output sequence. In complex inference tasks, the model may have high uncertainty at a certain intermediate inference step, but existing methods struggle to capture and utilize such fine-grained signals in real time. Furthermore, existing uncertainty quantization methods are mostly post-evaluation methods, meaning uncertainty analysis is performed only after the model generates output. This lag prevents uncertainty signals from being fed back into the inference process in real time, thus hindering the dynamic adjustment of inference strategies to avoid or mitigate the negative impact of uncertainty.

[0004] In summary, how to achieve synergistic optimization of inference efficiency and output reliability in large-scale code generation models is a pressing technical problem that needs to be solved. Summary of the Invention

[0005] In view of this, the purpose of this invention is to provide a method, apparatus, device, and medium for optimizing the inference of large code generation models, which can achieve synergistic optimization of inference efficiency and output reliability of large code generation models. The specific solution is as follows:

[0006] Firstly, this application provides an inference optimization method for large-scale code generation models, including:

[0007] During the inference process of the target code generation model in the code generation task, for each newly generated token, a multi-granularity uncertainty quantification step is executed in parallel to obtain the corresponding multi-granularity uncertainty score. The multi-granularity uncertainty quantification includes token-level uncertainty quantification, phrase-level uncertainty quantification, inference step-level uncertainty quantification, model cognition uncertainty quantification, and input data uncertainty quantification. The multi-granularity uncertainty score includes target token-level uncertainty score, target phrase-level uncertainty score, inference step-level uncertainty score, model cognition uncertainty score, and input data uncertainty score.

[0008] The current state space vector is constructed based on the multi-granularity uncertainty score, and a preset reinforcement learning policy network is used to evaluate the selection probability of several preset inference optimization policies based on the preset smooth decision mechanism and the state space vector, so as to determine the corresponding target inference optimization policy according to the obtained selection probability; the preset smooth decision mechanism is a mechanism that smooths the policy selection of continuous time steps through the exponential moving average method.

[0009] If the target reasoning optimization strategy is a preset reasoning acceleration strategy, then the reasoning is optimized by speculative decoding; if the target reasoning optimization strategy is a preset exploration optimization strategy, then the reasoning is optimized by preset multi-path sampling technology and preset knowledge enhancement technology; if the target reasoning optimization strategy is a preset fuzzy processing strategy, then the reasoning is optimized by using multiple candidate outputs as the target reasoning output.

[0010] The inference result obtained after this inference optimization process is determined, and the decision-making effect of this inference optimization strategy is evaluated based on the inference result, so as to optimize the preset reinforcement learning strategy network based on the evaluation result.

[0011] Optionally, the multi-granularity uncertainty quantification step during the parallel execution inference process, to obtain the corresponding multi-granularity uncertainty score, includes:

[0012] The probability distribution vector of the newly added Token is determined, and the information entropy is calculated using the probability distribution vector and a preset entropy calculation algorithm to quantify the uncertainty at the Token level. The obtained predicted entropy value is then normalized to obtain the corresponding target Token level uncertainty score. The preset entropy calculation algorithm includes a pre-calculated logarithmic lookup table and vectorization operations.

[0013] Extract several attention weights related to the newly added token from the last preset number of layers of the target code generation large model, in order to determine the target average attention weight of the newly added token, and determine several average attention weights and several token-level uncertainty scores of several historically generated tokens;

[0014] Based on the aforementioned average attention weights and the target average attention weights, the target token-level uncertainty score and the aforementioned token-level uncertainty scores are weighted and aggregated to perform phrase-level uncertainty quantification, thereby obtaining the corresponding target phrase-level uncertainty score.

[0015] Determine the reasoning complexity of the current reasoning step, and determine the perturbation intensity based on the reasoning complexity; the perturbation intensity includes the number of perturbations and the perturbation amplitude;

[0016] The first mapping from the current inference step to the final inference output is obtained through the target fast forward propagation technique, and the current inference step is perturbed based on the perturbation intensity. The second mapping from the current inference step to the final inference output after the perturbation is obtained through the target fast forward propagation technique. The output difference between the first mapping and the second mapping is measured to quantify the uncertainty at the inference step level and obtain the corresponding uncertainty score at the inference step level.

[0017] Based on variance decomposition technology, the model parameters of the target code generation large model are analyzed to obtain the corresponding model cognitive uncertainty score, and the current input data is analyzed to obtain the corresponding input data uncertainty score.

[0018] Optionally, the step of constructing the current state space vector based on the multi-granularity uncertainty score, and using a preset reinforcement learning policy network to evaluate the selection probability of several preset inference optimization strategies based on a preset smooth decision-making mechanism and the state space vector, so as to determine the corresponding target inference optimization strategy according to the obtained selection probability, includes:

[0019] The task type, inference progress information, and available resource status of the code generation task are determined, and the current state space vector is constructed using the task type, the inference progress information, the available resource status, and the multi-granularity uncertainty score.

[0020] The state space vector is input into a preset reinforcement learning policy network to obtain the selection probabilities of several preset inference optimization policies through the preset reinforcement learning policy network;

[0021] The selection probability is smoothed using the exponential moving average method and historical strategy selection probability, so that the target inference optimization strategy with the highest probability is determined from the several preset inference optimization strategies based on the probability results obtained after processing.

[0022] Optionally, if the target inference optimization strategy is a preset inference acceleration strategy, then the inference is optimized through speculative decoding, including:

[0023] If the target inference optimization strategy is a preset inference acceleration strategy, then the task characteristics of the code generation task are determined, and a target draft model is selected from the preset draft model pool according to the task characteristics.

[0024] The historical success rate is statistically analyzed using a preset sliding window, and the target step size adjustment parameter is determined by the historical success rate and the corresponding historical weighting factor.

[0025] The target step size for this speculative decoding is determined based on the target step size adjustment parameter and the multi-granularity uncertainty score, so as to optimize the speculative decoding of this inference according to the target step size.

[0026] Optionally, if the target reasoning optimization strategy is a preset exploration optimization strategy, then the current reasoning is optimized using preset multi-path sampling technology and preset knowledge enhancement technology, including:

[0027] If the target reasoning optimization strategy is a preset exploration optimization strategy, then the task domain and task type related to the code generation task are determined, and a preset external knowledge source is queried according to the task domain and the task type. The query results are then filtered and sorted using a preset relevance evaluation model to obtain the corresponding knowledge enhancement results.

[0028] The target number of sampling paths for this multi-path sampling is determined based on the model's cognitive uncertainty score, the preset maximum number of sampling paths, and the preset minimum number of sampling paths.

[0029] The target sampling path is explored using a preset sampling strategy, and several target candidate inference outputs are generated in parallel based on the knowledge enhancement results; the preset sampling strategy is a sampling strategy composed of temperature sampling, kernel sampling, and typical sampling.

[0030] The inference outputs of the several target candidates are integrated to generate the target inference output.

[0031] Optionally, if the target inference optimization strategy is a preset fuzzy processing strategy, then the inference is optimized by using multiple candidate outputs as the target inference output, including:

[0032] If the target reasoning optimization strategy is a preset fuzzy processing strategy, then the preset multi-path sampling technology and preset knowledge enhancement technology are used to generate the several target candidate reasoning outputs;

[0033] The several target candidate inference outputs are analyzed to obtain the corresponding confidence analysis results, and the set of multiple candidate outputs including the several target candidate inference outputs, the confidence analysis results, and the uncertainty score of the input data is determined as the target inference output.

[0034] Optionally, evaluating the decision-making effect of the current inference optimization strategy based on the inference result, and optimizing the preset reinforcement learning strategy network based on the obtained evaluation result, includes:

[0035] The inference results are compared with the multi-granularity uncertainty score, and the inference optimization strategy is verified by the A / B testing mechanism to evaluate the decision-making effect of the inference optimization strategy and obtain the corresponding effect evaluation results.

[0036] Using the performance evaluation results, the preset reinforcement learning strategy network is optimized and adjusted based on preset incremental learning techniques and preset reward functions.

[0037] Secondly, this application provides an inference optimization device for a large code generation model, comprising:

[0038] The step execution module is used to perform multi-granularity uncertainty quantification steps in parallel during the inference process of the target code generation large model for code generation tasks, for each newly generated token, to obtain the corresponding multi-granularity uncertainty score. The multi-granularity uncertainty quantification includes token-level uncertainty quantification, phrase-level uncertainty quantification, inference step-level uncertainty quantification, model cognition uncertainty quantification, and input data uncertainty quantification. The multi-granularity uncertainty score includes target token-level uncertainty score, target phrase-level uncertainty score, inference step-level uncertainty score, model cognition uncertainty score, and input data uncertainty score.

[0039] The probability evaluation module is used to construct the current state space vector based on the multi-granularity uncertainty score, and to evaluate the selection probability of several preset inference optimization strategies based on the preset smoothing decision mechanism and the state space vector using a preset reinforcement learning policy network, so as to determine the corresponding target inference optimization strategy according to the obtained selection probability; the preset smoothing decision mechanism is a mechanism that smooths the strategy selection of continuous time steps through the exponential moving average method.

[0040] The optimization processing module is used to optimize the current inference by speculative decoding if the target inference optimization strategy is a preset inference acceleration strategy; to optimize the current inference by using preset multi-path sampling technology and preset knowledge enhancement technology if the target inference optimization strategy is a preset exploration optimization strategy; and to optimize the current inference by using multiple candidate outputs as the target inference output if the target inference optimization strategy is a preset fuzzy processing strategy.

[0041] The effect evaluation module is used to determine the inference result obtained after this inference optimization process, and evaluate the decision effect of this inference optimization strategy based on the inference result, so as to optimize the preset reinforcement learning strategy network based on the evaluation result.

[0042] Thirdly, this application provides an electronic device, comprising:

[0043] Memory, used to store computer programs;

[0044] A processor for executing the computer program to implement the aforementioned inference optimization method for large code generation models.

[0045] Fourthly, this application provides a computer-readable storage medium for storing a computer program; wherein, when the computer program is executed by a processor, it implements the aforementioned inference optimization method for a large code generation model.

[0046] In this application, during the inference process of the target code generation model performing code generation tasks, for each newly generated token, a multi-granularity uncertainty quantification step is executed in parallel to obtain the corresponding multi-granularity uncertainty score. The multi-granularity uncertainty quantification includes token-level uncertainty quantification, phrase-level uncertainty quantification, inference step-level uncertainty quantification, model cognition uncertainty quantification, and input data uncertainty quantification. The multi-granularity uncertainty score includes the target token-level uncertainty score, target phrase-level uncertainty score, inference step-level uncertainty score, model cognition uncertainty score, and input data uncertainty score. Based on the multi-granularity uncertainty score, a current state space vector is constructed, and a preset reinforcement learning policy network is used to evaluate several preset inference optimization policies based on a preset smoothing decision-making mechanism and the state space vector. The selection probability is used to determine the corresponding target inference optimization strategy. The preset smoothing decision mechanism is a mechanism that smooths the strategy selection in continuous time steps using an exponential moving average method. If the target inference optimization strategy is a preset inference acceleration strategy, the inference is optimized by speculative decoding. If the target inference optimization strategy is a preset exploration optimization strategy, the inference is optimized using preset multi-path sampling and preset knowledge enhancement techniques. If the target inference optimization strategy is a preset fuzzy processing strategy, the inference is optimized by using multiple candidate outputs as the target inference output. The inference result obtained after the optimization is determined, and the decision effect of the optimization strategy is evaluated based on the inference result. The preset reinforcement learning strategy network is then optimized based on the evaluation result. As can be seen from the above, when generating large-scale model inference using the target code, this application simultaneously executes multi-granularity uncertainty quantification steps at the token level, phrase level, inference step level, model cognition level, and input data level for each newly generated token, obtaining the corresponding multi-granularity uncertainty score. Then, based on these scores, a current state space vector is constructed. Using a preset reinforcement learning strategy network, combined with a preset smoothing decision mechanism and the state space vector, the selection probability of multiple preset inference optimization strategies is evaluated, thereby determining the target inference optimization strategy. Differentiated optimization is performed according to the different target strategies: if it is a preset inference acceleration strategy, inference is optimized through speculative decoding; if it is a preset exploration optimization strategy, it is optimized using preset multi-path sampling technology and knowledge enhancement technology; if it is a preset fuzzy processing strategy, it is optimized by outputting multiple candidate results. Finally, the optimized inference result is determined, the decision-making effect of this strategy is evaluated based on the result, and the preset reinforcement learning strategy network is optimized based on the evaluation result.In this way, through the process described above in this application, the real-time parallel multi-granularity uncertainty quantification during inference can comprehensively capture the sources of uncertainty in each stage of inference, avoiding risk omissions caused by single-dimensional evaluation; constructing a state space vector based on uncertainty scores and combining it with reinforcement learning to evaluate policy probabilities ensures that policy selection not only fits the real-time state of the current inference but also avoids inference fluctuations caused by extreme policies through a smooth decision-making mechanism, improving the rationality and stability of policy selection; inference acceleration strategies can improve inference efficiency in high-efficiency scenarios, exploration optimization strategies can enhance the accuracy of complex inference, and fuzzy processing strategies can cope with the reliability requirements of results under high uncertainty; evaluating and iterating the reinforcement learning policy network based on inference results allows the policy network to continuously learn effective experience in inference optimization, taking into account inference efficiency, accuracy, and reliability; transforming uncertainty quantification from post-evaluation to real-time decision-driven, significantly improving inference speed while ensuring output quality, thereby achieving synergistic optimization of inference efficiency and output reliability in large code generation models. Attached Figure Description

[0047] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on the provided drawings without creative effort.

[0048] Figure 1 This application discloses a flowchart of an inference optimization method for a large code generation model.

[0049] Figure 2 This is a flowchart and timing diagram of the reasoning optimization method for a large code generation model disclosed in this application;

[0050] Figure 3 This is a schematic diagram of the inference optimization device for a large code generation model disclosed in this application;

[0051] Figure 4 This is a structural diagram of an electronic device disclosed in this application. Detailed Implementation

[0052] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0053] To address the inefficiency of LLM inference, existing speculative decoding techniques significantly improve inference speed by using a small draft model to predict multiple tokens and then verifying them in parallel with a large model. However, this acceleration is highly dependent on the prediction accuracy of the draft model; when the draft model's prediction quality is poor, frequent rollback operations can negate the acceleration effect. Model quantization techniques reduce computational and storage overhead by decreasing the precision of model parameters. However, quantization often comes with a loss of precision, and the output quality of the model may significantly decrease. On the other hand, research on the reliability problem of LLM mainly focuses on uncertainty quantization. However, most uncertainty quantization methods focus on the granularity at the token level or the entire output sequence. In complex inference tasks, the model may have high uncertainty at a certain intermediate inference step, but existing methods struggle to capture and utilize such fine-grained signals in real time. Furthermore, existing uncertainty quantization methods are mostly post-evaluation methods, meaning uncertainty analysis is performed only after the model generates output. This lag prevents uncertainty signals from being fed back into the inference process in real time, thus hindering the dynamic adjustment of inference strategies to avoid or mitigate the negative impact of uncertainty.

[0054] To overcome the aforementioned technical problems, this application provides an inference optimization method for large code generation models, which can achieve synergistic optimization of inference efficiency and output reliability of large code generation models.

[0055] See Figure 1 As shown in the figure, this invention discloses an inference optimization method for a large code generation model, including:

[0056] Step S11: During the inference process of the target code generation model in the code generation task, for each newly generated token, a multi-granularity uncertainty quantification step is executed in parallel during the inference process to obtain the corresponding multi-granularity uncertainty score; the multi-granularity uncertainty quantification includes token-level uncertainty quantification, phrase-level uncertainty quantification, inference step-level uncertainty quantification, model cognition uncertainty quantification, and input data uncertainty quantification; the multi-granularity uncertainty score includes target token-level uncertainty score, target phrase-level uncertainty score, inference step-level uncertainty score, model cognition uncertainty score, and input data uncertainty score.

[0057] In this embodiment, a pipelined processing mechanism is employed. As each new token is generated by the target code generation model, the internal state and intermediate outputs of the LLM inference process are monitored. A hierarchical computing architecture is used to perform multi-granularity uncertainty quantification operations in parallel. The quantification dimensions cover token-level (uncertainty in the generation of a single token), phrase-level (uncertainty in combinations of code phrases), inference step-level (uncertainty in the steps of the model's inference process), model cognition-level (uncertainty in the model's own understanding of the generated content), and input data-level (uncertainty brought about by the original input data). Quantization gradually aggregates from the lower-level token-level quantization to the higher-level inference step-level quantization, ultimately resulting in five categories of multi-granularity uncertainty scores: target token-level, target phrase-level, inference step-level, model cognition-level, and input data-level, achieving near real-time uncertainty perception. It is understood that the multi-granularity uncertainty quantification operations are performed through a lightweight parallel computing architecture to ensure that the uncertainty quantification process does not significantly increase inference latency.

[0058] It should be noted that, such as Figure 2The diagram shows the flow and timing of an inference optimization method for a large-scale code generation model provided in this application. This application mainly includes five core modules: a real-time uncertainty quantification module, an adaptive inference strategy decision-making module, a large-scale language model inference engine, an external knowledge base call interface, and a feedback and self-optimization module. The real-time uncertainty quantification module, as the system's perception center, is responsible for continuously monitoring internal state changes during the LLM inference process. Specifically, this module utilizes the attention weights and hidden states already calculated during the LLM inference process to extract uncertainty signals through efficient mathematical operations. The adaptive inference strategy decision-making module acts as the system's intelligent brain, receiving the output signal from the uncertainty quantification module and, combined with current task characteristics and resource status, selecting the optimal inference strategy through a pre-trained reinforcement learning strategy network. This module's decision-making process is based on large-scale learning from historical inference data, enabling accurate prediction of the effects of different strategy combinations. The large-scale language model inference engine is the system's execution core, responsible for actual text generation and inference calculations. This engine has been specially modified to include a dynamic control interface, enabling it to adjust inference parameters in real time in response to instructions from the decision-making module. This design enables the system to seamlessly switch between different optimization strategies during inference. The external knowledge base call interface provides the system with knowledge enhancement capabilities. When the system detects high cognitive uncertainty, this interface can quickly call external search engines, knowledge graphs, or professional databases to provide reliable factual support for the inference process. The interface adopts an asynchronous call mechanism to minimize interference with the inference process. The feedback and self-optimization module ensures that the system has the ability to continuously learn and improve. This module collects detailed performance data for each inference, including inference time, output quality, and uncertainty change trajectory, and continuously optimizes the accuracy of uncertainty quantification and the effectiveness of strategy decisions through online learning algorithms.

[0059] It should be noted that the processing flow of the multi-granularity uncertainty quantification step in the parallel execution inference process is as follows: The probability distribution vector of the newly added Token is determined, and information entropy is calculated using the probability distribution vector and a preset entropy calculation algorithm to perform Token-level uncertainty quantification. The obtained predicted entropy value is then normalized to obtain the corresponding target Token-level uncertainty score. The preset entropy calculation algorithm includes pre-computing a logarithmic lookup table and vectorization operations. Several attention weights related to the newly added Token are extracted from the final preset number of layers in the target code generation model to determine the target average attention weight of the newly added Token, and several average attention weights and several Token-level uncertainty scores of several historically generated Tokens are determined. Based on the several average attention weights and the target average attention weight, the target Token-level uncertainty score and the several Token-level uncertainty scores are... The phrase-level uncertainty score is obtained by weighted aggregation of the phrase-level uncertainty scores. The reasoning complexity of the current reasoning step is determined, and the perturbation intensity is determined based on this complexity. The perturbation intensity includes the number of perturbations and the perturbation amplitude. A first mapping from the current reasoning step to the final reasoning output is obtained using the target fast forward propagation technique. Based on the perturbation intensity, a perturbation is applied to the current reasoning step, and a second mapping from the perturbated current reasoning step to the final reasoning output is obtained using the target fast forward propagation technique. The output difference between the first and second mappings is measured to quantify the reasoning step-level uncertainty, resulting in a corresponding reasoning step-level uncertainty score. The model parameters of the target code generation model are analyzed using variance decomposition technology to obtain a corresponding model cognitive uncertainty score. The current input data is also analyzed to obtain a corresponding input data uncertainty score. Specifically, for the token level, the reliability of the prediction is evaluated by analyzing the concentration of the model output probability distribution. The probability distribution of each position is extracted from the LLM output layer, and then the prediction entropy is calculated in parallel. Specifically, the probability distribution vector of the newly added token is first determined, and then the information entropy is calculated using a preset entropy calculation algorithm. The specific formula for calculating the predicted information entropy is as follows:

[0060] ;

[0061] in, The newly generated token represents the token. The predicted entropy value characterizing the newly added token; Let V represent the predicted probability of the j-th token in the vocabulary, and let V represent the vocabulary size; log represents the logarithmic operation. After obtaining the predicted entropy value, it is normalized to the range [0,1] for easier comparison and threshold setting, thus obtaining the target token-level uncertainty score. The corresponding normalization calculation formula is as follows:

[0062] ;

[0063] in, This represents the normalized target token-level uncertainty score. It is understood that during token-level quantization, this embodiment can significantly reduce computational overhead through pre-computing a logarithmic lookup table and vectorization operations. For phrase-level quantization, a weighted aggregation method is used. Considering the differences in the importance of different tokens in semantic expression, an attention-based weighting method is used for aggregation. The attention mechanism within the LLM is used as the basis for weight allocation. Specifically, attention weights related to the newly added tokens are extracted from the final preset number of layers of the target code generation model to determine the target average attention weight. Simultaneously, several average attention weights and token-level uncertainty scores of several historically generated tokens are obtained. The most important historically generated tokens for the current generation are identified. Then, based on importance, the target and historical token-level uncertainty scores are weighted and aggregated to obtain the target phrase-level uncertainty score. The specific calculation formula for the weighted aggregation is as follows:

[0064] ;

[0065] in, This represents a phrase containing N tokens; The attention weight of the newly added token reflects its importance in the current generation context; This represents the target phrase-level uncertainty score. For inference step-level quantification, a lightweight perturbation analysis method is employed. This involves applying small perturbations to intermediate inference states and observing the stability of subsequent inference paths. Specifically, as inference becomes more complex and system states become more unstable, more and larger perturbations are needed to fully detect potential uncertainties. First, the inference complexity of the current inference step is determined to clarify the perturbation intensity, including the number and magnitude of perturbations. For example, simple tasks use 3-5 small-amplitude perturbations, while complex inference tasks use 8-12 medium-amplitude perturbations, ensuring effective detection of uncertainty without excessive interference with the inference process. The first mapping from the current inference step to the final inference output is obtained using the target fast forward propagation technique. Subsequently, based on the perturbation intensity, small-amplitude Gaussian noise perturbations are applied to the hidden state at key inference nodes (such as problem decomposition points and logical reasoning inflection points) of the current inference step. The second mapping after perturbation is obtained using the target fast forward propagation technique, and the output difference between the two is measured to obtain the inference step-level uncertainty score. The specific formula for calculating the inference step-level uncertainty score is as follows:

[0066] ;

[0067] in, This represents the intermediate state of the k-th reasoning step; This represents the uncertainty score at the inference step level; This represents the m-th perturbation; This represents the mapping function from the current state to the final output. This represents the second mapping. This represents the first mapping; The output difference is represented by a metric function, where M represents the number of perturbations. To distinguish between model cognitive uncertainty and input data uncertainty, a variance decomposition-based method is used. Specifically, for model cognitive level quantization, variance decomposition is used to analyze the model parameters of the target code-generated large model to obtain a model cognitive uncertainty score. The estimation formula for the model cognitive uncertainty score is as follows:

[0068] ;

[0069] in, represents the model parameters; x represents the current input context (including the historical token sequence); y represents the model's predicted output (the probability distribution of the next token). This represents the cognitive uncertainty score of the model; Var is the variance operator used to measure the degree of variation. Operations representing mathematical expectation; Characterize in a given x and Below, the conditional probability distribution of output y; Given y under the condition of x, [] represents the statistical operation performed on the random variable. For the input data level, the inherent fuzziness of the current input data is analyzed using variance decomposition technology to obtain the input data uncertainty score. The specific estimation formula for the input data uncertainty score is as follows:

[0070] ;

[0071] in, This represents the uncertainty score of the input data; The system performs a joint expectation operation to represent the average predicted value that simultaneously considers the uncertainty of model parameters and the uncertainty of output distribution. Understandably, the distinction between cognitive uncertainty and data uncertainty is achieved by analyzing different characteristic patterns of probability distributions. Cognitive uncertainty typically manifests as the model giving relatively high probabilities to multiple candidate answers, resulting in a relatively flat probability distribution; while data uncertainty typically manifests as the model giving a clear prediction, but the prediction result has multiple reasonable semantic interpretations. The system can identify these two distribution patterns using a specially trained binary classifier. The classifier takes the statistical characteristics of the probability distribution (such as entropy, kurtosis, skewness, etc.) as input and outputs the probability of belonging to either cognitive uncertainty or data uncertainty. In this way, the multi-granularity uncertainty quantification operation, which runs in parallel with token generation during the inference process in this embodiment, can capture risk points at each stage of code generation in real time, avoiding the omission of uncertainty caused by single-dimensional evaluation. Token-level quantization, with the help of efficient entropy calculation algorithms, quickly and accurately reflects the reliability of individual token generation. Phrase-level quantization, through attention weight aggregation, reflects the logical stability of code fragment combinations. Inference step-level quantization, through perturbation analysis, clearly presents the sensitivity of the inference process to the final output. Quantization at the model cognition and input data levels reveals the impact of the model's own parameter characteristics and the quality of input information on the generated results, respectively. By transforming uncertainty quantification from post-evaluation to real-time decision-driven, real-time quantization does not require waiting for complete code generation before evaluation, significantly improving the efficiency of uncertainty analysis, helping to improve the accuracy, logic, and reliability of generated code, effectively reducing the illusion rate, and reducing the cost of subsequent manual modification.

[0072] Step S12: Construct the current state space vector based on the multi-granularity uncertainty score, and use a preset reinforcement learning policy network to evaluate the selection probability of several preset inference optimization policies based on a preset smoothing decision mechanism and the state space vector, so as to determine the corresponding target inference optimization policy according to the obtained selection probability; the preset smoothing decision mechanism is a mechanism for smoothing the policy selection of continuous time steps through the exponential moving average method.

[0073] In this embodiment, the current state space vector is first constructed based on the multi-granularity uncertainty score. Then, a preset reinforcement learning policy network, combined with a preset smoothing decision mechanism and the state space vector, is used to evaluate the selection probability of several preset inference optimization policies. Finally, the corresponding target inference optimization policy is determined based on the evaluated selection probability. The preset smoothing decision mechanism is a mechanism that smooths the policy selection across continuous time steps using an exponential moving average method. It should be noted that the preset reinforcement learning policy network adopts a Transformer structure, which can effectively handle variable-length state sequences and complex feature interactions. It learns the correlation between different features through an attention mechanism, especially the complex mapping between uncertainty signals and optimal policy selection. The network is trained using the Proximal Policy Optimization (PPO) algorithm, trained offline in a large-scale simulated inference environment, and then fine-tuned online in actual deployment.

[0074] Specifically, the task type, inference progress information, and available resource status of the code generation task are determined. A current state space vector is constructed using the task type, inference progress information, available resource status, and multi-granularity uncertainty score. This state space vector is then input into a preset reinforcement learning policy network to obtain selection probabilities for several preset inference optimization policies. The selection probabilities are smoothed using an exponential moving average method and historical policy selection probabilities. Based on the processed probability results, the target inference optimization policy with the highest probability is determined from the several preset inference optimization policies. That is, the task type, inference progress information, and available resource status of the code generation task are first determined. The embedded representation of the task type, the encoding of the available resource status, and the summary information of the inference history (i.e., the inference progress information) are used together with the multi-granularity uncertainty score to construct the current state space vector, thereby encoding the current state. The specific formula for expressing the state space vector is as follows:

[0075] ;

[0076] in, The state space vector at time t represents the state space vector. Indicates the task type encoding. Indicates available computing resources. This indicates the inference progress. Next, the state space vector is input into a preset reinforcement learning policy network. The network calculates the initial selection probabilities of several preset inference optimization policies. The action space of the preset reinforcement learning policy network includes three models: efficient acceleration mode, deep exploration mode, and fuzzy processing mode, corresponding to different inference optimization directions. The specific formula for expressing the action space is as follows:

[0077] ;

[0078] in, The action space is defined as follows: the efficient acceleration mode corresponds to a preset inference acceleration strategy; the deep exploration mode corresponds to a preset exploration optimization strategy; and the fuzzy processing mode corresponds to a preset fuzzy processing strategy. Finally, to avoid overly aggressive strategy switching, an exponential moving average method is used, combined with historical strategy selection probabilities, to smooth the initial selection probabilities of consecutive time steps. Based on the processed probability results, the strategy with the highest probability is selected as the target inference optimization strategy from among the several preset inference optimization strategies. In this way, this embodiment constructs a state space vector based on multi-granularity uncertainty scores, task type, inference progress, and available resources, obtaining a structured decision-making basis, improving the adaptability of the strategy and the scenario, enabling the network to fully grasp the current inference state, and avoiding strategy misjudgment due to incomplete information. The preset smoothing decision mechanism smooths the strategy selection through exponential moving average, while incorporating historical selection experience, improving the stability and consistency of strategy selection. The reinforcement learning policy network evaluates the strategy selection probability based on the multi-dimensional state space vector, ensuring that the selected strategy can specifically alleviate the uncertainty problem in the current inference, taking into account both the adaptability and stability of the strategy.

[0079] Step S13: If the target reasoning optimization strategy is a preset reasoning acceleration strategy, then the reasoning is optimized by speculative decoding; if the target reasoning optimization strategy is a preset exploration optimization strategy, then the reasoning is optimized by using preset multi-path sampling technology and preset knowledge enhancement technology; if the target reasoning optimization strategy is a preset fuzzy processing strategy, then the reasoning is optimized by using multiple candidate outputs as the target reasoning output.

[0080] In this embodiment, the inference path and resource allocation are dynamically adjusted based on the output of the preset reinforcement learning strategy network. When low uncertainty is detected, i.e., the determined target inference optimization strategy is a preset inference acceleration strategy, aggressive speculative decoding is used to optimize the current code generation inference process. If high cognitive uncertainty is detected, i.e., a preset exploration optimization strategy, inference optimization is performed by combining preset multi-path sampling technology and preset knowledge enhancement technology to improve output reliability. If high data uncertainty is detected, i.e., a preset fuzzy processing strategy, a single definite answer is not required; instead, uncertainty is honestly expressed, and the optimization of this inference is completed by outputting multiple reasonable candidate results. The preset knowledge enhancement technology is implemented based on pre-configured external knowledge sources, maintaining a knowledge source index that includes search engine APIs (Application Programming Interfaces), structured knowledge graphs, professional databases, etc. That is, the preset inference optimization strategy of this application includes the preset inference acceleration strategy, the preset exploration optimization strategy, and the preset fuzzy processing strategy. This embodiment performs corresponding inference optimization operations for the determined strategies.

[0081] It should be noted that if the target inference optimization strategy is a preset inference acceleration strategy, the process of optimizing the inference through speculative decoding is as follows: If the target inference optimization strategy is a preset inference acceleration strategy, the task characteristics of the code generation task are determined, and a target draft model is selected from the preset draft model pool based on the task characteristics; the historical success rate is statistically analyzed using a preset sliding window, and the target step size adjustment parameter is determined through the historical success rate and the corresponding historical weight factor; the target step size for this speculative decoding is determined based on the target step size adjustment parameter and the multi-granularity uncertainty score, so as to optimize the speculative decoding of this inference according to the target step size. The preset draft model pool includes draft models of different sizes and levels of specialization. That is, if the target inference optimization strategy is a preset inference acceleration strategy, inference acceleration is achieved by dynamically adjusting the parameters of speculative decoding. First, the task characteristics of the code generation task are defined. Based on the task characteristics and performance requirements, a suitable target draft model is selected from the preset draft model pool. Then, a preset sliding window is used to statistically analyze the historical success rate of the most recent 100 speculative decoding attempts, and the target step size adjustment parameter is calculated in combination with the corresponding historical weight factor. The specific formula for determining the target step size adjustment parameter is as follows:

[0082] ;

[0083] Where n is the historical success rate; The historical weighting factor; These are the preset basic adjustment parameters; The target step size adjustment parameter is characterized. Then, based on the target step size adjustment parameter and the multi-granularity uncertainty score, the target step size for this speculative decoding is determined. The formula for determining the target step size is as follows:

[0084] ;

[0085] in, This represents the target step size for speculative decoding; Indicates the preset base step size; This represents the multi-granularity uncertainty score. That is, the target step size is dynamically adjusted according to the uncertainty level. When the uncertainty is low and the historical verification success rate is high, the system increases the speculative step size, up to a maximum of 8-10 tokens for parallel prediction. Finally, speculative decoding optimization processing is performed on the current inference process according to the target step size.

[0086] It should be further noted that if the target reasoning optimization strategy is a preset exploration optimization strategy, the processing flow for optimizing this reasoning using preset multi-path sampling technology and preset knowledge enhancement technology is as follows: If the target reasoning optimization strategy is a preset exploration optimization strategy, then the task domain and task type related to the code generation task are determined, and preset external knowledge sources are queried according to the task domain and task type. The query results are then filtered and sorted using a preset relevance evaluation model to obtain corresponding knowledge enhancement results. The target sampling path number for this multi-path sampling is determined based on the model's cognitive uncertainty score, the preset maximum number of sampling paths, and the preset minimum number of sampling paths. The target sampling path number is explored using a preset sampling strategy to generate several target candidate reasoning outputs in parallel based on the knowledge enhancement results. The preset sampling strategy is a sampling strategy composed of temperature sampling, kernel sampling, and typical sampling. The several target candidate reasoning outputs are integrated to generate the target reasoning output. That is, if the target reasoning optimization strategy is a preset exploration optimization strategy, firstly, the task domain and specific task type of the code generation task are determined. Based on this, preset external knowledge sources are queried, and the query results are filtered and sorted using a preset relevance evaluation model to ensure that the provided knowledge information is accurate and reliable, thus obtaining knowledge enhancement results. Then, the target sampling path number for this multi-path sampling is determined by combining the model's cognitive uncertainty score and the preset maximum and minimum number of sampling paths. The specific formula for determining the target sampling path number is as follows:

[0087] ;

[0088] in, Indicates the number of target sampling paths; and These represent the minimum and maximum number of sampling paths, respectively; This represents the cognitive uncertainty score of the model; The representation is rounded up. Then, using a preset sampling strategy consisting of temperature sampling, kernel sampling, and typical sampling, the sampling paths for the target sampling path number are explored. Combined with the knowledge enhancement results, several target candidate inference outputs are generated in parallel. Finally, these candidate outputs are independently evaluated for quality, and the final target inference output is generated through a voting mechanism or a weighted average based on confidence.

[0089] It should be noted that if the target inference optimization strategy is a preset fuzzy processing strategy, the processing flow for optimizing multiple candidate outputs as the target inference output is as follows: If the target inference optimization strategy is a preset fuzzy processing strategy, then the several target candidate inference outputs are generated using preset multi-path sampling technology and preset knowledge enhancement technology; the several target candidate inference outputs are analyzed to obtain the corresponding confidence analysis results, and the set of multiple candidate outputs including the several target candidate inference outputs, the confidence analysis results, and the uncertainty score of the input data is determined as the target inference output. That is, if the target inference optimization strategy is a preset fuzzy processing strategy, firstly, several target candidate inference outputs are generated using preset multi-path sampling technology and preset knowledge enhancement technology, then these candidate outputs are analyzed to obtain confidence analysis results, including confidence assessment and supporting reasons, while clearly pointing out the ambiguous or contradictory parts in the input information, suggesting that the user provide more information or seek professional advice, and finally, the set of multiple candidate outputs including the several target candidate inference outputs, the confidence analysis results, and the uncertainty score of the input data is determined as the target inference output. The specific formula for expressing the set of multiple candidate outputs is as follows:

[0090] ;

[0091] in, This represents the set of multiple candidate outputs; This represents the k-th candidate output, where K represents the number of candidates. In this way, this embodiment executes the corresponding inference optimization method according to the determined strategy, making the inference optimization more targeted; it selects the target draft model based on task characteristics, providing more accurate basic predictions for speculative decoding and reducing invalid computations; it determines the step size adjustment parameter by statistically analyzing historical success rates through a sliding window and combining them with historical weight factors, making the step size adjustment more closely match actual performance; it determines the target step size by combining multi-granularity uncertainty scores, dynamically adjusting the decoding step size while ensuring generation quality, ultimately achieving a significant improvement in inference efficiency through speculative decoding, while maintaining the reliability of code generation, achieving a balance between efficiency and quality; it accurately queries and filters external knowledge based on the task domain and type, effectively compensating for the model's own knowledge limitations and improving the generated content. The system ensures professionalism and accuracy; it dynamically determines the number of sampling paths based on model cognitive uncertainty, increasing paths to expand the exploration range when uncertainty is high and reducing paths to improve efficiency when uncertainty is low, achieving a balance between exploration depth and resource consumption; the combined use of multiple sampling strategies avoids the limitations of a single sampling method, while taking into account the diversity and rationality of the output, effectively reducing generation bias in high uncertainty scenarios and improving the accuracy and innovation of code logic; the system incorporates confidence analysis results and input data uncertainty scores into the target inference output, and the fuzzy processing strategy of outputting multiple candidate results can provide users with more reliable options when inference uncertainty is high and it is difficult to determine a unique optimal solution, reducing the risks that may exist with a single output.

[0092] Step S14: Determine the inference result obtained after this inference optimization process, and evaluate the decision effect of this inference optimization strategy based on the inference result, so as to optimize the preset reinforcement learning strategy network based on the evaluation result.

[0093] In this embodiment, the final inference result obtained from this inference optimization process is first determined. Then, based on the inference result, the decision-making effect of the selected inference optimization strategy is evaluated. Finally, based on the evaluation result, the parameters of the preset reinforcement learning strategy network are adjusted and the model is optimized. The inference result is detailed performance data of the inference, including multi-dimensional indicators such as inference time, output quality, uncertainty change trajectory, user satisfaction, and uncertainty prediction accuracy, which are obtained in a non-intrusive manner.

[0094] It should be noted that the process of obtaining the evaluation results and optimizing the preset reinforcement learning strategy network based on them is as follows: The inference results are compared with the multi-granularity uncertainty scores, and the inference optimization strategy is verified through an A / B testing mechanism to evaluate its decision-making effect and obtain the corresponding effect evaluation results. Using the effect evaluation results, the preset reinforcement learning strategy network is optimized and adjusted based on preset incremental learning techniques and a preset reward function. That is, firstly, the actual inference results are compared with the predicted multi-granularity uncertainty scores to continuously improve the accuracy of uncertainty quantification. By analyzing which situations the uncertainty prediction is too high or too low, the system can identify the deviation patterns of the quantization algorithm and perform targeted optimization. Simultaneously, the inference optimization strategy is verified using A / B testing (an optimization method) and causal inference. A small portion (e.g., 5-10%) of the current inference task is randomly assigned to different policy versions for parallel processing. By comparing the performance of these sub-tasks, the optimality of the current policy decision is verified, thereby evaluating the policy's decision-making effectiveness and obtaining corresponding performance evaluation results. Then, using these performance evaluation results, combined with a preset incremental learning technique, new feedback data is combined with historical data through an experience replay mechanism, and a preset reward function, the preset reinforcement learning policy network is optimized and adjusted. It should be further noted that the preset reward function comprehensively considers inference speed, output quality, and resource consumption, and its specific formula is as follows:

[0095] ;

[0096] in, This indicates a reward for reasoning speed; Indicates the calculation cost; This indicates the output quality score; Indicates a reliability bonus; This represents the corresponding weight parameters; The preset reward function is used to represent this. It should be noted that this embodiment also includes a security mechanism. During policy execution, the system continuously monitors the execution effect. When a significant performance degradation is detected, it quickly rolls back to a stable alternative version to ensure service reliability. In this way, this embodiment evaluates the policy decision effect based on the inference results, verifying the adaptability of the optimized strategy to the current task scenario and providing feedback for subsequent policy network optimization. The A / B testing mechanism provides a comparative basis for policy verification, ensuring the objectivity and reliability of the effect evaluation results. Based on the effect evaluation results, incremental learning technology is used to optimize the reinforcement learning policy network. This allows the network to continuously learn from experience without forgetting existing knowledge, gradually improving the accuracy of policy selection and enabling the network to continuously adapt to the inference needs of different scenarios. The introduction of the preset reward function guides the network to adjust towards better policy decisions, enhancing the overall performance of the large code generation model.

[0097] As can be seen from the above, in the large model inference process of target code generation in this embodiment, for each newly generated Token, multi-granularity uncertainty quantification steps at the Token level, phrase level, inference step level, model cognition level, and input data level are executed synchronously and in parallel to obtain the corresponding multi-granularity uncertainty score. Then, based on these scores, the current state space vector is constructed. With the help of a preset reinforcement learning strategy network, combined with a preset smoothing decision mechanism and the state space vector, the selection probability of multiple preset inference optimization strategies is evaluated, and then the target inference optimization strategy is determined. Differentiated optimization is performed according to the different target strategies: if it is a preset inference acceleration strategy, inference is optimized by speculative decoding; if it is a preset exploration optimization strategy, it is optimized by using preset multi-path sampling technology and knowledge enhancement technology; if it is a preset fuzzy processing strategy, it is optimized by outputting multiple candidate results. Finally, the result after inference optimization is determined, the decision effect of this strategy is evaluated based on the result, and the preset reinforcement learning strategy network is optimized based on the evaluation result. In this way, through the above-described process in the embodiments of this application, the real-time parallel multi-granularity uncertainty quantification during the inference process can comprehensively capture the sources of uncertainty in each stage of inference, avoiding risk omissions caused by single-dimensional evaluation; constructing a state space vector based on uncertainty scores and combining it with reinforcement learning to evaluate policy probabilities ensures that policy selection not only fits the real-time state of the current inference but also avoids inference fluctuations caused by extreme policies through a smooth decision-making mechanism, thereby improving the rationality and stability of policy selection; inference acceleration strategies can improve inference efficiency in high-efficiency scenarios, exploration optimization strategies can enhance the accuracy of complex inference, and fuzzy processing strategies can meet the reliability requirements of results under high uncertainty; evaluating and iterating the reinforcement learning policy network based on inference results allows the policy network to continuously learn effective experience in inference optimization, taking into account inference efficiency, accuracy, and reliability; transforming uncertainty quantification from post-evaluation to real-time decision-driven, significantly improving inference speed while ensuring output quality, thereby achieving synergistic optimization of inference efficiency and output reliability in large code generation models.

[0098] Accordingly, see Figure 3 As shown in the embodiments of this application, an inference optimization apparatus for a large code generation model is also provided, including:

[0099] The step execution module 11 is used to perform multi-granularity uncertainty quantification steps in parallel during the inference process of the target code generation large model for code generation tasks, for each newly generated token, to obtain the corresponding multi-granularity uncertainty score; the multi-granularity uncertainty quantification includes token-level uncertainty quantification, phrase-level uncertainty quantification, inference step-level uncertainty quantification, model cognition uncertainty quantification, and input data uncertainty quantification; the multi-granularity uncertainty score includes target token-level uncertainty score, target phrase-level uncertainty score, inference step-level uncertainty score, model cognition uncertainty score, and input data uncertainty score;

[0100] The probability evaluation module 12 is used to construct the current state space vector based on the multi-granularity uncertainty score, and to evaluate the selection probability of several preset inference optimization strategies based on the preset smoothing decision mechanism and the state space vector using a preset reinforcement learning policy network, so as to determine the corresponding target inference optimization strategy according to the obtained selection probability; the preset smoothing decision mechanism is a mechanism for smoothing the strategy selection of continuous time steps through the exponential moving average method.

[0101] The optimization processing module 13 is used to optimize the current inference by speculative decoding if the target inference optimization strategy is a preset inference acceleration strategy; to optimize the current inference by using preset multi-path sampling technology and preset knowledge enhancement technology if the target inference optimization strategy is a preset exploration optimization strategy; and to optimize the current inference by using multiple candidate outputs as the target inference output if the target inference optimization strategy is a preset fuzzy processing strategy.

[0102] The effect evaluation module 14 is used to determine the inference result obtained after this inference optimization process, and evaluate the decision effect of this inference optimization strategy based on the inference result, so as to optimize the preset reinforcement learning strategy network based on the evaluation result.

[0103] In some specific embodiments, the step execution module 11 may specifically include:

[0104] The entropy processing unit is used to determine the probability distribution vector of the newly added Token, and to calculate the information entropy using the probability distribution vector and a preset entropy calculation algorithm to quantify the uncertainty at the Token level, and to normalize the obtained predicted entropy value to obtain the corresponding target Token level uncertainty score; the preset entropy calculation algorithm includes a pre-calculated logarithmic lookup table and vectorization operations.

[0105] The weight extraction unit is used to extract several attention weights related to the newly added Token from the last preset number of layers of the target code generation model, so as to determine the target average attention weight of the newly added Token, and determine several average attention weights and several Token-level uncertainty scores of several historically generated Tokens.

[0106] The score aggregation unit is used to perform weighted aggregation of the target token-level uncertainty score and the several token-level uncertainty scores based on the several average attention weights and the target average attention weight, so as to perform phrase-level uncertainty quantification and obtain the corresponding target phrase-level uncertainty score.

[0107] A complexity determination unit is used to determine the reasoning complexity of the current reasoning step, so as to determine the perturbation intensity based on the reasoning complexity; the perturbation intensity includes the number of perturbations and the perturbation amplitude;

[0108] The perturbation application unit is used to obtain a first mapping from the current inference step to the final inference output through the target fast forward propagation technique, and to apply a perturbation to the current inference step based on the perturbation intensity, so as to obtain a second mapping from the current inference step to the final inference output after the perturbation through the target fast forward propagation technique, and to measure the output difference between the first mapping and the second mapping to perform inference step-level uncertainty quantification and obtain the corresponding inference step-level uncertainty score.

[0109] The data analysis unit is used to analyze the model parameters of the large model generated from the target code based on variance decomposition technology to obtain the corresponding model cognitive uncertainty score, and to analyze the current input data to obtain the corresponding input data uncertainty score.

[0110] In some specific embodiments, the probability evaluation module 12 may specifically include:

[0111] A state determination unit is used to determine the task type, inference progress information, and available resource status of the code generation task, so as to construct the current state space vector using the task type, the inference progress information, the available resource status, and the multi-granularity uncertainty score;

[0112] A vector input unit is used to input the state space vector into a preset reinforcement learning policy network, so as to obtain the selection probability of several preset inference optimization policies through the preset reinforcement learning policy network.

[0113] The probability smoothing unit is used to smooth the selection probability using the exponential moving average method and historical strategy selection probability, so as to determine the target inference optimization strategy with the highest probability from the several preset inference optimization strategies based on the probability results obtained after processing.

[0114] In some specific embodiments, the optimization processing module 13 may specifically include:

[0115] The model selection unit is used to determine the task characteristics of the code generation task if the target inference optimization strategy is a preset inference acceleration strategy, and select a target draft model from the preset draft model pool according to the task characteristics.

[0116] The parameter determination unit is used to statistically analyze the historical success rate using a preset sliding window, and to determine the target step size adjustment parameter through the historical success rate and the corresponding historical weight factor.

[0117] The step size determination unit is used to determine the target step size for this speculative decoding based on the target step size adjustment parameter and the multi-granularity uncertainty score, so as to optimize the speculative decoding of this inference according to the target step size.

[0118] In some specific embodiments, the optimization processing module 13 may specifically include:

[0119] The type determination unit is used to determine the task domain and task type related to the code generation task if the target reasoning optimization strategy is a preset exploration optimization strategy, so as to query a preset external knowledge source according to the task domain and the task type, and use a preset relevance evaluation model to filter and sort the obtained query results to obtain the corresponding knowledge enhancement results.

[0120] The quantity determination unit is used to determine the target number of sampling paths for this multi-path sampling based on the model's cognitive uncertainty score, the preset maximum number of sampling paths, and the preset minimum number of sampling paths.

[0121] The path exploration unit is used to explore a number of target sampling paths using a preset sampling strategy, and to generate several target candidate inference outputs in parallel based on the knowledge enhancement results; the preset sampling strategy is a sampling strategy composed of temperature sampling, kernel sampling, and typical sampling.

[0122] The output integration unit is used to integrate the several target candidate inference outputs to generate the target inference output.

[0123] In some specific embodiments, the optimization processing module 13 may specifically include:

[0124] The output generation unit is used to generate the plurality of target candidate inference outputs by using a preset multi-path sampling technique and a preset knowledge enhancement technique if the target inference optimization strategy is a preset fuzzy processing strategy.

[0125] The output analysis unit is used to analyze the plurality of target candidate inference outputs to obtain the corresponding confidence analysis results, and to determine the multi-candidate output set, including the plurality of target candidate inference outputs, the confidence analysis results, and the uncertainty score of the input data, as the target inference output.

[0126] In some specific embodiments, the effect evaluation module 14 may specifically include:

[0127] The strategy verification unit is used to compare the inference result with the multi-granularity uncertainty score, and to verify the inference optimization strategy through an A / B testing mechanism to evaluate the decision-making effect of the inference optimization strategy and obtain the corresponding effect evaluation result.

[0128] The network adjustment unit is used to optimize and adjust the preset reinforcement learning strategy network based on the effect evaluation results, a preset incremental learning technique, and a preset reward function.

[0129] Furthermore, embodiments of this application also disclose an electronic device, Figure 4 This is a structural diagram of an electronic device 20 according to an exemplary embodiment. The content of the diagram should not be construed as limiting the scope of this application. Specifically, the electronic device 20 may include: at least one processor 21, at least one memory 22, a power supply 23, a communication interface 24, an input / output interface 25, and a communication bus 26. The memory 22 stores a computer program, which is loaded and executed by the processor 21 to implement the relevant steps in the inference optimization method for the large code generation model disclosed in any of the foregoing embodiments. Furthermore, the electronic device 20 in this embodiment may specifically be an electronic computer.

[0130] In this embodiment, the power supply 23 is used to provide operating voltage for each hardware device on the electronic device 20; the communication interface 24 can create a data transmission channel between the electronic device 20 and external devices, and the communication protocol it follows can be any communication protocol applicable to the technical solution of this application, and is not specifically limited here; the input / output interface 25 is used to acquire external input data or output data to the outside world, and its specific interface type can be selected according to specific application needs, and is not specifically limited here.

[0131] In addition, the memory 22, as a carrier for resource storage, can be a read-only memory, random access memory, disk or optical disk, etc. The resources stored thereon can include operating system 221, computer program 222, etc., and the storage method can be temporary storage or permanent storage.

[0132] The operating system 221 is used to manage and control the various hardware devices on the electronic device 20 and the computer program 222, which may be Windows Server, Netware, Unix, Linux, etc. In addition to including a computer program capable of performing the inference optimization method for the large code generation model executed by the electronic device 20 as disclosed in any of the foregoing embodiments, the computer program 222 may further include computer programs capable of performing other specific tasks.

[0133] Furthermore, this application also discloses a computer-readable storage medium for storing a computer program; wherein, when the computer program is executed by a processor, it implements the aforementioned inference optimization method for a large code generation model. Specific steps of this method can be found in the corresponding content disclosed in the foregoing embodiments, and will not be repeated here.

[0134] The various embodiments in this specification are described in a progressive manner, with each embodiment focusing on its differences from other embodiments. Similar or identical parts between embodiments can be referred to interchangeably. For the apparatus disclosed in the embodiments, since it corresponds to the method disclosed in the embodiments, the description is relatively simple; relevant parts can be referred to in the method section.

[0135] Those skilled in the art will further recognize that the units and algorithm steps of the various examples described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, computer software, or a combination of both. To clearly illustrate the interchangeability of hardware and software, the components and steps of the various examples have been generally described in terms of functionality in the foregoing description. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this application.

[0136] The steps of the methods or algorithms described in conjunction with the embodiments disclosed herein can be implemented directly by hardware, a software module executed by a processor, or a combination of both. The software module can be located in random access memory (RAM), main memory, read-only memory (ROM), electrically programmable ROM, electrically erasable programmable ROM, registers, hard disk, removable disk, CD-ROM, or any other form of storage medium known in the art.

[0137] Finally, it should be noted that in this document, relational terms such as "first" and "second" are used only to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitations, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes said element.

[0138] The technical solutions provided in this application have been described in detail above. Specific examples have been used to illustrate the principles and implementation methods of this application. The descriptions of the above embodiments are only for the purpose of helping to understand the methods and core ideas of this application. At the same time, for those skilled in the art, there will be changes in the specific implementation methods and application scope based on the ideas of this application. Therefore, the content of this specification should not be construed as a limitation of this application.

Claims

1. A method for optimizing inference in a large code generation model, characterized in that, include: During the reasoning process of code generation task in the target code generation large model, for each newly generated token, the multi-granularity uncertainty quantification step in the reasoning process is executed in parallel to obtain the corresponding multi-granularity uncertainty score. The multi-granularity uncertainty quantification includes token-level uncertainty quantification, phrase-level uncertainty quantification, inference step-level uncertainty quantification, model cognition uncertainty quantification, and input data uncertainty quantification; the multi-granularity uncertainty score includes target token-level uncertainty score, target phrase-level uncertainty score, inference step-level uncertainty score, model cognition uncertainty score, and input data uncertainty score; The current state space vector is constructed based on the multi-granularity uncertainty score, and the selection probability of several preset inference optimization strategies is evaluated based on the preset smooth decision-making mechanism and the state space vector using a preset reinforcement learning policy network, so as to determine the corresponding target inference optimization strategy according to the obtained selection probability. The preset smoothing decision-making mechanism is a mechanism that smooths the strategy selection for consecutive time steps using the exponential moving average method. If the target inference optimization strategy is a preset inference acceleration strategy, then the inference is optimized by speculative decoding. If the target reasoning optimization strategy is a preset exploration optimization strategy, then the preset multi-path sampling technology and preset knowledge enhancement technology are used to optimize the reasoning process. If the target reasoning optimization strategy is a preset fuzzy processing strategy, then the reasoning is optimized by using multiple candidate outputs as the target reasoning output. The inference result obtained after this inference optimization process is determined, and the decision-making effect of this inference optimization strategy is evaluated based on the inference result, so as to optimize the preset reinforcement learning strategy network based on the evaluation result.

2. The inference optimization method for large code generation models according to claim 1, characterized in that, The multi-granularity uncertainty quantification step in the parallel execution inference process, to obtain the corresponding multi-granularity uncertainty score, includes: The probability distribution vector of the newly added Token is determined, and the information entropy is calculated using the probability distribution vector and a preset entropy calculation algorithm to quantify the uncertainty at the Token level. The obtained predicted entropy value is then normalized to obtain the corresponding target Token level uncertainty score. The preset entropy calculation algorithm includes a pre-calculated logarithmic lookup table and vectorization operations. Extract several attention weights related to the newly added token from the last preset number of layers of the target code generation large model, in order to determine the target average attention weight of the newly added token, and determine several average attention weights and several token-level uncertainty scores of several historically generated tokens; Based on the aforementioned average attention weights and the target average attention weights, the target token-level uncertainty score and the aforementioned token-level uncertainty scores are weighted and aggregated to perform phrase-level uncertainty quantification, thereby obtaining the corresponding target phrase-level uncertainty score. Determine the reasoning complexity of the current reasoning step, and determine the perturbation intensity based on the reasoning complexity; the perturbation intensity includes the number of perturbations and the perturbation amplitude; The first mapping from the current inference step to the final inference output is obtained through the target fast forward propagation technique, and the current inference step is perturbed based on the perturbation intensity. The second mapping from the current inference step to the final inference output after the perturbation is obtained through the target fast forward propagation technique. The output difference between the first mapping and the second mapping is measured to quantify the uncertainty at the inference step level and obtain the corresponding uncertainty score at the inference step level. Based on variance decomposition technology, the model parameters of the target code generation large model are analyzed to obtain the corresponding model cognitive uncertainty score, and the current input data is analyzed to obtain the corresponding input data uncertainty score.

3. The inference optimization method for large code generation models according to claim 1, characterized in that, The process of constructing a current state space vector based on the multi-granularity uncertainty score, and using a preset reinforcement learning policy network to evaluate the selection probabilities of several preset inference optimization strategies based on a preset smooth decision-making mechanism and the state space vector, and determining the corresponding target inference optimization strategy according to the obtained selection probabilities, includes: The task type, inference progress information, and available resource status of the code generation task are determined, and the current state space vector is constructed using the task type, the inference progress information, the available resource status, and the multi-granularity uncertainty score. The state space vector is input into a preset reinforcement learning policy network to obtain the selection probabilities of several preset inference optimization policies through the preset reinforcement learning policy network; The selection probability is smoothed using the exponential moving average method and historical strategy selection probability, so that the target inference optimization strategy with the highest probability is determined from the several preset inference optimization strategies based on the probability results obtained after processing.

4. The inference optimization method for large code generation models according to claim 1, characterized in that, If the target inference optimization strategy is a preset inference acceleration strategy, then the inference is optimized through speculative decoding, including: If the target inference optimization strategy is a preset inference acceleration strategy, then the task characteristics of the code generation task are determined, and a target draft model is selected from the preset draft model pool according to the task characteristics. The historical success rate is statistically analyzed using a preset sliding window, and the target step size adjustment parameter is determined by the historical success rate and the corresponding historical weighting factor. The target step size for this speculative decoding is determined based on the target step size adjustment parameter and the multi-granularity uncertainty score, so as to optimize the speculative decoding of this inference according to the target step size.

5. The inference optimization method for large code generation models according to claim 1, characterized in that, If the target reasoning optimization strategy is a preset exploration optimization strategy, then the current reasoning is optimized using preset multi-path sampling technology and preset knowledge enhancement technology, including: If the target reasoning optimization strategy is a preset exploration optimization strategy, then the task domain and task type related to the code generation task are determined, and a preset external knowledge source is queried according to the task domain and the task type. The query results are then filtered and sorted using a preset relevance evaluation model to obtain the corresponding knowledge enhancement results. The target number of sampling paths for this multi-path sampling is determined based on the model's cognitive uncertainty score, the preset maximum number of sampling paths, and the preset minimum number of sampling paths. The target sampling path is explored using a preset sampling strategy, and several target candidate inference outputs are generated in parallel based on the knowledge enhancement results; the preset sampling strategy is a sampling strategy composed of temperature sampling, kernel sampling, and typical sampling. The inference outputs of the several target candidates are integrated to generate the target inference output.

6. The inference optimization method for large code generation models according to claim 5, characterized in that, If the target inference optimization strategy is a preset fuzzy processing strategy, then the inference is optimized by using multiple candidate outputs as the target inference output, including: If the target reasoning optimization strategy is a preset fuzzy processing strategy, then the preset multi-path sampling technology and preset knowledge enhancement technology are used to generate the several target candidate reasoning outputs; The several target candidate inference outputs are analyzed to obtain the corresponding confidence analysis results, and the set of multiple candidate outputs including the several target candidate inference outputs, the confidence analysis results, and the uncertainty score of the input data is determined as the target inference output.

7. The inference optimization method for a large code generation model according to any one of claims 1 to 6, characterized in that, The step of evaluating the decision-making effect of the current inference optimization strategy based on the inference result, and optimizing the preset reinforcement learning strategy network based on the obtained evaluation result, includes: The inference results are compared with the multi-granularity uncertainty score, and the inference optimization strategy is verified by the A / B testing mechanism to evaluate the decision-making effect of the inference optimization strategy and obtain the corresponding effect evaluation results. Using the performance evaluation results, the preset reinforcement learning strategy network is optimized and adjusted based on preset incremental learning techniques and preset reward functions.

8. An inference optimization device for a large code generation model, characterized in that, include: The step execution module is used to perform multi-granularity uncertainty quantification steps in parallel during the inference process of code generation task in the target code generation large model, so as to obtain the corresponding multi-granularity uncertainty score. The multi-granularity uncertainty quantification includes token-level uncertainty quantification, phrase-level uncertainty quantification, inference step-level uncertainty quantification, model cognition uncertainty quantification, and input data uncertainty quantification; the multi-granularity uncertainty score includes target token-level uncertainty score, target phrase-level uncertainty score, inference step-level uncertainty score, model cognition uncertainty score, and input data uncertainty score; The probability evaluation module is used to construct the current state space vector based on the multi-granularity uncertainty score, and to evaluate the selection probability of several preset inference optimization strategies based on the preset smooth decision-making mechanism and the state space vector using a preset reinforcement learning policy network, so as to determine the corresponding target inference optimization strategy according to the obtained selection probability. The preset smoothing decision-making mechanism is a mechanism that smooths the strategy selection for consecutive time steps using the exponential moving average method. An optimization processing module is used to optimize the current inference by speculative decoding if the target inference optimization strategy is a preset inference acceleration strategy. If the target reasoning optimization strategy is a preset exploration optimization strategy, then the preset multi-path sampling technology and preset knowledge enhancement technology are used to optimize the reasoning process. If the target reasoning optimization strategy is a preset fuzzy processing strategy, then the reasoning is optimized by using multiple candidate outputs as the target reasoning output. The effect evaluation module is used to determine the inference result obtained after this inference optimization process, and evaluate the decision effect of this inference optimization strategy based on the inference result, so as to optimize the preset reinforcement learning strategy network based on the evaluation result.

9. An electronic device, characterized in that, include: Memory, used to store computer programs; A processor for executing the computer program to implement the inference optimization method for a large code generation model as described in any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that, Used to store computer programs; wherein, when the computer programs are executed by a processor, they implement the inference optimization method for a large code generation model as described in any one of claims 1 to 7.

Images