Parameter adjustment method of cloud device and electronic device

Abstract

The embodiment of the application provides a parameter adjustment method of a cloud device and an electronic device, and relates to the field of computers, wherein the method comprises the following steps: acquiring performance characteristics of a cloud device where a large language model is located, wherein the performance characteristics are used to indicate running characteristics of the cloud device in a working state; using a strategy network to perform prediction based on the performance characteristics to obtain a predicted execution action; performing strategy updating on the strategy network based on the performance characteristics and historical execution actions corresponding to the performance characteristics, wherein the historical execution actions comprise adjustment actions that have been performed on the performance characteristics of the cloud device; performing instruction mapping on the predicted execution action to obtain at least one parameter adjustment instruction corresponding to the predicted execution action; and executing the at least one parameter adjustment instruction in a register of the cloud device.

Description

TECHNICAL FIELD

[0001] Embodiments of the present application relate to the computer field, in particular, to a parameter adjustment method of a cloud device and an electronic device. BACKGROUND

[0002] In a cloud device (cloud computing device) environment, when a large language model plays a core role in a high-performance computing task, a traditional parameter adjustment method often uses a static configuration or a fixed rule-based adjustment method, which has poor parameter adjustment effect on the cloud device, resulting in insufficient or excessive performance optimization of the cloud device.

[0003] Therefore, there is a technical problem of poor parameter adjustment effect of the cloud device in the related art. SUMMARY

[0004] Embodiments of the present application provide a parameter adjustment method of a cloud device and an electronic device to at least solve the technical problem of poor parameter adjustment effect of the cloud device in the related art.

[0005] According to an embodiment of the present application, a parameter adjustment method of a cloud device is provided, comprising: obtaining a performance characteristic of a cloud device where a large language model is located, wherein the performance characteristic is used to indicate a running characteristic of the cloud device in a working state; using a policy network to perform prediction based on the performance characteristic to obtain a predicted execution action; performing policy update on the policy network based on the performance characteristic and a historical execution action corresponding to the performance characteristic, wherein the historical execution action includes an adjustment action that has been performed on the performance characteristic of the cloud device; performing instruction mapping on the predicted execution action to obtain at least one parameter adjustment instruction corresponding to the predicted execution action; and executing the at least one parameter adjustment instruction in a register of the cloud device.

[0006] According to another embodiment of the present application, a parameter adjustment device of a cloud device is provided, comprising: an obtaining unit configured to obtain a performance characteristic of a cloud device where a large language model is located, wherein the performance characteristic is used to indicate a running characteristic of the cloud device in a working state; a prediction unit configured to use a policy network to perform prediction based on the performance characteristic to obtain a predicted execution action; an updating unit configured to perform policy update on the policy network based on the performance characteristic and a historical execution action corresponding to the performance characteristic, wherein the historical execution action includes an adjustment action that has been performed on the performance characteristic of the cloud device; a mapping unit configured to perform instruction mapping on the predicted execution action to obtain at least one parameter adjustment instruction corresponding to the predicted execution action; and an adjustment unit configured to execute the at least one parameter adjustment instruction in a register of the cloud device.

[0007] According to another embodiment of the present application, a computer readable storage medium is also provided, and the computer readable storage medium stores a computer program, wherein the computer program is configured to execute the steps in any of the method embodiments described above when running.

[0008] According to another embodiment of the present application, an electronic device is also provided, comprising a memory and a processor, wherein the memory stores a computer program, and the processor is configured to execute the computer program to perform the steps in any of the method embodiments described above.

[0009] Through the embodiments provided by the present application, the performance characteristics of the cloud device where the large language model is located are collected in real time and dynamically predicted using a policy network, overcoming the limitations of static configuration, and the predicted execution actions can be directly converted into specific parameter adjustment instructions, such as being executed in the registers of the cloud device, quickly reflecting the optimization effect. The policy network not only generates predicted execution actions, but also learns and updates strategies according to the current and historical states (performance characteristics and historical execution actions), ensuring continuous improvement and adaptability of the optimization strategy, thereby improving the accuracy of the predicted execution actions, and achieving the technical effect of improving the parameter adjustment effect of the cloud device, solving the technical problem of poor parameter adjustment effect of the cloud device in the related art. BRIEF DESCRIPTION OF DRAWINGS

[0010] Figure 1 is a hardware structure block diagram of a parameter adjustment method of a cloud device according to an embodiment of the present application;

[0011] Figure 2 is a flowchart of a parameter adjustment method of a cloud device according to an embodiment of the present application;

[0012] Figure 3 is a technical schematic diagram of GPU large language model inference CPU parameter optimization based on model reinforcement learning according to an embodiment of the present application;

[0013] Figure 4 is a state space definition and real-time collection schematic diagram according to an embodiment of the present application;

[0014] Figure 5 is a PPO policy optimization technical schematic diagram of a hybrid action space according to an embodiment of the present application;

[0015] Figure 6 is a lightweight intermediate layer instruction mapping schematic diagram according to an embodiment of the present application;

[0016] Figure 7 is a multi-objective reward function dynamic adjustment schematic diagram according to an embodiment of the present application;

[0017] Figure 8is a schematic diagram of experience playback and online learning according to an embodiment of the present application;

[0018] Figure 9 is a structural block diagram of a parameter adjustment device of a cloud device according to an embodiment of the present application. DETAILED DESCRIPTION

[0019] Hereinafter, the embodiments of the present application will be described in detail with reference to the accompanying drawings and in conjunction with embodiments.

[0020] It should be noted that the terms "first", "second", and the like in the description and claims of the present application and the above-mentioned accompanying drawings are used to distinguish similar objects, and do not necessarily have to be used to describe a specific order or sequence. It should be understood that the data thus used can be interchanged under appropriate circumstances, so that the embodiments of the present application described herein can be implemented in an order other than that illustrated or described herein. In addition, the terms "include" and "have" and any variations thereof are intended to cover non-exclusive inclusion, for example, a process, method, system, product or device including a series of steps or units does not have to be limited to only those steps or units clearly listed, but can include other steps or units not clearly listed or inherent to the process, method, product or device.

[0021] The method embodiments provided in the embodiments of the present application can be executed in a computer terminal or similar computing device. Taking a computer terminal as an example, Figure 1 is a hardware structural block diagram of a computer terminal of a parameter adjustment method of a cloud device according to an embodiment of the present application. As Figure 1 shown, the computer terminal can include one or more (only one is shown in Figure 1 ) processor 102 (the processor 102 can include but is not limited to a processing device such as a microprocessor MCU or a programmable logic device FPGA) and a memory 104 for storing data, wherein the above-mentioned computer terminal can further include a transmission device 106 for communication function and an input and output device 108. Those skilled in the art can understand that Figure 1 the structure shown is only schematic, which does not limit the structure of the above-mentioned computer terminal. For example, the computer terminal can further include more or less components than Figure 1 shown, or have a different configuration than Figure 1 shown.

[0022] The memory 104 can be used to store computer programs, such as software programs of application software and modules, such as a computer program corresponding to the mapping relationship determination method in the embodiments of the present application. The processor 102 performs various functional applications and data processing by running the computer programs stored in the memory 104, that is, implements the above method. The memory 104 can include a high-speed random access memory, and can also include a non-volatile memory, such as one or more magnetic storage devices, flash memories, or other non-volatile solid-state memories. In some examples, the memory 104 can further include a memory remotely arranged with respect to the processor 102, which can be connected to the computer terminal through a network. Examples of the above network include but are not limited to the Internet, an intranet, a local area network, a mobile communication network, and a combination thereof.

[0023] The transmission device 106 is used to receive or send data via a network. Specific examples of the above network can include a wireless network provided by a communication service provider of the computer terminal. In one example, the transmission device 106 includes a network adapter (Network Interface Controller, NIC) which can be connected to other network devices through a base station so as to communicate with the Internet. In one example, the transmission device 106 can be a radio frequency (Radio Frequency, RF) module used for wireless communication with the Internet.

[0024] As an optional solution, a parameter adjustment method of a cloud device is shown in the following. Figure 2 The specific steps include:

[0025] S202, obtaining a performance characteristic of a cloud device where a large language model is located, wherein the performance characteristic is used to indicate a running characteristic of the cloud device in a working state;

[0026] S204, using a policy network to perform prediction based on the performance characteristic to obtain a predicted execution action;

[0027] S206, performing policy update on the policy network based on the performance characteristic and a historical execution action corresponding to the performance characteristic, wherein the historical execution action includes an adjustment action that has been performed on the performance characteristic of the cloud device;

[0028] S208, performing instruction mapping on the predicted execution action to obtain at least one parameter adjustment instruction corresponding to the predicted execution action;

[0029] S210, executing the at least one parameter adjustment instruction in a register of the cloud device.

[0030] Optionally, in this embodiment, large language models, such as those based on the Transformer architecture, are typically deployed in a cloud environment due to their complex structure and large parameter quantity, utilizing powerful computing resources (such as GPU clusters) for training and inference. Cloud devices are physical or virtual hardware facilities that host and execute these models, including computing nodes, storage devices, and network components.

[0031] Optionally, in this embodiment, performance characteristics refer to a series of indicators that can quantify and describe the system health and resource usage efficiency of cloud devices when executing large language model inference or other computing tasks. These characteristics typically include but are not limited to: GPU memory bandwidth utilization, CPU cache hit rate, inter-process communication average latency, CPU power consumption, CPU temperature.

[0032] Optionally, in this embodiment, the performance characteristics of cloud devices are important basis for monitoring and evaluating the current running state of cloud devices, directly related to system efficiency, computing delay, energy consumption and stability. In the inference process of large language models, the changes of performance characteristics can reflect whether the cooperation efficiency between GPU and CPU is good, whether the prefetch strategy is reasonable, whether the cache allocation is appropriate, whether the thread scheduling is optimized, etc.

[0033] Optionally, in this embodiment, the purpose of obtaining performance characteristics is to dynamically adjust the parameters on the cloud device to adapt to the changes in resource requirements during the inference process of large language models, and to improve the overall performance of the system. For example, if the CPU cache hit rate is found to be low, it may mean that the cache allocation or prefetch strategy needs to be adjusted; if the GPU utilization is not high, it may need to optimize the thread scheduling strategy to fully utilize GPU resources; if the CPU temperature is too high, it may need to adjust the power control strategy to prevent overheating and frequency reduction.

[0034] Optionally, in this embodiment, the policy network is a neural network used in the reinforcement learning framework to predict the optimal action based on the current environment state. In this embodiment, the policy network combines the Long Short-Term Memory (LSTM) network and the Proximal Policy Optimization (PPO) algorithm to predict CPU parameter adjustment actions.

[0035] Optionally, in this embodiment, action prediction is the optimization action predicted by the policy network based on the current performance characteristics, including but not limited to prefetch aggressiveness, cache allocation ratio, and thread scheduling strategy adjustment.

[0036] Optionally, in this embodiment, the historical execution action refers to the CPU parameter adjustment action that the system has performed in the past under the same or similar performance characteristics. This part of information is used for the continuous learning and policy optimization of the policy network. The predicted execution action is the to-be-executed action for adjusting the CPU parameters predicted by the policy network based on the current conditions. The parameter adjustment instruction is the instruction set converted from the predicted execution action and directly acting on the CPU register, which is used to dynamically adjust the prefetch strategy, cache allocation, and thread scheduling, etc.

[0037] Optionally, in this embodiment, the GPU memory bandwidth utilization, CPU cache hit rate, and other indicators are monitored in real time by hardware sensors such as GPU power sensors and CPU temperature sensors. Then, the data acquisition module is used for summarizing, and the sliding average filter is used to eliminate abnormal fluctuations, thereby generating a state vector reflecting the current system state.

[0038] Optionally, in this embodiment, a policy network combining LSTM and PPO is used, which inputs the current system state vector and the action executed last time to predict the next state and reward value. The hybrid action generated by the policy network includes the prefetch aggressiveness, cache allocation ratio, and thread scheduling strategy, which is processed through Gaussian sampling, Sigmoid normalization, or Softmax to ensure that the action meets the actual operation conditions.

[0039] Optionally, in this embodiment, the advantage function of the action is calculated according to the current state and its corresponding historical action, and the policy gradient is calculated through the PPO algorithm to update the policy network parameters to adapt to new workloads and model characteristics. The target network parameters are synchronized through a soft update mechanism, and the priority experience replay mechanism is used to ensure efficient learning.

[0040] Optionally, in this embodiment, the predicted execution action can be understood as a high-level action prediction result, which is mapped to low-level CPU register operation instructions (at least one parameter adjustment instruction) through a lightweight intermediate layer and is immediately executed in the register of the cloud device to realize the rapid adjustment of the CPU parameters. This mechanism ensures that the policy can be executed across different CPU architectures, maintaining the universality and low overhead of the optimized policy.

[0041] It can be understood that the embodiment constructs an optimization framework based on model reinforcement learning, which can dynamically adjust parameters such as prefetching strategy, cache allocation and thread scheduling according to real-time system running conditions, so as to achieve the best balance of delay reduction, GPU utilization improvement and CPU power consumption control. By capturing the time sequence characteristics of the system state through the LSTM network and combining the PPO algorithm for policy optimization, the method overcomes the limitations of static optimization, realizes global dynamic adjustment of CPU parameters, and improves the efficiency and energy efficiency ratio of large language model inference. In addition, through the online learning mechanism, the system can continuously adapt to new model architectures and load changes without human intervention to maintain excellent performance.

[0042] Through the embodiments provided in the present application, the performance characteristics of the cloud device where the large language model is located are collected in real time and dynamically predicted using a policy network, overcoming the limitations of static configuration, and the predicted execution actions can be directly converted into specific parameter adjustment instructions, such as being executed in the registers of the cloud device, quickly reflecting the optimization effect. The policy network not only generates predicted execution actions, but also learns and updates strategies based on the current and historical states (performance characteristics and historical execution actions), ensuring continuous improvement and adaptability of the optimization strategy, thereby improving the accuracy of the predicted execution actions, and achieving the technical effect of improving the parameter adjustment effect of the cloud device.

[0043] As an optional solution, based on the performance characteristics and the historical execution actions corresponding to the performance characteristics, the policy updating of the policy network comprises:

[0044] Performing time sequence prediction on the performance characteristics and the historical execution actions to obtain predicted performance characteristics and expected reward parameters of the cloud device;

[0045] Performing advantage difference calculation on the predicted performance characteristics, the expected reward parameters and the performance characteristics to obtain target advantage parameters of the policy network;

[0046] Updating the policy gradient of the policy network using the target advantage parameters.

[0047] Optionally, in the present embodiment, the time sequence prediction refers to the process of predicting the future state and reward parameters of the system, which is an important part of the learning and decision-making of the policy network. Through the time sequence prediction capability of the LSTM network, the policy network can predict the state of the large language model after action execution and the expected reward based on current and historical data.

[0048] Optionally, in the present embodiment, the advantage difference calculation is used to evaluate the quality of an action relative to the average action quality, i.e., whether the weighted sum of the immediate reward and the expected future reward brought by the action is better than the average reward predicted by the policy network. By calculating the advantage difference, the policy network can determine which parameter adjustment action is more beneficial, thereby guiding the optimization direction of the policy.

[0049] Optionally, in this embodiment, the policy gradient update is a core step in the reinforcement learning algorithm based on the policy network, aiming to update the parameters of the policy network to maximize the expected cumulative reward. By using the calculated target advantage parameters, the policy network can adjust its policy of predicting actions, achieving more efficient optimization of large language model inference performance.

[0050] Optionally, in this embodiment, by using the LSTM network to perform time series prediction on performance characteristics and historical execution actions, the predicted performance characteristics and expected reward parameters of the cloud device after taking the predicted execution action are obtained. This prediction process is based on the current state vector and the previous executed action to predict the next state and reward value. The time series prediction capability of the LSTM network can capture and model the long-term dependence of system state, thus providing more accurate prediction results for the policy network, facilitating the subsequent calculation of advantage difference.

[0051] After time series prediction, the predicted performance characteristics, expected reward parameters and current actual performance characteristics are compared, and the advantage difference of the action, i.e. the target advantage parameter, is calculated by the advantage function method. Here, the advantage difference calculation is not only based on the immediate reward, but also considers the long-term expected reward by discounting future rewards to form a long-term impact assessment of the current action. The target advantage parameter reflects the additional benefit of taking a certain action compared to the policy network's predicted average action, which is an important basis for the policy network to update its parameters.

[0052] Finally, the calculated target advantage parameter is used to update the policy gradient of the policy network. This process is based on the PPO algorithm, which updates the weight parameters in the policy network through gradient descent method to take actions that can improve the reward with a higher probability. The PPO algorithm limits the amplitude of policy update, avoiding large jumps in policy, ensuring the stability of the learning process. By continuously repeating this process, the policy network can gradually optimize its parameters, achieving continuous optimization of large language model inference performance.

[0053] Through the embodiments provided in this application, the limitations of static parameter configuration in handling dynamic load are overcome, and real-time adaptive adjustment of CPU parameters is achieved. Through the continuous learning ability of the policy network, the system can adapt to changes in different large language model architectures and workload characteristics without the need for manual reconfiguration to maintain high performance.

[0054] As an optional solution, the advantage difference calculation is performed on the predicted performance characteristics, expected reward parameters and performance characteristics to obtain the target advantage parameters of the policy network, including:

[0055] The state value calculation is performed on the predicted performance feature and the performance feature to obtain a first state value parameter corresponding to the predicted performance feature and a second state value parameter corresponding to the performance feature.

[0056] The target advantage parameter is obtained by multiplying the expected reward parameter by the first state value parameter and then subtracting the second state value parameter.

[0057] Optionally, in this embodiment, the predicted performance feature is a state that the system is expected to reach after performing a predicted action in the future, which is obtained by the policy network through time series prediction on the current system state and historical execution actions, including GPU memory bandwidth utilization, CPU cache hit rate and other key indicators.

[0058] Optionally, in this embodiment, the expected reward parameter is a reward value that is expected to be obtained according to the future state predicted by the policy network after taking the predicted execution action. The reward parameter includes the combination of immediate reward and future discounted reward, which is used to evaluate the long-term benefit of the action.

[0059] Optionally, in this embodiment, the state value calculation is used to evaluate the value of a state, i.e. the expected cumulative reward that can be obtained by taking a certain action from that state. State value is an important concept in reinforcement learning, which is used to guide the policy network to make better decisions.

[0060] Optionally, in this embodiment, the target advantage parameter is used to measure the additional benefit of taking a specific action compared to the average action quality, which is a key indicator for guiding policy updates in reinforcement learning algorithms. The calculation of the target advantage parameter is based on the difference between the expected reward and the two state value parameters, which can effectively evaluate the long-term potential value of the current action.

[0061] Optionally, in this embodiment, the state value calculation is performed on the predicted performance feature and the performance feature. This step involves calculating the state value of two states: the predicted performance feature and the current performance feature. The calculation of the state value parameter depends on the value function in the policy network, which can predict the expected cumulative reward that can be obtained from any given state according to the current policy. Through the value function, a first state value parameter corresponding to the predicted performance feature and a second state value parameter corresponding to the current performance feature are obtained.

[0062] On the basis of obtaining the first and second state value parameters, the target advantage parameter is calculated using the expected reward parameter.

[0063] Through the time series prediction ability of the prediction network (LSTM network) provided in the present application, the policy network can predict the cloud device state after performing future actions, that is, predict the performance characteristics. Subsequently, based on this predicted state and the immediate reward parameter, the target advantage parameter is obtained through state value calculation and reward discounting, which is an important basis for the policy network to update its policy gradient. This method enables the policy network to more accurately evaluate the long-term value of actions, thereby improving the decision quality of parameter adjustment, which is of great significance for realizing dynamic and adaptive large language model inference performance optimization.

[0064] As an optional solution, updating the policy gradient of the policy network using the target advantage parameter comprises:

[0065] obtaining a first execution probability corresponding to the predicted execution action and a second execution probability corresponding to the historical execution action, wherein the first execution probability is used to indicate the execution probability of the predicted execution action under the performance characteristics, and the second execution probability is used to indicate the execution probability of the historical execution action before the historical execution action is executed;

[0066] determining a log value corresponding to the first execution probability as a first gradient parameter corresponding to the policy network;

[0067] determining a clipping parameter value corresponding to the ratio of the first execution probability and the second execution probability as a second gradient parameter corresponding to the policy network;

[0068] updating the policy gradient of the policy network using the product of the first gradient parameter, the second gradient parameter, and the target advantage parameter.

[0069] Optionally, in the present embodiment, in the policy network, the first execution probability refers to the probability that the policy network predicts will take a certain specific action (predicted execution action) under the current system state (performance characteristics). The second execution probability refers to the probability that the policy network predicts will take a certain historical action before the historical execution action is executed. These two probabilities are part of the output of the policy network and reflect the tendency of the policy network to various possible actions.

[0070] Optionally, in the present embodiment, the log value is mainly used to calculate the first gradient parameter of the policy network. In reinforcement learning, the log probability is often used to calculate the policy gradient because it can simplify the mathematical expression of the gradient, making it easier to understand and calculate.

[0071] Optionally, in this embodiment, the clipping parameter is a mechanism used in reinforcement learning algorithms such as Proximal Policy Optimization (PPO) to limit the magnitude of policy updates. By limiting the ratio of the first execution probability to the second execution probability within a certain range, over-adjustment of the policy can be avoided, thereby maintaining the stability of the learning process.

[0072] Optionally, in this embodiment, the gradient parameter is the derivative required when the policy network updates its weights (i.e., parameters), which guides the direction of parameter adjustment. In this embodiment, the first gradient parameter and the second gradient parameter are calculated based on the clipping parameter value of the log execution probability and the execution probability ratio, respectively, and are key elements of the policy network gradient update.

[0073] Optionally, in this embodiment, the first execution probability of the predicted execution action and the second execution probability of the historical execution action are obtained from the policy network. These two probability values reflect the policy network's tendency to take these actions and are the basis for subsequent gradient calculation.

[0074] Next, the log value corresponding to the first execution probability is determined as the first gradient parameter. The greater the probability of the policy network predicting an execution action, the more significant the impact of this log value on the gradient, which helps the policy network to take these high-probability actions more in the future, thereby achieving performance optimization.

[0075] Then, the clipping parameter value corresponding to the ratio of the first execution probability and the second execution probability is calculated and determined as the second gradient parameter. The introduction of the clipping parameter is to limit the magnitude of policy updates and prevent the policy network from producing too much fluctuation during the update process, affecting the stability of the overall performance.

[0076] Finally, the gradient update value of the policy network is obtained by multiplying the first gradient parameter, the second gradient parameter, and the calculated target advantage parameter. This product result is used to update the parameters of the policy network, i.e., the policy gradient. Specifically, we use this product result to multiply the parameter gradient of the policy network, and adjust the weights of the policy network through the backpropagation method, so that the policy network is more likely to take actions that can bring higher target advantages in future decision-making.

[0077] Through the policy gradient update mechanism provided in this application, the information of immediate reward and future expected reward is combined, and the long-term value of the action is measured by the target advantage parameter, which guides the parameter adjustment of the policy network in combination with the first gradient parameter and the second gradient parameter. This gradient update method ensures that the policy network can achieve rapid and stable policy optimization in dynamic CPU parameter adjustment tasks, thereby significantly improving the inference efficiency of large language models on GPUs, optimizing inference latency, and controlling CPU power consumption.

[0078] As an optional solution, the performance characteristics and history are executed to perform timing prediction to obtain the predicted performance characteristics and the expected reward parameter of the cloud device, which comprises:

[0079] The inference delay characteristic parameter of the policy network is obtained, and the GPU utilization and the CPU power consumption of the cloud device are obtained from the performance characteristics, wherein the inference delay characteristic parameter is used to indicate the difference between the ratio of the inference delay of the policy network to the maximum threshold of the inference delay and 1;

[0080] The value of the inference delay characteristic parameter is multiplied by the first weight, the value of the GPU utilization is multiplied by the second weight, and the value of the CPU power consumption is multiplied by the third weight to obtain the expected reward parameter.

[0081] Optionally, in the embodiment, the inference delay characteristic parameter of the policy network is the inference time delay generated by the policy network in the process of processing the current performance characteristics and generating the predicted execution action, which is compared with the maximum threshold of the inference delay and the difference with 1. This parameter reflects the speed and efficiency of the policy network in decision-making.

[0082] Optionally, in the embodiment, the GPU utilization and the CPU power consumption are directly extracted from the performance characteristics, the GPU utilization reflects the activity level of the GPU in processing large language model inference tasks, and the CPU power consumption represents the energy consumption level when the CPU parameter adjustment strategy is executed. Both are key factors for calculating the expected reward parameter in the reinforcement learning framework.

[0083] Optionally, in the embodiment, in the calculation of the composite reward function, different performance indicators are given different weights, and the first weight, the second weight and the third weight determine the relative importance of each performance indicator in the overall reward calculation. Among them, the first weight is used to control the influence degree of the inference delay of the policy network on the expected reward, the second weight is used to emphasize the contribution of the GPU utilization, and the third weight is used to balance the negative influence of the CPU power consumption.

[0084] Optionally, in the embodiment, the inference delay characteristic parameter of the policy network in the current decision-making cycle is obtained. This parameter calculation is based on the comparison of the inference delay generated by the policy network with its maximum allowed threshold, which is used to reflect the real-time response speed of the policy network.

[0085] When the inference delay of the policy network approaches or is lower than the maximum threshold, the inference delay characteristic parameter approaches 0, indicating that the policy network performs well in fast decision-making; otherwise, the value of the inference delay characteristic parameter is larger, indicating that the decision-making speed of the policy network needs to be improved.

[0086] Next, the GPU utilization and CPU power consumption of the cloud device are extracted from the performance characteristics, which are important indicators for evaluating the efficiency of GPU-accelerated inference and the control of CPU power consumption.

[0087] Finally, the calculation of the expected reward parameter integrates the influence of the three dimensions of policy network inference delay, GPU utilization and CPU power consumption.

[0088] Through the embodiments provided in the present application, fine regulation of the policy network is achieved, ensuring that the parameter adjustment strategy can balance between different performance indicators, avoiding the problem of over-optimizing one aspect while ignoring others.

[0089] As an optional solution, the method further comprises:

[0090] In the case where the inference delay of the policy network is detected to be greater than the preset delay threshold, the first weight is increased;

[0091] In the case where the CPU temperature of the cloud device is detected to be greater than the preset temperature threshold, the third weight is increased.

[0092] Optionally, in the present embodiment, the preset delay threshold is a predefined maximum allowed value of the inference delay of the policy network, used to monitor the decision speed of the policy network. If the inference delay of the policy network is higher than this threshold, it indicates that the decision speed is slow, which may affect the real-time inference performance of the large language model.

[0093] Optionally, in the present embodiment, the preset temperature threshold is a safety upper limit value of the CPU temperature. If the CPU temperature exceeds the preset temperature threshold, it may trigger an overheating warning, or even cause automatic frequency reduction to protect the hardware, thereby affecting the running efficiency of the entire system.

[0094] Optionally, in the present embodiment, in the calculation of the expected reward parameter, the first weight controls the influence degree of the policy network inference delay feature parameter on the expected reward, while the third weight adjusts the importance of CPU power consumption in the reward calculation. When encountering a situation of excessively high inference delay or excessively high CPU temperature, by increasing the first weight or the third weight, the policy network will pay more attention to reducing the delay or controlling the power consumption, in order to ensure the stable and efficient operation of the system.

[0095] Through the embodiments provided in the present application, the policy network can quickly adjust the policy when facing performance bottlenecks or thermal management problems, and give priority to reducing inference delay or controlling CPU power consumption. For example, in a high-load environment, if the inference delay of the policy network exceeds the preset threshold, increasing the first weight will force the policy network to pay more attention to improving the decision speed to reduce the impact on the overall inference time. Similarly, if the CPU temperature is close to or exceeds the preset temperature threshold, the policy network will give priority to those CPU parameter adjustment strategies that can effectively reduce power consumption by increasing the third weight to prevent hardware overheating.

[0096] As an optional solution, using the policy network to make predictions based on performance characteristics, the predicted execution action includes:

[0097] Using the policy network to make continuous action predictions on the performance characteristics, a plurality of continuous action values are obtained;

[0098] Gaussian distribution sampling is performed on the plurality of continuous action values to obtain a prefetch aggressiveness parameter, and non-linear conversion is performed on the plurality of continuous action values to obtain a cache allocation ratio parameter;

[0099] Using the policy network to make discrete action predictions on the performance characteristics, a plurality of discrete action values are obtained;

[0100] Probability normalization conversion is performed on the plurality of discrete action values to obtain a thread scheduling strategy parameter;

[0101] The prefetch aggressiveness parameter, the cache allocation ratio parameter, and the thread scheduling strategy parameter are combined to obtain the predicted execution action.

[0102] Optionally, in the present embodiment, the output of the policy network can be divided into two types of continuous action and discrete action. Continuous action prediction involves prefetch aggressiveness and cache allocation ratio, which are continuously valued within a certain numerical range; while discrete action prediction focuses on the selection of thread scheduling strategy, which is usually one of several predefined strategies.

[0103] Optionally, in the present embodiment, Gaussian distribution sampling is used to randomly select the prefetch aggressiveness parameter from the continuous action prediction output of the policy network, to ensure the randomness and diversity of parameter adjustment. Non-linear conversion, such as Sigmoid function, is used to convert the original value output by the policy network into the effective range of the cache allocation ratio parameter, i.e. between 0.1 and 0.9.

[0104] Optionally, in the present embodiment, the discrete action prediction output of the policy network is subjected to probability normalization processing to ensure that the sum of the prediction probabilities of all possible thread scheduling strategies is equal to 1, so that the most suitable thread scheduling strategy can be selected by random sampling based on the probability distribution.

[0105] Optionally, in the present embodiment, the policy network predicts a plurality of continuous action values of the prefetch aggressiveness and the cache allocation ratio based on the current performance characteristics. These values constitute a continuous action space, providing candidate solutions for subsequent specific parameter adjustment.

[0106] For the prefetch aggressiveness, the continuous action prediction value of the policy network is sampled through a Gaussian distribution to obtain a prefetch aggressiveness parameter closer to the actual system dynamics; and for the cache allocation ratio, the prediction value of the policy network is constrained between 0.1 and 0.9 through a nonlinear transformation (such as a Sigmoid function) to generate a reasonable cache allocation ratio parameter.

[0107] The policy network also performs discrete action prediction on the performance characteristics to obtain prediction values of a plurality of thread scheduling strategies, which constitute a discrete action space of thread scheduling strategies.

[0108] After being predicted by the policy network, the plurality of discrete action values are processed through probability normalization to generate a probability distribution, from which a thread scheduling strategy parameter can be randomly selected. The normalized probability distribution ensures that each discrete action has a chance to be selected, but the probability size reflects the degree of goodness of the action as considered by the policy network.

[0109] Finally, the prefetch aggressiveness parameter obtained by Gaussian distribution sampling, the cache allocation ratio parameter after nonlinear transformation, and the thread scheduling strategy parameter after probability normalization are combined to form a complete predicted execution action. This action contains a comprehensive adjustment suggestion for CPU parameters to optimize the performance of large language model inference.

[0110] Through the above steps, the policy network can predict a series of parameter adjustment actions based on the current system state and historical execution actions. The randomness and diversity of these actions come from the Gaussian distribution sampling and probability normalization process. The prefetch aggressiveness controls the depth of data pre-reading by the system, the cache allocation ratio determines the share of GPU-related processes in the CPU cache, and the thread scheduling strategy affects the way CPU resources are allocated. Finally, these processed parameter values are combined into a predicted execution action to guide real-time adjustment of CPU parameters to optimize the performance of large language model inference.

[0111] As an optional solution, the instruction mapping of the predicted execution action to obtain at least one parameter adjustment instruction corresponding to the predicted execution action includes: obtaining a first instruction corresponding to the prefetch aggressiveness parameter, a second instruction corresponding to the cache allocation ratio parameter, and a third instruction corresponding to the thread scheduling strategy parameter through a mapping function;

[0112] In the registers of the cloud device, executing the at least one parameter adjustment instruction includes:

[0113] a first instruction is executed in a prefetch control register of the cloud device;

[0114] a second instruction is executed in a cache allocation register of the cloud device;

[0115] a third instruction is executed in a thread scheduling mode register of the cloud device.

[0116] Optionally, in this embodiment, the mapping function is a function of converting the high-level parameter adjustment suggestions output by the policy network into low-level register operation instructions. In this application, the mapping function converts the prefetch aggressiveness, cache allocation ratio and thread scheduling strategy parameters in the predicted execution action into the corresponding first instruction, second instruction and third instruction, so as to be directly executed in the registers of the CPU of the cloud device.

[0117] Optionally, in this embodiment, the prefetch control register, the cache allocation register and the thread scheduling mode register are all registers inside the CPU, which are used to control the prefetch strategy, the cache allocation strategy and the thread scheduling strategy of the CPU. The prefetch control register (such as IA32_PREFETCH_CTL in x86 architecture) is responsible for setting the depth and mode of the prefetch operation; the cache allocation register (such as the related cache management register in x86 architecture) controls the allocation ratio of the CPU cache; and the thread scheduling mode register (such as SCHED_EL1 in ARM architecture) is used to set the scheduling algorithm and priority of the thread among the CPU cores.

[0118] Optionally, in this embodiment, the policy network outputs the prefetch aggressiveness parameter, the cache allocation ratio parameter and the thread scheduling strategy parameter after predicting the execution action. In order to make these high-level parameter adjustment suggestions take effect on the physical hardware, we need to convert them into specific instructions that can directly act on the internal registers of the CPU through the mapping function, i.e. the first instruction, the second instruction and the third instruction.

[0119] The first instruction is sent to the prefetch control register for adjusting the prefetch aggressiveness. According to the prediction of the policy network, the prefetch control register will dynamically adjust the depth and mode of the prefetch operation according to the instruction of the first instruction, so as to optimize the data pre-reading strategy in the large language model inference process, improve the data access efficiency and reduce the delay.

[0120] The second instruction acts on the cache allocation register for adjusting the share of the GPU related process in the CPU cache. The adjustment of the cache allocation ratio aims to balance the allocation of the CPU cache resources, ensure that the data required by the GPU large language model inference can be quickly accessed, and at the same time control the CPU power consumption to avoid overheating.

[0121] The third instruction is sent to the thread scheduling mode register to set the optimal thread scheduling strategy. This step adjusts the thread scheduling algorithm and priority on the CPU to ensure that the cooperation between the GPU and the CPU is in the best state, improving the overall computing efficiency.

[0122] Through the embodiments provided in the present application, the model's decision is converted into actual improvement of system performance through precise register operation. Not only does it emphasize the intelligence of the policy network in generating parameter adjustment schemes, but it also embodies the efficiency and accuracy of parameter adjustment instruction execution, ensuring closed-loop control of the CPU parameter optimization process and providing a stable, efficient, and energy-saving operating environment for GPU large language model inference.

[0123] As an optional solution, obtaining the performance characteristics of the large language model includes:

[0124] Obtaining the GPU memory bandwidth utilization of the cloud device, the CPU cache hit rate of the cloud device, the inter-process communication average delay of the cloud device, the CPU power consumption of the cloud device, and the CPU temperature of the cloud device.

[0125] The five-dimensional vector composed of the GPU memory bandwidth utilization, the CPU cache hit rate, the inter-process communication average delay, the CPU power consumption, and the CPU temperature is determined as the performance characteristics.

[0126] Optionally, in the present embodiment, the GPU memory bandwidth utilization is used to measure the ratio of GPU memory read-write speed to maximum available bandwidth, directly reflecting the memory access efficiency of the GPU. The CPU cache hit rate is used to indicate the frequency of directly obtaining data from the cache when the CPU accesses data. The higher the cache hit rate, the lower the overall system delay.

[0127] Optionally, in the present embodiment, the inter-process communication average delay is used to evaluate the average value of the time required for data exchange between different processes during the inference process of the cloud device. Communication delay directly affects the overall computing efficiency. CPU power consumption is used to indicate the power consumed by the CPU during operation. The control of power consumption is crucial for high-performance computing and energy management of mobile devices. CPU temperature is the temperature of the CPU during operation. Excessive CPU temperature may trigger the system's automatic cooling mechanism, reducing CPU frequency and affecting performance.

[0128] Optionally, in the present embodiment, the following five types of key performance data are collected from the real-time running cloud device: GPU memory bandwidth utilization, CPU cache hit rate, inter-process communication average delay, CPU power consumption, and CPU temperature. These data provide a comprehensive view of the current running state of the system and are an important basis for subsequent optimization decisions.

[0129] Then, the five types of data collected above are integrated into a five-dimensional vector, i.e., performance characteristics. This vector contains key parameters that affect the inference performance of the cloud device under the cooperative working state of GPU and CPU, and can provide accurate state representation for model training and strategy generation.

[0130] It can be understood that the GPU memory bandwidth utilization is directly related to the computing efficiency of the GPU, the CPU cache hit rate affects the response speed and overall delay of the CPU, and the inter-process communication average delay determines the efficiency of data exchange between different computing units. In addition, monitoring of CPU power consumption and CPU temperature ensures that the system can maintain reasonable energy consumption and stable operating temperature while pursuing high performance, avoiding performance loss caused by overheating.

[0131] Through the embodiments provided in the present application, accurate state information is provided for subsequent timing prediction, strategy generation and parameter adjustment. By constructing a five-dimensional vector including GPU memory bandwidth utilization, CPU cache hit rate, inter-process communication average delay, CPU power consumption and CPU temperature, the system can comprehensively evaluate the current running status, and make the next decision and optimization based on the state, ensuring the real-time and effectiveness of the CPU parameter adjustment strategy, and improving the efficiency and stability of the large language model inference.

[0132] As an optional solution, the parameter adjustment method of the cloud device is used in the GPU large language model inference CPU parameter optimization scenario based on model reinforcement learning. In this scenario, with the wide application of large language models in natural language processing, recommendation systems and other fields, the optimization of inference performance has become a key challenge. GPU, as the main computing unit of large language model inference, its performance is often limited by the static nature of CPU parameter configuration, such as the aggressiveness of prefetch strategy, cache allocation ratio and thread scheduling strategy, etc. Traditional methods usually use heuristic rules or offline analysis tools for parameter tuning, but such methods cannot adapt to dynamic changes in load characteristics and model architecture differences.

[0133] To solve the above problems, the present embodiment forms a closed loop optimization around "state collection-timing prediction-strategy generation-instruction execution-reward adjustment-experience learning", and proposes a schematic diagram of GPU large language model inference CPU parameter optimization based on model reinforcement learning, as shown in Figure 3 Specifically, first, through step 1 (state collection), the state vector s t containing GPU memory bandwidth utilization, CPU cache hit rate, etc. is collected in real time, and the result is used as the input of the prediction network in step 2 (timing prediction, such as LSTM timing prediction), combined with the previous action a t-1 to output the next state s t+1 and the reward value r t; the prediction result of step 2 is input into the PPO policy network of step 3 to generate a mixed action a containing pre-fetch aggressiveness, cache allocation ratio and thread scheduling strategy t ; the action generated by step 3 (policy generation, such as PPO policy) is mapped to CPU register operation instruction through a lightweight intermediate layer in step 4 (instruction mapping execution) and executed; after execution, the real-time state s t is adjusted according to the real-time state s t , the reward function weight is dynamically adjusted, and the composite reward value is calculated; finally, in step 6 (experience replay learning), the experience tuple of state, action and reward is stored in the buffer, and the policy network is updated using priority experience replay and gradient descent, forming a complete feedback link from data collection to policy optimization, realizing real-time adaptive adjustment of CPU parameters.

[0134] Specifically, in this embodiment, a state space definition and real-time collection schematic diagram is as shown in Figure 4 The hardware sensor (such as GPU power sensor, CPU temperature sensor) collects raw data in real time, which is summarized through the data collection module, and then the abnormal values are removed through the sliding average filter, and finally a state vector s t containing 5-dimensional indicators is generated.

[0135] The system state vector is as shown in formula (1):

[0136] (1)

[0137] Wherein, , U GPU (t) is the GPU memory bandwidth utilization, B used (t) is the real-time bandwidth occupation, B total (t) is the total bandwidth; , H cache (t) is the CPU cache hit rate, C hit (t) and C miss (t) are the hit / miss times, respectively; , L IPC (t) is the average inter-process communication delay, T comm.i (t) is the i-th communication delay, and n is the sampling number; P CPU (t) is the CPU power consumption (W) directly collected through the power sensor; T CPU (t) is the CPU core temperature, which is obtained through the on-chip temperature sensor.

[0138] In this embodiment, in the LSTM time series prediction network modeling step, the current state s t is input into the LSTM time series prediction network to generate the prediction result of the next state s t-1The input vector is composed, and the time sequence features are extracted through two layers of LSTM network in turn, and finally the next state prediction value s is output by the full connection layer t+1 and the reward value r t .

[0139] The LSTM unit structure is shown in formula (2), which includes input gate i t , forget gate f t , output gate o t and cell state c t .

[0140] (2)

[0141] where, is a Sigmoid activation function, and tanh is a hyperbolic tangent activation function; W is , W fs , W os , W cs are weight matrices, and b is , b fs , b os , b cs are bias vectors; h t is a hidden state, which is used to predict the next state s t+1 and the reward r t : .

[0142] In this embodiment, a schematic diagram of a PPO policy optimization technology of a hybrid action space is shown in Figure 5 , the policy network receives the current state vector s t , generates continuous actions (pre-fetch aggressiveness, cache allocation ratio) through continuous action sampling and generates discrete actions (thread scheduling strategy) through discrete action sampling, wherein the continuous action sampling is processed through Gaussian sampling and action clipping, and the discrete action sampling is processed through normalization (Softmax normalization), and finally the hybrid action a t is composed.

[0143] Optionally, the policy network outputs the hybrid action (continuous value + discrete value):

[0144] Pre-fetch aggressiveness P prefetch : sampled through Gaussian distribution, .

[0145] Cache allocation ratio C ratio : normalized to [0.1, 0.9] through Sigmoid, .

[0146] Thread scheduling strategy T policy : discrete probability distribution, .

[0147] Optionally, advantage function calculation (GAE) is performed as shown in equation (3):

[0148] (3)

[0149] where, is the reward discount factor; is the GAE parameter; is the state value function output by the shared layer of the policy network.

[0150] Optionally, policy gradient update is performed as shown in equation (4):

[0151] (4)

[0152] where, is the policy gradient target, which is the gradient of the policy network parameters for updating the policy to maximize the expected cumulative reward ; is the expected operator, which is averaged over all possible state-action pairs , usually approximated by sampled experience trajectories; is the action probability logarithm, which is the logarithm of the probability of the current policy selecting action a in state s; is the probability ratio, which is the ratio of the probability of the new policy to the probability of the old policy for the same action a, if the ratio > 1, it means that the new policy is more inclined to select a, if the ratio < 1, it means that the new policy is more inclined to avoid a; is the clipping function, which limits the probability ratio to the interval to prevent the policy update from being too large; is the PPO clipping parameter, which ensures the stability of the policy update.

[0153] In this embodiment, a lightweight intermediate layer instruction mapping diagram is shown in Figure 6 , in which the mixed action (high-level action) a t is converted into a CPU register operation instruction through an instruction mapping layer, and a parameter adjustment instruction (such as an MSR instruction of x86 or an MCR instruction of ARM) is obtained through cross-platform compilation, and finally written into a CPU register for execution.

[0154] The space design of the action is shown in equation (5), which adopts a mixed discrete-continuous action space:

[0155] (5)

[0156] where, Pprefetch is the aggressiveness of prefetching (0-1 continuous value, 0 is conservative prefetching, and 1 is aggressive prefetching); C ratio is the L3 cache allocation ratio (0.1-0.9 continuous value); T policy is the thread scheduling strategy (discrete value: 0=polling, 1=priority).

[0157] The mapping function of the high-level action to the low-level register operation is shown in formula (6):

[0158] (6)

[0159] wherein, The mapping logic of is quantized by 8-bit precision, and is mapped to the CPU prefetch control register (such as IA32_PREFETCH_CTL of x86).

[0160] The level prefetch step adjustment; is the floor function; The mapping logic of is converted into an integer from 1 to 9, corresponding to the L3 cache allocation ratio of 10%-90% (such as CachePart=5 representing 50% cache allocation to the GPU related process), and is written into the cache partition register; is the ceiling function; The mapping logic of is directly to control the mode bit of the CPU thread scheduling register (such as SCHED_EL1 of ARM), 0x00 triggers the polling algorithm, and 0x01 enables the priority preemption.

[0161] In the embodiment, a multi-target reward function dynamic adjustment schematic diagram is shown in Figure 7 , the system state s t is monitored in real time, whether the delay and temperature are out of limits is detected, if the delay exceeds the threshold, the delay weight w1 is increased, if the temperature exceeds 80℃, the power consumption weight w3 is increased, and finally the reward function weight is updated.

[0162] The reward function is designed, and the composite reward function is shown in formula (7):

[0163] (7)

[0164] wherein, L t is the current inference delay; U t is the GPU utilization; P t is the CPU power consumption; w1, w2, and w3 are adjustable weights.

[0165] The composite reward function weight adaptive mechanism is shown in formula (8):

[0166] (8)

[0167] wherein, : base weight; : increase delay weight when delay exceeds threshold (delay threshold, can be adjusted according to QoS requirement of real-time service) : increase power consumption weight when temperature exceeds threshold (temperature threshold, avoid overheat and frequency reduction, can be adjusted according to project requirement) : is an indicator function, 1 if condition is met, otherwise 0.

[0168] In this embodiment, an experience replay and online learning schematic diagram is shown in FIG. 1, and an experience tuple (state-action-reward-next state tuple) is stored in an experience buffer after each decision. A priority experience replay mechanism is used to sample data according to an advantage function absolute value. A gradient is calculated, and a policy network is updated. A target network parameter is synchronized through a soft update mechanism. Figure 8

[0169] The experience tuple is stored as , and a priority experience replay (PER) is used, as shown in equation (9):

[0170] (9)

[0171] wherein, is a priority experience replay weight; is an experience priority, based on an advantage function absolute value; is a PPO clipping parameter, to avoid zero probability.

[0172] In this embodiment, an online updated policy is shown in equation (10):

[0173] (10)

[0174] wherein, is a learning rate.

[0175] A batch of data is sampled from the buffer every 100 ms, and a policy network is updated through gradient descent.

[0176] ​​​It should be noted that the CPU parameter adjustment in the embodiment is modeled as a partially observable Markov decision process (POMDP) problem in a hybrid discrete-continuous action space, and the joint optimization of prefetching strategy, cache allocation and thread scheduling is realized, and the reward function weight can be dynamically adjusted according to the system state. The LSTM prediction network supports partial parameter fine-tuning and can quickly migrate between different LLM architectures. Fast parameters (such as prefetching) are adjusted at a period of 10ms, and slow parameters (such as cache allocation) are adjusted at a period of 100ms. A multi-objective weighted reward mechanism is used to dynamically balance the optimization objectives of inference delay, GPU utilization and CPU power consumption, wherein the reward function weight can be automatically adjusted according to the system thermal threshold and power consumption limit. It can model long-term temporal dependencies, predict the impact of different CPU parameter configurations on GPU inference performance, and use the prediction results as the state input of reinforcement learning. It supports simultaneous processing of discrete parameter adjustment and continuous parameter optimization, including but not limited to: continuous value adjustment of prefetch aggressiveness; dynamic division of L3 cache allocation ratio; discrete selection of thread scheduling strategy.

[0177] It should also be noted that the embodiment can be applied to, but is not limited to, edge computing scenarios, adapt to resource-constrained devices, and support real-time inference for the Internet of Things. It can also be applied to, but is not limited to, multi-tenant cloud service scenarios, maintain LSTM hidden states for each tenant independently, and realize CPU parameter isolation in combination with the virtualization layer to avoid resource preemption. It can also be applied to, but is not limited to, autonomous driving scenarios, dynamically optimize CPU configurations for sensor data preprocessing for a vehicle heterogeneous computing platform (such as CPU+GPU+TPU), and reduce autonomous driving delay.

[0178] Through the embodiments provided in the present application, the limitations of traditional static parameter configuration are broken through, and multiple interrelated CPU parameters including prefetching strategy, cache allocation ratio and thread scheduling strategy can be dynamically adjusted according to real-time system state, and global optimal system performance is realized. Through the carefully designed composite reward function, the system can achieve the best balance between reducing inference delay, improving GPU utilization and controlling CPU power consumption and other multiple mutually restrictive objectives. The system has the ability of continuous learning and can maintain optimal performance without human intervention.

[0179] Through the description of the above embodiments, those skilled in the art can clearly understand that the method according to the above embodiments can be realized by means of software on a general hardware platform as necessary, and of course can also be realized by hardware, but in many cases the former is a better embodiment. Based on such understanding, the technical solutions of the present application can be embodied in the form of a software product in essence or the part that contributes to the prior art, and the computer software product is stored in a storage medium (such as a ROM / RAM, a magnetic disk, or an optical disk), and includes a plurality of instructions for causing a terminal device (which can be a mobile phone, a computer, a server, or a network device) to execute the method of each embodiment of the present application.

[0180] In this embodiment, a parameter adjustment apparatus of a cloud device is also provided, which is used to implement the above embodiments and preferred embodiments, and will not be described again. As used below, the term "module" can be a combination of software and / or hardware that implements a predetermined function. Although the apparatus described in the following embodiments is preferably implemented in software, hardware or a combination of software and hardware is also possible and contemplated.

[0181] Figure 9 is a structural block diagram of the parameter adjustment apparatus of the cloud device according to the embodiments of the present application, as shown in Figure 9 , the apparatus includes:

[0182] The acquisition unit 902 is configured to acquire performance characteristics of a cloud device where a large language model is located, wherein the performance characteristics are used to indicate running characteristics of the cloud device in a working state;

[0183] The prediction unit 904 is configured to use a policy network to perform prediction based on the performance characteristics to obtain a predicted execution action;

[0184] The update unit 906 is configured to perform policy update on the policy network based on the performance characteristics and historical execution actions corresponding to the performance characteristics, wherein the historical execution actions include adjustment actions that have been performed on the performance characteristics of the cloud device;

[0185] The mapping unit 908 is configured to perform instruction mapping on the predicted execution action to obtain at least one parameter adjustment instruction corresponding to the predicted execution action;

[0186] The adjustment unit 910 is configured to execute the at least one parameter adjustment instruction in a register of the cloud device.

[0187] As an optional solution, the update unit 906 includes:

[0188] A first prediction module is configured to perform time series prediction on the performance characteristics and the historical execution actions to obtain predicted performance characteristics of the cloud device and an expected reward parameter;

[0189] The computing module is configured to perform advantage difference calculation on the predicted performance feature, the expected reward parameter and the performance feature to obtain a target advantage parameter of the policy network.

[0190] The updating module is configured to update a policy gradient of the policy network using the target advantage parameter.

[0191] As an optional solution, the computing module comprises:

[0192] The first computing submodule is configured to perform state value calculation on the predicted performance feature and the performance feature to obtain a first state value parameter corresponding to the predicted performance feature and a second state value parameter corresponding to the performance feature.

[0193] The second computing submodule is configured to obtain the target advantage parameter by multiplying the expected reward parameter by the first state value parameter and then subtracting the second state value parameter.

[0194] As an optional solution, the updating module comprises:

[0195] The first obtaining submodule is configured to obtain a first execution probability corresponding to the predicted execution action and a second execution probability corresponding to the historical execution action, wherein the first execution probability is used to indicate an execution probability of the predicted execution action in the performance feature, and the second execution probability is used to indicate an execution probability of the historical execution action before the historical execution action is executed.

[0196] The first determining submodule is configured to determine a logarithmic value corresponding to the first execution probability as a first gradient parameter corresponding to the policy network.

[0197] The second determining submodule is configured to determine a clipping parameter value corresponding to a ratio of the first execution probability and the second execution probability as a second gradient parameter corresponding to the policy network.

[0198] The third determining submodule is configured to update the policy gradient of the policy network using a product result of the first gradient parameter, the second gradient parameter and the target advantage parameter.

[0199] As an optional solution, the prediction module comprises:

[0200] The second obtaining submodule is configured to obtain an inference delay feature parameter of the policy network and obtain a GPU utilization and a CPU power consumption of the cloud device from the performance feature, wherein the inference delay feature parameter is used to indicate a difference between a ratio of an inference delay of the policy network to a maximum threshold of the inference delay and 1.

[0201] The third computing submodule is configured to obtain the expected reward parameter by multiplying a value of the inference delay feature parameter by the first weight, adding a value of the GPU utilization by the second weight, and then subtracting a value of the CPU power consumption by the third weight.

[0202] As an optional solution, the apparatus further comprises:

[0203] a first increasing module, configured to increase the first weight when it is detected that the inference delay of the policy network is greater than a preset delay threshold;

[0204] a second increasing module, configured to increase the third weight when it is detected that the CPU temperature of the cloud device is greater than a preset temperature threshold.

[0205] As an optional solution, the prediction unit 904 comprises:

[0206] a second prediction module, configured to perform continuous action prediction on the performance features using the policy network to obtain a plurality of continuous action values;

[0207] a sampling module, configured to perform Gaussian distribution sampling on the plurality of continuous action values to obtain a prefetch aggressiveness parameter, and perform nonlinear conversion on the plurality of continuous action values to obtain a cache allocation ratio parameter;

[0208] a third prediction module, configured to perform discrete action prediction on the performance features using the policy network to obtain a plurality of discrete action values;

[0209] a conversion module, configured to perform probability normalization conversion on the plurality of discrete action values to obtain a thread scheduling strategy parameter;

[0210] a combination module, configured to combine the prefetch aggressiveness parameter, the cache allocation ratio parameter, and the thread scheduling strategy parameter to obtain a predicted execution action.

[0211] As an optional solution, the mapping unit 908 comprises:

[0212] a first obtaining module, configured to obtain, through a mapping function, a first instruction corresponding to the prefetch aggressiveness parameter, a second instruction corresponding to the cache allocation ratio parameter, and a third instruction corresponding to the thread scheduling strategy parameter;

[0213] The adjustment unit 910 comprises:

[0214] a first adjustment module, configured to execute the first instruction in a prefetch control register of the cloud device;

[0215] a second adjustment module, configured to execute the second instruction in a cache allocation register of the cloud device;

[0216] a third adjustment module, configured to execute the third instruction in a thread scheduling mode register of the cloud device.

[0217] As an optional solution, the obtaining unit 902 comprises:

[0218] The second obtaining module is configured to obtain GPU memory bandwidth utilization of the cloud device, CPU cache hit rate of the cloud device, inter-process communication average delay of the cloud device, CPU power consumption of the cloud device, and CPU temperature of the cloud device.

[0219] The determining module is configured to determine a five-dimensional vector composed of the GPU memory bandwidth utilization, the CPU cache hit rate, the inter-process communication average delay, the CPU power consumption, and the CPU temperature as the performance feature.

[0220] The specific examples in the embodiments can refer to the examples described in the above embodiments and exemplary embodiments, and the embodiments will not be described here again.

[0221] Through the above description of the embodiments, those skilled in the art can clearly understand that the method according to the above embodiments can be realized by means of software and a necessary general hardware platform, and of course, it can also be realized by hardware, but in many cases, the former is a better embodiment. Based on such understanding, the technical solutions of the present application can be embodied in the form of a software product, which is stored in a storage medium (such as a ROM / RAM, a magnetic disk, or an optical disk) and includes a plurality of instructions for causing a terminal device (which can be a mobile phone, a computer, a server, or a network device) to execute the methods of the various embodiments of the present application.

[0222] It should be noted that the above various modules can be realized by software or hardware, and for the latter, the following implementation manners can be used, but are not limited thereto: the above modules are located in the same processor; or the above various modules are located in different processors in any combination.

[0223] The embodiments of the present application also provide a computer readable storage medium, which stores a computer program, and the computer program is configured to execute the steps in any one of the method embodiments when running.

[0224] In an exemplary embodiment, the above computer readable storage medium can include, but is not limited to, a U disk, a read-only memory (ROM), a random access memory (RAM), a mobile hard disk, a magnetic disk or an optical disk, and various media that can store computer programs.

[0225] The embodiments of the present application also provide an electronic device, which includes a memory and a processor, the memory stores a computer program, and the processor is configured to execute the computer program to execute the steps in any one of the method embodiments.

[0226] In one example embodiment, the electronic device described above can further include a transmission device connected to the processor and an input / output device connected to the processor.

[0227] Embodiments of the present application also provide a computer program product, comprising a non-volatile computer readable storage medium, the non-volatile computer readable storage medium storing a computer program product, the computer program being executed by a processor to implement the steps of the method in the various embodiments of the present application.

[0228] The specific examples in the embodiments can refer to the examples described in the above embodiments and exemplary embodiments, which will not be repeated here.

[0229] Obviously, those skilled in the art should understand that the modules or steps of the present application described above can be realized by general computing devices, which can be concentrated on a single computing device or distributed on a network composed of multiple computing devices, which can be realized by program codes executable by computing devices, so that they can be stored in storage devices and executed by computing devices, and in some cases, the steps shown or described can be executed in different order, or they can be manufactured into individual integrated circuit modules, or multiple modules or steps can be manufactured into a single integrated circuit module. Thus, the present application is not limited to any specific combination of hardware and software.

[0230] The above is only the preferred embodiment of the present application and is not intended to limit the present application. Those skilled in the art can make various modifications and changes to the present application. Any modification, equivalent replacement, improvement, etc. within the principles of the present application shall be included in the protection scope of the present application.

Claims

1. A parameter adjustment method of a cloud device, characterized by, The method comprises: obtaining performance characteristics of a cloud device where a large language model is located, wherein the performance characteristics are used to indicate running characteristics of the cloud device in a working state; using a policy network to perform continuous action prediction on the performance characteristics to obtain a plurality of continuous action values; performing Gaussian distribution sampling on the plurality of continuous action values to obtain a pre-fetch aggressiveness parameter, and performing non-linear conversion on the plurality of continuous action values to obtain a cache allocation ratio parameter; using the policy network to perform discrete action prediction on the performance characteristics to obtain a plurality of discrete action values; performing probability normalization conversion on the plurality of discrete action values to obtain a thread scheduling strategy parameter; combining the pre-fetch aggressiveness parameter, the cache allocation ratio parameter, and the thread scheduling strategy parameter to obtain a predicted execution action; performing policy update on the policy network based on the performance characteristics and historical execution actions corresponding to the performance characteristics, wherein the historical execution actions include adjustment actions that have been performed on the performance characteristics of the cloud device; performing instruction mapping on the predicted execution action to obtain at least one parameter adjustment instruction corresponding to the predicted execution action; executing the at least one parameter adjustment instruction in a register of the cloud device.

2. The method of claim 1, wherein, The policy update on the policy network based on the performance characteristics and the historical execution actions corresponding to the performance characteristics comprises: performing time series prediction on the performance characteristics and the historical execution actions to obtain predicted performance characteristics of the cloud device and an expected reward parameter; performing advantage difference calculation on the predicted performance characteristics, the expected reward parameter, and the performance characteristics to obtain a target advantage parameter of the policy network; updating a policy gradient of the policy network using the target advantage parameter.

3. The method of claim 2, wherein, The advantage difference calculation on the predicted performance characteristics, the expected reward parameter, and the performance characteristics to obtain the target advantage parameter of the policy network comprises: performing state value calculation on the predicted performance characteristics and the performance characteristics to obtain a first state value parameter corresponding to the predicted performance characteristics and a second state value parameter corresponding to the performance characteristics; multiplying the first state value parameter by the expected reward parameter, and then subtracting the second state value parameter to obtain the target advantage parameter.

4. The method of claim 2, wherein, The updating of the policy gradient of the policy network using the target advantage parameter comprises: obtaining a first execution probability corresponding to the predicted execution action and a second execution probability corresponding to the historical execution action, wherein the first execution probability is used to indicate an execution probability of the predicted execution action under the performance characteristics, and the second execution probability is used to indicate an execution probability of the historical execution action before the historical execution action is executed; determining a logarithmic value corresponding to the first execution probability as a first gradient parameter corresponding to the policy network; determining a clipping parameter value corresponding to a ratio of the first execution probability and the second execution probability as a second gradient parameter corresponding to the policy network; The policy gradient of the policy network is updated using a product result of the first gradient parameter, the second gradient parameter, and the target advantage parameter.

5. The method of claim 2, wherein, The performance feature and the historical execution action are used to perform time series prediction to obtain a predicted performance feature and an expected reward parameter of the cloud device. An inference delay feature parameter of the policy network is obtained, and GPU utilization and CPU power consumption of the cloud device are obtained from the performance feature, where the inference delay feature parameter is used to indicate a difference between a ratio of inference delay of the policy network to a maximum threshold of inference delay and 1. The expected reward parameter is obtained by multiplying a first weight by a value of the inference delay feature parameter, adding a second weight multiplied by a value of the GPU utilization, and subtracting a third weight multiplied by a value of the CPU power consumption.

6. The method of claim 5, wherein, The method further includes: In a case where it is detected that the inference delay of the policy network is greater than a preset delay threshold, the first weight is increased. In a case where it is detected that the CPU temperature of the cloud device is greater than a preset temperature threshold, the third weight is increased.

7. The method of claim 1, wherein The instruction mapping of the predicted execution action to obtain at least one parameter adjustment instruction corresponding to the predicted execution action includes: A first instruction corresponding to the prefetch aggressiveness parameter, a second instruction corresponding to the cache allocation ratio parameter, and a third instruction corresponding to the thread scheduling strategy parameter are obtained through a mapping function; The execution of the at least one parameter adjustment instruction in the register of the cloud device includes: The first instruction is executed in a prefetch control register of the cloud device; The second instruction is executed in a cache allocation register of the cloud device; The third instruction is executed in a thread scheduling mode register of the cloud device.

8. The method according to any one of claims 1 to 7, characterized in that, The performance feature of the cloud device on which the large language model is located includes: The GPU memory bandwidth utilization of the cloud device, the CPU cache hit rate of the cloud device, the inter-process communication average delay of the cloud device, the CPU power consumption of the cloud device, and the CPU temperature of the cloud device are obtained. A five-dimensional vector composed of the GPU memory bandwidth utilization, the CPU cache hit rate, the inter-process communication average delay, the CPU power consumption, and the CPU temperature is determined as the performance feature.

9. An electronic device comprising a memory, a processor, and a computer program stored on the memory and executable on the processor, characterized in that, The processor executes the computer program to implement the steps of the method of any one of claims 1 to 8. The processor executes the computer program to implement the steps of the method of any one of claims 1 to 8.

Images