Server fan adaptive speed regulation method and system based on reinforcement learning and model predictive control

Abstract

This application discloses a server fan adaptive speed adjustment method and system based on reinforcement learning and model predictive control, relating to the field of server heat dissipation control technology. The disclosed server fan adaptive speed adjustment method and system based on reinforcement learning and model predictive control acquires real-time operating data, calculates the optimal fan speed sequence, generates control commands, generates adjustment strategies based on model prediction error input reinforcement learning agent, securely updates the model and executes iteratively, thereby realizing adaptive adjustment of fan speed and dynamic model correction. It can effectively integrate advanced prediction mechanism and dynamic optimization capability, minimizing energy consumption while ensuring temperature safety boundaries, and avoiding control lag and temperature spikes.

Description

Technical Field

[0001] This application relates to the field of server heat dissipation control technology, and in particular to a server fan adaptive speed adjustment method and system based on reinforcement learning and model predictive control. Background Technology

[0002] During data center operation, the energy consumed by the cooling system accounts for a significant proportion of the total energy consumption. Forced ventilation by fans, as the core cooling mechanism, directly impacts operating costs and equipment stability. Traditional control strategies rely on fixed threshold triggers or classic PID algorithms, passively increasing fan speed only after a temperature sensor detects an overheating, failing to predict subsequent heat load trends. This response mechanism leads to continuous oscillations in temperature and fan speed, exacerbating equipment noise and mechanical fatigue, and forcing the system to maintain a high baseline speed to prevent extreme overheating, resulting in significant energy waste. As server computing loads exhibit high-frequency dynamic changes, the heat exchange coupling effect between multiple fans and heat-generating components becomes increasingly complex. Traditional methods struggle to comprehensively manage the overall situation and minimize total power consumption while ensuring temperature safety boundaries. Existing improvement solutions attempt to introduce linear models for instantaneous optimization, such as calculating energy-saving speed setpoints based on current temperature data. However, these methods remain limited to static open-loop operation, lacking the ability to proactively predict multi-cycle temperature trajectories and cannot actively adjust fan speeds to avoid potential overheating risks. When the server workload changes abruptly, temperature spikes still occur frequently because the load information is not integrated to implement feedforward compensation, and the fundamental contradiction between control lag and energy consumption optimization goals cannot be effectively alleviated.

[0003] The above content is only used to help understand the technical solution of this application and does not represent an admission that the above content is prior art. Summary of the Invention

[0004] The main purpose of this application is to provide a server fan adaptive speed control method and system based on reinforcement learning and model predictive control, which aims to avoid control lag and temperature spikes.

[0005] To achieve the above objectives, this application proposes a server fan adaptive speed control method based on reinforcement learning and model predictive control, the method comprising: Acquire real-time operating data of the server; the real-time operating data includes temperature monitoring data of multiple heat-generating components, current speed data of each fan, and power consumption data reflecting the server's workload; Based on the real-time operating data and the preset model prediction controller, the optimal fan speed sequence in the prediction time domain is calculated; wherein the preset model prediction controller has a built-in thermal dynamic model for describing the dynamic relationship between fan speed and component temperature. Based on the first speed value in the optimal fan speed sequence, a first control command is generated and sent to the fan drive module to adjust the speed of each fan; Based on the temperature monitoring data at the current moment and the temperature data predicted by the thermal dynamic model in the prediction time domain, the model prediction error index is calculated. The model prediction error index, the real-time operating data, and the fan speed data corresponding to the first control command are input together into the pre-trained reinforcement learning agent. The pre-trained reinforcement learning agent analyzes and makes decisions based on the input data to generate a model parameter adjustment strategy. According to the model parameter adjustment strategy, the parameters of the thermal dynamic model are updated safely to correct the thermal dynamic model; The updated thermal dynamic model will be used as the thermal dynamic model for calculating the optimal fan speed sequence in the next control cycle, and the step of acquiring real-time operating data will be re-executed.

[0006] In one embodiment, the step of calculating the optimal fan speed sequence in the prediction time domain based on the real-time operating data and a preset model prediction controller includes: The temperature monitoring data, current rotation speed data, and power consumption data from the real-time operating data are input into the state observer for fusion processing to estimate the server's current complete thermal state data. Based on the thermal dynamics model and the thermal state data, predict the future temperature change trajectory data of each heating component in the prediction time domain; A multi-objective optimization problem is constructed; the objective functions of the multi-objective optimization problem include minimizing the deviation between the future temperature change trajectory data and the preset safe temperature range, and minimizing the energy consumption of the fan; Under the condition that the temperature monitoring data does not exceed the safety limit, the fan speed does not exceed the physical limit, and the fan speed change rate does not exceed the smoothness limit, the multi-objective optimization problem is solved to obtain the optimal fan speed sequence.

[0007] In one embodiment, constructing a multi-objective optimization problem includes: By analyzing the degree to which the future temperature change trajectory data is close to the preset safe temperature range, the temperature safety margin assessment result is obtained; When the temperature safety margin assessment result indicates that the safety margin is lower than the preset safety threshold, the penalty weight for the deviation between the future temperature change trajectory data and the preset safe temperature range is increased in the objective function, while the optimization weight for the energy consumption of the fan is reduced. When the temperature safety margin assessment result indicates that the safety margin is higher than the preset safety threshold, the penalty weight for the deviation between the future temperature change trajectory data and the preset safe temperature range is reduced in the objective function, while the optimization weight for the energy consumption of the fan is increased. The multi-objective optimization problem is reconstructed based on the dynamically adjusted weights.

[0008] In one embodiment, the steps of inputting the model prediction error index, the real-time operating data, and the fan speed data corresponding to the first control command into a pre-trained reinforcement learning agent, and having the pre-trained reinforcement learning agent analyze and make decisions based on the input data to generate a model parameter adjustment strategy, include: The model prediction error index, the real-time operating data, and the fan speed data corresponding to the first control command are combined to form the current state data of the pre-trained reinforcement learning agent. The pre-trained reinforcement learning agent selects actions from a preset action space based on the current state data; the actions are the model parameter adjustment strategies, used to represent the magnitude and direction of adjustment to multiple parameter items of the thermal dynamic model; After the pre-trained reinforcement learning agent performs the action, the thermal dynamic model is updated through a subsequent safety update step, and the control effect after the model update is evaluated based on the recalculated model prediction error index in the subsequent control cycle, and the reward signal is calculated. The reward signal is fed back to the pre-trained reinforcement learning agent to update its decision network, thereby optimizing its ability to generate future model parameter adjustment strategies.

[0009] In one embodiment, the step of safely updating the parameters of the thermal dynamic model according to the model parameter adjustment strategy to correct the thermal dynamic model includes: Based on the adjustment range indicated by the model parameter adjustment strategy, the severity of the current model mismatch is determined, and the corresponding safe update mode is selected accordingly. Based on the selected security update mode and the model parameter adjustment strategy, parameter updates are performed on the learning and validation model that runs in parallel with the preset model prediction controller to obtain the candidate thermal dynamic model after parameter updates. The candidate thermal dynamics model is tested and evaluated using a validation dataset to generate model validation results; When the model verification results meet the preset security, accuracy and stability requirements, the thermal dynamic model in the preset model prediction controller is replaced with the candidate thermal dynamic model to complete the security update.

[0010] In one embodiment, the step of determining the severity of the current model mismatch based on the adjustment magnitude indicated by the model parameter adjustment strategy, and selecting the corresponding safe update mode accordingly, includes: If the adjustment range is less than the first threshold, it is determined to be a slight mismatch. The parameter fine-tuning mode is selected, and the first adjustment of the parameters is performed on the learning and verification model using the gradient descent method based on historical running data. If the adjustment magnitude is greater than or equal to the first threshold and less than the second threshold, it is determined to be a mismatch. The model reconstruction mode is selected, and the recently collected running data is used on the learning and verification model to adjust the second magnitude of the parameters through the recursive least squares method. If the adjustment range is greater than or equal to the second threshold, it is determined to be a severe mismatch. The model relearning mode is selected, the online update of the learning and verification model is suspended, and a new thermal dynamic model is retrained using the stored historical data set and the system identification algorithm as the candidate thermal dynamic model.

[0011] In one embodiment, the step of testing and evaluating the candidate thermal dynamics model using a validation dataset to generate model validation results includes: Construct simulated input data for multiple verification scenarios, wherein the simulated input data includes simulated fan speed sequences and simulated workload sequences; The simulated input data is input into the candidate thermal dynamics model to obtain the simulated temperature prediction results; The simulated temperature prediction results are evaluated in multiple dimensions, including: prediction accuracy evaluation, calculating the deviation between the simulated prediction results and the preset reference value; constraint satisfaction evaluation, checking whether the temperature prediction exceeds the safety threshold in all simulated scenarios; and control stability evaluation, analyzing whether the control commands derived from the candidate thermal dynamic model have fluctuations exceeding the preset threshold. Based on the results of the multi-dimensional evaluation, the model validation result containing the scores of each evaluation item is generated; wherein, only when the weighted total score of all evaluation items in the model validation result exceeds the preset qualification threshold is it determined to meet the preset security, accuracy and stability requirements.

[0012] In one embodiment, the step of replacing the thermal dynamic model in the preset model prediction controller with the candidate thermal dynamic model includes: Starting from the current control cycle, the thermal dynamic model will not be directly replaced in subsequent consecutive control cycles; In each subsequent control cycle, the first optimal fan speed sequence and the second optimal fan speed sequence are calculated based on the current thermal dynamic model and the candidate thermal dynamic model, respectively. Based on a fusion coefficient that increases linearly from 0 to 1, the first optimal fan speed sequence and the second optimal fan speed sequence are weighted and fused to generate the fused optimal fan speed sequence for actual use. When the fusion coefficient reaches 1, the model switch is completed, and thereafter the candidate thermal dynamic model is used entirely for calculation.

[0013] In one embodiment, prior to the step of acquiring the server's real-time operational data, the method further includes: Based on historical operating data, multiple performance indicators of the thermal dynamics model are calculated, including at least: long-term prediction accuracy, short-term prediction stability, and parameter confidence. The comprehensive health score of the thermal dynamic model is calculated by weighting and fusing multiple performance indicators. The exploration strategy of the pre-trained reinforcement learning agent is dynamically adjusted according to the different health level ranges in which the comprehensive health score is located. Specifically, when the comprehensive health score is in the low health level range, a conservative exploration strategy is adopted to limit the adjustment range of the model parameter adjustment strategy. When the comprehensive health score is in the high health level range, an aggressive search strategy is adopted, allowing the model parameter adjustment strategy to attempt adjustments beyond the preset range.

[0014] Furthermore, to achieve the above objectives, this application also proposes a server fan adaptive speed control system based on reinforcement learning and model predictive control. The server fan adaptive speed control system based on reinforcement learning and model predictive control includes: a memory, a processor, and a server fan adaptive speed control program based on reinforcement learning and model predictive control stored in the memory and executable on the processor. The server fan adaptive speed control program based on reinforcement learning and model predictive control is configured to implement the steps of the server fan adaptive speed control method based on reinforcement learning and model predictive control.

[0015] The server fan adaptive speed control method and system proposed in this application, based on reinforcement learning and model predictive control, achieves adaptive adjustment of fan speed and dynamic model correction by acquiring real-time operating data, calculating the optimal fan speed sequence, generating control commands, generating adjustment strategies based on the model prediction error input reinforcement learning agent, safely updating the model and executing it cyclically. It can effectively integrate advanced prediction mechanism and dynamic optimization capability, minimize energy consumption while ensuring temperature safety boundary, and avoid control lag and temperature spike phenomena. Attached Figure Description

[0017] The accompanying drawings, which are incorporated in and form part of this specification, illustrate embodiments consistent with this application and, together with the description, serve to explain the principles of this application.

[0018] To more clearly illustrate the technical solutions in the embodiments of this application or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, for those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0019] Figure 1 This is a flowchart illustrating an embodiment of the server fan adaptive speed control method based on reinforcement learning and model predictive control provided in this application. Figure 2 This is a schematic diagram of a server fan adaptive speed control system based on reinforcement learning and model predictive control, as provided in this application.

[0020] Explanation of icon numbers: 10. Memory; 20. Processor.

[0021] The purpose, features, and advantages of this application will be further explained in conjunction with the embodiments and with reference to the accompanying drawings. Detailed Implementation

[0023] The technical solutions of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments. The components of this application described and shown in the accompanying drawings can generally be arranged and designed in various different configurations. Therefore, the following detailed description of the embodiments of this application provided in the accompanying drawings is not intended to limit the scope of this application, but merely represents selected embodiments of this application. All other embodiments obtained by those skilled in the art based on the embodiments of this application without inventive effort are within the scope of protection of this application.

[0024] It should be understood that similar reference numerals and letters in the following figures indicate similar items; therefore, once an item is defined in one figure, it does not need to be further defined and explained in subsequent figures. Furthermore, in the description of this application, the terms "first," "second," etc., are used only to distinguish descriptions and should not be construed as indicating or implying relative importance.

[0025] In existing technologies, server fan control methods, such as those based on fixed thresholds or PID control, suffer from limitations such as response lag, temperature and speed fluctuations, increased noise and wear, and energy waste. While existing improvement schemes attempt to introduce linear models, they still lack predictive capabilities and global collaborative optimization, making it impossible to proactively avoid temperature exceeding limits and difficult to achieve minimum energy consumption while ensuring safety constraints.

[0026] Based on this, embodiments of this application provide a server fan adaptive speed control method based on reinforcement learning and model predictive control, referring to... Figure 1 The server fan adaptive speed control method based on reinforcement learning and model predictive control includes steps S100 to S700, wherein: Step S100: Obtain real-time operating data of the server; the real-time operating data includes temperature monitoring data of multiple heat-generating components, current speed data of each fan, and power consumption data reflecting the server's workload; Step S200: Based on the real-time operating data and the preset model prediction controller, calculate the optimal fan speed sequence in the prediction time domain; wherein the preset model prediction controller has a built-in thermal dynamic model for describing the dynamic relationship between fan speed and component temperature. Step S300: Based on the first speed value in the optimal fan speed sequence, generate a first control command and send the first control command to the fan drive module to adjust the speed of each fan; Step S400: Based on the temperature monitoring data at the current moment and the temperature data predicted by the thermal dynamic model in the prediction time domain, calculate the model prediction error index. Step S500: The model prediction error index, the real-time running data, and the fan speed data corresponding to the first control command are input together into the pre-trained reinforcement learning agent. The pre-trained reinforcement learning agent analyzes and makes decisions based on the input data to generate a model parameter adjustment strategy. Step S600: According to the model parameter adjustment strategy, the parameters of the thermal dynamic model are updated safely to correct the thermal dynamic model; In step S700, the updated thermal dynamic model is used as the thermal dynamic model for calculating the optimal fan speed sequence in the next control cycle, and the step of acquiring real-time operating data is re-executed.

[0027] In this embodiment, the real-time operating data of the server refers to the data set collected in real time by various sensors and system interfaces during the server's operation, reflecting the server's current working status and environmental conditions. This data typically includes temperature monitoring data of multiple heat-generating components, current fan speed data, and power consumption data reflecting the server's workload. A model predictive controller is a control strategy that uses a system model to predict system behavior over a future period and calculates the optimal control input sequence for the objective function using an optimization algorithm. The first control input in the sequence is then applied to the system. In model predictive control, the prediction time domain refers to the time range within which the controller predicts the future behavior of the system, typically encompassing multiple control cycles. The optimal fan speed sequence refers to a series of fan speed setpoints calculated by the model predictive controller within the prediction time domain, aiming to meet system constraints and optimize performance indicators such as energy consumption or temperature. The thermal dynamics model describes the mathematical model of the dynamic heat exchange relationship between the temperature of internal heat-generating components and factors such as fan speed, ambient temperature, and power consumption, used to predict component temperature changes over time. The reinforcement learning agent is an intelligent entity based on reinforcement learning algorithms. Through interaction with the environment, it learns and selects actions based on the current state to maximize cumulative rewards; in this application, it is used to generate model parameter adjustment strategies. Model parameter tuning strategy refers to a set of instructions or parameters generated by a reinforcement learning agent to guide the direction and magnitude of modifications to various parameters in a thermal dynamic model, thereby correcting the model's prediction accuracy. Safety update refers to employing a series of verification and evaluation mechanisms when adjusting thermal dynamic model parameters to ensure that the updated model meets requirements in terms of safety, accuracy, and stability, avoiding the introduction of potential risks.

[0028] In this embodiment, the server fan adaptive speed control method based on reinforcement learning and model predictive control first acquires real-time operating data of the server during the implementation of the server fan adaptive speed control method. This data can be collected by various sensors and system interfaces inside the server. For example, temperature monitoring data can be obtained through temperature sensors deployed on heat-generating components such as the CPU, GPU, and memory; the current fan speed data can be obtained from the fan's built-in speed sensor or through feedback from the fan drive module; power consumption data can be provided by the power management unit or power consumption monitoring module. This data is collected periodically to reflect the server's real-time operating status.

[0029] In this embodiment, based on the acquired real-time operating data and a preset model predictive controller, the optimal fan speed sequence within the prediction time domain is calculated. This model predictive controller has a built-in thermal dynamic model describing the dynamic relationship between fan speed and component temperature. In one implementation, future temperatures can be directly predicted using a simplified linear model based on real-time operating data. Combined with a preset energy consumption target, a simple iterative search or table lookup method is used to determine the fan speed combination that maintains the temperature within a safe range over the next few time steps, thus forming the optimal fan speed sequence.

[0030] Furthermore, based on the first speed value in the optimal fan speed sequence, a first control command is generated and sent to the fan drive module to adjust the speed of each fan. Specifically, the first speed value in the optimal fan speed sequence is extracted as the target speed for each fan in the current control cycle. This target speed is then converted into an electrical signal or digital command and sent to the fan drive module via the communication interface. Upon receiving the command, the fan drive module adjusts the fan's power supply voltage or PWM signal accordingly to bring the fan speed to the target value.

[0031] In this embodiment, a model prediction error index is calculated based on the current temperature monitoring data and the temperature data predicted by the thermal dynamic model within the prediction time domain. For example, the actual measured component temperature can be compared point by point with the corresponding component temperature predicted by the thermal dynamic model, and the sum of the mean square error or absolute error between the two can be calculated as the model prediction error index. This index reflects the accuracy of the thermal dynamic model in predicting the actual system behavior.

[0032] In this embodiment, the model prediction error index, the real-time operating data, and the fan speed data corresponding to the first control command are input into a pre-trained reinforcement learning agent. The pre-trained reinforcement learning agent analyzes and makes decisions based on the input data to generate a model parameter adjustment strategy. For example, the reinforcement learning agent can be a rule-based system. When the model prediction error index exceeds a certain fixed threshold, it generates a fixed parameter adjustment amount according to a preset rule base, such as increasing or decreasing a certain thermal resistance parameter by a preset percentage, as the model parameter adjustment strategy.

[0033] In this embodiment, the parameters of the thermal dynamic model are safely updated according to the model parameter adjustment strategy to correct the thermal dynamic model. For example, the corresponding parameter values ​​in the thermal dynamic model can be directly modified according to the model parameter adjustment strategy. If the strategy indicates that a parameter needs to be increased, the adjustment amount is directly added to the current model parameter; if it indicates a decrease, the adjustment amount is directly subtracted. This update aims to make the thermal dynamic model better reflect the actual thermal dynamic characteristics of the server.

[0034] In this embodiment, the thermal dynamic model, after undergoing a security update, is used as the thermal dynamic model for calculating the optimal fan speed sequence in the next control cycle, and the step of acquiring real-time operating data is re-executed. This means that at the start of the next control cycle, the model predictive controller will use the corrected and updated thermal dynamic model to re-perform prediction and optimization calculations. This process forms a closed loop, enabling the thermal dynamic model to continuously adapt to changes in the server's operating environment, thereby achieving adaptive adjustment of fan speed.

[0035] In this embodiment, by combining real-time data acquisition, model predictive control and reinforcement learning, this method can proactively predict changes in server heat load, adaptively adjust fan speed, effectively avoid temperature exceeding limits, and at the same time reduce fan energy consumption and improve the overall efficiency and stability of the server cooling system while ensuring the safe operation of the system.

[0036] In one feasible implementation, the step of calculating the optimal fan speed sequence in the prediction time domain based on the real-time operating data and a preset model prediction controller includes: inputting the temperature monitoring data, the current fan speed data, and the power consumption data from the real-time operating data into a state observer for fusion processing to estimate the server's current complete thermal state data; predicting the future temperature change trajectory data of each heat-generating component in the prediction time domain based on the thermal dynamic model and the thermal state data; constructing a multi-objective optimization problem; the objective function of the multi-objective optimization problem includes minimizing the deviation between the future temperature change trajectory data and the preset safe temperature range, and minimizing the fan's energy consumption; and solving the multi-objective optimization problem under the condition that the temperature monitoring data does not exceed the safety upper limit, the fan speed does not exceed the physical limit, and the fan speed change rate does not exceed the smoothness limit, to obtain the optimal fan speed sequence.

[0037] In this embodiment, to obtain complete thermal state data of the server, the application inputs temperature monitoring data, current rotation speed data, and power consumption data from real-time operation data into a state observer for fusion processing. A state observer is an algorithm that uses a system model and measurement data to estimate the internal state of a system, providing accurate estimates even if some states cannot be directly measured or are subject to noise. For example, methods such as Kalman filters or extended Kalman filters can be used to combine real-time data from different sensors with a thermal dynamic model, thereby estimating the precise temperature, heat flux density, and other key thermal parameters of each heat-generating component inside the server, compensating for potential limitations or incompleteness in sensor data.

[0038] In this embodiment, after obtaining the server's current complete thermal state data, the future temperature change trajectory data of each heat-generating component is predicted within the prediction time domain based on the thermal dynamic model and the thermal state data. Specifically, the thermal dynamic model uses the currently estimated thermal state as initial conditions and, combined with possible workload changes and fan speed assumptions within the prediction time domain, uses model simulation or iterative calculation to deduce how the temperature of each heat-generating component will evolve over time in the future. This allows the controller to anticipate potential overheating risks and make proactive adjustments.

[0039] In this embodiment, to ensure server thermal safety while also considering energy efficiency, a multi-objective optimization problem is constructed. The objective function of this multi-objective optimization problem includes minimizing the deviation between the future temperature change trajectory data and the preset safe temperature range, and minimizing fan energy consumption. This means the optimization algorithm needs to consider two mutually constraining objectives simultaneously: first, to keep the component temperature within a safe and ideal range, avoiding excessively high or low temperatures; and second, to minimize the power consumption required for fan operation. The objective function typically combines these objectives through a weighted summation, for example, by weighting the square of the temperature deviation term with the fan power consumption term.

[0040] In this embodiment, a series of hard constraints must be satisfied when solving the multi-objective optimization problem, including that the temperature monitoring data does not exceed the safety upper limit, the fan speed does not exceed the physical limit, and the fan speed change rate does not exceed the smoothness limit. These constraints ensure the safety, physical feasibility, and stability of the control strategy. For example, the temperature constraint ensures that the predicted temperature of any component will not exceed its safety threshold; the fan speed constraint ensures that the fan will not be required to operate at a speed beyond its physical capabilities; and the fan speed change rate constraint avoids frequent and drastic start-stop or speed changes of the fan, thereby extending the fan's lifespan and reducing noise. Solving such constrained multi-objective optimization problems can employ numerical optimization algorithms such as quadratic programming, sequential quadratic programming, or interior-point methods.

[0041] In this embodiment, through the above technical solution, when calculating the optimal fan speed sequence, this application first utilizes a state observer to fuse real-time operating data, effectively overcoming the limitations of sensor noise and unmeasured states, thereby obtaining more accurate and complete internal thermal state data of the server. Based on this accurate thermal state data and thermal dynamic model, the system can accurately predict the temperature change trajectory of each heat-generating component in the future time domain, achieving early warning and proactive avoidance of potential overheating risks. Furthermore, by constructing a multi-objective optimization problem that includes temperature safety and energy efficiency, and strictly considering hard constraints such as temperature upper limits, fan physical limits, and speed change rate, this application can intelligently balance the server's heat dissipation performance and energy consumption, ensuring that the server can operate with the lowest energy consumption while meeting thermal safety requirements under any operating conditions. This refined method for calculating the optimal fan speed sequence improves the accuracy, robustness, safety, and energy efficiency of server fan speed regulation, enabling the entire adaptive speed regulation system to better cope with the complexity of server workload and environmental changes.

[0042] In one feasible implementation, constructing a multi-objective optimization problem includes: analyzing the proximity of the future temperature change trajectory data to a preset safe temperature range to obtain a temperature safety margin assessment result; when the temperature safety margin assessment result indicates that the safety margin is lower than a preset safety threshold, increasing the penalty weight for the deviation between the future temperature change trajectory data and the preset safe temperature range in the objective function, while decreasing the optimization weight for the energy consumption of the fan; when the temperature safety margin assessment result indicates that the safety margin is higher than the preset safety threshold, decreasing the penalty weight for the deviation between the future temperature change trajectory data and the preset safe temperature range in the objective function, while increasing the optimization weight for the energy consumption of the fan; and reconstructing the multi-objective optimization problem based on the dynamically adjusted weights.

[0043] In this embodiment, when constructing a multi-objective optimization problem, it is first necessary to analyze the proximity of the future temperature change trajectory data to the preset safe temperature range to obtain a temperature safety margin assessment result. This step aims to quantify the distance between the current or predicted temperature state and the safe threshold, thereby determining the "safety margin" of the server operation. For example, based on the future temperature change trajectory data of each heat-generating component predicted in the prediction time domain using the thermal dynamic model and the thermal state data, the difference between the highest predicted temperature and the upper limit of the preset safe temperature range can be calculated. If the difference is large, it indicates a high safety margin; if the difference is small or negative, it indicates a low safety margin. In addition, factors such as the temperature change rate and temperature duration can be considered for a more comprehensive assessment.

[0044] In this embodiment, the weights of each term in the multi-objective optimization problem are dynamically adjusted based on the temperature safety margin assessment results. When the temperature safety margin assessment results indicate that the safety margin is lower than a preset safety threshold, the system determines that the server component is at risk of overheating. At this time, the penalty weight for the deviation between the future temperature change trajectory data and the preset safe temperature range is increased in the objective function, while the optimization weight for the fan's energy consumption is decreased. For example, a safety threshold can be preset; when the difference between the highest predicted temperature and the safety upper limit is less than a certain preset percentage (e.g., 5%), the safety margin is considered low. In this case, in the multi-objective optimization function, the coefficients of terms related to temperature deviation (e.g., the term representing the squared difference between the predicted temperature and the safety upper limit) are increased, while the coefficients of terms related to fan energy consumption (e.g., the term representing the total power consumption of the fan) are correspondingly decreased. Conversely, when the temperature safety margin assessment result indicates that the safety margin is higher than the preset safety threshold, the system determines that the server component temperature is within a safe range and has sufficient margin. In this case, the penalty weight for the deviation between the future temperature change trajectory data and the preset safe temperature range is reduced in the objective function, while the optimization weight for the fan's energy consumption is increased. For example, when the difference between the highest predicted temperature and the safe upper limit is greater than a certain preset percentage (e.g., 15%), the safety margin is considered high. In this case, in the multi-objective optimization function, the coefficient of the term related to temperature deviation is reduced, while the coefficient of the term related to fan energy consumption is increased.

[0045] In this embodiment, the multi-objective optimization problem is finally reconstructed based on the dynamically adjusted weights. This step ensures that the optimizer can solve for the optimal fan speed sequence using the most appropriate weight configuration based on the latest safety margin assessment results in each control cycle. Specifically, at the beginning of each control cycle, the weight coefficients of each term in the multi-objective optimization function can be updated in real time according to the above assessment and adjustment logic, and then the optimization problem with the new weights can be input into the solver for solving. This can be achieved by modifying the mathematical expression of the optimization problem in real time, or by passing different weight parameters to a general optimization solving module.

[0046] In this embodiment, by introducing an analysis of the proximity of future temperature change trajectory data to a preset safe temperature range, and dynamically adjusting the penalty and optimization weights in the multi-objective optimization problem based on the temperature safety margin assessment results, this application enables the server fan speed control strategy to achieve a more intelligent and flexible balance under different operating conditions. When the server component temperature approaches the safe limit, the system prioritizes temperature safety by increasing the penalty weight for temperature deviation, prompting the fan to dissipate heat more actively and effectively avoiding the risk of overheating. Conversely, when the temperature is within the safe range and has sufficient margin, the system prioritizes energy consumption optimization by reducing the penalty weight for temperature deviation and increasing the energy consumption optimization weight, thereby minimizing fan operating energy consumption while ensuring safety. This adaptive weight adjustment mechanism allows the fan speed control strategy to be dynamically optimized based on the server's real-time thermal state and workload, improving the server's operational safety, stability, and energy efficiency.

[0047] In one feasible implementation, the steps of inputting the model prediction error index, the real-time operating data, and the fan speed data corresponding to the first control command into a pre-trained reinforcement learning agent, and having the pre-trained reinforcement learning agent analyze and make decisions based on the input data to generate a model parameter adjustment strategy, include: combining the model prediction error index, the real-time operating data, and the fan speed data corresponding to the first control command into the current state data of the pre-trained reinforcement learning agent; the pre-trained reinforcement learning agent selecting an action from a preset action space based on the current state data; the action being the model parameter adjustment strategy, used to represent the magnitude and direction of adjustment to multiple parameters of the thermal dynamic model; after the pre-trained reinforcement learning agent executes the action, updating the thermal dynamic model through a subsequent safety update step, and evaluating the control effect after the model update based on the recalculated model prediction error index in subsequent control cycles to calculate a reward signal; and feeding the reward signal back to the pre-trained reinforcement learning agent to update its decision network, thereby optimizing its ability to generate the model parameter adjustment strategy in the future.

[0048] In this embodiment, the model prediction error index, real-time operating data, and fan speed data corresponding to the first control command are combined to form the current state data of the pre-trained reinforcement learning agent. The model prediction error index quantifies the deviation between the predicted values ​​of the thermal dynamic model and the actual observed values, directly reflecting the model's current accuracy. The real-time operating data provides comprehensive information on the server's current operating state, including temperature monitoring data for multiple heat-generating components, the current speed data of each fan, and power consumption data reflecting the server's workload. The fan speed data corresponding to the first control command records the actual fan adjustment within the current control cycle. These heterogeneous data are fused, for example through feature concatenation, weighted averaging, or neural network encoding, to form a unified and comprehensive state vector, which serves as the basis for the reinforcement learning agent's decision-making. This comprehensive state representation ensures that the agent can fully understand the current system's thermal state, the degree of model mismatch, and the impact of control actions.

[0049] In this embodiment, the pre-trained reinforcement learning agent selects an action from a preset action space based on the current state data. This action is a model parameter adjustment strategy, representing the magnitude and direction of adjustments to multiple parameters of the thermal dynamics model. The pre-trained reinforcement learning agent has already learned a certain decision-making ability during the offline phase, enabling it to initially determine how to adjust model parameters based on the input state. The preset action space defines all legal adjustment operations that the agent can take, such as incremental or decremental adjustments to model parameters like thermal resistance, thermal capacity, and fan efficiency coefficient. The adjustment magnitude can be discrete steps or a continuous range. The agent maps the current state to a specific action in the action space through its internal policy network (e.g., a deep neural network), thereby generating a clear model parameter adjustment strategy.

[0050] In this embodiment, after the pre-trained reinforcement learning agent executes the action, the thermal dynamic model is updated through a subsequent safety update step. In subsequent control cycles, the control effect after the model update is evaluated based on the recalculated model prediction error index, and a reward signal is calculated. When the agent generates a model parameter adjustment strategy, this strategy is not immediately applied directly to the online thermal dynamic model, but undergoes a safety update process (e.g., testing and validation on a parallel learning validation model). Once the model is safely updated, the system continues to run, and in subsequent control cycles, a new model prediction error index is recalculated. By comparing the change in model prediction error before and after the update, as well as other performance indicators (such as temperature stability, energy consumption, etc.), the system can quantify the effect of this model adjustment and generate a reward signal accordingly. For example, if the model prediction error decreases, a positive reward is given; if the error increases or causes system instability, a negative reward is given.

[0051] In this embodiment, the reward signal is fed back to the pre-trained reinforcement learning agent to update its decision network, thereby optimizing its ability to generate future model parameter adjustment strategies. The reward signal is the core driving force of reinforcement learning. After receiving the reward signal, the agent uses reinforcement learning algorithms (such as policy gradient, Q-learning, etc.) to adjust its decision network (e.g., updating the weights of the neural network). Through this continuous online learning process, the agent can gradually correct and optimize its strategy, enabling it to generate more effective and accurate model parameter adjustment strategies when facing different server operating states and model mismatches. This iterative optimization mechanism allows the reinforcement learning agent to continuously adapt to changes in the server operating environment, thereby achieving true adaptive control.

[0052] In this embodiment, a closed-loop online learning mechanism is constructed through the above technical solution, effectively solving the problem that the decision-making ability of a pre-trained reinforcement learning agent may not be able to maintain its optimal state when facing a complex and ever-changing server operating environment. Specifically, by combining the model prediction error index, real-time operating data, and fan speed data corresponding to the first control command into comprehensive current state data, the reinforcement learning agent can obtain accurate perception of the system's current thermal state, model mismatch degree, and immediate control effect, providing sufficient basis for its subsequent decisions. Based on this comprehensive state, the agent can select a refined model parameter adjustment strategy from the preset action space. This strategy not only indicates the magnitude of the adjustment but also clarifies the direction of the adjustment, thereby ensuring the pertinence and effectiveness of model correction. More importantly, by evaluating the control effect after the model update based on the model prediction error index recalculated in subsequent control cycles after the agent executes the action and calculating the reward signal, the system can quantify the quality of the agent's decision. This reward signal is fed back to the reinforcement learning agent to continuously update its decision network, enabling the agent to continuously optimize its strategy based on actual operating feedback. This online learning and iterative optimization process enables the reinforcement learning agent to continuously adapt to changes in the server's operating environment, overcoming the limitations of pre-trained models and generating more accurate and robust model parameter tuning strategies. Ultimately, this ensures that the thermal dynamics model maintains high accuracy, allowing the model predictive controller to continuously output optimal fan speed sequences, further improving heat dissipation efficiency and reducing energy consumption while ensuring safe server operation.

[0053] In one feasible implementation, the step of safely updating the parameters of the thermal dynamic model to correct the thermal dynamic model according to the model parameter adjustment strategy includes: determining the severity of the current model mismatch based on the adjustment range indicated by the model parameter adjustment strategy, and selecting a corresponding safe update mode accordingly; performing parameter updates on a learning and validation model running in parallel with the preset model prediction controller based on the selected safe update mode and the model parameter adjustment strategy to obtain a candidate thermal dynamic model with updated parameters; testing and evaluating the candidate thermal dynamic model using a validation dataset to generate model validation results; and replacing the thermal dynamic model in the preset model prediction controller with the candidate thermal dynamic model when the model validation results meet preset safety, accuracy, and stability requirements, thus completing the safe update.

[0054] In this embodiment, the severity of the current model mismatch is determined based on the adjustment magnitude indicated by the model parameter adjustment strategy, and a corresponding safe update mode is selected accordingly. This means that after receiving the model parameter adjustment strategy generated by the reinforcement learning agent, the system first analyzes the parameter adjustment amount or direction indicated in the strategy. This adjustment magnitude can be a comprehensive indicator reflecting the degree of deviation between the model's current behavior and the actual system behavior. Based on a preset threshold, the adjustment magnitude is divided into different levels, such as slight or severe mismatch. For different degrees of mismatch severity, the system dynamically selects the most suitable safe update mode. For example, for slight mismatch, a fine-tuning mode might be selected; for severe mismatch, a partial reconstruction mode might be selected; and for severe mismatch, a comprehensive model relearning might be necessary. This hierarchical processing mechanism ensures the flexibility and targeting of the model update process, avoiding unnecessary over-adjustment or under-adjustment.

[0055] In this embodiment, based on the selected safe update mode and model parameter adjustment strategy, parameter updates are performed on a learning and validation model running in parallel with the preset model predictive controller to obtain a candidate thermal dynamic model with updated parameters. This means that after determining a suitable safe update mode, the system does not immediately apply the adjustment strategy to the thermal dynamic model in the currently running model predictive controller. Instead, it performs parameter updates on a "learning and validation model" independent of the main control loop. This learning and validation model is a copy of the thermal dynamic model or a model specifically for testing, running in parallel with the preset model predictive controller, but its update operation does not directly affect the server's real-time fan speed control. In this way, the model parameter adjustment strategy can be safely applied, and the learning and validation model can be updated with parameters according to the selected safe update mode (e.g., using a specific optimization algorithm or data range), thereby generating a "candidate thermal dynamic model." This candidate model contains parameter adjustments suggested by the reinforcement learning agent, but which have not yet been adopted by the main controller.

[0056] In this embodiment, using a validation dataset to test and evaluate candidate thermal dynamic models and generate model validation results means that after obtaining a candidate thermal dynamic model, it needs to be rigorously tested and evaluated to ensure its performance and safety. The system utilizes a dedicated validation dataset, which may contain historical operating data, simulated extreme operating condition data, or artificially constructed test sequences. This data is input into the candidate thermal dynamic model to simulate its behavior under various conditions, and its predictions are compared with actual observations or preset safety boundaries. Evaluation metrics may include prediction accuracy (e.g., mean squared error), satisfaction of temperature constraints, and stability under dynamic changes. Through multi-dimensional testing and evaluation, the system generates a detailed model validation result, which quantifies the various performance metrics of the candidate model.

[0057] In this embodiment, when the model validation results meet the preset safety, accuracy, and stability requirements, the thermal dynamic model in the preset model prediction controller is replaced with a candidate thermal dynamic model to complete the safe update. This means that the system will only allow a candidate thermal dynamic model to replace the thermal dynamic model currently used in the preset model prediction controller if the candidate model passes all preset rigorous validation tests and its model validation results show that it meets the preset standards in terms of safety, accuracy, and stability. These requirements are crucial to ensuring the normal operation of the server and avoiding potential risks. For example, safety requirements may include that the predicted temperature must not exceed a specific threshold; accuracy requirements may involve that the prediction error must be within an acceptable range; and stability requirements may focus on the model's convergence and robustness in the face of disturbances. Once all requirements are met, the system will perform a model switching operation, deploying the validated candidate thermal dynamic model to the main controller, thereby completing a safe and reliable model update.

[0058] In this embodiment, the most suitable safe update mode is dynamically selected based on the adjustment range of the model parameter adjustment strategy, ensuring the targeted and efficient nature of the update process. Secondly, by updating parameters on a learning and validation model running in parallel with the main controller, the potential risks of the model update process to the real-time control system are effectively isolated, avoiding system instability or performance degradation caused by improper updates. Thirdly, the candidate model is rigorously tested and evaluated using a validation dataset, ensuring the safety, accuracy, and stability of the new model and providing data support for reliable model deployment. Finally, replacement is only performed when the candidate model meets all preset requirements, ensuring that the server fan adaptive speed control system always operates in an optimal and safe state. This mechanism improves the robustness and reliability of the entire system, enabling the server to operate continuously and efficiently while minimizing the risks caused by improper model updates, guaranteeing long-term stable operation and energy efficiency optimization of the server.

[0059] In one feasible implementation, the step of determining the severity of the current model mismatch based on the adjustment magnitude indicated by the model parameter adjustment strategy, and selecting the corresponding safe update mode accordingly, includes: if the adjustment magnitude is less than a first threshold, it is determined to be a slight mismatch, and a parameter fine-tuning mode is selected, using gradient descent based on historical running data to perform a first magnitude adjustment of the parameters on the learning and verification model; if the adjustment magnitude is greater than or equal to the first threshold and less than a second threshold, it is determined to be a mismatch, and a model reconstruction mode is selected, using recently collected running data on the learning and verification model to perform a second magnitude adjustment of the parameters through recursive least squares; if the adjustment magnitude is greater than or equal to the second threshold, it is determined to be a severe mismatch, and a model relearning mode is selected, pausing online updates to the learning and verification model, and instead using the stored complete set of historical data to retrain a new thermal dynamic model as the candidate thermal dynamic model through a system identification algorithm.

[0060] In this embodiment, the adjustment magnitude refers to the quantitative indicator of adjusting the thermal dynamic model parameters in the model parameter adjustment strategy generated by the pre-trained reinforcement learning agent after analyzing and deciding based on the model prediction error index, real-time operating data, and fan speed data corresponding to the first control command. This adjustment magnitude reflects the magnitude of the deviation between the current thermal dynamic model and the actual system dynamics. To accurately determine the severity of model mismatch, this application sets a first threshold and a second threshold. The first and second thresholds are preset numerical boundaries used to divide the adjustment magnitude into different intervals, corresponding to different degrees of model mismatch severity. These thresholds are typically set through extensive offline testing, simulation analysis, or based on domain expert experience, aiming to balance the computational cost of model updates with the model accuracy requirements. For example, the first threshold can be set as a small percentage (e.g., 1%-5%) of the typical parameter variation range, while the second threshold can be set as a larger percentage (e.g., 10%-20%).

[0061] In this embodiment, when the adjustment magnitude is less than the first threshold, the system determines that the thermal dynamic model is in a slightly mismatched state. In this case, the parameter fine-tuning mode is selected. The parameter fine-tuning mode aims to make small, gradual corrections to the model parameters. Specifically, on the learning and validation model running in parallel with the preset model prediction controller, the first magnitude adjustment of the parameters is performed using gradient descent based on historical running data. Gradient descent is an iterative optimization algorithm that calculates the gradient of the loss function (e.g., model prediction error) with respect to the model parameters and updates the parameters in the opposite direction of the gradient to gradually reduce the error. Using historical running data for adjustment ensures the stability and reliability of the fine-tuning process and avoids over-correction caused by instantaneous noise or abnormal data.

[0062] In this embodiment, if the adjustment magnitude is greater than or equal to the first threshold and less than the second threshold, the system determines that the thermal dynamic model is in a mismatch state. In this case, the model reconstruction mode is selected. The model reconstruction mode aims to make more substantial updates to the model parameters to adapt to larger changes in system dynamics. Specifically, on the learning and verification model, the most recently collected running data is used to perform a second magnitude adjustment of the parameters using recursive least squares. Recursive least squares is an online parameter estimation method that can update model parameters in real time as new data arrives, and usually assigns higher weights to recent data, thereby enabling the model to quickly adapt to the current working state of the system or changes in the environment. The second magnitude adjustment is usually larger than the first magnitude adjustment to cope with more model biases.

[0063] In this embodiment, if the adjustment magnitude is greater than or equal to the second threshold, the system determines that the thermal dynamic model is in a severely mismatched state. In this case, the model relearning mode is selected. The model relearning mode aims to completely rebuild the thermal dynamic model to solve the fundamental model mismatch problem. Specifically, online updates to the learning and validation model are suspended to avoid unstable online adjustments when the model is severely mismatched. Instead, the entire set of stored historical data is used to retrain a new thermal dynamic model as the candidate thermal dynamic model through a system identification algorithm. System identification algorithms are powerful tools that can build mathematical models from the system's input and output data; for example, prediction error methods, subspace identification methods, etc., can be used. Using the entire set of historical data ensures that the newly trained model has stronger generalization ability and robustness, thereby fundamentally correcting the severely mismatched model.

[0064] In this embodiment, through the above technical solution, this application can intelligently determine the severity of mismatch in the thermal dynamic model based on the adjustment range indicated by the model parameter adjustment strategy generated by the reinforcement learning agent, and dynamically select the most suitable safe update mode accordingly. This hierarchical and adaptive model update mechanism ensures efficient parameter fine-tuning when the model has slight mismatch, rapid model reconstruction to adapt to system changes when mismatch occurs, and thorough model relearning to restore model accuracy when there is severe mismatch. This improves the accuracy and robustness of the thermal dynamic model under different operating conditions and system aging, avoids control performance degradation or system instability caused by model mismatch, and thus ensures the accuracy and energy efficiency optimization of server fan adaptive speed regulation.

[0065] In one feasible implementation, the step of testing and evaluating the candidate thermal dynamic model using a validation dataset to generate model validation results includes: constructing simulated input data for multiple validation scenarios, the simulated input data including simulated fan speed sequences and simulated workload sequences; inputting the simulated input data into the candidate thermal dynamic model to obtain simulated temperature prediction results; performing multi-dimensional evaluation on the simulated temperature prediction results, including: prediction accuracy evaluation, calculating the deviation between the simulated prediction results and preset reference values; constraint satisfaction evaluation, checking whether the temperature prediction exceeds a safety threshold in all simulated scenarios; control stability evaluation, analyzing whether there are fluctuations in the control commands derived from the candidate thermal dynamic model that exceed a preset threshold; and generating the model validation results containing various evaluation scores based on the results of the multi-dimensional evaluation; wherein, only when the weighted total score of all evaluation scores in the model validation results exceeds a preset pass threshold is it determined to meet the preset safety, accuracy, and stability requirements.

[0066] In this embodiment, simulated input data for multiple verification scenarios is constructed. This simulated input data includes simulated fan speed sequences and simulated workload sequences. To ensure a comprehensive and rigorous evaluation of the candidate thermal dynamics model, this simulated input data is not simply reused from historical operating data. Instead, it is constructed by systematically generating simulated fan speed sequences and simulated workload sequences. The simulated fan speed sequences can cover the entire operating range of the fan, from low to high speed, and include various acceleration, deceleration, and stable operation modes to simulate the dynamic changes that may occur under actual control commands. The simulated workload sequences can represent various typical server operating states, such as idle, light load, medium load, and full load, as well as scenarios with rapid or periodic load changes. In this way, a more challenging and diverse test environment than actual operating data can be created, thereby more fully exposing potential defects in the candidate thermal dynamics model.

[0067] In this embodiment, the simulated input data is input to the candidate thermal dynamics model to obtain simulated temperature prediction results. After constructing the simulated input data, it is provided as input to the candidate thermal dynamics model to be verified. The candidate thermal dynamics model receives these simulated fan speed sequences and simulated workload sequences, and calculates and outputs simulated temperature prediction results under these simulated operating conditions based on its internal thermal dynamic relationships. These prediction results reflect the candidate thermal dynamics model's ability to predict temperature changes of internal heat-generating components of the server under different simulated scenarios.

[0068] In this embodiment, the simulated temperature prediction results are evaluated from multiple dimensions, including: prediction accuracy evaluation, which calculates the deviation between the simulated prediction results and preset reference values; constraint satisfaction evaluation, which checks whether the temperature prediction exceeds a safety threshold in all simulated scenarios; and control stability evaluation, which analyzes whether the control commands derived from the candidate thermal dynamic model exhibit fluctuations exceeding a preset threshold. To comprehensively measure the performance of the candidate thermal dynamic model, this application proposes a multi-dimensional evaluation of its simulated temperature prediction results. Prediction accuracy evaluation aims to measure the closeness between the model's predicted value and the actual value or a high-precision reference value. Specifically, the model's prediction accuracy is quantified by calculating the deviation between the simulated prediction results and preset reference values, such as mean squared error (MSE) or mean absolute error (MAE). The preset reference value can be derived from a high-fidelity physical simulation model, an offline accurate identification model, or rigorously calibrated experimental data. Constraint satisfaction evaluation is a crucial step in ensuring model safety. This evaluation checks whether the temperature predicted by the candidate thermal dynamic model consistently remains within a preset safety threshold in all simulated scenarios. If the temperature prediction exceeds the safety threshold in any simulated scenario, it indicates that the model may have safety risks and requires further correction. Control stability assessment focuses on the dynamic behavior that the model may exhibit in the actual control loop. The model's stability can be determined by analyzing whether the control commands derived from the candidate thermal dynamic model (e.g., fan speed commands predicted by the simulation model when the controller uses the candidate model) fluctuate beyond a preset threshold. For example, frequent and significant changes in control commands may lead to increased fan wear, increased noise, and even affect the overall system stability. The preset threshold can be defined as the maximum permissible change in fan speed per unit time.

[0069] In this embodiment, based on the results of the multi-dimensional evaluation, a model validation result containing various evaluation scores is generated. Only when the weighted total score of all evaluation scores in the model validation result exceeds a preset passing threshold is it determined to meet the preset safety, accuracy, and stability requirements. After completing the above multi-dimensional evaluation, this application quantifies the results of each evaluation into corresponding evaluation scores. For example, prediction accuracy can be converted into a score negatively correlated with the magnitude of the deviation, constraint satisfaction can be penalized based on the degree of violation of the safety threshold, and control stability can be scored based on the degree of fluctuation. Subsequently, these evaluation scores are weighted and fused to calculate a comprehensive weighted total score. The weights of each evaluation score can be configured according to the different emphases on safety, accuracy, and stability in practical applications; for example, safety-related evaluations are usually given higher weights. Finally, only when this weighted total score exceeds the preset passing threshold is the candidate thermal dynamic model determined to meet the preset safety, accuracy, and stability requirements and can be safely deployed to the preset model prediction controller. This comprehensive evaluation mechanism ensures the comprehensiveness and reliability of model updates.

[0070] In this embodiment, by constructing simulated input data for multiple verification scenarios, the operating states of the server under different working conditions can be fully covered, avoiding testing blind spots that may result from relying solely on historical data. These simulated data are input into candidate thermal dynamic models and evaluated in multiple dimensions, including prediction accuracy, constraint satisfaction, and control stability, enabling a comprehensive measurement of model performance from multiple key perspectives. In particular, by calculating a weighted total score for each evaluation item and comparing it with a passing threshold, it is ensured that the model is adopted only when it meets preset standards in terms of safety, accuracy, and stability. This improves the reliability and security of thermal dynamic model updates, thereby guaranteeing that the server fan adaptive speed control system can achieve efficient, stable, and safe temperature control in various complex environments, effectively avoiding overheating risks or unnecessary energy consumption due to model mismatch.

[0071] In one feasible implementation, the step of replacing the thermal dynamic model in the preset model prediction controller with the candidate thermal dynamic model includes: starting from the current control cycle, not directly replacing the thermal dynamic model in subsequent consecutive control cycles; in each subsequent control cycle, calculating a first optimal fan speed sequence and a second optimal fan speed sequence based on the current thermal dynamic model and the candidate thermal dynamic model; performing weighted fusion of the first optimal fan speed sequence and the second optimal fan speed sequence based on a fusion coefficient that increases linearly from 0 to 1 to generate the fused optimal fan speed sequence actually used; when the fusion coefficient reaches 1, completing the model switching, and thereafter using the candidate thermal dynamic model for calculations.

[0072] In this embodiment, after the model verification results meet the preset requirements, the system does not immediately replace the currently used thermal dynamic model with a new candidate thermal dynamic model. This "non-direct replacement" strategy aims to avoid transient shocks or instabilities in the control system that may result from sudden model switching, providing a buffer period for the online adaptation and verification of the new model, and ensuring that the system can still maintain stable operation during the transition. To achieve a smooth transition, in each subsequent control cycle, the system uses two models in parallel for prediction and optimization. The current thermal dynamic model is the model currently used in the main controller, while the candidate thermal dynamic model is a new model that has been verified offline and is ready for replacement. By calculating separately, two independent and optimal control strategies can be obtained, namely the first optimal fan speed sequence and the second optimal fan speed sequence, providing a basis for subsequent smooth switching. The first optimal fan speed sequence represents the control suggestion based on the old model, while the second optimal fan speed sequence represents the control suggestion based on the new model. Subsequently, the system performs weighted fusion of the first optimal fan speed sequence and the second optimal fan speed sequence based on a fusion coefficient that increases linearly from 0 to 1 to generate the fused optimal fan speed sequence used in practice. The fusion coefficient is a value between 0 and 1 that increases linearly with the progression of the control cycle. For example, at the start of the switch, the fusion coefficient might be 0, indicating that the first optimal fan speed sequence is used entirely; as time progresses, the fusion coefficient gradually increases, for example, 0.1, 0.2, ..., until it reaches 1. Weighted fusion refers to linearly combining two optimal fan speed sequences according to the fusion coefficient, for example: fused optimal fan speed sequence = (1 - fusion coefficient) × first optimal fan speed sequence + fusion coefficient × second optimal fan speed sequence. This gradual fusion method allows the control command to smoothly transition from the old model to the new model, avoiding abrupt changes. Finally, when the fusion coefficient reaches 1, it means that the fused optimal fan speed sequence will be completely determined by the second optimal fan speed sequence (i.e., the result calculated based on the candidate thermal dynamic model). At this point, the system can consider that the candidate thermal dynamic model has completely taken over the control task and its effectiveness and stability have been verified through online smooth transition. Afterward, the current thermal dynamic model will be completely replaced by the candidate thermal dynamic model, and subsequent calculations will only rely on the new thermal dynamic model.

[0073] In this embodiment, the above-described technical solution effectively addresses the instability of the control system that may occur during online model updates. By introducing a gradual model switching mechanism—that is, running the old and new models in parallel for a period of time and weighted fusion of their control outputs—the system can avoid transient shocks caused by sudden model switching. The linear increase of the fusion coefficient ensures a smooth transition of control commands from the old model to the new model, making fan speed adjustment smoother and avoiding temperature fluctuations or abnormal energy consumption. This soft switching strategy not only ensures the continuity and stability of server heat dissipation control but also provides opportunities for online adaptation and verification of the new model, improving the robustness and reliability of the entire adaptive speed regulation method and ensuring that the server can continue to operate efficiently and stably during model updates.

[0074] In one feasible implementation, prior to the step of acquiring real-time operational data from the server, the method further includes: calculating multiple performance metrics of the thermal dynamic model based on historical operational data, wherein the performance metrics include at least: long-term prediction accuracy, short-term prediction stability, and parameter confidence; weightedly fusing the multiple performance metrics to calculate a comprehensive health score for the thermal dynamic model; and dynamically adjusting the exploration strategy of the pre-trained reinforcement learning agent according to the different health level ranges in which the comprehensive health score falls; wherein, when the comprehensive health score is in a low health level range, a conservative exploration strategy is adopted to limit the adjustment range of the model parameter adjustment strategy; and when the comprehensive health score is in a high health level range, an aggressive search strategy is adopted to allow the model parameter adjustment strategy to attempt adjustments exceeding a preset range.

[0075] In this embodiment, historical operating data can cover information such as server temperature, fan speed, and power consumption under different operating conditions. This data is typically stored in the server's log system or a dedicated database. Long-term prediction accuracy can be quantified by dividing historical data into training and test sets, using the model to make long-term predictions on the test set, and comparing these predictions with actual observations. Statistical metrics such as mean squared error (MSE), mean absolute error (MAE), or R-squared value can be used to calculate these metrics. Short-term prediction stability focuses on evaluating the model's responsiveness to temperature changes and prediction volatility within a short time window. For example, it checks whether the deviation between the model's predicted values ​​and actual values ​​remains within a preset range over consecutive short periods, or whether the trend of predicted value changes is smooth. Parameter confidence can be evaluated using statistical analysis methods, such as analysis of covariance or sensitivity analysis, to assess the reliability and stability of the model's internal parameters. For example, if the model parameters exhibit large variations when trained on different historical datasets, their confidence level may be low.

[0076] Based on this, multiple performance indicators are weighted and fused to calculate the overall health score of the thermal dynamic model. Weighted fusion typically employs a linear weighted summation method, where each performance indicator is assigned a weight, and then the values ​​of each indicator are multiplied by their corresponding weights before summing. For example, the overall health score can be expressed as "w1 × long-term prediction accuracy + w2 × short-term prediction stability + w3 × parameter confidence". These weights can be determined based on experience, expert knowledge, or optimized using machine learning methods (such as regression analysis) to reflect the relative importance of different indicators to the overall health of the model. In this way, a single overall health score can comprehensively and intuitively reflect the overall quality and reliability of the thermal dynamic model.

[0077] In this embodiment, the exploration strategy of the pre-trained reinforcement learning agent is dynamically adjusted according to the different health level intervals in which the comprehensive health score falls. The preset health level intervals can divide the comprehensive health score into low, medium, and high levels. For example, [0, L1) represents a low health level, [L1, L2) a medium health level, and [L2,1] a high health level. These thresholds can be calibrated according to actual system requirements and model characteristics. When the comprehensive health score is in the low health level interval, a conservative exploration strategy is adopted, limiting the adjustment range of the model parameter adjustment strategy. The conservative exploration strategy can reduce the exploration rate of the reinforcement learning agent (e.g., by setting the ε value in the ε-greedy strategy to a smaller value), making it more inclined to choose actions with known good results (exploitation) rather than trying new, uncertain actions (exploration). Simultaneously, a hard constraint is placed on the numerical range of the model parameter adjustment strategy generated by the reinforcement learning agent. For example, it is stipulated that the increment or decrement of each parameter adjustment must not exceed a certain preset maximum value to avoid aggressive updates due to model inaccuracy or instability. Conversely, when the overall health score is in the high health level range, an aggressive search strategy is adopted, allowing the model parameter adjustment strategy to attempt adjustments exceeding a preset range. The aggressive search strategy can increase the exploration rate of the reinforcement learning agent (e.g., by setting a larger ε value in the ε-greedy strategy), making it more likely to try new actions. Simultaneously, the numerical range restrictions on the model parameter adjustment strategies generated by the reinforcement learning agent are relaxed, allowing it to attempt larger parameter adjustment steps to accelerate convergence to a better solution.

[0078] In this embodiment, through the above technical solution, this application can evaluate the current state of the thermal dynamic model in real time and intelligently adjust the exploration behavior of the reinforcement learning agent accordingly. When the model health is low, the reinforcement learning agent adopts a conservative exploration strategy, limiting the adjustment range of the model parameter adjustment strategy, thereby effectively avoiding aggressive updates caused by inaccurate or unstable models, reducing system risks, and ensuring the stability of server operation. Simultaneously, when the model health is high, the reinforcement learning agent adopts an aggressive search strategy, allowing the model parameter adjustment strategy to attempt adjustments exceeding a preset range. This helps the reinforcement learning agent discover better model parameter configurations more quickly, accelerating the correction and optimization process of the thermal dynamic model, thereby improving the adaptability and energy efficiency of fan speed control. This adaptive exploration strategy based on model health makes the decision-making process of the reinforcement learning agent more intelligent and efficient, improving the robustness and performance of the entire server fan adaptive speed control method.

[0079] In the embodiments of this application, the server fan adaptive speed regulation method based on reinforcement learning and model predictive control achieves adaptive adjustment of fan speed and dynamic model correction by acquiring real-time operating data, calculating the optimal fan speed sequence, generating control commands, generating adjustment strategies based on the model prediction error input reinforcement learning agent, safely updating the model and executing it cyclically. It can effectively integrate advanced prediction mechanism and dynamic optimization capability, minimize energy consumption while ensuring temperature safety boundary, and avoid control lag and temperature spike phenomena.

[0080] It should be noted that the above examples are only for understanding this application and do not constitute a limitation on the server fan adaptive speed control method based on reinforcement learning and model predictive control in this application. Any simple modifications based on this technical concept are within the protection scope of this application.

[0081] This application also provides a server fan adaptive speed control system based on reinforcement learning and model predictive control, referenced... Figure 2 The server fan adaptive speed control system based on reinforcement learning and model predictive control includes: a memory 10, a processor 20, and a server fan adaptive speed control program based on reinforcement learning and model predictive control stored in the memory 10 and executable on the processor 20. The server fan adaptive speed control program based on reinforcement learning and model predictive control is configured to implement the steps of the server fan adaptive speed control method based on reinforcement learning and model predictive control.

[0082] The server fan adaptive speed control system based on reinforcement learning and model predictive control provided in this application, employing the server fan adaptive speed control method based on reinforcement learning and model predictive control in the above embodiments, can avoid control lag and temperature spikes. Compared with the prior art, the beneficial effects of the server fan adaptive speed control system based on reinforcement learning and model predictive control provided in this application are the same as those of the server fan adaptive speed control method based on reinforcement learning and model predictive control provided in the above embodiments, and other technical features of the server fan adaptive speed control system based on reinforcement learning and model predictive control are the same as those disclosed in the methods of the above embodiments, and will not be repeated here.

[0083] It should be understood that the various parts disclosed in this application can be implemented using hardware, software, firmware, or a combination thereof. In the description of the above embodiments, specific features, structures, materials, or characteristics can be combined in any suitable manner in one or more embodiments or examples.

[0084] The above description is merely a specific embodiment of this application, but the scope of protection of this application is not limited thereto. All equivalent structural transformations made under the technical concept of this application using the contents of the specification and drawings of this application, or direct / indirect applications in other related technical fields, are included within the scope of patent protection of this application.

Claims

1. A server fan adaptive speed control method based on reinforcement learning and model predictive control, characterized in that, The method includes: Acquire real-time operating data of the server; the real-time operating data includes temperature monitoring data of multiple heat-generating components, current speed data of each fan, and power consumption data reflecting the server's workload; Based on the real-time operating data and the preset model prediction controller, the optimal fan speed sequence in the prediction time domain is calculated; wherein the preset model prediction controller has a built-in thermal dynamic model for describing the dynamic relationship between fan speed and component temperature. Based on the first speed value in the optimal fan speed sequence, a first control command is generated and sent to the fan drive module to adjust the speed of each fan; Based on the temperature monitoring data at the current moment and the temperature data predicted by the thermal dynamic model in the prediction time domain, the model prediction error index is calculated. The model prediction error index, the real-time operating data, and the fan speed data corresponding to the first control command are input together into the pre-trained reinforcement learning agent. The pre-trained reinforcement learning agent analyzes and makes decisions based on the input data to generate a model parameter adjustment strategy. According to the model parameter adjustment strategy, the parameters of the thermal dynamic model are updated safely to correct the thermal dynamic model; The updated thermal dynamic model will be used as the thermal dynamic model for calculating the optimal fan speed sequence in the next control cycle, and the step of acquiring real-time operating data will be re-executed.

2. The server fan adaptive speed control method based on reinforcement learning and model predictive control as described in claim 1, characterized in that, The steps for calculating the optimal fan speed sequence in the prediction time domain based on the real-time operating data and the preset model prediction controller include: The temperature monitoring data, current rotation speed data, and power consumption data from the real-time operating data are input into the state observer for fusion processing to estimate the server's current complete thermal state data. Based on the thermal dynamics model and the thermal state data, predict the future temperature change trajectory data of each heat-generating component in the prediction time domain; A multi-objective optimization problem is constructed; the objective functions of the multi-objective optimization problem include minimizing the deviation between the future temperature change trajectory data and the preset safe temperature range, and minimizing the energy consumption of the fan; Under the condition that the temperature monitoring data does not exceed the safety limit, the fan speed does not exceed the physical limit, and the fan speed change rate does not exceed the smoothness limit, the multi-objective optimization problem is solved to obtain the optimal fan speed sequence.

3. The server fan adaptive speed control method based on reinforcement learning and model predictive control as described in claim 2, characterized in that, Constructing multi-objective optimization problems includes: By analyzing the degree of closeness between the future temperature change trajectory data and the preset safe temperature range, the temperature safety margin assessment result is obtained; When the temperature safety margin assessment result indicates that the safety margin is lower than the preset safety threshold, the penalty weight for the deviation between the future temperature change trajectory data and the preset safe temperature range is increased in the objective function, while the optimization weight for the energy consumption of the fan is reduced. When the temperature safety margin assessment result indicates that the safety margin is higher than the preset safety threshold, the penalty weight for the deviation between the future temperature change trajectory data and the preset safe temperature range is reduced in the objective function, while the optimization weight for the energy consumption of the fan is increased. The multi-objective optimization problem is reconstructed based on the dynamically adjusted weights.

4. The server fan adaptive speed control method based on reinforcement learning and model predictive control as described in claim 1, characterized in that, The steps of inputting the model prediction error index, the real-time operating data, and the fan speed data corresponding to the first control command into a pre-trained reinforcement learning agent, and having the pre-trained reinforcement learning agent analyze and make decisions based on the input data to generate a model parameter adjustment strategy, include: The model prediction error index, the real-time operating data, and the fan speed data corresponding to the first control command are combined to form the current state data of the pre-trained reinforcement learning agent. The pre-trained reinforcement learning agent selects actions from a preset action space based on the current state data; the actions are the model parameter adjustment strategies, used to represent the magnitude and direction of adjustment to multiple parameter items of the thermal dynamic model; After the pre-trained reinforcement learning agent performs the action, the thermal dynamic model is updated through a subsequent safety update step, and the control effect after the model update is evaluated based on the recalculated model prediction error index in the subsequent control cycle, and the reward signal is calculated. The reward signal is fed back to the pre-trained reinforcement learning agent to update its decision network, thereby optimizing its ability to generate future model parameter adjustment strategies.

5. The server fan adaptive speed control method based on reinforcement learning and model predictive control as described in claim 1, characterized in that, The steps for safely updating the parameters of the thermal dynamic model according to the model parameter adjustment strategy to correct the thermal dynamic model include: Based on the adjustment range indicated by the model parameter adjustment strategy, the severity of the current model mismatch is determined, and the corresponding safe update mode is selected accordingly. Based on the selected security update mode and the model parameter adjustment strategy, parameter updates are performed on the learning and validation model that runs in parallel with the preset model prediction controller to obtain the candidate thermal dynamic model after parameter updates. The candidate thermal dynamics model is tested and evaluated using a validation dataset to generate model validation results; When the model verification results meet the preset security, accuracy and stability requirements, the thermal dynamic model in the preset model prediction controller is replaced with the candidate thermal dynamic model to complete the security update.

6. The server fan adaptive speed control method based on reinforcement learning and model predictive control as described in claim 5, characterized in that, The steps of determining the severity of the current model mismatch based on the adjustment range indicated by the model parameter adjustment strategy, and selecting the corresponding safe update mode accordingly, include: If the adjustment range is less than the first threshold, it is determined to be a slight mismatch. The parameter fine-tuning mode is selected, and the first adjustment of the parameters is performed on the learning and verification model using the gradient descent method based on historical running data. If the adjustment magnitude is greater than or equal to the first threshold and less than the second threshold, it is determined to be a mismatch. The model reconstruction mode is selected, and the recently collected running data is used on the learning and verification model to adjust the second magnitude of the parameters through the recursive least squares method. If the adjustment range is greater than or equal to the second threshold, it is determined to be a severe mismatch. The model relearning mode is selected, the online update of the learning and verification model is suspended, and a new thermal dynamic model is retrained using the stored historical data set and the system identification algorithm as the candidate thermal dynamic model.

7. The server fan adaptive speed control method based on reinforcement learning and model predictive control as described in claim 5, characterized in that, The steps of testing and evaluating the candidate thermal dynamics model using a validation dataset and generating model validation results include: Construct simulated input data for multiple verification scenarios, the simulated input data including simulated fan speed sequences and simulated workload sequences; The simulated input data is input into the candidate thermal dynamics model to obtain the simulated temperature prediction results; The simulated temperature prediction results are evaluated in multiple dimensions, including: prediction accuracy evaluation, calculating the deviation between the simulated prediction results and the preset reference value; constraint satisfaction evaluation, checking whether the temperature prediction exceeds the safety threshold in all simulated scenarios; and control stability evaluation, analyzing whether the control commands derived from the candidate thermal dynamic model have fluctuations exceeding the preset threshold. Based on the results of the multi-dimensional evaluation, the model validation result containing the scores of each evaluation item is generated; wherein, only when the weighted total score of all evaluation items in the model validation result exceeds the preset qualification threshold is it determined to meet the preset security, accuracy and stability requirements.

8. The server fan adaptive speed control method based on reinforcement learning and model predictive control as described in claim 5, characterized in that, The step of replacing the thermal dynamic model in the preset model prediction controller with the candidate thermal dynamic model includes: Starting from the current control cycle, the thermal dynamic model will not be directly replaced in subsequent consecutive control cycles; In each subsequent control cycle, the first optimal fan speed sequence and the second optimal fan speed sequence are calculated based on the current thermal dynamic model and the candidate thermal dynamic model, respectively. Based on a fusion coefficient that increases linearly from 0 to 1, the first optimal fan speed sequence and the second optimal fan speed sequence are weighted and fused to generate the fused optimal fan speed sequence for actual use. When the fusion coefficient reaches 1, the model switch is completed, and thereafter the candidate thermal dynamic model is used entirely for calculation.

9. The server fan adaptive speed control method based on reinforcement learning and model predictive control as described in claim 1, characterized in that, Prior to the step of acquiring the server's real-time operational data, the method further includes: Based on historical operating data, multiple performance indicators of the thermal dynamics model are calculated, including at least: long-term prediction accuracy, short-term prediction stability, and parameter confidence. The comprehensive health score of the thermal dynamic model is calculated by weighting and fusing multiple performance indicators. The exploration strategy of the pre-trained reinforcement learning agent is dynamically adjusted according to the different health level ranges in which the comprehensive health score is located. Specifically, when the comprehensive health score is in the low health level range, a conservative exploration strategy is adopted to limit the adjustment range of the model parameter adjustment strategy. When the comprehensive health score is in the high health level range, an aggressive search strategy is adopted, allowing the model parameter adjustment strategy to attempt adjustments beyond the preset range.

10. A server fan adaptive speed control system based on reinforcement learning and model predictive control, characterized in that, The server fan adaptive speed control system based on reinforcement learning and model predictive control includes: a memory, a processor, and a server fan adaptive speed control program based on reinforcement learning and model predictive control stored in the memory and executable on the processor. The server fan adaptive speed control program based on reinforcement learning and model predictive control is configured to implement the steps of the server fan adaptive speed control method based on reinforcement learning and model predictive control as described in any one of claims 1 to 9.

Images