An intelligent fan control method and device based on simulation and reinforcement learning

Abstract

The application discloses an intelligent fan control method and device based on simulation and reinforcement learning, and belongs to the technical field of server heat dissipation control. The application combines historical data and control actions to build a processor temperature prediction model with physical constraints, thereby avoiding unreasonable prediction results. A fan power consumption prediction model is constructed according to historical control sequences. A simulation environment is built relying on the two types of prediction models, and a discrete action reinforcement learning controller is trained, so that the controller can output fan control instructions according to system states. The trained controller is deployed on a server management chip, and the device periodically collects system states, and the controller issues control instructions in real time to complete heat dissipation regulation and control. The application takes into account the temperature control effect and power consumption performance, and realizes automatic and intelligent heat dissipation management of the server fan.

Description

Technical Field

[0001] This invention relates to the field of server heat dissipation control technology, and in particular to an intelligent fan control method and device based on simulation and reinforcement learning. Background Technology

[0002] As a core infrastructure for cloud computing, artificial intelligence, and high-performance computing, the CPU heat dissipation control of servers directly affects system stability and energy efficiency. In related technologies, a basic system for server heat dissipation control is formed by collecting parameters such as CPU temperature, power consumption, and fan speed using a BMC chip, and then employing a PID controller to output a PWM duty cycle to adjust the fan speed. Specifically, this system covers the entire process from temperature sensing and load monitoring to fan speed adjustment, including traditional PID, PID based on BP neural network parameter tuning, and continuous motion reinforcement learning control schemes.

[0003] However, existing control methods, which directly employ fixed or indirectly adjusted PID parameters, do not fully consider the dynamic response requirements under rapid fluctuations in CPU load. This can lead to issues such as temperature response lag, overshoot, and excessive fan power consumption. Furthermore, continuous-action reinforcement learning methods are not perfectly compatible with the discrete control characteristics of server fan PWM. Direct deployment on a BMC (Browser Control Center) is also limited by inference complexity, feedback cycle, and hardware compatibility. In addition, training a reinforcement learning controller on a real server carries the risk of overheating, and temperature predictors that rely solely on data fitting may produce non-physical predictions, affecting the effectiveness of the control strategy. Summary of the Invention

[0004] The present invention aims to at least partially solve one of the technical problems in the related art.

[0005] Therefore, the first objective of this invention is to propose an intelligent fan control method based on simulation and reinforcement learning.

[0006] Another objective of this invention is to propose an intelligent fan control device based on simulation and reinforcement learning.

[0007] The third objective of this invention is to provide a computer device.

[0008] A fourth objective of this invention is to provide a non-transitory computer-readable storage medium.

[0009] To achieve the above objectives, a first aspect of the present invention proposes an intelligent fan control method based on simulation and reinforcement learning, comprising:

[0010] S10, Construct a temperature prediction model. The temperature prediction model predicts the processor temperature at the next moment based on historical operating data and current control actions, and introduces physical constraint terms during the training process to suppress non-physical prediction results. S20, Construct a power consumption prediction model, which predicts fan power consumption based on historical control action sequences; S30, a simulation environment is constructed using the temperature prediction model and the power consumption prediction model, and a discrete action reinforcement learning controller is trained in the simulation environment. The controller outputs discrete fan control actions according to the system state. S40, the trained controller is deployed to the server management chip. In each control cycle, the server management chip collects the current system status and outputs the corresponding fan control action through the controller to perform heat dissipation control.

[0011] In one embodiment of the present invention, S10 includes: A time fusion network is constructed, which includes a historical sequence branch, a current state branch, and a fusion output branch. The historical sequence branch uses the multi-dimensional sequence features of the first k-1 time steps as input to an LSTM network. The current state branch uses the current CPU power and current PWM action as inputs to the MLP network. The features of the historical sequence branch and the current state branch are fused and the predicted temperature is obtained by the linear output layer.

[0012] In one embodiment of the present invention, introducing physical constraints during training to suppress non-physical prediction results includes: A physical consistency loss is introduced into the total loss of the temperature prediction model. The physical consistency loss adopts the Pearson correlation penalty between the PWM sequence and the predicted temperature change. When the correlation is positive and it does not belong to the rapid temperature rise masking condition, the loss is increased for the training sample.

[0013] In one embodiment of the present invention, the total loss consists of the prediction error loss and the physical consistency loss, and the total loss is expressed as: , in Represents a correlation metric. For physical constraint weights, This is the operating condition mask.

[0014] In one embodiment of the present invention, S20 includes: A lightweight LSTM model is used, taking the PWM sequence as input and outputting the prediction of fan power consumption. The power consumption prediction relationship is expressed as follows: ,in This represents the power predictor parameters.

[0015] In one embodiment of the present invention, S30 includes: The discrete action reinforcement learning controller is implemented using a deep Q-network (DQN), and the DQN network is input to the system state. The controller outputs the Q value corresponding to each candidate PWM action, and selects the PWM action with the largest Q value as the current control output.

[0016] In one embodiment of the present invention, S40 includes: The DQN controller parameters are frozen and deployed to the server BMC chip or the hardware management module that works in conjunction with the BMC. Only lightweight forward inference is performed during the deployment phase. BMC reads CPU power, current temperature, and the previous PWM cycle to construct a state vector. ; The DQN network calculates the Q value of each discrete PWM action; Select the PWM action corresponding to the maximum Q value; the BMC writes the PWM into the fan control interface.

[0017] To achieve the above objectives, a second aspect of the present invention provides an intelligent fan control device based on simulation and reinforcement learning, comprising: A temperature prediction model building module is used to build a temperature prediction model. The temperature prediction model predicts the processor temperature at the next moment based on historical operating data and current control actions, and introduces physical constraint terms during the training process to suppress non-physical prediction results. A power consumption prediction model building module is used to build a power consumption prediction model, which predicts fan power consumption based on historical control action sequences. The reinforcement learning controller training module is used to construct a simulation environment using the temperature prediction model and the power consumption prediction model, and to train a discrete action reinforcement learning controller in the simulation environment. The controller outputs discrete fan control actions according to the system state. The fan cooling control deployment and execution module is used to deploy the trained controller to the server management chip. In each control cycle, the server management chip collects the current system status and outputs the corresponding fan control action through the controller to perform cooling control.

[0018] This invention provides an intelligent fan control method and device based on simulation and reinforcement learning, which solves the problems of traditional PID control such as response lag, overshoot, and high energy consumption, as well as the problems of reinforcement learning in real server training risk, action space mismatch, poor physical consistency, and poor adaptability to multiple setpoints. It reduces the training risk of real servers, improves the physical consistency of temperature prediction, adapts to discrete PWM, enhances over-temperature suppression capability, and balances heat dissipation and energy consumption.

[0019] To achieve the above objectives, a third aspect of this application provides a computer device comprising: a processor and a memory; wherein the processor runs a program corresponding to the executable program code by reading executable program code stored in the memory, for implementing the method described in the first aspect embodiment.

[0020] To achieve the above objectives, a fourth aspect of this application provides a non-transitory computer-readable storage medium having a computer program stored thereon that, when executed by a processor, implements the method described in the first aspect.

[0021] Additional aspects and advantages of the invention will be set forth in part in the description which follows, and in part will be obvious from the description, or may be learned by practice of the invention. Attached Figure Description

[0022] Figure 1 This is a flowchart of an intelligent fan control method based on simulation and reinforcement learning according to an embodiment of the present invention; Figure 2 This is a system block diagram according to an embodiment of the present invention; Figure 3 This is an architecture diagram of a TFN temperature simulator according to an embodiment of the present invention; Figure 4 This is a comparison chart of the accuracy of power predictors according to an embodiment of the present invention; Figure 5 This is a diagram of a reinforcement learning controller architecture according to an embodiment of the present invention; Figure 6 This is a diagram showing the I-PID controller temperature control curve according to an embodiment of the present invention; Figure 7 This is a structural diagram of an intelligent fan control device based on simulation and reinforcement learning according to an embodiment of the present invention; Figure 8 It is a computer device according to an embodiment of the present invention. Detailed Implementation

[0023] It should be noted that, unless otherwise specified, the embodiments and features described in the present invention can be combined with each other. The present invention will now be described in detail with reference to the accompanying drawings and embodiments.

[0024] To enable those skilled in the art to better understand the present invention, the technical solutions of the present invention will be clearly and completely described below with reference to the accompanying drawings of the embodiments of the present invention. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort should fall within the scope of protection of the present invention.

[0025] The following description, with reference to the accompanying drawings, describes an intelligent fan control method and apparatus based on simulation and reinforcement learning according to an embodiment of the present invention.

[0026] Figure 1 This is a flowchart of an intelligent fan control method based on simulation and reinforcement learning according to an embodiment of the present invention, such as... Figure 1 As shown, it includes: S10, Construct a temperature prediction model. The temperature prediction model predicts the processor temperature at the next moment based on historical operating data and current control actions, and introduces physical constraint terms during the training process to suppress non-physical prediction results. S20, Construct a power consumption prediction model, which predicts fan power consumption based on historical control action sequences; S30, a simulation environment is constructed using the temperature prediction model and the power consumption prediction model, and a discrete action reinforcement learning controller is trained in the simulation environment. The controller outputs discrete fan control actions according to the system state. S40, the trained controller is deployed to the server management chip. In each control cycle, the server management chip collects the current system status and outputs the corresponding fan control action through the controller to perform heat dissipation control.

[0027] This invention provides an intelligent fan control system based on simulation and reinforcement learning. The system includes a data acquisition module, a temperature prediction module, a power consumption prediction module, a simulation environment module, a DQN discrete controller training module, a dynamic setpoint training module, and a BMC deployment and execution module. The overall approach adopts a technical route of "real data acquisition—predictor modeling—virtual environment training—controller freezing—BMC deployment and inference." The hardware relationships between the BMC chip, CPU, fan, PWM control signal, and heat dissipation path in the server are as follows: Figure 2 As shown.

[0028] In one embodiment of the present invention, the system variables and control objectives are defined as follows: The server CPU heat dissipation control process operates in discrete time steps t. The core variables collected or calculated by the system include: instantaneous CPU power. Fan PWM duty cycle CPU temperature Fan power consumption and temperature setpoint .in Used to characterize workload intensity The discrete control actions output by the controller. The temperature of the controlled object, Energy consumption optimization indicators.

[0029] Furthermore, the controller's objective is to reduce average fan power consumption and improve response speed to load fluctuations while ensuring that the CPU temperature does not exceed the set temperature or minimizes the over-temperature range. This objective can be expressed as: minimizing the combined cost of average fan power consumption and temperature deviation over a period of time under safety constraints, where δ is the allowable temperature margin.

[0030] Furthermore, the PWM action space is discretized based on the duty cycle resolution supported by the BMC and fan hardware. For example, the action set can be represented as:

[0031] Example of a control target:

[0032] In one embodiment of the present invention, multi-scenario data acquisition and preprocessing are performed to construct a simulator that can replace the real environment. First, multi-dimensional time-series data is collected on a cloud server and OpenBMC system platform. The collected data includes CPU power sequence W, PWM control sequence P, CPU real-time temperature sequence T, and fan power consumption sequence E.

[0033] Furthermore, the data acquisition covers three typical scenarios: random power load, square wave power load, and full power load. For example, it can acquire data for 2 hours of random power load, 2 hours of square wave power load, and 3 hours of full power load; the PWM control sequence outputs random PWM values ​​for the first 6 hours to cover the control space, and keeps the PWM at 0 for the last hour to record the thermal response without cooling intervention.

[0034] Furthermore, after data acquisition, time alignment, outlier filtering, normalization, and sliding window slicing are performed on the data, which is then divided into training and validation sets. The training set is used to fit the temperature predictor and power predictor, while the validation set is used to evaluate the consistency between the simulation environment and the real system.

[0035] In one embodiment of the present invention, a temperature prediction module Furthermore, the temperature prediction module It is implemented using a time fusion network (TFN), with the specific structure as follows: Figure 3As shown, this module includes a history sequence branch, a current state branch, and a fused output branch. The history sequence branch uses multi-dimensional sequence features from the previous k-1 time steps of the LSTM network input, such as historical CPU power sequences, historical PWM sequences, and historical CPU temperature sequences, to extract the temporal relationship between load changes, fan control, and temperature inertia.

[0036] Furthermore, the current state branch uses an MLP network to input the current CPU power. and current PWM action This is used to extract the immediate impact of the current control action on the temperature at the next moment. The features from the two branches are fused through an attention layer or a feature concatenation layer, and the predicted temperature is obtained from the linear output layer. The temperature prediction relationship can be expressed as:

[0037] in, This represents the parameters of the temperature predictor.

[0038] Furthermore, to verify the effectiveness of the TFN structure, ablation experiments can be used to compare TFN models with only MLP branching, only LSTM branching, and different time window lengths. Example experimental results are shown in Table 1. The temperature prediction MAE of TFN at ts=5 reaches 1.83℃, which is superior to the single-branch structure.

[0039] Table 1

[0040] In one embodiment of the present invention, during predictor training with physical constraints, to prevent the temperature predictor from generating non-physical predictions by relying solely on data correlation, the present invention introduces a physical constraint term into the temperature predictor training loss. This constraint term is used to suppress the situation where the predicted temperature rises synchronously with the PWM increase during non-rapid temperature rise moments.

[0041] In one implementation, the total loss of the temperature predictor consists of prediction error loss and physical consistency loss. The prediction error loss can be SmoothL1 loss or mean squared error loss; the physical consistency loss can be a Pearson correlation penalty between the PWM sequence and the predicted temperature change, with an additional loss applied to the training sample when the correlation is positive and it does not belong to the rapid temperature rise masking condition.

[0042] Furthermore, its function is to ensure that the simulation environment not only makes accurate numerical predictions, but also that the output direction conforms to the physical laws of heat dissipation, thereby avoiding the DQN controller from being trained on erroneous feedback.

[0043]

[0044] Where ρ represents the relevance measure, and λ is the physical constraint weight. This is the operating condition mask. This is used to reduce constraint strength in scenarios where the temperature rises rapidly and the system reasonably increases the PWM to suppress the temperature. Example experimental results are shown in Table 2: Table 2

[0045] The MTD in Table 2 reflects the difference in the direction of the predictor's temperature response under PWM variations. By introducing physical constraints, the predictor output can more stably meet the heat dissipation physical consistency requirements set by this invention, reducing the interference of non-physical predictions on controller training.

[0046] In one embodiment of the present invention, a power prediction module Power Prediction Module A lightweight LSTM model is used to predict fan power consumption by taking the PWM sequence as input and outputting it. Unlike prediction methods that simply assume a fixed square relationship between PWM and fan power, LSTM power predictors can learn dynamic characteristics such as fan speed switching, the cumulative effect of historical PWM, and instantaneous power fluctuations. The power prediction relationship can be expressed as:

[0047] in This represents the parameters of the power predictor. In the example experiment, the prediction accuracy of the two predictors is compared as follows: Figure 4 As shown, the LSTM power predictor The efficiency is 0.9365, significantly better than the 0.5784 of the squared predictor. This module provides energy feedback to the reward function, enabling the controller to balance over-temperature suppression and energy saving.

[0048] In one embodiment of the present invention, a simulation environment module is provided, which comprises a temperature prediction module. and power prediction module Together, they constitute the system. During training, the controller does not directly act on the real fan; instead, it inputs candidate PWM actions into the simulation environment, which then returns the predicted temperature. Predicted power consumption Next state and rewards .

[0049] Furthermore, the purpose of this design is to transfer high-risk, long-cycle real-world interactions to a virtual environment, thereby avoiding CPU overheating or fan malfunctions that may occur during the reinforcement learning exploration phase, while also reducing reliance on repeated deployment and debugging of BMC hardware.

[0050] In one embodiment of the present invention, a DQN discrete controller is implemented using a deep Q-network (DQN). The interactive training structure between the DQN controller and the simulation environment is as follows. Figure 5 As shown. DQN network input system status. The controller outputs the Q value corresponding to each candidate PWM action, and selects the PWM action with the largest Q value as the current control output. Since DQN is naturally adapted to discrete action spaces, it can directly match the discrete control method of server fan PWM.

[0051] In one implementation, the system state is defined as follows:

[0052] in, CPU power at the previous moment. The PWM value at the previous moment. This refers to the CPU temperature at the previous moment. This represents the proportional characteristics of temperatures approaching a set point.

[0053] Furthermore, the reward function is segmented based on whether the predicted temperature exceeds the setpoint. When the predicted temperature exceeds the setpoint, the reward is positively correlated with the predicted fan power consumption to encourage the controller to increase PWM and enhance cooling; when the predicted temperature does not exceed the setpoint, the reward is negatively correlated with the predicted fan power consumption to encourage the controller to reduce energy consumption at a safe temperature.

[0054] The reward function can be expressed as:

[0055] Furthermore, the DQN training process includes: selecting actions according to an ε-greedy policy; executing actions in a simulation environment and obtaining rewards and new states; storing samples consisting of states, actions, rewards, and the next state into an experience replay pool; randomly sampling mini-batch data from the replay pool; calculating the target Q-value using the current Q-network and the target Q-network; minimizing the Q-value error and updating the network parameters; and synchronizing the current Q-network parameters to the target network every preset number of steps.

[0056] In one embodiment of the present invention, dynamic setpoint training is employed to enable the controller to adapt to different operating temperature thresholds. During the training phase, multiple temperature setpoints are selected, such as 50°C, 55°C, 60°C, 65°C, and 70°C, and the training data is evenly distributed according to these setpoints. This allows the controller to learn the energy consumption range and over-temperature penalty relationship under different temperature targets.

[0057] Furthermore, after dynamic setpoint training, the controller does not need to be readjusted for each setpoint, and can output appropriate PWM under light load and low temperature, high load and high temperature, and setpoint switching scenarios. This strategy improves the controller's versatility and deployment convenience.

[0058] In one embodiment of the present invention, the BMC deployment execution process involves freezing the DQN controller parameters and deploying them to the server BMC chip or a hardware management module that works in conjunction with the BMC after training is completed. During the deployment phase, reinforcement learning training is no longer performed; only lightweight forward inference is executed.

[0059] Furthermore, within each control cycle, the BMC reads the CPU power, current temperature, and the PWM from the previous cycle to construct a state vector. The DQN network calculates the Q value of each discrete PWM action; selects the PWM action corresponding to the maximum Q value; the BMC writes the PWM into the fan control interface; then the system enters the next control cycle.

[0060] Furthermore, this deployment method avoids complex control algorithms from undergoing long-term online learning on the BMC, retaining only the rapid inference process, thus making it more suitable for server hardware management scenarios with limited resources and high real-time requirements.

[0061] In one embodiment of the present invention, the performance evaluation method, for evaluating the controller effect, uses the average temperature deviation. Maximum temperature deviation Average overtemperature value Average fan power consumption And the five indicators of comprehensive cost control: 3C. , , , The smaller the 3C value, the better the control performance.

[0062] Furthermore, each evaluation indicator can be expressed as:

[0063] The comparison results of examples in real and simulated environments are shown in Tables 3, 4, and 5: Table 3

[0064] Table 4

[0065] Table 5

[0066] Furthermore, Table 3 shows that, under real-world conditions, the solution of this invention, compared to Naïve-PID and BP-PID, exhibits lower maximum temperature deviation. Average overtemperature value It is superior in terms of overall cost control and 3C aspects; Table 4 shows that, under simulation environment, the solution of this invention is superior. , Both the cost and the 3C cost are significantly lower than those of the comparative controller; Table 5 shows that the solution of the present invention can reduce or maintain a low overall control cost at multiple set points.

[0067] The embodiments of this invention also have the following technical effects: Reduced training risks on real servers. This invention constructs a high-fidelity simulation environment through a temperature predictor and a power consumption predictor, allowing the reinforcement learning controller to primarily complete exploration and training in a virtual environment, reducing the risk of overheating and hardware wear caused by trial and error directly on real servers. Improved physical consistency of temperature prediction. This invention introduces a physical constraint term into the temperature predictor loss function, ensuring that the predictor output conforms to the basic law of enhanced heat dissipation as PWM increases, reducing control strategy deviations caused by non-physical feedback from the training source. Adaptation to discrete PWM action space. This invention uses a DQN controller to directly output discrete PWM actions, avoiding mapping errors between continuous control algorithms and discrete fan interfaces, making it more suitable for BMC fan control scenarios. Enhanced over-temperature suppression capability. Experimental results show that the maximum temperature deviation of the controller of this invention in a real environment is significantly reduced. The average overtemperature value is 0.61. The cost is 0.32, and the overall control cost (3C) is 0.33; in a simulation environment, the controller of this invention... It is 0.02. The values ​​are 0.01 for 0.01 and 0.03 for 3C, both demonstrating good temperature stability control capabilities. Compared to the I-PID controller, the controller of this invention exhibits better response speed and temperature overshoot suppression performance. Figure 6 As shown. Balancing heat dissipation and energy consumption. The reward function encourages enhanced cooling when temperatures exceed limits and reduced power consumption when temperatures are within safe limits, allowing the controller to dynamically balance cooling intensity and fan energy consumption based on temperature conditions. Supporting multi-setpoint deployment. Through dynamic setpoint training, the controller of this invention can adapt to various temperature threshold scenarios within the range of 50℃ to 70℃, reducing the need for re-tuning and repetitive training. Improving BMC deployment feasibility. After training, the BMC only needs to perform lightweight DQN forward inference and output PWM actions, eliminating the need for complex online training on the BMC and reducing deployment difficulty.

[0068] To achieve the above embodiments, such as Figure 7 As shown, this embodiment also provides an intelligent fan control device 10 based on simulation and reinforcement learning, including: Temperature prediction model building module 100 is used to build a temperature prediction model. The temperature prediction model predicts the processor temperature at the next moment based on historical operating data and current control actions, and introduces physical constraint terms during the training process to suppress non-physical prediction results. A power consumption prediction model building module 200 is used to build a power consumption prediction model, which predicts fan power consumption based on historical control action sequences. The reinforcement learning controller training module 300 is used to construct a simulation environment using the temperature prediction model and the power consumption prediction model, and to train a discrete action reinforcement learning controller in the simulation environment. The controller outputs discrete fan control actions according to the system state. The fan cooling control deployment and execution module 400 is used to deploy the trained controller to the server management chip. In each control cycle, the server management chip collects the current system status and outputs the corresponding fan control action through the controller to perform cooling control.

[0069] This invention provides an intelligent fan control device based on simulation and reinforcement learning, which solves the problems of traditional PID control such as response lag, overshoot, and high energy consumption, as well as the problems of reinforcement learning in real server training risk, action space mismatch, poor physical consistency, and poor adaptability to multiple setpoints. It reduces the training risk of real servers, improves the physical consistency of temperature prediction, adapts to discrete PWM, enhances over-temperature suppression capability, and balances heat dissipation and energy consumption.

[0070] To implement the methods of the above embodiments, the present invention also provides a computer device, such as... Figure 8 As shown, the computer device 600 includes a memory 601 and a processor 602; wherein, the processor 602 reads executable program code stored in the memory 601 to run a program corresponding to the executable program code, so as to implement the various steps of the method described above.

[0071] To implement the above embodiments, this application also proposes a non-transitory computer-readable storage medium storing a computer program thereon, which, when executed by a processor, implements the method described in the foregoing embodiments.

[0072] In the description of this specification, the references to terms such as "one embodiment," "some embodiments," "example," "specific example," or "some examples," etc., refer to specific features, structures, materials, or characteristics described in connection with that embodiment or example, which are included in at least one embodiment or example of the present invention. In this specification, the illustrative expressions of the above terms do not necessarily refer to the same embodiment or example. Furthermore, the specific features, structures, materials, or characteristics described may be combined in any suitable manner in one or more embodiments or examples. Moreover, without contradiction, those skilled in the art can combine and integrate the different embodiments or examples described in this specification, as well as the features of different embodiments or examples.

[0073] Furthermore, the terms "first" and "second" are used for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the number of technical features indicated. Thus, a feature defined as "first" or "second" may explicitly or implicitly include at least one of that feature. In the description of this invention, "a plurality of" means at least two, such as two, three, etc., unless otherwise explicitly specified.

Claims

1. A smart fan control method based on simulation and reinforcement learning, characterized in that, include: S10, Construct a temperature prediction model. The temperature prediction model predicts the processor temperature at the next moment based on historical operating data and current control actions, and introduces physical constraint terms during the training process to suppress non-physical prediction results. S20, Construct a power consumption prediction model, which predicts fan power consumption based on historical control action sequences; S30, a simulation environment is constructed using the temperature prediction model and the power consumption prediction model, and a discrete action reinforcement learning controller is trained in the simulation environment. The controller outputs discrete fan control actions according to the system state. S40, the trained controller is deployed to the server management chip. In each control cycle, the server management chip collects the current system status and outputs the corresponding fan control action through the controller to perform heat dissipation control.

2. The method as described in claim 1, characterized in that, S10 includes: A time fusion network is constructed, which includes a historical sequence branch, a current state branch, and a fusion output branch. The historical sequence branch uses the multi-dimensional sequence features of the first k-1 time steps as input to an LSTM network. The current state branch uses the current CPU power and current PWM action as inputs to the MLP network. The features of the historical sequence branch and the current state branch are fused and the predicted temperature is obtained by the linear output layer.

3. The method as described in claim 2, characterized in that, The method of introducing physical constraints during training to suppress non-physical prediction results includes: A physical consistency loss is introduced into the total loss of the temperature prediction model. The physical consistency loss adopts the Pearson correlation penalty between the PWM sequence and the predicted temperature change. When the correlation is positive and it does not belong to the rapid temperature rise masking condition, the loss is increased for the training sample.

4. The method as described in claim 3, characterized in that, The total loss consists of the prediction error loss and the physical consistency loss, and the total loss is expressed as follows: ， in Represents a correlation metric. For physical constraint weights, This is the operating condition mask.

5. The method as described in claim 1, characterized in that, S20 includes: A lightweight LSTM model is used, taking the PWM sequence as input and outputting the prediction of fan power consumption. The power consumption prediction relationship is expressed as follows: ,in This represents the power predictor parameters.

6. The method as described in claim 1, characterized in that, S30 includes: The discrete action reinforcement learning controller is implemented using a deep Q-network (DQN), and the DQN network is input to the system state. The controller outputs the Q value corresponding to each candidate PWM action, and selects the PWM action with the largest Q value as the current control output.

7. The method as described in claim 1, characterized in that, S40 includes: The DQN controller parameters are frozen and deployed to the server BMC chip or the hardware management module that works in conjunction with the BMC. Only lightweight forward inference is performed during the deployment phase. BMC reads CPU power, current temperature, and the previous PWM cycle to construct a state vector. ; The DQN network calculates the Q value of each discrete PWM action; Select the PWM action corresponding to the maximum Q value; the BMC writes the PWM into the fan control interface.

8. An intelligent fan control device based on simulation and reinforcement learning, characterized in that, include: A temperature prediction model building module is used to build a temperature prediction model. The temperature prediction model predicts the processor temperature at the next moment based on historical operating data and current control actions, and introduces physical constraint terms during the training process to suppress non-physical prediction results. A power consumption prediction model building module is used to build a power consumption prediction model, which predicts fan power consumption based on historical control action sequences. The reinforcement learning controller training module is used to construct a simulation environment using the temperature prediction model and the power consumption prediction model, and to train a discrete action reinforcement learning controller in the simulation environment. The controller outputs discrete fan control actions according to the system state. The fan cooling control deployment and execution module is used to deploy the trained controller to the server management chip. In each control cycle, the server management chip collects the current system status and outputs the corresponding fan control action through the controller to perform cooling control.

9. A computer device, characterized in that, Including processor and memory; The processor reads executable program code stored in the memory to run a program corresponding to the executable program code, so as to implement the method as described in any one of claims 1-7.

10. A non-transitory computer-readable storage medium having a computer program stored thereon, characterized in that, When the program is executed by the processor, it implements the method as described in any one of claims 1-7.

Images