Air conditioners, their control methods, and computer program products

Abstract

This application discloses an air conditioner, its control method, and a computer program product. The method includes: acquiring a first temperature and a first humidity in each area of ​​a data center at the current moment, as well as a first load power of electronic equipment; using the first temperature, first humidity, and first load power as inputs to a target evaluation network to output decision scores for multiple control operations on the air conditioner, wherein training samples are input to the evaluation network and the target network respectively to obtain a first decision score and a second decision score, and when the difference between the first decision score and the second decision score reaches a preset minimum value, the evaluation network is updated to obtain the target evaluation network; and control operations are determined based on the decision scores to enable the air conditioner to operate based on the control operations. The method of this application can quickly respond to changes in the load of electronic equipment, accurately control the cooling capacity of the air conditioner, and optimize the overall energy consumption of the data center.

Description

Technical Field

[0001] This application relates to the field of air conditioner technology, and more particularly to air conditioners, control methods thereon, and computer program products. Background Technology

[0002] With the continuous expansion of data center scale and the rapid development of information technology, the energy consumption problem of data centers is becoming increasingly prominent. Electronic equipment in data centers has stringent requirements for the operating environment temperature, requiring continuous cooling from air conditioning systems to ensure their stable operation. This makes air conditioning systems one of the main energy-consuming devices in data centers.

[0003] Currently, most data centers employ air conditioning control strategies based on temperature thresholds. Multiple temperature sensors are installed within the data center to monitor the temperature in different areas in real time. When the temperature in a certain area exceeds a preset high-temperature threshold, the air conditioning system increases its cooling capacity; when the temperature falls below a preset low-temperature threshold, the air conditioning system reduces its cooling capacity or stops cooling.

[0004] The aforementioned methods have a slow response time. When the load on electronic devices changes suddenly, the temperature sensor detects the temperature change and transmits it to the air conditioning system, which then makes adjustments. This process involves a significant delay, potentially causing electronic devices to be in a high-temperature environment for a short period, affecting their performance and lifespan. Furthermore, it cannot achieve globally optimal energy-saving effects. It only adjusts based on local temperature changes, which may cause some areas to over-cool while others under-cool, resulting in energy waste. Moreover, it lacks intelligence; its temperature threshold-based control method is ill-suited to complex and changing operating environments. Summary of the Invention

[0005] This application provides an air conditioner and its control method and computer program product to at least solve the problems of slow response speed, inability to achieve global energy saving and lack of intelligence in related technologies. It can quickly respond to changes in the load of electronic equipment, accurately regulate the cooling capacity of the air conditioner, optimize the overall energy consumption of the data center, and ensure that the electronic equipment is always in a suitable operating temperature range.

[0006] This application provides a method for controlling an air conditioner, including:

[0007] Obtain the current temperature and humidity of each area of ​​the data center, as well as the current load power of electronic devices;

[0008] The first temperature, the first humidity, and the first load power are used as inputs to the target evaluation network to output decision scores when performing multiple control operations on the air conditioner. Training samples are input to the evaluation network and the target network respectively to obtain a first decision score and a second decision score. When the difference between the first decision score and the second decision score reaches a preset minimum value, the evaluation network is updated to obtain the target evaluation network.

[0009] A random number generation function is determined based on a greedy strategy algorithm, and random numbers are generated based on the random number generation function.

[0010] If the random number is less than a preset probability threshold, a target index is randomly generated based on the random number generation function, and a corresponding control operation is selected from the plurality of control operations based on the target index as the control operation;

[0011] If the random number is greater than the preset probability threshold, the control operation corresponding to the highest decision score will be used as the control operation.

[0012] This application also provides a computer program product, including a computer program / instructions, which, when executed by a processor, implement the above-described air conditioner control method.

[0013] This application also provides an air conditioner, including a memory, a processor, and a program stored in the memory and executable on the processor. When the processor executes the program, it implements the above-described control method for the air conditioner.

[0014] This application obtains the first temperature and humidity of each area in a data center at the current moment, as well as the first load power of electronic devices. The first temperature, humidity, and load power are used as inputs to a target evaluation network to output decision scores for multiple control operations on the air conditioner. Training samples are input to both the evaluation network and the target network to obtain a first decision score and a second decision score. When the difference between the first and second decision scores reaches a preset minimum, the evaluation network is updated to obtain the target evaluation network. A greedy strategy algorithm is used to determine a random number generation function, which generates random numbers. If the random number is less than a preset probability threshold, a target index is randomly generated based on the random number generation function, and a corresponding control operation is selected from multiple control operations based on the target index. If the random number is greater than the preset probability threshold, the control operation corresponding to the highest decision score is selected as the control operation. Therefore, this method can quickly respond to changes in the load of electronic devices, accurately control the cooling capacity of air conditioners, optimize the overall energy consumption of the data center, and ensure that electronic devices are always within a suitable operating temperature range. Attached Figure Description

[0015] To more clearly illustrate the embodiments of this application, the accompanying drawings used in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0016] Figure 1 This is a flowchart of an air conditioner control method according to an embodiment of this application;

[0017] Figure 2 A flowchart illustrating a control method for an air conditioner according to a specific example of this application;

[0018] Figure 3 This is a block diagram of an air conditioner according to an embodiment of this application.

[0019] Reference numerals: 200-Air conditioner, 210-Memory, 220-Processor. Detailed Implementation

[0020] The embodiments of this application are described in detail below. Examples of these embodiments are shown in the accompanying drawings, wherein the same or similar reference numerals denote the same or similar elements or elements having the same or similar functions throughout. The embodiments described below with reference to the accompanying drawings are exemplary and intended to explain this application, and should not be construed as limiting this application.

[0021] The control method for an air conditioner, the air conditioner, and the computer program product proposed in this application are described below with reference to the accompanying drawings.

[0022] Figure 1 This is a flowchart of a control method for an air conditioner according to an embodiment of this application.

[0023] like Figure 1 As shown, the control method of the air conditioner in this application embodiment may include the following steps:

[0024] S1, obtain the first temperature and first humidity of each area of ​​the data center and the first load power of electronic devices at the current moment;

[0025] S2, the first temperature, the first humidity and the first load power are used as inputs to the target evaluation network to output decision scores when performing multiple control operations on the air conditioner. The training samples are input to the evaluation network and the target network respectively to obtain the first decision score and the second decision score. When the difference between the first decision score and the second decision score reaches a preset minimum value, the evaluation network is updated to obtain the target evaluation network.

[0026] S3, determine the random number generation function based on the greedy strategy algorithm, generate random numbers based on the random number generation function, and then proceed to steps S4 and S5.

[0027] S4. If the random number is less than the preset probability threshold, a target index is randomly generated based on the random number generation function, and a corresponding control operation is selected from multiple control operations based on the target index.

[0028] S5: If the random number is greater than the preset probability threshold, the control operation corresponding to the highest decision score will be used as the control operation.

[0029] Specifically, the first step is to obtain the current temperature and humidity levels in each area of ​​the data center, as well as the current load power of the electronic equipment. In other words, it's about collecting current operational status data for the data center, which will serve as the basis for regulating air conditioning. For example, temperature and humidity sensors are installed in key areas of the data center to monitor and record the ambient temperature and humidity in real time. Power monitoring devices, such as electricity meters or other energy consumption monitoring tools, are installed on the electronic equipment to track the energy consumption and overall load power of each device in real time. The initial load power of the electronic equipment represents the total power consumption of all servers, storage devices, network devices, and other related hardware. This can be a cumulative measurement representing the total energy consumption of all electronic equipment in the data center at a given moment. Wired or wireless sensor networks can be used to transmit data to a central monitoring system, and automated data acquisition systems can be used to collect and store this data periodically. This allows for the acquisition of accurate temperature readings in various areas of the data center, including server rooms, cooling zones, and storage areas, which is crucial for controlling the internal environment of the data center. It also allows for the acquisition of real-time load power data for electronic equipment, reflecting the current energy consumption and operational status of the equipment.

[0030] In addition, the collected raw data can be preprocessed to ensure data quality and usability, supporting subsequent analysis and decision-making. For example, outliers or erroneous readings, such as extreme temperature or humidity readings due to sensor malfunctions, can be removed. Temperature, humidity, and power data can be converted to uniform measures, such as degrees Celsius, percentages, and kilowatts, to facilitate comparison and analysis. Finally, data from different sensors can be integrated into a unified data structure, such as time series data or tables.

[0031] After acquiring the first temperature, first humidity, and first load power, these parameters can be used as inputs to the target evaluation network to output decision scores for multiple control operations on the air conditioner. In other words, these real-time data are input vectors into the evaluation network, which converts them into internal representations for understanding and processing. The evaluation network can assess the expected effects of all possible control actions (such as increasing or decreasing cooling capacity, adjusting fan speed, etc.) based on the input state data, and each action has a corresponding Q-value (decision score). The decision score represents the expected reward or value of performing that action in the current state.

[0032] For example, in a data center, the following data is acquired from the data center's sensors: a first temperature T1 = 21℃, a first humidity H1 = 45%, and a first load power P1 = 50kW. This data is provided as input vectors to a target evaluation network. The target evaluation network is a deep neural network trained to understand the relationship between these inputs and air conditioning control actions. It consists of multiple layers, each of which can learn different features of the input data. That is, after receiving the input data, the target evaluation network processes this data through its internal multi-layer neural network. The input layer receives the raw data, namely the values ​​of temperature, humidity, and load power. The hidden layers consist of multiple neurons, each of which can learn a specific feature of the input data. Each neuron performs a weighted sum with the input data through weights and biases, and then performs a nonlinear transformation through an activation function (such as ReLU (Rectified Linear Unit), sigmoid (Sigmoid Function), etc.). The activation function introduces nonlinearity, enabling the network to learn complex patterns. The hidden layers automatically extract useful features from the input data by learning weights and biases. For example, the network might learn that the combination of temperature and humidity has a significant impact on the Q-value of a certain action. The output layer has one neuron corresponding to each possible action, outputting the predicted Q-value for that action. The activation function of the output layer may be linear, directly outputting the Q-value. Data propagates forward from the input layer through the hidden layers, with each layer transforming and extracting features. After multiple layers of processing, the network finally outputs the Q-value for each action, representing the expected reward of performing each action in the current state. Assume the network outputs the following Q-values ​​(decision scores) for three possible control actions: Action A1 (increase cooling capacity): Q(A1) = 0.3, Action A2 (decrease cooling capacity): Q(A2) = 0.5, Action A3 (increase air supply temperature): Q(A3) = 0.4.

[0033] It should be noted that training samples can be input into the evaluation network and the target network respectively to obtain a first decision score and a second decision score. When the difference between the first decision score and the second decision score reaches a preset minimum value, the evaluation network can be updated to obtain the target evaluation network. In other words, during the operation of the data center, data such as temperature, humidity, and load power are collected in real time. This data can be stored as training samples in an experience replay pool. Each sample can include the current state, the action performed, the reward obtained, and the new state. When the number of samples in the experience replay pool reaches a certain threshold, a batch of samples can be randomly selected for training. The selected samples are input into the evaluation network, which predicts the Q-value (first decision score) of each possible action based on the current state. The same samples are input into the target network, which predicts the Q-value (second decision score) of each possible action based on the new state. The difference between the first decision score and the second decision score can be calculated. If the difference between the first decision score and the second decision score reaches a preset minimum value (or threshold), the parameters θ (such as the learning rate) of the evaluation network are updated to make its predictions closer to the output of the target network. In a dual-deep Q-network, the evaluation network is responsible for predicting the Q-value of each action based on the current state and selecting the action to execute. Its parameters are continuously updated during training to minimize prediction error. The target network provides a stable target Q-value (a second decision score) used to train the evaluation network. Its parameters are updated less frequently, typically copied from the evaluation network at regular time steps to stabilize the training process. By separating the evaluation and target networks, dual-deep Q-networks can reduce the overestimation of Q-values ​​and improve the stability and efficiency of learning. During training, the parameters of the evaluation network are continuously updated to reduce the discrepancy between its predictions and those of the target network, thereby improving the quality of decision-making.

[0034] Therefore, by comparing the outputs of the evaluation network and the target network, the evaluation network gradually learns how to better predict the expected reward of each action. When the evaluation network's prediction is close enough to the target network's prediction, the parameters of the evaluation network are updated to improve its performance. This process enables the evaluation network to more accurately predict the expected reward of performing different actions in different states, thus making better decisions. In this way, the evaluation network can learn to select the optimal action in different states to maximize cumulative reward. The introduction of the target network helps reduce the overestimation of Q-values ​​and improves the stability of learning, while the use of the experience replay pool helps to break temporal correlations and improve learning efficiency.

[0035] After determining multiple decision scores, control operations can be determined based on these scores to make the air conditioner operate accordingly. For example, a random number generation function can be determined based on a greedy strategy algorithm to generate a random number between 0 and 1. This random number is used to decide whether the agent should explore or exploit. The relationship between the random number and a preset probability threshold is judged. If the random number is less than the preset probability threshold, a target index is randomly generated based on the random number generation function, and the corresponding control operation is selected from multiple control operations based on the target index. That is, if the generated random number is less than the preset probability threshold ϵ, exploration is performed, and a target index is generated using the random number generation function. An action is selected as the control operation from multiple possible control operations based on the target index. If the random number is greater than the preset probability threshold, the control operation corresponding to the highest decision score is selected as the control operation. That is, if the generated random number is greater than or equal to the preset probability threshold ϵ, exploitation is performed, and the action with the highest decision score is selected as the control operation. Therefore, in this way, we can utilize the known best strategy (choosing the action with the highest Q value) in most cases, while retaining a certain probability of exploration (choosing random actions) to discover potentially better strategies. This approach helps the evaluation network and the target network to learn and adapt quickly when facing unknown or changing environments, while avoiding getting trapped in local optima.

[0036] Alternatively, multiple decision scores can be sorted in ascending order, and the control action corresponding to the decision score at the median can be selected. It should be noted that if the environment is relatively stable and the evaluation network is already quite accurate, the action with the highest score can be chosen. This method is simple and direct, selecting the action with the highest expected reward, and is easy to understand and implement, helping the agent quickly learn a strategy that maximizes long-term cumulative rewards. If the environment is dynamically changing or the evaluation network is still being trained, choosing the action with the median score may be more appropriate, avoiding excessively high or low scores, and helping to reduce extreme or unstable behaviors.

[0037] This allows air conditioners to operate based on control actions. For example, in the example above, assume the evaluation network outputs the following Q values ​​(decision scores) for three possible control actions: Action A1 (increase cooling capacity): Q(A1) = 0.3, Action A2 (decrease cooling capacity): Q(A2) = 0.5, Action A3 (increase supply air temperature): Q(A3) = 0.4. The control action with the highest Q value can be selected, i.e., increasing the supply air temperature, to control the air conditioner to operate according to the action of increasing the supply air temperature. By executing the selected control action, the data center's air conditioning system will adjust its operating state accordingly, such as changing the cooling capacity, air volume, or air temperature, in order to achieve better temperature and humidity control, while also considering energy efficiency.

[0038] Furthermore, to maintain the stability of the data center's internal environment and protect electronic equipment from temperature fluctuations, the cooling capacity adjustment step size is set to 1%-5% of the system's rated cooling capacity, depending on the data center's size and requirements. For example, for a system with a rated cooling capacity of 1000kW, the adjustment step size can be set from 10kW to 50kW. When cooling capacity needs adjustment, the air conditioning control system will increase or decrease the cooling output in steps of 10kW to 50kW based on current load demands and environmental conditions until the required cooling capacity is achieved. The air conditioning control system will also monitor the supply air temperature and make fine adjustments as needed, ensuring that temperature changes do not exceed an accuracy range of ±0.5℃. This helps maintain the stability of the data center's internal environment and prevents temperature fluctuations from adversely affecting electronic equipment. Therefore, by optimizing the control of cooling capacity and supply air temperature, the energy efficiency of the air conditioning system can be improved, energy consumption reduced, and the stability of the data center's internal environment helped protect electronic equipment from temperature fluctuations.

[0039] Therefore, an adaptive air conditioner control method was implemented using a dual-depth Q-network algorithm. This method can make rapid decisions based on real-time data and respond to environmental changes. Through continuous learning and optimization, the evaluation network gradually improves its predictive accuracy, making data center air conditioning control more efficient and energy-saving. Simultaneously, the stability of the learning process is maintained by periodically updating the parameters of the evaluation network to the target network. This method reduces the need for manual intervention and improves the system's automation and intelligence. It can quickly respond to changes in electronic equipment load, precisely control the air conditioner's cooling capacity, optimize the overall energy consumption of the data center, and ensure that electronic equipment is always operating within a suitable temperature range.

[0040] According to one embodiment of this application, after the air conditioner is operated based on the control operation, the control method of the air conditioner further includes: obtaining the reward value corresponding to the operation of the air conditioner based on the control operation, and obtaining the second temperature and second humidity of each area of ​​the data center and the second load power of the electronic device after the air conditioner is operated based on the control operation; storing the reward value, second temperature, second humidity and second load power into the training sample, wherein the training sample is used to train the evaluation network.

[0041] Specifically, based on the action selected by the evaluation network, the air conditioner performs corresponding control operations, such as adjusting the cooling capacity, air volume, or air temperature. After the control operation is performed, the environmental state of the data center changes, requiring remeasurement of the temperature, humidity, and load power of electronic equipment in each area of ​​the data center. These data reflect the effectiveness of the control operation. A reward value can be calculated based on the new state and preset objectives (such as maintaining a specific temperature range, minimizing energy consumption, etc.). The reward value reflects the quality of the action; a positive reward indicates that the action helps achieve the objective, while a negative reward indicates that the action has a negative impact on the objective. Therefore, the reward value, second temperature, second humidity, and second load power can be stored as training samples. Here, the reward value represents a quantitative indicator of the action's effectiveness, the second temperature represents the new temperature in each area of ​​the data center after the control operation, the second humidity represents the new humidity in each area of ​​the data center after the control operation, and the second load power represents the load power of electronic equipment after the control operation. These training samples will be used to subsequently train the evaluation network to optimize its decision-making strategy, thereby enabling the trained evaluation network to serve as the target evaluation network to determine the optimal control operation for the air conditioner.

[0042] Therefore, by collecting feedback information after actions are executed, the evaluation network can continuously learn and adjust its strategies, thereby achieving intelligent control of data center air conditioning systems and improving energy efficiency and reliability. This method enables agents to make decisions based on real-time data and continuously optimize their behavior according to changes in the environment.

[0043] According to one embodiment of this application, the training samples further include a first temperature, a first humidity, a first load power, and the current control operation. The training samples are input into an evaluation network and a target network, respectively, to obtain a first decision score and a second decision score. This includes: using the first temperature, first humidity, first load power, and the current control operation as input to the evaluation network to obtain a first decision score; using the second temperature, second humidity, and second load power as input to the evaluation network to output a decision score when multiple control operations are performed on the air conditioner; determining the updated control operation corresponding to the highest decision score; and using the second temperature, second humidity, second load power, and the updated control operation as input to the target network to obtain a second decision score.

[0044] Specifically, the training samples include not only the new state (second temperature, second humidity, second load power) and reward value after the action is executed, but also the state (first temperature, first humidity, first load power) and the current control operation before the action is executed. This information is used to train the evaluation network and the target network to optimize the decision-making process. The state (first temperature, first humidity, first load power) and the executed action before the control operation are collected. After the action is executed, the new state (second temperature, second humidity, second load power) and the corresponding reward value are collected. This data is stored as training samples in the experience replay pool.

[0045] The evaluation network takes a first temperature, a first humidity level, a first load power, and the current control operation as input to obtain a first decision score (first Q-value). For example, these data are provided as an input vector to the evaluation network. This data reflects the current operating state of the data center and the actions the agent plans to perform. The input data enters the evaluation network through its input layer, initiating the forward propagation process. The data passes through multiple hidden layers, each containing multiple neurons. These neurons are weighted and summed with the input data using weights and biases, and then nonlinearly transformed using an activation function. The hidden layers learn features of the input data that are crucial for predicting the Q-value. For example, the network might learn that the combination of temperature and humidity significantly impacts the Q-value of a particular action. After multiple layers of processing, the data reaches the output layer, which typically has one neuron corresponding to each possible action. The output layer generates a Q-value for each action, representing the expected reward of performing each action in the current state. The appropriate action is then selected from the Q-values ​​output by the evaluation network. This action can be randomly selected or correspond to the highest Q-value.

[0046] Similarly, the second temperature, second humidity, and second load power are used as inputs to the evaluation network to output decision scores for multiple control operations on the air conditioner; and the updated control operation corresponding to the highest decision score is determined. These real-time collected parameters are used as inputs to the evaluation network. The evaluation network's role is to evaluate the Q-value (i.e., decision score) of each possible control operation (such as increasing cooling capacity, decreasing airflow, etc.) based on the current state. A higher Q-value indicates that the operation is more likely to bring good results in the current state (such as reducing energy consumption, maintaining temperature stability, etc.). That is, the control operation with the highest Q-value output from the evaluation network is selected; this operation is considered the most likely to bring the best effect in the current state and is called the updated control operation. The second temperature, second humidity, second load power, and the updated control operation are then used as inputs to the target network to obtain the second decision score. The target network provides a stable benchmark for calculating the target Q-value. Therefore, the target network outputs a target Q-value (second decision score) based on the input state and the updated control operation. This target Q-value is used to evaluate the actual effect of the updated control operation and serves as a reference for training the evaluation network.

[0047] Therefore, by using real-time collected data as input, the network output decision score is evaluated, the optimal control operation is selected, and the result is verified and optimized through the target network. This method not only enables rapid response to changes in the load of electronic devices but also achieves global energy consumption optimization, improves control accuracy and reliability, and demonstrates significant energy-saving effects and operational stability.

[0048] According to one embodiment of this application, the control method for an air conditioner further includes: determining a target loss function based on a first decision score and a second decision score; and minimizing the target loss function based on a gradient descent algorithm so that the difference between the first decision score and the second decision score reaches a preset minimum value.

[0049] Specifically, determining and minimizing the target loss function is a crucial step in the training process. The target loss function can be determined based on the first decision score and the second decision score. The first decision score, derived from the Q-value output by the evaluation network based on the current state, represents the expected utility of the evaluation network for each possible regulatory operation. The second decision score, derived from the target network's target Q-value output based on the current state and the updated regulatory operation, represents an estimate of the actual utility of the target network for the updated regulatory operation. The target loss function can be determined based on the difference between the Q-value output by the evaluation network and the target Q-value output by the target network.

[0050] After determining the target loss function, it can be minimized using the gradient descent algorithm to minimize the difference between the first and second decision scores. Gradient descent is an optimization algorithm used to minimize the loss function L(θ). Its basic idea is to calculate the gradient of the loss function with respect to the parameter θ and update the parameters in the opposite direction of the gradient to gradually reduce the value of the loss function. In each iteration, a batch of samples is randomly selected from the experience replay pool, and the loss function L(θ) for these samples is calculated. The gradient of the loss function with respect to the parameter θ is also calculated, and the parameters θ of the evaluation network are updated according to the gradient descent update rule. During training, the goal is to minimize the difference between the first and second decision scores, i.e., to minimize the value of the loss function L(θ). A preset minimum value is usually set; when the value of the loss function falls below this threshold, the training is considered to have converged, and the algorithm has learned a relatively accurate control strategy.

[0051] Therefore, by minimizing the objective loss function, the evaluation network can learn a more accurate Q-value, thereby making better control decisions in actual operation. This helps improve the energy efficiency and operational stability of data center air conditioning systems, enabling rapid response to changes in electronic equipment load, precise control of the cooling capacity and airflow of the air conditioning system, saving energy consumption, and extending equipment lifespan.

[0052] According to one embodiment of this application, determining a target loss function based on a first decision score and a second decision score includes: using the product of a discount factor and a second decision score plus a reward value as the target decision score; and determining the target loss function based on the difference between the first decision score and the target decision score.

[0053] Specifically, when determining the target loss function based on the first and second decision scores, the sum of the product of the discount factor and the second decision score and the reward value can be used as the target decision score, and the target loss function can be determined based on the difference between the first and target decision scores. For example, the formula can be used... Determine the target loss function. Wherein, Let be the target loss function. It is to evaluate the network in state Next action The output is the first decision score. It's a reward value. It is a discount factor (a value between 0 and 1 used to balance the importance of immediate rewards and future rewards). This is the maximum Q-value among all possible actions a' of the target network in the next state s′. It should be noted that in the above formula, state s represents the first temperature, first humidity, and first load power of the electronic equipment in each area of ​​the current data center. Action a refers to the action chosen by the network in the current state s, such as increasing cooling capacity, decreasing cooling capacity, increasing airflow, decreasing airflow, adjusting airflow temperature, etc. State s′ represents the adjusted temperature, humidity, and load power of the air conditioner, and a′ is the action corresponding to the maximum Q-value among all possible actions considered by the target network in the new state s′, such as increasing cooling capacity, decreasing cooling capacity, increasing airflow, decreasing airflow, adjusting airflow temperature, etc.

[0054] Therefore, by introducing a target loss function, the stability and efficiency of the algorithm are improved, enabling it to better handle high-dimensional state and action spaces, as well as complex environmental dynamics. In practical applications, such as data center air conditioning control, this can lead to better control strategies and improved performance.

[0055] According to one embodiment of this application, obtaining the reward value corresponding to the operation of an air conditioner based on a control operation includes: obtaining the temperature uniformity index of a data center, the temperature stability index of an electronic device, and the energy efficiency index of an air conditioner; and determining the reward value based on the sum of the product of a first weighting coefficient and the temperature uniformity index, the product of a second weighting coefficient and the temperature stability index, and the product of a third weighting coefficient and the energy efficiency index.

[0056] Specifically, the reward value is the feedback obtained by the agent (in this case, the air conditioning control system) from the environment, used to evaluate the quality of the actions performed. The design of the reward value is crucial to the training process and the performance of the final policy. When obtaining the reward value corresponding to the operation of the air conditioner based on the control operation, the following metrics can be obtained: temperature uniformity index of the data center, temperature stability index of electronic equipment, and energy efficiency index of the air conditioner. The temperature uniformity index measures the consistency of temperature in different areas within the data center. High temperature uniformity means small temperature differences between areas within the data center, helping to protect electronic equipment from localized overheating. The temperature stability index of electronic equipment assesses the temperature fluctuations in the operating environment of electronic equipment. High temperature stability means small temperature changes around the electronic equipment, helping to improve the operating efficiency and lifespan of the equipment. The energy efficiency index of the air conditioner measures the energy efficiency of the air conditioner in providing cooling effects. High energy efficiency means consuming less energy to achieve the same cooling effect.

[0057] Since different performance metrics can have varying degrees of impact on data center operating efficiency and costs, a weighting coefficient needs to be assigned to each metric. The first weighting coefficient is assigned to the temperature evenness metric, reflecting its importance in overall performance; the second weighting coefficient is assigned to the temperature stability metric, reflecting its importance; and the third weighting coefficient is assigned to the energy efficiency metric, reflecting its importance. Therefore, the reward value can be determined based on the sum of the products of the first weighting coefficient and the temperature evenness metric, the second weighting coefficient and the temperature stability metric, and the third weighting coefficient and the energy efficiency metric.

[0058] Therefore, by comprehensively considering temperature uniformity, temperature stability, and energy efficiency, the reward value can more comprehensively reflect the effect of air conditioning control operations. A reasonable reward value design can guide the air conditioning control system to make better decisions, thereby improving the overall performance of the data center. Furthermore, the weighting coefficients can be adjusted according to actual conditions to adapt to different operational needs and optimization objectives. In this way, the reward value not only reflects the direct effect of air conditioning control operations but also considers its impact on the long-term operating efficiency and cost of the data center, thus helping to train more efficient and reliable air conditioning control strategies.

[0059] According to one embodiment of this application, obtaining a temperature uniformity index for a data center includes: calculating an average temperature based on the actual temperature of each area of ​​the data center; and determining a temperature uniformity index based on the actual temperature of each area of ​​the data center and its corresponding weighting coefficient and average temperature.

[0060] Specifically, when obtaining the temperature uniformity index of a data center, the average temperature can be calculated based on the actual temperature of each area of ​​the data center. First, actual temperature data for each area of ​​the data center needs to be collected, which can be obtained through temperature sensors deployed in different locations within the data center. Then, the average of these temperature data is calculated. That is, the average temperature equals the sum of the actual temperatures of each area divided by the total number of data area centers. The temperature uniformity index is usually calculated based on the deviation between the actual temperature and the average temperature of each area. To more accurately reflect the importance of different areas, weighting coefficients can be introduced. These weighting coefficients can be determined based on factors such as the importance of the area, equipment density, or heat load. For example, the temperature uniformity index is calculated using the formula... Confirmed. Among them, It is the weight coefficient of the i-th region. It is the actual temperature of the i-th region. It is the average temperature across all areas of the data center.

[0061] Therefore, by ensuring uniform temperature distribution within the data center, unnecessary energy waste can be avoided. For example, there is no need to overcool certain areas. Temperature balance helps prevent electronic equipment from being damaged by localized overheating, thereby improving equipment reliability and lifespan. Furthermore, by adjusting the weighting coefficients, the specific needs of different areas within the data center can be flexibly adapted.

[0062] According to one embodiment of this application, obtaining the temperature stability index of an electronic device includes: calculating the maximum and minimum temperature values ​​of the electronic device within a preset time period; determining the temperature fluctuation difference based on the maximum and minimum values; and determining the temperature stability index based on the temperature fluctuation difference and a preset mapping relationship, wherein the preset mapping relationship is used to indicate the relationship between the temperature fluctuation difference and the temperature stability index.

[0063] Specifically, the temperature stability of electronic devices is a key factor in ensuring their normal operation and extending their lifespan. Temperature stability indicators can help assess the temperature variations in the operating environment of the equipment, thus providing a basis for air conditioning control strategies. When obtaining temperature stability indicators for electronic devices, firstly, temperature data needs to be collected over a preset time period. This data can be obtained through the device's built-in temperature sensor or externally deployed temperature monitoring equipment. Then, the maximum and minimum temperature values ​​within this time period are calculated. Based on these maximum and minimum values, the temperature fluctuation difference is determined; this difference reflects the magnitude of temperature change within the preset time period.

[0064] After determining the temperature fluctuation difference, a temperature stability index can be determined based on the temperature fluctuation difference and a preset mapping relationship. In other words, the temperature stability index is determined based on the temperature fluctuation difference and the preset mapping relationship. The preset mapping relationship is a function or rule used to convert the temperature fluctuation difference into a temperature stability index. For example, by pre-determining the relationship between the temperature fluctuation difference and the temperature stability index, the temperature stability index can be obtained by directly calling the preset mapping relationship after the temperature fluctuation difference is determined. Furthermore, this mapping relationship can be designed according to actual needs and equipment characteristics, and can include linear mappings, nonlinear mappings, etc. For example, a linear mapping relationship could be: Temperature stability index = 1 - α * ΔT / preset threshold. Here, α is a coefficient used to adjust the sensitivity of the mapping, ΔT is the temperature fluctuation difference, and the preset threshold is an upper limit for temperature fluctuation set according to equipment characteristics and operating requirements.

[0065] Therefore, by monitoring temperature stability, temperature fluctuations that may lead to equipment failure can be detected and resolved in a timely manner. Temperature stability indicators can serve as an important tool for evaluating and optimizing data center environmental control, helping to ensure that electronic equipment operates stably under suitable temperature conditions.

[0066] According to one embodiment of this application, obtaining the energy efficiency index of an air conditioner includes: determining an energy efficiency ratio (EER) based on the ratio of the air conditioner's cooling capacity to its power consumption; if the EER is greater than a preset EER threshold, calculating a first EER difference between the EER and the preset EER threshold, and determining the energy efficiency index based on the product of the first EER difference and a positive bonus value; if the EER is less than the preset EER threshold, calculating a second EER difference between the EER and the preset EER threshold, and determining the energy efficiency index based on the product of the second EER difference and a negative bonus value. The preset EER threshold can be determined according to actual conditions.

[0067] Specifically, when obtaining the energy efficiency index of an air conditioner, the energy efficiency ratio (EER) can be determined based on the ratio of the air conditioner's cooling capacity to its power consumption. Cooling capacity is the cooling energy provided by the air conditioner per unit time, and power consumption is the electrical energy consumed by the air conditioner while providing cooling capacity. A preset EER threshold is a benchmark value used to assess whether the air conditioner's energy efficiency meets the expected standard. Based on the comparison between the actual EER and the preset EER threshold, the calculation method for the energy efficiency index can be determined.

[0068] When the energy efficiency ratio (EER) is greater than the preset EER threshold, the difference between the actual EER and the preset EER threshold is calculated, i.e., the first EER difference = EER - preset EER threshold. The energy efficiency index is then determined based on the product of the first EER difference and a positive bonus value, i.e., energy efficiency index = first EER difference × positive bonus value. In this case, the energy efficiency index is positive, indicating that the air conditioner's energy efficiency is better than the preset standard, and a positive bonus should be given. When the EER is less than the preset EER threshold, the difference between the preset EER threshold and the actual EER is calculated, i.e., the second EER difference = preset EER threshold - EER. The energy efficiency index is then determined based on the product of the second EER difference and a negative bonus value, i.e., energy efficiency index = second EER difference × negative bonus value. In this case, the energy efficiency index is negative, indicating that the air conditioner's energy efficiency is lower than the preset standard, and a negative penalty should be given. The preset EER threshold can be 3, the first EER difference can be 10, and the second EER difference can be -5.

[0069] Therefore, by rewarding high-efficiency operation and penalizing low-efficiency operation, we can incentivize the optimization of air conditioner operating strategies, thereby improving overall energy efficiency. In this way, energy efficiency indicators can serve as an important tool for evaluating and optimizing air conditioner operating efficiency, contributing to energy conservation and improved operational quality.

[0070] According to one embodiment of this application, the air conditioner control method further includes: monitoring the status of the air conditioner; and adjusting the training parameters of the evaluation network in the event of an air conditioner malfunction, wherein the air conditioner malfunction includes at least one of the following: in each area of ​​the data center, the temperature of at least one area is greater than a first preset temperature threshold for a first preset duration, or the temperature of at least one area is less than a second preset temperature threshold for a second preset duration; the suction pressure and discharge pressure of the air conditioner compressor are not within a preset pressure range; or the flow rates of chilled water and cooling water are not within a preset flow rate range. Adjusting the training parameters of the evaluation network includes reducing at least one of the following: experience replay pool capacity, batch training size, learning rate, and target network update frequency. The first preset duration, second preset duration, first preset temperature threshold, second preset temperature threshold, preset pressure range, and preset flow rate range can be determined according to actual conditions.

[0071] Specifically, when controlling the air conditioner, its status can also be monitored. Real-time operating data, including but not limited to temperature, humidity, pressure, current, and voltage, is collected through sensors and monitoring systems. This data is analyzed to assess the health and performance of the air conditioner. When an anomaly is detected, measures need to be taken to adjust the training parameters of the evaluation network to adapt to the current operating conditions and prevent further deterioration of system performance. For example, if the temperature in at least one area of ​​the data center exceeds a first preset temperature threshold, and this condition persists for a first preset duration (e.g., 30 minutes), a high-temperature anomaly is considered to have occurred. If the temperature in at least one area is below a second preset temperature threshold, and this condition persists for a second preset duration (e.g., 20 minutes), a low-temperature anomaly is considered to have occurred. Furthermore, the suction and discharge pressures of the air conditioner compressor are monitored; if these pressures are outside the preset pressure range, a pressure anomaly is considered to have occurred. The flow rates of chilled water and cooling water are monitored; if the flow rates are outside the preset flow rate range, a flow rate anomaly is considered to have occurred. Therefore, when an anomaly is detected in the air conditioner, the training parameters of the evaluation network need to be adjusted to adapt to the new operating conditions and optimize the control strategy.

[0072] Training parameters can include adjusting at least one of the following: experience replay pool size, batch size, learning rate, and target network update frequency. Adjusting the experience replay pool size (reducing its capacity) decreases the number of samples used in training, helping the network adapt to new operating conditions more quickly. Reducing the batch size increases the update frequency during training, further helping the network adapt to changes faster. Lowering the learning rate smooths out parameter updates, reducing instability caused by excessively large parameter updates. Reducing the update frequency decreases fluctuations during training, improving learning stability. Furthermore, the extent of adjustment can be determined based on the severity of the air conditioner anomaly. For example, severe temperature anomalies may require more significant adjustments to the training parameters.

[0073] Therefore, by adjusting the training parameters of the evaluation network when the air conditioner malfunctions, the robustness of the system can be improved, enabling it to better cope with uncertainties and changes during operation. Timely adjustments can prevent the air conditioner from continuing to operate under abnormal conditions, thereby protecting the equipment from damage. By adjusting the training parameters, the performance of the air conditioner under different operating conditions can be optimized, improving energy efficiency and reliability.

[0074] According to one embodiment of this application, the air conditioner control method further includes: obtaining the difference between the control parameters determined by the evaluation network during the control operation and the control parameters of the feedback controller when controlling the air conditioner; and controlling the air conditioner to operate based on the control parameters of the feedback controller when controlling the air conditioner if the difference exceeds a preset difference threshold and continues for a preset time. The preset difference threshold and preset time can be determined according to actual conditions.

[0075] Specifically, the evaluation network determines the control parameters for the control operation, and the feedback controller determines the control parameters for the air conditioner. The evaluation network determines a set of control operation parameters based on current environmental conditions (such as temperature and humidity) and historical data. These parameters may include cooling capacity, air volume, and air supply temperature. The feedback controller (such as a PID controller) calculates another set of control operation parameters based on real-time sensor data and preset control logic. The difference between the two sets of control parameters can be an absolute difference or a norm (such as Euclidean distance).

[0076] A difference threshold is set to determine if the difference between two sets of control parameters is large enough to require action, and a time threshold is set to determine if the difference exceeding the threshold persists for a sufficiently long time. If the difference exceeds the preset difference threshold and this state persists for a preset time, the output of the evaluation network is considered potentially unstable or unreliable. When the conditions are met, the control system switches to operating the air conditioner based on the control parameters of the feedback controller. This is a safety mechanism to prevent potential errors in the evaluation network from adversely affecting the operation of the air conditioner. Furthermore, a smooth transition is required during the switch to avoid impacting the air conditioner's operation; this can be achieved by gradually adjusting the control parameters.

[0077] Therefore, by switching to the feedback controller when the evaluation network output is unreliable, the robustness of the system can be improved. When the evaluation network output may damage the air conditioner, switching to the feedback controller can enhance the system's safety. When the evaluation network output is stable and reliable, it can be used for more precise control to optimize performance. In this way, the air conditioner's control method can be more flexible and reliable, enabling intelligent switching between the evaluation network and the feedback controller to adapt to different operating conditions, thereby improving overall operating efficiency and reliability.

[0078] According to one embodiment of this application, the control method for an air conditioner further includes: acquiring the compressor suction pressure and discharge pressure of the air conditioner, and acquiring the temperature difference between cooling water and chilled water in the cooling circuit of the air conditioner; using the suction pressure, discharge pressure, temperature difference, first temperature, first humidity, and first load power as inputs to an evaluation network to output a decision score when performing multiple control operations on the air conditioner.

[0079] Specifically, to enable the evaluation network to more accurately predict the effects of different control operations, the compressor's suction and discharge pressures, as well as the temperature difference between the cooling water and chilled water in the air conditioner's cooling circuit, can be obtained. The compressor suction and discharge pressures reflect the compressor's operating status. Excessively low suction pressure or excessively high discharge pressure may indicate a compressor problem or the need to adjust operating parameters to improve efficiency. The temperature difference between the cooling water and chilled water reflects the heat exchange efficiency of the cooling circuit. Ideally, the temperature difference between the cooling water and chilled water should be large enough to ensure effective heat exchange.

[0080] After obtaining the compressor's suction and discharge pressures and the temperature difference between the cooling water and chilled water in the air conditioner's cooling circuit, the suction pressure, discharge pressure, temperature difference, first temperature, first humidity, and first load power can be used as inputs to an evaluation network to output decision scores for multiple control operations on the air conditioner. That is, the evaluation network outputs decision scores for multiple control operations on the air conditioner based on the input parameters. These scores reflect the expected effect of each possible control operation (such as adjusting the compressor's operating frequency, changing the airflow, or adjusting the cooling water flow) under the current condition. A higher score indicates that the control operation is more likely to bring good results under the current condition, such as improving energy efficiency or maintaining temperature stability. The specific method for determining the decision scores is the same as the method described above, where the first temperature, first humidity, and first load power are used as inputs to the target evaluation network to output decision scores for multiple control operations on the air conditioner, and will not be repeated here.

[0081] Therefore, by considering the compressor's operating status and the heat exchange efficiency of the cooling circuit, the air conditioner can be more precisely controlled to adapt to the current operating conditions. Precise control can improve the air conditioner's energy efficiency and reduce energy waste. Thus, the control strategy can be automatically adjusted according to real-time operating parameters and environmental conditions to achieve the best operating effect.

[0082] According to one embodiment of this application, the ratio of the power difference of the first load power at adjacent time points to the time interval between adjacent time points is obtained as the heat load change rate; when the heat load change rate is greater than zero, the air conditioner is pre-controlled based on a first strategy, wherein the first strategy includes at least one of increasing the cooling capacity, increasing the air volume, and decreasing the air supply temperature; when the heat load change rate is less than zero, the air conditioner is pre-controlled based on a second strategy, wherein controlling the air conditioner includes at least one of decreasing the cooling capacity, decreasing the air volume, and increasing the air supply temperature.

[0083] Specifically, the heat load change rate parameter of electronic equipment is a key predictive indicator. It helps us understand the heat load change trend of electronic equipment in the data center, allowing us to predict heat load trends based on the heat load change rate parameter and adjust the air conditioning cooling strategy in advance. The heat load change rate is obtained by calculating the difference in load power between two adjacent time points t and t-1 and dividing it by the time interval Δt. After determining the heat load change rate, it can be assessed. If the heat load change rate is greater than zero, it indicates that an increase in heat load is predicted. Cooling capacity can be increased in advance to cope with the upcoming heat load peak, and airflow and air temperature can be adjusted. Increasing the airflow and decreasing the air temperature allows for more effective distribution of cool air and improves cooling efficiency.

[0084] When the rate of change of heat load is less than zero, indicating that the heat load is predicted to decrease, appropriate control and adjustments are still necessary to avoid energy waste or other potential problems caused by over-cooling. For example, the cooling capacity of the air conditioning system can be reduced according to the degree of heat load reduction to match the current heat load demand. Furthermore, the air supply volume and temperature can be adjusted, such as reducing the air supply volume and increasing the air supply temperature, to utilize the cooling air more effectively, improve cooling efficiency, and avoid over-cooling.

[0085] Therefore, appropriate control and adjustment can be made according to the increase or decrease of heat load to ensure the stable operation and energy efficiency of the data center.

[0086] In one embodiment of the application, the rate of change of heat load in electronic devices can be monitored to more accurately predict future heat load trends, allowing for advance adjustment of the air conditioner's cooling strategy. For example, real-time operating data of electronic devices, including CPU (Central Processing Unit) utilization, memory usage, and network traffic, can be collected and analyzed to calculate the rate of change of heat load. A combined LSTM (Long Short-Term Memory) and Kalman filter model is employed, combining the advantages of both to improve the accuracy and real-time performance of heat load prediction. The LSTM network structure consists of three hidden layers, each with 128 neurons, a structure that helps capture long-term dependencies in time-series data. The model's input consists of load data from the past 24 hours and current traffic flow, providing sufficient historical information and current state for the LSTM network to learn patterns in heat load changes. The process noise covariance is set to 0.01 to represent the uncertainty of the system model, and the measurement noise covariance is set to 0.05 to represent the uncertainty of the measurement data. The Kalman filter uses these parameters to correct the prediction error of the LSTM network in real time, thereby improving the accuracy of the prediction.

[0087] Therefore, real-time operating data of electronic devices is collected, including heat load, CPU utilization, memory usage, and network traffic. The rate of change of heat load is calculated and used as one of the features. This data, along with the load data from the past 24 hours and current business traffic, is input into the LSTM network. The LSTM network predicts future heat load based on the input data. The prediction results are then fed into a Kalman filter, which uses process noise covariance and measurement noise covariance to correct prediction errors in real time, resulting in a more accurate heat load prediction. Based on the corrected heat load prediction, cooling strategies can be adjusted in advance, such as adjusting compressor operating frequency and changing air volume, to ensure that the temperature of the data center is controlled within a suitable range.

[0088] Therefore, combining LSTM and Kalman filters can improve the accuracy of heat load prediction, thereby managing the temperature of data centers more effectively. By predicting heat load in advance and adjusting cooling strategies, energy efficiency can be improved and energy consumption reduced. In this way, the air conditioning system of the data center can operate more intelligently and efficiently, providing a stable and reliable operating environment for electronic equipment.

[0089] The following is combined Figure 2 The method described in this application is used to describe the method.

[0090] As a specific example, the control method for the air conditioner of this application may include the following steps:

[0091] S101, obtain the first temperature and first humidity of each area of ​​the data center and the first load power of electronic devices at the current moment.

[0092] S102, taking the first temperature, first humidity and first load power as inputs to the target evaluation network, and outputting a decision score when performing multiple control operations on the air conditioner.

[0093] S103, determine the random number generation function based on the greedy strategy algorithm, and generate random numbers based on the random number generation function.

[0094] S104. Determine whether the random number is less than a preset probability threshold. If yes, proceed to step S105; otherwise, proceed to step S113.

[0095] S105, randomly generate a target index based on a random number generation function, and select the corresponding control operation from multiple control operations based on the target index, so that the air conditioner operates based on the control operation.

[0096] S106, obtain the reward value corresponding to the operation of the air conditioner based on the control operation, obtain the second temperature and second humidity of each area of ​​the data center and the second load power of each electronic device after the air conditioner is operated based on the control operation, and store the reward value, second temperature, second humidity and second load power into the training sample.

[0097] S107, determine whether the number of training samples meets the preset threshold. If yes, proceed to step S108; if no, proceed to step S101.

[0098] S108, the training samples are input into the evaluation network and the target network respectively to obtain the first decision score and the second decision score.

[0099] S109, the product of the discount factor and the second decision score and the sum of the reward value are used as the target decision score, and the target loss function is determined based on the difference between the first decision score and the target decision score.

[0100] S110, the product of the discount factor and the second decision score and the sum of the reward value are used as the target decision score, and the target loss function is determined based on the difference between the first decision score and the target decision score.

[0101] S111 minimizes the target loss function based on the gradient descent algorithm so that the difference between the first decision score and the second decision score reaches a preset minimum value.

[0102] S112, update the evaluation network to obtain the target evaluation network, so that the control operation is determined based on the target evaluation network when the air conditioner is controlled next time, and copy the parameters of the updated evaluation network to the target network.

[0103] S113, take the control operation corresponding to the highest decision score as the control operation, so that the air conditioner operates based on the control operation, and proceed to step S106.

[0104] In summary, the air conditioner control method according to the embodiments of this application obtains the first temperature and first humidity of each area of ​​the data center and the first load power of the electronic equipment at the current moment; uses the first temperature, first humidity, and first load power as inputs to a target evaluation network to output decision scores when performing multiple control operations on the air conditioner. Training samples are input to the evaluation network and the target network respectively to obtain a first decision score and a second decision score. When the difference between the first decision score and the second decision score reaches a preset minimum value, the evaluation network is updated to obtain the target evaluation network. A random number generation function is determined based on a greedy strategy algorithm, and random numbers are generated based on the random number generation function. If the random number is less than a preset probability threshold, a target index is randomly generated based on the random number generation function, and a corresponding control operation is selected from multiple control operations based on the target index. If the random number is greater than the preset probability threshold, the control operation corresponding to the highest decision score is selected as the control operation. Therefore, this method can quickly respond to changes in the load of electronic equipment, accurately control the cooling capacity of the air conditioner, optimize the overall energy consumption of the data center, and ensure that the electronic equipment is always within a suitable operating temperature range.

[0105] Corresponding to the above embodiments, this application also proposes a computer program product.

[0106] The computer program product of this application includes a computer program / instruction, which, when executed by a processor, implements the above-described air conditioner control method.

[0107] The computer program product according to the embodiments of this application can quickly respond to changes in the load of electronic equipment, accurately regulate the cooling capacity of the air conditioner, optimize the overall energy consumption of the data center, and ensure that the electronic equipment is always in a suitable operating temperature range by executing the above-described air conditioner control method.

[0108] Corresponding to the above embodiments, this application also proposes an air conditioner.

[0109] like Figure 3 As shown, the air conditioner 200 in this embodiment may include: a memory 210, a processor 220, and a program stored in the memory 210 and executable on the processor 220. When the processor 220 executes the program, it implements the above-described air conditioner control method.

[0110] According to the embodiments of this application, the air conditioner, by executing the above-described air conditioner control method, can quickly respond to changes in the load of electronic equipment, accurately regulate the cooling capacity of the air conditioner, optimize the overall energy consumption of the data center, and ensure that the electronic equipment is always within a suitable operating temperature range.

[0111] It should be noted that the logic and / or steps represented in the flowchart or otherwise described herein, for example, can be considered as a sequenced list of executable instructions for implementing logical functions, and can be specifically implemented in any computer-readable medium for use by, or in conjunction with, an instruction execution system, apparatus, or device (such as a computer-based system, a processor-included system, or other system that can fetch and execute instructions from, an instruction execution system, apparatus, or device). For the purposes of this specification, "computer-readable medium" can be any means that can contain, store, communicate, propagate, or transmit programs for use by, or in conjunction with, an instruction execution system, apparatus, or device. More specific examples (a non-exhaustive list) of computer-readable media include: an electrical connection having one or more wires (electronic device), a portable computer disk drive (magnetic device), random access memory (RAM), read-only memory (ROM), erasable and editable read-only memory (EPROM or flash memory), fiber optic devices, and portable optical disc read-only memory (CDROM). Alternatively, the computer-readable medium may be paper or other suitable media on which the program can be printed, since the program can be obtained electronically, for example, by optically scanning the paper or other medium, followed by editing, interpreting, or otherwise processing as necessary, and then stored in a computer memory.

[0112] It should be understood that various parts of this application can be implemented using hardware, software, firmware, or a combination thereof. In the above embodiments, multiple steps or methods can be implemented using software or firmware stored in memory and executed by a suitable instruction execution system. For example, if implemented in hardware, as in another embodiment, it can be implemented using any one or a combination of the following techniques known in the art: discrete logic circuits having logic gates for implementing logical functions on data signals, application-specific integrated circuits (ASICs) having suitable combinational logic gates, programmable gate arrays (PGAs), field-programmable gate arrays (FPGAs), etc.

[0113] In the description of this specification, the references to terms such as "one embodiment," "some embodiments," "example," "specific example," or "some examples," etc., indicate that a specific feature, structure, material, or characteristic described in connection with that embodiment or example is included in at least one embodiment or example of this application. In this specification, the illustrative expressions of the above terms do not necessarily refer to the same embodiment or example. Furthermore, the specific features, structures, materials, or characteristics described may be combined in any suitable manner in one or more embodiments or examples.

[0114] Furthermore, the terms "first" and "second" are used for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the number of technical features indicated. Thus, a feature defined as "first" or "second" may explicitly or implicitly include at least one of that feature. In the description of this application, "multiple" means at least two, such as two, three, etc., unless otherwise explicitly specified.

[0115] In this application, unless otherwise expressly specified and limited, the terms "installation," "connection," "joining," and "fixing," etc., should be interpreted broadly. For example, they can refer to a fixed connection, a detachable connection, or an integral part; they can refer to a mechanical connection or an electrical connection; they can refer to a direct connection or an indirect connection through an intermediate medium; they can refer to the internal communication of two components or the interaction between two components, unless otherwise expressly limited. Those skilled in the art can understand the specific meaning of the above terms in this application according to the specific circumstances.

[0116] Although embodiments of this application have been shown and described above, it is understood that the above embodiments are exemplary and should not be construed as limiting this application. Those skilled in the art can make changes, modifications, substitutions and variations to the above embodiments within the scope of this application.

Claims

1. A control method for an air conditioner, characterized in that, The method includes: Obtain the current temperature and humidity of each area of ​​the data center, as well as the current load power of electronic devices; The first temperature, the first humidity, and the first load power are used as inputs to the target evaluation network to output decision scores when performing multiple control operations on the air conditioner. Training samples are input to the evaluation network and the target network respectively to obtain a first decision score and a second decision score. When the difference between the first decision score and the second decision score reaches a preset minimum value, the evaluation network is updated to obtain the target evaluation network. A random number generation function is determined based on a greedy strategy algorithm, and random numbers are generated based on the random number generation function. If the random number is less than a preset probability threshold, a target index is randomly generated based on the random number generation function, and a corresponding control operation is selected from the plurality of control operations based on the target index as the control operation; If the random number is greater than the preset probability threshold, the control operation corresponding to the highest decision score will be used as the control operation.

2. The control method for an air conditioner according to claim 1, characterized in that, After the air conditioner is operated based on the control operation, the method further includes: Obtain the reward value corresponding to the operation of the air conditioner based on the control operation, and obtain the second temperature and second humidity of each area of ​​the data center and the second load power of each electronic device after the air conditioner is operated based on the control operation; The reward value, the second temperature, the second humidity, and the second load power are stored in the training samples, wherein the training samples are used to train the evaluation network.

3. The control method for an air conditioner according to claim 2, characterized in that, The training samples also include the first temperature, the first humidity, the first load power, and the current time's control operation. The step of inputting the training samples into the evaluation network and the target network respectively to obtain a first decision score and a second decision score includes: The first temperature, the first humidity, the first load power, and the current adjustment operation are used as inputs to the evaluation network to obtain a first decision score. The second temperature, the second humidity, and the second load power are used as inputs to the evaluation network to output a decision score when performing multiple control operations on the air conditioner. Determine the update and control operation corresponding to the highest decision score; The second temperature, the second humidity, the second load power, and the update control operation are used as inputs to the target network to obtain a second decision score.

4. The control method for an air conditioner according to claim 3, characterized in that, The method further includes: The target loss function is determined based on the first decision score and the second decision score; The objective loss function is minimized based on the gradient descent algorithm so that the difference between the first decision score and the second decision score reaches a preset minimum value.

5. The control method for an air conditioner according to claim 4, characterized in that, The step of determining the target loss function based on the first decision score and the second decision score includes: The product of the discount factor and the second decision score, plus the sum of the reward value, is used as the target decision score; The target loss function is determined based on the difference between the first decision score and the target decision score.

6. The control method for an air conditioner according to claim 2, characterized in that, The acquisition of the reward value corresponding to the operation of the air conditioner based on the control operation includes: Obtain the temperature uniformity index of the data center, the temperature stability index of the electronic equipment, and the energy efficiency index of the air conditioner; The reward value is determined by the sum of the product of the first weighting coefficient and the temperature uniformity index, the product of the second weighting coefficient and the temperature stability index, and the product of the third weighting coefficient and the energy efficiency index.

7. The control method for an air conditioner according to claim 6, characterized in that, Obtaining the temperature uniformity index of the data center includes: The average temperature was calculated based on the actual temperature in each area of ​​the data center. The temperature uniformity index is determined based on the actual temperature of each area of ​​the data center, its corresponding weighting coefficient, and the average temperature.

8. The control method for an air conditioner according to claim 6, characterized in that, Obtaining the temperature stability indicators of the electronic device includes: Calculate the maximum and minimum temperature values ​​of the electronic device within a preset time period; The temperature fluctuation difference is determined based on the maximum and minimum values; The temperature stability index is determined based on the temperature fluctuation difference and a preset mapping relationship, wherein the preset mapping relationship is used to indicate the relationship between the temperature fluctuation difference and the temperature stability index.

9. The control method for an air conditioner according to claim 6, characterized in that, Obtaining the energy efficiency index of the air conditioner includes: The energy efficiency ratio is determined based on the ratio of the cooling capacity of the air conditioner to the power consumption of the air conditioner. When the energy efficiency ratio is greater than a preset energy efficiency ratio threshold, a first energy efficiency ratio difference between the energy efficiency ratio and the preset energy efficiency ratio threshold is calculated, and the energy efficiency index is determined based on the product of the first energy efficiency ratio difference and the positive reward value. If the energy efficiency ratio is less than the preset energy efficiency ratio threshold, calculate the second energy efficiency ratio difference between the energy efficiency ratio and the preset energy efficiency ratio threshold, and determine the energy efficiency index based on the product of the second energy efficiency ratio difference and the negative bonus value.

10. The control method for an air conditioner according to claim 1, characterized in that, The method further includes: Monitor the status of the air conditioner; In the event of an malfunction in the air conditioner, the training parameters of the evaluation network are adjusted. The malfunction in the air conditioner includes at least one of the following: in each area of ​​the data center, the temperature in at least one area is greater than a first preset temperature threshold and lasts for a first preset duration, or the temperature in at least one area is less than a second preset temperature threshold and lasts for a second preset duration; the suction pressure and discharge pressure of the air conditioner's compressor are not within a preset pressure range; or the flow rates of chilled water and cooling water are not within a preset flow rate range. Adjusting the training parameters of the evaluation network includes reducing at least one of the following: experience replay pool capacity, batch training size, learning rate, and target network update frequency.

11. The control method for an air conditioner according to claim 1, characterized in that, The method further includes: The difference between the control parameters determined by the evaluation network during the control operation and the control parameters of the feedback controller when controlling the air conditioner is obtained. If the difference exceeds a preset difference threshold and continues for a preset time, the air conditioner is controlled to operate based on the control parameters used by the feedback controller when controlling the air conditioner.

12. The control method for an air conditioner according to claim 1, characterized in that, The method further includes: The compressor suction pressure and discharge pressure of the air conditioner are obtained, and the temperature difference between the cooling water and chilled water in the cooling circuit of the air conditioner is obtained; The intake pressure, the exhaust pressure, the temperature difference, the first temperature, the first humidity, and the first load power are used as inputs to the evaluation network to output a decision score when performing multiple control operations on the air conditioner.

13. The control method for an air conditioner according to claim 1, characterized in that, The method further includes: The ratio of the power difference of the first load at adjacent time points to the time interval between adjacent time points is obtained as the heat load change rate. When the heat load change rate is greater than zero, the air conditioner is controlled in advance based on a first strategy, wherein the first strategy includes at least one of increasing the cooling capacity, increasing the air supply volume, and decreasing the air supply temperature. When the rate of change of heat load is less than zero, the air conditioner is controlled in advance based on a second strategy, wherein controlling the air conditioner includes at least one of reducing the cooling capacity, reducing the air supply volume, and increasing the air supply temperature.

14. A computer program product comprising a computer program / instructions, characterized in that, When the computer program / instructions are executed by the processor, they implement the control method for the air conditioner according to any one of claims 1-13.

15. An air conditioner, characterized in that, include: A memory, a processor, and a program stored in the memory and executable on the processor, wherein when the processor executes the program, it implements the control method for an air conditioner according to any one of claims 1-13.

Images