Intelligent energy-saving control method and system for air conditioner frequency conversion

Abstract

The invention discloses an intelligent energy-saving control method and system for air conditioner frequency conversion. The intelligent energy-saving control method comprises the steps that a real-time state vector of an air conditioner is obtained; inputting the real-time state vector into a pre-trained energy-saving control model; wherein the energy-saving control model is formed by iterative training of a reinforcement learning model of a built-in intelligent agent; according to the real-time state vector, the intelligent agent executes a target control strategy based on iterative training, and an optimal action combination is selected from an action space so as to adjust the rotating speed of a compressor, the rotating speed of a fan and the power of an air conditioner; according to the invention, the reinforcement learning algorithm is introduced, so that the self-adaptive capability of the intelligent agent in a dynamic environment is exerted; the intelligent agent can continuously optimize a control strategy according to real-time feedback, and find an optimal balance point between energy conservation and comfort; through reinforcement learning, the air conditioning system can learn how to adjust operation parameters under different environment conditions, and intelligent control is achieved.

Description

Technical Field

[0001] The present invention relates to intelligent control of air conditioners, and specifically to an intelligent energy-saving control method for variable-frequency air conditioners. Background Art

[0002] With the increase in energy consumption and the improvement of environmental protection requirements, intelligent energy-saving control has become a research focus, especially in the application of air-conditioning systems. The introduction of variable-frequency technology enables air conditioners to adapt to environmental changes by adjusting the speed of the compressor, thereby achieving a certain degree of energy saving. Although variable-frequency technology can be adjusted according to temperature requirements, it cannot solve the problem of long-term energy efficiency optimization. With the rapid development of artificial intelligence technology, reinforcement learning, as an effective optimization method, has been widely applied to intelligent control systems. However, traditional reinforcement learning methods usually face high trial-and-error costs in practical applications, especially in the real-time operation of air-conditioning systems. If traditional reinforcement learning algorithms are used for training, the agent needs to go through a large number of trial-and-error processes to obtain an effective control strategy, which is very time-consuming and impractical in reality. Therefore, existing technologies usually adopt methods based on historical data and simulation training to pre-define the state space and action space of the air-conditioning system in advance, thereby reducing the trial-and-error costs in actual operation.

[0003] The patent document with the patent publication number CN118031385A discloses an energy-saving control method and system for air conditioners based on reinforcement learning and digital twin models, which realizes information sharing and collaborative control of air-conditioning equipment in different spaces in a multi-air-conditioning system environment through network communication, achieves cross-space collaborative control, and maximizes the energy-saving effect.

[0004] Despite this, most of the objective definitions combined with reinforcement learning only focus on the optimization of a single objective, such as minimizing temperature error or energy consumption, while ignoring the mutual relationship between these two objectives. The disadvantage of single-objective optimization is that it cannot comprehensively consider the trade-off between temperature comfort and energy consumption, which may lead to ignoring one objective when pursuing the other. For example, simply reducing the temperature error may cause the system to start and stop frequently, increasing energy consumption; while simply pursuing the lowest energy consumption may cause the indoor temperature to deviate from the set temperature, affecting the comfort of users. Therefore, single-objective optimization often cannot achieve the best balance between comfort and energy-saving effect in air-conditioning systems. Summary of the Invention

[0005] This embodiment provides an intelligent energy-saving control method, system, electronic device and storage medium for variable-frequency air conditioners, exploring how to solve the problem of dual-objective balance in air-conditioning control.

[0006] In a first aspect, the present invention provides an intelligent energy-saving control method for variable-frequency air conditioners, including:

[0007] S1. Obtain the real-time state vector of the air conditioner;

[0008] S2. Input the real-time state vector into a pre-trained energy-saving control model; wherein, the energy-saving control model is iteratively trained by a reinforcement learning model with a built-in agent;

[0009] S3. According to the real-time state vector, the agent executes the target control strategy after iterative training, selects the optimal action combination from the action space to adjust the compressor speed, fan speed, and air conditioner power;

[0010] Wherein, the optimal action combination aims to maximize the long-term cumulative reward, and the long-term cumulative reward includes minimizing the temperature error between the set temperature and the indoor temperature and simultaneously minimizing the long-term energy consumption.

[0011] In some embodiments, obtaining the real-time state vector of the air conditioner includes:

[0012] S1-1. Obtain several real-time state parameters of the air conditioner, and the several real-time state parameters include: indoor temperature, indoor humidity, external temperature, set temperature, load power, compressor speed, and fan speed;

[0013] S1-2. After standardizing the several real-time state parameters, generate several real-time state features;

[0014] S1-3. Concatenate the several real-time state features to generate a one-dimensional real-time state vector.

[0015] In some embodiments, the modeling steps of the energy-saving control model being iteratively trained by a reinforcement learning model with a built-in agent include:

[0016] S2-1. Define the state space and action space of the reinforcement learning model;

[0017] S2-2. Calculate the immediate reward of the initial state-action pair according to the state space and action space;

[0018] S2-3. Update the model policy according to the immediate reward of the initial state-action pair to generate a policy update model for the next training time step;

[0019] S2-4. Calculate the immediate reward of the next state-action pair according to the policy update model for the next training time step;

[0020] S2-5. Iteratively update the model policy according to the immediate reward of the next state-action pair;

[0021] S2-6. Loop through S2-4 to S2-5 and calculate the long-term cumulative reward for several time steps;

[0022] The calculation expression of the long-term cumulative reward is as follows:

[0023] Among them, G t is the long-term cumulative reward starting from the initial training time step; γ is the discount factor, with a value between 0 and 1, used to attenuate the weight of future rewards; R(s t+k , a t+k+1 ) represents the immediate reward obtained by the agent when taking action a t+k at time step t + k in state s t+k+1 ; k is the offset of the training time step, used to represent the relative position from the current time step to the future time step;

[0024] S2-7. When the actions selected at several time steps maximize the long-term cumulative reward, it indicates that the policy update model has iteratively converged to the energy-saving control model;

[0025] S2-8. Derive the selected actions from the action space of the energy-saving control model and define them as the target control strategy.

[0026] In some of these embodiments, defining the state space and action space of the reinforcement learning model includes:

[0027] S2-1-1. Collect the indoor temperature, indoor humidity, external temperature, set temperature, load power, compressor speed, and fan speed at time step t, and define them as the state features at time step t after standardization processing;

[0028] S2-1-2. Collect the state features of several time steps along the time axis and combine them into the state space of the reinforcement learning model;

[0029] S2-1-3. Collect the actions corresponding to several time steps along the time axis and combine them into the action space of the reinforcement learning model.

[0030] In some of these embodiments, calculating the immediate reward of the initial state-action pair according to the state space and action space includes:

[0031] S2-2-1. Establish a time axis of the same scale in the state space and action space, and mark the initial training time step on the time axis;

[0032] S2-2-2. According to the marked initial training time step, extract several state features from the state space as the input of the reinforcement learning model;

[0033] S2-2-3. Mark the next training time step on the time axis and extract several actions from the action space as the output of the reinforcement learning model;

[0034] S2-2-4. Calculate the immediate reward of the initial state-action pair based on several state features at the initial training time step and the action at the next training time step;

[0035] The functional expression of the immediate reward is: R(s t , a t+1 ) = -(α·|T t - T set | + β·E t );

[0036] Among them, R(s t , a t+1 ) is the immediate reward at time step t, s t is the state at time step t, a t+1 is the action taken by the agent at time step t + 1, α is the weight coefficient of the temperature error, used to balance the relationship between the temperature error and the energy efficiency; T t represents the indoor temperature at time step t, T set is the set temperature of the air conditioner, |T t - T set | is the temperature error between the indoor temperature and the set temperature; E t represents the energy consumption at time step t, β is the weight coefficient of the energy consumption, used to balance the weight between the energy consumption and the temperature error.

[0037] In some of these embodiments, updating the model policy according to the immediate reward to generate a policy update model for the next training time step includes:

[0038] S2-3-1. Calculate the true Q value of the initial state-action pair; among them, the true Q value represents the return after taking action a t under state s t+1 ;

[0039] S2-3-2. Calculate the target Q value of the initial state-action pair according to the true Q value and the immediate reward of the initial state-action pair;

[0040] The calculation expression of the target Q value is:

[0041] Among them, Q TGT is the target Q value, used to calculate the long-term expected return of the first state-action pair, based on the current immediate reward and the true Q value; represents that in the next state s t+1 , select the action a t+2 that can obtain the maximum true Q value, representing the long-term cumulative reward of the agent's optimal expectation of future returns.

[0042] S2-3-3. Calculate the Q-value loss based on the target Q-value and the true Q-value of the initial state-action pair.

[0043] The functional expression of the Q-value loss is: L(θ) = (Q TGT - Q(s t , a t+1 )) 2 ;

[0044] where L(θ) is the Q-value loss, representing the error between the true Q-value and the target Q-value of the initial state-action pair of the current model; Q(s t , a t+1 ) is the true Q-value corresponding to the current state s t and the current action a t+1 , representing the long-term reward predicted when the agent takes the current action in the current state; (Q TGT - Q(s t , a t+1 )) 2 represents the squared difference of the true Q-value error;

[0045] S2-3-4. Calculate the gradient of the Q-value loss with respect to the model based on the Q-value loss.

[0046] S2-3-5. Update the model parameters using gradient descent according to the gradient of the Q-value loss with respect to the model to generate the policy update model.

[0047] In some of these embodiments, calculating the immediate reward of the next state-action pair according to the policy update model of the next training time step includes:

[0048] S2-4-1. Obtain several state features of the next training time step.

[0049] S2-4-2. Use the several state features of the next training time step as the input of the policy update model and output the action of the next second training time step.

[0050] S2-4-3. Calculate the immediate reward of the next state-action pair according to the several state features of the next training time step and the action of the next second training time step.

[0051] In some of these embodiments, iteratively updating the model policy according to the immediate reward of the next state-action pair includes S2-5-1. Calculate the true Q-value of the next state-action pair.

[0052] S2-5-2. Calculate the target Q-value of the next state-action pair according to the true Q-value and the immediate reward of the next state-action pair.

[0053] S2-5-3. Calculate the Q-value loss between the target Q-value of the next state-action pair and the true Q-value of the next state-action pair;

[0054] S2-5-4. Calculate the model gradient of the Q-value loss and iteratively update the model policy in the way of gradient descent.

[0055] Compared with the prior art, an intelligent energy-saving control method for air-conditioning frequency conversion according to the present invention introduces a reinforcement learning algorithm to exert the adaptive ability of the agent in a dynamic environment. The agent can continuously optimize the control strategy according to real-time feedback and find the optimal balance point between energy saving and comfort. Through reinforcement learning, the air-conditioning system can learn how to adjust the operating parameters under different environmental conditions to achieve intelligent control. This ability of adaptive adjustment not only improves the operating efficiency of the system, but also reduces human intervention and enhances the user experience.

[0056] Furthermore, through the long-term cumulative reward mechanism, two goals of minimizing temperature error and minimizing energy consumption are combined to achieve the dual optimization of the air-conditioning system. By precisely adjusting the air-conditioning parameters, not only the error between the set temperature and the indoor temperature is minimized to improve the user's comfort, but also the energy consumption of the air-conditioning is effectively reduced. This design of balancing two goals ensures that the air-conditioning system improves energy efficiency without sacrificing comfort, achieving a more efficient and economical energy-saving effect.

[0057] In a second aspect, the present invention provides a system for an intelligent energy-saving control method for air-conditioning frequency conversion. The control system includes:

[0058] A state acquisition module for acquiring the real-time state vector of the air conditioner;

[0059] A state input module for inputting the real-time state vector into a pre-trained energy-saving control model; wherein, the energy-saving control model is iteratively trained by a reinforcement learning model with an embedded agent;

[0060] An action output module for, according to the real-time state vector, the agent executing a target control strategy based on iterative training to select an optimal action combination from the action space to adjust the compressor speed, the fan speed and the air-conditioning power;

[0061] Wherein, the optimal action combination aims to maximize the long-term cumulative reward, and the long-term cumulative reward includes minimizing the temperature error between the set temperature and the indoor temperature and simultaneously minimizing the long-term energy consumption.

[0062] Compared with the prior art, the beneficial effects of an intelligent energy-saving control system for air-conditioning frequency conversion according to the present invention are the same as those of the above-mentioned intelligent energy-saving control method for air-conditioning frequency conversion, so details are not described herein again. Description of the Drawings

[0063] Figure 1 It is a schematic flow chart of an intelligent energy-saving control method for variable-frequency air conditioners of the present invention;

[0064] Figure 2 It is a flow chart of the modeling steps of the energy-saving control model of the present invention;

[0065] Figure 3 It is a structural block diagram of an intelligent energy-saving control system for variable-frequency air conditioners of the present invention. Specific embodiments

[0066] Next, the technical solutions in the embodiments of the present invention will be clearly and completely described in conjunction with the accompanying drawings in the embodiments of the present invention. Obviously, the described embodiments are only a part of the embodiments of the present invention, rather than all the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those of ordinary skill in the art without creative efforts shall fall within the protection scope of the present invention.

[0067] Embodiment 1: Please refer to Figures 1 to 2 , the present invention provides an intelligent energy-saving control method for variable-frequency air conditioners, including the following steps:

[0068] S1. Obtain the real-time state vector of the air conditioner;

[0069] Among them, the step S1 specifically includes:

[0070] S1-1. Obtain a number of real-time state parameters of the air conditioner, and the number of real-time state parameters includes: indoor temperature, indoor humidity, external temperature, set temperature, load power, compressor speed, and fan speed;

[0071] S1-2. After standardizing the number of real-time state parameters, generate a number of real-time state features;

[0072] S1-3. Concatenate the number of real-time state features to generate a real-time state vector with a one-dimensional structure.

[0073] In this embodiment, by obtaining a number of real-time state parameters of the air conditioner and standardizing these parameters, a number of real-time state features are generated. Through feature concatenation, the features are transformed into a real-time state vector with a one-dimensional structure as the input for the agent to make decisions, thereby ensuring the efficient use of the information of various parameters of the air conditioner and further optimizing the overall control efficiency.

[0074] S2. Input the real-time state vector into a pre-trained energy-saving control model; among them, the energy-saving control model is iteratively trained by a reinforcement learning model with an embedded agent;

[0075] S3. Based on the real-time state vector, the agent executes the target control policy after iterative training, selects the optimal action combination from the action space to adjust the compressor speed, fan speed, and air conditioner power;

[0076] Among them, the optimal action combination aims to maximize the long-term cumulative reward, and the long-term cumulative reward includes minimizing the temperature error between the set temperature and the indoor temperature, and simultaneously minimizing the long-term energy consumption.

[0077] In this embodiment, by inputting the real-time state vector into the pre-trained energy-saving control model, the intelligent optimization of the operating state of the variable-frequency air conditioner is realized. The model is iteratively trained by the agent and can select the optimal action combination from the state space to adjust parameters such as the compressor speed, fan speed, and air conditioner power, so as to achieve the purpose of maximizing the long-term cumulative reward. The long-term cumulative reward not only considers the error between the set temperature and the indoor temperature, but also takes into account the minimization of energy consumption. Through the long-term cumulative reward method, the agent can minimize energy consumption to the greatest extent while ensuring temperature comfort.

[0078] In this embodiment, the modeling steps of the energy-saving control model iteratively trained by the reinforcement learning model with an embedded agent include:

[0079] S2-1. Define the state space and action space of the reinforcement learning model;

[0080] S2-2. Calculate the immediate reward of the initial state-action pair according to the state space and action space;

[0081] S2-3. Update the model policy according to the immediate reward of the initial state-action pair to generate the policy update model for the next training time step;

[0082] S2-4. Calculate the immediate reward of the next state-action pair according to the policy update model of the next training time step;

[0083] S2-5. Iteratively update the model policy according to the immediate reward of the next state-action pair;

[0084] S2-6. Loop through S2-4 to S2-5 and calculate the long-term cumulative reward for several time steps;

[0085] The calculation expression of the long-term cumulative reward is:

[0086] where G tis the long-term cumulative reward starting from the initial training time step; it represents the return obtained by the agent from the initial training time step, based on all the actions it takes and the feedback rewards from the environment. The long-term cumulative reward is the goal for the agent to optimize its behavior. In reinforcement learning, the agent aims to maximize this cumulative reward to achieve the optimal policy. γ is the discount factor, with a value between 0 and 1, used to attenuate the weight of future rewards; a smaller γ value indicates that the agent pays more attention to the current reward, while a larger γ value means the agent also has a greater concern for future rewards. The role of the γ value is to gradually weaken the influence of future rewards, thus preventing the agent from only focusing on future returns and ignoring the current reward. By using γ, the agent can balance short-term and long-term returns during optimization. R(s t+k ,a t+k+1 ) represents the immediate reward obtained by the agent when taking action a t+k in state s t+k+1 at the training time step t + k; this reward reflects the agent's performance at that moment and state, meaning the immediate reward the agent will obtain in future time steps, and as k increases, it represents rewards in the more distant future. k is the offset of the training time step, used to represent the relative position from the current time step to a future time step; that is, k represents the position of future rewards, and as k increases, the rewards calculated are for more distant futures. During the calculation process, k acts as an exponential factor to represent the attenuation degree of future rewards. Specifically, the influence of future rewards will gradually decrease as k increases.

[0087] In summary, the long-term cumulative reward calculates the long-term cumulative reward starting from the initial training time step, and the cumulative reward is obtained by weighted summation of the immediate rewards at each time step, where the weights are determined by the discount factor γ.

[0088] S2-7. When the actions selected at several time steps maximize the long-term cumulative reward, it indicates that the policy update model has iteratively converged to the energy-saving control model;

[0089] S2-8. Derive the selected actions from the action space of the energy-saving control model and define them as the target control policy.

[0090] In this embodiment, by defining the state space and action space of the reinforcement learning model and continuously iterating and updating the model policy, the agent can optimize its behavior based on the immediate reward of each state-action pair. After each policy update, the agent calculates the immediate reward of the next state-action pair according to the new policy and continuously adjusts the policy until the long-term cumulative reward is maximized, thereby achieving the optimal control policy. Through the role of the discount factor, the short-term and long-term rewards are balanced, ensuring that the agent not only pays attention to the current immediate reward but also considers future rewards. When the policy update reaches convergence, the model is finalized as an energy-saving control model, and finally an optimal target control policy is derived to control the operation of the air conditioner, thus realizing intelligent energy-saving control.

[0091] Further, the step S2-1 specifically includes:

[0092] S2-1-1, Collect the indoor temperature, indoor humidity, external temperature, set temperature, load power, compressor speed, and fan speed at time step t, and define them as the state features at time step t after standardization processing;

[0093] S2-1-2, Collect the state features of several time steps along the time axis and combine them into the state space of the reinforcement learning model;

[0094] S2-1-3, Collect the actions corresponding to several time steps along the time axis and combine them into the action space of the reinforcement learning model.

[0095] In this embodiment, by collecting multiple real-time state parameters of the air conditioner (such as indoor temperature, humidity, external temperature, set temperature, load power, compressor speed, and fan speed) and performing standardization processing on the parameters, the state features of each time step can be defined. These state features are then concatenated along the time axis to form the state space of the reinforcement learning model, which serves as the input for the agent's decision-making. In addition, the agent also constitutes the action space through the action parameters collected along the time axis. Through the state space and action space constructed based on the time axis, each state and corresponding action in the air conditioner environment can be effectively represented.

[0096] Further, the step S2-2 specifically includes:

[0097] S2-2-1, Establish a time axis of the same scale in the state space and action space and mark the initial training time step on the time axis;

[0098] S2-2-2, According to the marked initial training time step, extract several state features from the state space as the input of the reinforcement learning model;

[0099] S2-2-3. Mark the next training time step on the time axis, and extract several actions from the action space as the output of the reinforcement learning model;

[0100] S2-2-4. Calculate the immediate reward of the initial state-action pair according to several state features at the initial training time step and the actions at the next training time step;

[0101] The functional expression of the immediate reward is: R(s t ,a t+1 ) = -(α·|T t -T set | + β·E t );

[0102] Among them, R(s t ,a t+1 ) is the immediate reward at time step t, which is the return obtained by the agent taking action a t under state s t+1 . s t is the state at time step t, (for example: the current indoor temperature T t , humidity, external temperature, set temperature, etc.). a t+1 is the action taken by the agent at time step t + 1, (for example: adjusting the compressor speed, fan speed or air conditioner power, etc.). α is the weight coefficient of the temperature error, used to balance the relationship between the temperature error and energy efficiency; T t represents the indoor temperature at time step t, T set is the set temperature of the air conditioner, that is, the temperature that the user hopes to maintain, |T t -T set | is the temperature error between the indoor temperature and the set temperature; E t represents the energy consumption at time step t, which may be the power consumption of the air conditioner, running time, etc.; β is the weight coefficient of the energy consumption, used to balance the weight between the energy consumption and the temperature error.

[0103] Specifically, the immediate reward reflects the return obtained by the agent at time step t after taking action a t under state s t+1 . That is, the reward is the result feedback by the agent after executing the action. And the smaller the temperature error means the better the agent performs, so it will get a higher reward. The energy consumption represents the energy consumption of the air conditioner when executing the current action. When the energy consumption is low, the agent will get more rewards. For the agent, reducing energy consumption is one of the goals.

[0104] In this embodiment, by establishing a time axis of the same scale on the state space and the action space and marking the initial training time step, relevant state features can be extracted from the state space as the input of the reinforcement learning model. Then, according to the marked time step, the corresponding action is extracted from the action space as the output, and the immediate reward of the initial state-action pair is calculated. The immediate reward is calculated based on the weighted values of the temperature error and the energy consumption, specifically considering the difference between the indoor temperature and the set temperature, as well as the energy consumption of the air conditioner. By calculating the immediate reward, the agent can adjust the control strategy according to the immediate feedback to ensure minimizing the energy consumption while guaranteeing the comfort, achieving intelligent energy-saving control.

[0105] Further, the step S2-3 specifically includes:

[0106] S2-3-1. Calculate the true Q value of the initial state-action pair; wherein, the true Q value represents the return after taking action a t under state s t+1 .

[0107] S2-3-2. Calculate the target Q value of the initial state-action pair according to the true Q value and the immediate reward of the initial state-action pair;

[0108] The calculation expression of the target Q value is:

[0109] where Q TGT is the target Q value, used to calculate the long-term expected return of the first state-action pair, based on the current immediate reward and the true Q value; R(s t , a t+1 ) as previously mentioned, represents the immediate reward obtained by taking action a t under state s t+1 . represents that in the next state s t+1 , the action a t+2 that can obtain the maximum true Q value is selected, representing the long-term cumulative reward of the agent's optimal expectation of future returns.

[0110] S2-3-3. Calculate the Q value loss according to the target Q value and the true Q value of the initial state-action pair;

[0111] The function expression of the Q value loss is: L(θ) = (Q TGT - Q(s t , a t+1 )) 2 .

[0112] Among them, L(θ) is the Q-value loss, representing the error between the true Q-value and the target Q-value of the current initial state-action pair of the model; its goal is to minimize the Q-value loss function in order to update the parameters of the reinforcement learning model (for example, the weights in a neural network) so that the true Q-value is as close as possible to the target Q-value. Q(s t ,a t+1 ) is the true Q-value corresponding to the current state s t and the current action a t+1 , representing the long-term return predicted when the agent takes the current action in the current state; (that is, the expected return it believes for this action). (Q TGT -Q(s t ,a t+1 )) 2 represents the mean squared error of the true Q-value error, penalizing larger errors through a squared exponential; by calculating the square of the error and minimizing it, the agent can gradually correct its policy, reducing the gap between the predicted true Q-value and the target Q-value.

[0113] S2-3-4. Calculate the gradient of the Q-value loss with respect to the model according to the Q-value loss;

[0114] S2-3-5. Update the model parameters using gradient descent according to the gradient of the Q-value loss with respect to the model to generate the policy update model.

[0115] In this embodiment, by calculating the true Q-value of the initial state-action pair and calculating the target Q-value based on the immediate reward and the true Q-value, the agent can optimize its policy and update the model. The target Q-value combines the current immediate reward and the agent's expectation of future rewards, reflecting the long-term expected reward of the agent after taking a specific action in the current state. Next, by calculating the loss between the target Q-value and the true Q-value and using the gradient descent method to update the parameters of the model, the agent can gradually reduce the gap between the predicted Q-value and the target Q-value, thereby optimizing its control strategy.

[0116] Furthermore, the step S2-4 specifically includes:

[0117] S2-4-1. Obtain several state features at the next training time step;

[0118] S2-4-2. Use the several state features at the next training time step as the input of the policy update model and output the action at the next two training time steps;

[0119] S2-4-3. Calculate the immediate reward of the next state-action pair according to the several state features at the next training time step and the action at the next two training time steps.

[0120] In this embodiment, by obtaining the state features of the next training time step and inputting them into the model after policy update, the agent can output the action of the next training time step. Based on this action and the state features, the agent calculates the immediate reward of the next state-action pair. Through the immediate reward of the next state-action pair, the agent continuously adjusts the control policy according to the new input and feedback, so as to achieve the dynamic optimization of air conditioner operation and balance energy saving and comfort.

[0121] Further, the step S2-5 specifically includes

[0122] S2-5-1. Calculate the true Q value of the next state-action pair;

[0123] S2-5-2. Calculate the target Q value of the next state-action pair according to the true Q value of the next state-action pair and the immediate reward;

[0124] S2-5-3. Calculate the Q value loss between the target Q value of the next state-action pair and the true Q value of the next state-action pair;

[0125] S2-5-4. Calculate the model gradient of the Q value loss, and iteratively update the model policy in the way of gradient descent.

[0126] In this embodiment, by calculating the true Q value of the next state-action pair and calculating the target Q value in combination with the immediate reward, the agent is updated by the same steps as the initial state-action pair. That is, this step belongs to a part of the iterative update. By gradually reducing the difference between the target Q value and the true Q value, the parameters of the model are adjusted using the gradient descent method. Each iteration optimizes the model policy to ensure that the agent can continuously improve its decision-making quality, so as to converge to the optimal policy after multiple training cycles and achieve the optimal balance of energy saving and comfort of the air conditioner system.

[0127] Embodiment 2: Refer to Figure 3 , Embodiment 2 also provides an intelligent energy-saving control method system for air conditioner frequency conversion. This system is used to implement the above method embodiment, and those that have been described will not be repeated. The following terms "module", "unit", "sub-unit", etc. can be a combination of software and / or hardware that can achieve a predetermined function. Although the system described in the following embodiments is preferably implemented in software, implementation in hardware, or a combination of software and hardware is also possible and contemplated.

[0128] As Figure 3 shown, Figure 3 is the structural block diagram of an intelligent energy-saving control method system for air conditioner frequency conversion according to the present invention. This system includes:

[0129] A state acquisition module, used to acquire the real-time state vector of the air conditioner;

[0130] A state input module for inputting the real-time state vector into a pre-trained energy-saving control model; wherein, the energy-saving control model is iteratively trained by a reinforcement learning model with a built-in agent.

[0131] An action output module for, according to the real-time state vector, the agent executing a target control policy after iterative training to select an optimal action combination from an action space to adjust the compressor speed, the fan speed, and the air-conditioning power.

[0132] Wherein, the optimal action combination aims to maximize the long-term cumulative reward, and the long-term cumulative reward includes minimizing the temperature error between the set temperature and the indoor temperature and simultaneously minimizing the long-term energy consumption.

[0133] In the above system, the real-time state vector of the air conditioner is obtained through the state acquisition module; the real-time state vector is input into the pre-trained energy-saving control model through the state input module; and the optimal action combination is selected from the action space through the action output module; thereby solving the problem of the balance of the dual objectives in air-conditioning control.

[0134] The above embodiments can be implemented in whole or in part by software, hardware, firmware, or any arbitrary combination thereof. When implemented using software, the above embodiments can be implemented in whole or in part in the form of a computer program product. The computer program product includes one or more computer instructions or computer programs. When the computer instructions or computer programs are loaded or executed on a computer, the processes or functions described in the embodiments of the present application are generated in whole or in part. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable systems. The computer instructions can be stored in a computer-readable storage medium or transmitted from one computer-readable storage medium to another computer-readable storage medium. For example, the computer instructions can be transmitted from one website, computer, server, or data center to another website, computer, server, or data center by wire (such as infrared, wireless, microwave, etc.).

[0135] The computer-readable storage medium can be any available medium that can be accessed by a computer or a data storage device such as a server or a data center that includes one or more collections of available media. The available media can be magnetic media (such as floppy disks, hard disks, magnetic tapes), optical media (such as DVDs), or semiconductor media. The semiconductor media can be a solid-state drive.

[0136] In several embodiments provided by the present application, it should be understood that the disclosed systems, systems, and methods can be implemented in other ways. For example, the system embodiments described above are merely illustrative. For example, the division of the units is only a logical function division of an underwater topographic change analysis system and method for a waterway. In actual implementation, there may be other division methods. For example, multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the displayed or discussed coupling or direct coupling or communication connection between each other can be through some interfaces. The indirect coupling or communication connection of the system or unit can be in an electrical, mechanical, or other form.

[0137] As described above, this is only the specific implementation manner of the present application, but the protection scope of the present application is not limited thereto. Any person skilled in the art within the technical scope disclosed by the present application can easily think of changes or substitutions, which should all be covered within the protection scope of the present application.

Claims

1. An intelligent energy-saving control method for air-conditioning frequency conversion, characterized in that: include: S1. Obtain the real-time state vector of the air conditioner; S2. Inputting the real-time state vector into a pre-trained energy-saving control model; wherein the energy-saving control model is iteratively trained by a reinforcement learning model of a built-in intelligent agent; S3, according to the real-time state vector, the intelligent agent executes the target control strategy based on iterative training, and selects the optimal action combination from the action space to adjust the compressor speed, fan speed and air conditioning power; The optimal action combination aims to maximize the long-term cumulative reward, which includes minimizing the temperature error between the set temperature and the indoor temperature and minimizing the long-term energy consumption.

2. The intelligent energy-saving control method for air-conditioning frequency conversion according to claim 1 is characterized in that: Get the real-time status vector of the air conditioner, including: S1-1, obtaining several real-time status parameters of the air conditioner, the several real-time status parameters including: indoor temperature, indoor humidity, external temperature, set temperature, load power, compressor speed and fan speed; S1-2, after standardizing the plurality of real-time status parameters, generating a plurality of real-time status features; S1-3, concatenate several real-time state features to generate a real-time state vector with a one-dimensional structure.

3. The intelligent energy-saving control method for air-conditioning frequency conversion according to claim 1 is characterized in that: The energy-saving control model is modeled by iterative training of a reinforcement learning model of a built-in intelligent agent, including: S2-1. Define the state space and action space of the reinforcement learning model; S2-2. Calculate the immediate reward of the initial state-action pair based on the state space and action space; S2-3, updating the model strategy according to the instant reward of the initial state-action pair to generate a strategy update model for the next training time step; S2-4, updating the model according to the strategy of the next training time step, and calculating the immediate reward of the next state-action pair; S2-5, iteratively update the model strategy based on the immediate reward of the next state-action pair; S2-6, loop through S2-4 to S2-5, and calculate the long-term cumulative rewards for several time steps; The calculation expression of the long-term cumulative reward is: Among them, G t is the long-term cumulative reward starting from the initial training time step; γ is the discount factor, which takes a value between 0 and 1 and is used to attenuate the weight of future rewards; R(s t+k ,a t+k+1 ) means that at the training time step t+k, the agent is in state s t+k Take action a t+k+1 The immediate reward obtained; k is the offset of the training time step, which is used to indicate the relative position from the current time step to the future time step; S2-7, when the actions selected in several time steps maximize the long-term cumulative reward, it indicates that the strategy update model has iteratively converged to the energy-saving control model; S2-8. Derive a selected action from the action space of the energy-saving control model and define it as the target control strategy.

4. The intelligent energy-saving control method for air-conditioning frequency conversion according to claim 3 is characterized in that: Define the state space and action space of the reinforcement learning model, including: S2-1-1, collect the indoor temperature, indoor humidity, external temperature, set temperature, load power, compressor speed and fan speed at time step t, and define them as the state characteristics of time step t after standardization; S2-1-2, collecting state features of several time steps along the time axis, and combining them into the state space of the reinforcement learning model; S2-1-3. Collect actions corresponding to several time steps along the time axis and combine them into the action space of the reinforcement learning model.

5. The intelligent energy-saving control method for air-conditioning frequency conversion according to claim 3 is characterized in that: According to the state space and action space, calculate the immediate reward of the initial state-action pair, including: S2-2-1, establishing a time axis of the same scale in the state space and the action space, and marking the initial training time step on the time axis; S2-2-2, extracting a number of state features from the state space as inputs of the reinforcement learning model according to the marked initial training time step; S2-2-3, mark the next training time step on the timeline, and extract several actions from the action space as the output of the reinforcement learning model; S2-2-4, calculate the immediate reward of the initial state-action pair based on several state features of the initial training time step and the action of the next training time step; The functional expression of the instant reward is: R(s t ,a t+1 )=-(a·|T t -T set |+β·E t ); Among them, R(s t ,a t+1 ) is the immediate reward at time step t, s t is the state at time step t, a t+1 is the action taken by the agent at time step t+1, α is the weight coefficient of temperature error, which is used to balance the relationship between temperature error and energy efficiency; T t represents the indoor temperature at time step t, T set is the set temperature of the air conditioner, |T t -T set | is the temperature error between the indoor temperature and the set temperature; E t represents the energy consumption at time step t, and β is the weight coefficient of energy consumption, which is used to balance the weight between energy consumption and temperature error.

6. The intelligent energy-saving control method for air-conditioning frequency conversion according to claim 3 is characterized in that: According to the instant reward update model strategy, a strategy update model for the next training time step is generated, including: S2-3-1. Calculate the real Q value of the initial state-action pair; wherein the real Q value represents the state s t Take action a t+1 The return after S2-3-2, calculating the target Q value of the initial state-action pair according to the actual Q value and the immediate reward of the initial state-action pair; The calculation expression of the target Q value is: Among them, Q TGT is the target Q value, which is used to calculate the long-term expected return of the first state-action pair based on the current immediate reward and the true Q value; Indicates that in the next state s t+1 In the above example, we select the action a that can obtain the maximum true Q value. t+2 , represents the long-term cumulative reward of the agent’s optimal expectation of future returns; S2-3-3, calculating the Q value loss according to the target Q value and the actual Q value of the initial state action pair; The function expression of the Q value loss is: L(θ)=(Q TGT -Q(s t ,a t+1 )) 2 ; Among them, L(θ) is the Q value loss, which represents the error between the actual Q value and the target Q value of the current initial state action pair of the model; Q(s t ,a t+1 ) is the current state s t and the current action a t+1 The corresponding true Q value represents the predicted long-term reward when the agent takes the current action in the current state; (Q TGT -Q(s t ,a t+1 )) 2 Represents the squared error of the true Q value; S2-3-4. Calculate the gradient of the Q-value loss relative to the model based on the Q-value loss. S2-3-5. According to the gradient of the Q-value loss relative to the model, the model parameters are updated using gradient descent to generate the strategy update model.

7. The intelligent energy-saving control method for air-conditioning frequency conversion according to claim 3 is characterized in that: The strategy update model of the next training time step is used to calculate the immediate reward of the next state-action pair, including: S2-4-1, obtain several state features of the next training time step; S2-4-2, using the state features of the next training time step as input to the strategy update model, and outputting the actions of the next two training time steps; S2-4-3. Calculate the immediate reward for the next state-action pair based on several state features of the next training time step and the actions of the next two training time steps.

8. The intelligent energy-saving control method for air-conditioning frequency conversion according to claim 1, characterized in that: According to the immediate reward of the next state-action pair, the model strategy is iteratively updated, including S2-5-1, calculating the true Q value of the next state-action pair; S2-5-2, calculating the target Q value of the next state-action pair according to the real Q value and the immediate reward of the next state-action pair; S2-5-3, calculate the Q value loss between the target Q value of the next state-action pair and the actual Q value of the next state-action pair; S2-5-4. Calculate the model gradient of the Q-value loss and iteratively update the model strategy by gradient descent.

9. An intelligent energy-saving control system for air conditioning frequency conversion, using an intelligent energy-saving control method for air conditioning frequency conversion as claimed in any one of claims 1 to 8, characterized in that: The control system comprises: A state acquisition module is used to obtain the real-time state vector of the air conditioner; A state input module, used for inputting the real-time state vector into a pre-trained energy-saving control model; wherein the energy-saving control model is iteratively trained by a reinforcement learning model of a built-in intelligent agent; An action output module, configured to enable the agent to execute a target control strategy based on iterative training according to the real-time state vector, and select an optimal action combination from the action space to adjust the compressor speed, fan speed and air conditioning power; The optimal action combination aims to maximize the long-term cumulative reward, which includes minimizing the temperature error between the set temperature and the indoor temperature and minimizing the long-term energy consumption.

Images