An intelligent energy-saving control method and system for air conditioner variable frequency

Abstract

The application discloses a kind of intelligent energy-saving control method and system of air conditioner variable frequency, comprising: obtaining the real-time state vector of air conditioner;The real-time state vector is input to pre-trained energy-saving control model;Wherein, the energy-saving control model is iteratively trained by reinforcement learning model of built-in intelligent agent;According to the real-time state vector, the intelligent agent executes target control strategy based on iterative training, selects optimal action combination from action space, to adjust compressor speed, fan speed and air conditioner power;The application introduces reinforcement learning algorithm, and plays the self-adaptive ability of intelligent agent in dynamic environment;Intelligent agent can continuously optimize control strategy according to real-time feedback, and find the optimal balance point between energy saving and comfort;Through reinforcement learning, air conditioner system can learn how to adjust operating parameters under different environmental conditions, realize intelligent control.

Description

TECHNICAL FIELD

[0001] The present application relates to intelligent control of air conditioners, in particular to an intelligent energy-saving control method for air conditioner variable frequency. BACKGROUND

[0002] With the increase of energy consumption and the improvement of environmental protection demand, intelligent energy-saving control has become the focus of research, especially in air conditioning systems. The introduction of variable frequency technology enables air conditioners to adapt to environmental changes by adjusting the speed of the compressor, thereby achieving a certain degree of energy saving. Although variable frequency technology can adjust according to temperature requirements, it cannot solve the problem of long-term energy efficiency optimization. With the rapid development of artificial intelligence technology, reinforcement learning as an effective optimization method has been widely applied in intelligent control systems. However, traditional reinforcement learning methods often face high trial and error costs in practical applications, especially in real-time operation of air conditioning systems. If a traditional reinforcement learning algorithm is used for training, the agent needs to go through a large number of trial and error processes to obtain an effective control strategy, which is very time-consuming and impractical in practice. Therefore, existing technologies usually adopt a method based on historical data and simulation training to define the state space and action space of the air conditioning system in advance, thereby reducing the trial and error cost in actual operation.

[0003] The patent document with patent publication number CN118031385A discloses an air conditioner energy-saving control method and system based on reinforcement learning and digital twin model, which realizes information sharing and collaborative control of air conditioner equipment in different spaces in a multi-air conditioner system environment through network communication, achieves cross-space collaborative control, and maximizes energy-saving effect.

[0004] Despite this, most goal definitions combined with reinforcement learning only focus on the optimization of a single target, such as minimizing temperature error or energy consumption, while ignoring the relationship between the two targets. The disadvantage of single-target optimization is that it cannot fully consider the trade-off between temperature comfort and energy consumption, which may lead to neglecting one target while pursuing the other. For example, simply reducing temperature error may result in frequent start and stop of the system, increasing energy consumption; while simply pursuing the lowest energy consumption may cause the indoor temperature to deviate from the set temperature, affecting user comfort. Therefore, single-target optimization often cannot achieve the best balance between comfort and energy-saving effect of the air conditioning system. SUMMARY

[0005] The present embodiment provides an intelligent energy-saving control method, system, electronic device and storage medium for air conditioner variable frequency, which explores how to solve the problem of double-target balance in air conditioner control.

[0006] In a first aspect, the present application provides an intelligent energy-saving control method for air conditioner variable frequency, comprising:

[0007] S1, obtaining a real-time state vector of an air conditioner;

[0008] S2, inputting the real-time state vector into a pre-trained energy-saving control model; wherein the energy-saving control model is iteratively trained by a reinforcement learning model with an embedded agent;

[0009] S3, according to the real-time state vector, the agent executes a target control strategy based on iteration training, selects an optimal action combination from an action space to adjust the compressor speed, fan speed and air conditioner power;

[0010] Wherein, the optimal action combination aims to maximize the long-term cumulative reward, which includes minimizing the temperature error between the set temperature and the indoor temperature, and simultaneously minimizing the long-term energy consumption.

[0011] In some embodiments, obtaining a real-time state vector of an air conditioner, comprising:

[0012] S1-1, obtaining a plurality of real-time state parameters of the air conditioner, including indoor temperature, indoor humidity, outdoor temperature, set temperature, load power, compressor speed and fan speed;

[0013] S1-2, after standardizing the plurality of real-time state parameters, generating a plurality of real-time state features;

[0014] S1-3, concatenating the plurality of real-time state features to generate a one-dimensional real-time state vector.

[0015] In some embodiments, the modeling step of the energy-saving control model iteratively trained by the reinforcement learning model with the embedded agent, comprising:

[0016] S2-1, defining the state space and action space of the reinforcement learning model;

[0017] S2-2, according to the state space and action space, calculating the immediate reward of the initial state-action pair;

[0018] S2-3, according to the immediate reward of the initial state-action pair, updating the model strategy to generate a strategy update model of the next training time step;

[0019] S2-4, according to the strategy update model of the next training time step, calculating the immediate reward of the next state-action pair;

[0020] S2-5, iteratively updating the model strategy according to the immediate reward of the next state-action pair;

[0021] S2-6, S2-4 to S2-5 are executed in a loop, and the long-term cumulative reward of a plurality of time steps is calculated.

[0022] The calculation expression of the long-term cumulative reward is:

[0023] Wherein, G t is the long-term cumulative reward from the initial training time step; γ is a discount factor, which is between 0 and 1, used to attenuate the weight of future rewards; R(s t+k ,a t+k+1 ) represents the immediate reward obtained by the agent in state s t+k under action a t+k+1 at training time step t+k; k is the offset of the training time step, which represents the relative position from the current time step to the future time step;

[0024] S2-7, when the action selected at several time steps maximizes the long-term cumulative reward, it is characterized that the policy update model has been iteratively converged to the energy-saving control model;

[0025] S2-8, the selected action is derived from the action space of the energy-saving control model, which is defined as the target control policy.

[0026] In some embodiments thereof, the state space and the action space of the reinforcement learning model are defined, comprising:

[0027] S2-1-1, the indoor temperature, the indoor humidity, the outdoor temperature, the set temperature, the load power, the compressor speed and the fan speed at time step t are collected and standardized to define the state features at time step t;

[0028] S2-1-2, the state features at several time steps along the time axis are collected and combined as the state space of the reinforcement learning model;

[0029] S2-1-3, the actions corresponding to several time steps along the time axis are collected and combined as the action space of the reinforcement learning model.

[0030] In some embodiments thereof, according to the state space and the action space, the immediate reward of the initial state-action pair is calculated, comprising:

[0031] S2-2-1, a time axis with the same scale is established in the state space and the action space, and the initial training time step is marked on the time axis;

[0032] S2-2-2, according to the marked initial training time step, several state features are extracted from the state space as the input of the reinforcement learning model;

[0033] S2-2-3, the next training time step is marked on the time axis, and several actions are extracted from the action space as the output of the reinforcement learning model;

[0034] S2-2-4, calculating an immediate reward of the initial state-action pair according to the state features of the initial training time step and the action of the next training time step;

[0035] The function expression of the immediate reward is: R(s t ,a t+1 ) = -(a · |T t -T set | + b · E t );

[0036] Wherein, R(s t ,a t+1 ) is the immediate reward at time step t, s t is the state at time step t, a t+1 is the action taken by the agent at time step t+1, a is the weight coefficient of temperature error, used to balance the relationship between temperature error and energy efficiency; T t represents the indoor temperature at time step t, T set is the set temperature of the air conditioner, |T t -T set | is the temperature error between the indoor temperature and the set temperature; E t represents the energy consumption at time step t, b is the weight coefficient of energy consumption, used to balance the weight between energy consumption and temperature error.

[0037] In some embodiments, the model policy is updated according to the immediate reward, and a policy update model of the next training time step is generated, including:

[0038] S2-3-1, calculating the real Q value of the initial state-action pair; wherein the real Q value represents the return after taking action a t in state s t+1 ;

[0039] S2-3-2, calculating the target Q value of the initial state-action pair according to the real Q value of the initial state-action pair and the immediate reward;

[0040] The calculation expression of the target Q value is:

[0041] Wherein, Q TGT is the target Q value, used to calculate the long-term expected return of the first state-action pair, based on the current immediate reward and the real Q value; represents selecting the action a t+1 that can obtain the maximum real Q value in the next state s t+2 , and represents the long-term cumulative reward of the optimal expectation of the agent on the future return.

[0042] S2-3-3. Calculate the Q-value loss based on the target Q-value and the actual Q-value of the initial state action pair;

[0043] The functional expression for the Q-value loss is: L(θ) = (Q TGT -Q(s t ,a t+1 )) 2 ;

[0044] Where L(θ) is the Q-value loss, representing the error between the true Q-value and the target Q-value of the current initial state-action pair of the model; Q(s t ,a t+1 ) represents the current state s t and current action a t+1 The corresponding true Q-value represents the predicted long-term reward when the agent takes the current action in the current state; (Q TGT -Q(s t ,a t+1 )) 2 This represents the squared difference of the error in the true Q value;

[0045] S2-3-4. Based on the Q-value loss, calculate the gradient of the Q-value loss relative to the model;

[0046] S2-3-5. Based on the gradient of the Q-value loss relative to the model, update the model parameters using gradient descent and generate the strategy to update the model.

[0047] In some embodiments, the immediate reward for the next state-action pair is calculated based on the policy update model at the next training time step, including:

[0048] S2-4-1. Obtain several state features for the next training time step;

[0049] S2-4-2. Take several state features of the next training time step as input to the policy update model and output the action of the next two training time steps.

[0050] S2-4-3. Based on several state features of the next training time step and the actions of the next two training time steps, calculate the immediate reward of the next state-action pair.

[0051] In some of these embodiments, the model policy is iteratively updated based on the immediate reward of the next state action pair, including S2-5-1, calculating the true Q value of the next state action pair;

[0052] S2-5-2. Calculate the target Q value of the next state action pair based on the true Q value and the immediate reward.

[0053] S2-5-3, calculating a Q-value loss between a target Q-value of the next state-action pair and a real Q-value of the next state-action pair;

[0054] S2-5-4, calculating a model gradient of the Q-value loss, and iteratively updating the model policy in a gradient descent manner.

[0055] Compared with the prior art, the intelligent energy-saving control method for variable frequency air conditioning of the present application introduces a reinforcement learning algorithm to exert the self-adaptive ability of the agent in a dynamic environment. The agent can continuously optimize the control strategy according to real-time feedback and find the optimal balance point between energy saving and comfort. Through reinforcement learning, the air conditioning system can learn how to adjust the operating parameters under different environmental conditions to achieve intelligent control. This adaptive adjustment capability not only improves the operating efficiency of the system but also reduces human intervention and improves user experience.

[0056] Further, through a long-term cumulative reward mechanism, the two goals of minimizing temperature error and minimizing energy consumption are combined to achieve dual optimization of the air conditioning system. By accurately adjusting the air conditioning parameters, the error between the set temperature and the indoor temperature is minimized to improve user comfort, and the energy consumption of the air conditioner is effectively reduced. This dual-target balancing design ensures that the air conditioning system improves energy efficiency without sacrificing comfort, achieving more efficient and more economical energy-saving effect.

[0057] In a second aspect, the present application provides an intelligent energy-saving control method system for variable frequency air conditioning, the control system comprising:

[0058] a state acquisition module for acquiring a real-time state vector of the air conditioner;

[0059] a state input module for inputting the real-time state vector to a pre-trained energy-saving control model; wherein the energy-saving control model is iteratively trained by a reinforcement learning model with an embedded agent;

[0060] an action output module for selecting an optimal action combination from the action space to adjust the compressor speed, fan speed and air conditioner power according to the real-time state vector and the target control strategy executed by the agent based on the iterative training;

[0061] wherein the optimal action combination aims to maximize the long-term cumulative reward, and the long-term cumulative reward includes minimizing the temperature error between the set temperature and the indoor temperature and simultaneously minimizing the long-term energy consumption.

[0062] Compared with the prior art, the intelligent energy-saving control system for variable frequency air conditioning of the present application has the same beneficial effects as the intelligent energy-saving control method for variable frequency air conditioning described above, and therefore will not be described here. BRIEF DESCRIPTION OF DRAWINGS

[0063] Figure 1 A flowchart of a variable frequency intelligent energy-saving control method of an air conditioner according to the present application;

[0064] Figure 2 A flowchart of a modeling step of the energy-saving control model according to the present application;

[0065] Figure 3 A structure block diagram of a variable frequency intelligent energy-saving control system of an air conditioner according to the present application. DETAILED DESCRIPTION

[0066] The technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the drawings in the embodiments of the present application. Obviously, the described embodiments are only a part of the embodiments of the present application, rather than all the embodiments of the present application. Based on the embodiments in the present application, all other embodiments obtained by those skilled in the art without creative efforts belong to the scope of protection of the present application.

[0067] Embodiment 1: Please refer to Figures 1 to 2 The present application provides a variable frequency intelligent energy-saving control method of an air conditioner, comprising the following steps:

[0068] S1, obtaining a real-time state vector of an air conditioner;

[0069] The step S1 specifically comprises:

[0070] S1-1, obtaining a plurality of real-time state parameters of the air conditioner, wherein the plurality of real-time state parameters comprise indoor temperature, indoor humidity, external temperature, set temperature, load power, compressor speed and fan speed;

[0071] S1-2, generating a plurality of real-time state features after standardizing the plurality of real-time state parameters;

[0072] S1-3, performing feature splicing on the plurality of real-time state features to generate a one-dimensional structure real-time state vector.

[0073] In the embodiment, a plurality of real-time state features are generated by obtaining a plurality of real-time state parameters of an air conditioner and standardizing the parameters. The features are converted into a one-dimensional structure real-time state vector through feature splicing, which is used as the input of the intelligent agent for decision-making, thereby ensuring efficient use of information of various parameters of the air conditioner and optimizing the overall control efficiency.

[0074] S2, inputting the real-time state vector into a pre-trained energy-saving control model, wherein the energy-saving control model is iteratively trained by a reinforcement learning model of an embedded intelligent agent;

[0075] S3, according to the real-time state vector, the intelligent agent executes the target control strategy based on iterative training to select an optimal action combination from the action space to adjust the compressor speed, the fan speed and the air conditioner power;

[0076] Wherein, the optimal action combination aims to maximize the long-term cumulative reward, which includes minimizing the temperature error between the set temperature and the indoor temperature, and simultaneously minimizing the long-term energy consumption.

[0077] In this embodiment, the intelligent optimization of the variable frequency air conditioner running state is realized by inputting the real-time state vector into the pre-trained energy-saving control model. The model is trained by an intelligent agent, which can select the optimal action combination from the state space to adjust the compressor speed, fan speed, air conditioner power and other parameters, so as to maximize the long-term cumulative reward. The long-term cumulative reward not only considers the error between the set temperature and the indoor temperature, but also takes into account the minimization of energy consumption. Through the long-term cumulative reward, the intelligent agent can ensure temperature comfort while minimizing energy consumption.

[0078] In this embodiment, the energy-saving control model is iteratively trained by a reinforcement learning model with an embedded intelligent agent. The modeling step includes:

[0079] S2-1, defining the state space and action space of the reinforcement learning model;

[0080] S2-2, calculating the immediate reward of the initial state-action pair according to the state space and action space;

[0081] S2-3, updating the model strategy according to the immediate reward of the initial state-action pair to generate a strategy update model for the next training time step;

[0082] S2-4, calculating the immediate reward of the next state-action pair according to the strategy update model for the next training time step;

[0083] S2-5, iteratively updating the model strategy according to the immediate reward of the next state-action pair;

[0084] S2-6, S2-4 to S2-5 are executed in a loop, and the long-term cumulative reward of several time steps is calculated;

[0085] The calculation expression of the long-term cumulative reward is:

[0086] Wherein, G tR(s t+k ,a t+k+1 ) represents the immediate reward obtained by the agent taking action a t+k in state s t+k+1 at training time step t+k; this reward reflects the agent's performance at that moment and state, meaning the immediate reward obtained by the agent at future time steps, and as k increases, it represents the reward further in the future. k is the offset of the training time step, used to represent the relative position from the current time step to the future time step; that is, k represents the position of the future reward, and as k increases, the reward further in the future is calculated. In the calculation process, k is used as an exponential factor to represent the degree of attenuation of the future reward. Specifically, the influence of the future reward will gradually decrease as k increases.

[0087] In summary, the long-term cumulative reward calculates the long-term cumulative reward from the initial training time step, and the cumulative reward is obtained by weighting and summing the immediate reward of each time step, where the weight is determined by the discount factor γ.

[0088] S2-7, when the action selected by the plurality of time steps maximizes the long-term cumulative reward, it is characterized that the policy update model has been iteratively converged to the energy-saving control model;

[0089] S2-8, derive the selected action from the action space of the energy-saving control model, and define it as the target control policy.

[0090] In this embodiment, by defining the state space and action space of the reinforcement learning model, and by constantly updating the model policy through iteration, the agent can optimize behavior according to the immediate reward of each state-action pair. After each policy update, the agent will calculate the immediate reward of the next state-action pair according to the new policy, and continuously adjust the policy until the long-term cumulative reward is maximized, thereby achieving the optimal control policy. Through the role of the discount factor, the balance between short-term and long-term returns is achieved, ensuring that the agent not only focuses on the current immediate reward, but also considers future rewards. When the policy update converges, the model is finalized as an energy-saving control model, and an optimal target control policy is finally derived for controlling the operation of the air conditioner, thereby achieving intelligent energy-saving control.

[0091] Further, the step S2-1 specifically comprises:

[0092] S2-1-1, collecting the indoor temperature, indoor humidity, external temperature, set temperature, load power, compressor speed and fan speed at time step t, and defining the state features at time step t after standardization processing;

[0093] S2-1-2, collecting state features at several time steps along the time axis, and combining them into the state space of the reinforcement learning model;

[0094] S2-1-3, collecting actions corresponding to several time steps along the time axis, and combining them into the action space of the reinforcement learning model.

[0095] In this embodiment, by collecting multiple real-time state parameters of the air conditioner (such as indoor temperature, humidity, external temperature, set temperature, load power, compressor speed and fan speed), and standardizing the parameters, the state features at each time step can be defined. These state features are then spliced along the time axis to form the state space of the reinforcement learning model, which serves as the input for the agent's decision-making. In addition, the agent also constitutes the action space by collecting action parameters along the time axis. Through the state space and action space constructed based on the time axis, each state and corresponding action of the air conditioner environment can be effectively represented.

[0096] Further, the step S2-2 specifically comprises:

[0097] S2-2-1, establishing a time axis of the same scale in the state space and action space, and marking the initial training time step on the time axis;

[0098] S2-2-2, extracting several state features from the state space as the input of the reinforcement learning model according to the marked initial training time step;

[0099] S2-2-3, mark the next training time step on the time axis, and extract a number of actions from the action space as the output of the reinforcement learning model;

[0100] S2-2-4, calculate the immediate reward of the initial state-action pair according to the state features of the initial training time step and the action of the next training time step;

[0101] The function expression of the immediate reward is: R(s t ,a t+1 ) = -(a·|T t -T set | + b·E t );

[0102] Wherein, R(s t ,a t+1 ) is the immediate reward at time step t, which is the reward obtained by the agent taking action a t+1 in state s t . s t is the state at time step t (for example: current indoor temperature T t+1 , humidity, external temperature, set temperature, etc.). a t is the action taken by the agent at time step t+1 (for example: adjusting the compressor speed, fan speed or air conditioning power, etc.). a is the weight coefficient of temperature error, used to balance the relationship between temperature error and energy efficiency; T set represents the indoor temperature at time step t, T t is the set temperature of the air conditioner, i.e. the temperature that the user wants to maintain, |T set -T t | is the temperature error between the indoor temperature and the set temperature; E t represents the energy consumption at time step t, which can be the power usage, running time, etc. of the air conditioner; b is the weight coefficient of energy consumption, used to balance the weight between energy consumption and temperature error.

[0103] Specifically, the immediate reward reflects the reward obtained by the agent taking action a t+1 in state s t at time step t. That is, the reward is the result of the feedback of the agent after executing the action. And the smaller the temperature error means that the agent performs better, so it will get a higher reward. Energy consumption represents the energy consumption of the air conditioner when executing the current action. When the energy consumption is lower, the agent will get more rewards. For the agent, reducing energy consumption is one of the goals.

[0104] In the embodiment, by establishing a time axis of the same scale on the state space and the action space, and marking the initial training time step, the relevant state features can be extracted from the state space as the input of the reinforcement learning model. Then, according to the marked time step, the corresponding action is extracted from the action space as the output, and the immediate reward of the initial state-action pair is calculated. The immediate reward is calculated according to the weighted value of the temperature error and the energy consumption, and the difference between the indoor temperature and the set temperature, and the energy consumption of the air conditioner are specifically considered. By calculating the immediate reward, the agent can adjust the control strategy according to the immediate feedback to ensure that the comfort is guaranteed while the energy consumption is minimized, and the intelligent energy-saving control is realized.

[0105] Further, the step S2-3 specifically comprises:

[0106] S2-3-1, calculating a real Q value of the initial state-action pair; wherein the real Q value represents a state s t and an action a t+1 after the action a is taken.

[0107] S2-3-2, calculating a target Q value of the initial state-action pair according to the real Q value of the initial state-action pair and the immediate reward.

[0108] The calculation expression of the target Q value is:

[0109] wherein Q TGT is the target Q value, used for calculating a long-term expected return of the first state-action pair, based on the current immediate reward and the real Q value; R(s t ,a t+1 ) is as described above, representing the immediate reward obtained by taking the action a t in the state s t+1 . represents selecting the action a t+2 that can obtain the maximum real Q value in the next state s t+1 , representing the long-term cumulative reward of the optimal expectation of the agent on the future return.

[0110] S2-3-3, calculating a Q value loss according to the target Q value and the real Q value of the initial state-action pair.

[0111] The function expression of the Q value loss is: L(θ)=(Q TGT -Q(s t ,a t+1 )) 2 ;

[0112] wherein L(Q) is a Q-value loss, representing the error between the true Q-value and the target Q-value for the initial state-action pair of the model; the goal is to minimize the Q-value loss function so as to update the parameters of the reinforcement learning model (e.g., the weights in the neural network) such that the true Q-value is as close as possible to the target Q-value. The true Q-value Q(s t ,a t+1 ) is the true Q-value corresponding to the current state s t and the current action a t+1 , representing the long-term return predicted by the agent when taking the current action in the current state (i.e., it considers the expected return of the action). TGT -Q(s t ,a t+1 )) 2 represents the squared difference of the true Q-value error, which is penalized by the squared exponential for larger errors; the Q-value loss function is minimized by computing the squared error, and the agent can gradually correct its policy so that the difference between the predicted true Q-value and the target Q-value is smaller.

[0113] S2-3-4, calculating the gradient of the Q-value loss with respect to the model according to the Q-value loss;

[0114] S2-3-5, updating the model parameters using gradient descent according to the gradient of the Q-value loss with respect to the model, to generate the policy update model.

[0115] In this embodiment, by calculating the true Q-value of the initial state-action pair, and calculating the target Q-value according to the immediate reward and the true Q-value, the agent can optimize its policy and update the model. The target Q-value combines the current immediate reward and the agent's expectation of future returns, reflecting the long-term expected return of the agent after taking a particular action in the current state. Next, by calculating the loss between the target Q-value and the true Q-value, and updating the parameters of the model using the gradient descent method, the agent can gradually reduce the difference between the predicted Q-value and the target Q-value, thereby optimizing its control policy.

[0116] Further, the step S2-4 specifically comprises:

[0117] S2-4-1, obtaining a plurality of state features of the next training time step;

[0118] S2-4-2, taking the plurality of state features of the next training time step as input of the policy update model, and outputting the action of the next training time step;

[0119] S2-4-3, calculating the immediate reward of the next state-action pair according to the plurality of state features of the next training time step and the action of the next training time step.

[0120] In the embodiment, the agent can output the action of the next training time step by obtaining the state feature of the next training time step and inputting it into the model updated in the policy. According to the action and the state feature, the agent calculates the immediate reward of the next state-action pair. Through the immediate reward of the next state-action pair, the agent continuously adjusts the control policy according to the new input and feedback, thereby realizing the dynamic optimization of the operation of the air conditioner to achieve the balance between energy saving and comfort.

[0121] Further, the step S2-5 specifically comprises

[0122] S2-5-1, calculating the real Q value of the next state-action pair;

[0123] S2-5-2, calculating the target Q value of the next state-action pair according to the real Q value of the next state-action pair and the immediate reward;

[0124] S2-5-3, calculating the Q value loss between the target Q value of the next state-action pair and the real Q value of the next state-action pair;

[0125] S2-5-4, calculating the model gradient of the Q value loss, and iteratively updating the model policy in the gradient descent manner.

[0126] In the embodiment, the agent updates through the same steps as the initial state-action pair by calculating the real Q value of the next state-action pair and calculating the target Q value in combination with the immediate reward. That is, this step belongs to part of the iterative update, and the parameters of the model are adjusted by using the gradient descent method by gradually reducing the difference between the target Q value and the real Q value. Each iteration optimizes the model policy to ensure that the agent can continuously improve its decision quality so as to converge to the optimal policy after multiple training cycles and realize the optimal balance between energy saving and comfort of the air conditioning system.

[0127] Embodiment 2: refer to Figure 3 Embodiment 2 also provides an intelligent energy-saving control method system for variable frequency air conditioners, which is used to implement the above method embodiments and has been described. The terms "module", "unit", "sub-unit" and the like used below can be a combination of software and / or hardware that realizes a predetermined function. Although the system described in the following embodiments is preferably implemented in software, hardware or a combination of software and hardware is also possible and is conceived.

[0128] As Figure 3 shown, Figure 3 is a structural block diagram of an intelligent energy-saving control method system for variable frequency air conditioners of the present application, which comprises:

[0129] a state acquisition module for acquiring a real-time state vector of the air conditioner;

[0130] a state input module configured to input the real-time state vector into a pre-trained energy-saving control model, wherein the energy-saving control model is iteratively trained by a reinforcement learning model with an embedded agent;

[0131] an action output module configured to select an optimal action combination from an action space according to the real-time state vector, so as to adjust the compressor speed, the fan speed and the air conditioner power, wherein the optimal action combination is executed by the agent based on the target control strategy after iteration training.

[0132] The optimal action combination aims to maximize a long-term cumulative reward, and the long-term cumulative reward includes minimizing a temperature error between a set temperature and an indoor temperature and simultaneously minimizing a long-term energy consumption.

[0133] In the above system, the real-time state vector of the air conditioner is obtained by the state acquisition module, the real-time state vector is input into the pre-trained energy-saving control model by the state input module, and the optimal action combination is selected from the action space by the action output module, thereby solving the problem of double-target balance in air conditioner control.

[0134] The above embodiments can be realized wholly or partially by software, hardware, firmware or any other combination. When realized by software, the above embodiments can be realized wholly or partially in the form of a computer program product. The computer program product includes one or more computer instructions or computer programs. When the computer instructions or computer programs are loaded or executed on a computer, the processes or functions described in the embodiments of the present application are wholly or partially generated. The computer can be a general-purpose computer, a special-purpose computer, a computer network or other programmable systems. The computer instructions can be stored in a computer-readable storage medium or transferred from one computer-readable storage medium to another, for example, the computer instructions can be transferred from one website, computer, server or data center to another by wired (for example, infrared, wireless, microwave, etc.) or wireless means.

[0135] The computer-readable storage medium can be any available medium accessible by a computer or a data storage device such as a server, data center and the like containing a set of one or more available media. The available media can be a magnetic medium (for example, a floppy disk, a hard disk, a magnetic tape), an optical medium (for example, a DVD) or a semiconductor medium. The semiconductor medium can be a solid state disk.

[0136] In several embodiments provided in the present application, it should be understood that the disclosed system, system and method can be implemented in other ways. For example, the above-described system embodiments are only illustrative, for example, the division of the units is only a logical function division of the channel underwater topographic change analysis system and method, and actual implementation can have another division manner, for example, a plurality of units or components can be combined or integrated into another system, or some features can be ignored or not executed. In addition, the coupling or direct coupling or communication connection between the displayed or discussed units can be indirect coupling or communication connection through some interfaces, systems or units, which can be electrical, mechanical or other forms.

[0137] The above describes only specific embodiments of the present application, but the protection scope of the present application is not limited thereto, and any person skilled in the art can easily think of changes or replacements within the technical range disclosed in the present application, which should be covered within the protection scope of the present application.

Claims

1. A method for intelligent energy-saving control of air conditioner frequency conversion, characterized in that, The method comprises the following steps: S1, obtaining a real-time state vector of an air conditioner; S2, inputting the real-time state vector into a pre-trained energy-saving control model; wherein the energy-saving control model is iteratively trained by a reinforcement learning model with an embedded agent; S3, according to the real-time state vector, the agent executes a target control strategy based on the iteration training, selects an optimal action combination from an action space to adjust the compressor speed, fan speed and air conditioner power; wherein the optimal action combination aims to maximize the long-term cumulative reward, which includes minimizing the temperature error between the set temperature and the indoor temperature, and simultaneously minimizing the long-term energy consumption; The energy-saving control model is iteratively trained by a reinforcement learning model with an embedded agent, comprising: defining the state space and action space of the reinforcement learning model; calculating the immediate reward of the initial state-action pair according to the state space and action space; The calculation of the immediate reward of the initial state-action pair according to the state space and action space comprises: establishing a time axis of the same scale in the state space and action space, and marking the initial training time step on the time axis; According to the marked initial training time step, extract a plurality of state features from the state space as the input of the reinforcement learning model; Mark the next training time step on the time axis, and extract a plurality of actions from the action space as the output of the reinforcement learning model; According to the state features of the initial training time step and the actions of the next training time step, the immediate reward of the initial state-action pair is calculated; The function expression of the instant reward is: R(s t ,a t+1 ) = -(α·|T t -T set | + β·E t ); wherein R(s t ,a t+1 ) is the immediate reward at time step t, s t is the state at time step t, a t+1 is the action taken by the agent at time step t+1, a is a weight coefficient of temperature error for balancing the relationship between temperature error and energy efficiency; T t represents the indoor temperature at time step t, T set is the set temperature of the air conditioner, |T t -T set | is the temperature error between the indoor temperature and the set temperature; E t represents the energy consumption at time step t, and b is a weight coefficient of energy consumption for balancing the weight between energy consumption and temperature error. By establishing a time axis of the same scale on the state space and action space, and marking the initial training time step, relevant state features can be extracted from the state space as the input of the reinforcement learning model; according to the marked time step, the corresponding actions are extracted from the action space as the output, and the immediate reward of the initial state-action pair is calculated.

2. The intelligent energy-saving control method for variable frequency of an air conditioner according to claim 1, characterized in that, Obtaining a real-time state vector of an air conditioner comprises: S1-1, obtaining a plurality of real-time state parameters of the air conditioner, including indoor temperature, indoor humidity, outdoor temperature, set temperature, load power, compressor speed and fan speed; S1-2, after standardizing the plurality of real-time state parameters, a plurality of real-time state features are generated; S1-3, a one-dimensional real-time state vector is generated by splicing the plurality of real-time state features.

3. The intelligent energy-saving control method for variable frequency of an air conditioner according to claim 1, characterized in that, The calculation of the immediate reward of the next state-action pair further comprises: According to the immediate reward of the initial state-action pair, update the model strategy to generate a policy update model of the next training time step; According to the policy update model of the next training time step, the immediate reward of the next state-action pair is calculated; According to the immediate reward of the next state-action pair, iteratively update the model strategy; The update of the model strategy is executed in a loop, and the long-term cumulative reward of a plurality of time steps is calculated; The calculation expression of the long-term accumulated prize is: where G t is the long-term cumulative reward from the initial training time step; γ is a discount factor, which is between 0 and 1, used to decay the weight of future rewards; R(s t+k ,a t+k+1 ) represents the immediate reward obtained by the agent taking action a t+k in state s t+k+1 at training time step t+k; k is the offset of the training time step, which is used to represent the relative position from the current time step to the future time step. When the actions selected in the plurality of time steps maximize the long-term cumulative reward, it indicates that the policy update model has iteratively converged to the energy-saving control model; The selected actions are derived from the action space of the energy-saving control model, and are defined as the target control strategy.

4. The intelligent energy-saving control method for variable frequency of an air conditioner according to claim 3, characterized in that, Defining the state space and action space of the reinforcement learning model comprises: S2-1-1, collect indoor temperature, indoor humidity, outdoor temperature, set temperature, load power, compressor speed and fan speed of the time step t, and define the state features of the time step t after standardization processing; S2-1-2, collect state features of several time steps along the time axis, and combine them into the state space of the reinforcement learning model; S2-1-3, collect actions corresponding to several time steps along the time axis, and combine them into the action space of the reinforcement learning model.

5. The intelligent energy-saving control method for variable frequency of an air conditioner according to claim 3, characterized in that, According to the instant reward, update the model policy to generate a policy update model of the next training time step, including: S2-3-1, calculate a real Q value of the initial state-action pair; wherein the real Q value represents a state s t Take action a t+1 After the reward; S2-3-2, calculate the target Q value of the initial state-action pair according to the real Q value of the initial state-action pair and the instant reward; The calculation expression of the target Q value is: where Q TGT is the target Q-value used to compute the long-term expected return for the first state-action pair, based on the current immediate reward and the true Q-value; represents the maximum true Q-value that can be obtained in the next state s t+1 , where the action a t+2 is selected that maximizes the true Q-value; and represents the long-term cumulative reward that the agent optimally expects from the future rewards. S2-3-3, calculate the Q value loss according to the target Q value and the real Q value of the initial state-action pair; The function expression of the Q value loss is: L(0) = (Q TGT - Q(s t ,a t+1 )) 2 ; Wherein, L(θ) is the Q value loss, which represents the error between the true Q value and the target Q value of the initial state action pair of the model at present; Q(s t ,a t+1 ) is the true Q value corresponding to the current state s t and the current action a t+1 , which represents the long-term return predicted by the agent when taking the current action in the current state; (Q TGT -Q(s t ,a t+1 )) 2 represents the square difference of the true Q value error; S2-3-4, calculate the gradient of the Q value loss with respect to the model; S2-3-5, update the model parameters using gradient descent according to the gradient of the Q value loss with respect to the model, and generate the policy update model.

6. The intelligent energy-saving control method for variable frequency of an air conditioner according to claim 3, characterized in that, According to the policy update model of the next training time step, calculate the instant reward of the next state-action pair, including: S2-4-1, obtain several state features of the next training time step; S2-4-2, input the several state features of the next training time step into the policy update model, and output the action of the next training time step; S2-4-3, calculate the instant reward of the next state-action pair according to the several state features of the next training time step and the action of the next training time step.

7. The intelligent energy-saving control method for variable frequency of an air conditioner according to claim 1, characterized in that, According to the instant reward of the next state-action pair, iteratively update the model policy, including S2-5-1, calculate the real Q value of the next state-action pair; S2-5-2, calculate the target Q value of the next state-action pair according to the real Q value of the next state-action pair and the instant reward; S2-5-3, calculate the Q value loss between the target Q value of the next state-action pair and the real Q value of the next state-action pair; S2-5-4, calculate the model gradient of the Q value loss, and iteratively update the model policy in the form of gradient descent.

8. An intelligent energy-saving control system for air conditioner frequency conversion, applying the intelligent energy-saving control method for air conditioner frequency conversion according to any one of claims 1 to 7, characterized in that, The control system comprises: a state acquisition module for acquiring a real-time state vector of the air conditioner; a state input module for inputting the real-time state vector into a pre-trained energy-saving control model; wherein the energy-saving control model is iteratively trained by a reinforcement learning model with an embedded agent; an action output module for selecting an optimal action combination from the action space to adjust the compressor speed, fan speed and air conditioner power according to the real-time state vector and the target control strategy executed by the agent based on the iterative training; wherein the optimal action combination aims to maximize the long-term cumulative reward, which includes minimizing the temperature error between the set temperature and the indoor temperature, and simultaneously minimizing the long-term energy consumption; The energy-saving control model is iteratively trained by a reinforcement learning model with an embedded agent, including: defining the state space and action space of the reinforcement learning model; calculating the instant reward of the initial state-action pair according to the state space and the action space; The instant reward of the initial state-action pair is calculated according to the state space and the action space, and the instant reward of the initial state-action pair is calculated according to the state space and the action space, and the instant reward of the initial state-action pair is calculated according to the state space and the action space, and the instant reward of the initial state-action pair is calculated according to the state space and the action space, and the instant reward of the initial state-action pair is calculated according to the state space and the action space, and the instant reward of the initial state-action pair is calculated according to the state space and the action space, and the instant reward of the initial state-action pair is calculated according to the state space and the action space, and the instant reward of the initial state-action pair is calculated according to the state space and the action space, and the instant reward of the initial state-action pair is calculated according to the state space and the action space, and the instant reward of the initial state-action pair is calculated according to the state space and the action space, and the instant reward of the initial state-action pair is calculated according to the state space and the action space, and the instant reward of the initial state-action pair is calculated according to the state space and the action space, and the instant reward of the initial state-action pair is calculated according to the state space and the action space, and the instant reward of the initial state-action pair is calculated according to the state space and the action space, and the instant reward of the initial state-action pair is calculated according to the state space and the action space, and the instant reward of the initial state-action pair is calculated according to the state space and the action space, and the instant reward of the initial state-action pair is calculated according to the state space and the action space, and the instant reward of the initial state-action pair is calculated according to the state space and the action space, and the instant reward of the initial state-action pair is calculated according to the state space and the action space, and the instant reward of the initial state-action pair is calculated according to the state space and the action space, and the instant reward of the initial state-action pair is calculated according to the state space and the action space, and the instant reward of the initial state-action pair is calculated according to the state space ​ ​ ​ ​ The function expression of the instant reward is: R(s t ,a t+1 ) = -(α·|T t -T set | + β·E t ); wherein R(s t , a t+1 ) is the immediate reward at time step t, s t is the state at time step t, a t+1 is the action taken by the agent at time step t+1, a is a weight coefficient of temperature error for balancing the relationship between temperature error and energy efficiency; T t represents the indoor temperature at time step t, T set is the set temperature of the air conditioner, |T t -T set | is the temperature error between the indoor temperature and the set temperature; E t represents the energy consumption at time step t, and b is a weight coefficient of energy consumption for balancing the weight between energy consumption and temperature error. ​

Images