Control method of air conditioner and air conditioner

Abstract

The invention provides a control method of an air conditioner and the air conditioner. The control method of the air conditioner comprises the steps that a trigger signal for starting the air conditioner is received, and the set temperature of the air conditioner is obtained; detecting the actual temperature of the indoor environment; determining an intelligent agent according to the actual temperature and the set temperature; and the air conditioner is controlled according to the determined intelligent agent. According to the scheme, different reinforcement learning agents can be dispatched to control the air conditioner according to the actual situation, the comfort level of a user is considered, energy consumption is reduced, and the use experience of the user is improved; reasonable reward function guidance agents are set in different operation stages of the air conditioner, the convergence difficulty of the reinforcement learning algorithm on the air conditioner control method is reduced, the stability of the reinforcement learning algorithm is improved, and the feasibility of the reinforcement learning algorithm is improved.

Description

Technical Field

[0001] The present invention relates to the technical field of household appliances, and in particular to a control method of an air conditioner and the air conditioner. Background Art

[0002] Numerous control methods have been developed or proposed for HVAC systems. Currently, ON / OFF control and PID control strategies are still used in many HVAC systems due to their simplicity, but these methods do not achieve good performance in many systems. Advances in data storage, computing, and communication equipment have led to the introduction of advanced control theories such as variable gain, model predictive control, and reinforcement learning into the intelligent control of residential HVAC systems.

[0003] Comprehensive theoretical analysis shows that the main problem facing household air conditioning control is the complexity and variability of control scenarios, and the excessive number of unobservable factors in different usage scenarios. As a result, control strategies developed in the laboratory often cannot meet the needs of users in real-world usage scenarios. To address this problem, deep learning-based model predictive control algorithms can modify control strategies based on data from the usage phase, but they still require manual modulation of some system-level hyperparameters. Deep learning-based reinforcement learning methods can solve more complex and multi-dimensional control tasks. However, reinforcement learning requires the establishment of a reasonable reward function to guide the intelligent agent. However, the requirement of rapid temperature attainment for comfort and the requirement of energy saving for energy conservation are inherently conflicting. How to configure the reinforcement learning algorithm so that it can simultaneously meet the requirements of comfort and energy conservation is the main difficulty in the implementation phase of the project.

[0004] Specifically, existing air conditioning control algorithms have the following limitations: First, traditional control algorithms like PID require extensive manual parameter tuning and experimentation in the laboratory and are not adaptive to the user's living environment. Second, while model predictive control (MPC) algorithms have adaptive capabilities, they struggle to construct mechanism-based linear models for air conditioning and room systems. Data-driven room load prediction models built using big data consume significant computing resources due to their architecture, which requires the high-frequency operation of neural networks and nonlinear optimization algorithms. This leads to latency and cost pressures in practical applications. Third, compared to algorithms like PID, reinforcement learning offers adaptive and self-evolving capabilities, and its inference phase consumes significantly less computing power than MPC. However, the convergence of reinforcement learning relies on setting a reasonable reward function to guide the agent. The inherent conflict between the comfort requirement of quickly achieving temperature and the energy efficiency requirement of saving energy makes convergence of reinforcement learning algorithms extremely difficult for air conditioning control methods. Summary of the Invention

[0005] One purpose of the present invention is to dispatch different reinforcement learning agents to control the air conditioner according to actual conditions, taking into account both user comfort and energy conservation.

[0006] A further purpose of the present invention is to reduce the convergence difficulty of the reinforcement learning algorithm in the air conditioning control method, improve the stability of the reinforcement learning algorithm, and increase the feasibility of the implementation of the reinforcement learning algorithm.

[0007] In particular, the present invention provides a method for controlling an air conditioner, comprising: receiving a trigger signal for turning on the air conditioner and obtaining a set temperature of the air conditioner; detecting the actual temperature of the indoor environment; determining an intelligent agent based on the actual temperature and the set temperature; and controlling the air conditioner based on the determined intelligent agent.

[0008] Optionally, the step of determining the intelligent agent based on the actual temperature and the set temperature includes: judging whether the absolute value of the difference between the actual temperature and the set temperature is greater than a preset threshold; and if so, determining that the intelligent agent is a temperature-seeking intelligent agent, if not, determining that the intelligent agent is a temperature-controlling intelligent agent.

[0009] Optionally, the reward function for training the thermophilic agent is: , T is the actual temperature, T set For set temperature, abs refers to absolute value.

[0010] Optionally, the reward function for training the temperature control agent is: , u1, u2, and u3 are the set frequency of the air conditioner compressor, the set opening of the electronic expansion valve, and the set speed of the outdoor fan, respectively, and α1, α2, and α3 are preset weight values.

[0011] Optionally, after determining that the agent is a temperature-trending agent, the method further includes: controlling the air conditioner according to the temperature-trending agent; judging abs(TT set )<1 whether it lasts for a first preset time period; and if so, switching to the temperature control intelligent body to control the air conditioner.

[0012] Optionally, when switching from a temperature-seeking agent to a temperature-controlling agent, if abs(TT set )>1 continues for the second preset time, switching to the temperature-trending intelligent agent to control the air conditioner.

[0013] Optionally, after determining that the agent is a temperature control agent, the method further includes: controlling the air conditioner according to the temperature control agent; judging abs(TT set )>1 continues for a third preset time period; and if so, switch to the temperature-trending intelligent body to control the air conditioner.

[0014] Optionally, when the temperature control agent switches to the temperature trending agent, if abs(TT set )<1 continues for a fourth preset time, switching to the temperature control intelligent body to control the air conditioner.

[0015] Optionally, the training of the temperature-seeking agent and the temperature-controlling agent both includes input state parameters and action parameters, and the state parameters include: indoor humidity, actual temperature, set temperature, indoor target humidity, speed of the indoor fan, status of the indoor fan, status of the compressor, operating frequency of the compressor, outdoor temperature, power of the outdoor unit, status of the four-way valve, opening of the electronic expansion valve, speed of the outdoor fan; the action parameters include: set frequency of the compressor, set opening of the electronic expansion valve, set speed of the outdoor fan.

[0016] According to another aspect of the present invention, an air conditioner is provided, comprising: a controller, the controller comprising a memory and a processor, wherein the memory stores a machine executable program, and when the machine executable program is executed by the processor, any of the above-mentioned air conditioner control methods is implemented.

[0017] The control method and air conditioner of the air conditioner of the present invention receive a trigger signal to turn on the air conditioner and obtain the set temperature of the air conditioner, detect the actual temperature of the indoor environment, determine an intelligent agent based on the actual temperature and the set temperature, and control the air conditioner based on the determined intelligent agent. Different reinforcement learning intelligent agents can be dispatched to control the air conditioner according to actual conditions, taking into account user comfort and energy saving, thereby improving the user experience.

[0018] Furthermore, the control method and air conditioner of the air conditioner of the present invention divide the control cycle of the air conditioner into a temperature-trending stage and a temperature-control stage, and set two reward functions respectively, and train two intelligent agents to serve the temperature-trending control and temperature-control control of the air conditioner respectively. The former ensures the rapid cooling / heating of the air conditioner and guarantees the user's comfort as soon as possible, and the latter ensures that the air conditioner does not overshoot in the temperature-control stage and saves energy as much as possible. Reasonable reward functions are set to guide the intelligent agent in different operating stages of the air conditioner, reduce the convergence difficulty of the reinforcement learning algorithm in the air-conditioning control method, improve the stability of the reinforcement learning algorithm, and increase the feasibility of the reinforcement learning algorithm.

[0019] Based on the following detailed description of specific embodiments of the present invention in conjunction with the accompanying drawings, those skilled in the art will become more aware of the above and other objects, advantages and features of the present invention. BRIEF DESCRIPTION OF THE DRAWINGS

[0020] Hereinafter, some specific embodiments of the present invention will be described in detail in an exemplary and non-limiting manner with reference to the accompanying drawings. The same reference numerals in the accompanying drawings indicate the same or similar components or parts. It should be understood by those skilled in the art that these drawings are not necessarily drawn to scale. In the accompanying drawings: Figure 1 is a schematic diagram of a method for controlling an air conditioner according to an embodiment of the present invention; Figure 2is a detailed flow chart of a method for controlling an air conditioner according to an embodiment of the present invention; Figure 3 is a schematic block diagram of a controller for an air conditioner according to an embodiment of the present invention; Figure 4 is a schematic diagram of a machine-readable storage medium according to an embodiment of the present invention. DETAILED DESCRIPTION

[0021] This embodiment first provides a control method for an air conditioner, which can dispatch different reinforcement learning agents to control the air conditioner according to actual conditions, taking into account user comfort and energy saving, thereby improving the user experience. Figure 1 FIG. 1 is a schematic diagram of a method for controlling an air conditioner according to an embodiment of the present invention. Figure 1 As shown, the control method of the air conditioner may include the following steps: Step S102, receiving a trigger signal for turning on the air conditioner and obtaining a set temperature of the air conditioner; Step S104, detecting the actual temperature of the indoor environment; Step S106, determining an intelligent agent according to the actual temperature and the set temperature; Step S108: Control the air conditioner according to the determined intelligent agent.

[0022] In the above steps, step S102 receives a trigger signal for turning on the air conditioner and obtains the set temperature of the air conditioner. In a specific embodiment, the trigger signal and the set temperature can be received from a mobile terminal bound to the air conditioner, or from a display device, voice device, or remote control of the air conditioner. The mobile terminal can be a portable smart device, such as a smartphone or a smart tablet.

[0023] Step S104 detects the actual temperature of the indoor environment, wherein the indoor environment refers to the indoor environment where the indoor unit of the air conditioner is located. In a specific embodiment, the indoor unit of the air conditioner can be provided with a temperature sensor, which can detect the actual temperature of the indoor environment and accurately reflect the temperature of the indoor environment.

[0024] Step S106 determines an agent based on the actual temperature and the set temperature. Specifically, the agent can be determined based on the absolute value of the difference between the actual temperature and the set temperature. In a preferred embodiment, the agent can include a temperature-trending agent and a temperature-controlling agent. Based on the absolute value of the difference between the actual temperature and the set temperature, it can be determined whether the air conditioner is currently in the temperature-trending stage or the temperature-controlling stage, and the agent is correspondingly determined to be a temperature-trending agent or a temperature-controlling agent.

[0025] The absolute value of the difference between the actual temperature and the set temperature is large, indicating that the actual temperature of the indoor environment is far from the set temperature that the user feels comfortable with. At this time, it can be determined that the air conditioner is currently in the temperature-trending stage, and the intelligent agent is determined to be a temperature-trending intelligent agent. "Temperature-trending" means that the air conditioner adjusts the actual temperature of the indoor environment as quickly as possible so that it reaches the set temperature as soon as possible to meet the user's comfort needs.

[0026] The absolute value of the difference between the actual temperature and the set temperature is small, indicating that the actual temperature of the indoor environment is close to the set temperature that the user feels comfortable. At this time, it can be determined that the air conditioner is currently in the temperature control stage, and the intelligent agent is determined to be a temperature control intelligent agent. "Temperature control" means that the air conditioner controls the actual temperature of the indoor environment to around the set temperature, for example, controlling the actual temperature within ±1°C of the set temperature, meeting the user's comfort needs while reducing energy consumption as much as possible.

[0027] In a preferred embodiment, the step of determining the intelligent agent based on the actual temperature and the set temperature may include: judging whether the absolute value of the difference between the actual temperature and the set temperature is greater than a preset threshold; and if so, determining that the intelligent agent is a temperature-seeking intelligent agent; if not, determining that the intelligent agent is a temperature-control intelligent agent.

[0028] In a specific embodiment, the preset threshold value may be 1°C. If the absolute value of the difference between the actual temperature and the set temperature is greater than 1°C, it is considered that the actual temperature of the indoor environment is significantly different from the set temperature at which the user feels comfortable. If the absolute value of the difference between the actual temperature and the set temperature is less than or equal to 1°C, it is considered that the actual temperature of the indoor environment is relatively close to the set temperature at which the user feels comfortable. It should be noted that the specific values ​​of the preset threshold values ​​described above are merely examples and are not limitations of the present invention. In other embodiments, the preset threshold values ​​may be set to other values ​​based on actual conditions.

[0029] After determining an agent based on the actual temperature and the set temperature in step S106, step S108 can be executed to control the air conditioner based on the determined agent. For example, if the agent is determined to be a temperature-trending agent, the air conditioner can be controlled based on the temperature-trending agent; if the agent is determined to be a temperature-controlling agent, the air conditioner can be controlled based on the temperature-controlling agent. By invoking an agent that matches the current actual situation and the air conditioner's operating stage, both user comfort needs and energy conservation requirements can be met.

[0030] In a specific embodiment, the reward function for training the thermophilic agent is: , T is the actual temperature, T set For setting temperature, abs refers to absolute value. In the case of .exist In the case of . Reward function The purpose of setting is to allow the air conditioner to reach the set temperature as quickly as possible to meet the user's comfort requirements.

[0031] The reward function for training the temperature control agent is: , u1, u2, u3 are respectively the set frequency of the air conditioner compressor, the set opening of the electronic expansion valve, and the set speed of the outdoor fan, and α1, α2, α3 are the preset weight values. In the case of .exist In the case of . Reward function The purpose of setting is to allow the air conditioner to control the actual temperature of the indoor environment to be around the set temperature as much as possible, for example, to control the actual temperature to be within ±1°C of the set temperature, so as to meet the user's comfort needs while meeting the energy-saving needs as much as possible.

[0032] α1, α2, and α3 are the preset weights for the air conditioner's compressor frequency, electronic expansion valve opening, and outdoor fan speed, respectively. These can be set based on actual conditions or empirical data. For example, a higher-weighted parameter can have a larger preset weight, while a lower-weighted parameter can have a smaller preset weight.

[0033] In addition, the training of the temperature-trending agent and the temperature-controlling agent both includes input state parameters and action parameters. In addition, the state parameters include: indoor humidity, actual temperature, set temperature, indoor target humidity, indoor fan speed, indoor fan status, compressor status, compressor operating frequency, outdoor temperature, outdoor unit power, four-way valve status, electronic expansion valve opening, outdoor fan speed. The state parameters can be denoted as S, where the state parameter at time T is S t The action parameters include: the set frequency of the compressor, the set opening of the electronic expansion valve, and the set speed of the outdoor fan. The action parameter can be recorded as A, where the action parameter at time T is A t .

[0034] In a specific embodiment, the training method of the temperature-seeking agent and the temperature-controlling agent may include algorithms of all reinforcement learning frameworks, including but not limited to DQN, PG, PPO and other algorithms.

[0035] The Deep Q-Network (DQN) uses a neural network to approximate the Q-value function to achieve optimal policies. As an extension of Q-Learning in the deep learning era, DQN addresses the limitations of traditional Q-Learning in high-dimensional state spaces. DQN offers the following advantages: It eliminates the need for explicit modeling of environmental dynamics, requiring only the learning of the Q-value function; and its experience replay mechanism improves data utilization efficiency.

[0036] Policy Gradient (PG) no longer indirectly learns the value function, but instead directly parameterizes the policy, maximizing the expected cumulative reward through gradient ascent. PG has the following advantages: it naturally supports continuous action spaces; it makes policy optimization more direct, and it can potentially find more complex solutions.

[0037] Proximal Policy Optimization (PPO) improves upon the traditional policy gradient algorithm by constraining the policy update step size to avoid performance degradation caused by excessively large updates. PPO offers the following advantages: high sample efficiency and effective use of historical data; stable training and low sensitivity to hyperparameters; simple implementation, and ease of scalability.

[0038] In summary, the control method of the air conditioner in this embodiment receives a trigger signal to turn on the air conditioner and obtains the set temperature of the air conditioner, detects the actual temperature of the indoor environment, determines an intelligent agent based on the actual temperature and the set temperature, and controls the air conditioner based on the determined intelligent agent. It can dispatch different reinforcement learning intelligent agents to control the air conditioner according to actual conditions, taking into account user comfort and energy saving, and improving the user experience.

[0039] In some optional embodiments, the air conditioner can achieve higher technical effects by further optimizing and configuring the above steps. The following describes in detail the control method of the air conditioner of this embodiment in combination with an introduction to an optional execution process of this embodiment. This embodiment is only an example of the execution process. During specific implementation, the execution order and operating conditions of some steps can be modified according to specific implementation requirements. Figure 2 FIG. 1 is a detailed flow chart of a method for controlling an air conditioner according to an embodiment of the present invention. The method for controlling an air conditioner includes the following steps: Step S202, receiving a trigger signal for turning on the air conditioner and obtaining a set temperature of the air conditioner; Step S204, detecting the actual temperature of the indoor environment; Step S206, determine whether abs(T-Tset) is greater than a preset threshold, if so, execute step S208, if not, execute step S220; Step S208, determining that the agent is a temperature-seeking agent; Step S210, controlling the air conditioner according to the temperature-trending intelligent agent; Step S212, determine whether abs(T-Tset)<1 lasts for a first preset time period, if so, execute step S214, if not, execute step S210; Step S214, switching to the temperature control agent to control the air conditioner; Step S216, determining whether abs(T-Tset)>1 lasts for a second preset time after the switch. If so, execute step S218; if not, execute step S214; Step S218, switching to the temperature-trending agent to control the air conditioner, and executing step S228; Step S220, determining that the agent is a temperature control agent; Step S222, controlling the air conditioner according to the temperature control agent; Step S224, determine whether abs(T-Tset)>1 lasts for a third preset time period, if so, execute step S226, if not, execute step S222; Step S226, switching to the temperature-trending agent to control the air conditioner; Step S228, determining whether abs(T-Tset)<1 continues for a fourth preset time period after the switch; if so, executing step S230; if not, executing step S226; In step S230, the temperature control agent is switched to control the air conditioner, and step S216 is executed.

[0040] In the above steps, after step S202 receives the trigger signal to turn on the air conditioner and obtains the set temperature of the air conditioner, step S204 can be executed to detect the actual temperature of the indoor environment. Then, step S206 is executed to determine whether abs(T-Tset), that is, the absolute value of the difference between the actual temperature and the set temperature, is greater than a preset threshold. If the result of step S206 is yes, that is, the absolute value of the difference between the actual temperature and the set temperature is greater than the preset threshold, steps S208 and S210 are executed to determine that the agent is a temperature-seeking agent, and the air conditioner is controlled according to the temperature-seeking agent.

[0041] If the absolute value of the difference between the actual temperature and the set temperature is greater than the preset threshold, it means that the actual temperature of the indoor environment is far from the set temperature that the user feels comfortable with. At this time, the intelligent agent can be determined to be a temperature-seeking intelligent agent. The air conditioner can be controlled according to the temperature-seeking intelligent agent. The air conditioner can adjust the actual temperature of the indoor environment as quickly as possible so that it reaches the set temperature as soon as possible to meet the user's comfort needs.

[0042] Then, step S212 can be executed to determine whether abs(T-Tset)<1 continues for a first preset time period. If the result of step S212 is yes, that is, abs(T-Tset)<1 continues for the first preset time period, step S214 is executed to switch to the temperature control agent to control the air conditioner. If the result of step S212 is no, that is, abs(T-Tset)<1 does not continue for the first preset time period, step S210 is executed to control the air conditioner according to the temperature trending agent.

[0043] After the air conditioner is controlled by the temperature-trending agent, if abs(T-Tset) < 1 for a first preset period of time, it indicates that the actual temperature of the indoor environment has reached the set temperature, and the air conditioner can be switched to the temperature-control agent to control the air conditioner. If abs(T-Tset) < 1 does not continue for the first preset period of time, it indicates that the actual temperature of the indoor environment has not reached the set temperature, and the air conditioner needs to continue to be controlled by the temperature-trending agent.

[0044] Furthermore, after the temperature-seeking agent switches to the temperature-controlling agent to control the air conditioner in step S214, step S216 may be executed to determine whether abs(T-Tset)>1 persists for a second preset time period after the switch. If the determination result in step S216 is yes, i.e., abs(T-Tset)>1 persists for the second preset time period, step S218 is executed to switch to the temperature-seeking agent to control the air conditioner. If the determination result in step S216 is no, i.e., abs(T-Tset)>1 does not persist for the second preset time period, step S214 is executed to switch to the temperature-controlling agent to control the air conditioner.

[0045] After the temperature-seeking agent switches to the temperature-controlling agent to control the air conditioner, if abs(T-Tset)>1 persists for a second preset duration after the switch, it indicates that the actual indoor temperature has deviated from the set temperature, and the air conditioner can be switched to the temperature-seeking agent to control the air conditioner. If abs(T-Tset)>1 does not persist for the second preset duration, it indicates that the actual indoor temperature has not deviated from the set temperature, and control of the air conditioner needs to be switched to the temperature-controlling agent. In fact, the air conditioner continues to be controlled by the temperature-controlling agent.

[0046] If the judgment result in step S206 is no, that is, the absolute value of the difference between the actual temperature and the set temperature is less than or equal to the preset threshold, steps S220 and S222 are executed to determine that the intelligent agent is a temperature control intelligent agent, and the air conditioner is controlled according to the temperature control intelligent agent.

[0047] If the absolute value of the difference between the actual temperature and the set temperature is less than or equal to the preset threshold, it means that the actual temperature of the indoor environment is close to the set temperature that the user feels comfortable with. At this time, the intelligent agent can be determined as a temperature control intelligent agent. The air conditioner can be controlled according to the temperature control intelligent agent. The air conditioner can control the actual temperature of the indoor environment to around the set temperature, for example, controlling the actual temperature to within ±1°C of the set temperature, thereby meeting the user's comfort needs while reducing energy consumption as much as possible.

[0048] Then, step S224 can be executed to determine whether abs(T-Tset)>1 continues for a third preset time period. If the result of step S224 is yes, that is, abs(T-Tset)>1 continues for the third preset time period, step S226 is executed to switch to the temperature-trending agent to control the air conditioner. If the result of step S224 is no, that is, abs(T-Tset)>1 does not continue for the third preset time period, step S222 is executed to control the air conditioner according to the temperature-control agent.

[0049] After the temperature-control agent controls the air conditioner, if abs(T-Tset)>1 persists for a third preset period of time, indicating that the actual indoor temperature has deviated from the set temperature, the temperature-trending agent can be used to control the air conditioner. If abs(T-Tset)>1 does not persist for the third preset period of time, indicating that the actual indoor temperature has not deviated from the set temperature, the temperature-control agent should continue to control the air conditioner.

[0050] Furthermore, after the temperature-control agent switches to the temperature-trending agent to control the air conditioner in step S226, step S228 may be executed to determine whether abs(T-Tset)<1 persists for a fourth preset time period after the switch. If the determination result in step S228 is yes, i.e., abs(T-Tset)<1 persists for the fourth preset time period, step S230 is executed to switch to the temperature-control agent to control the air conditioner. If the determination result in step S228 is no, i.e., abs(T-Tset)<1 does not persist for the fourth preset time period, step S226 is executed to switch to the temperature-trending agent to control the air conditioner.

[0051] After the temperature-control agent switches to the temperature-trending agent to control the air conditioner, if abs(T-Tset) < 1 for a fourth preset period of time after the switch, it indicates that the actual temperature of the indoor environment has reached the set temperature, and the air conditioner can be switched to the temperature-control agent to control the air conditioner. If abs(T-Tset) < 1 does not continue for the fourth preset period of time, it indicates that the actual temperature of the indoor environment has not reached the set temperature, and the air conditioner needs to be switched to the temperature-trending agent to control the air conditioner. In fact, the air conditioner continues to be controlled by the temperature-trending agent.

[0052] It should be noted that after switching to the temperature-trending agent to control the air conditioner in step S218, step S228 can be executed to determine whether abs(T-Tset) < 1 continues for a fourth preset time period after the switch. After switching to the temperature-control agent to control the air conditioner in step S230, step S216 can be executed to determine whether abs(T-Tset) > 1 continues for a second preset time period after the switch. The corresponding steps can be executed based on the judgment results of steps S228 and S216, respectively. This ensures that the control of the air conditioner is a dynamically adjusted process.

[0053] In a specific embodiment, the first preset duration and the third preset duration may be 3 minutes; the second preset duration and the fourth preset duration may be 1 minute. It should be noted that the specific values ​​of the above-mentioned preset durations are merely examples and are not limitations of the present invention. In other embodiments, the preset durations may be set to other values ​​according to actual circumstances.

[0054] Several specific embodiments are introduced below: During the startup phase of the air conditioner, when abs(T-Tset)>1, the temperature-tending agent controls the air conditioner; when abs(T-Tset)<1 for 3 minutes, the temperature-control agent is switched to control the air conditioner, and one control cycle of the temperature-tending agent is considered completed.

[0055] During the startup phase of the air conditioner, when abs(T-Tset)<1, the temperature control agent controls the air conditioner; when abs(T-Tset)>1 for 3 minutes, the temperature trending agent controls the air conditioner, and one control cycle of the temperature control agent is considered completed.

[0056] When the temperature-trending agent switches to the temperature-controlling agent, and abs(T-Tset)>1 lasts for 1 minute after the switch, the temperature-trending agent is switched to control the air conditioner, and it is considered that one control cycle of the temperature-controlling agent is completed.

[0057] When the temperature control agent switches to the temperature trending agent, if abs(T-Tset)<1 for 1 minute after the switch, the temperature control agent is switched to control the air conditioner, and it is considered that one control cycle of the temperature trending agent is completed.

[0058] In summary, the control method of the air conditioner in this embodiment divides the control cycle of the air conditioner into a temperature-trending stage and a temperature-control stage, and sets two reward functions respectively, and trains two intelligent agents to serve the temperature-trending control and temperature-control control of the air conditioner respectively. The former ensures the rapid cooling / heating of the air conditioner and ensures the user's comfort as soon as possible, and the latter ensures that the air conditioner does not overshoot in the temperature control stage and saves energy as much as possible. By setting reasonable reward functions to guide the intelligent agent in different operating stages of the air conditioner, the convergence difficulty of the reinforcement learning algorithm in the air conditioning control method is reduced, the stability of the reinforcement learning algorithm is improved, and the feasibility of the reinforcement learning algorithm is increased.

[0059] This embodiment further provides an air conditioner, which may include a controller 300 . Figure 3 FIG. 3 is a schematic block diagram of a controller 300 for an air conditioner according to an embodiment of the present invention. Figure 3 As shown, the controller 300 may include: a processor 310 and a memory 320 , wherein the memory 320 stores a machine executable program 321 , and when the machine executable program 321 is executed by the processor 310 , it is used to implement any of the above-mentioned air conditioner control methods.

[0060] Processor 310 can be a central processing unit (CPU) or a digital processing unit, etc. Processor 310 sends and receives data via a communication interface. Memory 320 is used to store machine-executable programs 321 executed by processor 310. Memory 320 is any medium capable of carrying or storing desired program code in the form of instructions or data structures and accessible by a computer, and can also be a combination of multiple memories 320.

[0061] The machine executable program 321 can be downloaded from a computer-readable storage medium to a corresponding computing / processing device or downloaded via a network (e.g., the Internet, a local area network, a wide area network, and / or a wireless network) and installed in the controller 300. The controller 300 can be used to control the operation of the air conditioner itself, as well as receive and send signals to other devices, for example, to send and receive signals to a mobile terminal.

[0062] It should be noted that the control method of the air conditioner implemented by the air conditioner in this embodiment receives a trigger signal to turn on the air conditioner and obtains the set temperature of the air conditioner, detects the actual temperature of the indoor environment, determines the intelligent agent based on the actual temperature and the set temperature, and controls the air conditioner based on the determined intelligent agent. It can dispatch different reinforcement learning intelligent agents to control the air conditioner according to actual conditions, taking into account user comfort and energy saving, and improving the user experience.

[0063] In addition, the control cycle of the air conditioner is divided into the temperature-trending stage and the temperature-control stage, and two reward functions are set respectively. Two intelligent agents are trained to serve the temperature-trending control and temperature-control control of the air conditioner respectively. The former ensures the rapid cooling / heating of the air conditioner and guarantees the user's comfort as soon as possible. The latter ensures that the air conditioner does not overshoot in the temperature control stage and saves energy as much as possible. Reasonable reward functions are set to guide the intelligent agent in different operation stages of the air conditioner, reduce the convergence difficulty of the reinforcement learning algorithm in the air conditioning control method, improve the stability of the reinforcement learning algorithm, and increase the feasibility of the reinforcement learning algorithm.

[0064] This embodiment also provides a machine-readable storage medium 400, Figure 4 3 is a schematic diagram of a machine-readable storage medium 400 according to an embodiment of the present invention. The machine-readable storage medium 400 stores a machine-executable program 321. When the machine-executable program 321 is executed by the processor 310, the air conditioner control method of any of the above embodiments is implemented.

[0065] The machine-readable storage medium 400 of this embodiment can be an electronic memory such as a flash memory, an EEPROM (Electrically Erasable Programmable Read-Only Memory), an EPROM, a hard disk, or a ROM. The machine-readable storage medium 400 has storage space for a machine-executable program 321 for executing any of the method steps described above. These machine-executable programs 321 can be read from or written to one or more computer program products.

[0066] These computer program products include program code carriers such as hard disks, compact disks (CDs), memory cards, or floppy disks. When the device where the machine-readable storage medium 400 is located runs the machine-executable program 321 , each step of the method described above can be executed.

[0067] In the description of the present embodiment, reference to the terms "one embodiment," "some embodiments," "illustrative embodiments," "example," "specific example," or "some examples" means that the specific features, structures, materials, or characteristics described in conjunction with the embodiment or example are included in at least one embodiment or example of the present invention. In this specification, the exemplary expressions of the above terms do not necessarily refer to the same embodiment or example. Moreover, the specific features, structures, materials, or characteristics described may be combined in any appropriate manner in any one or more embodiments or examples.

[0068] At this point, those skilled in the art will recognize that, although a number of exemplary embodiments of the present invention have been shown and described in detail herein, many other variations or modifications consistent with the principles of the present invention may be directly determined or derived from the disclosure of the present invention without departing from the spirit and scope of the present invention. Therefore, the scope of the present invention should be understood and deemed to cover all such other variations or modifications.

Claims

1. A method for controlling an air conditioner, characterized in that: include: receiving a trigger signal for turning on the air conditioner and obtaining a set temperature of the air conditioner; Detect the actual temperature of the indoor environment; determining an intelligent agent according to the actual temperature and the set temperature; as well as The air conditioner is controlled according to the determined intelligent agent.

2. The method according to claim 1, wherein the step of determining an intelligent agent according to the actual temperature and the set temperature comprises: Determining whether the absolute value of the difference between the actual temperature and the set temperature is greater than a preset threshold; as well as If so, it is determined that the agent is a thermophilic agent. If not, it is determined that the intelligent agent is a temperature control intelligent agent.

3. The method according to claim 2, wherein: The reward function for training the thermophilic agent is: , T is the actual temperature, T set is the set temperature, and abs refers to the absolute value.

4. The method according to claim 3, wherein: The reward function for training the temperature control agent is: , u1, u2, and u3 are respectively the set frequency of the compressor of the air conditioner, the set opening of the electronic expansion valve, and the set speed of the outdoor fan, and α1, α2, and α3 are preset weight values.

5. The method according to claim 4, wherein after the step of determining that the agent is a thermotropic agent, the method further comprises: controlling the air conditioner according to the temperature-trending intelligent agent; Determine abs(TT set )<1 whether it lasts for the first preset time; as well as If so, the temperature control agent is switched to control the air conditioner.

6. The method according to claim 5, wherein: In the case of switching from the temperature-trending agent to the temperature-controlling agent, if abs(TT set )>1 continues for a second preset time, switching to the temperature-trending intelligent agent to control the air conditioner.

7. The method according to claim 4, wherein after the step of determining that the intelligent agent is a temperature control intelligent agent, the method further comprises: controlling the air conditioner according to the temperature control intelligent agent; Determine abs(TT set )>1 whether it lasts for the third preset time period; as well as If so, the temperature-trending intelligent agent is switched to control the air conditioner.

8. The method according to claim 7, wherein: In the case of switching from the temperature control agent to the temperature trending agent, if abs(TT set )<1 continues for a fourth preset time period, switching to the temperature control intelligent body to control the air conditioner.

9. The method according to claim 2, wherein: Training the temperature-trending agent and the temperature-controlling agent both includes inputting state parameters and action parameters, and The state parameters include: indoor humidity, the actual temperature, the set temperature, indoor target humidity, indoor fan speed, indoor fan status, compressor status, compressor operating frequency, outdoor temperature, outdoor unit power, four-way valve status, electronic expansion valve opening, and outdoor fan speed; The operation parameters include: a set frequency of the compressor, a set opening of the electronic expansion valve, and a set speed of the outdoor fan.

10. An air conditioner, characterized in that: include: A controller comprising a memory and a processor, wherein the memory stores a machine executable program, and when the machine executable program is executed by the processor, the control method of the air conditioner according to any one of claims 1 to 9 is implemented.

Images