Air conditioner control method and system, electronic equipment and storage medium

Abstract

The invention discloses an air conditioner control method and system, electronic equipment and a storage medium, and relates to the field of air conditioner control. The method comprises the following steps: determining an energy-saving award coefficient and a comfort award coefficient according to a current time period and a current season; constructing an energy-saving award function according to the energy-saving award coefficient, constructing a comfort award function according to the comfort award coefficient, and constructing an instant award function according to the energy-saving award function, the comfort award function and a punishment mechanism; state data and action data of operation of the air conditioner are obtained, the state data comprise the indoor temperature, the outdoor temperature and the temperature, the mode and the air speed set by a user, and the action data comprise air speed adjustment, mode adjustment and temperature adjustment; and the optimal action of the next state is determined according to the state data, the action data and the instant reward function, and the operation parameters of the air conditioner are adjusted according to the optimal action. By implementing the technical scheme provided by the invention, a more efficient and environment-friendly air conditioner control method is provided.

Description

Technical Field

[0001] The present application relates to the technical field of air conditioning control, and in particular to a control method, system, electronic device and storage medium for an air conditioning. Background Art

[0002] With the continuous expansion of campus scale and the increasing energy consumption of buildings, the energy-saving optimization problem of air-conditioning systems, as the main component of campus building energy consumption, has become increasingly prominent. Traditional air-conditioning systems mainly rely on manual adjustment and simple temperature control logic to achieve basic functional requirements. In order to improve the energy efficiency and user experience of air-conditioning systems, commonly used methods include timed on / off, constant temperature control, remote control, etc.

[0003] However, the above traditional methods still have some shortcomings in actual use. For example, although timed on / off can save energy, it cannot respond to environmental changes and changes in user needs in real time; although constant temperature control is stable, it may lead to excessive energy consumption under extreme weather conditions; although remote control is convenient, it requires active user intervention and lacks automated and intelligent decision-making capabilities.

[0004] Therefore, how to achieve more efficient energy utilization while ensuring user comfort has become the main challenge facing current air-conditioning control technology. Summary of the invention

[0005] The present application provides an air-conditioning control method, system, electronic device and storage medium, which improve the user experience through intelligent decision-making and provide a more efficient, environmentally friendly and humanized control strategy for the air-conditioning system.

[0006] In a first aspect of the present application, a method for controlling an air conditioner is provided, characterized in that the method is applied to an air conditioner control platform, and the method comprises: Determine energy saving bonus coefficient and comfort bonus coefficient according to the current time period and season; An energy-saving reward function is constructed according to the energy-saving reward coefficient, a comfort reward function is constructed according to the comfort reward coefficient, and an instant reward function is constructed according to the energy-saving reward function, the comfort reward function and a penalty mechanism; Acquire status data and action data of the air conditioner operation, wherein the status data includes indoor temperature, outdoor temperature, and temperature, mode and wind speed set by the user, and the action data includes adjustment of wind speed, mode and temperature; An optimal action for the next state is determined according to the state data, the action data and the immediate reward function, and an operating parameter of the air conditioner is adjusted according to the optimal action.

[0007] Optionally, determining the energy-saving bonus coefficient and the comfort bonus coefficient according to the current time period and season includes: The energy saving bonus coefficient is determined by the following formula: η=η0·f(T,S) Where η represents the energy-saving reward coefficient, η0 represents the basic energy-saving reward coefficient, and f(T, S) represents the function for adjusting the energy-saving reward according to the time period T and season S. The comfort bonus factor is determined by the following formula: K=K0·g(T,S) Among them, K represents the comfort reward coefficient, K0 represents the basic comfort reward coefficient, and g(T, S) represents the function of adjusting the comfort reward according to the time period T and season S.

[0008] Optionally, constructing an energy-saving reward function according to the energy-saving reward coefficient and constructing a comfort reward function according to the comfort reward coefficient includes: The energy-saving reward function is constructed as follows: R energy =η(BaseEnergy-ActualEnergy) Among them, R energy represents the energy-saving reward function, η represents the energy-saving reward coefficient, BaseEnergy represents the preset baseline energy consumption, ActualEnergy represents the actual energy consumption of the air conditioner in the current state, The comfort reward function is constructed as follows: R comfort =K·min(∣IndoorTemp-SetTemp∣,Threshold) Among them, R comfort represents the comfort reward function, K represents the comfort reward coefficient, IndoorTemp represents the actual temperature in the room, SetTemp represents the temperature set by the user, Threshold represents the maximum allowable temperature deviation, |IndoorTemp-SetTemp| represents the absolute value of the difference between the actual temperature in the room and the temperature set by the user, and min() represents the minimum value.

[0009] Optionally, constructing an immediate reward function according to the energy-saving reward function, the comfort reward function and the penalty mechanism includes: The penalty mechanism is constructed as follows: R penalty =-μ·max(0,∣IndoorTemp-SetTemp∣-Threshold) Among them, R penaltyrepresents the penalty function, μ is the penalty coefficient, and max() represents the maximum value. The immediate reward function is constructed as follows: R t =R energy +R comfort +R penalty Among them, R t represents the immediate reward function at time t.

[0010] Optionally, determining the optimal action for the next state according to the state data, the action data and the immediate reward function includes: The optimal action for the next state is determined by the following formula: Q(s, a)←Q(s, a)+α[R t +γmax_{a'}Q(s', a')-Q(s, a)] Among them, Q(s, a) represents the Q value of executing action a in state s, α is the learning rate, γ is the discount factor, and R t represents the immediate reward function at time t, and max_{a'}Q(s',a') is the maximum Q value of all possible actions a' in the next state s'.

[0011] Optionally, determining the optimal action for the next state according to the state data, the action data and the immediate reward function, and adjusting the operating parameters of the air conditioner according to the optimal action includes: Calculate the immediate reward value obtained under the current state-action pair according to the immediate reward function; Predict the discounted reward obtained after executing each candidate action in a preset future time period based on the discount factor; Update the Q value of each state-action pair by combining the current state, action, immediate reward value and the discounted reward; The action with the highest Q value is selected as the optimal action for the current state.

[0012] Optionally, the method further includes constructing a long-term reward function according to the immediate reward function and the long-term reward coefficient, specifically including: The long-term reward function is constructed as follows: Among them, R long-term represents the long-term reward function, λ represents the long-term reward coefficient, γ represents the discount factor, and T0 is the time range.

[0013] In a second aspect of the present application, a control system for an air conditioner is provided, characterized in that it includes a coefficient module, a function module, a collection module and an execution module, wherein: A coefficient module configured to determine an energy-saving bonus coefficient and a comfort bonus coefficient according to a current time period and season; A function module configured to construct an energy-saving reward function according to the energy-saving reward coefficient, construct a comfort reward function according to the comfort reward coefficient, and construct an instant reward function according to the energy-saving reward function, the comfort reward function and a penalty mechanism; A collection module configured to obtain status data and action data of the air conditioner operation, wherein the status data includes indoor temperature, outdoor temperature, and temperature, mode and wind speed set by the user, and the action data includes adjustment of wind speed, adjustment of mode and adjustment of temperature; An execution module is configured to determine an optimal action for a next state according to the state data, the action data and the immediate reward function, and adjust operating parameters of the air conditioner according to the optimal action.

[0014] In the third aspect of the present application, an electronic device is provided, including a processor, a memory, a user interface and a network interface, the memory is used to store instructions, the user interface and the network interface are both used to communicate with other devices, and the processor is used to execute the instructions stored in the memory so that the electronic device executes any one of the methods described above.

[0015] In a fourth aspect of the present application, a computer-readable storage medium is provided, wherein the computer-readable storage medium stores instructions, and when the instructions are executed, any of the methods described above is executed.

[0016] In summary, one or more technical solutions provided in the embodiments of the present application have at least the following technical effects or advantages: 1. By introducing energy-saving incentive coefficients and comfort incentive coefficients, two important aspects of energy saving and user experience can be comprehensively considered. The energy-saving incentive coefficient encourages the system to adopt a more energy-saving operation strategy, while the comfort incentive coefficient ensures that users maintain a suitable indoor environment when enjoying air-conditioning services. This balanced strategy helps to achieve efficient use of energy while meeting user needs; 2. By constructing energy-saving reward functions, comfort reward functions, and instant reward functions combined with penalty mechanisms, an intelligent decision-making framework is provided for air-conditioning control. The system can dynamically calculate the optimal air-conditioning operating parameter adjustment strategy based on current environmental conditions (such as indoor and outdoor temperatures, user-set temperatures, modes, and wind speeds) and past operating data (such as motion data); 3. Since the instant reward function dynamically determines the energy-saving reward coefficient and the comfort reward coefficient according to the current time period and season, it has strong adaptability and flexibility. It can automatically adjust the control strategy according to different seasons and different time periods of the day to adapt to the changing external environment and user needs; 4. By precisely controlling the operating parameters of the air conditioner (such as wind speed, mode and temperature), it can provide users with a more comfortable and personalized indoor environment. This not only helps to improve user satisfaction, but also improves the user's life and work efficiency to a certain extent; 5. Since this method fully considers energy-saving factors in the decision-making process, it helps to reduce the energy consumption of air conditioners and reduce carbon emissions. This is of great significance for achieving sustainable development and addressing climate change. BRIEF DESCRIPTION OF THE DRAWINGS

[0017] Figure 1 It is a flowchart of the air conditioner control method disclosed in the embodiment of the present application; Figure 2 It is a module schematic diagram of the control system of the air conditioner disclosed in the embodiment of the present application; Figure 3 It is a structural schematic diagram of an electronic device disclosed in an embodiment of the present application.

[0018] Explanation of the reference numerals: 201, coefficient module; 202, function module; 203, acquisition module; 204, execution module; 301, processor; 302, communication bus; 303, user interface; 304, network interface; 305, memory. DETAILED DESCRIPTION

[0019] In order to enable technicians in this field to better understand the technical solutions in this specification, the technical solutions in the embodiments of this specification will be clearly and completely described below in conjunction with the drawings in the embodiments of this specification. Obviously, the described embodiments are only part of the embodiments of this application, not all of the embodiments.

[0020] In the description of the embodiments of the present application, words such as "for example" or "for example" are used to indicate examples, illustrations or explanations. Any embodiment or design described as "for example" or "for example" in the embodiments of the present application should not be interpreted as being more preferred or more advantageous than other embodiments or designs. Specifically, the use of words such as "for example" or "for example" is intended to present related concepts in a specific way.

[0021] In the description of the embodiments of the present application, the meaning of the term "multiple" refers to two or more. For example, multiple systems refer to two or more systems, and multiple screen terminals refer to two or more screen terminals. In addition, the terms "first" and "second" are used for descriptive purposes only and cannot be understood as indicating or implying relative importance or implicitly indicating the indicated technical features. Thus, the features defined as "first" and "second" may explicitly or implicitly include one or more of the features. The terms "include", "comprise", "have" and their variations all mean "including but not limited to", unless otherwise specifically emphasized.

[0022] This embodiment discloses a method for controlling an air conditioner, which is applied to an air conditioner control platform. Figure 1 is a flow chart of the air conditioner control method disclosed in the embodiment of the present application, such as Figure 1 As shown, the method comprises the following steps: S101. Determine an energy-saving bonus coefficient and a comfort bonus coefficient according to the current time period and season; S102, constructing an energy-saving reward function according to the energy-saving reward coefficient, constructing a comfort reward function according to the comfort reward coefficient, and constructing an instant reward function according to the energy-saving reward function, the comfort reward function and a penalty mechanism; S103, acquiring status data and action data of the air conditioner operation, wherein the status data includes indoor temperature, outdoor temperature, and temperature, mode, and wind speed set by the user, and the action data includes adjustment of wind speed, mode, and temperature; S104: determining an optimal action for the next state according to the state data, the action data and the immediate reward function, and adjusting operating parameters of the air conditioner according to the optimal action.

[0023] The system needs to determine two key coefficients based on the current time (such as daytime, night) and season (such as summer, winter): the energy-saving reward coefficient and the comfort reward coefficient. The energy-saving reward coefficient and the comfort reward coefficient reflect the importance of the system to energy saving and comfort under different time and seasonal conditions. During peak energy demand (such as summer daytime), the energy-saving reward coefficient may be higher, encouraging the system to adopt a more energy-saving operation strategy. Under extreme weather conditions (such as extremely hot or cold weather), the comfort reward coefficient may be higher to ensure that the indoor environment reaches the user's desired comfort level. Based on the energy-saving reward coefficient, an energy-saving reward function is constructed to evaluate the impact of each action (such as adjusting temperature, wind speed or mode) on energy consumption. The better the energy-saving effect, the higher the reward. Based on the comfort reward coefficient, a comfort reward function is constructed to evaluate the impact of each action on the comfort of the indoor environment. The higher the comfort, the higher the reward. In addition to positive rewards, a penalty mechanism can also be introduced to prevent the system from taking overly extreme or inefficient actions. For example, if the system frequently adjusts the temperature or wind speed, it may cause user discomfort or energy waste, and a certain penalty should be given at this time. The state data includes information such as indoor temperature, outdoor temperature, user-set temperature, mode and wind speed. These data are the basis for the system to make decisions. The action data includes possible operations such as wind speed adjustment, mode adjustment and temperature adjustment. The system needs to select the best one among these actions to execute. Combining the energy-saving reward function, the comfort reward function and the penalty mechanism, an immediate reward function is constructed. The immediate reward function is used to evaluate the immediate reward that each possible action can obtain in the current state. According to the immediate reward function, the system selects an optimal action among the possible actions, that is, the action that can maximize the immediate reward. When the optimal action is determined, the system will execute this action and adjust the operating parameters of the air conditioner (such as temperature, wind speed, mode, etc.). This process is dynamic, and the system will continuously adjust and optimize its decision-making process based on the new state data and the immediate reward function to achieve long-term energy saving and comfort goals.

[0024] By determining the energy-saving reward coefficient according to the current time period and season, the operation strategy of the air conditioner can be intelligently adjusted to adapt to different environmental and climatic conditions, thereby reducing unnecessary energy consumption. The construction of the energy-saving reward function enables the system to prioritize energy-saving effects while ensuring comfort, thereby improving the overall energy efficiency. The introduction of the comfort reward coefficient enables the air-conditioning system to intelligently adjust the operating parameters according to the user's preferences such as temperature, mode and wind speed, as well as the current indoor and outdoor temperatures, to achieve the user's desired comfort level. The construction of the comfort reward function further strengthens the system's focus on user comfort, ensuring that the user's comfort experience is not sacrificed while saving energy. By obtaining the state data and action data of the air conditioner operation, the immediate reward function is used to determine the optimal action for the next state. This data-based decision-making method enables the system to continuously learn and optimize its control strategy. Over time, the system can accumulate more operating data and experience, thereby further improving its decision-making ability and control effect. Since this method can dynamically adjust the energy-saving reward coefficient and the comfort reward coefficient according to different time periods and seasons, it has strong flexibility and adaptability.

[0025] The embodiment of the present application senses the changes in the indoor and outdoor environment and user needs in real time, and uses a reinforcement learning algorithm to dynamically adjust the air conditioning operation strategy to minimize energy consumption and maximize indoor comfort. At the same time, it reduces manual intervention, reduces maintenance costs, and improves the overall energy efficiency of the campus air conditioning system.

[0026] Optionally, determining the energy-saving bonus coefficient and the comfort bonus coefficient according to the current time period and season includes: The energy saving bonus coefficient is determined by the following formula: η=η0·f(T,S) Where η represents the energy-saving reward coefficient, η0 represents the basic energy-saving reward coefficient, and f(T, S) represents the function for adjusting the energy-saving reward according to the time period T and season S. The comfort bonus factor is determined by the following formula: K=K0·g(T,S) Among them, K represents the comfort reward coefficient, K0 represents the basic comfort reward coefficient, and g(T, S) represents the function of adjusting the comfort reward according to the time period T and season S.

[0027] The basic energy-saving bonus coefficient is a fixed value that represents the basic energy-saving bonus level of the system when time periods and seasonal factors are not considered. It is usually determined based on the initial design of the system, energy efficiency standards, and expected operating efficiency. The adjustment function f(T, S) is a function that changes according to time periods (T) and seasons (S) and is used to adjust the energy-saving bonus coefficient. Its design takes into account the impact of different time periods and seasons on energy consumption. For example, in summer and winter, energy consumption may increase due to the need for more cooling or heating, so f(T, S) may reduce the energy-saving bonus coefficient accordingly to encourage the system to operate more energy-efficiently. On the contrary, in spring and autumn or certain time periods (such as nighttime), energy consumption may be lower, and f(T, S) may increase the energy-saving bonus coefficient to encourage the system to maintain a higher energy efficiency. The basic comfort bonus coefficient is also a fixed value that represents the basic comfort bonus level of the system when time periods and seasonal factors are not considered. It is usually determined based on the user's expectations of comfort, the initial design of the system, and the expected operating results. The adjustment function g(T, S) is a function that changes according to the time period (T) and season (S) and is used to adjust the comfort reward coefficient. Its design takes into account the impact of different time periods and seasons on user comfort. For example, in summer, users may expect lower indoor temperatures, so g(T, S) may increase the comfort reward coefficient to encourage the system to provide lower indoor temperatures. Conversely, in winter, users may expect higher indoor temperatures, and g(T, S) may also adjust the comfort reward coefficient accordingly.

[0028] By introducing time period T and season S as variables for adjusting energy-saving rewards and comfort rewards, the operation strategy of the air-conditioning system can be controlled more finely. Different time periods and seasons have different effects on the energy consumption and comfort requirements of the air-conditioning system. Therefore, adjusting the reward coefficient according to these variables can make the control strategy more in line with the actual situation. The determination of the energy-saving reward coefficient η takes into account the basic energy-saving reward coefficient and the function f(T, S) adjusted according to the time period T and season S. This means that the system can intelligently adjust the energy-saving strategy according to the current environmental conditions and climatic conditions, thereby further improving the energy-saving effect. The determination of the comfort reward coefficient K also takes into account the basic comfort reward coefficient and the function g(T, S) adjusted according to the time period T and season S. This enables the system to intelligently adjust the operating parameters according to the user's comfort requirements and current environmental conditions to provide the best comfort experience. Through fine control and optimization of the comfort experience, this method can significantly improve the user's satisfaction with the air-conditioning system. Users can set different time periods and season parameters according to their needs and preferences to obtain a more personalized service experience.

[0029] Optionally, constructing an energy-saving reward function according to the energy-saving reward coefficient and constructing a comfort reward function according to the comfort reward coefficient includes: The energy-saving reward function is constructed as follows: R energy =η(BaseEnergy-ActualEnergy) Among them, R energy represents the energy-saving reward function, η represents the energy-saving reward coefficient, BaseEnergy represents the preset baseline energy consumption, ActualEnergy represents the actual energy consumption of the air conditioner in the current state, The comfort reward function is constructed as follows: R comfort =K·min(∣IndoorTemp-SetTemp∣,Threshold) Among them, R comfort represents the comfort reward function, K represents the comfort reward coefficient, IndoorTemp represents the actual temperature in the room, SetTemp represents the temperature set by the user, Threshold represents the maximum allowable temperature deviation, |IndoorTemp-SetTemp| represents the absolute value of the difference between the actual temperature in the room and the temperature set by the user, and min() represents the minimum value.

[0030] BaseEnergy is the preset baseline energy consumption, which indicates the energy consumption level that the air conditioning system should theoretically achieve under the same conditions (such as outdoor temperature, humidity, indoor set temperature, etc.). This value can be based on historical data, experimental data or theoretical calculations. ActualEnergy is the actual energy consumption of the air conditioning system in the current state, which indicates the actual energy consumption value of the air conditioning system in the current operating state. The energy-saving reward function measures the energy-saving effect of the air conditioning system by calculating the difference between the actual energy consumption and the preset baseline energy consumption. If the actual energy consumption is lower than the baseline energy consumption, the difference is positive, indicating energy saving; otherwise, the difference is negative, indicating high energy consumption. The difference is multiplied by the energy-saving reward coefficient to obtain the final energy-saving reward. If η is positive and large, it means that the system attaches great importance to energy-saving effects in the current time period and season; if η is small or negative, it may mean that the system pays more attention to other aspects of performance (such as comfort) under current conditions. The comfort reward function first calculates the absolute value of the difference between the current actual indoor temperature and the user-set temperature as a measure of temperature deviation. This absolute value of the difference is compared with the maximum allowable temperature deviation Threshold, and the smaller value of the two is taken. This is done to prevent the reward value from being too large when the temperature deviation is too large, which may cause the system to over-pursue energy saving at the expense of comfort. Threshold is the maximum temperature deviation allowed, and no comfort reward will be given if it exceeds this value. Multiply the smaller value obtained by the comfort reward coefficient K to get the final comfort reward. If K is positive and large, it means that the system attaches more importance to comfort in the current time period and season; if K is small or negative, it may mean that the system pays more attention to other aspects of performance (such as energy saving) under current conditions.

[0031] The energy-saving reward function achieves a quantitative evaluation of the energy-saving effect by calculating the difference between the preset baseline energy consumption and the actual energy consumption and multiplying it by the energy-saving reward coefficient. This method enables the system to clearly understand the energy-saving effect under the current operation strategy, so as to adjust the control strategy in a targeted manner to further improve energy-saving efficiency. The design of the energy-saving reward function encourages the system to adopt a more energy-saving operation strategy. When the actual energy consumption is lower than the preset baseline energy consumption, the system will receive a positive energy-saving reward, which helps guide the system to minimize energy consumption while ensuring comfort. The comfort reward function achieves a quantitative evaluation of user comfort by calculating the difference between the current actual indoor temperature and the user-set temperature and multiplying it by the comfort reward coefficient, while taking into account the maximum allowable temperature deviation. This method ensures that the system will not sacrifice the user's comfort experience while saving energy. The design of the comfort reward function enables the system to find a balance between energy saving and comfort. When the difference between the indoor temperature and the user-set temperature is within the maximum allowable temperature deviation range, the system will receive a positive comfort reward, which helps guide the system to minimize energy consumption while maintaining comfort.

[0032] Optionally, constructing an immediate reward function according to the energy-saving reward function, the comfort reward function and the penalty mechanism includes: The penalty mechanism is constructed as follows: R penalty =-μ·max(0,∣IndoorTemp-SetTemp∣-Threshold) Among them, R penalty represents the penalty function, μ is the penalty coefficient, and max() represents the maximum value. The immediate reward function is constructed as follows: R t =R energy +R comfort +R penalty Among them, R t represents the immediate reward function at time t.

[0033] The penalty function is intended to impose a penalty on the system when the difference between the actual indoor temperature and the user-set temperature exceeds the maximum allowable temperature deviation. The degree of penalty is determined by the penalty coefficient. The embodiment of the present application uses max(0, |IndoorTemp-SetTemp|-Threshold) to ensure that a penalty is only incurred when the temperature deviation exceeds the Threshold. If the temperature deviation is within the Threshold, the penalty is 0. The instant reward function comprehensively considers energy-saving rewards, comfort rewards and penalty mechanisms, and is used to evaluate the overall performance of the system at time t. The instant reward function is obtained by adding the energy-saving reward function, the comfort reward function and the penalty function. The instant reward function can comprehensively reflect the system's performance in energy saving, comfort and temperature control.

[0034] The instant reward function enables the system to find a balance between energy saving and comfort by comprehensively considering energy-saving rewards and comfort rewards. At the same time, the introduction of the penalty mechanism ensures that the system will not excessively sacrifice the user's comfort while pursuing energy saving. By constructing an instant reward function, the system can automatically calculate and obtain instant rewards based on the current state and operation strategy. This intelligent control method based on reinforcement learning enables the system to continuously learn and optimize its control strategy and improve its intelligence level. The construction of the instant reward function enables the system to more accurately evaluate the user's comfort and energy-saving needs, thereby providing more personalized services. This helps to enhance the user experience and improve user satisfaction. By incentivizing energy-saving behavior and ensuring user comfort, the instant reward function helps to reduce energy consumption and environmental pollution and promote sustainable development. This is of great significance to the current energy crisis and climate change issues facing the world.

[0035] Optionally, determining the optimal action for the next state according to the state data, the action data and the immediate reward function includes: The optimal action for the next state is determined by the following formula: Q(s, a)←Q(s, a)+α[R t +γmax_{a'}Q(s', a')-Q(s, a)] Among them, Q(s, a) represents the Q value of executing action a in state s, α is the learning rate, γ is the discount factor, and R t represents the immediate reward function at time t, and max_{a'}Q(s',a') is the maximum Q value of all possible actions a' in the next state s'.

[0036] Define the state space S as the environmental state of the air conditioning system, including indoor and outdoor temperature, user-set temperature, mode and wind speed, etc. Define the action space A as the operations that the air conditioning system can perform, such as adjusting wind speed, adjusting mode, adjusting temperature, etc. The state space S includes multiple states s, and the action space A includes multiple actions a. Q(s, a) represents the Q value of executing action a in state s. This value gradually approaches the true value through learning, and it reflects the expected return that can be obtained by executing the action in a given state. The learning rate determines the speed at which new information covers old information. The larger the learning rate, the faster the algorithm updates the Q value, but it may also cause the learning process to be unstable. On the contrary, the smaller the learning rate, the slower the algorithm updates the Q value, but the learning process is more stable. The discount factor determines the degree of influence of future returns on current decisions. The closer the discount factor is to 1, the greater the influence of future returns on current decisions; the closer it is to 0, the smaller the influence of future returns on current decisions. This reflects the importance of the algorithm to future returns. max_{a'}Q(s', a') represents the maximum value of the Q value among all possible actions a' in the next state s'. This value represents the expected maximum reward that can be obtained by following the optimal strategy starting from the next state. It reflects the algorithm's prediction of the optimal strategy in the future. Specifically, the algorithm first calculates the difference between the current Q value and the new estimate based on future rewards (i.e., R t +γmax_{a'}Q(s',a')-Q(s,a)), then multiply this difference by the learning rate α, and finally add this result to the current Q value to get the updated Q value. The above formula is based on the idea of ​​"temporal difference learning". Temporal difference learning is a method of updating estimated values ​​using differential information in time. In the Q-learning algorithm, temporal difference learning is reflected in updating the Q value using the difference between the current estimated Q value and the new estimate based on future rewards. This update method enables the algorithm to gradually approach the true optimal strategy because each update is based on a more accurate estimate of future rewards.

[0037] By continuously interacting with the environment, the Q-learning algorithm can adaptively learn the value of each action in different states, that is, the Q value. This allows the system to select the best action based on the current state without knowing the complete model of the environment in advance. The discount factor in the algorithm allows the system to consider the impact of future rewards for long-term planning. This means that when the system chooses the current action, it will consider the potential impact of the action on future states and rewards, which helps to achieve longer-term goals (such as energy saving and comfort balance). By iteratively updating the Q value, the Q-learning algorithm can gradually converge to the optimal strategy. This means that as learning proceeds, the system will become more and more adept at choosing actions that maximize the long-term total reward in a given state. Since Q-learning is model-free, it does not require assumptions about the dynamics of the environment. Therefore, when the environment changes (such as indoor temperature, external weather, etc.), the system can flexibly adjust its strategy to adapt to the new situation. By learning the optimal strategy, the system can more effectively control the operation of the air conditioner, thereby finding the best balance between energy saving and comfort. This helps to improve the overall performance of the system, including energy saving efficiency and user satisfaction.

[0038] Optionally, determining the optimal action for the next state according to the state data, the action data and the immediate reward function, and adjusting the operating parameters of the air conditioner according to the optimal action includes: Calculate the immediate reward value obtained under the current state-action pair according to the immediate reward function; Predict the discounted reward obtained after executing each candidate action in a preset future time period based on the discount factor; Update the Q value of each state-action pair by combining the current state, action, immediate reward value and the discounted reward; The action with the highest Q value is selected as the optimal action for the current state.

[0039] The system needs to calculate the immediate reward value obtained under the current state-action pair based on the current state data (such as indoor temperature, humidity, etc.) and the executed action data (such as the working mode and wind speed of the air conditioner), as well as the pre-defined immediate reward function. This immediate reward value reflects the immediate benefits (such as energy saving effect, comfort improvement, etc.) brought by executing the action in the current state. The system needs to predict the discounted reward that may be obtained in the preset future time period (the so-called "future step") after executing each candidate action. Here, the discount factor plays a key role, which determines the importance of future rewards to the current decision. The closer γ is to 1, the greater the impact of future rewards on the current decision; the closer γ is to 0, the more important the current reward is. When calculating specifically, the system needs to consider all future states that may be reached after executing each candidate action, as well as the immediate reward values ​​that may be obtained in these states, and then multiply these future reward values ​​by the corresponding discount factor to obtain the discounted reward. With the immediate reward value and the discounted reward, the system can update the Q value of each state-action pair. The Q value reflects the expected value of the long-term total reward that can be obtained by executing an action in a given state. The system needs to select the action with the highest Q value as the optimal action in the current state. This optimal action is obtained after considering multiple factors such as the current state, immediate reward, future reward, and learning rate, so it is expected to bring the largest long-term total reward in the current state.

[0040] The system is able to adaptively learn and optimize its control strategy based on the real-time collected state data (such as indoor temperature, humidity, external weather conditions, etc.) and action data (such as the air conditioner's operating mode, wind speed, temperature setting, etc.), as well as the energy saving and comfort information fed back by the immediate reward function. By considering the discount factor, the system not only focuses on the immediate reward of the current action, but also predicts and considers the discounted rewards that may be obtained in the future time period. This helps the system make more long-term decisions to maximize the long-term benefits of energy saving and comfort. The system is able to intelligently select the optimal action in the current state based on the updated Q value. This means that the system can dynamically adjust the operating parameters of the air conditioner according to the real-time situation to cope with different environmental conditions and user needs. By optimizing the control strategy, the system can more effectively manage the energy consumption of the air conditioner while ensuring the comfort of the indoor environment. This helps to reduce unnecessary energy waste, improve energy efficiency, and meet users' expectations for indoor environmental comfort.

[0041] Optionally, the method further includes constructing a long-term reward function according to the immediate reward function and the long-term reward coefficient, specifically including: The long-term reward function is constructed as follows: Among them, R long-termrepresents the long-term reward function, λ represents the long-term reward coefficient, γ represents the discount factor, and T0 is the time range.

[0042] R long-term Represents the long-term reward function, which combines all immediate rewards from time step 1 to time step T0, and takes into account the effect of these rewards decaying over time. The long-term reward coefficient λ is used to adjust the importance of long-term rewards in the entire optimization process. This coefficient allows us to balance the relationship between long-term and short-term rewards according to actual needs. The discount factor γ is used to calculate the value of future rewards at the current point in time. The discount factor usually takes a value between 0 and 1, which reflects the importance we attach to future rewards. A smaller discount factor means that we attach more importance to current rewards, while a larger discount factor attaches more importance to future rewards. T0: time frame, indicating the total number of time steps considered when calculating long-term rewards. The choice of this time frame depends on the specific context and requirements of the problem. R t : An immediate reward function, which represents the reward obtained at time step t. This reward is usually calculated based on the current state or behavior, and may include energy-saving rewards, comfort rewards, and possible penalties. Introducing long-term reward terms in the reward function encourages consideration of long-term system health and user satisfaction while pursuing short-term energy-saving effects.

[0043] This embodiment also discloses a control system for an air conditioner. Figure 2 is a module diagram of the air conditioner control system disclosed in the embodiment of the present application, such as Figure 2 As shown, the system includes a coefficient module 201, a function module 202, a collection module 203 and an execution module 204, wherein: A coefficient module 201 is configured to determine an energy-saving bonus coefficient and a comfort bonus coefficient according to a current time period and season; Function module 202, configured to construct an energy-saving reward function according to the energy-saving reward coefficient, construct a comfort reward function according to the comfort reward coefficient, and construct an instant reward function according to the energy-saving reward function, the comfort reward function and a penalty mechanism; The acquisition module 203 is configured to obtain the state data and action data of the air conditioner operation, wherein the state data includes the indoor temperature, the outdoor temperature, and the temperature, mode and wind speed set by the user, and the action data includes the adjustment of the wind speed, the adjustment of the mode and the adjustment of the temperature; The execution module 204 is configured to determine the optimal action for the next state according to the state data, the action data and the immediate reward function, and adjust the operating parameters of the air conditioner according to the optimal action.

[0044] Optionally, the coefficient module 201 is configured to: The energy saving bonus coefficient is determined by the following formula: η=η0·f(T,S) Where η represents the energy-saving reward coefficient, η0 represents the basic energy-saving reward coefficient, and f(T, S) represents the function for adjusting the energy-saving reward according to the time period T and season S. The comfort bonus factor is determined by the following formula: K=K0·g(T,S) Among them, K represents the comfort reward coefficient, K0 represents the basic comfort reward coefficient, and g(T, S) represents the function of adjusting the comfort reward according to the time period T and season S.

[0045] Optionally, the function module 202 is configured to: The energy-saving reward function is constructed as follows: R energy =η(BaseEnergy-ActualEnergy) Among them, R energy represents the energy-saving reward function, η represents the energy-saving reward coefficient, BaseEnergy represents the preset baseline energy consumption, ActualEnergy represents the actual energy consumption of the air conditioner in the current state, The comfort reward function is constructed as follows: R comfort =K·min(∣IndoorTemp-SetTemp∣,Threshold) Among them, R comfort represents the comfort reward function, K represents the comfort reward coefficient, IndoorTemp represents the actual temperature in the room, SetTemp represents the temperature set by the user, Threshold represents the maximum allowable temperature deviation, |IndoorTemp-SetTemp| represents the absolute value of the difference between the actual temperature in the room and the temperature set by the user, and min() represents the minimum value.

[0046] Optionally, the function module 202 is configured to: The penalty mechanism is constructed as follows: R penalty =-μ·max(0,∣IndoorTemp-SetTemp∣-Threshold) Among them, R penalty represents the penalty function, μ is the penalty coefficient, and max() represents the maximum value. The immediate reward function is constructed as follows: R t =R energy +R comfort +R penalty Among them, R t represents the immediate reward function at time t.

[0047] Optionally, the execution module 204 is configured to: The optimal action for the next state is determined by the following formula: Q(s, a)←Q(s, a)+α[R t +γmax_{a'}Q(s', a')-Q(s, a)] Among them, Q(s, a) represents the Q value of executing action a in state s, α is the learning rate, γ is the discount factor, and R t represents the immediate reward function at time t, and max_{a'}Q(s',a') is the maximum Q value of all possible actions a' in the next state s'.

[0048] Optionally, the execution module 204 is configured to: Calculate the immediate reward value obtained under the current state-action pair according to the immediate reward function; Predict the discounted reward obtained after executing each candidate action in a preset future time period based on the discount factor; Update the Q value of each state-action pair by combining the current state, action, immediate reward value and the discounted reward; The action with the highest Q value is selected as the optimal action for the current state.

[0049] Optionally, the system further comprises a long-term module, wherein the long-term module is configured to: The long-term reward function is constructed as follows: Among them, R long-term represents the long-term reward function, λ represents the long-term reward coefficient, γ represents the discount factor, and T0 is the time range.

[0050] It should be noted that: when the device provided in the above embodiment realizes its function, only the division of the above functional modules is used as an example. In actual application, the above functions can be assigned to different functional modules as needed, that is, the internal structure of the device is divided into different functional modules to complete all or part of the functions described above. In addition, the device and method embodiments provided in the above embodiment belong to the same concept, and the specific implementation process is detailed in the method embodiment, which will not be repeated here.

[0051] This embodiment also discloses an electronic device, referring to Figure 3 The electronic device may include: at least one processor 301 , at least one communication bus 302 , a user interface 303 , a network interface 304 , and at least one memory 305 .

[0052] The communication bus 302 is used to realize the connection and communication between these components.

[0053] The user interface 303 may include a display screen (Display) and a camera (Camera). The optional user interface 303 may also include a standard wired interface and a wireless interface.

[0054] The network interface 304 may optionally include a standard wired interface or a wireless interface (such as a WI-FI interface).

[0055] Among them, the processor 301 may include one or more processing cores. The processor 301 uses various interfaces and lines to connect various parts in the entire server, and executes various functions of the server and processes data by running or executing instructions, programs, code sets or instruction sets stored in the memory 305, and calling data stored in the memory 305. Optionally, the processor 301 can be implemented in at least one hardware form of digital signal processing (Digital Signal Processing, DSP), field programmable gate array (Field-Programmable Gate Array, FPGA), and programmable logic array (Programmable Logic Array, PLA). The processor 301 can integrate one or a combination of a central processing unit (Central Processing Unit, CPU), a graphics processing unit (Graphics Processing Unit, GPU) and a modem. Among them, the CPU mainly processes the operating system, user interface and application programs; the GPU is responsible for rendering and drawing the content to be displayed on the display screen; the modem is used to process wireless communications. It can be understood that the above-mentioned modem may not be integrated into the processor 301, and it can be implemented separately through a chip.

[0056] Among them, the memory 305 may include a random access memory (Random Access Memory, RAM) and may also include a read-only memory (Read-Only Memory). Optionally, the memory 305 includes a non-transitory computer-readable storage medium. The memory 305 can be used to store instructions, programs, codes, code sets or instruction sets. The memory 305 may include a program storage area and a data storage area, wherein the program storage area may store instructions for implementing an operating system, instructions for at least one function (such as a touch function, a sound playback function, an image playback function, etc.), instructions for implementing the above-mentioned various method embodiments, etc.; the data storage area may store data involved in the above-mentioned various method embodiments, etc. The memory 305 may optionally be at least one storage device located away from the aforementioned processor 301. As Figure 3 As shown, the memory 305 as a computer storage medium may include an operating system, a network communication module, a user interface module, and an application program of an air conditioner control method.

[0057] exist Figure 3 In the electronic device shown, the user interface 303 is mainly used to provide an input interface for the user and obtain data input by the user; and the processor 301 can be used to call the application program storing the air conditioner control method in the memory 305. When executed by one or more processors 301, the electronic device executes one or more methods in the above-mentioned embodiments.

[0058] It should be noted that, for the aforementioned method embodiments, for the sake of simplicity, they are all described as a series of action combinations, but those skilled in the art should be aware that the present application is not limited by the order of the actions described, because according to the present application, certain steps can be performed in other orders or simultaneously. Secondly, those skilled in the art should also be aware that the embodiments described in the specification are all preferred embodiments, and the actions and modules involved are not necessarily required for the present application.

[0059] In the above embodiments, the description of each embodiment has its own emphasis. For parts that are not described in detail in a certain embodiment, reference can be made to the relevant descriptions of other embodiments.

[0060] In the several embodiments provided in this application, it should be understood that the disclosed devices can be implemented in other ways. For example, the device embodiments described above are only schematic, such as the division of units, which is only a logical function division. There may be other division methods in actual implementation, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some service interfaces, and the indirect coupling or communication connection of devices or units can be electrical or other forms.

[0061] The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0062] In addition, each functional unit in each embodiment of the present application may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit. The above-mentioned integrated unit may be implemented in the form of hardware or in the form of software functional units.

[0063] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable memory. Based on this understanding, the technical solution of the present application, or the part that contributes to the prior art, or all or part of the technical solution can be embodied in the form of a software product, which is stored in a memory 305 and includes several instructions for a computer device (which can be a personal computer, server or network device, etc.) to perform all or part of the steps of the various embodiments of the present application. The aforementioned memory 305 includes: various media that can store program codes, such as a USB flash drive, a mobile hard disk, a magnetic disk or an optical disk.

[0064] The above is only an exemplary embodiment of the present disclosure and cannot be used to limit the scope of the present disclosure. That is, any equivalent changes and modifications made according to the teachings of the present disclosure are still within the scope of the present disclosure. After considering the disclosure of the specification, those skilled in the art will easily think of other embodiments of the present disclosure. This application is intended to cover any modification, use or adaptation of the present disclosure, which follows the general principles of the present disclosure and includes common knowledge or customary technical means in the technical field that are not recorded in the present disclosure. The description and examples are only regarded as exemplary, and the scope and spirit of the present disclosure are defined by the claims.

Claims

1. A method for controlling an air conditioner, characterized in that: Applied to an air conditioning control platform, the method comprises: Determine energy saving bonus coefficient and comfort bonus coefficient according to the current time period and season; An energy-saving reward function is constructed according to the energy-saving reward coefficient, a comfort reward function is constructed according to the comfort reward coefficient, and an instant reward function is constructed according to the energy-saving reward function, the comfort reward function and a penalty mechanism; Acquire status data and action data of the air conditioner operation, wherein the status data includes indoor temperature, outdoor temperature, and temperature, mode and wind speed set by the user, and the action data includes adjustment of wind speed, mode and temperature; An optimal action for the next state is determined according to the state data, the action data and the immediate reward function, and an operating parameter of the air conditioner is adjusted according to the optimal action.

2. The air conditioner control method according to claim 1, characterized in that: Determining the energy saving bonus coefficient and the comfort bonus coefficient according to the current time period and season includes: The energy saving bonus coefficient is determined by the following formula: η=η0·f(T,S) Where η represents the energy-saving reward coefficient, η0 represents the basic energy-saving reward coefficient, and f(T, S) represents the function for adjusting the energy-saving reward according to the time period T and season S. The comfort bonus factor is determined by the following formula: K=K0·g(T,S) Among them, K represents the comfort reward coefficient, K0 represents the basic comfort reward coefficient, and g(T, S) represents the function of adjusting the comfort reward according to the time period T and season S.

3. The air conditioner control method according to claim 1, characterized in that: The constructing of the energy-saving reward function according to the energy-saving reward coefficient and the constructing of the comfort reward function according to the comfort reward coefficient include: The energy-saving reward function is constructed as follows: R energy =η(BaseEnergy-ActualEnergy) Among them, R energy represents the energy-saving reward function, η represents the energy-saving reward coefficient, BaseEnergy represents the preset baseline energy consumption, ActualEnergy represents the actual energy consumption of the air conditioner in the current state, The comfort reward function is constructed as follows: R comfort =K·min(∣IndoorTemp-SetTemp∣,Threshold) Among them, R comfort represents the comfort reward function, K represents the comfort reward coefficient, IndoorTemp represents the actual temperature in the room, SetTemp represents the temperature set by the user, Threshold represents the maximum allowable temperature deviation, |IndoorTemp-SetTemp| represents the absolute value of the difference between the actual temperature in the room and the temperature set by the user, and min() represents the minimum value.

4. The air conditioner control method according to claim 3, characterized in that: The constructing of an instant reward function according to the energy-saving reward function, the comfort reward function and the penalty mechanism comprises: The penalty mechanism is constructed by the following formula: R penalty =-μ·max(0,∣IndoorTemp-SetTemp∣-Threshold) Among them, R penalty represents the penalty function, μ is the penalty coefficient, and max() represents the maximum value. The immediate reward function is constructed as follows: R t =R energy +R comfort +R penalty Among them, R t represents the immediate reward function at time t.

5. The air conditioner control method according to claim 4, characterized in that: The determining the optimal action for the next state according to the state data, the action data and the immediate reward function comprises: The optimal action for the next state is determined by the following formula: Q(s,a)←Q(s,a)+α[R t +γmax_{a'}Q(s',a')-Q(s,a)] Among them, Q(s, a) represents the Q value of executing action a in state s, α is the learning rate, γ is the discount factor, and R t represents the immediate reward function at time t, and max_{a'}Q(s',a') is the maximum Q value of all possible actions a' in the next state s'.

6. The air conditioner control method according to claim 5, characterized in that: Determining the optimal action for the next state according to the state data, the action data and the immediate reward function, and adjusting the operating parameters of the air conditioner according to the optimal action includes: Calculate the immediate reward value obtained under the current state-action pair according to the immediate reward function; Predict the discounted reward obtained after executing each candidate action in a preset future time period based on the discount factor; Update the Q value of each state-action pair by combining the current state, action, immediate reward value and the discounted reward; The action with the highest Q value is selected as the optimal action for the current state.

7. The air conditioner control method according to claim 4, characterized in that: The method further includes constructing a long-term reward function according to the immediate reward function and the long-term reward coefficient, specifically including: The long-term reward function is constructed as follows: Among them, R long-term represents the long-term reward function, λ represents the long-term reward coefficient, γ represents the discount factor, and T0 is the time range.

8. An air conditioning control system, characterized in that: It includes coefficient module, function module, acquisition module and execution module, among which: A coefficient module configured to determine an energy-saving bonus coefficient and a comfort bonus coefficient according to a current time period and season; A function module configured to construct an energy-saving reward function according to the energy-saving reward coefficient, construct a comfort reward function according to the comfort reward coefficient, and construct an instant reward function according to the energy-saving reward function, the comfort reward function and a penalty mechanism; A collection module configured to obtain status data and action data of the air conditioner operation, wherein the status data includes indoor temperature, outdoor temperature, and temperature, mode and wind speed set by the user, and the action data includes adjustment of wind speed, adjustment of mode and adjustment of temperature; An execution module is configured to determine an optimal action for a next state according to the state data, the action data and the immediate reward function, and adjust operating parameters of the air conditioner according to the optimal action.

9. An electronic device, characterized in that: It includes a processor, a memory, a user interface and a network interface, the memory is used to store instructions, the user interface and the network interface are both used to communicate with other devices, and the processor is used to execute the instructions stored in the memory so that the electronic device executes the method as described in any one of claims 1-7.

10. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores instructions, and when the instructions are executed, the method according to any one of claims 1 to 7 is executed.

Images