Air conditioner control method, device and system, electronic equipment and air conditioner

Abstract

The invention relates to the technical field of air conditioning, and provides an air conditioner control method, device and system, electronic equipment and an air conditioner. The air conditioner control method comprises the steps that input parameters are obtained, the input parameters comprise the current environment state, or the input parameters comprise the current environment state and the air conditioner use requirement of a user, and the environment state comprises the value of an environment factor related to noise in the air conditioner operation process; generating a noise control strategy according to the input parameters and a strategy generation model; according to the noise control strategy and the current environment state, predicting an updated environment state after the noise control strategy is adopted; and in response to user feedback aiming at the updated environment state and / or the noise control strategy, selectively executing the noise control strategy or updating the noise control strategy. By adopting the method and the device, noise control strategies meeting different user requirements can be obtained.

Description

Technical Field

[0001] The present application relates to the field of air conditioning technology, and in particular to an air conditioning control method, device, system, electronic equipment and air conditioning. Background Art

[0002] With the popularization of air conditioners, air conditioners have become a necessity in people's lives. People have also changed from the most basic cooling and heating needs to the pursuit of a higher level of comfort experience.

[0003] Air conditioning noise has always been a pain point in the comfort experience. At present, the general way to detect air conditioning noise is to collect noise decibels and implement noise reduction strategies for the air conditioner based on the noise decibels and the preset noise threshold. However, the method of presetting noise reduction decibels and noise reduction strategies cannot meet the personalized needs of different users and is difficult to meet the actual needs of users. Summary of the invention

[0004] In order to solve the problem that the prior art is difficult to meet the personalized needs of different users, the embodiments of the present application provide an air conditioning control method, device, system, electronic equipment and air conditioner.

[0005] A first aspect of an embodiment of the present application provides an air conditioning control method, the method comprising:

[0006] Acquire input parameters, wherein the input parameters include a current environmental state, or the input parameters also include at least one of a user's air conditioning use demand and an air conditioning setting value, wherein the environmental state includes a value of an environmental factor related to noise during air conditioning operation;

[0007] generating a noise control strategy according to the input parameters and the strategy generation model;

[0008] According to the noise control strategy and the current environmental state, predicting an updated environmental state after adopting the noise control strategy;

[0009] In response to user feedback regarding the updated environmental status and / or the noise control strategy, the noise control strategy is selectively executed or updated.

[0010] In combination with the first aspect, in an implementation of the first aspect, the environmental factor includes at least one of noise decibel, indoor temperature, fan speed, compressor frequency, and air conditioning usage time.

[0011] In combination with the first aspect, in an implementation of the first aspect, the strategy generation model is a deep reinforcement learning model, and generating the noise control strategy according to the input parameters and the strategy generation model includes:

[0012] The input parameters are input into the deep reinforcement learning model to obtain the noise control strategy output by the deep reinforcement learning model, wherein the deep reinforcement learning model takes the input parameters as input and takes the noise control strategy as output.

[0013] In combination with the first aspect, in an implementation of the first aspect, the deep reinforcement learning model is trained in the following manner:

[0014] Obtain a first value estimate for executing a first noise control strategy under an environmental state at a first moment;

[0015] Obtain a second value estimate for executing a second noise control strategy under the environmental state at a second moment;

[0016] Based on the first value estimate and the second value estimate, updating model parameters of the deep reinforcement learning model;

[0017] Among them, the first noise control strategy is the noise control strategy obtained by the deep reinforcement learning model with the first environmental state at the first moment, the user's first air-conditioning usage demand and the first air-conditioning setting value as input, the environmental state at the second moment is the environmental state predicted based on the first environmental state at the first moment and the first action, and the second noise control strategy is the noise control strategy obtained by the deep reinforcement learning model with the second environmental state, the user's second air-conditioning usage demand and the second air-conditioning setting value as input.

[0018] In combination with the first aspect, in an implementation of the first aspect, predicting, according to the noise control strategy and the current environmental state, an updated environmental state after adopting the noise control strategy includes:

[0019] The noise control strategy and the current environmental state are input into a state prediction model to obtain the updated environmental state output by the state prediction model, wherein the state prediction model takes the noise control strategy and the current environmental state as input and takes the updated environmental state as output.

[0020] In combination with the first aspect, in an implementation of the first aspect, the training data used by the state prediction model during the training process includes:

[0021] a first air conditioning setting value within a target time period and a first environmental state corresponding to the first air conditioning setting value;

[0022] Wherein, the first air conditioning setting value includes at least one of a target temperature, a target fan speed, and a target operation mode;

[0023] The first environmental state includes at least one value of actual indoor temperature, actual fan speed, actual noise decibel, actual air conditioning usage time and actual compressor frequency.

[0024] In combination with the first aspect, in an implementation of the first aspect, in response to user feedback on the updated environment state and / or the noise control strategy, selectively executing the noise control strategy or updating the noise control strategy includes:

[0025] If the user is satisfied with the updated environment status and / or the noise control strategy, executing the noise control strategy; and / or,

[0026] In the case that the user is not satisfied with the updated environmental status and / or the noise control strategy, the noise control strategy is regenerated according to the user's improved air-conditioning usage requirements and the current environmental status.

[0027] In combination with the first aspect, in an implementation manner of the first aspect, the method further includes:

[0028] The air-conditioning usage requirement is extracted from the user requirements input by the user based on the language model, and the form of the air-conditioning usage requirement meets the format requirements of the input parameters of the strategy generation model.

[0029] In combination with the first aspect, in an implementation manner of the first aspect, the method further includes:

[0030] Sending the noise control strategy and the updated environment state corresponding to the noise control strategy to a user, and obtaining user feedback on the updated environment state and / or the noise control strategy;

[0031] Among them, the number of the strategy generation models is at least one, and the number of the noise control strategies is at least one.

[0032] A second aspect of an embodiment of the present application provides an air conditioning control device, the air conditioning control device comprising:

[0033] An input parameter acquisition module, used to acquire input parameters, wherein the input parameters include a current environmental state, or the input parameters also include at least one of a user's air conditioning use demand and an air conditioning setting value, wherein the environmental state includes a value of an environmental factor related to noise during air conditioning operation;

[0034] A strategy generation model, used for generating a noise control strategy according to the input parameters;

[0035] A state prediction model, used to predict an updated environmental state after adopting the noise control strategy according to the noise control strategy and the current environmental state;

[0036] A strategy processing module is used to selectively execute the noise control strategy or update the noise control strategy in response to user feedback on the updated environmental status and / or the noise control strategy.

[0037] In conjunction with the second aspect, in an implementation of the second aspect, the method further includes:

[0038] A strategy presentation module, used to send or present the noise control strategy and the updated environmental status to a user;

[0039] A user feedback module is used to obtain user feedback on the updated environment state and / or the noise control strategy.

[0040] According to a third aspect of an embodiment of the present application, there is provided an electronic device, the electronic device comprising:

[0041] Memory, for storing computer instructions;

[0042] The processor is used to call the computer instructions to implement the air conditioning control method described in the first aspect of the embodiment of the present application.

[0043] A fourth aspect of an embodiment of the present application provides an air conditioner, which adopts the air conditioning control method described in the first aspect of the embodiment of the application, or includes the air conditioning control device of the second aspect of the embodiment of the present application, or includes the electronic device of the third aspect of the embodiment of the present application.

[0044] A fifth aspect of the present application provides an air conditioning control system, the air conditioning control system comprising:

[0045] A server adopts the air conditioning control method described in the first aspect of the embodiment of the present application;

[0046] a user device, configured to receive the updated environment status and / or the noise control strategy sent by the server, present the updated environment status and / or the noise control strategy to a user for selection by the user, and

[0047] If the user feedback on the updated environment status and / or the noise control strategy is satisfactory, a control instruction is sent to the air conditioner according to the noise control strategy, or,

[0048] In the case where the user feedback regarding the updated environment status and / or the noise control strategy is not satisfactory, the updated user requirements are sent to the server so that the server regenerates the noise control strategy.

[0049] The method provided in this embodiment, on the one hand, generates a noise control strategy based on input parameters and a strategy generation model, which has higher flexibility than the fixed use of a preset noise control scheme in the prior art; on the other hand, predicts the updated environmental state after the noise control strategy is adopted based on the noise control strategy and the current environmental state, which is conducive to presenting the possible environmental state corresponding to the noise control strategy to the user, thereby providing the user with objective expectations; on another hand, in response to user feedback on the updated environmental state and / or noise control strategy, the noise control strategy is selectively executed or the noise control is updated, so that the noise control strategy can meet the personalized needs of the user and improve the comfort experience of air-conditioning control. BRIEF DESCRIPTION OF THE DRAWINGS

[0050] Figure 1 is a flow chart of an air conditioning control method according to an embodiment of the present application;

[0051] Figure 2 is a flow chart of an air conditioning control method according to an embodiment of the present application;

[0052] Figure 3 A schematic diagram of a training process of a deep reinforcement learning model according to an embodiment of the present application;

[0053] Figure 4 4 is a block diagram of an air conditioning control device according to an embodiment of the present application. DETAILED DESCRIPTION

[0054] Embodiments of the present invention are described in detail below, examples of which are shown in the accompanying drawings, wherein the same or similar reference numerals throughout represent the same or similar elements or elements having the same or similar functions. The embodiments described below with reference to the accompanying drawings are exemplary and are intended to be used to explain the present invention, and should not be construed as limiting the present invention.

[0055] Throughout the specification and claims, the following terms have at least the meanings explicitly associated herein, unless the context dictates otherwise. The meanings identified below do not necessarily limit the terms, but merely provide illustrative examples of the terms.

[0056] In the description of the present invention, the phrase "in one embodiment" does not necessarily refer to the same embodiment, although it may refer to the same embodiment. Similarly, the phrase "in some embodiments" as used herein, when used multiple times, does not necessarily refer to the same embodiment, although it may refer to the same embodiment. As used herein, the term "or" is an inclusive "or" operator and is equivalent to the term "and / or", unless the context clearly dictates otherwise. The term "based on" is not exclusive and allows for additional factors not described, unless the context clearly dictates otherwise. The word "exemplary" means "used as an example, instance or illustration" in this article. Any embodiment described herein as "exemplary" is not necessarily interpreted as being superior or superior to other embodiments. The scope of the present invention is limited only by the scope of the appended claims, and any examples set forth in this specification are not intended to be limiting, but only to set forth some of the many possible embodiments of the claimed invention. The various embodiments provided by the present invention should not be interpreted as limiting the scope of protection of the present invention.

[0057] In the description of the present invention, it should be understood that the terms "center", "longitudinal", "lateral", "length", "width", "thickness", "up", "down", "front", "back", "left", "right", "vertical", "horizontal", "top", "bottom", "inside", "outside", "clockwise", "counterclockwise" and the like indicate orientations or positional relationships based on the orientations or positional relationships shown in the accompanying drawings, and are only for the convenience of describing the present invention and simplifying the description, rather than indicating or implying that the referred device or element must have a specific orientation, be constructed and operated in a specific orientation, and therefore should not be understood as limiting the present invention.

[0058] In addition, the terms "first" and "second" are used for descriptive purposes only and should not be understood as indicating or implying relative importance or implicitly indicating the number of the indicated technical features. Therefore, the features defined as "first" and "second" may explicitly or implicitly include one or more of the features. In the description of the present invention, the meaning of "plurality" is two or more, unless otherwise clearly and specifically defined.

[0059] In the present invention, unless otherwise clearly specified and limited, the terms "installed", "connected", "connected", "fixed" and the like should be understood in a broad sense, for example, it can be a fixed connection, a detachable connection, or an integral connection; it can be a mechanical connection or an electrical connection; it can be a direct connection or an indirect connection through an intermediate medium, it can be the internal connection of two elements or the interaction relationship between two elements. For ordinary technicians in this field, the specific meanings of the above terms in the present invention can be understood according to specific circumstances.

[0060] In the present invention, unless otherwise clearly specified and limited, a first feature being "above" or "below" a second feature may include that the first and second features are in direct contact, or may include that the first and second features are not in direct contact but are in contact through another feature between them. Moreover, a first feature being "above", "above" and "above" a second feature includes that the first feature is directly above and obliquely above the second feature, or simply indicates that the first feature is higher in level than the second feature. A first feature being "below", "below" and "below" a second feature includes that the first feature is directly below and obliquely below the second feature, or simply indicates that the first feature is lower in level than the second feature.

[0061] First, the terms and names involved in the present application are explained as follows. These explanations are intended to facilitate those skilled in the art to understand the embodiments of the present application, but are not intended to limit the understanding of the embodiments of the present application.

[0062] Deep reinforcement learning: Deep reinforcement learning (DRL) is a technology that combines reinforcement learning (RL) with deep learning (DL), aiming to enable the agent to learn the optimal behavior strategy through interaction with the environment.

[0063] The working principle of the deep reinforcement learning model can be summarized as the following steps: Perceiving the environment: The agent obtains the state information of the environment through sensors or interfaces; State representation: The state information usually needs to be preprocessed (such as image scaling, feature extraction, etc.) before being input into the deep neural network; Decision making: The neural network outputs the probability distribution of one or more actions or directly outputs an action based on the input state information; Executing actions: The agent performs corresponding behaviors according to the actions or action probability distributions output by the network; Receiving feedback: After the agent performs an action, it receives immediate rewards from the environment and enters a new state; Updating the model: According to the results of the agent's behavior (rewards and new states), the parameters of the neural network are updated through a certain learning algorithm (such as Q-learning, policy gradient, etc.) to make better decisions in the future.

[0064] The main components of deep reinforcement learning include: Agent: the subject that learns and executes strategies; Environment: the external world where the agent is located, which can provide state information and rewards; State: describes the current status of the environment; Action: the behavior that the agent can perform; Reward: the feedback obtained from the environment after the agent performs an action, which is used to evaluate the quality of the action; Policy: the principle of the agent's selection of actions, which can be deterministic or random; Value Function: evaluates the value of a state or state-action pair, such as Q function or V function; Deep Neural Network: a model used to learn policies or value functions.

[0065] LSTM (Long Short-Term Memory): is a special recurrent neural network (RNN) designed to solve the gradient vanishing or gradient exploding problems encountered by traditional RNNs when processing long sequence data. LSTM introduces a gate mechanism to enable the network to learn long-term dependencies and effectively retain important information.

[0066] Air conditioning noise has always been a pain point in the comfort experience. At present, the general way to detect air conditioning noise is to collect noise decibels and implement noise reduction strategies for the air conditioner based on the noise decibels and the preset noise threshold. However, the method of presetting noise reduction decibels and noise reduction strategies cannot meet the personalized needs of different users and is difficult to meet the actual needs of users.

[0067] For example, in some noise reduction control methods, by obtaining the air conditioning noise and the ambient noise, comparing the difference between the air conditioning noise and the ambient noise with the preset difference, adjusting the speed of the indoor fan and the frequency of the compressor, the air conditioning noise is controlled to be close to the ambient noise. This control strategy set according to the noise difference only considers the noise factor, and does not consider the impact of reducing the wind speed, the compressor frequency and the different feelings of different users between the noise.

[0068] For another example, in the noise control method for fresh air air conditioners, a pre-configured wind speed control strategy is used according to the current ambient noise to adapt the fresh air speed to the current ambient noise to achieve the purpose of reducing noise. This is also based on the objective factors of ambient noise to control the operation of the air conditioner, without considering people's subjective ideas about noise, and the use of preset control strategies cannot meet the different usage needs of different groups of people.

[0069] In this regard, the embodiment of the present application provides an air conditioning control method suitable for meeting the personalized needs of users to improve the user's comfort experience. Figure 1 FIG. 1 is a flow chart of an air conditioning control method according to an embodiment of the present application. Figure 1 , the method includes the following processing steps.

[0070] S100: Obtain input parameters.

[0071] The input parameters include the current environmental status, or the input parameters also include at least one of the user's air conditioning usage requirements and air conditioning setting values.

[0072] Exemplarily, the environmental state refers to the actual state of the air conditioner operation, including the value of the environmental factor related to the noise during the operation of the air conditioner. For example, the environmental factor includes at least one of noise decibel, indoor temperature, fan speed, compressor frequency, and air conditioner use time. The noise decibel value includes the decibel at the air outlet of the air conditioner indoor unit and / or the decibel of the air conditioner outdoor unit.

[0073] Exemplarily, the user's air conditioning usage requirements may include target control parameters of the air conditioner (i.e., which parameters of the air conditioner are desired to be adjusted), for example, at least one of the compressor frequency, the internal fan speed, the internal unit, the external unit, and the like.

[0074] Exemplarily, the air conditioner set value reflects the intervention in the future state of the air conditioner (for example, the next moment after the current moment), for example, including at least one of the target temperature set by the user (affecting the external unit compressor), the target fan speed (affecting the fan speed), and the target operating mode (affecting the operation of the whole machine, for example, cooling mode, heating mode; for example, cooling sleep mode, cooling energy-saving mode, etc.).

[0075] S102: Generate a noise control strategy according to input parameters and a strategy generation model.

[0076] The strategy generation model at least takes input parameters as input and takes noise control strategy as output.

[0077] S104: According to the noise control strategy and the current environmental state, predict an updated environmental state after the noise control strategy is adopted.

[0078] For example, a state prediction model is used to predict the environmental state at the next moment (ie, update the environmental state) based on the noise control strategy at the previous moment and the current environmental state.

[0079] S106: In response to user feedback for updating the environment status and / or the noise control strategy, selectively execute the noise control strategy or update the noise control strategy.

[0080] For example, if the user feedback is satisfactory, the noise control strategy is executed; if the user is dissatisfied, the noise control strategy is updated.

[0081] The method provided in this embodiment, on the one hand, generates a noise control strategy based on input parameters and a strategy generation model, which has higher flexibility than the fixed use of a preset noise control scheme in the prior art; on the other hand, predicts the updated environmental state after the noise control strategy is adopted based on the noise control strategy and the current environmental state, which is conducive to presenting the possible environmental state corresponding to the noise control strategy to the user, thereby providing the user with objective expectations; on another hand, in response to user feedback on the updated environmental state and / or noise control strategy, the noise control strategy is selectively executed or the noise control is updated, so that the noise control strategy can meet the personalized needs of the user and improve the comfort experience of air-conditioning control.

[0082] In addition, when the input parameters include the user's air conditioning usage needs, it is more conducive to the embodiment of the present application to provide a noise control strategy that meets the user's needs, thereby meeting the needs of different users and improving comfort and intelligent experience.

[0083] In addition, when the input parameters include air conditioner setting values, it is beneficial to generate a noise control strategy that better complies with the actual working characteristics of the air conditioner by comprehensively considering the actual operating state of the air conditioner and the ideal operating state of the air conditioner (ie, the state reflected by the user setting values).

[0084] Optionally, in an implementation of this embodiment, in S102, the strategy generation model is a deep reinforcement learning model, and the noise control strategy is generated according to the input parameters and the strategy generation model, including: inputting the input parameters into the deep reinforcement learning model to obtain the noise control strategy output by the deep reinforcement learning model, wherein the deep reinforcement learning model takes the input parameters as input and uses the noise control strategy as output.

[0085] By adopting this implementation method, a noise control strategy corresponding to the current environmental state (or the current environmental state and user needs) can be obtained based on the deep reinforcement learning model.

[0086] It should be noted that the deep reinforcement learning model is part of the Markov Decision Process (MDP). Therefore, in other embodiments, other MDPs may be used to replace the deep reinforcement learning model.

[0087] Exemplarily, the deep reinforcement learning model is trained in the following manner.

[0088] First, obtain the first value valuation of executing the first noise control strategy under the environmental state at the first moment; then, obtain the second value valuation of executing the second noise control strategy under the environmental state at the second moment; then, based on the first value valuation and the second value valuation, update the model parameters of the deep reinforcement learning model. Among them, the first noise control strategy is the noise control strategy obtained by taking the first environmental state at the first moment, the first air conditioning use demand of the user and the first air conditioning setting value (the "first moment" here is the historical moment corresponding to the training data, and the first environmental state, the first air conditioning use demand of the user and the first air conditioning setting value are all historical data) as input, the environmental state at the second moment is the environmental state predicted according to the environmental state at the first moment and the first action, and the second noise control strategy is the noise control strategy obtained by taking the second environmental state at the second moment, the second air conditioning use demand of the user and the second air conditioning setting value (the "second moment" here is the historical moment corresponding to the training data, and the second environmental state, the second air conditioning use demand of the user and the second air conditioning setting value are all historical data) as input by the deep reinforcement learning model. Among them, the second moment is the next moment of the first moment.

[0089] Among them, the first environmental state / the second environmental state includes at least one value of noise decibels, indoor temperature, fan speed, compressor frequency, and air conditioning usage time; the user's first air conditioning usage demand / the user's second air conditioning usage demand includes: at least one of compressor frequency, indoor fan speed, indoor unit, outdoor unit, etc.; the first air conditioning setting value / the second air conditioning setting value includes at least one of target temperature, target fan speed, and target operating mode.

[0090] In this way, the user's air-conditioning usage needs and air-conditioning setting values ​​are taken into account during the model training process, so that the trained deep reinforcement learning model can output a noise control strategy that meets user needs and the air-conditioning's own characteristics with a high probability.

[0091] Among them, illustratively, the LSTM model mentioned below can be used to predict the environmental state at the second moment.

[0092] Optionally, in an implementation method of the embodiment of the present application, in S104, based on the noise control strategy and the current environmental state, an updated environmental state after adopting the noise control strategy is predicted, including: inputting the noise control strategy and the current environmental state into a state prediction model, and obtaining an updated environmental state output by the state prediction model, wherein the state prediction model takes the noise control strategy and the current environmental state as input, and takes the updated environmental state as output.

[0093] Exemplarily, the inventors consider that the operating state of the air conditioner is a time-continuous transient state, and can use the LSTM (Long Short-Term Memory Recursive Neural Network) model to predict the noise value based on the operating parameters of the air conditioner, which is helpful to avoid repeated adjustment of parameters and excessive parameter changes that bring bad experience to users. That is, the state prediction model can be an LSTM (Long Short-Term Memory Recursive Neural Network) model. It should be noted that LSTM is an optimization algorithm of a recurrent neural network (RNN). Optionally, other RNNs can be used as substitutes in other embodiments. For example: basic RNN, RNN variants, LSTM variants, etc.

[0094] By adopting this implementation, the environmental state after the noise control strategy is executed under the current environmental state can be relatively accurately predicted according to the noise control strategy and the current environmental state, which is helpful to provide an intuitive reference for the user.

[0095] Exemplarily, the training data used by the state prediction model during the training process include: a first air conditioner setting value and a first environmental state within a target time period; wherein the first air conditioner setting value includes at least one of a target temperature, a target fan speed, and a target operating mode, and the first environmental state includes at least one of an actual indoor temperature, an actual fan speed, an actual noise decibel, an actual air conditioner usage time, and an actual compressor frequency. Of course, the air conditioner setting value is not limited to this example, and any value that has an impact on the operating state of the air conditioner can be included.

[0096] By using the training data provided in this embodiment and the conventional LSTM model training method, a state prediction model that can relatively accurately predict the environmental state at the next moment can be obtained.

[0097] Optionally, in an implementation of this embodiment, in S106, if the user is satisfied with the updated environmental status and / or noise control strategy, the noise control strategy may be executed. In this way, the air conditioning adjustment may meet the actual needs of the user.

[0098] Optionally, in an implementation of this embodiment, in S106, if the user is not satisfied with the updated environment state and / or noise control strategy, the noise control strategy is regenerated according to the user's improved air conditioning usage requirements and the current environment state. In this way, through cyclic iteration, it is helpful to generate a noise control strategy that meets the user's needs.

[0099] Optionally, in one implementation of the present embodiment, air conditioning usage requirements can be extracted from user requirements input by users based on a language model (for example, a large language model, GPT, Wenxin Yiyan, Doubao, Tongyi Qianwen, or a specially trained language model), and the form of the air conditioning usage requirements conforms to the format requirements of the input parameters of the policy generation model.

[0100] For example, user needs can be voice or text, and the content is natural language, for example: "I feel very hot, but I also feel it is very noisy indoors." The corresponding air conditioning usage needs can be: hot and cold - adjust temperature - compressor frequency; noisy - indoor fan speed, compressor power; indoor - indoor unit.

[0101] Optionally, in an implementation of an embodiment of the present application, before S106, the method may further include: sending the noise control strategy and the updated environmental state corresponding to the noise control strategy to the user, and obtaining user feedback on the updated environmental state and / or the noise control strategy. Among them, the number of strategy generation models is at least one, and the number of noise control strategies is at least one. In other words, input parameters can be input into multiple different strategy generation models to obtain different noise control strategies, and then different noise control strategies can be input into the same state prediction model to obtain environmental states corresponding to different noise control strategies, which are then provided to the user for selection at the same time.

[0102] Figure 2 is a flow chart of an air conditioning control method according to an embodiment of the present application, more specifically, a flow chart of a method for controlling air conditioning noise based on deep reinforcement learning, with reference to Figure 2 , the method includes the following processing steps.

[0103] 1) Information collection.

[0104] Specifically, because the air conditioning noise is directly affected by the frequency of the air conditioning fan and compressor, noise sensors are placed at the air outlet of the air conditioner and the outdoor unit (such as the outdoor unit casing or the outdoor unit fan outlet) to collect noise decibels. At the same time, the air conditioning main control chip uses the air conditioning set value, actual temperature value, fan speed value, compressor power value, air conditioning use time and noise decibel value collected by the noise sensor as current vector information.

[0105] The air conditioner set values ​​include: temperature, which affects the compressor of the external unit; wind speed, which affects the fan speed; and operation mode, which affects the operation of the whole machine. The operation mode can be cooling mode, heating mode, or subdivided into cooling sleep, cooling energy saving and other modes. The state of the air conditioner is a dynamic process, and the air conditioner set value can be regarded as an intervention in the future state of the air conditioner. The air conditioner set value should not be limited to the three examples above. As long as it has an impact on the operation state of the air conditioner, it should be regarded as a variable. LSTM will automatically divide the weights during training according to the degree of influence of the variable on the noise.

[0106] The above vector information is the parameter that has a significant impact on the air conditioning noise. It is not limited to the above example. Other information can be added as a vector. The LSTM model will automatically assign weights to each parameter according to the calculation results.

[0107] In a practical application, the collected information can be transmitted to the cloud platform through the main control chip-gateway-cloud platform method, and the cloud platform records the collected information as a set of vectors st.

[0108] 2) The LSTM model creates the environment state.

[0109] Specifically, the training can be performed based on the air conditioning set value, actual temperature value, fan speed value, compressor power value and noise decibel value obtained by the information collection part in the past period of time T to obtain a trained LSTM model. The noise decibel value, fan speed and indoor temperature contained in a certain time t constitute the environmental state st and are further input into the LSTM model, and the noise decibel value, actual temperature value and fan speed at the time t+1 are predicted and output to constitute the environment st+1. The information at the time t+1 is virtual information predicted by LSTM and will not affect the real environment.

[0110] In this embodiment, the environmental state includes noise decibel value, fan speed and indoor temperature as an example for description. In other embodiments, it can also include compressor frequency, air conditioner use time (indicates how long the air conditioner is used continuously), operation mode, etc.

[0111] The LSTM model is described as follows:

[0112] ①Call the LSTM model:

[0113] When calling the model, determine whether the user has updated the LSTM model: If not, the pre-trained model is called, and the data used by the pre-trained model is the noise judgment standard in the laboratory environment. If it has been updated, the user's personal LSTM model is called.

[0114] The current vector st and the operating state sequence ht-1 in the past period of time are used as input (when t=0, ht-1=0) to predict the current environment state st+1.

[0115] it=σ(Wi[ht-1,st]+bi)

[0116] ft=σ(Wf[ht-1,st]+bf)

[0117] ct=ftct-1+ittanh(Wc[ht-1,st]+bc)

[0118] ot=σ(Wo[ht-1,st]+bo)

[0119] st+1=ottanh(ct)

[0120] Among them, σ is the standard sigmoid function; i, f, o and c are the input gate, forget gate, output gate and memory unit respectively; bi, bf, bo and bc are the bias vectors of the input gate, forget gate, output gate and memory unit respectively; W is the weight matrix between each unit and the gate vector.

[0121] ②Model training update:

[0122] When it is judged that the noise decibel value and indoor temperature prediction deviate too much from the actual value, the n operation information vectors s = (s1, s2, .., sn) after the last model training are used to form a data set, and the LSTM model is trained and updated to adapt to the air conditioner's own state and environmental factors, and the impact of the air conditioner on the noise and indoor temperature is divided based on the length of the time series.

[0123] Short-term time series: Changes in a short period of time will have a greater impact on air conditioner dust accumulation, and the changes need to be memorized or forgotten in a short period of time, such as fan speed and compressor frequency;

[0124] Long-term time series: Changes in values ​​over a long period of time will affect the air conditioning noise, and there is no need to remember or forget the changes in a short period of time, such as the air conditioning usage time;

[0125] Under normal circumstances, short-term time series and long-term time series do not need to be manually divided and labeled. The LSTM model can automatically remember and forget based on the degree of influence of the factor characteristics on air-conditioning noise and indoor temperature.

[0126] Running the information vector s through the LSTM model outputs the result y:

[0127] {y1,y2,...,yn}=LSTM{s1,s2,..,sn}

[0128] Loss function:

[0129] The smaller the loss function, the higher the accuracy of the LSTM model, and the more accurately it can predict the noise generated by the air conditioner and the indoor temperature changes after the state is changed.

[0130] 3) Obtain different noise control strategies based on multiple strategy generation models. That is, multiple different noise control strategies can be obtained based on multiple strategy generation models obtained through training. Of course, in other embodiments, multiple different noise control strategies can also be obtained based on one strategy generation model.

[0131] 4) Predict the operation results of the noise control strategy based on the LSTM model (i.e., predict the environmental state at the next moment)

[0132] 5) Get user feedback. If the user feedback is satisfactory, save the deep reinforcement learning model that the user is satisfied with and update the LSTM model. If the user feedback is dissatisfied and proposes improvement requirements, return to step 3 based on the user requirements.

[0133] Specifically, multiple optimal value function action plans an obtained through multiple deep reinforcement learning models, as well as the corresponding expected noise reduction decibel value, indoor temperature, and fan speed changes can be presented to the user. The user selects the corresponding plan according to his or her own needs. If the user is not satisfied with the given plan and proposes improvement requirements, deep reinforcement learning changes the parameters and continues training; until the user is satisfied with the plan, the deep reinforcement learning model of the corresponding plan will be saved as a preference model, and the parameters in the LSTM model will be updated to increase the prediction accuracy.

[0134] 6) Control the air conditioner based on the local application. Specifically, the cloud platform provides the calculated noise control strategy, and the user selects the appropriate noise control strategy on the APP (i.e., the local application). The APP issues control instructions to the air conditioner according to the noise control strategy selected by the user, and the main control chip of the air conditioner adjusts the air conditioner load to achieve the purpose of reducing noise.

[0135] For example, after the air conditioner has been running for a period of time, the information collection module collects vector information for a period of time and transmits it to the cloud platform to create an LSTM environment. The deep reinforcement learning module calculates several different noise control strategies, and imports the air conditioner parameters corresponding to multiple noise control strategies into the previously created LSTM model to obtain the air conditioner operation results of the noise control strategy that is expected to be implemented (that is, the predicted environmental state). The cloud platform sends the noise control strategy and the predicted air conditioner operation results to be displayed on the user APP. For example: the fan speed in the air conditioner is reduced, the noise is reduced by 30 decibels; or the compressor frequency is reduced, the noise is reduced by 40 decibels, and the indoor temperature is expected to increase by 2 degrees Celsius. When the user feedback is dissatisfied (for example: the user feels that the noise can be slightly higher and a more comfortable indoor temperature has been obtained), the deep reinforcement learning can adjust the noise reduction strategy according to the needs and calculate again. Until the user feels satisfied, the APP issues a control instruction to the air conditioner according to the noise control strategy selected by the user, and the air conditioner main control chip receives the control instruction and adjusts the air conditioner load according to the control instruction. The deep reinforcement learning model saves the preference plan and updates the LSTM model to increase the prediction accuracy.

[0136] Figure 3 Schematic diagram of the training process of a deep reinforcement learning model according to an embodiment of the present application. Figure 3 Explain the training process of the deep reinforcement learning model.

[0137] In this embodiment, branches can be composed of different deep reinforcement learning models with multiple different strategies, for example: a different-strategy model that uses reducing noise decibel values ​​as a behavioral strategy and user comfort as a target strategy; a same-strategy model that uses reducing noise decibel values ​​as a behavioral strategy and a target strategy; a DQN network that encourages exploratory behavior, etc.

[0138] In this embodiment, refer to Figure 2 , the training process is as follows.

[0139] ① Based on user needs, a deep reinforcement learning model is used to simulate the air conditioning setting value and the current environmental state st (for example, fan speed, compressor frequency, and air conditioning usage time) to obtain an action a.

[0140] For example, the user's needs can be converted into the user's air conditioning usage needs through the GPT model or by directly setting the function. For example, the conversion results are: hot and cold - adjusting temperature - compressor frequency; noisy - internal fan speed, compressor power; indoor and outdoor - internal unit, external unit, etc.

[0141] Regarding action a, for example, if my demand is "I feel very hot, but I also feel that it is very noisy indoors", then this action a can be: lower the air conditioner temperature setting, increase the compressor frequency, and lower the internal fan speed. In the embodiment of this application, all parameters related to noise and user experience and controllable by the air conditioner can be included in action a, including but not limited to internal and external fan speeds, compressor frequency, air conditioner set temperature, etc.

[0142] ② Combine action a with the current environment st as the input of the LSTM model, and LSTM predicts the noise, indoor temperature, and fan speed for a period of time t+1 in the future as the state st+1. The length of each period can be customized as needed, for example, 10 minutes.

[0143] ③ The deep reinforcement learning model scores the returned function reward R(st,a,st+1) with the target strategy π as the standard γ(γ∈[0,1]), and the action value function can be expressed as:

[0144] Qπ(st,at)=E[Rt+γ·Rt+1+γ2·Rt+2+...+γn·Rt+n|St=st,At=at]

[0145] Among them, Rt+n is the reward at time t+n predicted by the LSTM model based on the current state st and action a.

[0146] The target strategy π can be a weighted combination of one or more of the action a noise, the indoor temperature value, or the user's habit matching degree (comfort).

[0147] ④ Input st, the user's air conditioning usage demand and the air conditioning setting value into the deep reinforcement learning model to obtain the action at. The action at is predicted by the LSTM model to obtain a value valuation qt and a new environment st+1. Then input the new environment st+1, the user's air conditioning usage demand and the air conditioning setting value into the deep reinforcement learning model to obtain a new action at+1 and a new valuation qt+1. It can be considered that the new environment st+1 is the target obtained based on the target strategy π, which generates an error between the target and the current value:

[0148] L(w)=1 / 2[Q(st,at;w)-(rt+γ·Q(st+1,at+1;w))]2

[0149] Update the current model parameter w:

[0150]

[0151] ⑤ The deep reinforcement learning model is trained based on the value score and behavior strategy of the target strategy π (which can be the same as or different from the target strategy. Different deep reinforcement learning models have different target strategies and behavior rewards) to obtain multiple optimal value function action plans an.

[0152] In this embodiment, the training of LSTM and deep reinforcement learning are both carried out on the cloud platform. When the training is completed, the model is saved on the cloud platform, and the energy management solution obtained by the deep reinforcement learning training is transmitted back to the user's smart device APP through the cloud platform-gateway-smart device APP.

[0153] Figure 4 is a block diagram of an air conditioning control device according to an embodiment of the present application. Figure 4 , the air conditioning control device includes the following modules.

[0154] The input parameter acquisition module is used to acquire input parameters, which also include at least one of the user's air conditioning usage requirements and air conditioning setting values, wherein the environmental state includes the value of environmental factors related to noise during air conditioning operation.

[0155] A strategy generation model is used to generate noise control strategies based on input parameters.

[0156] A state prediction model is used to predict the updated environmental state after the noise control strategy is adopted based on the noise control strategy and the current environmental state;

[0157] The strategy processing module is used to selectively execute the noise control strategy or update the noise control strategy in response to user feedback for updating the environment status and / or the noise control strategy.

[0158] The air-conditioning control device provided in this embodiment, on the one hand, generates a noise control strategy based on input parameters and a strategy generation model, which has higher flexibility than the fixed use of a preset noise control scheme in the prior art; on the other hand, based on the noise control strategy and the current environmental state, the updated environmental state after the noise control strategy is adopted is predicted, which is conducive to presenting the possible environmental state corresponding to the noise control strategy to the user, thereby providing the user with objective expectations; on another hand, in response to user feedback on the updated environmental state and / or noise control strategy, the noise control strategy is selectively executed or the noise control is updated, so that the noise control strategy can meet the personalized needs of the user and improve the comfort experience of air-conditioning control.

[0159] Optionally, in one implementation of this embodiment, the air conditioning control module also includes: a strategy display module, used to send or present the noise control strategy and updated environmental status to the user; a user feedback module, used to obtain user feedback on the updated environmental status and / or noise control strategy.

[0160] For detailed descriptions of each module in this embodiment, please refer to the description in the previous method embodiment, which will not be repeated here.

[0161] An embodiment of the present application also provides an electronic device, which includes a memory and a processor, wherein the memory is used to store computer instructions; the processor is used to call computer instructions to implement the method provided by the embodiment of the present application (for example, air conditioning control method, model training method).

[0162] Exemplarily, the electronic device may be a chip, a circuit board with an integrated chip, or hardware including a circuit board. For example, the electronic device may be a user terminal device.

[0163] An embodiment of the present application also provides an air conditioner, which adopts the air conditioner control method or model training method provided by the embodiment of the present application, or the air conditioner is integrated with the electronic device provided by the embodiment of the present application.

[0164] The embodiment of the present application also provides an air conditioning control system, the air conditioning control system comprising:

[0165] A server adopts the air conditioning control method provided in the embodiment of the present application;

[0166] The user device is used to receive the updated environment state and / or noise control strategy sent by the server, and present the updated environment state and / or noise control strategy to the user for the user to select. And, if the user feedback on the updated environment state and / or noise control strategy is satisfactory, send a control instruction to the air conditioner according to the noise control strategy, or, if the user feedback on the updated environment state and / or noise control strategy is dissatisfied, send the updated user demand to the server so that the server regenerates the noise control strategy.

[0167] In addition, the air conditioning control system may also include an air conditioner, and the main control chip of the air conditioner is used to receive the control instruction sent by the user equipment and adjust the corresponding load according to the control instruction. For example, when the control instruction includes the load to be controlled (for example, the compressor frequency, the fan speed) and the adjustment amplitude, the corresponding adjustment is performed according to the control instruction.

[0168] In the above-mentioned embodiments of the present application, the description of each embodiment has its own emphasis. For parts that are not described in detail in a certain embodiment, please refer to the relevant description of other embodiments. The steps shown in the relevant flow chart can be executed in a computer system such as a set of computer executable instructions, and although the logical order is shown in the flow chart, in some cases, the steps shown or described can be executed in an order different from that here. In other words, the order of steps described in the foregoing embodiments is only an example, and it is also within the scope of protection of the embodiments of the present application to reasonably adjust the order of steps based on the content of the embodiments of the present application.

[0169] The sequence of serial numbers or introduction of the embodiments of the present application is for description only and does not represent the superiority or inferiority of the embodiments.

[0170] In the description of this specification, the description with reference to the terms "one embodiment", "some embodiments", "example", "specific example", or "some examples" etc. means that the specific features, structures, materials or characteristics described in conjunction with the embodiment or example are included in at least one embodiment or example of the present invention. In this specification, the schematic representations of the above terms do not necessarily refer to the same embodiment or example. Moreover, the specific features, structures, materials or characteristics described may be combined in any one or more embodiments or examples in a suitable manner. In addition, those skilled in the art may combine and combine different embodiments or examples described in this specification.

[0171] Although the embodiments of the present invention have been shown and described above, it is to be understood that the above embodiments are exemplary and are not to be construed as limitations of the present invention. A person skilled in the art may change, modify, replace and vary the above embodiments within the scope of the present invention.

Claims

1. An air conditioning control method, characterized in that: The method comprises: Acquire input parameters, wherein the input parameters include a current environmental state, or the input parameters also include at least one of a user's air conditioning use demand and an air conditioning setting value, wherein the environmental state includes a value of an environmental factor related to noise during air conditioning operation; generating a noise control strategy according to the input parameters and the strategy generation model; According to the noise control strategy and the current environmental state, predicting an updated environmental state after adopting the noise control strategy; In response to user feedback regarding the updated environmental status and / or the noise control strategy, the noise control strategy is selectively executed or updated.

2. The air conditioning control method according to claim 1, characterized in that: The environmental factors include at least one of noise decibels, indoor temperature, fan speed, compressor frequency, and air conditioning usage time.

3. The air conditioning control method according to claim 1, characterized in that: The strategy generation model is a deep reinforcement learning model, and the noise control strategy is generated according to the input parameters and the strategy generation model, including: The input parameters are input into the deep reinforcement learning model to obtain the noise control strategy output by the deep reinforcement learning model, wherein the deep reinforcement learning model takes the input parameters as input and takes the noise control strategy as output.

4. The air conditioning control method according to claim 3, characterized in that: The deep reinforcement learning model is trained in the following way: Obtain a first value estimate for executing a first noise control strategy under an environmental state at a first moment; Obtain a second value estimate for executing a second noise control strategy under the environmental state at a second moment; Based on the first value estimate and the second value estimate, updating model parameters of the deep reinforcement learning model; Among them, the first noise control strategy is the noise control strategy obtained by the deep reinforcement learning model with the first environmental state at the first moment, the user's first air-conditioning usage demand and the first air-conditioning setting value as input, the environmental state at the second moment is the environmental state predicted based on the first environmental state at the first moment and the first action, and the second noise control strategy is the noise control strategy obtained by the deep reinforcement learning model with the environmental state at the second moment, the user's second air-conditioning usage demand and the second air-conditioning setting value as input.

5. The air conditioning control method according to claim 1, characterized in that: The step of predicting, based on the noise control strategy and the current environmental state, an updated environmental state after adopting the noise control strategy comprises: The noise control strategy and the current environmental state are input into a state prediction model to obtain the updated environmental state output by the state prediction model, wherein the state prediction model takes the noise control strategy and the current environmental state as input and takes the updated environmental state as output.

6. The air conditioning control method according to claim 5, characterized in that: The training data used by the state prediction model during the training process includes: a first air conditioning setting value and a first environmental state within a target time period; Wherein, the first air conditioning setting value includes at least one value of a target temperature, a target fan speed, and a target operation mode; The first environmental state includes at least one value of actual indoor temperature, actual fan speed, actual noise decibel, actual air conditioning usage time and actual compressor frequency.

7. The air conditioning control method according to claim 1, characterized in that: The selectively executing the noise control strategy or updating the noise control strategy in response to user feedback regarding the updated environmental state and / or the noise control strategy includes: If the user is satisfied with the updated environment status and / or the noise control strategy, executing the noise control strategy; and / or, In the case that the user is not satisfied with the updated environmental status and / or the noise control strategy, the noise control strategy is regenerated according to the user's improved air-conditioning usage requirements and the current environmental status.

8. The air conditioning control method according to claim 1, characterized in that: The method further comprises: The air-conditioning usage requirement is extracted from the user requirements input by the user based on the language model, and the form of the air-conditioning usage requirement meets the format requirements of the input parameters of the strategy generation model.

9. The air conditioning control method according to claim 1, characterized in that: The method further comprises: Sending the noise control strategy and the updated environment state corresponding to the noise control strategy to a user, and obtaining user feedback on the updated environment state and / or the noise control strategy; Among them, the number of the strategy generation models is at least one, and the number of the noise control strategies is at least one.

10. An air conditioning control device, characterized in that: The air conditioning control device comprises: An input parameter acquisition module, used to acquire input parameters, wherein the input parameters include a current environmental state, or the input parameters also include at least one of a user's air conditioning use demand and an air conditioning setting value, wherein the environmental state includes a value of an environmental factor related to noise during air conditioning operation; A strategy generation model, used for generating a noise control strategy according to the input parameters; A state prediction model, used to predict an updated environmental state after adopting the noise control strategy according to the noise control strategy and the current environmental state; A strategy processing module is used to selectively execute the noise control strategy or update the noise control strategy in response to user feedback on the updated environmental status and / or the noise control strategy.

11. The air conditioning control device according to claim 10, characterized in that: Also includes: A strategy presentation module, used to send or present the noise control strategy and the updated environmental status to a user; A user feedback module is used to obtain user feedback on the updated environment state and / or the noise control strategy.

12. An electronic device, characterized in that: The electronic device comprises: Memory, for storing computer instructions; A processor, configured to call the computer instructions to implement the air conditioning control method according to any one of claims 1 to 9.

13. An air conditioner, characterized in that: The air conditioner adopts the air conditioner control method as described in any one of claims 1 to 9, or includes the air conditioner control device as described in claim 10 or 11, or includes the electronic device as described in claim 12.

14. An air conditioning control system, characterized in that: The air conditioning control system comprises: A server, using the air conditioning control method according to any one of claims 1 to 9; a user device, configured to receive the updated environment status and / or the noise control strategy sent by the server, present the updated environment status and / or the noise control strategy to a user for selection by the user, and If the user feedback on the updated environment status and / or the noise control strategy is satisfactory, a control instruction is sent to the air conditioner according to the noise control strategy, or, In the case where the user feedback regarding the updated environment status and / or the noise control strategy is not satisfactory, the updated user requirements are sent to the server so that the server regenerates the noise control strategy.

Images