Device control method and apparatus

Abstract

A device control method and an apparatus, relating to the technical field of device control. The method comprises: acquiring the current operating environment state of a device; when it is determined that the reliability of an environment surrogate model fails to meet a standard, using a default control policy to control the operation of the device; acquiring interaction data between the default control policy and an environment; on the basis of the interaction data, updating a training data set; and, on the basis of the updated training data set, optimizing and adjusting weighting parameters of the environment surrogate model, so as to obtain an environment surrogate model of which the reliability meets the standard. The device control method enables the environment surrogate model to quickly adapt to environmental changes, and ensures that the control policy optimized on the basis of the environment surrogate model will not degrade, thus improving device performance and effects of energy conservation.

Description

Device control method and apparatus

[0001] The present application claims priority to the Chinese patent application No. 202411334151.0, filed on September 24, 2024, entitled "A device control method and apparatus", the whole content of which is incorporated herein by reference. TECHNICAL FIELD

[0002] The present application relates to the technical field of device control, and in particular to a device control method and apparatus. BACKGROUND

[0003] Air conditioners rank first in terms of power consumption among all mainstream home appliances, and most air conditioner manufacturers will make energy saving one of the main selling points of their products. To continuously iterate and optimize the energy saving effect of air conditioners, on the one hand, long-term research and development investment is needed for bottom-layer hardware and software such as compressor heat storage technology and other links, and on the other hand, online control strategies also need to be optimized. Traditional online control strategy optimization is generally based on rule-based algorithms obtained by air conditioner manufacturers through manual debugging in fixed laboratories, which often cannot guarantee the optimality of the performance of online control strategies. The main reason is that there is often a significant difference between customer environments and laboratory environments, resulting in performance degradation of online control strategies in customer environments. SUMMARY

[0004] Embodiments of the present application provide a device control method and apparatus, which can enable an environment agent model to quickly adapt to changes in the environment, ensure that a control strategy optimized based on the environment agent model does not deteriorate, and improve the performance and energy saving effect of the device.

[0005] In a first aspect, the present application provides a device control method, comprising: obtaining an environment state in which a device is currently working (which can be referred to as a current environment state); determining that the reliability of an environment agent model does not meet a standard, the environment agent model being used to predict an environment state at a future time based on an environment state at a current time and a control parameter; determining a target control parameter of the device based on the current environment state and a default control strategy; controlling the device based on the target control parameter; obtaining interaction data between the default control strategy and the environment, the interaction data including the current environment state, a control parameter issued by the default control strategy based on the current environment state, and an environment state at a next time; updating a training data set based on the interaction data, the training data including a plurality of training sample pairs, each training sample pair including an environment state at time t, a control parameter at time t, and an environment state at time t+1; and optimizing and adjusting a weight parameter of the environment agent model based on the updated training data set to obtain an environment agent model with reliable reliability.

[0006] The device control method provided in the application guarantees that the control strategy of the device does not deteriorate obviously by controlling the device by using a default control strategy (such as a control strategy provided by the device when it is shipped from the factory) when the reliability of the environment agent model does not meet the standard (for example, the prediction accuracy of the environment agent model is lower than a set threshold due to a dramatic change in the environment), and then updating the training data set based on the interaction data of the default control strategy and the environment, obtaining new training samples based on the interaction data, updating the training data set by using the new training samples, and training the environment agent model by using the updated training data set to optimize the environment agent model so that the reliability of the environment agent model meets the standard.

[0007] In one possible implementation, one specific implementation of updating the training data set based on the interaction data is as follows: obtaining a new training sample pair based on the interaction data; adding the new training sample pair to the training data set; determining a target sliding window from a candidate sliding window set, the candidate sliding window set including sliding windows of different lengths; selecting the latest N training samples from the training data set based on the target sliding window; and obtaining an updated training data set based on the N training samples.

[0008] The dynamic sliding window algorithm is used to update the training data set, so as to guarantee the rationality of the update of the training data set and improve the adaptation speed of the environment agent model to the environment.

[0009] In another possible implementation, one specific implementation of determining the target sliding window from the candidate sliding window set is as follows: determining the target sliding window based on the probabilities of the sliding windows in the candidate sliding window set, the probability of each sliding window being determined based on feedback information corresponding to the sliding window, the feedback information corresponding to each sliding window indicating the prediction error of the environment agent model trained based on the training samples selected from the training data set by using each sliding window.

[0010] In another possible implementation, the device control method provided in the application further includes: periodically updating the probabilities of the sliding windows in the candidate sliding window set based on the AMAB algorithm.

[0011] The AMAB algorithm is used to quickly select an optimal sliding window length, so as to improve the adaptation speed and accuracy performance of the environment agent model to the environment.

[0012] In another possible implementation, one specific implementation of determining the target control parameter of the device based on the current environment state and the default control strategy is as follows: determining a first control parameter based on the current environment state in which the device is currently operating and the default control strategy; and perturbing the first control parameter within a preset amplitude to obtain the target control parameter.

[0013] The control parameter issued by the default control strategy is randomly perturbed, so as to improve the diversity of the collected training samples and improve the modeling speed of the environment agent model to the environmental change.

[0014] In another possible implementation, the device control method provided by the present application further includes: determining that the environment agent model reliability meets a standard; performing optimization solving based on the current working environment state of the device and the environment agent model to obtain second control parameters; and determining target control parameters based on the second control parameters.

[0015] When the environment agent model reliability meets the standard, i.e., the environment agent model can accurately model the changes of the environment, the online control strategy based on the optimization of the environment agent model is used to control the device, thereby improving the performance and energy saving effect of the device.

[0016] In another possible implementation, one specific implementation of determining the target control parameters based on the second control parameters is: performing optimization correction on the second control parameters based on prior information to obtain the target control parameters, and the prior information includes an optimal proportion of control parameters with correlation in the control parameter set.

[0017] The control parameters issued by the control strategy based on the optimization of the environment agent model are further corrected based on expert prior knowledge, thereby further ensuring the accuracy of the control parameters issued by the control strategy and further improving the performance of the device and reducing the working energy consumption of the device.

[0018] In another possible implementation, the initial environment agent model is trained based on an initial training data set, and the initial training data set is obtained based on the interaction data between the device and the environment during initial startup.

[0019] The training data set is obtained by exploration of the device in a specific customer environment, and the environment agent model is trained based on the training data set. The environment agent model is more suitable for the environment and can more accurately predict the changes of the environment, thereby improving the accuracy of the control strategy and further improving the performance and energy saving effect of the device.

[0020] In another possible implementation, the interaction data between the device and the environment during initial startup includes: a current environment state, third control parameters, and an environment state at a next time; and the third control parameters are obtained based on a preset amplitude of disturbance on the control parameters issued by a default control strategy based on the current environment state.

[0021] In this way, the sample complexity of the environment agent model fitting is greatly reduced while ensuring that the performance does not significantly degrade during sample exploration, thereby solving the cold start problem.

[0022] In another possible implementation, the reliability of the environment agent model is determined based on the accuracy and / or prior correlation of the environment agent model.

[0023] In another possible implementation, the device is an air conditioner; the environment state includes at least one of an indoor temperature, an actual value of an indoor fan rotating speed, an indoor coil temperature, an actual value of a compressor frequency, a compressor exhaust temperature, an actual value of an expansion valve opening degree, a cumulative power, an actual value of an outdoor fan rotating speed, and an outdoor temperature; and the target control parameter includes at least one of a target value of the indoor fan rotating speed, a target value of the compressor frequency, a target value of the expansion valve opening degree, and a target value of the outdoor fan rotating speed.

[0024] In another possible implementation, the environment agent model is a light-weight linear model or a decision tree model, so as to reduce the algorithmic power consumption and enable the environment agent model to be deployed on a terminal device with limited algorithmic power.

[0025] In a second aspect, the present application also provides a device control apparatus, comprising a first acquisition module, a reliability judgment module, a determination module, a control module, a second acquisition module, an update module, and an optimization module, wherein the first acquisition module is configured to acquire an environment state in which a device is currently working (which can be referred to as a current environment state); the reliability judgment module is configured to determine that the reliability of an environment agent model does not meet a standard, the environment agent model being configured to predict an environment state at a future time based on an environment state at a current time; the determination module is configured to determine a target control parameter of the device based on the current environment state and a default control strategy; the control module is configured to control the device based on the target control parameter; the second acquisition module is configured to acquire interaction data between the default control strategy and the environment, the interaction data including the current environment state, a control parameter issued by the default control strategy based on the current environment state, and an environment state at a next time; the update module is configured to update a training data set based on the interaction data, the training data including a plurality of training sample pairs, each training sample pair including an environment state at time t, a control parameter at time t, and an environment state at time t+1; and the optimization module is configured to optimize and adjust weight parameters of the environment agent model based on the updated training data set, to obtain an environment agent model that meets the reliability standard.

[0026] In a possible implementation, the update module is specifically configured to: based on the interaction data, obtain a new training sample pair; add the new training sample pair to the training data set; determine a target sliding window from a candidate sliding window set, the candidate sliding window set including a plurality of sliding windows with different lengths; select the most recent N training samples from the training data set based on the target sliding window; and based on the N training samples, obtain the updated training data set.

[0027] In another possible implementation, one specific implementation of determining the target sliding window from the candidate sliding window set is: determining the target sliding window based on probabilities of the sliding windows in the candidate sliding window set, the probability of each sliding window being determined based on feedback information corresponding to the sliding window, the feedback information corresponding to each sliding window indicating a prediction error of the environment agent model trained based on the training sample pairs selected from the training data set based on each sliding window.

[0028] In another possible implementation, the updating module is further configured to periodically update the probabilities of the candidate sliding windows based on the AMAB algorithm.

[0029] In another possible implementation, the determining module is specifically configured to determine the first control parameter based on the current working environment state of the device and a default control strategy; and perturb the first control parameter within a preset amplitude to obtain the target control parameter.

[0030] In another possible implementation, the determining module is further configured to determine that the environment agent model is reliable; perform optimization and solving based on the current working environment state of the device and the environment agent model to obtain a second control parameter; and determine the target control parameter based on the second control parameter.

[0031] In another possible implementation, the determination of the target control parameter based on the second control parameter is specifically implemented as follows: the second control parameter is optimized and corrected based on prior information to obtain the target control parameter, and the prior information includes an optimal proportion of control parameters with relevance in the control parameter set.

[0032] In another possible implementation, the initial environment agent model is trained based on an initial training data set, and the initial training data set is obtained based on interaction data of the device and the environment during initial booting.

[0033] In another possible implementation, the interaction data of the device and the environment during initial booting includes a current environment state, a third control parameter, and a next time environment state, and the third control parameter is obtained by perturbing a control parameter issued based on the default control strategy based on the current environment state within a preset amplitude.

[0034] In another possible implementation, the reliability of the environment agent model is determined based on the accuracy and / or prior relevance of the environment agent model.

[0035] In another possible implementation, the device is an air conditioner, and the environment state includes at least one of indoor temperature, actual value of indoor fan rotating speed, indoor coil temperature, actual value of compressor frequency, compressor discharge temperature, actual value of expansion valve opening degree, cumulative power, actual value of outdoor fan rotating speed, and outdoor temperature; and the target control parameter includes at least one of target value of indoor fan rotating speed, target value of compressor frequency, target value of expansion valve opening degree, and target value of outdoor fan rotating speed.

[0036] In another possible implementation, the environment agent model is a lightweight linear model or a decision tree model.

[0037] In a third aspect, an embodiment of the present application provides a computing device, comprising a memory and a processor, wherein the memory stores instructions, and when the instructions are executed by the processor, the method described in the first aspect or any possible implementation manner of the first aspect is implemented.

[0038] In a fourth aspect, an embodiment of the present application provides a computer readable storage medium, which stores a computer program, and when the computer program is executed by a processor, the method described in the first aspect or any possible implementation manner of the first aspect is implemented.

[0039] In a fifth aspect, an embodiment of the present application further provides a computer program or a computer program product, which comprises instructions, and when the instructions are executed, the computer executes the method described in the first aspect or any possible implementation manner of the first aspect.

[0040] In a sixth aspect, an embodiment of the present application further provides a chip, comprising at least one processor and a communication interface, wherein the processor is configured to execute the method described in the first aspect or any possible implementation manner of the first aspect. BRIEF DESCRIPTION OF DRAWINGS

[0041] FIG. 1 shows a structural schematic diagram of an air conditioner to which the device control method provided by the embodiment of the present application can be applied;

[0042] FIG. 2 shows an implementation flowchart of the device control method provided by the embodiment of the present application;

[0043] FIG. 3 is a flowchart of a device control method provided by the embodiment of the present application;

[0044] FIG. 4 is a structural schematic diagram of a device control apparatus provided by the embodiment of the present application;

[0045] FIG. 5 is a structural schematic diagram of a computing device provided by the embodiment of the present application. DETAILED DESCRIPTION

[0046] The term “and / or” mentioned in the present document is a description of the association relationship of the associated objects, which means that there can be three relationships, for example, A and / or B, which can represent the following three cases: A exists alone, A and B exist simultaneously, and B exists alone. The symbol “ / ” in the present document represents an or relationship of the associated objects, for example, A / B represents A or B.

[0047] The terms "first" and "second" and the like in the description and in the claims of this patent document are used for distinguishing between similar elements and not necessarily for describing a specific sequential or chronological order. It is to be understood that the terms so used are interchangeable under appropriate circumstances such that the embodiments of the present application described herein are, for example, capable of use in either order. It is to be understood that the terminology used herein is for the purpose of describing particular embodiments only and is not intended to be limiting; the use of the singular is not intended to exclude the plural and vice versa, unless the context clearly indicates otherwise. The terms "comprising", "comprises" and "including" are to be construed as open-ended terms (i.e., meaning "including, but not limited to"), unless the context dictates otherwise. Thus, use of such terms is not intended to exclude other elements from the claims, although the use of such terms in the claims is not intended to exclude other elements from the claims. The use of any of the terms "first", "second", and the like does not imply that the elements so described must be in a given order or be treated as being of obvious spatial, chronological or causal significance unless expressly stated otherwise.

[0048] In the description of the embodiments of the present application, the meaning of "a plurality of" is two or more, for example, a plurality of processing units means two or more processing units, and the like; a plurality of elements means two or more elements, and the like, unless otherwise specified.

[0049] In the description of the embodiments of the present application, the meaning of "a plurality of" is two or more, for example, a plurality of processing units means two or more processing units, and the like; a plurality of elements means two or more elements, and the like, unless otherwise specified.

[0050] In the past few decades, almost all air conditioner manufacturers have only focused on the optimization of refrigeration efficiency at the hardware level, but there is no supporting environmental self-adaptive algorithm to make full use of the hardware performance (such as the household air conditioner often overshoots during the start-up stage, causing energy waste), which also brings new space for air conditioning energy saving. Many air conditioner manufacturers have realized the opportunity of AI-based air conditioning energy saving algorithm in recent years. Data-driven AI algorithms have some unique advantages in air conditioning energy saving: (1) strong adaptability: the optimal control strategy can be adaptively found based on different environments. (2) Strong compatibility: it will not conflict with the existing system framework and is easy to transplant. (3) Weak dependence on experts: even if not a heating and ventilation expert, one can learn better energy saving solutions through AI algorithms and some basic heating knowledge through AI algorithms and data learning.

[0051] However, there are difficulties in replacing traditional air conditioner online control strategies with data-driven AI algorithms: (1) Limited end-side computing power: The memory and computing power of air conditioner chips are limited and cannot support complex models and algorithms. (2) High generalization ability requirement: The algorithm and model are required to quickly adapt and solve the optimal strategy in a non-fully observed and non-steady-state environment. (3) High robustness requirement: The algorithm and model are required to be no worse than the baseline under any environmental conditions and not to fail in temperature control. Related technologies are mostly based on proxy models with single-step strategy benefits, such as Google's use of deep neural networks to fit the energy consumption of large data centers under different operating conditions in 2017, and air conditioner manufacturers using simpler models such as Bayesian linear regression to fit the energy consumption of air conditioners under different operating conditions. The premise of achieving ideal energy saving for these schemes is that these proxy models can prepare the mapping relationship between the issued strategy and energy consumption under different operating conditions.

[0052] For example, related technology one, data center energy saving algorithm based on deep neural network. The core idea is to first collect data from data centers under different operating conditions, and then use the neural network model in the figure to fit the mapping relationship between different operating conditions and energy consumption. When the model accuracy meets the requirements, the trained model can be used to solve more energy-saving strategies.

[0053] However, this scheme has the following problems:

[0054] High computing power requirement: Cannot be applied when the end-side computing power is limited.

[0055] High sample complexity: A large amount of diverse data is required to ensure the accuracy of the model.

[0056] Poor environmental adaptability: If the observed state cannot completely describe the entire system or the external environment changes rapidly, the neural network-based proxy model does not have the ability to adapt quickly, resulting in unreliable strategies based on model optimization.

[0057] No consideration of long-term benefits: Single-step greedy optimization cannot guarantee global optimality, and even if the model accuracy is high, temperature overshoot may still occur.

[0058] Related technology two, air conditioner online control strategy optimization scheme is a rule-based algorithm manually adjusted by HVAC experts in a specific laboratory based on expert experience. The optimization period is usually long, typically several months.

[0059] This scheme has the following problems:

[0060] Poor generalization ability: The control strategy adjusted in the laboratory is often not optimal in a variety of customer environments.

[0061] No consideration of long-term benefits: strategies based on expert experience often cannot support high-complexity strategy optimization, and debugging strategies only optimize single-step benefits, so the optimized strategies are prone to temperature overshoot, resulting in waste of energy.

[0062] The embodiment of the application considers how to accurately model the environment with a small amount of data, and does not significantly degrade compared to the baseline during interaction, laying the foundation for subsequent strategy optimization. And how to make the environment agent model quickly adapt to the change of the environment, so that the performance of the issued strategy does not degrade in the subsequent online learning process.

[0063] Therefore, the embodiment of the application proposes a device control method, which introduces the ideas of safe exploration and dynamic local linearization modeling, so that the lightweight agent model can quickly adapt to changes in the environment, thereby efficiently solving the energy-saving strategy based on the environment agent model.

[0064] The device control method provided by the embodiment of the application can be applied to the control of various devices, and realizes the energy-saving optimization of the device, for example, can realize the energy-saving optimization of network devices (such as routers), energy-saving optimization of air conditioning devices, etc. Especially can be applied to the energy-saving optimization of the control strategy of the air conditioner, the embodiment of the application takes the air conditioner as an example to introduce the specific implementation of the device control method provided by the embodiment of the application, and other devices are similar. The specific implementation can refer to the control method for the air conditioner. For the sake of brevity, the embodiment of the application will not be repeated.

[0065] In order to understand the scheme of the embodiment of the application, the technical terms involved in this paper are first explained.

[0066] Linear model: a machine learning model that assumes that the input and output satisfy a linear mathematical relationship. This model is suitable for local modeling based on data when the data volume is small and the computing power is limited, and has the advantage of stronger explanation compared to the commonly used neural network model, and is easier to integrate into expert priori.

[0067] Safe exploration: the algorithm has no prior knowledge of the environment in the early stage, and needs to ensure that the collected data is diverse while ensuring that the issued strategy does not significantly degrade compared to the baseline, facilitating more accurate modeling of the environment.

[0068] Dynamic sliding window: a machine learning algorithm for non-steady-state environment, which uses the data of the recent time period as the input for model fitting, and dynamically adjusts how much data is selected based on historical feedback.

[0069] FIG. 1 shows a structural schematic diagram of an air conditioner to which the device control method provided by the embodiments of the present application can be applied. As shown in FIG. 1, the air conditioner includes an indoor unit and an outdoor unit, the indoor unit includes an indoor fan and an evaporator, and the outdoor unit includes a compressor, a condenser, an outdoor fan and an expansion valve. A user inputs target temperature, mode, wind force and other user settings required for the air conditioner to adjust through an air conditioner remote control or a control panel on the air conditioner, and the air conditioner responds to the user settings, sends control signals to various components in the air conditioner based on its own control strategy, for example, sends a control signal to the indoor fan to control the rotating speed of the indoor fan, sends a control signal (the control signal is generated according to a control parameter) to the evaporator to control the operation of the evaporator, sends a control signal to the compressor to control the operating frequency of the compressor, sends a control signal to the expansion valve to control the opening degree of the expansion valve, and sends a control signal to the outdoor fan to control the rotating speed of the outdoor fan; so that the air conditioner works to adjust the temperature to the target temperature set by the user.

[0070] With the operation of the air conditioner, the external environment state will also change (for example, with the air conditioner cooling, the temperature of the external environment will decrease) and the control strategy of the air conditioner will adjust the control signals sent to various components according to the current environment state of the environment where the air conditioner is located, so as to achieve effective temperature control and energy optimization of the air conditioner.

[0071] The control parameter of the air conditioner refers to the operating parameter of some components (for example, the compressor, the indoor fan, the expansion valve and the outdoor fan) in the air conditioner, so that the air conditioner works to adjust the temperature to the target temperature required by the user. As shown in FIG. 1, the control quantity (i.e., the control parameter) of the air conditioner includes an indoor fan rotating speed target value (i.e., the indoor fan rotating speed value indicated by the control parameter), a compressor frequency target value, an expansion valve opening degree target value and an outdoor fan rotating speed target value.

[0072] The environment state refers to some state parameters of the environment where the air conditioner works, or some environment state parameters that will affect the control decision of the air conditioner. As shown in FIG. 1, the state quantity (i.e., the environment state) can include indoor temperature, actual value of indoor fan rotating speed (i.e., actual rotating speed of the indoor fan), indoor coil temperature, actual value of compressor frequency, compressor discharge temperature, actual value of expansion valve opening degree, cumulative power, actual value of outdoor fan rotating speed and outdoor temperature.

[0073] In some other examples, the control parameter of the air conditioner can only include part of the control parameter items in the indoor fan rotating speed target value, the compressor frequency target value, the expansion valve opening degree target value and the outdoor fan rotating speed target value. The environment state can also include part of the environment state items in the indoor temperature, the actual value of the indoor fan rotating speed, the indoor coil temperature, the actual value of the compressor frequency, the compressor discharge temperature, the actual value of the expansion valve opening degree, the cumulative power, the actual value of the outdoor fan rotating speed and the outdoor temperature.

[0074] The environmental state can be detected by various sensors, such as indoor temperature, indoor coil temperature, compressor discharge temperature, and outdoor temperature, which can be detected by temperature sensors arranged at corresponding positions. For example, a temperature sensor is arranged in the indoor space where the air conditioner is located to detect the indoor temperature. A temperature sensor is arranged at the indoor coil to detect the temperature of the indoor coil. A temperature sensor is arranged at the discharge port of the compressor to detect the temperature of the gas discharged by the compressor. A temperature sensor is arranged in the outdoor space where the outdoor unit is located to detect the outdoor temperature. For example, the actual value of the speed of the indoor fan and the actual value of the speed of the outdoor fan can be detected by a speed sensor. For example, a speed sensor can be arranged on the indoor fan and the outdoor fan to detect the speed of the indoor fan and the speed of the outdoor fan, respectively, to obtain the actual speed of the indoor fan and the actual speed of the outdoor fan. For example, the actual value of the frequency of the compressor can be detected by a frequency sensor. The actual opening degree of the expansion valve can be detected by a distance sensor. The cumulative power consumption of the air conditioner can be detected by an energy consumption sensor.

[0075] Some environmental states can also be detected without sensors and can be obtained by other means. For example, by communication from other channels. For example, for a working scene where other smart home devices or monitoring systems are installed, the temperature of indoor air can be obtained according to the records of other smart devices, and the temperature of outdoor air can be obtained from an online server according to weather forecasts and other information.

[0076] The device control method provided by the embodiment of the present application is deployed in the air conditioner shown in FIG. 1. Considering that the computing power of the air conditioner is usually limited, the device control method of the embodiment of the present application predicts future environmental changes by using a lightweight environmental agent model, and greatly improves the environmental adaptation speed by using an online strategy optimization framework based on dynamic sliding window local linearization modeling, so as to efficiently solve the optimal control strategy in a dynamic environment, thereby efficiently optimizing and saving energy for the air conditioner.

[0077] FIG. 2 shows a flowchart of an implementation of the device control method provided by the present application. The method can be implemented by the air conditioner shown in FIG. 1.

[0078] First, the interaction data of the air conditioner and the working environment where the air conditioner is located is obtained, and an environmental agent model is modeled according to the interaction data, to obtain an environmental agent model for the specific working environment where the air conditioner is currently located. Compared with the model modeled in the laboratory environment, the prediction of the environmental agent model of the embodiment of the present application is more accurate, and the optimization and solution of the online control strategy based on the environmental agent model are more accurate.

[0079] In order to ensure the diversity of the collected interaction data, and to prevent the performance degradation (e.g. temperature control failure) of the air conditioner during the process of obtaining the interaction data, the embodiments of the present application add appropriate amplitude of perturbation to the baseline control strategy of the air conditioner when the air conditioner is initially started, so as to achieve accurate local modeling of the environment with only a small number of environmental interactions, and no significant degradation compared to the baseline control strategy during the interaction process.

[0080] For example, after the air conditioner is initially started, appropriate amplitude of random perturbation is added to the control parameters issued by the original control strategy of the air conditioner, and then the interaction data between the control strategy and the environment is collected, that is, the control parameters issued by the control strategy are executed to make the air conditioner work, and then the environment is affected to cause the change of the environment state. According to the interaction data, the triple (x t , u t , x t+1 ) data is obtained, wherein x t represents the environment state at time t, u t represents the control parameter issued by the control strategy at time t, and x t+1 represents the environment state at time t+1.

[0081] It should be noted that the original control strategy of the air conditioner refers to the online control strategy provided by the air conditioner factory, which can be any type of control strategy, for example, the control strategy can be a rule-based algorithm based on expert manual debugging.

[0082] The meaning of appropriate amplitude of perturbation is that the perturbation to the control parameters issued by the original control strategy of the air conditioner will not affect the temperature control of the air conditioner, for example, after the control parameters with perturbation are executed, the air conditioner can still control the temperature at the target temperature.

[0083] For example, the appropriate perturbation amplitude of the inner fan speed is ±5, the appropriate perturbation amplitude of the compressor frequency is ±3, the appropriate perturbation amplitude of the expansion valve opening is ±2, and the appropriate perturbation amplitude of the outer fan speed is ±5. Then, according to the control parameters to be issued by the original control strategy of the air conditioner, the target value of the inner fan speed is M1, the target value of the compressor frequency is M2, the target value of the expansion valve opening is M3, and the target value of the outer fan speed is M4. After appropriate amplitude of random perturbation is applied to each control parameter, the target value of the inner fan speed is M1+4, the target value of the compressor frequency is M2-2, the target value of the expansion valve opening is M3+1, and the target value of the outer fan speed is M4-3.

[0084] In this way, by adding appropriate amplitude of random perturbation to the control parameters issued by the original control strategy of the air conditioner, sufficient modeling accuracy and diverse data can be quickly obtained.

[0085] Then, the obtained multiple triple data are used for modeling to obtain an environment agent model, or the obtained multiple triple data are used as a training data set to train the environment agent model to obtain an environment agent model with a satisfactory prediction accuracy.

[0086] The safe exploration is performed until the number of triple data with sufficient modeling accuracy is obtained, and it is assumed that the sufficient number is 90 triple data. For example, a sampling interval of two minutes is used, that is, one triple data (x t , u t , x t+1 ) is obtained every two minutes. Thirty triple data can be collected in one hour, and 90 triple data can be collected in three hours. Then, the random disturbance applied to the control parameter issued by the original control strategy is terminated, and the 90 triple data collected are used to model the environment agent model.

[0087] The environment agent model obtained by modeling can predict the environment state at a future time t+1 based on the environment state at a time t and the control parameter at the time t. In other words, the environment state at the time t and the control parameter at the time t are used as inputs of the environment agent model, and the environment agent model can output the environment state at the time t+1 after reasoning.

[0088] The safe exploration can be used to quickly obtain training samples with diversity and a required number in the absence of prior knowledge of the environment, and the training samples collected are used to train the environment agent model to obtain an environment agent model that is preliminarily trained, thereby solving the cold start problem.

[0089] Considering the limited computing power of the air conditioner, the environment agent model adopts a lightweight model, for example, the environment agent model can be a linear model or a decision tree model, to reduce the computing power overhead and facilitate deployment on the air conditioner.

[0090] The lightweight model refers to a neural network model with a small number of parameters. For example, the environment agent model in the embodiment has a parameter quantity of 10M, and the computing power requirement of the device for deploying the environment agent model is also small, for example, less than 1000 mops. Therefore, the environment agent model provided in the embodiment can be deployed on a terminal device with limited computing power, for example, an air conditioner. The air conditioner can run an online control strategy optimized based on the environment agent model, thereby ensuring the temperature control performance and energy saving effect of the air conditioner.

[0091] After obtaining the trained environment agent model, the current environment state and the environment agent model are used for optimization and solving to obtain the control parameter currently required to be issued by the air conditioner. In other words, an optimization model composed of the environment agent model is used to perform optimization and solving based on the current environment state to obtain an initial solution, and the control parameter currently issued by the air conditioner is determined based on the initial solution.

[0092] An exemplary optimization model includes the following optimization equation set:

[0093] wherein x τ represents the environment state at time τ, u τ represents the control parameter at time τ, x t represents the environment state at time t, u t represents the control parameter at time t, x t+1 represents the environment state at time t+1, Q and R are weight matrices defining a single-step loss function, A and B are weight parameters of an environment agent model fitted based on historical environment, and X is a feasible region of the control parameter.

[0094] By solving the above optimization equation set, an initial solution is obtained. In one example, the initial solution can be directly used as the current control parameter and issued to control the air conditioner. For example, by solving the above optimization equation set, the initial solution is obtained as u τ1 = M5, u τ2 = M6, u τ3 = M7, and u τ4 = M8. The control parameter to be issued by the air conditioner at present is directly determined according to the initial solution as follows: the target value of the inner fan speed is M5, the target value of the compressor frequency is M6, the target value of the expansion valve opening degree is M7, and the target value of the outer fan speed is M8.

[0095] In another example, in order to ensure the quality of the control parameter issued by the air conditioner (i.e., the air conditioner is controlled according to the control parameter, the temperature control of the air conditioner is more stable, and the air conditioner is more energy-saving), after obtaining the initial solution of the optimization equation set, the initial solution is further processed, and the control parameter to be issued by the air conditioner at present is determined according to the processed initial solution.

[0096] Exemplarily, the initial solution is optimized and corrected by expert priori, and the corresponding components in the air conditioner are controlled by the control parameter after the optimization and correction. For example, the expert priori includes an optimal proportion of different control parameters, the control parameter obtained by the initial solution is adjusted to meet the optimal proportion, the initial solution is optimized and adjusted, and the initial solution after the optimization and adjustment is issued as the current control parameter to control the corresponding components of the air conditioner to work. That is, the initial solution is further mapped by using expert experience, so that the control parameter meets the expert priori.

[0097] For example, the expert experience knows that the optimal opening size of the expansion valve and the compressor frequency size are positively correlated, but due to the limited ability of the optimization model, the matching relationship cannot be well modeled, so the expert experience is needed to do a post-mapping on the initial solution obtained by solving the optimization equation, so that the control parameters meet the expert prior. For example, the compressor frequency obtained by solving the model is M6, and the expansion valve opening is M7, but the prior knowledge is that the optimal ratio of the compressor frequency and the expansion valve opening is 1:2, so the expansion valve opening is adjusted to 2M6, which is equivalent to making a correction on the original basis, further improving the air conditioning refrigeration / heating efficiency or energy utilization efficiency.

[0098] The accuracy of the initial solution obtained by the optimization model and the reliability of the environmental agent model are closely related, or in other words, the reliability of the environmental agent model determines the reliability of the initial solution obtained by the optimization model. Therefore, in order to ensure the accuracy of the online control strategy of the air conditioner, when the control parameters need to be issued during the actual operation of the air conditioner, the reliability of the environmental agent model needs to be judged first. When the reliability of the environmental agent model meets the requirements, the current environmental state and the optimization model optimization issuing strategy are used, and then the optimization issuing strategy is corrected based on the expert experience, and the air conditioner is controlled according to the expert experience optimized issuing strategy. When the reliability of the environmental agent model does not meet the requirements, the default control strategy is used to issue the control parameters, so as to guarantee the quality of the online control strategy of the air conditioner and avoid the obvious degradation of the control strategy of the air conditioner.

[0099] The reliability of the environmental agent model can be determined based on the accuracy and / or prior correlation of the environmental agent model. For example, when the prediction accuracy of the environmental agent model is lower than the preset threshold, it is judged that the reliability of the environmental agent model does not meet the requirements. For example, when the prediction accuracy of the environmental agent model is lower than the preset threshold for several steps in a row, it is judged that the reliability of the environmental agent model does not meet the requirements. And / or, when the prior correlation of the environmental agent model does not meet the requirements, it is judged that the reliability of the environmental agent model does not meet the requirements. For example, when the control parameter of the compressor frequency is relatively high, it has positive benefits for temperature reduction, but the temperature predicted by the environmental agent model does not decrease, so it is judged that the prior correlation of the environmental agent model does not meet the requirements, and the reliability of the environmental agent model is judged to be not meet the requirements. For example, when the prediction accuracy of the environmental agent model is lower than the preset threshold, and the prior correlation of the environmental agent model does not meet the requirements, it is judged that the reliability of the environmental agent model does not meet the requirements.

[0100] The default control strategy can be an online control strategy provided by the air conditioner factory, for example, the control strategy is a control strategy based on a proportional integral derivative (PID) control algorithm. Random perturbations of appropriate amplitudes are performed on the basis of the control parameters issued under the default control strategy of the air conditioner, to ensure the diversity of the data. Then, the interaction data of the default control strategy and the environment are collected to obtain new triple data (x t , u t , x t+1 ), the new triple data (x t , u t , x t+1 ) are used to update the training data set, and the updated training data set is used to optimize and train the environment agent model, so that the environment agent model quickly adapts to the changes of the environment and ensures the prediction accuracy of the environment agent model.

[0101] Since the change of the environment is not stable, the working environment of the air conditioner can change sharply, at this time, the reliability of the environment agent model can be insufficient, which can make the optimization strategy based on the environment agent model unreliable. Therefore, the embodiment of the present application proposes an online strategy optimization framework based on dynamic sliding window local linear modeling, so that the environment agent model can quickly adapt to the changes of the environment and can efficiently solve the optimal energy-saving strategy in a dynamic environment.

[0102] For example, when it is determined that the reliability of the environment agent model is not up to standard, the control parameters are issued by using the default control strategy of the air conditioner, the interaction data of the default control strategy and the environment are collected to obtain new triple data, and the training data set is updated based on the new triple data. For example, each new triple data is added to the training data set, and the oldest triple data is deleted, to ensure the stability of the data amount in the training data set. Then, the length of the dynamic sliding window is updated, a plurality of triple data is selected from the training data set based on the updated dynamic sliding window, a training data subset is formed by using the selected plurality of triple data, the parameters of the optimized environment agent model are trained by using the training data subset, so that the environment agent model quickly adapts to the changes of the environment.

[0103] The embodiment of the present application pre-constructs a feasible sliding window set J w ={W0, W1,..., W N} of different lengths, and the different lengths of the sliding windows mean that the time span of the triple data selected from the training data set is different, and the prediction accuracy of the trained environment agent model is also different. How to select the optimal sliding window (also referred to as the sliding window of the optimal length) from the sliding window set determines the prediction accuracy of the optimized environment agent model.

[0104] According to the feedback information (for example, modeling error, that is, the prediction error of the environment agent model trained by using the training data subset composed of the triple data selected from the training data set by using different sliding windows) of different sliding window lengths selected at the past time, the probability of selection of each sliding window in the sliding window set is determined. The better the feedback information of the sliding window (that is, the smaller the modeling error), the greater the probability of selection of the sliding window in the sliding window set, that is, when the environment agent model is optimized and trained, the target sliding window is selected from the sliding window set (the sliding window with the greater probability is more likely to be selected as the target sliding window this time), a plurality of triple data is selected from the training data set according to the target sliding window, the weight parameters of the environment agent model are trained by using the plurality of triple data selected by the target sliding window, and the environment agent model is updated until the environment agent model adapts to the changed environment, and the reliability of the environment agent model in the changed environment meets the standard, and then the environment agent model is optimized.

[0105] In order to further increase the adaptation speed of the environment agent model to the environmental change, the embodiments of the present application update the probability of selection of each sliding window in the sliding window set according to the feedback of the sliding window selected at the past time based on a preset period, for example, every time interval T0, using an adversarial multi-armed bandit (AMAB) algorithm, in other words, the AMAB algorithm is used to dynamically update the length of the sliding window selected in the next time interval (that is, the sliding window with the highest probability), so as to achieve the effect of quickly learning the optimal sliding window length with the accumulation of time, and the algorithm has the guarantee of theoretical optimality. In this way, the optimal sliding window length is quickly selected by using the AMAB algorithm, and the adaptation speed and prediction accuracy of the environment agent model are improved.

[0106] To sum up, the device control method provided in the embodiments of the present application first collects diverse triple data through safe exploration in the initial stage (for example, three hours) of the initial start of the air conditioner, and obtains an environment agent model by modeling based on the collected triple data. In the online strategy control stage, it is first determined whether the current environment agent model is reliable at each moment, if the environment agent model is reliable, the control parameters are optimized and issued based on the environment agent model, otherwise the control parameters are issued based on the default control strategy. Then in the process of interaction between the default control strategy and the environment, interaction data is collected to obtain new triple data, the training data set is updated according to the new triple data, the optimized environment agent model is updated based on the updated training data set, and after it is determined that the reliability of the updated agent model meets the standard, the control parameters are optimized and issued according to the updated environment agent model, and the cycle is repeated. The device control method provided in the embodiments of the present application greatly reduces the complexity of the fitting sample of the environment agent model through safe exploration while ensuring that the performance of the air conditioner does not deteriorate significantly during exploration, solves the cold start problem; through online strategy optimization based on dynamic sliding window local linearization modeling, the environment adaptation speed is greatly improved, the optimal control strategy of the device can be efficiently solved in a dynamic environment, the quality of the optimized strategy of the optimization model is further improved through the hierarchical optimization embedded with expert experience (that is, the initial solution is optimized and corrected through expert priori); the optimal sliding window length is quickly selected through the AMAB algorithm, and the environment adaptation speed and prediction accuracy of the environment agent model are improved.

[0107] FIG. 3 is a flowchart of a device control method provided in the embodiments of the present application. The method can be implemented by a device applying the device control method provided in the embodiments of the present application, for example, an air conditioner. The device control method provided in the embodiments of the present application will be introduced below taking the air conditioner as an example, and the method at least includes steps S301 to S308.

[0108] In step S301, the current environment state is obtained.

[0109] The environment state refers to some state parameters of the working environment of the air conditioner, or some environment state parameters that will affect the control decision of the air conditioner. As shown in FIG. 1, the environment state can include indoor temperature, actual value of indoor fan speed (i.e., actual speed of the indoor fan), indoor coil temperature, actual value of compressor frequency, compressor discharge temperature, actual value of expansion valve opening degree, cumulative power, actual value of outdoor fan speed, and outdoor temperature.

[0110] In some other examples, the environment state can also include some of the environment state items in the indoor temperature, actual value of indoor fan speed, indoor coil temperature, actual value of compressor frequency, compressor discharge temperature, actual value of expansion valve opening degree, cumulative power, actual value of outdoor fan speed, and outdoor temperature.

[0111] The current environment state refers to one or more of the current environment state of the air conditioner, such as indoor temperature, actual value of indoor fan speed, indoor coil temperature, actual value of compressor frequency, compressor discharge temperature, actual value of expansion valve opening degree, cumulative power, actual value of outdoor fan speed, and outdoor temperature at the current time.

[0112] The current environment state can be obtained by various methods, for example, can be directly measured by a sensor, or can be obtained from other channels by communication. For brevity, the specific acquisition method is not described here.

[0113] In step S302, it is determined that the reliability of the environment agent model does not meet the standard, and the environment agent model is used to predict the environment state at the future time based on the environment state at the current time and the control parameter.

[0114] The accuracy of the control strategy based on the environment agent model optimization (also referred to as an optimization model) in the embodiments of the present application is closely related to the reliability of the environment agent model, or in other words, the reliability of the environment agent model determines the reliability of the initial solution obtained by the optimization model optimization. Therefore, in order to ensure the accuracy of the online control strategy of the air conditioner, when the control parameter needs to be issued during the actual operation of the air conditioner, it is necessary to first determine whether the reliability of the environment agent model meets the standard, and then determine the subsequent control strategy of the air conditioner according to whether the reliability of the environment agent model meets the standard.

[0115] Whether the reliability of the environment agent model meets the standard can be determined based on the accuracy and / or prior correlation of the environment agent model. For brevity, the specific method for determining the reliability of the environment agent model is not described here.

[0116] In another example, considering the limited computing power of the air conditioner, the environment agent model adopts a lightweight model, for example, the environment agent model can be a linear model or a decision tree model, etc. Lightweight model, in order to reduce the computing power overhead and facilitate deployment on the air conditioner side.

[0117] In step S303, the target control parameter of the device is determined based on the current environment state and the default control strategy.

[0118] After it is determined that the reliability of the environment agent model does not meet the standard, that is, the accuracy of the solution output by the optimization model is unreliable, the control parameter is issued by using the default control strategy, thereby guaranteeing the quality of the online control strategy of the air conditioner and avoiding the obvious degradation of the control strategy of the air conditioner.

[0119] The default control strategy can be an online control strategy provided by the air conditioner when it is shipped, for example, the control strategy is a control strategy based on a PID control algorithm, or a rule-based algorithm manually debugged by a heating and ventilation expert in a specific laboratory based on expert experience, for example, the current environment state is xt , the rule-based algorithm can calculate x t The corresponding control parameter is determined as the target control parameter.

[0120] The control parameter of the air conditioner refers to the working parameter of controlling some components (such as compressor, indoor fan, expansion valve and outdoor fan) in the air conditioner, so as to adjust the temperature to the target temperature required by the user. As shown in FIG. 1, the control parameter of the air conditioner includes the target value of the indoor fan speed (i.e. the value of the indoor fan speed indicated by the control parameter), the target value of the compressor frequency, the target value of the expansion valve opening degree, and the target value of the outdoor fan speed.

[0121] In some other examples, the control parameter of the air conditioner can also include only part of the control parameter items of the target value of the indoor fan speed, the target value of the compressor frequency, the target value of the expansion valve opening degree, and the target value of the outdoor fan speed.

[0122] In step S304, the equipment is controlled based on the target control parameter.

[0123] After determining the target control parameter, the corresponding components in the air conditioner are controlled to work by using the target control parameter, so as to realize the influence on the external temperature / humidity. For example, the target control parameter includes the compressor frequency M1, the indoor fan speed M2, the expansion valve opening degree M3, and the outdoor fan speed M4, and then the processor of the air conditioner controls the compressor frequency M1, the indoor fan speed M2, the expansion valve opening degree M3, and the outdoor fan speed M4 to work.

[0124] In step S305, the interaction data between the default control strategy and the environment is obtained.

[0125] In the process of working of the air conditioner in the default control strategy, the interaction data between the default control strategy and the environment is collected, that is, the control parameter of the air conditioner and the environment state. According to the interaction data, the triple (x t , u t , x t+1 ) data is obtained, wherein x t represents the environment state at time t, u t represents the control parameter issued by the control strategy at time t, and x t+1 represents the environment state at time t+1.

[0126] In another example, on the basis of the control parameter issued by the default control strategy of the air conditioner, a random disturbance of appropriate amplitude is carried out to ensure the diversity of the data. Then the interaction data between the default control strategy and the environment is collected to obtain new triple data (x t , u t , x t+1), to ensure the diversity of the new triple data, and further ensure the sample diversity of the training data set updated by the new triple, so as to enable the environment agent model to quickly adapt to the change of the environment and ensure the prediction accuracy of the environment agent model.

[0127] In step S306, the training data set is updated based on the interaction data.

[0128] For example, one way to update the training data set can be to add the new triple data to the training data set every time a new triple data is obtained, and delete the oldest triple data, to ensure the stability of the data amount in the training data set. Then the length of the dynamic sliding window is updated, and multiple triple data are selected from the training data set based on the updated dynamic sliding window, a training data subset is formed by the selected multiple triple data, and the parameters of the environment agent model are trained by using the training data subset, so that the environment agent model quickly adapts to the change of the environment.

[0129] The specific method of updating the training data set based on the dynamic sliding window can be referred to the description above, and will not be repeated here for brevity.

[0130] In step S307, the weight parameters of the environment agent model are optimized and adjusted based on the updated training data set, to obtain an environment agent model that meets the reliability requirements.

[0131] After the reliability of the environment agent model meets the requirements, the control strategy optimized based on the environment agent model is used to control the operating parameters of the air conditioner. For example, the current environment state and the optimization model composed of the environment agent model mentioned above are used to optimize and solve, to obtain an initial solution. One way is to directly use the initial solution as the current control parameter for issuing and controlling the air conditioner. Another way is to further process the initial solution after obtaining the initial solution of the optimization equation set, and determine the control parameter to be issued by the air conditioner according to the processed initial solution. For example, the initial solution is optimized and corrected by expert priori, and the corresponding components in the air conditioner are controlled by the optimized and corrected control parameter. For example, the expert priori includes the optimal proportion of different control parameters, the control parameter obtained by the initial solution is adjusted to meet the optimal proportion, to optimize and adjust the initial solution, and then the optimized and adjusted initial solution is issued as the current control parameter to control the corresponding components of the air conditioner to work. That is, the initial solution is further mapped by using the expert experience, so that the control parameter meets the expert priori.

[0132] The initial environment agent model can collect training data sets in a safe exploration manner to model the environment changes. For example, in order to ensure the diversity of the collected interaction data, and to prevent the air conditioner from performance degradation (such as temperature control failure) during the process of obtaining the interaction data, the embodiments of the present application add appropriate amplitude of disturbance to the air conditioner based on the baseline control strategy when the air conditioner is initially started, so as to realize accurate local modeling of the environment with only a small number of environment interactions, and no obvious degradation compared with the baseline control strategy during the interaction process.

[0133] The specific implementation of the safe exploration can be referred to the description above, and will not be repeated here for brevity.

[0134] Based on the same concept as the foregoing embodiment of the device control method, the embodiments of the present application also provide a device control apparatus 400, which can be deployed in a terminal device, such as an air conditioner, to improve the temperature control performance and energy saving effect of the air conditioner. The device control apparatus 400 includes units or modules to implement the steps in the device control method shown in FIGS. 2 and 3.

[0135] FIG. 4 is a structural schematic diagram of a device control apparatus according to an embodiment of the present application. As shown in FIG. 4, the device control apparatus 400 includes a first obtaining module 401, a reliability determining module 402, a determining module 403, a control module 404, a second obtaining module 405, an updating module 406, and an optimizing module 407. The first obtaining module 401 is configured to obtain an environment state in which the device is currently working (which can be referred to as a current environment state). The reliability determining module 402 is configured to determine that the reliability of an environment agent model does not meet a standard, and the environment agent model is configured to predict an environment state at a future time based on an environment state at a current time. The determining module 403 is configured to determine a target control parameter of the device based on the current environment state and a default control strategy. The control module 404 is configured to control the device based on the target control parameter. The second obtaining module 405 is configured to obtain interaction data between the default control strategy and the environment, the interaction data including the current environment state, a control parameter issued by the default control strategy based on the current environment state, and an environment state at a next time. The updating module 406 is configured to update a training data set based on the interaction data, the training data including a plurality of training sample pairs, each training sample pair including an environment state at time t, a control parameter at time t, and an environment state at time t+1. The optimizing module 407 is configured to optimize and adjust weight parameters of the environment agent model based on the updated training data set, to obtain an environment agent model with reliable reliability.

[0136] In a possible implementation, the updating module 406 is specifically configured to: obtain a new training sample pair based on the interaction data; add the new training sample pair to the training data set; determine a target sliding window from a candidate sliding window set, the candidate sliding window set including sliding windows of different lengths; select the latest N training samples from the training data set based on the target sliding window; and obtain an updated training data set based on the N training samples.

[0137] In another possible implementation, a specific implementation of determining the target sliding window from the candidate sliding window set includes: determining the target sliding window based on probabilities of sliding windows in the candidate sliding window set, the probability of each sliding window being determined based on feedback information corresponding to the sliding window, the feedback information corresponding to each sliding window indicating a prediction error of an environmental agent model trained based on training samples selected from the training data set based on each sliding window.

[0138] In another possible implementation, the updating module 406 is further configured to periodically update the probabilities of the sliding windows in the candidate sliding window set based on the AMAB algorithm.

[0139] In another possible implementation, the determining module 403 is specifically configured to: determine a first control parameter based on the current working environment state of the device and a default control strategy; and perturb the first control parameter within a preset amplitude to obtain the target control parameter.

[0140] In another possible implementation, the determining module 403 is further configured to: determine that the environmental agent model is reliable; perform optimization and solving based on the current working environment state of the device and the environmental agent model to obtain a second control parameter; and determine the target control parameter based on the second control parameter.

[0141] In another possible implementation, a specific implementation of determining the target control parameter based on the second control parameter includes: performing optimization and correction on the second control parameter based on prior information to obtain the target control parameter, the prior information including an optimal proportion of control parameters in the control parameter set that have relevance.

[0142] In another possible implementation, the initial environmental agent model is trained based on an initial training data set, and the initial training data set is obtained based on interaction data of the device in initial booting and the environment.

[0143] In another possible implementation, the interaction data of the device in initial booting and the environment includes: a current environment state, a third control parameter, and an environment state at a next time point; and the third control parameter is obtained by perturbing a control parameter issued based on the default control strategy based on the current environment state within a preset amplitude.

[0144] In another possible implementation, the reliability of the environmental agent model is determined based on an accuracy of the environmental agent model and / or prior relevance.

[0145] In another possible implementation, the device is an air conditioner; the environment state includes at least one of an indoor temperature, an actual value of an indoor fan rotating speed, an indoor coil temperature, an actual value of a compressor frequency, a compressor exhaust temperature, an actual value of an expansion valve opening degree, a cumulative power, an actual value of an outdoor fan rotating speed, and an outdoor temperature; and the target control parameter includes at least one of a target value of the indoor fan rotating speed, a target value of the compressor frequency, a target value of the expansion valve opening degree, and a target value of the outdoor fan rotating speed.

[0146] In another possible implementation, the environment agent model is a light-weight linear model or a decision tree model.

[0147] The device control apparatus 400 according to the embodiments of the present application can correspond to performing the methods described in the embodiments of the present application, and the above and other operations and / or functions of the various modules in the device control apparatus 400 are respectively for realizing the corresponding flows of the various methods in FIGS. 2-3, and for brevity, will not be described here again.

[0148] FIG. 5 is a structural schematic diagram of a computing device provided by the embodiments of the present application.

[0149] As shown in FIG. 5, the computing device 500 includes at least one processor 501, a memory 502, and a communication interface 503. The processor 501, the memory 502, and the communication interface 503 are communicatively connected, and can be communicatively connected in a wired (for example, a bus) manner or in a wireless manner. The communication interface 503 is configured to send and / or receive data sent by other devices; the memory 502 stores computer instructions, and the processor 501 executes the computer instructions to execute the methods in the foregoing method embodiments, so as to improve the control performance and energy saving effect of the device.

[0150] It should be understood that, in the embodiments of the present application, the processor 501 can be a central processing unit CPU, and the processor 501 can also be other general-purpose processors, digital signal processors (DSPs), application specific integrated circuits (ASICs), field programmable gate arrays (FPGAs) or other programmable logic devices, discrete gates or transistor logic devices, discrete hardware components, etc. The general-purpose processor can be a microprocessor or any conventional processor, etc.

[0151] The memory 502 can include read-only memory and random access memory, and provide instructions and data to the processor 501. The memory 502 can also include non-volatile random access memory. Optionally, the random storage memory can be a high bandwidth memory (HBM), for example.

[0152] The memory 502 can be a volatile memory or a non-volatile memory, or can include both volatile and non-volatile memories. Among them, the non-volatile memory can be a read-only memory (ROM), a programmable ROM (PROM), an erasable programmable ROM (EPROM), an electrically EPROM (EEPROM), or a flash memory. The volatile memory can be a random access memory (RAM) used as an external cache. By way of example, but not limitation, many forms of RAM are available, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), double data rate synchronous DRAM (DDR SDRAM), enhanced SDRAM (ESDRAM), synchlink DRAM (SLDRAM), and direct rambus RAM (DR RAM).

[0153] It should be understood that the computing device 500 according to the embodiments of the present application can perform the method shown in FIGS. 2-3 of the embodiments of the present application, and the detailed description of the method implemented is described above. For the sake of brevity, it will not be repeated here.

[0154] The embodiments of the present application provide a computer readable storage medium having stored thereon computer program, when the computer instructions are executed by a processor, the above-mentioned method is implemented.

[0155] The embodiments of the present application provide a chip, which includes at least one processor and an interface, the at least one processor determines program instructions or data through the interface; the at least one processor is used to execute the program instructions to implement the above-mentioned method.

[0156] Embodiments of the present application provide a computer program or computer program product comprising instructions which, when executed, cause a computer to perform the above-mentioned method.

[0157] Those skilled in the art should further understand that the units and algorithm steps of the examples described in combination with the embodiments disclosed herein can be realized in electronic hardware, computer software or a combination of both. In order to clearly illustrate the interchangeability of hardware and software, the components and steps of the examples have been described in the above description in general terms. Whether the functions are realized in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to realize the described functions for each specific application, but such implementation should not be considered beyond the scope of the present application.

[0158] The steps of the methods or algorithms described in combination with the embodiments disclosed herein can be implemented in hardware, software executed by a processor, or a combination of both. The software modules can be placed in random access memory (RAM), internal memory, read-only memory (ROM), electrically programmable ROM, electrically erasable programmable ROM, registers, hard disks, removable disks, CD-ROMs, or any other form of storage medium known in the art.

[0159] The above detailed description of the specific implementation is further detailed for the purpose of the present application, technical solutions and beneficial effects. It should be understood that the above description is only a specific implementation of the present application and is not intended to limit the protection scope of the present application. Any modification, equivalent replacement, improvement, etc. made within the spirit and principles of the present application should be included in the protection scope of the present application.

Claims

1. A device control method characterized by, The method comprises: obtaining an environment state in which a device is currently operating; determining that an environment agent model is unreliable, the environment agent model being configured to predict an environment state at a future time based on an environment state at a current time and a control parameter; determining a target control parameter for the device based on the environment state in which the device is currently operating and a default control strategy; controlling the device based on the target control parameter; obtaining interaction data between the default control strategy and the environment, the interaction data comprising the environment state in which the device is currently operating, a control parameter issued by the default control strategy based on the environment state in which the device is currently operating, and an environment state at a next time; updating a training data set based on the interaction data, the training data comprising a plurality of training sample pairs, each training sample pair comprising an environment state at a time t, a control parameter at the time t, and an environment state at a time t+1; optimizing and adjusting weight parameters of the environment agent model based on the updated training data set to obtain a reliable environment agent model.

2. The method of claim 1, wherein, The method further comprises: obtaining a new training sample pair based on the interaction data; adding the new training sample pair to the training data set; determining a target sliding window from a candidate sliding window set, the candidate sliding window set comprising a plurality of sliding windows of different lengths; selecting the most recent N training samples from the training data set based on the target sliding window; obtaining the updated training data set based on the N training samples.

3. The method of claim 2, wherein, The method further comprises: determining the target sliding window based on probabilities of the sliding windows in the candidate sliding window set, the probabilities of the sliding windows being determined based on feedback information corresponding to the sliding windows, the feedback information indicating prediction errors of an environment agent model trained based on training sample pairs selected from the training data set based on the sliding windows.

4. The method of claim 3, wherein, The method further comprises: periodically updating the probabilities of the sliding windows in the candidate sliding window set based on the AMAB algorithm.

5. The method according to any one of claims 1 to 4, characterized in that, The method further comprises: determining a first control parameter based on the environment state in which the device is currently operating and the default control strategy; perturbing the first control parameter within a preset amplitude to obtain the target control parameter.

6. The method according to any one of claims 1 to 5, characterized in that, The method further comprises: determining that the environment agent model is reliable; optimizing and solving a second control parameter based on the environment state in which the device is currently operating and the environment agent model; determining the target control parameter based on the second control parameter.

7. The method of claim 6, wherein, The method further comprises: optimizing and correcting the second control parameter based on prior information to obtain the target control parameter, the prior information comprising an optimal proportion of control parameters in a control parameter set that are correlated.

8. The method according to any one of claims 1 to 7, characterized in that, The initial environment agent model is trained based on an initial training data set, the initial training data set being obtained based on interaction data between the device and the environment during initial booting.

9. The method of claim 8, wherein, The interaction data of the device and the environment in the initial booting includes: an environment state in which the device currently works, a third control parameter, and an environment state at a next time; The third control parameter is obtained based on preset amplitude perturbation on a control parameter issued by the default control strategy based on the environment state in which the device currently works.

10. The method according to any one of claims 1 to 9, characterized in that, The reliability of the environment agent model is determined based on accuracy and / or prior correlation of the environment agent model.

11. The method according to any one of claims 1 to 10, characterized in that, The device is an air conditioner. The environment state includes at least one of indoor temperature, actual value of indoor fan rotating speed, indoor coil temperature, actual value of compressor frequency, compressor discharge temperature, actual value of expansion valve opening degree, cumulative power, actual value of outdoor fan rotating speed, and outdoor temperature. The target control parameter includes at least one of target value of indoor fan rotating speed, target value of compressor frequency, target value of expansion valve opening degree, and target value of outdoor fan rotating speed.

12. The method according to any one of claims 1 to 11, characterized in that, The environment agent model is a lightweight linear model or a decision tree model.

13. An apparatus control device characterized by comprising: The method comprises: The first acquisition module is configured to acquire an environment state in which a device currently works; The reliability determination module is configured to determine that the reliability of the environment agent model is not up to standard, the environment agent model being used to predict an environment state at a future time based on an environment state at a current time; The determination module is configured to determine a target control parameter of the device based on the environment state in which the device currently works and a default control strategy; The control module is configured to control the device based on the target control parameter; The second acquisition module is configured to acquire interaction data of the default control strategy and the environment, the interaction data including the environment state in which the device currently works, a control parameter issued by the default control strategy based on the environment state in which the device currently works, and an environment state at a next time; The update module is configured to update a training data set based on the interaction data, the training data including a plurality of training sample pairs, each training sample pair including an environment state at time t, a control parameter at time t, and an environment state at time t+1; The optimization module is configured to optimize and adjust weight parameters of the environment agent model based on the updated training data set, to obtain the environment agent model with up-to-standard reliability.

14. A computing device comprising a memory and a processor, wherein: The memory stores instructions, when the instructions are executed by the processor, the method of any one of claims 1-12 is implemented.

15. A computer readable storage medium having stored thereon a computer program, characterized in that, The computer program, when executed by the processor, causes the method of any one of claims 1-12 to be implemented.

Images