Equipment control method and device

Abstract

The invention provides an equipment control method and device. The method comprises the following steps: acquiring a current working environment state of equipment; determining that the reliability of the environment agent model does not reach the standard, and adopting a module control strategy to control equipment to work; obtaining interaction data of the default control strategy and the environment; updating the training data set based on the interaction data; and based on the updated training data set, carrying out optimization adjustment on weight parameters of the environment agent model to obtain the environment agent model with the reliability reaching the standard. According to the equipment control method provided by the invention, the environment agent model can quickly adapt to the change of the environment, the control strategy optimized based on the environment agent model is not degraded, and the performance and the energy-saving effect of the equipment are improved.

Description

Technical Field

[0001] The present application relates to the field of equipment control technology, and in particular to an equipment control method and device. Background Art

[0002] Air conditioners are among the most power-hungry household appliances, and most air conditioner manufacturers use energy saving as one of the main selling points of their products. To continuously iterate and optimize the energy-saving effect of air conditioners, it is necessary to invest long-term R&D in multiple links such as the underlying hardware and software, such as compressor thermal energy storage technology, and also to optimize online control strategies. Traditional online control strategy optimization is generally based on rule-based algorithms manually debugged by air conditioner manufacturers' experts in fixed laboratories, which often cannot guarantee the optimality of online control strategy performance. The main reason is that there are often significant differences between the customer environment and the laboratory environment, resulting in performance degradation of the online control strategy in the customer environment. Summary of the invention

[0003] The embodiments of the present application provide a device control method and apparatus, which can enable an environmental agent model to quickly adapt to environmental changes, ensure that the control strategy optimized based on the environmental agent model is not degraded, and improve the performance and energy-saving effect of the device.

[0004] In the first aspect, the present application provides a device control method, including obtaining the current working environment state of the device (which can be referred to as the current environment state); determining that the reliability of the environment proxy model does not meet the standard, and the environment proxy model is used to predict the environment state at a future time based on the environment state and control parameters at the current time; based on the current environment state and the default control strategy, determine the target control parameters of the device; control the device based on the target control parameters; obtain the interaction data between the default control strategy and the environment, the interaction data includes the current environment state, and the control parameters issued by the default control strategy based on the current environment state, and the environment state at the next time; update the training data set based on the interaction data, the training data includes multiple training sample pairs, each training sample pair includes the environment state at time t, the control parameters at time t, and the environment state at time t+1; based on the updated training data set, optimize and adjust the weight parameters of the environment proxy model to obtain an environment proxy model with met reliability standards.

[0005] The device control method provided in the present application controls the device by adopting a default control strategy (such as the control strategy that comes with the device when it leaves the factory) to control the device when the reliability of the environmental proxy model does not meet the standard (for example, the prediction accuracy of the environmental proxy model is lower than the set threshold due to a sudden change in the environment), thereby ensuring that the control strategy of the device does not show obvious degradation, and then obtains new training samples based on the interaction data by collecting the interaction data between the default control strategy and the environment, and uses the new training samples to update the training data set, and uses the updated training data set to train the environmental proxy model to tune the environmental proxy model so that its reliability meets the standard.

[0006] In a possible implementation, a specific implementation of updating the training data set based on the interaction data is: based on the interaction data, obtain new training sample pairs; add the new training sample pairs to the training data set; determine the target sliding window from the candidate sliding window set, the candidate sliding window set includes multiple sliding windows of different lengths; based on the target sliding window, select the nearest N training samples from the training data set; based on the N training samples, obtain an updated training data set.

[0007] The training data set is updated through a dynamic sliding window algorithm to ensure the rationality of the training data set update and improve the speed at which the environmental agent model adapts to the environment.

[0008] In another possible implementation, a specific implementation of determining the target sliding window from the candidate sliding window set is: determining the target sliding window based on the probability of each sliding window in the candidate sliding window set, the probability of each sliding window is determined based on feedback information corresponding to each sliding window, and the feedback information corresponding to each sliding window indicates the prediction error of the training sample selected from the training data set based on each sliding window for the trained environment agent model.

[0009] In another possible implementation, the device control method provided by the present application further includes: periodically updating the probability of each sliding window in the candidate sliding window set based on the AMAB algorithm.

[0010] The AMAB algorithm is used to quickly select the optimal sliding window length to improve the adaptation speed and accuracy of the environmental agent model to the environment.

[0011] In another possible implementation, a specific implementation of determining the target control parameter of the device based on the current environmental state and the default control strategy is: determining a first control parameter based on the current working environment state of the device and the default control strategy; perturbing the first control parameter within a preset range to obtain the target control parameter.

[0012] By randomly perturbing the control parameters issued by the default control strategy, the diversity of the collected training samples is improved, and the modeling speed of the environmental agent model for environmental changes is improved.

[0013] In another possible implementation, the device control method provided in the present application also includes: determining whether the reliability of the environmental proxy model meets the standard; optimizing and solving based on the current working environment state of the device and the environmental proxy model to obtain a second control parameter; and determining the target control parameter based on the second control parameter.

[0014] When the reliability of the environmental proxy model meets the standard, that is, the environmental proxy model can accurately model the changes in the environment, the online control strategy based on the environmental proxy model optimization is used to control the equipment to improve the performance and energy saving effect of the equipment.

[0015] In another possible implementation, a specific implementation of determining the target control parameter based on the second control parameter is: optimizing and correcting the second control parameter based on prior information to obtain the target control parameter, wherein the prior information includes an optimal proportion of control parameters with correlation in a control parameter set.

[0016] The control parameters issued by the control strategy based on environmental agent model optimization are further corrected through expert prior knowledge to further ensure the accuracy of the parameters issued by the control strategy, so as to further improve the equipment performance and reduce the equipment's working energy consumption.

[0017] In another possible implementation, the initial environment proxy model is trained based on an initial training data set, and the initial training data set is obtained based on interaction data between the device and the environment when it is powered on for the first time.

[0018] By exploring the device in a specific customer environment to obtain a training data set, and training an environmental proxy model based on the training data set, the environmental proxy model is more adapted to the environment, can more accurately predict environmental changes, improve the accuracy of the control strategy, and thus improve the performance and energy-saving effect of the equipment.

[0019] In another possible implementation, the interaction data between the device and the environment when it is turned on for the first time includes: the current environmental state, the third control parameter, and the environmental state at the next moment; the third control parameter is obtained by perturbing the control parameter issued by the default control strategy based on the current environmental state with a preset amplitude.

[0020] This greatly reduces the sample fitting complexity of the environmental proxy model while ensuring that the performance does not deteriorate significantly during sample exploration, thus solving the cold start problem.

[0021] In another possible implementation, the reliability of the environment proxy model is determined based on the accuracy and / or a priori relevance of the environment proxy model.

[0022] In another possible implementation, the device is an air conditioner; the environmental state includes at least one of the indoor temperature, the actual value of the internal fan speed, the indoor coil temperature, the actual value of the compressor frequency, the compressor exhaust temperature, the actual value of the expansion valve opening, the accumulated power, the actual value of the external fan speed, and the outdoor temperature; the target control parameter includes at least one of the target value of the internal fan speed, the target value of the compressor frequency, the target value of the expansion valve opening, and the target value of the external fan speed.

[0023] In another possible implementation, the environment proxy model is a lightweight linear model or a decision tree model, which reduces computing power overhead so that the environment proxy model can be deployed on terminal devices with limited computing power.

[0024] In the second aspect, the present application also provides a device control device, including a first acquisition module, a reliability judgment module, a determination module, a control module, a second acquisition module, an update module and an optimization module, wherein the first acquisition module is used to obtain the current working environment state of the device (which can be referred to as the current environment state for short); the reliability judgment module is used to determine that the reliability of the environment agent model does not meet the standard, and the environment agent model is used to predict the environment state at a future time based on the environment state at a current time; the determination module is used to determine the target control parameters of the device based on the current environment state and the default control strategy; the control module is used to control the device based on the target control parameters; the second acquisition module is used to obtain the interaction data between the default control strategy and the environment, the interaction data including the current environment state, and the control parameters issued by the default control strategy based on the current environment state, and the environment state at the next time; the update module is used to update the training data set based on the interaction data, the training data including multiple training sample pairs, each training sample pair including the environment state at time t, the control parameters at time t, and the environment state at time t+1; the optimization module is used to optimize and adjust the weight parameters of the environment agent model based on the updated training data set to obtain an environment agent model with reliability that meets the standard.

[0025] In one possible implementation, the update module is specifically used to: obtain new training sample pairs based on interaction data; add the new training sample pairs to the training data set; determine a target sliding window from a candidate sliding window set, the candidate sliding window set includes multiple sliding windows of different lengths; select the nearest N training samples from the training data set based on the target sliding window; and obtain an updated training data set based on the N training samples.

[0026] In another possible implementation, a specific implementation of determining the target sliding window from the candidate sliding window set is: determining the target sliding window based on the probability of each sliding window in the candidate sliding window set, the probability of each sliding window is determined based on feedback information corresponding to each sliding window, and the feedback information corresponding to each sliding window indicates the prediction error of the training sample selected from the training data set based on each sliding window for the trained environment agent model.

[0027] In another possible implementation, the updating module is further used to periodically update the probability of each sliding window in the candidate sliding window set based on the AMAB algorithm.

[0028] In another possible implementation, the determination module is specifically used to: determine the first control parameter based on the current working environment state of the device and the default control strategy; and perturb the first control parameter within a preset range to obtain a target control parameter.

[0029] In another possible implementation, the determination module is also used to determine whether the reliability of the environmental proxy model meets the standard; perform optimization and solution based on the current working environment state of the device and the environmental proxy model to obtain a second control parameter; and determine the target control parameter based on the second control parameter.

[0030] In another possible implementation, a specific implementation of determining the target control parameter based on the second control parameter is: optimizing and correcting the second control parameter based on prior information to obtain the target control parameter, wherein the prior information includes an optimal proportion of control parameters with correlation in a control parameter set.

[0031] In another possible implementation, the initial environment proxy model is trained based on an initial training data set, and the initial training data set is obtained based on interaction data between the device and the environment when it is powered on for the first time.

[0032] In another possible implementation, the interaction data between the device and the environment when it is turned on for the first time includes: the current environmental state, the third control parameter, and the environmental state at the next moment; the third control parameter is obtained by perturbing the control parameter issued by the default control strategy based on the current environmental state with a preset amplitude.

[0033] In another possible implementation, the reliability of the environment proxy model is determined based on the accuracy and / or a priori relevance of the environment proxy model.

[0034] In another possible implementation, the device is an air conditioner; the environmental state includes at least one of the indoor temperature, the actual value of the internal fan speed, the indoor coil temperature, the actual value of the compressor frequency, the compressor exhaust temperature, the actual value of the expansion valve opening, the accumulated power, the actual value of the external fan speed, and the outdoor temperature; the target control parameter includes at least one of the target value of the internal fan speed, the target value of the compressor frequency, the target value of the expansion valve opening, and the target value of the external fan speed.

[0035] In another possible implementation, the environment agent model is a lightweight linear model or a decision tree model.

[0036] In a third aspect, an embodiment of the present application provides a computing device, comprising a memory and a processor, wherein the memory stores instructions, and when the instructions are executed by the processor, the method described in the first aspect or any possible implementation of the first aspect is implemented.

[0037] In a fourth aspect, an embodiment of the present application provides a computer-readable storage medium having a computer program stored thereon. When the computer program is executed by a processor, the method described in the first aspect or any possible implementation of the first aspect is implemented.

[0038] In a fifth aspect, an embodiment of the present application further provides a computer program or a computer program product, which includes instructions that, when executed, cause a computer to execute the method described in the first aspect or any possible implementation of the first aspect.

[0039] In a sixth aspect, an embodiment of the present application further provides a chip, comprising at least one processor and a communication interface, wherein the processor is used to execute the method described in the first aspect or any possible implementation manner of the first aspect. BRIEF DESCRIPTION OF THE DRAWINGS

[0040] Figure 1 A schematic diagram of the structure of an air conditioner to which the device control method provided by an embodiment of the present application can be applied is shown;

[0041] Figure 2 A schematic diagram of an implementation flow of a device control method provided by the present application is shown;

[0042] Figure 3 A flow chart of a device control method provided for the implementation of this application;

[0043] Figure 4 A schematic diagram of the structure of a device control device provided in an embodiment of the present application;

[0044] Figure 5 A schematic diagram of the structure of a computing device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0045] The term "and / or" mentioned in this article is a kind of relationship that describes the association of associated objects, indicating that there can be three relationships. For example, A and / or B can mean: A exists alone, A and B exist at the same time, and B exists alone. The symbol " / " in this article indicates that the associated objects are in an or relationship, for example, A / B means A or B.

[0046] The terms "first" and "second" etc. in the specification and claims herein are used to distinguish different objects, rather than to describe the specific order of objects. It should be understood that the terms used in this way can be interchangeable in appropriate circumstances, which is merely a way of distinguishing the objects of the same attributes when describing them in the embodiments of the present application. In addition, the terms "including" and "having" and any of their variations are intended to cover non-exclusive inclusions, so that the process, method, system, product or equipment comprising a series of units need not be limited to those units, but may include other units that are not clearly listed or inherent to these processes, methods, systems, products or equipment.

[0047] In the embodiments of the present application, words such as "exemplary" or "for example" are used to indicate examples, illustrations or descriptions. Any embodiment or design described as "exemplary" or "for example" in the embodiments of the present application should not be interpreted as being more preferred or more advantageous than other embodiments or designs. Specifically, the use of words such as "exemplary" or "for example" is intended to present related concepts in a specific way.

[0048] In the description of the embodiments of the present application, unless otherwise specified, "multiple" means two or more than two. For example, multiple processing units refer to two or more processing units, etc.; multiple elements refer to two or more elements, etc.

[0049] In the past few decades, almost all air conditioning manufacturers have only focused on optimizing cooling efficiency at the hardware level, but there is no supporting environment-adaptive algorithm to fully utilize the hardware performance (for example, household air conditioners often overshoot during the startup phase, resulting in energy waste), which also brings new space for air conditioning energy saving. In recent years, many air conditioning manufacturers have realized the opportunity of AI-based air conditioning energy-saving algorithms. Data-driven AI algorithms have some unique advantages in air conditioning energy saving: (1) Strong adaptability: It can adaptively find the optimal control strategy based on different environments. (2) Strong compatibility: It will not conflict with the existing system framework and is easy to transplant. (3) Weak dependence on experts: Even if you are not an expert in HVAC, you can learn better energy-saving solutions through AI algorithms and some basic HVAC knowledge through AI algorithms and data.

[0050] However, replacing traditional air-conditioning online control strategies with data-driven AI algorithms has the following difficulties: (1) Limited computing power on the terminal side: The memory and computing power of air-conditioning chips are limited and cannot support highly complex models and algorithms. (2) High generalization capability requirements: The algorithms and models are required to quickly adapt and solve the optimal strategy in non-fully observed and non-steady-state environments. (3) High robustness requirements: The algorithms and models are required not to degrade compared to the baseline under any environmental conditions, and temperature control failure will not occur. Most related technologies are based on the single-step strategy benefits of the proxy model. For example, Google used deep neural networks to fit the energy consumption of large data centers under different working conditions in 2017. Some air-conditioning manufacturers also use simpler models such as Bayesian linear regression to fit the energy consumption of air conditioners under different working conditions. The premise for these solutions to achieve ideal energy saving is that these proxy models can be prepared to model the mapping relationship between the issued strategy and energy consumption under different working conditions.

[0051] For example, the first related technology is a data center energy-saving algorithm based on deep neural networks. The core idea is to first collect data from the data center under different working conditions, and then use the neural network model in the figure above to fit the mapping relationship between different working conditions and energy consumption. When the model accuracy meets the requirements, the trained model can be used to solve a more energy-saving strategy.

[0052] However, this solution has the following problems:

[0053] High computing power requirements: This feature cannot be used when the computing power on the client side is limited.

[0054] High sample complexity: A large amount of diverse data is required to ensure the accuracy of the model.

[0055] Poor adaptability to the environment: If the observed state cannot fully describe the entire system, or the external environment changes dramatically, the neural network-based agent model does not have the ability to adapt quickly, resulting in unreliable strategies based on model optimization.

[0056] Failure to consider long-term benefits: Single-step greedy optimization cannot guarantee global optimality, and temperature overshoot may occur even if the model is highly accurate.

[0057] Related technology 2: The air conditioning online control strategy tuning solution is a rule-based algorithm that is manually debugged by HVAC experts based on expert experience in a specific laboratory. The tuning cycle is often relatively long, generally several months.

[0058] This solution has the following problems:

[0059] Poor generalization ability: Control strategies debugged in the laboratory are often not optimal in a variety of customer environments.

[0060] Failure to consider long-term benefits: Strategies based on expert experience often cannot support high-complexity strategy optimization. The debugging strategy only optimizes the benefits of a single step. The optimized strategy is prone to temperature overshoot, resulting in energy waste.

[0061] The embodiments of this application consider how to accurately model the environment with a small amount of data, and not significantly degrade compared to the baseline during the interaction, laying the foundation for subsequent strategy optimization. And how to make the environment agent model adapt quickly when the environment changes, so that the performance of the issued strategy is not degraded in the subsequent online learning process.

[0062] To this end, an embodiment of the present application proposes a device control method, which introduces the idea of ​​safety exploration and dynamic local linearization modeling, so that the lightweight agent model can quickly adapt to environmental changes, thereby efficiently solving energy-saving strategies based on the environmental agent model.

[0063] The device control method provided in the embodiment of the present application can be applied to the control of various devices to achieve energy-saving optimization of the devices, for example, it can achieve energy-saving optimization for network devices (such as routers), energy-saving optimization for air-conditioning equipment, etc. It can be especially applied to energy-saving optimization of the control strategy of air-conditioning. The embodiment of the present application takes air-conditioning as an example to introduce the specific implementation of the device control method provided in the embodiment of the present application. Other devices are similar to it. The specific implementation can refer to the control method for air-conditioning. For the sake of brevity, the embodiment of the present application will not be repeated.

[0064] To facilitate understanding of the solutions of the embodiments of the present application, the technical terms involved in this document are first explained below.

[0065] Linear model: A machine learning model that assumes that the input and output satisfy a linear mathematical relationship. This model is suitable for local modeling based on data when the amount of data is small and the computing power is limited. Compared with the commonly used neural network model, it has the advantage of being more interpretable and easier to incorporate expert priors.

[0066] Security exploration: The algorithm does not have any prior knowledge of the environment in the early stages. It ensures the diversity of collected data while ensuring that the issued strategy does not degrade significantly compared to the baseline, so as to facilitate more accurate modeling of the environment.

[0067] Dynamic sliding window: A machine learning algorithm for non-steady-state environments that uses data from a recent period of time as input for model fitting. The specific amount of data that is selected can be dynamically adjusted based on historical feedback.

[0068] Figure 1 The structure diagram of an air conditioner to which the device control method provided in the embodiment of the present application can be applied is shown. Figure 1As shown, the air conditioner includes an indoor unit and an outdoor unit, the indoor unit includes an indoor fan and an evaporator, and the outdoor unit includes a compressor, a condenser, an outdoor fan and an expansion valve. The user inputs the target temperature, mode, wind speed and other user settings that need to be adjusted by the air conditioner through the air conditioner remote control or the control panel on the air conditioner. The air conditioner responds to the user settings and sends control signals to various components in the air conditioner based on its own control strategy. For example, a control signal is sent to the indoor fan to control the speed of the indoor fan, a control signal is sent to the evaporator (the control signal is generated according to the control parameter) to control the operation of the evaporator, a control signal is sent to the compressor to control the operating frequency of the compressor, a control signal is sent to the expansion valve to control the opening of the expansion valve, and a control signal is sent to the outdoor fan to control the speed of the outdoor fan; so that the air conditioner works to adjust the temperature to the target temperature set by the user.

[0069] As the air conditioner works, the external environment will also change (for example, as the air conditioner cools, the temperature of the external environment will drop). The air conditioner's control strategy will adjust the control signals sent to each component according to the current environmental state of the air conditioner's environment to achieve effective temperature control and energy-saving optimization of the air conditioner.

[0070] The control parameters of an air conditioner refer to the working parameters of some components in the air conditioner (such as the compressor, indoor fan, expansion valve and outdoor fan, etc.) so that the air conditioner can adjust the temperature to the target temperature required by the user. Figure 1 As shown, the control quantity (ie, control parameter) of the air conditioner includes an internal fan speed target value (ie, the internal fan speed value indicated by the control parameter), a compressor frequency target value, an expansion valve opening target value, and an external fan speed target value.

[0071] Environmental status refers to some state parameters of the working environment of the air conditioner, or some environmental state parameters that will affect the control decision of the air conditioner, such as Figure 1 As shown, the state quantity (i.e., environmental state) may include indoor temperature, actual value of indoor fan speed (i.e., actual speed of the indoor fan), indoor coil temperature, actual value of compressor frequency, compressor exhaust temperature, actual value of expansion valve opening, accumulated power, actual value of outdoor fan speed, and outdoor temperature.

[0072] In some other examples, the control parameters of the air conditioner may also include only some of the control parameter items among the target value of the internal fan speed, the target value of the compressor frequency, the target value of the expansion valve opening, and the target value of the external fan speed. The environmental state may also include some of the environmental state items among the indoor temperature, the actual value of the internal fan speed, the indoor coil temperature, the actual value of the compressor frequency, the compressor exhaust temperature, the actual value of the expansion valve opening, the accumulated power, the actual value of the external fan speed, and the outdoor temperature.

[0073] The environmental state can be detected by various sensors. For example, the indoor temperature, indoor coil temperature, compressor exhaust temperature, and outdoor temperature can be detected by temperature sensors set at corresponding positions. For example, a temperature sensor is set in the indoor space where the air conditioner is located, and the indoor temperature is detected to obtain the indoor temperature. A temperature sensor is set at the indoor coil, and the temperature of the indoor coil is detected to obtain the indoor coil temperature. A temperature sensor is set at the exhaust port of the compressor, and the temperature of the gas discharged by the compressor is detected to obtain the compressor exhaust temperature. A temperature sensor is set in the outdoor space where the outdoor unit is located, and the outdoor temperature of the outdoor unit is detected to obtain the outdoor temperature. For example, the actual value of the speed of the inner fan and the actual value of the speed of the outer fan can be detected by a speed sensor. For example, speed sensors can be set at the inner fan and the outer fan respectively, and the speed of the inner fan and the speed of the outer fan can be detected respectively to obtain the actual speed of the inner fan and the actual speed of the outer fan. For another example, the actual value of the compressor frequency can be detected by a frequency sensor. The actual value of the expansion valve opening can be obtained by detecting the actual opening of the expansion valve through a distance sensor. The accumulated power refers to the accumulated power consumed by the air conditioner, which can be detected by an energy consumption sensor.

[0074] Some environmental conditions can be obtained by other means instead of sensors. For example, they can be obtained from other channels through communication. For example, for a work scenario where other smart home devices or monitoring systems are installed, the indoor air temperature can be obtained based on records from other smart devices, and the outdoor air temperature can be obtained from an online server based on weather forecasts and other information.

[0075] The embodiments of this application are Figure 1 The device control method provided by the embodiment of the present application is deployed in the air conditioner shown. Considering that the computing power of the air conditioner is usually limited, the device control method of the embodiment of the present application adopts a lightweight environmental agent model to predict future environmental changes. Through an online strategy optimization framework based on dynamic sliding window local linearization modeling, the environmental adaptation speed is greatly improved, and the optimal control strategy can be efficiently solved in a dynamic environment to perform efficient energy-saving tuning on the air conditioner.

[0076] Figure 2 A schematic diagram of an implementation flow of the device control method provided by the present application is shown. The method can be Figure 1 The air conditioning shown is implemented.

[0077] First, the interaction data between the air conditioner and the working environment in which the air conditioner is located is obtained, and an environmental proxy model is modeled according to the interaction data to obtain an environmental proxy model for the specific working environment in which the air conditioner is currently located. Compared with the model modeled in a laboratory environment, the environmental proxy model of the embodiment of the present application has more accurate predictions, thereby making the optimization solution of the online control strategy based on the environmental proxy model more precise.

[0078] In order to ensure the diversity of the collected interaction data and prevent the air conditioner from experiencing performance degradation (such as temperature control failure) in the process of obtaining the interaction data, the embodiment of the present application adopts a safe exploration method to add a disturbance of appropriate amplitude on the basis of the air conditioner baseline control strategy when the air conditioner is turned on for the first time, so as to achieve accurate local modeling of the environment with only a small number of interactions with the environment, and no obvious degradation occurs during the interaction process compared with the baseline control strategy.

[0079] For example, after the air conditioner is turned on for the first time, a random disturbance of appropriate amplitude is added to the control parameters issued by the original control strategy of the air conditioner, and then the interaction data between the control strategy and the environment is collected, that is, the control parameters issued by the control strategy are affected by the air conditioner after the air conditioner is working, resulting in changes in the environmental state. The triple (x t ,u t ,x t+1 ) data, where x t represents the environmental state at time t, u t represents the control parameters issued by the control strategy at time t, x t+1 Represents the environmental state at time t+1.

[0080] It should be noted that the original control strategy of the air conditioner refers to the online control strategy that comes with the air conditioner when it leaves the factory. The control strategy can be any type of control strategy, for example, the control strategy can be a rule-based algorithm based on manual debugging by experts.

[0081] The meaning of a disturbance of an appropriate magnitude is that the disturbance of the control parameters issued by the original control strategy of the air conditioner will not affect the temperature control of the air conditioner. For example, after the control parameters after adding the disturbance are executed, the air conditioner can still control the temperature at the target temperature.

[0082] For example, the appropriate disturbance amplitude for the internal fan speed is ±5, the appropriate disturbance amplitude for the compressor frequency is ±3, the appropriate disturbance amplitude for the expansion valve opening is ±2, and the appropriate disturbance amplitude for the external fan speed is ±5. According to the original control strategy of the air conditioner, the control parameters to be issued are the internal fan speed target value M1, the compressor frequency target value M2, the expansion valve opening target value M3, and the external fan speed target value M4. After applying random disturbances of appropriate amplitudes to each control parameter, the internal fan speed target value is M1+4, the compressor frequency target value is M2-2, the expansion valve opening target value is M3+1, and the external fan speed target value is M4-3.

[0083] In this way, by making random perturbations of appropriate magnitude on the control parameters issued by the original control strategy of the air conditioner, diverse data with sufficient modeling accuracy can be quickly obtained.

[0084] Then, the obtained multiple triple data are used to build a model to obtain an environmental proxy model, or in other words, the obtained multiple triple data are used as a training data set to train the environmental proxy model to obtain an environmental proxy model with a prediction accuracy that meets the standard.

[0085] Perform security exploration until a sufficient number of triple data with modeling accuracy is obtained. Assume that the sufficient number is 90 triple data. For example, the sampling interval is two minutes, that is, a triple data is sampled every two minutes (x t ,u t , x t+1 ), 30 triplet data can be collected in one hour, and 90 triplet data can be collected in three hours. Then, the random perturbation of the control parameters issued by the original control strategy is terminated, and the environmental proxy model is modeled using the 90 triplet data collected.

[0086] The environmental agent model obtained by modeling can predict the environmental state at the future time t+1 based on the environmental state at time t and the control parameters at time t. In other words, the environmental state at time t and the control parameters at time t are used as the input of the environmental agent model, and the environmental agent model can output the environmental state at time t+1 after reasoning.

[0087] Through safe exploration, it is possible to quickly obtain diverse and sufficient training samples without prior knowledge of the environment, and use the collected training samples to train the environmental proxy model to obtain a preliminarily trained environmental proxy model, thus solving the problem of cold start.

[0088] Considering the limited computing power of the air conditioner, the environmental proxy model adopts a lightweight model. For example, the environmental proxy model can be a lightweight model such as a linear model or a decision tree model to reduce computing power overhead and facilitate deployment on the air conditioner end.

[0089] A lightweight model refers to a neural network model with a small number of parameters. For example, the number of parameters of the environmental proxy model in the embodiment of the present application is 10M, and the computing power required for the device to deploy the environmental proxy model is also small, for example, less than 1000mops. This makes it easy to deploy on terminal devices with limited computing power. For example, the environmental proxy model provided in the embodiment of the present application is deployed on an air conditioner, so that the air conditioner can run an online control strategy optimized based on the environmental proxy model to ensure the temperature control performance and energy-saving effect of the air conditioner.

[0090] After the trained environment proxy model is obtained, the current environment state and the environment proxy model are used to optimize and solve, and the control parameters that the air conditioner currently needs to send are obtained. In other words, the optimization model composed of the environment proxy model is optimized and solved according to the current environment state to obtain an initial solution, and the control parameters currently sent by the air conditioner are determined according to the initial solution.

[0091] Exemplarily, the optimization model includes the following optimization equations:

[0092]

[0093] stx t+1 =Ax t +Bu t

[0094] x t ∈X

[0095] Among them, x τ represents the environmental state at time τ, u τ represents the control parameter at time τ, x t represents the environmental state at time t, u t represents the control parameter at time t, x t+1 represents the environmental state at time t+1, Q and R are the weight matrices that define the single-step loss function, A and B are the weight parameters of the environmental proxy model based on historical environment fitting, and X is the feasible domain of the control parameters.

[0096] By solving the above optimization equations, an initial solution is obtained. In one example, the initial solution can be directly issued as the current control parameter to control the air conditioner. For example, by solving the above optimization equations, the initial solution is U is u τ1 =M5,u τ2 =M6,u τ3 =M7,u τ4 =M8, directly based on the initial solution, the control parameters to be issued by the air conditioner are: the target value of the internal fan speed is M5, the target value of the compressor frequency is M6, the target value of the expansion valve opening is M7, and the target value of the external fan speed is M8.

[0097] In another example, in order to ensure the quality of the control parameters issued by the air conditioner (i.e., the air conditioner is controlled according to the control parameters, and the temperature control of the air conditioner is more stable and the air conditioner is more energy-efficient), after obtaining the initial solution of the optimization equation group, the initial solution will be further processed, and the control parameters to be issued by the air conditioner are determined based on the processed initial solution.

[0098] Exemplarily, the initial solution is optimized and corrected by expert priori, and the corresponding components in the air conditioner are controlled by the optimized and corrected control parameters. For example, the expert priori includes the optimal ratio of different control parameters, and the control parameters obtained by the initial solution are adjusted to meet the optimal ratio to optimize the initial solution, and then the optimized and adjusted initial solution is issued as the current control parameter to control the corresponding components of the air conditioner to work. That is, the initial solution is post-mapped using expert experience so that the control parameters meet the expert priori.

[0099] For example, expert experience and knowledge know that the optimal opening of the expansion valve and the frequency of the compressor are positively correlated, but due to the limited ability of the optimization model, the matching relationship cannot be well modeled. Therefore, expert experience is needed to perform a post-mapping of the initial solution obtained by solving the optimization equation group so that the control parameters meet the expert prior. For example, according to the model solution, the compressor frequency is M6 and the expansion valve opening is M7, but the prior knowledge is that the optimal ratio of the compressor frequency and the expansion valve opening is 1:2, then adjusting the expansion valve opening from M7 to 2M6 is equivalent to making a correction on the original basis, further improving the air conditioning cooling / heating efficiency or energy utilization efficiency.

[0100] The accuracy of the initial solution obtained by solving the optimization model is closely related to the reliability of the environmental proxy model, or in other words, the reliability of the environmental proxy model determines the reliability of the initial solution obtained by optimizing the optimization model. Therefore, in order to ensure the accuracy of the air-conditioning online control strategy, in the actual operation of the air-conditioning, when it is necessary to send control parameters, it is necessary to first determine whether the reliability of the environmental proxy model meets the standard. When the reliability of the environmental proxy model meets the standard, the current environmental state and the optimization model are used to optimize the sending strategy, and then the optimized sending strategy is corrected based on expert experience. The air-conditioning operation is controlled according to the sending strategy optimized by expert experience. When the reliability of the environmental proxy model does not meet the standard, the default control strategy is used to send the control parameters, so as to ensure the quality of the online control strategy of the air-conditioning and avoid obvious degradation of the control strategy of the air-conditioning.

[0101] Whether the reliability of the environmental proxy model meets the standard can be determined based on the accuracy and / or prior correlation of the environmental proxy model. For example, when the prediction accuracy of the environmental proxy model is lower than the preset threshold, the reliability of the environmental proxy model is judged to be not up to standard. For example, if the prediction accuracy of the environmental proxy model in the past few steps is continuously lower than the preset threshold, the reliability of the environmental proxy model is judged to be not up to standard. And / or, when the prior correlation of the environmental proxy model does not meet the requirements, the reliability of the environmental proxy model is judged to be not up to standard. For example, when the control parameter of the compressor frequency is relatively high, there is a positive benefit for cooling, but the temperature predicted by the environmental proxy model does not decrease, then it is judged that the prior correlation of the environmental proxy model does not meet the requirements, and then the reliability of the environmental proxy model is judged to be not up to standard. For another example, when the prediction accuracy of the environmental proxy model is lower than the preset threshold, and the prior correlation of the environmental proxy model does not meet the requirements, the reliability of the environmental proxy model is judged to be not up to standard.

[0102] The default control strategy can be the online control strategy that comes with the air conditioner when it leaves the factory. For example, the control strategy is based on the proportional integral derivative (PID) control algorithm. Based on the control parameters issued by the default control strategy of the air conditioner, random disturbances of appropriate amplitude are performed to ensure data diversity. Then, the interaction data between the default control strategy and the environment are collected to obtain new triplet data ( x t,u t , x t+1 ), using the new triple data (x t ,u t , x t+1 ) Update the training data set, and then use the updated training data set to optimize the training of the environment proxy model, so that the environment proxy model can quickly adapt to changes in the environment and ensure the prediction accuracy of the environment proxy model.

[0103] Since environmental changes are not steady-state, the working environment of the air conditioner may change dramatically. At this time, the reliability of the environmental proxy model will be insufficient, making the optimization strategy based on the environmental proxy model unreliable. For this reason, the embodiment of the present application proposes an online strategy optimization framework based on dynamic sliding window local linearization modeling, so that the environmental proxy model can quickly adapt to environmental changes and efficiently solve the optimal energy-saving strategy in a dynamic environment.

[0104] Exemplarily, when it is determined that the reliability of the environmental proxy model does not meet the standard, the default control strategy of the air conditioner is used to issue control parameters, and the interaction data between the default control strategy and the environment is collected to obtain new triple data, and the training data set is updated based on the new triple data. For example, each time a new triple data is obtained, the new triple data is added to the training data set, and the oldest triple data is deleted to ensure the stability of the data volume in the training data set. Then, the length of the dynamic sliding window is updated, and multiple triple data are selected from the training data set based on the updated dynamic sliding window. The training data subset formed by the selected multiple triple data is used to train and optimize the parameters of the environmental proxy model, so that the environmental proxy model can quickly adapt to changes in the environment.

[0105] The embodiment of the present application pre-constructs a feasible sliding window set J of different lengths w = {W0, W1, ..., W N}, different sliding windows of different lengths mean that the time spans of the triplet data selected from the training data set are different, and the prediction accuracy of the trained environment proxy model is also different. How to select the optimal sliding window (also called the sliding window of optimal length) from the sliding window set determines the prediction accuracy of the trained and optimized environment proxy model.

[0106] According to the feedback information of different sliding window lengths selected in the past (such as modeling error, i.e., the prediction error of the environment proxy model obtained by training the training data subset composed of triple data selected from the training data set using different sliding windows), the probability of each sliding window being selected in the sliding window set is determined. The better the feedback information of the sliding window (i.e., the smaller the modeling error), the greater the probability of the sliding window being selected in the sliding window set, that is, when the environment proxy model is optimized for training, the target sliding window is selected from the sliding window set (the sliding window with a greater probability is more likely to be selected as the target sliding window this time), multiple triple data are selected from the training data set according to the target sliding window, and the weight parameters of the environment proxy model are optimized by using the multiple triple data selected by the target sliding window, and the environment proxy model is updated until the environment proxy model adapts to the changed environment, and the reliability of the environment proxy model meets the standard in the changed environment, and then the environment proxy model is optimized.

[0107] In order to further increase the adaptation speed of the environmental proxy model to environmental changes, the embodiment of the present application selects different sliding windows based on the feedback of the past time according to a preset cycle, for example, every certain time interval T0, and uses the adversarial multi-armed bandit (AMAB) algorithm to update the probability of each sliding window in the sliding window set being selected. In other words, the AMAB algorithm is used to dynamically update the sliding window length selected in the next time interval (i.e., the sliding window with the highest probability), so as to achieve the effect of quickly learning the optimal sliding window length as time accumulates, and the algorithm has the guarantee of theoretical optimality. In this way, the optimal sliding window length is quickly selected by the AMAB algorithm to improve the adaptation speed and prediction accuracy of the environmental proxy model.

[0108] In summary, the device control method provided by the embodiment of the present application first collects diverse triplet data through security exploration in the early stage (for example, three hours) when the air conditioner is first turned on for the first time, and uses the collected triplet data to model and obtain an environmental proxy model. In the online policy control stage, at each moment, it is first determined whether the current environmental proxy model is reliable. If the environmental proxy model is reliable, the control parameters are optimized and sent based on the environmental proxy model, otherwise the control parameters are sent based on the default control policy. Then, in the process of interaction between the default control policy and the environment, the interaction data is collected to obtain new triplet data, and the training data set is updated according to the new triplet data. The environmental proxy model is updated and optimized based on the updated training data set. After determining that the reliability of the updated proxy model meets the standard, the control parameters are optimized and sent according to the updated environmental proxy model, and this cycle is repeated. The device control method provided in the embodiment of the present application greatly reduces the complexity of the sample fitting of the environmental proxy model through safe exploration while ensuring that the performance of the air conditioner does not deteriorate significantly during the exploration period, thereby solving the cold start problem; through online strategy optimization based on dynamic sliding window local linearization modeling, the environmental adaptation speed is greatly improved, and the optimal control strategy of the equipment can be efficiently solved in a dynamic environment. The quality of the optimization strategy of the optimization model is further improved through hierarchical optimization embedded with expert experience (i.e., optimizing and correcting the initial solution through expert priori); the optimal sliding window length is quickly selected through the AMAB algorithm, thereby improving the environmental adaptation speed and prediction accuracy of the environmental proxy model.

[0109] Figure 3 A flow chart of a device control method provided for the implementation of the present application. The method can be implemented by a device that applies the device control method provided in the embodiment of the present application, such as an air conditioner. The device control method provided in the embodiment of the present application is introduced below using an air conditioner as an example. The method includes at least steps S301 to S308.

[0110] In step S301, the current environment state is obtained.

[0111] Environmental status refers to some state parameters of the working environment of the air conditioner, or some environmental state parameters that will affect the control decision of the air conditioner, such as Figure 1 As shown, the environmental status may include indoor temperature, actual value of indoor fan speed (i.e., actual speed of the indoor fan), indoor coil temperature, actual value of compressor frequency, compressor exhaust temperature, actual value of expansion valve opening, accumulated power, actual value of outdoor fan speed, and outdoor temperature.

[0112] In some other examples, the environmental status may also include some environmental status items such as indoor temperature, actual value of internal fan speed, indoor coil temperature, actual value of compressor frequency, compressor exhaust temperature, actual value of expansion valve opening, accumulated power, actual value of external fan speed, and outdoor temperature.

[0113] The current environmental status refers to the current environmental status of the air conditioner, such as one or more of the current indoor temperature, the actual value of the indoor fan speed, the indoor coil temperature, the actual value of the compressor frequency, the compressor exhaust temperature, the actual value of the expansion valve opening, the accumulated power, the actual value of the outdoor fan speed, and the outdoor temperature.

[0114] The current environmental state can be obtained by a variety of methods, such as directly measuring it through a sensor, or obtaining it from other channels through communication. The specific acquisition method can be found in the above description, and for the sake of brevity, it will not be repeated here.

[0115] In step S302, it is determined that the reliability of the environment proxy model does not meet the standard, and the environment proxy model is used to predict the environment state at a future time based on the environment state at a current time and the control parameters.

[0116] The accuracy of the control strategy based on the optimization of the environmental proxy model in the embodiment of the present application (also referred to as the optimization model) is closely related to the reliability of the environmental proxy model, or in other words, the reliability of the environmental proxy model determines the reliability of the initial solution obtained by optimizing the optimization model. Therefore, in order to ensure the accuracy of the online control strategy of the air conditioner, when the control parameters need to be issued during the actual operation of the air conditioner, it is necessary to first determine whether the reliability of the environmental proxy model meets the standard, and determine the subsequent control strategy of the air conditioner based on whether the reliability of the environmental proxy model meets the standard.

[0117] Whether the reliability of the environmental proxy model meets the standard can be determined based on the accuracy and / or prior relevance of the environmental proxy model. The specific method for determining the reliability of the environmental proxy model can be found in the above description, and will not be repeated here for the sake of brevity.

[0118] In another example, considering the limited computing power of the air conditioner, the environmental proxy model adopts a lightweight model. For example, the environmental proxy model can be a lightweight model such as a linear model or a decision tree model to reduce computing power overhead and facilitate deployment on the air conditioner end.

[0119] In step S303, target control parameters of the device are determined based on the current environmental state and the default control strategy.

[0120] After determining that the reliability of the environmental proxy model does not meet the standard, that is, the accuracy of the solution output by the optimization model is unreliable, the control parameters are issued using the default control strategy, thereby ensuring the quality of the air conditioner's online control strategy and avoiding obvious degradation of the air conditioner's control strategy.

[0121] The default control strategy can be the online control strategy that comes with the air conditioner when it leaves the factory. For example, the control strategy is based on the PID control algorithm, or it can be a rule-based algorithm that is manually debugged by HVAC experts in a specific laboratory based on their expert experience. For example, if the current environmental state is x t , the rule-based algorithm can calculate x t The corresponding control parameter is determined as the target control parameter.

[0122] The control parameters of an air conditioner refer to the working parameters of some components in the air conditioner (such as the compressor, indoor fan, expansion valve and outdoor fan, etc.) so that the air conditioner can adjust the temperature to the target temperature required by the user. Figure 1 As shown, the control parameters of the air conditioner include an internal fan speed target value (ie, an internal fan speed value indicated by the control parameters), a compressor frequency target value, an expansion valve opening target value, and an external fan speed target value.

[0123] In some other examples, the control parameters of the air conditioner may also include only some control parameter items of the internal fan speed target value, the compressor frequency target value, the expansion valve opening target value, and the external fan speed target value.

[0124] In step S304, the device is controlled based on the target control parameter.

[0125] After determining the target control parameters, the target control parameters are used to control the corresponding components in the air conditioner to work, so as to achieve the influence on the external temperature / humidity. For example, if the compressor frequency in the target control parameters is M1, the internal fan speed is M2, the expansion valve opening is M3, and the external fan speed is M4, then the air conditioner processor controls the compressor frequency to be M1, the internal fan speed to be M2, the expansion valve opening to be M3, and the external fan speed to be M4 for operation.

[0126] In step S305, the interaction data between the default control strategy and the environment is obtained.

[0127] When the air conditioner is working with the default control strategy, the interaction data between the default control strategy and the environment is collected, that is, the control parameters of the air conditioner and the environmental state. t ,u t, x t+1 ) data, where x t represents the environmental state at time t, u t represents the control parameters issued by the control strategy at time t, x t+1 Represents the environmental state at time t+1.

[0128] In another example, random perturbations of appropriate magnitude are performed based on the control parameters issued by the default control strategy of the air conditioner to ensure data diversity. Then, the interaction data between the default control strategy and the environment is collected to obtain new triple data (x t ,u t , x t+1 ) to ensure the diversity of the new triplet data, and then to ensure the sample diversity of the training data set after the new triplet is updated, so that the environmental proxy model can quickly adapt to the changes in the environment and ensure the prediction accuracy of the environmental proxy model.

[0129] In step S306, the training data set is updated based on the interaction data.

[0130] Exemplarily, one way to update the training data set is: each time a new triple data is obtained, the new triple data is added to the training data set, and the oldest triple data is deleted to ensure the stability of the data volume in the training data set. Then, the length of the dynamic sliding window is updated, and multiple triple data are selected from the training data set based on the updated dynamic sliding window. The training data subset formed by the selected multiple triple data is used to train and optimize the parameters of the environment agent model, so that the environment agent model can quickly adapt to changes in the environment.

[0131] The specific method for updating the training data set based on the dynamic sliding window can be found in the above description, and for the sake of brevity, it will not be repeated here.

[0132] In step S307, the weight parameters of the environmental proxy model are optimized and adjusted based on the updated training data set to obtain an environmental proxy model that meets the reliability standards.

[0133] After the reliability of the environmental proxy model meets the standard, the control strategy based on the environmental proxy model optimization is used to control the operating parameters of the air conditioner. For example, the current environmental state and the optimization model composed of the environmental proxy model mentioned above are used to optimize and solve to obtain the initial solution. One way is to directly use the initial solution as the current control parameter to control the air conditioner. Another way is that in order to ensure the quality of the control parameters issued by the air conditioner, after obtaining the initial solution of the optimization equation group, the initial solution will be further processed, and the control parameters to be issued by the air conditioner are determined according to the processed initial solution. For example, the initial solution is optimized and corrected by expert priori, and the corresponding components in the air conditioner are controlled by the optimized and corrected control parameters. For example, the expert priori includes the optimal ratio of different control parameters, and the control parameters obtained by the initial solution are adjusted to meet the optimal ratio to optimize and adjust the initial solution, and then the optimized and adjusted initial solution is issued as the current control parameter to control the corresponding components of the air conditioner to work. That is, the initial solution is post-mapped using expert experience so that the control parameters meet the expert priori.

[0134] The initial environmental proxy model can be obtained by collecting training data sets in a safe exploration manner to model environmental changes. For example, in order to ensure the diversity of the collected interaction data and prevent the air conditioner from experiencing performance degradation (such as temperature control failure) in the process of obtaining the interaction data, the embodiment of the present application adopts a safe exploration method to add a disturbance of appropriate amplitude on the basis of the air conditioner baseline control strategy when the air conditioner is turned on for the first time, so as to achieve accurate local modeling of the environment with only a small number of interactions with the environment, and no obvious degradation occurs during the interaction process compared to the baseline control strategy.

[0135] The specific implementation of security exploration can be found in the above description. For the sake of brevity, it will not be repeated here.

[0136] Based on the same concept as the above-mentioned device control method embodiment, the present application embodiment also provides a device control device 400, which can be deployed in a terminal device, such as an air conditioner, to improve the temperature control performance and energy saving effect of the air conditioner. The device control device 400 includes a Figure 2 and 3 The units or modules of each step in the device control method shown.

[0137] Figure 4 This is a schematic diagram of the structure of a device control device provided in an embodiment of the present application. Figure 4As shown, the device control device 400 includes a first acquisition module 401, a reliability judgment module 402, a determination module 403, a control module 404, a second acquisition module 405, an update module 406 and an optimization module 407, wherein the first acquisition module 401 is used to obtain the current working environment state of the device (which can be referred to as the current environment state); the reliability judgment module 402 is used to determine that the reliability of the environment agent model does not meet the standard, and the environment agent model is used to predict the environment state at a future time based on the environment state at the current time; the determination module 403 is used to determine the target control parameters of the device based on the current environment state and the default control strategy; the control module 404 is used to The invention is used to control the device based on the target control parameters; the second acquisition module 405 is used to obtain the interaction data between the default control strategy and the environment, the interaction data includes the current environment state, and the control parameters issued by the default control strategy based on the current environment state, and the environment state at the next moment; the update module 406 is used to update the training data set based on the interaction data, the training data includes a plurality of training sample pairs, each training sample pair includes the environment state at time t, the control parameters at time t, and the environment state at time t+1; the optimization module 407 is used to optimize and adjust the weight parameters of the environment proxy model based on the updated training data set to obtain an environment proxy model with satisfactory reliability.

[0138] In one possible implementation, the update module 406 is specifically used to: obtain new training sample pairs based on the interaction data; add the new training sample pairs to the training data set; determine a target sliding window from a candidate sliding window set, where the candidate sliding window set includes multiple sliding windows of different lengths; select the nearest N training samples from the training data set based on the target sliding window; and obtain an updated training data set based on the N training samples.

[0139] In another possible implementation, a specific implementation of determining the target sliding window from the candidate sliding window set is: determining the target sliding window based on the probability of each sliding window in the candidate sliding window set, the probability of each sliding window is determined based on feedback information corresponding to each sliding window, and the feedback information corresponding to each sliding window indicates the prediction error of the training sample selected from the training data set based on each sliding window for the trained environment agent model.

[0140] In another possible implementation, the updating module 406 is further configured to periodically update the probability of each sliding window in the candidate sliding window set based on the AMAB algorithm.

[0141] In another possible implementation, the determination module 403 is specifically used to: determine the first control parameter based on the current working environment state of the device and the default control strategy; and perturb the first control parameter within a preset range to obtain a target control parameter.

[0142] In another possible implementation, the determination module 403 is also used to determine whether the reliability of the environmental proxy model meets the standard; perform optimization and solution based on the current working environment state of the device and the environmental proxy model to obtain the second control parameter; and determine the target control parameter based on the second control parameter.

[0143] In another possible implementation, a specific implementation of determining the target control parameter based on the second control parameter is: optimizing and correcting the second control parameter based on prior information to obtain the target control parameter, wherein the prior information includes an optimal proportion of control parameters with correlation in a control parameter set.

[0144] In another possible implementation, the initial environment proxy model is trained based on an initial training data set, and the initial training data set is obtained based on interaction data between the device and the environment when it is powered on for the first time.

[0145] In another possible implementation, the interaction data between the device and the environment when it is turned on for the first time includes: the current environmental state, the third control parameter, and the environmental state at the next moment; the third control parameter is obtained by perturbing the control parameter issued by the default control strategy based on the current environmental state with a preset amplitude.

[0146] In another possible implementation, the reliability of the environment proxy model is determined based on the accuracy and / or a priori relevance of the environment proxy model.

[0147] In another possible implementation, the device is an air conditioner; the environmental state includes at least one of the indoor temperature, the actual value of the internal fan speed, the indoor coil temperature, the actual value of the compressor frequency, the compressor exhaust temperature, the actual value of the expansion valve opening, the accumulated power, the actual value of the external fan speed, and the outdoor temperature; the target control parameter includes at least one of the target value of the internal fan speed, the target value of the compressor frequency, the target value of the expansion valve opening, and the target value of the external fan speed.

[0148] In another possible implementation, the environment agent model is a lightweight linear model or a decision tree model.

[0149] The device control apparatus 400 according to the embodiment of the present application may correspond to executing the method described in the embodiment of the present application, and the above and other operations and / or functions of each module in the device control apparatus 400 are respectively to implement Figure 2-3 For the sake of brevity, the corresponding processes of each method in are not repeated here.

[0150] Figure 5 A schematic diagram of the structure of a computing device provided in an embodiment of the present application.

[0151] like Figure 5As shown, the computing device 500 includes at least one processor 501, a memory 502 and a communication interface 503. The processor 501, the memory 502 and the communication interface 503 are connected in communication, and the communication connection can be achieved by wired means (such as a bus) or by wireless means. The communication interface 503 is used to send and / or receive data sent by other devices; the memory 502 stores computer instructions, and the processor 501 executes the computer instructions and executes the method in the aforementioned method embodiment to improve the control performance and energy saving effect of the device.

[0152] It should be understood that in the embodiment of the present application, the processor 501 may be a central processing unit CPU, and the processor 501 may also be other general-purpose processors, digital signal processors (DSP), application specific integrated circuits (ASIC), field programmable gate arrays (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. The general-purpose processor may be a microprocessor or any conventional processor, etc.

[0153] The memory 502 may include a read-only memory and a random access memory, and provide instructions and data to the processor 501. The memory 502 may also include a non-volatile random access memory. Optionally, the random access memory may be, for example, a high bandwidth memory (HBM).

[0154] The memory 502 may be a volatile memory or a nonvolatile memory, or may include both volatile and nonvolatile memories. Among them, the nonvolatile memory may be a read-only memory (ROM), a programmable ROM (PROM), an erasable programmable ROM (EPROM), an electrically erasable programmable ROM (EEPROM), or a flash memory. The volatile memory may be a random access memory (RAM), which is used as an external cache. By way of example and not limitation, many forms of RAM are available, such as static RAM (SRAM), dynamic random access memory (DRAM), synchronous dynamic random access memory (SDRAM), double data rate synchronous dynamic random access memory (DDR SDRAM), enhanced synchronous dynamic random access memory (ESDRAM), synchronous link DRAM (SLDRAM), and direct rambus RAM (DR RAM).

[0155] It should be understood that the computing device 500 according to the embodiment of the present application can execute the implementation of the embodiment of the present application. Figure 2-3 The method shown, the detailed description of the implementation of this method can be found above, for the sake of brevity, it will not be repeated here.

[0156] An embodiment of the present application provides a computer-readable storage medium having a computer program stored thereon. When the computer instructions are executed by a processor, the above-mentioned method is implemented.

[0157] An embodiment of the present application provides a chip, which includes at least one processor and an interface, wherein the at least one processor determines program instructions or data through the interface; the at least one processor is used to execute the program instructions to implement the method mentioned above.

[0158] An embodiment of the present application provides a computer program or a computer program product, wherein the computer program or the computer program product comprises instructions, and when the instructions are executed, the computer is caused to execute the above-mentioned method.

[0159] Those of ordinary skill in the art should further appreciate that the units and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, computer software, or a combination of the two. In order to clearly illustrate the interchangeability of hardware and software, the composition and steps of each example have been generally described in the above description according to function. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those of ordinary skill in the art may use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of this application.

[0160] The steps of the method or algorithm described in conjunction with the embodiments disclosed herein may be implemented as a software module executed by hardware, a processor, or a combination of the two. The software module may be placed in a random access memory (RAM), a memory, a read-only memory (ROM), an electrically programmable ROM, an electrically erasable programmable ROM, a register, a hard disk, a removable disk, a CD-ROM, or any other form of storage medium known in the art.

[0161] The specific implementation methods described above further illustrate the purpose, technical solutions and beneficial effects of the present application in detail. It should be understood that the above description is only the specific implementation method of the present application and is not intended to limit the scope of protection of the present application. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of the present application should be included in the scope of protection of the present application.

Claims

1. A device control method, characterized in that: include: Get the current working environment status of the device; Determining that the reliability of an environmental proxy model does not meet a standard, wherein the environmental proxy model is used to predict the environmental state at a future time based on the environmental state at a current time and control parameters; Determining target control parameters of the device based on the current working environment state of the device and the default control strategy; controlling the device based on the target control parameter; Acquire interaction data between the default control strategy and the environment, the interaction data including the current working environment state of the device, the control parameters issued by the default control strategy based on the current working environment state of the device, and the environment state at the next moment; Update a training data set based on the interaction data, wherein the training data includes a plurality of training sample pairs, each training sample pair includes an environment state at time t, a control parameter at time t, and an environment state at time t+1; The weight parameters of the environmental proxy model are optimized and adjusted based on the updated training data set to obtain the environmental proxy model with satisfactory reliability.

2. The method according to claim 1, characterized in that The updating of the training data set based on the interaction data comprises: Based on the interaction data, obtaining new training sample pairs; Adding the new training sample pair to the training data set; Determining a target sliding window from a set of candidate sliding windows, wherein the set of candidate sliding windows includes a plurality of sliding windows of different lengths; Selecting the most recent N training samples from the training data set based on the target sliding window; Based on the N training samples, the updated training data set is obtained.

3. The method according to claim 2, characterized in that The determining of the target sliding window from the candidate sliding window set includes: The target sliding window is determined based on the probability of each sliding window in the candidate sliding window set, the probability of each sliding window is determined based on feedback information corresponding to each sliding window, and the feedback information corresponding to each sliding window indicates the prediction error of the training sample selected from the training data set based on each sliding window for the trained environment agent model.

4. The method according to claim 3, characterized in that Also includes: The probability of each sliding window in the candidate sliding window set is periodically updated based on the AMAB algorithm.

5. The method according to any one of claims 1 to 4, characterized in that: The determining the target control parameter of the device based on the current environmental state and the default control strategy includes: Determine a first control parameter based on the current operating environment state of the device and the default control strategy; The first control parameter is disturbed within a preset range to obtain the target control parameter.

6. The method according to any one of claims 1 to 5, characterized in that: Also includes: Determining that the reliability of the environmental agent model meets the requirements; Performing optimization and solving based on the current working environment state of the device and the environment agent model to obtain a second control parameter; Based on the second control parameter, the target control parameter is determined.

7. The method according to claim 6, characterized in that The determining the target control parameter based on the second control parameter includes: The second control parameter is optimized and corrected based on prior information to obtain the target control parameter, wherein the prior information includes an optimal ratio of control parameters with correlation in a control parameter set.

8. The method according to any one of claims 1 to 7, characterized in that: The initial environment proxy model is trained based on an initial training data set, and the initial training data set is obtained based on interaction data between the device and the environment when it is powered on for the first time.

9. The method according to claim 8, characterized in that The interaction data between the device and the environment at the first startup includes: the current working environment state of the device, the third control parameter, and the environment state at the next moment; The third control parameter is obtained by perturbing the control parameter issued by the default control strategy based on the current working environment state of the device with a preset amplitude.

10. The method according to any one of claims 1 to 9, characterized in that: The reliability of the environment proxy model is determined based on the accuracy and / or a priori relevance of the environment proxy model.

11. The method according to any one of claims 1 to 10, characterized in that: The equipment is an air conditioner; The environmental status includes at least one of indoor temperature, actual value of internal fan speed, indoor coil temperature, actual value of compressor frequency, compressor exhaust temperature, actual value of expansion valve opening, accumulated power, actual value of external fan speed, and outdoor temperature; The target control parameter includes at least one of a target value for the speed of the inner fan, a target value for the frequency of the compressor, a target value for the opening of the expansion valve, and a target value for the speed of the outer fan.

12. The method according to any one of claims 1 to 11, characterized in that: The environmental agent model is a lightweight linear model or a decision tree model.

13. A device control device, characterized in that: include: The first acquisition module is used to obtain the current working environment status of the device; A reliability judgment module, used to determine that the reliability of the environmental proxy model does not meet the standard, wherein the environmental proxy model is used to predict the environmental state at a future time based on the environmental state at a current time; A determination module, used to determine the target control parameters of the device based on the current working environment state of the device and the default control strategy; A control module, configured to control the device based on the target control parameter; A second acquisition module is used to acquire interaction data between the default control strategy and the environment, wherein the interaction data includes the current working environment state of the device, the control parameters issued by the default control strategy based on the current working environment state of the device, and the environment state at the next moment; An updating module, configured to update a training data set based on the interaction data, wherein the training data includes a plurality of training sample pairs, each training sample pair including an environment state at time t, a control parameter at time t, and an environment state at time t+1; The optimization module is used to optimize and adjust the weight parameters of the environmental proxy model based on the updated training data set to obtain the environmental proxy model with satisfactory reliability.

14. A computing device comprising a memory and a processor, characterized in that: Instructions are stored in the memory, and when the instructions are executed by the processor, the method according to any one of claims 1 to 12 is implemented.

15. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the method according to any one of claims 1 to 12 is implemented.

Images