A device control method and apparatus

Abstract

The application provides a device control method and device, the method comprises: acquiring an environment state in which a device currently operates; determining that reliability of an environment agent model is not up to standard, and controlling the device to operate by using a default control strategy; acquiring interaction data between the default control strategy and the environment; updating a training data set based on the interaction data; and optimizing and adjusting weight parameters of the environment agent model based on the updated training data set to obtain an environment agent model with up-to-standard reliability. The device control method provided by the application enables the environment agent model to quickly adapt to changes in the environment, ensures that a control strategy optimized based on the environment agent model is not degraded, and improves performance and energy-saving effect of the device.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of device control, and in particular to a device control method and device. BACKGROUND

[0002] Air conditioners rank first in terms of power consumption among all mainstream home appliances, and most air conditioner manufacturers will take energy saving as one of the main selling points of the product. To continuously iterate and optimize the energy saving effect of the air conditioner, on the one hand, long-term research and development investment is needed for the underlying hardware and software, such as the compressor heat storage technology and other links, and on the other hand, the online control strategy also needs to be optimized. The traditional online control strategy optimization is generally based on the rule-based algorithm obtained by the air conditioner manufacturer manually debugging in the fixed laboratory, which often cannot guarantee the optimality of the performance of the online control strategy. The main reason is that there is a not small difference between the customer environment and the laboratory environment, which leads to performance degradation of the online control strategy in the customer environment. SUMMARY

[0003] Embodiments of the present application provide a device control method and device, which can enable the environment agent model to quickly adapt to changes in the environment, ensure that the control strategy optimized based on the environment agent model does not degrade, and improve the performance and energy saving effect of the device.

[0004] In a first aspect, the present application provides a device control method, comprising: obtaining an environment state in which a device is currently working (which can be referred to as a current environment state); determining that the reliability of an environment agent model does not meet a standard, the environment agent model being used to predict an environment state at a future time based on an environment state at a current time and a control parameter; determining a target control parameter of the device based on the current environment state and a default control strategy; controlling the device based on the target control parameter; obtaining interaction data of the default control strategy and the environment, the interaction data including the current environment state, a control parameter issued by the default control strategy based on the current environment state, and an environment state at a next time; updating a training data set based on the interaction data, the training data including a plurality of training sample pairs, each training sample pair including an environment state at time t, a control parameter at time t, and an environment state at time t+1; and optimizing and adjusting a weight parameter of the environment agent model based on the updated training data set to obtain an environment agent model with reliable reliability.

[0005] The device control method provided in the application guarantees that the control strategy of the device does not deteriorate obviously by controlling the device by using a default control strategy (such as a control strategy provided by the device when it is shipped from the factory) when the reliability of the environment agent model does not meet the standard (for example, the prediction accuracy of the environment agent model is lower than a set threshold due to a dramatic change in the environment), and then updating the training data set based on the interaction data of the default control strategy and the environment, obtaining new training samples based on the interaction data, updating the training data set by using the new training samples, and training the environment agent model by using the updated training data set to optimize the environment agent model so that the reliability of the environment agent model meets the standard.

[0006] In one possible implementation, one specific implementation of updating the training data set based on the interaction data is as follows: obtaining a new training sample pair based on the interaction data; adding the new training sample pair to the training data set; determining a target sliding window from a candidate sliding window set, the candidate sliding window set including sliding windows of different lengths; selecting the latest N training samples from the training data set based on the target sliding window; and obtaining an updated training data set based on the N training samples.

[0007] The training data set is updated by using the dynamic sliding window algorithm, so that the rationality of updating the training data set is guaranteed, and the adaptation speed of the environment agent model to the environment is improved.

[0008] In another possible implementation, one specific implementation of determining the target sliding window from the candidate sliding window set is as follows: determining the target sliding window based on the probabilities of the sliding windows in the candidate sliding window set, the probability of each sliding window being determined based on feedback information corresponding to the sliding window, and the feedback information corresponding to each sliding window indicating a prediction error of the environment agent model trained based on the training samples selected from the training data set by using each sliding window.

[0009] In another possible implementation, the device control method provided in the application further includes: periodically updating the probabilities of the sliding windows in the candidate sliding window set based on the AMAB algorithm.

[0010] The optimal sliding window length is quickly selected by using the AMAB algorithm, so that the adaptation speed and accuracy performance of the environment agent model to the environment are improved.

[0011] In another possible implementation, one specific implementation of determining the target control parameter of the device based on the current environment state and the default control strategy is as follows: determining a first control parameter based on the current working environment state of the device and the default control strategy; and perturbing the first control parameter within a preset amplitude to obtain the target control parameter.

[0012] The diversity of the collected training samples is improved by randomly perturbing the control parameter issued by the default control strategy, and the modeling speed of the environment agent model to the environmental change is improved.

[0013] In another possible implementation, the device control method provided by the present application further includes: determining that the environment agent model reliability meets a standard; performing optimization solving based on the current working environment state of the device and the environment agent model to obtain second control parameters; and determining target control parameters based on the second control parameters.

[0014] When the environment agent model reliability meets the standard, i.e., the environment agent model can accurately model the changes of the environment, the online control strategy based on the optimization of the environment agent model is used to control the device, thereby improving the performance and energy saving effect of the device.

[0015] In another possible implementation, one specific implementation of determining the target control parameters based on the second control parameters is: performing optimization correction on the second control parameters based on prior information to obtain the target control parameters, and the prior information includes an optimal proportion of control parameters with correlation in the control parameter set.

[0016] The control parameters issued by the control strategy based on the optimization of the environment agent model are further corrected based on expert prior knowledge, thereby further ensuring the accuracy of the parameters issued by the control strategy and further improving the performance of the device and reducing the working energy consumption of the device.

[0017] In another possible implementation, the initial environment agent model is trained based on an initial training data set, and the initial training data set is obtained based on the interaction data of the device and the environment during initial startup.

[0018] The training data set is obtained by exploration of the device in a specific customer environment, and the environment agent model is trained based on the training data set. The environment agent model is more suitable for the environment and can more accurately predict the changes of the environment, thereby improving the accuracy of the control strategy and further improving the performance and energy saving effect of the device.

[0019] In another possible implementation, the interaction data of the device and the environment during initial startup includes: a current environment state, third control parameters, and an environment state at a next time; and the third control parameters are obtained based on a preset amplitude of disturbance on the control parameters issued by a default control strategy based on the current environment state.

[0020] In this way, the sample complexity of the environment agent model fitting is greatly reduced while ensuring that the performance does not significantly degrade during sample exploration, thereby solving the cold start problem.

[0021] In another possible implementation, the reliability of the environment agent model is determined based on the accuracy and / or prior correlation of the environment agent model.

[0022] In another possible implementation, the device is an air conditioner; the environmental conditions include at least one of the following: indoor temperature, actual value of indoor fan speed, indoor coil temperature, actual value of compressor frequency, compressor discharge temperature, actual value of expansion valve opening, cumulative power consumption, actual value of outdoor fan speed, and outdoor temperature; the target control parameters include at least one of the following: target value of indoor fan speed, target value of compressor frequency, target value of expansion valve opening, and target value of outdoor fan speed.

[0023] In another possible implementation, the environment agent model is a lightweight linear model or decision tree model, which reduces computational overhead and allows the environment agent model to be deployed on terminal devices with limited computing power.

[0024] Secondly, this application also provides a device control apparatus, including a first acquisition module, a reliability judgment module, a determination module, a control module, a second acquisition module, an update module, and an optimization module. The first acquisition module acquires the current environmental state of the device (hereinafter referred to as the current environmental state); the reliability judgment module determines that the environmental proxy model's reliability is substandard, and the environmental proxy model predicts the environmental state at future times based on the current environmental state; the determination module determines the target control parameters of the device based on the current environmental state and the default control strategy; the control module controls the device based on the target control parameters; the second acquisition module acquires interaction data between the default control strategy and the environment, including the current environmental state, the control parameters issued by the default control strategy based on the current environmental state, and the environmental state at the next time step; the update module updates the training dataset based on the interaction data, the training data including multiple training sample pairs, each training sample pair including the environmental state at time t, the control parameters at time t, and the environmental state at time t+1; the optimization module optimizes and adjusts the weight parameters of the environmental proxy model based on the updated training dataset to obtain a reliable environmental proxy model.

[0025] In one possible implementation, the update module is specifically used for: obtaining new training sample pairs based on the interaction data; adding the new training sample pairs to the training dataset; determining the target sliding window from a set of candidate sliding windows, the set of candidate sliding windows including multiple sliding windows of different lengths; selecting the N nearest training samples from the training dataset based on the target sliding window; and obtaining the updated training dataset based on the N training samples.

[0026] In another possible implementation, a specific implementation of determining the target sliding window from the candidate sliding window set is as follows: the target sliding window is determined based on the probability of each sliding window in the candidate sliding window set, the probability of each sliding window is determined based on the feedback information corresponding to each sliding window, and the feedback information corresponding to each sliding window indicates the prediction error of the trained environment agent model based on the training samples selected from the training dataset based on each sliding window.

[0027] In another possible implementation, the update module is also used to periodically update the probability of each sliding window in the candidate sliding window set based on the AMAB algorithm.

[0028] In another possible implementation, the determining module is specifically used to: determine a first control parameter based on the current operating environment state of the device and the default control strategy; and perturb the first control parameter within a preset range to obtain the target control parameter.

[0029] In another possible implementation, the determination module is also used to determine whether the reliability of the environmental proxy model meets the standard; to perform optimization solutions based on the current operating environment of the device and the environmental proxy model to obtain the second control parameters; and to determine the target control parameters based on the second control parameters.

[0030] In another possible implementation, a specific way to determine the target control parameter based on the second control parameter is as follows: the second control parameter is optimized and corrected based on prior information to obtain the target control parameter, wherein the prior information includes the optimal proportion of control parameters that are correlated in the control parameter set.

[0031] In another possible implementation, the initial environment agent model is trained based on an initial training dataset, which is obtained from the interaction data between the device and the environment upon initial power-on.

[0032] In another possible implementation, the interaction data between the device and the environment upon initial power-on includes: the current environmental state, the third control parameter, and the environmental state at the next moment; the third control parameter is obtained by perturbing the control parameters issued by the default control strategy based on the current environmental state with a preset amplitude.

[0033] In another possible implementation, the reliability of the environment proxy model is determined based on the accuracy and / or prior correlation of the environment proxy model.

[0034] In another possible implementation, the device is an air conditioner; the environmental conditions include at least one of the following: indoor temperature, actual value of indoor fan speed, indoor coil temperature, actual value of compressor frequency, compressor discharge temperature, actual value of expansion valve opening, cumulative power consumption, actual value of outdoor fan speed, and outdoor temperature; the target control parameters include at least one of the following: target value of indoor fan speed, target value of compressor frequency, target value of expansion valve opening, and target value of outdoor fan speed.

[0035] In another possible implementation, the environment agent model is a lightweight linear model or a decision tree model.

[0036] Thirdly, embodiments of this application provide a computing device, including a memory and a processor, wherein the memory stores instructions that, when executed by the processor, cause the method described in the first aspect or any possible implementation of the first aspect to be implemented.

[0037] Fourthly, embodiments of this application provide a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, causes the method described in the first aspect or any possible implementation thereof to be implemented.

[0038] Fifthly, embodiments of this application also provide a computer program or computer program product, which includes instructions that, when executed, cause a computer to perform the method described in the first aspect or any possible implementation thereof.

[0039] In a sixth aspect, embodiments of this application also provide a chip including at least one processor and a communication interface, the processor being configured to perform the method described in the first aspect or any possible implementation thereof. Attached Figure Description

[0040] Figure 1 A schematic diagram of an air conditioner to which the device control method provided in the embodiments of this application can be applied is shown;

[0041] Figure 2 This invention provides a schematic diagram illustrating one implementation flow of the device control method.

[0042] Figure 3 A flowchart illustrating a device control method provided for the implementation of this application;

[0043] Figure 4 This is a schematic diagram of the structure of a device control apparatus provided in an embodiment of this application;

[0044] Figure 5 A schematic diagram of the structure of a computing device provided in an embodiment of this application. Detailed Implementation

[0045] The term "and / or" used in this article describes the relationship between related objects, indicating that three relationships can exist. For example, A and / or B can represent three cases: A alone, A and B simultaneously, and B alone. The symbol " / " in this article indicates that the related objects are in an "or" relationship; for example, A / B means A or B.

[0046] The terms "first" and "second," etc., used in the specification and claims herein are used to distinguish different objects, not to describe a specific order of objects. It should be understood that such terms are interchangeable where appropriate; this is merely a way of distinguishing objects with the same properties in the description of embodiments of this application. Furthermore, the terms "comprising" and "having," and any variations thereof, are intended to cover non-exclusive inclusion, so that a process, method, system, product, or apparatus that comprises a list of elements is not necessarily limited to those elements, but may include other elements not expressly listed or inherent to such processes, methods, systems, products, or apparatus.

[0047] In the embodiments of this application, the terms "exemplary" or "for example" are used to indicate that something is an example, illustration, or description. Any embodiment or design that is described as "exemplary" or "for example" in the embodiments of this application should not be construed as being more preferred or advantageous than other embodiments or design. Specifically, the use of the terms "exemplary" or "for example" is intended to present the relevant concepts in a specific manner.

[0048] In the description of the embodiments of this application, unless otherwise stated, "multiple" means two or more, for example, multiple processing units means two or more processing units, multiple elements means two or more elements, etc.

[0049] For decades, almost all air conditioner manufacturers have focused on optimizing cooling efficiency at the hardware level, but without supporting environmentally adaptive algorithms to fully utilize hardware performance (for example, household air conditioners often overshoot during startup, resulting in energy waste). This has created new opportunities for air conditioner energy saving. In recent years, many air conditioner manufacturers have realized the opportunities in AI-based air conditioner energy-saving algorithms. Data-driven AI algorithms have some unique advantages in air conditioner energy saving: (1) Strong adaptability: They can adaptively find the optimal control strategy based on different environments. (2) Strong compatibility: They will not conflict with existing system frameworks and are easy to port. (3) Low expert dependence: Even those who are not HVAC experts can learn better energy-saving solutions through AI algorithms and some basic HVAC knowledge.

[0050] However, replacing traditional air conditioning online control strategies with data-driven AI algorithms presents the following challenges: (1) Limited computing power on the edge: Air conditioning chips have limited memory and computing power, which cannot support highly complex models and algorithms. (2) High generalization capability requirements: Algorithms and models are required to quickly adapt to and solve optimal strategies in environments with incomplete observations and non-steady states. (3) High robustness requirements: Algorithms and models are required to remain unchanged from the baseline under any environmental conditions and not experience temperature control failures. Most related technologies are based on the single-step strategy benefits of proxy models. For example, in 2017, Google used deep neural networks to fit the energy consumption of large data centers under different operating conditions. Some air conditioning manufacturers also use simpler models, such as Bayesian linear regression, to fit the energy consumption of air conditioners under different operating conditions. The premise for these solutions to achieve ideal energy saving is that these proxy models can prepare to model the mapping relationship between the strategies and energy consumption under different operating conditions.

[0051] For example, one related technology is a data center energy-saving algorithm based on deep neural networks. Its core idea is to first collect data from the data center under different operating conditions. Based on this data, a neural network model (as shown in the diagram) is used to fit the mapping relationship between different operating conditions and energy consumption. Once the model's accuracy meets the requirements, a more energy-efficient strategy can be found using this trained model.

[0052] However, this solution has the following problems:

[0053] High computing power requirement: Cannot be used when computing power is limited at the edge.

[0054] High sample complexity: A large amount of diverse data is required to ensure the accuracy of the model.

[0055] Poor adaptability to the environment: If the observed state cannot fully describe the entire system, or if the external environment changes drastically, the surrogate model based on the neural network does not have the ability to adapt quickly, resulting in unreliable strategies based on model optimization.

[0056] Long-term benefits are not considered: greedy optimization based on a single step cannot guarantee global optimality, and temperature overshoot may still occur even if the model accuracy is very high.

[0057] The second related technology is the online control strategy optimization scheme for air conditioning. It is a rule-based algorithm that is manually debugged by HVAC experts in a specially set up laboratory based on their experience. The optimization cycle is often quite long, usually several months.

[0058] This solution has the following problems:

[0059] Poor generalization ability: Control strategies debugged in the laboratory are often not optimal in a wide variety of customer environments.

[0060] Long-term benefits are not considered: Strategies based on expert experience often cannot support the optimization of highly complex strategies. Debugging strategies only optimizes the benefits of a single step, and the optimized strategies are prone to temperature overshoot, resulting in wasted energy.

[0061] This application's embodiments consider how to accurately model the environment with a small amount of data, without significant degradation compared to the baseline during interaction, thus laying the foundation for subsequent policy optimization. It also addresses how to enable the environmental agent model to quickly adapt to environmental changes, ensuring that policy delivery performance does not degrade during subsequent online learning.

[0062] To address this, this application proposes a device control method that incorporates safety exploration and dynamic local linearization modeling, enabling a lightweight proxy model to quickly adapt to environmental changes and thus efficiently solve energy-saving strategies based on the environmental proxy model.

[0063] The device control method provided in this application can be applied to the control of various devices to achieve energy-saving optimization. For example, it can optimize energy saving for network devices (such as routers) and air conditioning devices. In particular, it can be applied to the energy-saving optimization of air conditioning control strategies. This application uses an air conditioner as an example to illustrate the specific implementation of the device control method provided in this application. The implementation of other devices is similar, and the specific implementation can be referred to the control method for air conditioners. For the sake of brevity, this application will not elaborate further.

[0064] To facilitate understanding of the solutions in the embodiments of this application, the technical terms involved in this document will be explained first.

[0065] Linear model: A machine learning model that assumes a linear mathematical relationship between input and output. This model is suitable for local modeling based on limited data when data volume is small and computing power is limited. Compared with commonly used neural network models, it has the advantage of stronger interpretability and is easier to incorporate expert priors.

[0066] Safety exploration: Since the algorithm has no prior knowledge of the environment in the early stages, it needs to ensure the diversity of collected data while ensuring that the deployed strategy does not deteriorate significantly compared to the baseline, so as to facilitate more accurate modeling of the environment.

[0067] Dynamic sliding window: A machine learning algorithm for non-steady-state environments that uses data from recent times as input for model fitting, and the specific amount of data selected can be dynamically adjusted based on historical feedback.

[0068] Figure 1 A schematic diagram of an air conditioner to which the device control method provided in the embodiments of this application can be applied is shown. Figure 1As shown, an air conditioner includes an indoor unit and an outdoor unit. The indoor unit includes an indoor fan and an evaporator, while the outdoor unit includes a compressor, a condenser, an outdoor fan, and an expansion valve. The user inputs the desired temperature, mode, fan speed, and other settings via the air conditioner's remote control or control panel. The air conditioner responds to these settings by sending control signals to various components based on its own control strategy. For example, it sends a control signal to the indoor fan to control its speed, a control signal to the evaporator (generated based on control parameters) to control its operation, a control signal to the compressor to control its operating frequency, a control signal to the expansion valve to control its opening, and a control signal to the outdoor fan to control its speed. This allows the air conditioner to adjust the temperature to the user-set target temperature.

[0069] As the air conditioner operates, the external environment changes accordingly (for example, as the air conditioner cools, the temperature of the outside environment decreases). The air conditioner's control strategy adjusts the control signals sent to various components based on the current environmental conditions of the environment in which the air conditioner is located, in order to achieve effective temperature control and energy-saving optimization.

[0070] Air conditioner control parameters refer to the operating parameters that control certain components within the air conditioner (such as the compressor, indoor unit fan, expansion valve, and outdoor fan) to adjust the temperature to the user's desired target temperature. For example... Figure 1 As shown, the control quantities (i.e. control parameters) of the air conditioner include the target value of the indoor fan speed (i.e., the indoor fan speed value indicated by the control parameters), the target value of the compressor frequency, the target value of the expansion valve opening, and the target value of the outdoor fan speed.

[0071] Environmental conditions refer to certain state parameters of the environment in which the air conditioner operates, or in other words, environmental state parameters that affect the control decisions of the air conditioner, such as... Figure 1 As shown, the state variables (i.e., environmental states) may include indoor temperature, actual indoor fan speed (i.e., actual indoor fan speed), indoor coil temperature, actual compressor frequency, compressor discharge temperature, actual expansion valve opening, cumulative power consumption, actual outdoor fan speed, and outdoor temperature.

[0072] In some other examples, the control parameters for the air conditioner may also include only some of the following: target indoor fan speed, target compressor frequency, target expansion valve opening, and target outdoor fan speed. Environmental conditions may also include only some of the following: indoor temperature, actual indoor fan speed, indoor coil temperature, actual compressor frequency, compressor discharge temperature, actual expansion valve opening, cumulative power consumption, actual outdoor fan speed, and outdoor temperature.

[0073] Environmental conditions can be detected by various sensors. For example, indoor temperature, indoor coil temperature, compressor exhaust temperature, and outdoor temperature can be detected by temperature sensors installed at corresponding locations. For instance, a temperature sensor can be installed in the indoor space where the indoor unit is located to detect the indoor temperature. A temperature sensor can be installed at the indoor coil to detect the indoor coil temperature. A temperature sensor can be installed at the compressor exhaust port to detect the compressor exhaust temperature. A temperature sensor can be installed in the outdoor space where the outdoor unit is located to detect the outdoor temperature. For example, the actual speed of the indoor fan and the actual speed of the outdoor fan can be detected by speed sensors. For instance, speed sensors can be installed at both the indoor and outdoor fans to detect their speeds, obtaining the actual speeds of the indoor and outdoor fans respectively. As another example, the actual compressor frequency can be detected by a frequency sensor. The actual expansion valve opening can be detected by a distance sensor. Accumulated power consumption refers to the cumulative power consumed by the air conditioner, which can be detected by an energy consumption sensor.

[0074] Some environmental conditions can be obtained without sensors, through other means. For example, they can be obtained from other channels via communication. For instance, in work scenarios with other smart home devices or monitoring systems installed, indoor air temperature can be obtained from records of other smart devices, while outdoor air temperature can be obtained from an online server based on weather forecasts and other information.

[0075] The embodiments of this application are as follows: Figure 1 The air conditioner shown is equipped with the device control method provided in this application embodiment. Considering that the computing power of air conditioners is usually limited, the device control method in this application embodiment uses a lightweight environmental proxy model to predict future environmental changes. Through an online strategy optimization framework based on dynamic sliding window local linearization modeling, the environmental adaptation speed is greatly improved. The optimal control strategy can be solved efficiently in dynamic environments to optimize the air conditioner for energy saving.

[0076] Figure 2 A schematic diagram illustrating an implementation flow of the device control method provided in this application is shown. This method can be implemented by... Figure 1 The air conditioning operation is shown below.

[0077] First, the interaction data between the air conditioner and its working environment is obtained. An environment proxy model is then modeled based on this interaction data to obtain an environment proxy model for the specific working environment in which the air conditioner is currently located. Compared with the model modeled in the laboratory environment, the environment proxy model in this embodiment of the application makes more accurate predictions, thereby making the optimization solution of the online control strategy based on the environment proxy model more precise.

[0078] To ensure the diversity of the collected interaction data and to prevent the air conditioner from experiencing performance degradation (such as temperature control failure) during the acquisition of interaction data, this application embodiment adopts a safe exploration approach. When the air conditioner is turned on for the first time, an appropriate perturbation is added to the air conditioner's baseline control strategy. This allows for accurate local modeling of the environment with only a small number of interactions with the environment, and the interaction process does not result in significant degradation compared to the baseline control strategy.

[0079] For example, after the air conditioner is initially turned on, a random disturbance of appropriate magnitude is added to the control parameters issued by the original control strategy of the air conditioner. Then, the interaction data between the control strategy and the environment is collected, that is, the control parameters issued by the control strategy are executed after the air conditioner operates, resulting in changes in the environmental state. Based on the interaction data, a triplet (x...) is obtained. t ,u t ,x t+1 ) data, where x t U represents the environmental state at time t. t x represents the control parameters issued by the control strategy at time t. t+1 This represents the environmental state at time t+1.

[0080] It should be noted that the original control strategy of the air conditioner refers to the online control strategy that comes with the air conditioner from the factory. This control strategy can be any type of control strategy, such as a rule-based algorithm based on manual debugging by experts.

[0081] The meaning of an appropriate level of disturbance is that the disturbance to the control parameters issued by the original control strategy of the air conditioner will not affect the temperature control of the air conditioner. For example, after the control parameters after the disturbance are executed, the air conditioner can still control the temperature at the target temperature.

[0082] For example, the appropriate perturbation range for the indoor fan speed is ±5, the appropriate perturbation range for the compressor frequency is ±3, the appropriate perturbation range for the expansion valve opening is ±2, and the appropriate perturbation range for the outdoor fan speed is ±5. Then, according to the original control strategy of the air conditioner, the target control parameters to be issued are: indoor fan speed target value M1, compressor frequency target value M2, expansion valve opening target value M3, and outdoor fan speed target value M4. After applying appropriate random perturbations to each control parameter, the target values ​​become: indoor fan speed target value M1+4, compressor frequency target value M2-2, expansion valve opening target value M3+1, and outdoor fan speed target value M4-3.

[0083] In this way, by subjecting the control parameters issued by the original control strategy of the air conditioner to random perturbations of an appropriate magnitude, diverse data with sufficient modeling accuracy can be quickly obtained.

[0084] Then, the acquired triple data is used to model the environment proxy model, or the acquired triple data is used as a training dataset to train the environment proxy model to obtain an environment proxy model with satisfactory prediction accuracy.

[0085] Perform security exploration until a sufficient number of triplet data points are obtained for modeling accuracy. Assuming a sufficient number is 90 triplet data points, for example, with a sampling interval of two minutes, i.e., one triplet data point (x) is sampled every two minutes. t u t x t+1 If 30 triplet data points can be collected in one hour and 90 triplet data points can be collected in three hours, then the random perturbation of the control parameters issued by the original control strategy will be terminated, and the 90 triplet data points collected will be used to model the environment proxy model.

[0086] The environmental proxy model obtained from modeling can predict the environmental state at time t+1 based on the environmental state at time t and the control parameters at time t. In other words, the environmental state at time t and the control parameters at time t are used as inputs to the environmental proxy model, and the environmental proxy model can output the environmental state at time t+1 through inference.

[0087] Security exploration enables the rapid acquisition of diverse and sufficient training samples without prior knowledge of the environment. These samples are then used to train the environment agent model, resulting in a preliminarily trained model that solves the cold start problem.

[0088] Considering the limited computing power of air conditioners, a lightweight model is adopted for the environmental proxy model. For example, the environmental proxy model can be a linear model or a decision tree model, etc., to reduce computing power consumption and facilitate deployment on the air conditioner.

[0089] Lightweight models refer to neural network models with fewer parameters. For example, the environmental proxy model in this embodiment has 10M parameters and requires less computing power from the device that deploys it, such as less than 1000 MOPs. This makes it easy to deploy on terminal devices with limited computing power, such as air conditioners. The environmental proxy model provided in this embodiment can be deployed on an air conditioner, enabling the air conditioner to run an online control strategy optimized based on the environmental proxy model, thus ensuring the air conditioner's temperature control performance and energy-saving effect.

[0090] After obtaining the trained environmental proxy model, the current environmental state and the environmental proxy model are used for optimization to obtain the control parameters that the air conditioner needs to issue. In other words, the optimization model composed of the environmental proxy model is used to optimize based on the current environmental state to obtain an initial solution, and the control parameters to be issued by the air conditioner are determined based on this initial solution.

[0091] For example, the optimization model includes the following set of optimization equations:

[0092]

[0093] stx t+1 =Ax t +Bu t

[0094] x t ∈X

[0095] Where, x τ U represents the environmental state at time τ. τ The control parameter at time τ, x t U represents the environmental state at time t. t The control parameter at time t, x t+1 Let represent the environmental state at time t+1, Q and R be the weight matrices defining the single-step loss function, A and B be the weight parameters of the environmental proxy model fitted based on the historical environment, and X be the feasible region of the control parameters.

[0096] By solving the above optimization equations, an initial solution is obtained. In one example, this initial solution can be directly used as the current control parameter to control the air conditioner's operation. For example, by solving the above optimization equations, the initial solution obtained is U = u. τ1 =M5, u τ2 =M6, u τ3 =M7, u τ4 =M8. Based on this initial solution, the control parameters to be issued by the air conditioner are: target value of indoor fan speed M5, target value of compressor frequency M6, target value of expansion valve opening M7, and target value of outdoor fan speed M8.

[0097] In another example, in order to ensure the quality of the control parameters issued by the air conditioner (i.e., the air conditioner is controlled according to the control parameters, so that the temperature control of the air conditioner is more stable and the air conditioner is more energy-efficient), after obtaining the initial solution of the optimization equation system, the initial solution will be further processed, and the control parameters to be issued by the air conditioner will be determined according to the processed initial solution.

[0098] For example, the initial solution is optimized and corrected using expert priors, and the optimized and corrected control parameters are used to control the corresponding components in the air conditioner. For instance, the expert priors may include optimal ratios of different control parameters. The control parameters obtained from the initial solution are adjusted to conform to these optimal ratios, thus optimizing the initial solution. The optimized and adjusted initial solution is then issued as the current control parameters to control the corresponding components of the air conditioner. In other words, expert experience is used to perform a post-mapping on the initial solution so that the control parameters satisfy the expert priors.

[0099] For example, expert experience knows that the optimal opening degree of the expansion valve and the compressor frequency are positively correlated. However, due to the limited capabilities of the optimization model, it cannot accurately model the matching relationship. Therefore, expert experience is needed to perform a post-mapping on the initial solution obtained from solving the optimization equations, so that the control parameters satisfy the expert's prior knowledge. For instance, the compressor frequency obtained from the model solution is M6, and the expansion valve opening degree is M7. However, the prior knowledge is that the optimal ratio of compressor frequency to expansion valve opening degree is 1:2. Adjusting the expansion valve opening degree from M7 to 2M6 is equivalent to making a correction on the original basis, further improving the air conditioning cooling / heating efficiency or energy utilization efficiency.

[0100] The accuracy of the initial solution obtained from the optimization model is closely related to the reliability of the environmental proxy model; in other words, the reliability of the environmental proxy model determines the reliability of the initial solution obtained from the optimization model. Therefore, to ensure the accuracy of the air conditioning online control strategy, during actual operation, when control parameters need to be issued, it is necessary to first determine whether the reliability of the environmental proxy model meets the standard. If the reliability of the environmental proxy model meets the standard, the issuance strategy is optimized using the current environmental state and the optimization model. Then, the issuance strategy is corrected and optimized based on expert experience, and the air conditioning is controlled according to the optimized issuance strategy. If the reliability of the environmental proxy model does not meet the standard, the default control strategy is used to issue control parameters. This ensures a minimum level of quality for the air conditioning online control strategy and prevents significant degradation of the control strategy.

[0101] The reliability of an environmental surrogate model can be determined based on its accuracy and / or prior correlation. For example, if the prediction accuracy of the environmental surrogate model is lower than a preset threshold, its reliability is deemed unacceptable. This is similar to the case where the prediction accuracy of the environmental surrogate model has consistently fallen below the preset threshold over several steps. Alternatively, if the prior correlation of the environmental surrogate model does not meet the requirements, its reliability is also deemed unacceptable. For instance, a high compressor frequency control parameter may have a positive effect on cooling, but if the temperature predicted by the environmental surrogate model does not decrease, then the prior correlation of the environmental surrogate model does not meet the requirements, and therefore its reliability is deemed unacceptable. Furthermore, if both the prediction accuracy and prior correlation of the environmental surrogate model are below the preset threshold, its reliability is deemed unacceptable.

[0102] The default control strategy can be the online control strategy that comes pre-installed with the air conditioner, such as a control strategy based on a proportional integral derivative (PID) control algorithm. Appropriate random perturbations are applied to the control parameters issued based on the default control strategy to ensure data diversity. Then, interaction data between the default control strategy and the environment are collected to obtain new triplet data (…). x t, u t x t+1 ), using new triplet data (x t u t x t+1 Update the training dataset, and then use the updated training dataset to optimize and train the environmental agent model, so that the environmental agent model can quickly adapt to changes in the environment and ensure the prediction accuracy of the environmental agent model.

[0103] Since environmental changes are not steady-state, the working environment of air conditioners may change drastically. In this case, the reliability of the environmental proxy model will be insufficient, making the optimization strategy based on the environmental proxy model unreliable. To address this, this application proposes an online strategy optimization framework based on dynamic sliding window local linearization modeling, which enables the environmental proxy model to quickly adapt to environmental changes and efficiently solve the optimal energy-saving strategy in dynamic environments.

[0104] For example, when the reliability of the environmental proxy model is determined to be substandard, the default control strategy of the air conditioner is adopted to issue control parameters, and the interaction data between the default control strategy and the environment is collected to obtain new triplet data. The training dataset is then updated based on the new triplet data. For instance, each time a new triplet data is obtained, it is added to the training dataset, and the oldest triplet data is deleted to ensure the stability of the data volume in the training dataset. Then, the length of the dynamic sliding window is updated, and multiple triplet data are selected from the training dataset based on the updated dynamic sliding window. The training data subset formed by the selected multiple triplet data is used to train and optimize the parameters of the environmental proxy model, enabling the environmental proxy model to quickly adapt to changes in the environment.

[0105] In this application embodiment, a feasible set of sliding windows of different lengths is pre-constructed. w ={W0, W1, ..., W N Different sliding window lengths mean different time spans for the triples selected from the training dataset, resulting in varying prediction accuracy of the trained environment agent model. The selection of the optimal sliding window (also known as the optimal length window) from the set of sliding windows determines the prediction accuracy of the trained and optimized environment agent model.

[0106] Based on feedback information from past selections of different sliding window lengths (e.g., modeling error, i.e., the prediction error of the environment proxy model trained using a subset of training data consisting of triples selected from the training dataset using different sliding windows), the probability of each sliding window in the set being selected is determined. The better the feedback information of a sliding window (i.e., the smaller the modeling error), the higher the probability of that sliding window being selected in the set. That is, during optimization training, the environment proxy model selects a target sliding window from the set (windows with higher probabilities are more likely to be selected as the target sliding window). Based on the target sliding window, multiple triples are selected from the training dataset. The weight parameters of the environment proxy model are trained and optimized using these triples selected by the target sliding window. The environment proxy model is updated until it adapts to the changed environment. If the reliability of the environment proxy model meets the standard under the changed environment, then optimization is performed using the environment proxy model.

[0107] To further enhance the adaptability of the environmental proxy model to environmental changes, this embodiment updates the probability of each window in the window set using an adversarial multi-armed bandit (AMAB) algorithm, based on feedback from past window selections at a preset period, such as every certain time interval T0. In other words, the AMAB algorithm dynamically updates the selected window length (i.e., the window with the highest probability) for the next time interval, thereby achieving the effect of quickly learning the optimal window length over time. Furthermore, this algorithm is theoretically optimal. Thus, by rapidly selecting the optimal window length using the AMAB algorithm, the adaptability and prediction accuracy of the environmental proxy model are improved.

[0108] In summary, the device control method provided in this application first collects diverse triplet data through safety exploration during the initial stage of the air conditioner's initial startup (e.g., three hours). This collected triplet data is then used to model an environment proxy model. During the online policy control phase, at each moment, the reliability of the current environment proxy model is first determined. If the environment proxy model is reliable, control parameters are optimized and issued based on it; otherwise, control parameters are issued based on the default control policy. Then, during the interaction between the default control policy and the environment, interaction data is collected to obtain new triplet data. The training dataset is updated based on the new triplet data, and the environment proxy model is updated and optimized based on the updated training dataset. After determining that the reliability of the updated environment proxy model meets the standard, control parameters are optimized and issued based on the updated environment proxy model. This process is repeated continuously. The device control method provided in this application significantly reduces the complexity of the environmental proxy model's fitting samples through safe exploration while ensuring that the air conditioner's performance does not deteriorate significantly during the exploration period, thus solving the cold start problem. Through online strategy optimization based on dynamic sliding window local linearization modeling, it greatly improves the speed of environmental adaptation and can efficiently solve the optimal control strategy for the device in dynamic environments. Through hierarchical optimization embedding expert experience (i.e., optimizing and correcting the initial solution through expert priors), it further improves the quality of the optimization model's optimization strategy. Finally, through the AMAB algorithm, it quickly selects the optimal sliding window length, improving the environmental adaptation speed and prediction accuracy of the environmental proxy model.

[0109] Figure 3 This is a flowchart illustrating a device control method provided for implementation of this application. This method can be implemented by a device applying the device control method provided in the embodiments of this application, such as an air conditioner. The device control method provided in the embodiments of this application is described below using an air conditioner as an example. This method includes at least steps S301 to S308.

[0110] In step S301, the current environmental state is obtained.

[0111] Environmental conditions refer to certain state parameters of the environment in which the air conditioner operates, or in other words, environmental state parameters that affect the control decisions of the air conditioner, such as... Figure 1 As shown, the environmental conditions may include indoor temperature, actual indoor fan speed (i.e., actual indoor fan speed), indoor coil temperature, actual compressor frequency, compressor discharge temperature, actual expansion valve opening, cumulative power consumption, actual outdoor fan speed, and outdoor temperature.

[0112] In some other examples, environmental status may also include some of the following environmental status items: indoor temperature, actual indoor fan speed, indoor coil temperature, actual compressor frequency, compressor discharge temperature, actual expansion valve opening, cumulative power consumption, actual outdoor fan speed, and outdoor temperature.

[0113] Current environmental status refers to the current environmental status of the air conditioner, such as one or more of the following: current indoor temperature, actual indoor fan speed, indoor coil temperature, actual compressor frequency, compressor discharge temperature, actual expansion valve opening, cumulative power consumption, actual outdoor fan speed, and outdoor temperature.

[0114] The current environmental status can be obtained in various ways, such as by direct measurement through sensors or by obtaining it from other channels through communication. For specific methods, please refer to the description above. For the sake of brevity, it will not be repeated here.

[0115] In step S302, it is determined that the reliability of the environmental proxy model is substandard. The environmental proxy model is used to predict the environmental state at future times based on the current environmental state and control parameters.

[0116] In this embodiment, the accuracy of the control strategy (also known as the optimization model) based on the environmental surrogate model is closely related to the reliability of the environmental surrogate model. In other words, the reliability of the environmental surrogate model determines the reliability of the initial solution obtained by the optimization model. Therefore, to ensure the accuracy of the online control strategy for the air conditioner, during actual operation, when control parameters need to be issued, it is necessary to first determine whether the reliability of the environmental surrogate model meets the standard, and then determine the subsequent control strategy for the air conditioner based on whether the reliability of the environmental surrogate model meets the standard.

[0117] Whether an environmental proxy model meets the reliability standards can be determined based on its accuracy and / or prior correlation. Specific methods for judging the reliability of an environmental proxy model can be found above; for brevity, they will not be repeated here.

[0118] In another example, considering the limited computing power of air conditioners, a lightweight model is adopted for the environmental proxy model. For example, the environmental proxy model can be a linear model or a decision tree model, etc., to reduce computing power overhead and facilitate deployment on the air conditioner.

[0119] In step S303, the target control parameters of the device are determined based on the current environmental state and the default control strategy.

[0120] If the reliability of the environmental proxy model is determined to be substandard, that is, the accuracy of the solution output by the optimized model is unreliable, then the default control strategy is used to issue control parameters. In this way, the quality of the air conditioner's online control strategy is guaranteed, and the control strategy of the air conditioner is prevented from deteriorating significantly.

[0121] The default control strategy can be an online control strategy that comes pre-installed with the air conditioner, such as a PID control algorithm, or a rule-based algorithm that HVAC experts manually debug based on their experience in a specially set up laboratory, for example, when the current environmental state is x. t The regular algorithm can calculate x t The corresponding control parameter is then determined as the target control parameter.

[0122] Air conditioner control parameters refer to the operating parameters that control certain components within the air conditioner (such as the compressor, indoor unit fan, expansion valve, and outdoor fan) to adjust the temperature to the user's desired target temperature. For example... Figure 1 As shown, the control parameters of the air conditioner include the target value of the indoor fan speed (i.e., the indoor fan speed value indicated by the control parameters), the target value of the compressor frequency, the target value of the expansion valve opening, and the target value of the outdoor fan speed.

[0123] In some other examples, the control parameters of the air conditioner may also include only some of the following control parameter items: target value of indoor fan speed, target value of compressor frequency, target value of expansion valve opening, and target value of outdoor fan speed.

[0124] In step S304, the device is controlled based on the target control parameters.

[0125] After determining the target control parameters, the corresponding components in the air conditioner are controlled using these parameters to influence the external temperature and humidity. For example, if the target control parameters are compressor frequency M1, indoor fan speed M2, expansion valve opening M3, and outdoor fan speed M4, then the air conditioner's processor will control the compressor frequency to be M1, the indoor fan speed to be M2, the expansion valve opening to be M3, and the outdoor fan speed to be M4.

[0126] In step S305, the interaction data between the default control policy and the environment is obtained.

[0127] During the operation of the air conditioner using its default control strategy, interaction data between the default control strategy and the environment is collected, namely, the air conditioner's control parameters and the environmental state. Based on this interaction data, a triplet (x...) is obtained. t u tx t+1 ) data, where x t U represents the environmental state at time t. t x represents the control parameters issued by the control strategy at time t. t+1 This represents the environmental state at time t+1.

[0128] In another example, the control parameters issued based on the default control strategy of the air conditioner are subject to appropriate random perturbations to ensure data diversity. Then, interaction data between the default control strategy and the environment are collected to obtain new triplet data (x...). t u t x t+1 This ensures the diversity of the new triplet data, thereby guaranteeing the sample diversity of the updated training dataset using the new triplets. This allows the environmental proxy model to quickly adapt to changes in the environment and ensures the prediction accuracy of the environmental proxy model.

[0129] In step S306, the training dataset is updated based on the interaction data.

[0130] For example, one way to update the training dataset is to add each new triplet to the training dataset and delete the oldest triplet to ensure the stability of the data volume. Then, update the length of the dynamic sliding window, select multiple triplets from the training dataset based on the updated dynamic sliding window, and use the selected subset of triplets to train and optimize the parameters of the environment proxy model, enabling the environment proxy model to quickly adapt to changes in the environment.

[0131] The specific method for updating the training dataset based on dynamic sliding window can be found in the description above, and will not be repeated here for the sake of brevity.

[0132] In step S307, the weight parameters of the environment agent model are optimized and adjusted based on the updated training dataset to obtain an environment agent model that meets the reliability standards.

[0133] After the reliability of the environmental proxy model meets the standards, the operating parameters of the air conditioner are controlled using a control strategy optimized based on the environmental proxy model. For example, the current environmental state and the optimization model composed of the environmental proxy model mentioned above are used to solve for an initial solution. One approach is to directly use this initial solution as the current control parameter to control the air conditioner. Another approach, to ensure the quality of the control parameters issued by the air conditioner, involves further processing the initial solution after obtaining the optimization equation set, and determining the control parameters to be issued by the air conditioner based on the processed initial solution. For example, the initial solution can be optimized and corrected using expert priors, and the corresponding components in the air conditioner can be controlled using the optimized and corrected control parameters. For instance, the expert priors may include the optimal ratio of different control parameters. The control parameters obtained from the initial solution are adjusted to conform to the optimal ratio to optimize the initial solution. Then, the optimized and adjusted initial solution is used as the current control parameter to control the corresponding components of the air conditioner. In other words, expert experience is used to perform a post-mapping on the initial solution so that the control parameters satisfy the expert priors.

[0134] The initial environmental proxy model can be obtained by collecting training datasets and modeling environmental changes in a safe exploration manner. For example, in order to ensure the diversity of the collected interaction data and to prevent the air conditioner from experiencing performance degradation (such as temperature control failure) during the acquisition of interaction data, this application embodiment adopts a safe exploration method to add an appropriate perturbation to the air conditioner baseline control strategy when the air conditioner is turned on for the first time. This enables accurate local modeling of the environment with only a small number of interactions with the environment, and the interaction process does not result in significant degradation compared to the baseline control strategy.

[0135] The specific implementation of secure exploration can be found in the description above, and will not be repeated here for the sake of brevity.

[0136] Based on the same concept as the aforementioned embodiment of a device control method, this application also provides a device control apparatus 400, which can be deployed in a terminal device, such as an air conditioner, to improve the temperature control performance and energy-saving effect of the air conditioner. The device control apparatus 400 includes components for implementing... Figure 2 and 3 The units or modules of each step in the device control method shown.

[0137] Figure 4 This is a schematic diagram of the structure of a device control apparatus provided in an embodiment of this application. Figure 4As shown, the device control unit 400 includes a first acquisition module 401, a reliability judgment module 402, a determination module 403, a control module 404, a second acquisition module 405, an update module 406, and an optimization module 407. The first acquisition module 401 acquires the current environmental state of the device (hereinafter referred to as the current environmental state); the reliability judgment module 402 determines that the reliability of the environmental proxy model is substandard, and the environmental proxy model is used to predict the environmental state at future times based on the current environmental state; the determination module 403 determines the target control parameters of the device based on the current environmental state and the default control strategy; the control module 404 uses... The system controls the device based on target control parameters; the second acquisition module 405 is used to acquire the interaction data between the default control strategy and the environment, the interaction data includes the current environment state, the control parameters issued by the default control strategy based on the current environment state, and the environment state at the next moment; the update module 406 is used to update the training dataset based on the interaction data, the training data includes multiple training sample pairs, each training sample pair includes the environment state at time t, the control parameters at time t, and the environment state at time t+1; the optimization module 407 is used to optimize and adjust the weight parameters of the environment proxy model based on the updated training dataset to obtain an environment proxy model with acceptable reliability.

[0138] In one possible implementation, the update module 406 is specifically used for: obtaining new training sample pairs based on the interaction data; adding the new training sample pairs to the training dataset; determining a target sliding window from a set of candidate sliding windows, the set of candidate sliding windows including multiple sliding windows of different lengths; selecting the N nearest training samples from the training dataset based on the target sliding window; and obtaining an updated training dataset based on the N training samples.

[0139] In another possible implementation, a specific implementation of determining the target sliding window from the candidate sliding window set is as follows: the target sliding window is determined based on the probability of each sliding window in the candidate sliding window set, the probability of each sliding window is determined based on the feedback information corresponding to each sliding window, and the feedback information corresponding to each sliding window indicates the prediction error of the trained environment agent model based on the training samples selected from the training dataset based on each sliding window.

[0140] In another possible implementation, the update module 406 is also used to periodically update the probability of each sliding window in the candidate sliding window set based on the AMAB algorithm.

[0141] In another possible implementation, the determining module 403 is specifically used to: determine a first control parameter based on the current operating environment state of the device and the default control strategy; and perturb the first control parameter within a preset range to obtain the target control parameter.

[0142] In another possible implementation, the determination module 403 is also used to determine whether the reliability of the environmental proxy model meets the standard; to perform optimization solution based on the current operating environment of the device and the environmental proxy model to obtain the second control parameter; and to determine the target control parameter based on the second control parameter.

[0143] In another possible implementation, a specific way to determine the target control parameter based on the second control parameter is as follows: the second control parameter is optimized and corrected based on prior information to obtain the target control parameter, wherein the prior information includes the optimal proportion of control parameters that are correlated in the control parameter set.

[0144] In another possible implementation, the initial environment agent model is trained based on an initial training dataset, which is obtained from the interaction data between the device and the environment upon initial power-on.

[0145] In another possible implementation, the interaction data between the device and the environment upon initial power-on includes: the current environmental state, the third control parameter, and the environmental state at the next moment; the third control parameter is obtained by perturbing the control parameters issued by the default control strategy based on the current environmental state with a preset amplitude.

[0146] In another possible implementation, the reliability of the environment proxy model is determined based on the accuracy and / or prior correlation of the environment proxy model.

[0147] In another possible implementation, the device is an air conditioner; the environmental conditions include at least one of the following: indoor temperature, actual value of indoor fan speed, indoor coil temperature, actual value of compressor frequency, compressor discharge temperature, actual value of expansion valve opening, cumulative power consumption, actual value of outdoor fan speed, and outdoor temperature; the target control parameters include at least one of the following: target value of indoor fan speed, target value of compressor frequency, target value of expansion valve opening, and target value of outdoor fan speed.

[0148] In another possible implementation, the environment agent model is a lightweight linear model or a decision tree model.

[0149] The device control apparatus 400 according to the embodiments of this application can correspond to executing the methods described in the embodiments of this application, and the above and other operations and / or functions of each module in the device control apparatus 400 are respectively for implementing Figures 2-3 For the sake of brevity, the corresponding processes of each method in the code will not be elaborated here.

[0150] Figure 5 A schematic diagram of the structure of a computing device provided in an embodiment of this application.

[0151] like Figure 5As shown, the computing device 500 includes at least one processor 501, a memory 502, and a communication interface 503. The processor 501, memory 502, and communication interface 503 are communicatively connected, which can be achieved via a wired (e.g., bus) or wireless connection. The communication interface 503 is used to send and / or receive data from other devices. The memory 502 stores computer instructions, which the processor 501 executes to perform the methods described in the aforementioned method embodiments, thereby improving the device's control performance and energy efficiency.

[0152] It should be understood that in the embodiments of this application, the processor 501 may be a central processing unit (CPU), or it may be other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. A general-purpose processor may be a microprocessor or any conventional processor.

[0153] The memory 502 may include read-only memory and random access memory, and provides instructions and data to the processor 501. The memory 502 may also include non-volatile random access memory. Optionally, the random access memory may be, for example, high bandwidth memory (HBM).

[0154] The memory 502 can be volatile memory or non-volatile memory, or it can include both. The non-volatile memory can be read-only memory (ROM), programmable read-only memory (PROM), erasable programmable read-only memory (EPROM), electrically erasable programmable read-only memory (EEPROM), or flash memory. The volatile memory can be random access memory (RAM), which is used as an external cache. By way of example, but not limitation, many forms of RAM are available, such as static random access memory (SRAM), dynamic random access memory (DRAM), synchronous dynamic random access memory (SDRAM), double data rate synchronous dynamic random access memory (DDR SDRAM), enhanced synchronous dynamic random access memory (ESDRAM), synchronous linked dynamic random access memory (SLDRAM), and direct rambus RAM (DR RAM).

[0155] It should be understood that the computing device 500 according to the embodiments of this application can execute the implementation of the embodiments of this application. Figures 2-3 The method shown is described in detail above, and will not be repeated here for the sake of brevity.

[0156] Embodiments of this application provide a computer-readable storage medium having a computer program stored thereon, wherein when the computer instructions are executed by a processor, the aforementioned method is implemented.

[0157] An embodiment of this application provides a chip including at least one processor and an interface, wherein the at least one processor determines program instructions or data through the interface; the at least one processor is used to execute the program instructions to implement the method mentioned above.

[0158] Embodiments of this application provide a computer program or computer program product that includes instructions that, when executed, cause a computer to perform the methods mentioned above.

[0159] Those skilled in the art will further recognize that the units and algorithm steps of the various examples described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, computer software, or a combination of both. To clearly illustrate the interchangeability of hardware and software, the components and steps of the various examples have been generally described in terms of functionality in the foregoing description. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this application.

[0160] The steps of the methods or algorithms described in conjunction with the embodiments disclosed herein can be implemented using hardware, a software module executed by a processor, or a combination of both. The software module can be located in random access memory (RAM), main memory, read-only memory (ROM), electrically programmable ROM, electrically erasable programmable ROM, registers, hard disk, removable disk, CD-ROM, or any other form of storage medium known in the art.

[0161] The specific embodiments described above further illustrate the purpose, technical solution, and beneficial effects of this application. It should be understood that the above description is only a specific embodiment of this application and is not intended to limit the scope of protection of this application. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of this application should be included within the scope of protection of this application.

Claims

1. A device control method, characterized in that, include: Obtain the current operating environment status of the device; The reliability of the environmental proxy model is determined to be substandard. The environmental proxy model is used to predict the environmental state at future times based on the current environmental state and control parameters. Based on the current operating environment and default control strategy of the device, determine the target control parameters of the device; Control the device based on the target control parameters; The interaction data between the default control strategy and the environment is obtained. The interaction data includes the current environmental state of the device, the control parameters issued by the default control strategy based on the current environmental state of the device, and the environmental state at the next moment. The training dataset is updated based on the interactive data. The training dataset includes multiple training sample pairs, and each training sample pair includes the environmental state at time t, the control parameters at time t, and the environmental state at time t+1. The weight parameters of the environment proxy model are optimized and adjusted based on the updated training dataset to obtain an environment proxy model that meets the reliability standards.

2. The method according to claim 1, characterized in that, The step of updating the training dataset based on the interaction data includes: Based on the interaction data, new training sample pairs are obtained; The new training sample pairs are added to the training dataset; The target sliding window is determined from a set of candidate sliding windows, the set of candidate sliding windows including multiple sliding windows of different lengths; Based on the target sliding window, select the N nearest training samples from the training dataset; Based on the N training samples, the updated training dataset is obtained.

3. The method according to claim 2, characterized in that, The step of determining the target sliding window from the candidate sliding window set includes: The target sliding window is determined based on the probability of each sliding window in the candidate sliding window set. The probability of each sliding window is determined based on the feedback information corresponding to each sliding window. The feedback information corresponding to each sliding window indicates the prediction error of the trained environment agent model based on the training samples selected from the training dataset based on each sliding window.

4. The method according to claim 3, characterized in that, Also includes: The probability of each sliding window in the candidate sliding window set is periodically updated based on the AMAB algorithm.

5. The method according to claim 1, characterized in that, The process of determining the target control parameters of the device based on its current operating environment and default control policy includes: The first control parameter is determined based on the current operating environment of the device and the default control strategy. The first control parameter is perturbed within a preset range to obtain the target control parameter.

6. The method according to claim 1, characterized in that, Also includes: The reliability of the environmental proxy model was confirmed to be up to standard. The second control parameter is obtained by optimizing the solution based on the current operating environment of the device and the environment proxy model. The target control parameters are determined based on the second control parameters.

7. The method according to claim 6, characterized in that, Determining the target control parameter based on the second control parameter includes: The second control parameter is optimized and corrected based on prior information to obtain the target control parameter. The prior information includes the optimal proportion of control parameters that are correlated in the control parameter set.

8. The method according to claim 1, characterized in that, The initial environment agent model is trained based on an initial training dataset, which is obtained based on the interaction data between the device and the environment upon initial power-on.

9. The method according to claim 8, characterized in that, The interaction data between the device and the environment during the initial power-on includes: the current environmental state of the device, the third control parameter, and the environmental state at the next moment; The third control parameter is obtained by perturbing the control parameters issued by the default control strategy based on the current operating environment of the device with a preset amplitude.

10. The method according to claim 1, characterized in that, The reliability of the environmental proxy model is determined based on the accuracy and / or prior correlation of the environmental proxy model.

11. The method according to claim 1, characterized in that, The device is an air conditioner; The environmental conditions include at least one of indoor temperature, indoor coil temperature, compressor exhaust temperature, and outdoor temperature; or, the environmental conditions include at least one of indoor temperature, indoor coil temperature, compressor exhaust temperature, and outdoor temperature, and at least one of the following: actual value of indoor fan speed, actual value of compressor frequency, actual value of expansion valve opening, cumulative power consumption, and actual value of outdoor fan speed. The target control parameters include at least one of the following: target value for internal fan speed, target value for compressor frequency, target value for expansion valve opening, and target value for external fan speed.

12. The method according to any one of claims 1-11, characterized in that, The environmental proxy model is a lightweight linear model or a decision tree model.

13. A device control apparatus, characterized in that, include: The first acquisition module is used to acquire the current operating environment status of the device; The reliability judgment module is used to determine that the reliability of the environmental proxy model is not up to standard. The environmental proxy model is used to predict the environmental state at future times based on the environmental state at the current time. The determination module is used to determine the target control parameters of the device based on the current operating environment and default control strategy of the device. The control module is used to control the device based on the target control parameters; The second acquisition module is used to acquire the interaction data between the default control strategy and the environment. The interaction data includes the current working environment state of the device, the control parameters issued by the default control strategy based on the current working environment state of the device, and the environment state at the next moment. The update module is used to update the training dataset based on the interaction data. The training dataset includes multiple training sample pairs, and each training sample pair includes the environmental state at time t, the control parameters at time t, and the environmental state at time t+1. The optimization module is used to optimize and adjust the weight parameters of the environment proxy model based on the updated training dataset to obtain the environment proxy model with acceptable reliability.

14. A computing device, comprising a memory and a processor, characterized in that, The memory stores instructions that, when executed by a processor, cause the method described in any one of claims 1-12 to be implemented.

15. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by a processor, it causes the method as described in any one of claims 1-12 to be implemented.

Images