Component control method, device, apparatus, storage medium and chip

Abstract

The present disclosure relates to a component control method, device, equipment, storage medium and chip, wherein the method comprises: obtaining a sensor parameter of a component to be controlled; inputting the sensor parameter of the component to be controlled into a reinforcement learning model to output an action control parameter of the component to be controlled; wherein the reinforcement learning model is obtained by training based on an optimization function corresponding to each of m time slots, a preset constraint condition and a reward function, the optimization function is used to reflect the total power consumption of n components in the electronic device in the corresponding time slot, the preset constraint condition is determined based on the running parameters of the n components, and the reward function is related to the optimization function, and m and n are positive integers. In this way, the technical problems of related technologies, such as the difficulty in learning the control of internal components, the refinement and accuracy of influence control, etc. can be solved.

Description

Technical Field

[0001] This disclosure relates to the field of terminal technology, and in particular to a component control method, apparatus, device, storage medium, and chip. Background Technology

[0002] With the continuous advancements in terminal technology, IoT technology, and reinforcement learning technology, more and more reinforcement learning techniques are being introduced into the field of terminal control. Taking air conditioning as an example, introducing reinforcement learning technology into air conditioning control can not only achieve intelligent control of the air conditioner but also improve its comfort and energy efficiency to a certain extent. With the development of multi-intelligent reinforcement learning, achieving intelligent and precise control of air conditioning components, while ensuring comfort, to achieve more accurate temperature control and greater energy efficiency has become a research hotspot in recent years.

[0003] However, most existing technologies treat air conditioners as a whole for temperature control, making it difficult to learn the control of internal components, thus affecting the precision and accuracy of control. Therefore, there is an urgent need to propose a better component control scheme. Summary of the Invention

[0004] To overcome the problems existing in the related technologies, this disclosure provides a component control method, device, equipment, storage medium, and chip to solve the technical problems in the above-mentioned related technologies, such as the difficulty in learning the control of internal components and the impact on the fineness and accuracy of control.

[0005] According to a first aspect of the present disclosure, a component control method is provided, comprising: Obtain the sensor parameters of the component to be controlled; The sensor parameters of the component to be controlled are input into the reinforcement learning model, and the action control parameters of the component to be controlled are output. The reinforcement learning model is obtained by training based on the optimization function, preset constraints and reward function corresponding to each of the m time slots. The optimization function is used to reflect the total power consumption of n components in the electronic device in the corresponding time slot. The preset constraints are determined based on the operating parameters of each of the n components. The reward function is related to the optimization function. Both m and n are positive integers.

[0006] In some embodiments, the method further includes: A target function is determined, which is used to indicate the sum of the energy consumption gains of each of the n components within a preset time period, wherein the preset time period is the time period corresponding to the m time slots; The objective function is transformed into a single time slot to obtain the optimization functions corresponding to each of the m time slots.

[0007] In some embodiments, determining the objective function includes: Determine the energy consumption gain corresponding to each of the n components in the m time slots. The energy consumption gain is used to indicate the difference between the power consumption of the corresponding component when it operates at a preset frequency and the power consumption when it operates at the current operating frequency in the time slot. The energy consumption gain for the preset time period is calculated based on the energy consumption gain of each of the n components within the m time slots, thus obtaining the objective function.

[0008] In some embodiments, performing single-slot transformation on the objective function to obtain optimization functions corresponding to each of the m slots includes: Based on the total power consumption of each of the n components in the m time slots and the average power consumption of the n components in the preset time period, a corresponding virtual power consumption queue is determined. The virtual power consumption queue is used to reflect the deviation between the total power consumption of the n components in the corresponding time slot and the average power consumption. Based on the corresponding virtual power consumption queue, the sum of the energy consumption gains of each of the n components within the m time slots is transformed to obtain the optimization function corresponding to each of the m time slots.

[0009] In some embodiments, the preset constraint includes at least one of the following: The operating parameters of each of the n components are within their respective preset parameter ranges; The total power consumption of each of the n components within the m time slots is less than or equal to the corresponding preset power consumption threshold. The average power consumption of the n components during the preset time period is less than or equal to the corresponding preset power threshold.

[0010] In some embodiments, the reward function includes individual reward functions for each of the n components and a collective reward function for the n components. The individual reward functions are determined based at least on the motion control parameters of the corresponding components, and the collective reward function is determined based at least on the optimization functions corresponding to each of the m time slots.

[0011] In some embodiments, the reward function further includes a preset penalty term; wherein, when the corresponding preset constraint condition is met, the reward function includes individual reward functions for each of the n components and a collective reward function for the n components; or, when the corresponding preset constraint condition is not met, the reward function includes the preset penalty term.

[0012] In some embodiments, the method further includes: The control operation of the component to be controlled is performed based on the motion control parameters of the component to be controlled.

[0013] According to a second aspect of the present disclosure, a component control device is provided, comprising: The acquisition module is configured to acquire sensor parameters of the component to be controlled; The processing module is configured to input the sensor parameters of the component to be controlled into the reinforcement learning model and output the motion control parameters of the component to be controlled. The reinforcement learning model is obtained by training based on the optimization function, preset constraints and reward function corresponding to each of the m time slots. The optimization function is used to reflect the total power consumption of n components in the electronic device in the corresponding time slot. The preset constraints are determined based on the operating parameters of each of the n components. The reward function is related to the optimization function. Both m and n are positive integers.

[0014] For any content not introduced or described in the embodiments of this disclosure, please refer to the relevant descriptions in the foregoing method embodiments. This disclosure does not limit the scope of the embodiments.

[0015] According to a third aspect of the present disclosure, a home appliance is provided, comprising: a processor; a memory for storing processor-executable instructions; wherein the processor is configured to execute the executable instructions to implement the steps of the component control method described above.

[0016] According to a fourth aspect of the present disclosure, a computer-readable storage medium is provided that stores computer program instructions thereon, which, when executed by a processor, implement the steps of the component control method provided in the first aspect of the present disclosure.

[0017] According to a fifth aspect of the present disclosure, a chip is provided, comprising: a processor and an interface; the processor is configured to read instructions to execute the steps of the component control method described above.

[0018] The technical solution provided by the embodiments of this disclosure can include the following beneficial effects: Home appliances acquire sensor parameters of a component to be controlled; the sensor parameters of the component to be controlled are input into a reinforcement learning model, and the action control parameters of the component to be controlled are output; wherein, the reinforcement learning model is obtained by training based on optimization functions, preset constraints, and reward functions corresponding to m time slots, the optimization functions reflect the total power consumption of n components in the electronic device within the corresponding time slots, the preset constraints are determined based on the operating parameters of the n components, the reward function is related to the optimization function, and m and n are both positive integers. In this way, home appliances can directly use the reinforcement learning model to calculate and output the action control parameters of the component to be controlled, facilitating efficient and accurate subsequent control of the component. This improves the convenience and accuracy / precision of component control. It also solves the technical problems in related technologies such as the difficulty in learning the control of internal components, which affects the refinement and precision of control.

[0019] It should be understood that the above general description and the following detailed description are exemplary and explanatory only, and are not intended to limit this disclosure. Attached Figure Description

[0020] The accompanying drawings, which are incorporated in and form a part of this specification, illustrate embodiments consistent with this disclosure and, together with the description, serve to explain the principles of this disclosure.

[0021] Figure 1 This is a schematic diagram of a system structure according to an exemplary embodiment.

[0022] Figure 2 This is a flowchart illustrating a component control method according to an exemplary embodiment.

[0023] Figure 3 This is a schematic diagram of the internal structure of a reinforcement learning model according to an exemplary embodiment.

[0024] Figure 4 This is a schematic diagram of the internal structure of an action network according to an exemplary embodiment.

[0025] Figure 5 This is a flowchart illustrating a model training method according to an exemplary embodiment.

[0026] Figure 6 This is a schematic diagram illustrating a process for determining an objective function according to an exemplary embodiment.

[0027] Figure 7 This is a schematic diagram illustrating a time slot conversion process according to an exemplary embodiment.

[0028] Figure 8 This is a schematic diagram of the structure of a component control device according to an exemplary embodiment.

[0029] Figure 9 This is a schematic diagram of the structure of a household appliance according to an exemplary embodiment.

[0030] Figure 10 This is a schematic diagram of the structure of a chip according to an exemplary embodiment. Detailed Implementation

[0031] Exemplary embodiments will now be described in detail, examples of which are illustrated in the accompanying drawings. When the following description relates to the drawings, unless otherwise indicated, the same numerals in different drawings denote the same or similar elements. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with this disclosure. Rather, they are merely examples of apparatuses and methods consistent with some aspects of this disclosure as detailed in the appended claims.

[0032] It should be noted that all actions involving the acquisition of signals, information, or data in this disclosure are performed with the authorization of the respective device owner.

[0033] Taking air conditioners as an example, current research on energy efficiency and comfort in air conditioning suffers from the following problems: While energy efficiency and comfort are the overall goals, there is a lack of research on the refined control of internal components. Furthermore, existing learning algorithms treat the air conditioner as a whole, making it difficult to learn the impact of environmental changes on the overall system, thus affecting the accuracy of control results. Existing learning algorithms incorporate historical data for prediction to improve control accuracy; however, such algorithms require a large amount of historical data to ensure accuracy, which is often difficult to implement in practice, significantly increasing the difficulty of algorithm implementation. There is also limited research on limiting overall power consumption within preset service periods.

[0034] To address the aforementioned problems, this disclosure provides a component control method, apparatus, device, storage medium, and chip. Some embodiments applicable to this disclosure are described below.

[0035] Please see Figure 1 This is a schematic diagram of a system structure according to an exemplary embodiment. For example... Figure 1 The system shown may include a home appliance 100 and a device controller 200. Optionally, it may also include a server 300. Wherein: The aforementioned household appliance 100 can refer to household appliances, such as, but not limited to, air conditioners, refrigerators, washing machines, dryers, dry cleaning machines, or other household appliances. For ease of description, this disclosure uses an air conditioner as an example for the following description and introduction of relevant content. This can be adjusted according to actual circumstances, and this disclosure does not constitute a limitation. The aforementioned household appliance 100 can include various components that make up the household appliance. For example, taking an air conditioner as an example, the air conditioner can include, but is not limited to, an indoor fan, an outdoor fan, a compressor, an expansion valve, a baffle plate, a sensor (also called a sensor), a microcontroller unit (MCU), or other controllable components, etc.

[0036] The aforementioned device controller 200 may include a data transceiver module 201, a data processing module 202, and an algorithm module 203. The data transceiver module 201 is primarily used for receiving and sending data, such as receiving user-input commands, like setting the air conditioner to 26°C.

[0037] The aforementioned data processing module 202 is mainly used to process the data received by the data transceiver module 201, such as format conversion, instruction translation, or custom preprocessing. For example, the data processing module 202 can convert the data received by the data transceiver module 201 into a format that the system can recognize, such as converting it into a format that the subsequent algorithm module 203 can recognize or process. This disclosure does not impose further limitations or details on this.

[0038] The aforementioned algorithm module 203 is mainly used to perform corresponding algorithm calculations on the data processed by the aforementioned data processing module 202 to obtain the corresponding processing results. This disclosure does not limit the specific algorithms involved in the aforementioned algorithm module 203; they can be pre-defined by the system or user according to actual functional requirements. For example, the aforementioned algorithm module 203 can use reinforcement learning algorithms to predict component actions on the data processed by the aforementioned data processing module 202, thereby obtaining the component's action control parameters, such as the target operating frequency of the compressor, the target opening degree of the expansion valve, etc. This disclosure does not impose further limitations or details on these parameters.

[0039] The aforementioned server 300 can refer to a local server, a cloud server, or a server with other deployment methods. The aforementioned server 300 can communicate with the aforementioned home appliance 100 and the aforementioned device controller 200 via a network. For example, the home appliance 100 can upload some of its historical data to the server 300 for backup storage, etc. This disclosure will not make further limitations or details in this regard.

[0040] Taking the aforementioned household appliance 100 as an air conditioner as an example, the following examples illustrate the definitions / representations of some parameter symbols involved in this disclosure. Specifically, in an air conditioning control scenario, the system may include, for example, an air conditioner controller M, an air conditioner N, and some internal components, such as the air conditioner's indoor fan. , outdoor fan ,compressor Expansion valve Windshield And various sensors, etc. That is, Once a user sets their desired temperature, the air conditioner needs to coordinate with its various components to reach the set temperature while ensuring energy efficiency and comfort, and then maintain that temperature around the desired level. In this scenario, the air conditioner's microcontroller unit (MCU) can receive air conditioner control commands, which instruct the control parameters of the components to be controlled. The MCU then responds to these commands by controlling the operation of the corresponding components, such as controlling the opening of the expansion valve p to 10°.

[0041] This disclosure can convert the preset time period T of temperature control into a series of time slots t, that is .in, This represents the i-th time slot, where i and n are both user-defined positive integers. For example, assuming the preset time period T corresponding to temperature control is 1 hour, the system can be set to be divided into 4 time slots. In this disclosure, T can be divided into 4 equal time slots, each time slot being 15 minutes.

[0042] This disclosure adopts Indicates indoor temperature. Indicates the outdoor temperature. Indicates indoor humidity. This indicates the inlet temperature of the evaporator. This indicates the outlet temperature of the evaporator. This indicates the inlet temperature of the condenser. This indicates the outlet temperature of the condenser. This indicates the compressor's exhaust temperature. This indicates the inlet pressure of the compressor. This indicates the compressor's outlet pressure. This indicates the operating frequency of the compressor. This indicates the operating current of the compressor. This indicates the operating speed of the internal fan. This indicates the operating speed of the external fan. This indicates the operating current of the internal fan. This indicates the operating current of the external fan. This indicates the operating current of the expansion valve. This indicates the opening degree of the expansion valve. This indicates the operating voltage of the compressor. This indicates the operating voltage of the internal fan. This indicates the operating voltage of the external fan. This indicates the operating voltage of the expansion valve. This indicates the operating voltage of the wind deflector.

[0043] Based on the above embodiments, please refer to Figure 2 This is a flowchart illustrating a component control method according to an exemplary embodiment. Figure 2 The method shown can be applied to the aforementioned household appliance 100, such as... Figure 2 The method shown may include the following implementation steps: S201. Obtain the sensor parameters of the component to be controlled.

[0044] In this disclosure, the aforementioned components to be controlled can refer to components in household appliances that need to be controlled. Taking an air conditioner as an example, the aforementioned components to be controlled may include, but are not limited to, indoor fans, outdoor fans, compressors, expansion valves, baffles, or other components that need to be controlled. This disclosure does not impose further limitations or provide detailed descriptions in this regard.

[0045] This disclosure does not limit the implementation method of obtaining the above-mentioned sensor parameters. For example, they can be directly acquired by the built-in sensor of the component to be controlled, or they can be obtained from other devices (such as other terminals or servers) through the network. This disclosure will not limit or elaborate on these aspects.

[0046] The sensor parameters of the controlled component mentioned above can refer to the observation values ​​of the controlled component on the environment in the current time slot. These observation values ​​can be obtained by the corresponding sensor or obtained from other devices through the network, etc. This disclosure does not impose any restrictions on this.

[0047] S202. Input the sensor parameters of the component to be controlled into the reinforcement learning model, and output the action control parameters of the component to be controlled; wherein, the reinforcement learning model is obtained by training based on the optimization function, preset constraints and overall reward function corresponding to each of the m time slots, the optimization function is used to reflect the total power consumption of the n components in the electronic device in the corresponding time slot, the preset constraints are determined based on the operating parameters of the n components, the overall reward function is related to the optimization function, and m and n are both positive integers.

[0048] After obtaining the sensor parameters of the aforementioned component to be controlled, this disclosure allows the sensor parameters of the component to be controlled to be input into a trained reinforcement learning model for inference calculation, thereby outputting the motion control parameters of the component to be controlled. This disclosure does not limit the number of components to be controlled; they can be customized according to actual conditions. For ease of description, this disclosure uses one component as an example to illustrate the relevant content, but this does not constitute a limitation. The reinforcement learning model will be described in detail below and will not be described here.

[0049] By implementing the embodiments of this disclosure, a home appliance acquires sensor parameters of a component to be controlled; the sensor parameters of the component to be controlled are input into a reinforcement learning model, and the action control parameters of the component to be controlled are output. The reinforcement learning model is trained based on optimization functions, preset constraints, and reward functions corresponding to m time slots. The optimization functions reflect the total power consumption of n components in the electronic device within the corresponding time slots. The preset constraints are determined based on the operating parameters of the n components. The reward function is related to the optimization function, and m and n are both positive integers. In this way, the home appliance can directly use the reinforcement learning model to calculate and output the action control parameters of the component to be controlled, facilitating efficient and accurate subsequent control of the component. This improves the convenience and accuracy of component control. It also solves the technical problems in related technologies, such as the difficulty in learning the control of internal components, which affects the refinement and accuracy of control.

[0050] In step S201 above, in practical applications, each component of the home appliance can observe its associated sensor parameters through built-in sensors, that is, observe the environmental observation values ​​of each component in the corresponding time slot (e.g., time slot t). Taking time slot t as an example, a brief introduction is given below.

[0051] The compressor's sensor parameters can include the compressor's (y) observations of the environment during time slot t, which can be specifically expressed as: .in, This represents the indoor temperature corresponding to time slot t. This represents the outdoor temperature corresponding to time slot t. This represents the indoor humidity corresponding to time slot t. This represents the exhaust temperature of compressor y in time slot t. This represents the inlet pressure of compressor y in time slot t. This represents the outlet pressure of compressor y in time slot t. This represents the operating frequency of compressor y in time slot t. This represents the operating current / current of compressor y in time slot t. This represents the operating voltage / current voltage of compressor y in time slot t.

[0052] The sensor parameters of the air conditioner fan can include the observations of the environment by the air conditioner fan f in time slot t, which can be specifically expressed as: The aforementioned air conditioning fan f may include an indoor fan f. in and external fan f out .in, This indicates the rotational speed of the air conditioner's internal fan in time slot t. This indicates the rotational speed of the air conditioner's outdoor fan in time slot t. This represents the operating current of the air conditioner's indoor fan during time slot t. This represents the operating current of the air conditioner's outdoor fan in time slot t. This indicates the operating voltage of the air conditioner's indoor fan in time slot t. This indicates the operating voltage of the air conditioner's outdoor fan in time slot t.

[0053] The sensor parameters of the expansion valve can include the observations of the expansion valve p on the environment during time slot t, specifically expressed as follows: .in, This represents the operating current of the expansion valve p in time slot t. This indicates the opening degree of the expansion valve p in time slot t (also simply called the opening degree). This indicates the inlet temperature of the evaporator in time slot t. This indicates the outlet temperature of the evaporator in time slot t. This indicates the inlet temperature of the condenser in time slot t. This indicates the outlet temperature of the condenser in time slot t. This represents the operating voltage of the expansion valve p in time slot t.

[0054] The sensor parameters of the wind deflector can include the observations of the environment by the wind deflector w in time slot t, which can be specifically expressed as: .in, This indicates the current angle / operating angle of the wind deflector w in time slot t. This indicates the current / operating position of the wind deflector w in time slot t. This represents the operating voltage of the wind deflector w in time slot t.

[0055] In an optional embodiment, the sensor parameters of the above-mentioned components can be uniformly defined as a set of sensors or a set of state spaces S, which can be expressed as shown in the following formula (1): (1) In step S202, this disclosure can employ a reinforcement learning model to solve for actions and optimization problems in a continuous action space. In practical applications, this disclosure can use a centralized training and distributed execution approach for model training and use. During the centralized training phase, the device controller can assist the agent in learning using its available information (e.g., sensor parameters, action space, etc.), helping the agent update the corresponding model parameters. The aforementioned reinforcement learning model can acquire learning information and update model parameters by receiving rewards / feedback from the environment for actions without requiring any pre-provided data. The reinforcement learning model can include, but is not limited to, models such as Multi-Agent Deep Deterministic Policy Gradient (MADDPG), Q-Learning, State-Action-Reward-State-Action (SARSA), Deep Deterministic Policy Gradient (DDPG), Advantage Actor-Critic (A2C), Proximal Policy Optimization (PPO), or other custom reinforcement learning models.

[0056] This disclosure does not limit the internal structure of the above reinforcement learning model; for example, please refer to [link to relevant documentation]. Figure 3 This is a schematic diagram illustrating the internal structure of a reinforcement learning model according to an exemplary embodiment. For example... Figure 3 The model shown may include at least one agent module. The illustration uses n agent modules as an example, but this is not a limitation; n is a positive integer that can be customized according to actual conditions. The input to each agent module is the sensor parameters of each component, i.e., the component's observation of the environment, o. The output of each agent module is the action control parameters of each component, i.e., the parameters a that control the component's actions, which will be described in detail below and will not be discussed here. Each agent module may include an action network (Actor), a critique network (Critic), and a loss unit (Loss). This disclosure does not limit the number of action networks and critique networks; the illustrations use two action networks and two critique networks as examples, which can be adjusted / determined according to actual conditions and are not a limitation. Wherein: The aforementioned action network is used to perform inference calculations on input information (e.g., sensor parameters of the component to be controlled). The aforementioned critique network is used to assist in the model training of the aforementioned action network, such as adjusting / updating the parameters of the aforementioned action network. The aforementioned loss unit is used to calculate the loss value r of the aforementioned action network and / or the aforementioned critique network, for example, by calculating the difference between the actual action parameters corresponding to the sensor parameters input to the aforementioned action network and the predicted action parameters output by the aforementioned action network inferring from the input sensor parameters, thereby obtaining the loss value r of the aforementioned action network. This disclosure does not impose many limitations on the internal structure of the aforementioned action network, the aforementioned critique network, and the aforementioned loss unit; for example, please refer to [link to relevant documentation]. Figure 4 This is a schematic diagram illustrating the internal structure of an action network according to an exemplary embodiment. For example... Figure 4 The action network shown may include a Multilayer Perceptron (MLP) and a Gated Recurrent Unit (GRU). This disclosure does not limit the number of MLPs and GRUs; the illustrations show examples with two MLPs and one GRU, but this is not a limitation. The MLP is a feedforward neural network that uses multi-layer linear transformations and non-linear activation functions to achieve feature extraction and knowledge acquisition. The GRU primarily learns long-term dependencies in the environment by inputting the hidden (feature) information from the previous time step and outputs the hidden (feature) information for the next time step.

[0057] In reinforcement learning models, the action network (Actor) outputs the final action control parameters, so the vectors from the last layer need to be mapped to the actual action space. Since the action control parameters of different components are inconsistent, the action control parameters output by the action network (Actor) need to be mapped and scaled differently to obtain the actual action control parameters.

[0058] This disclosure defines the motion space of each component as specific control parameter values ​​for different stages within a continuous time period, thereby enabling more refined control of each component. For example, this disclosure can provide five specific motion control parameters adjusted in chronological order within any time period, such as time period t0, thereby completing parameter control for the corresponding stage. The motion space can be specifically represented by the following formula (2): (2) in, This represents five action control parameters (specifically, the compressor's operating frequency) that change continuously in chronological order during time period t0. In other words, the compressor continuously adjusts and controls itself according to these five action control parameters during time t0 to achieve the value of the last action control parameter. This represents five control parameters (specifically, the operating speed of the internal fan) that change continuously in time sequence during time period t0. This represents five action control parameters (specifically, the operating speed of the external fan) that change continuously in time sequence during time period t0. This represents five action control parameters (specifically, the opening degree of the expansion valve) that change continuously in time sequence during time period t0. This represents five motion control parameters (specifically, the position of the wind deflector) that change continuously in time sequence during time period t0.

[0059] The reinforcement learning model described above can use experience pool replay and gradient descent to update the model parameters. The experience pool replay technique refers to training with training data from the same time slot. The gradient descent technique refers to updating the model parameters of the reinforcement learning model based on the gradient value of the objective function / optimization function under the current sensor parameters. The experience pool replay technique and the gradient descent technique can be referred to in the relevant introduction and description of traditional related techniques such as traditional gradient descent. This disclosure will not make any further limitations or details in this regard.

[0060] Based on this, some optional embodiments involved in this disclosure are described below.

[0061] In an optional embodiment, after obtaining the motion control parameters of the component to be controlled, this disclosure can control the component to be controlled based on / according to the motion control parameters. For example, if the motion control parameters of the component to be controlled include the operating frequency of the compressor, this disclosure can directly control the operation of the compressor according to the operating frequency of the compressor, thereby achieving the user-set required temperature, etc. This disclosure does not impose further limitations or details in this regard.

[0062] In another optional embodiment, before performing step S202, this disclosure further requires training the reinforcement learning model. Some training embodiments related to the reinforcement learning model are described below. Please refer to... Figure 5 This is a flowchart illustrating a model training method according to an exemplary embodiment. Figure 5 The method shown can be applied to training devices and may include the following implementation steps: S501. Determine the objective function, which is used to indicate the sum of the energy consumption gains of each of the n components within a preset time period T, where the preset time period T is the time period corresponding to the m time slots.

[0063] In this disclosure, the objective function can refer to a function that designs and finds the optimal motion control parameters for each component to achieve temperature control under the constraints of energy saving and ensuring comfort. Typically, the objective function can be described as the sum of energy consumption gains of n components in a home appliance within a preset time period T. The preset time period is the time period corresponding to the m time slots mentioned above. This disclosure does not limit the implementation method for determining the objective function; for example, please refer to... Figure 6 This is a schematic diagram illustrating a process for determining an objective function according to an exemplary embodiment. For example... Figure 6 The process shown may include the following implementation steps: S601. Determine the energy consumption gain corresponding to each of the n components in the m time slots. The energy consumption gain is used to indicate the energy consumption difference between the operating parameters of the corresponding component at a preset frequency and the operating parameters at the current operating frequency within the time slot.

[0064] In this disclosure, the aforementioned energy consumption gain can refer to / be defined as the difference between the power consumption of a corresponding component operating at a preset frequency and the power consumption operating at the current frequency within a corresponding time slot (e.g., time slot t). The preset frequency can be a frequency pre-defined by the system or the user, such as the maximum operating frequency supported by the component. Taking the maximum operating frequency as an example, the aforementioned energy consumption gain can also be defined as the difference between the corresponding component operating at full power and the current non-full power operation within time slot t.

[0065] Understandably, taking an air conditioner as an example, within time slot t, the power consumption of the compressor and fan is directly proportional to their operating speed and / or operating frequency. For the expansion valve, its power consumption is directly proportional to the power of the motor used to control the expansion valve's changes. For the baffle, the larger the opening shift each time, the greater its energy consumption. Therefore, this disclosure defines the energy consumption gain of each component within time slot t as shown in the following formula (3): (3) in, This indicates the maximum operating frequency supported by the compressor. This indicates the maximum operating speed supported by the internal fan. This indicates the maximum operating speed supported by the external fan. This indicates the maximum opening degree supported by the expansion valve. This indicates the current position of the wind deflector in time slot t. This indicates the current position of the wind deflector in time slot t-1. , , , and These are all factor parameters that are pre-defined by the system or the user, and this disclosure will not impose further restrictions or details on them.

[0066] This disclosure does not limit the implementation method for determining the above-mentioned energy consumption gain. For example, this disclosure can usually calculate the corresponding energy consumption gain according to the preset frequency and current frequency of the corresponding component using the corresponding formula / rule. Specifically, the energy consumption gain of the corresponding component in time slot t can be calculated according to the aforementioned formula (3). Similarly, the energy consumption gain of the corresponding component in m time slots can be calculated according to the above principle. This disclosure does not limit or elaborate on this.

[0067] S602. Based on the energy consumption gain of each of the n components in the m time slots, calculate the energy consumption gain for the preset time period to obtain the objective function.

[0068] This disclosure does not limit the specific implementation of the above-described energy consumption gain calculation. For example, this disclosure can first calculate the total energy consumption gain for each of the m time slots. Specifically, it can sum the energy consumption gains of the n components within the same time slot to obtain the total energy consumption gain of the n components within the same time slot. The total energy consumption gain is the sum of the energy consumption gains of the n components within the corresponding time slot. Then, the total energy consumption gains within the m time slots are summed and averaged to obtain the above-described objective function.

[0069] In practical applications, based on the example shown in the aforementioned formula (3), the total energy consumption gain of each component within time slot t can be calculated using the formula (3) as shown in the following formula (4): (4) in, This represents the total energy consumption gain mentioned above. It is understandable that this disclosure aims to find the optimal action control parameters for each component under the constraints of energy saving and ensuring comfort, thereby achieving air conditioning temperature regulation. Therefore, the objective function of this disclosure can be described as maximizing energy consumption gain, as shown in the following formula (5): (5) Where max represents taking the maximum value in mathematical operations, and T represents the aforementioned preset time period.

[0070] S502. Perform single-slot transformation on the objective function to obtain the optimization functions corresponding to each of the m slots.

[0071] This disclosure does not limit the specific implementation of the above-described single-slot conversion. For example, please refer to... Figure 7 This is a schematic diagram illustrating a time-slot conversion process according to an exemplary embodiment. For example... Figure 7 The process shown may include the following implementation steps: S701. Based on the total power consumption of each of the n components in the m time slots and the average power consumption of the n components in the preset time period, a corresponding virtual power consumption queue is determined. The virtual power consumption queue is used to reflect the deviation between the total power consumption of the n components in the corresponding time slot and the average power consumption.

[0072] This disclosure can define / determine the corresponding virtual power consumption queue based on the total power consumption of each of the n components within m time slots and the average power consumption of the n components within a preset time period. The details will be described in detail below. Based on this principle, the virtual power consumption queue corresponding to each time slot can be determined, etc., which will not be described in detail here.

[0073] S702. Based on the corresponding virtual power consumption queue, the sum of the energy consumption gains of each of the n components in the m time slots is transformed to obtain the optimization function corresponding to each of the m time slots.

[0074] This disclosure can transform the total energy consumption gain of each of the n components within m time slots based on a corresponding virtual power consumption queue, thereby obtaining the optimization function corresponding to each of the m time slots. This disclosure does not limit the specific implementation of the above transformation; for example, the drift penalty function in Lyapunov optimization can be used to perform a single-time-slot transformation of the objective function (i.e., the sum of the energy consumption gains of each of the n components within m time slots), thereby obtaining the optimization problem corresponding to each of the m time slots.

[0075] In practical applications, this disclosure can transform the aforementioned objective function into an equivalent single-slot optimization problem based on Lyapunov optimization and drift penalty methods. Then, a reinforcement learning model is combined to find the optimal motion control parameters of the component to achieve the goals / objectives of energy saving and comfort. The Lyapunov optimization method, Lyapunov optimization and drift penalty methods, and the transformed single-slot optimization problem involved in this disclosure are described in sequence below.

[0076] This disclosure defines a virtual power consumption queue Q(t+1), which also represents the virtual power consumption queue corresponding to time slot (t+1). This virtual power consumption queue Q(t+1) is used to reflect the deviation between the total power consumption and the average power consumption of each component in the home appliance within time slot t. This virtual power consumption queue Q(t+1) can be represented by the following formula (6): (6) Where Q(t) represents the virtual power consumption queue corresponding to time slot t, used to reflect the accumulation of the queue in time slot T; usually, when t is 0, Q(0) can be initialized to 0. Q(t+1) represents the virtual power consumption queue corresponding to time slot t+1, which is used to reflect the deviation between the total power consumption and the average power consumption of each component in the home appliance in time slot t. max{} represents taking the maximum value in mathematical operations.

[0077] For each time slot t, define the Lyapunov function L(Q(t)) and the Lyapunov drift function for a single time slot. As shown in the following formula (7): (7) Where | represents the OR operation. E[ ] represents the expectation operation in mathematical operations. This value reflects the temperature stability of the virtual power consumption queue; a smaller value indicates greater stability. To achieve a balance between power consumption and energy gain within a single time slot, a drift penalty function is applied to the Lyapunov optimization. The definition is shown in the following formula (8): (8) in, This represents an adjustable variable used to adjust / reflect whether power consumption or energy gain is more important within time slot t. Typically, . , which represents the total energy consumption gain of each component in a household appliance within time slot t.

[0078] This disclosure allows the drift penalty function in the aforementioned Lyapunov optimization to be obtained using the Lyapunov optimization theorem. The upper limit boundary is shown in the following formula (9): (9) Where B represents a constant, .

[0079] When solving the problem, the constant terms that have no effect can be removed, and the objective function formula (5) above can be transformed into a single-slot optimization problem, which is to say, an optimization of the above formula (9). The above single-slot optimization problem, for example, the optimization problem corresponding to slot t, can be shown in the following formula (10): (10) in, This represents the total power consumption of each component in a household appliance within time slot t.

[0080] S503. Determine the preset constraints.

[0081] In this disclosure, the aforementioned preset constraints can refer to the limiting conditions used to constrain the objective function, optimization function, or corresponding reward function. These constraints may be related to the parameters in the corresponding function and can be customized according to actual circumstances; this disclosure does not impose further limitations or descriptions on this. For example, the aforementioned preset constraints may include, but are not limited to, any combination of one or more of the following conditions: the operating parameters of each of the n components must be within the corresponding preset parameter range; the total power consumption of each of the n components within m time slots must be less than or equal to the corresponding preset power consumption threshold; and the average power consumption of the n components within a preset time period T must be less than or equal to the corresponding preset power consumption threshold.

[0082] Understandably, since every component in a household appliance consumes electricity, power consumption should be a limiting factor. This disclosure can approximate the total power consumption of the household appliance within time slot t by using the voltage, current, and equal-length gaps of each component. The specific formula is shown in formula (11): (11) Based on this, the present disclosure can limit / constrain the total power consumption and the average power consumption within the preset time period T, and the corresponding constraint conditions are shown in the following formula (12): (12) in, This represents the preset maximum threshold corresponding to the total power consumption, also known as the preset first power consumption threshold. This represents the preset maximum threshold corresponding to the average power consumption, also known as the preset second power consumption threshold. Both the aforementioned preset first power consumption threshold and the aforementioned preset second power consumption threshold can be power consumption thresholds pre-set by the system or the user. These can be empirical values ​​set based on user experience, or statistical values ​​calculated based on a series of experimental data, etc. This disclosure will not impose further limitations or details on them.

[0084] Based on this, for example, the above-mentioned preset constraints can be described as shown in the following formula (13): (13) In the above formula (12), constraint ① is the constraint on the speed of the internal and external fans. ② is the constraint on the evaporator temperature. ③ is the constraint on the condenser temperature. ④ is the constraint on the compressor. ⑤ is the constraint on the expansion valve and the baffle plate. ⑥ is the power consumption constraint.

[0085] The above This could refer to the preset minimum threshold corresponding to the fan speed. This could refer to the preset maximum threshold corresponding to the fan speed. This could refer to the preset maximum threshold corresponding to the inlet and outlet temperatures of the evaporator. This could refer to the preset maximum threshold corresponding to the inlet and outlet temperatures of the condenser. This could refer to the preset maximum threshold corresponding to the compressor's exhaust temperature. This could refer to the preset minimum threshold corresponding to the compressor's operating frequency. This could refer to the preset maximum threshold corresponding to the compressor's operating frequency. This could refer to the preset minimum threshold corresponding to the expansion valve opening degree. This could refer to the preset maximum threshold corresponding to the opening degree of the expansion valve. This could refer to the preset minimum threshold corresponding to the position of the windshield. This could refer to the preset maximum threshold corresponding to the position of the windshield. These preset minimum and maximum thresholds can be pre-defined by the system or the user, such as empirical values ​​set based on user experience, or statistical values ​​calculated based on a series of experimental data. This disclosure does not impose further limitations or elaborate on these.

[0086] S504. Determine the (overall) reward function. The reward function involved in this disclosure may also be called the overall reward function. For ease of description, the following disclosure will use the overall reward function as an example to illustrate the relevant content, but this does not constitute a limitation.

[0087] The overall reward function described above in this disclosure is a key component in model training, primarily used to guide the agent (i.e., various components in a home appliance) to learn the optimal policy in a complex environment. The purpose of setting the overall reward function in this disclosure is to guide the learning of the agent (specifically, various components in a home appliance). Therefore, in addition to the individual reward functions of each component, this disclosure also sets up a collective reward function and reward / penalty terms to aid in the learning and training of the reinforcement learning model. The specific settings are further described below.

[0088] When certain scenarios / conditions are met, corresponding positive rewards are obtained, as detailed below: A. Each component in the home appliance meets the comfort limit of the Predicted Mean Vote (PMV) intelligent human comfort control system, calculated as shown in the following formula (14): (14) in, It represents the latent heat of vaporization per unit time. This indicates energy consumption gain. This indicates the user's desired temperature setting. This indicates the amount of heat gained or lost during evaporation. This indicates indoor humidity, also known as relative humidity. This indicates the resistance of the clothing. Indicates the temperature of clothing. Indicates air temperature. It indicates the metabolic rate.

[0090] B. The overall collaboration of various components in home appliances enables the temperature to reach the user's set temperature.

[0091] Taking into account the impact of time, safety, and temperature difference on the final reward, this disclosure adds relevant reward and penalty items as follows: A. It encourages a relatively short initial temperature change time.

[0092] B. If the temperature of some sensors exceeds the limit, an additional major penalty will be imposed.

[0093] C. The greater the temperature difference after the temperature control reaches the later stage (after the user-set temperature), the greater the reward or penalty.

[0094] The individual reward function mentioned above can be set as shown in the following formula (15): (15) in, s represents the sensor parameters, a represents the motion control parameters of the corresponding component in the motion space, and t represents the time slot t. The action space corresponding to the above-mentioned components can be referred to in the relevant introduction in the aforementioned formula (13), which will not be repeated here. tem represents the current temperature of the corresponding component. This indicates the corresponding temperature threshold. This refers to reward or punishment items, also known as incentive items. and All of these are setting factors or setting coefficients corresponding to the aforementioned action control parameter a. They can be pre-defined according to the actual situation, such as experience values ​​set based on user experience, or statistical values ​​calculated based on a series of experimental data. This disclosure will not impose further limitations or details on these.

[0095] The above collective reward function can be set as shown in the following formula (16): (16) in, A represents the operating space of the corresponding component in the home appliance. and These are all reward and penalty items involved in the function, and can also be called reward items. This represents the preset minimum threshold corresponding to PMV. The preset maximum threshold corresponding to PMV can be customized in advance according to the actual situation. For example, it can be an empirical value set based on user experience, or a statistical value calculated based on a series of experimental data. This disclosure will not make any further limitations or details on this.

[0096] The overall reward function described above can be set as shown in the following formula (17): (17) in, This represents the overall reward function corresponding to time slot t. This refers to a reward item, which in this case can specifically refer to a penalty item. That is, if the relevant parameters involved in the above reward parameters meet the corresponding preset constraints during operation, then the overall reward function can be the sum of the individual reward functions and the collective reward function; conversely, if the corresponding preset constraints are not met, it indicates a problem, and this disclosure can directly assign a large negative reward. This helps the agent avoid such erroneous explorations.

[0097] In other words, the overall reward function mentioned above can include the individual reward functions of each of the n components in the home appliance and the collective reward function corresponding to the n components. The individual reward function can be determined based at least on the motion control parameters of the corresponding component, and optionally also based on the sensor parameters of the corresponding component, and optionally also by adding corresponding penalty terms, etc., which can be referred to the relevant explanation of the aforementioned formula (15), and will not be repeated here. The collective reward function mentioned above can be determined based at least on the optimization functions corresponding to each of the m time slots mentioned above, and optionally also based on the sensor parameters of the corresponding component and corresponding penalty terms, which can be referred to the relevant explanation of the aforementioned formula (16), and will not be repeated here.

[0098] In an optional embodiment, the overall reward function may further include a preset penalty term. This preset penalty term is custom-set by the system or user based on actual conditions. During operation, if the corresponding parameters meet the aforementioned preset constraints, the overall reward function, including the individual reward function and the collective reward function, as defined in formula (17) can be used for subsequent model training. Conversely, if the aforementioned preset constraints are not met, the preset penalty term in formula (17) can be used as the final overall reward function for subsequent model training. For a detailed introduction to the overall reward function, please refer to the aforementioned description; further details are omitted here.

[0099] S505. Based on the optimization functions corresponding to each of the above m time slots, the above preset constraints, and the above overall reward function, train the reinforcement learning model to be trained, thereby obtaining the trained reinforcement learning model.

[0100] This disclosure utilizes the optimization functions corresponding to the aforementioned m time slots, the aforementioned preset constraints, and the aforementioned overall reward function to train the reinforcement learning model to be trained, thereby obtaining the trained reinforcement learning model. This disclosure does not impose excessive limitations or details on the specific process of model training. For example, during model training, the optimization functions corresponding to the aforementioned m time slots can be maximized while satisfying the aforementioned preset constraints, and then the aforementioned overall reward function can be used to adjust / update the model parameters. Specifically, if the corresponding parameters satisfy the aforementioned preset constraints during the calculation of maximizing the aforementioned optimization functions, this disclosure can assign a reward term through the aforementioned overall reward function; conversely, if the corresponding preset constraints are not satisfied, this disclosure can assign a penalty term through the aforementioned overall reward function to adjust / update the model parameters, thereby guiding model training and making the model better, more accurate, and more precise.

[0101] It should be noted that this disclosure does not limit the execution order of the above implementation steps, which can be customized / set according to the actual situation. For example, this disclosure may execute steps S503 and S504 first, followed by steps S501 and S502; or it may execute step S504 first, then step S503, and finally steps S501 and S502; or it may execute steps S501 and S502 first, then step S504, and finally step S503, etc. This disclosure does not impose further limitations or details on these aspects. The training equipment mentioned in this disclosure may refer to equipment with model training capabilities, which may include, but is not limited to, home appliances, terminal devices, servers, platforms with training capabilities, or other equipment, etc. This disclosure does not impose further limitations or details on these aspects.

[0102] As can be seen, the above-disclosed scheme allows for more refined control of various components in home appliances. Each time slot can be continuously controlled, for example, the motion control parameters can be controlled five times within time slot t. Long-term energy consumption can be constrained; for instance, the stable control achieved through Lyapunov optimization constrains the energy consumption of each time slot. Since a comfort reward is incorporated into the overall reward function, components will adjust their motion control parameters to achieve comfort during operation, subject to corresponding energy consumption constraints. By using a reinforcement learning model to achieve refined control of internal components in home appliances, energy saving and comfort are further enhanced, helping users reduce long-term operating costs compared to existing technologies. In the specific implementation process, the home appliance acquires sensor parameters of the component to be controlled; these sensor parameters are input into a reinforcement learning model, which outputs the motion control parameters of the component. The reinforcement learning model is trained based on optimization functions, preset constraints, and an overall reward function corresponding to each of the m time slots. The optimization functions reflect the total power consumption of n components in the electronic device within their respective time slots. The preset constraints are determined based on the operating parameters of each of the n components. The overall reward function is related to the optimization functions, and both m and n are positive integers. In this way, the home appliance can directly use the reinforcement learning model to calculate and output the motion control parameters of the component to be controlled, facilitating efficient and accurate subsequent control of the component. This improves the convenience and accuracy of component control. It also solves the technical problems in related technologies, such as the difficulty in learning the control of internal components, which affects the refinement and accuracy of control.

[0103] Based on the foregoing embodiments, please refer to Figure 8 This is a schematic diagram illustrating the structure of a component control device according to an exemplary embodiment. For example... Figure 8 The illustrated device can be applied to household appliances. The device may include an acquisition module 801 and a processing module 802. Wherein: The acquisition module 801 is configured to acquire sensor parameters of the component to be controlled; The processing module 802 is configured to input the sensor parameters of the component to be controlled into the reinforcement learning model and output the motion control parameters of the component to be controlled. The reinforcement learning model is obtained by training based on the optimization function, preset constraints and reward function corresponding to each of the m time slots. The optimization function is used to reflect the total power consumption of n components in the electronic device in the corresponding time slot. The preset constraints are determined based on the operating parameters of each of the n components. The reward function is related to the optimization function. Both m and n are positive integers.

[0104] In some embodiments, the processing module 802 is further configured to: A target function is determined, which is used to indicate the sum of the energy consumption gains of each of the n components within a preset time period, wherein the preset time period is the time period corresponding to the m time slots; The objective function is transformed into a single time slot to obtain the optimization functions corresponding to each of the m time slots.

[0105] In some embodiments, the processing module 802 is configured to: Determine the energy consumption gain corresponding to each of the n components in the m time slots. The energy consumption gain is used to indicate the difference between the power consumption of the corresponding component when it operates at a preset frequency and the power consumption when it operates at the current operating frequency in the time slot. The energy consumption gain for the preset time period is calculated based on the energy consumption gain of each of the n components within the m time slots, thus obtaining the objective function.

[0106] In some embodiments, the processing module 802 is configured to: Based on the total power consumption of each of the n components in the m time slots and the average power consumption of the n components in the preset time period, a corresponding virtual power consumption queue is determined. The virtual power consumption queue is used to reflect the deviation between the total power consumption of the n components in the corresponding time slot and the average power consumption. Based on the corresponding virtual power consumption queue, the sum of the energy consumption gains of each of the n components within the m time slots is transformed to obtain the optimization function corresponding to each of the m time slots.

[0107] In some embodiments, the preset constraint includes at least one of the following: The operating parameters of each of the n components are within their respective preset parameter ranges; The total power consumption of each of the n components within the m time slots is less than or equal to the corresponding preset power consumption threshold. The average power consumption of the n components during the preset time period is less than or equal to the corresponding preset power threshold.

[0108] In some embodiments, the reward function includes individual reward functions for each of the n components and a collective reward function for the n components. The individual reward functions are determined based at least on the motion control parameters of the corresponding components, and the collective reward function is determined based at least on the optimization functions corresponding to each of the m time slots.

[0109] In some embodiments, the reward function further includes a preset penalty term; wherein, when the corresponding preset constraint condition is met, the reward function includes individual reward functions for each of the n components and a collective reward function for the n components; or, when the corresponding preset constraint condition is not met, the reward function includes the preset penalty term.

[0110] In some embodiments, the processing module 802 is further configured to: The control operation of the component to be controlled is performed based on the motion control parameters of the component to be controlled.

[0111] For any content not introduced or described in this disclosure, please refer to the relevant descriptions in the foregoing method embodiments; they will not be repeated here.

[0112] By implementing the embodiments of this disclosure, the above-described device can acquire sensor parameters of the component to be controlled; input the sensor parameters of the component to be controlled into a reinforcement learning model, and output the motion control parameters of the component to be controlled; wherein, the reinforcement learning model is obtained by training based on optimization functions, preset constraints, and reward functions corresponding to m time slots, the optimization functions are used to reflect the total power consumption of n components in the electronic device within the corresponding time slots, the preset constraints are determined based on the operating parameters of the n components, the reward function is related to the optimization function, and m and n are both positive integers. In this way, home appliances can directly use the reinforcement learning model to calculate and output the motion control parameters of the component to be controlled, which facilitates efficient and accurate subsequent control of the component. This is beneficial to improving the convenience and accuracy of component control. It also solves the technical problems in related technologies such as the difficulty in learning the control of internal components, which affects the refinement and accuracy of control.

[0113] This disclosure also provides a computer-readable storage medium having computer program instructions stored thereon, which, when executed by a processor, implement the steps of the component control method provided in this disclosure.

[0114] Figure 9 This is a schematic diagram illustrating the structure of a household appliance according to an exemplary embodiment. For example, the household appliance 500 may be an air conditioner, washing machine, dryer, clothes dryer, or other household appliances.

[0115] Reference Figure 9 The home appliance 500 may include one or more of the following components: processing component 502, memory 504, power supply component 506, multimedia component 508, audio component 510, input / output interface 512, sensor component 514, and communication component 516.

[0116] Processing component 502 typically controls the overall operation of device 500, such as operations associated with display, telephone calls, data communication, camera operation, and recording. Processing component 502 may include one or more processors 520 to execute instructions to complete all or part of the steps of the component control method described above. Furthermore, processing component 502 may include one or more modules to facilitate interaction between processing component 502 and other components. For example, processing component 502 may include a multimedia module to facilitate interaction between multimedia component 508 and processing component 502.

[0117] Memory 504 is configured to store various types of data to support the operation of device 500. Examples of this data include instructions for any application or method operating on device 500, contact data, phonebook data, messages, pictures, videos, etc. Memory 504 can be implemented by any type of volatile or non-volatile storage device or a combination thereof, such as static random access memory (SRAM), electrically erasable programmable read-only memory (EEPROM), erasable programmable read-only memory (EPROM), programmable read-only memory (PROM), read-only memory (ROM), magnetic storage, flash memory, magnetic disk, or optical disk.

[0118] Power supply component 506 provides power to various components of device 500. Power supply component 506 may include a power management system, one or more power supplies, and other components associated with generating, managing, and distributing power to device 500.

[0119] Multimedia component 508 includes a screen that provides an output interface between the device 500 and the user. In some embodiments, the screen may include a liquid crystal display (LCD) and a touch panel (TP). If the screen includes a touch panel, the screen may be implemented as a touchscreen to receive input signals from the user. The touch panel includes one or more touch sensors to sense touches, swipes, and gestures on the touch panel. The touch sensors may sense not only the boundaries of the touch or swipe action but also the duration and pressure associated with the touch or swipe operation. In some embodiments, multimedia component 508 includes a front-facing camera and / or a rear-facing camera. When the device 500 is in an operating mode, such as a shooting mode or a video mode, the front-facing camera and / or the rear-facing camera may receive external multimedia data. Each front-facing camera and rear-facing camera may be a fixed optical lens system or have focal length and optical zoom capabilities.

[0120] Audio component 510 is configured to output and / or input audio signals. For example, audio component 510 includes a microphone (MIC) configured to receive external audio signals when device 500 is in an operating mode, such as call mode, recording mode, and voice recognition mode. The received audio signals may be further stored in memory 504 or transmitted via communication component 516. In some embodiments, audio component 510 also includes a speaker for outputting audio signals.

[0121] Input / output interface 512 provides an interface between processing component 502 and peripheral interface modules, which may be keyboards, click wheels, buttons, etc. These buttons may include, but are not limited to, home buttons, volume buttons, start buttons, and lock buttons.

[0122] Sensor assembly 514 includes one or more sensors for providing state assessments of various aspects of device 500. For example, sensor assembly 514 may detect the on / off state of device 500, the relative positioning of components such as the display and keypad of device 500, changes in the position of device 500 or a component of device 500, the presence or absence of user contact with device 500, the orientation or acceleration / deceleration of device 500, and temperature changes of device 500. Sensor assembly 514 may include a proximity sensor configured to detect the presence of nearby objects without any physical contact. Sensor assembly 514 may also include a light sensor, such as a CMOS or CCD image sensor, for use in imaging applications. In some embodiments, sensor assembly 514 may also include an accelerometer, a gyroscope, a magnetometer, a pressure sensor, or a temperature sensor.

[0123] Communication component 516 is configured to facilitate wired or wireless communication between device 500 and other devices. Device 500 can access wireless networks based on communication standards, such as WiFi, 2G, or 3G, or combinations thereof. In one exemplary embodiment, communication component 516 receives broadcast signals or broadcast-related information from an external broadcast management system via a broadcast channel. In one exemplary embodiment, communication component 516 also includes a near-field communication (NFC) module to facilitate short-range communication. For example, the NFC module may be implemented based on radio frequency identification (RFID) technology, Infrared Data Association (IrDA) technology, ultra-wideband (UWB) technology, Bluetooth (BT) technology, and other technologies.

[0124] In an exemplary embodiment, device 500 may be implemented by one or more application-specific integrated circuits (ASICs), digital signal processors (DSPs), digital signal processing devices (DSPDs), programmable logic devices (PLDs), field-programmable gate arrays (FPGAs), controllers, microcontrollers, microprocessors, or other electronic components to perform the component control method described above.

[0125] In an exemplary embodiment, a non-transitory computer-readable storage medium including instructions is also provided, such as a memory 504 including instructions, which can be executed by the processor 520 of the device 500 to complete the above-described upper-level component control method. For example, the non-transitory computer-readable storage medium may be a ROM, random access memory (RAM), CD-ROM, magnetic tape, floppy disk, and optical data storage device, etc.

[0126] The aforementioned device can be a standalone electronic device or a part of a standalone electronic device. For example, in one embodiment, the device can be an integrated circuit (IC) or a chip, wherein the integrated circuit can be a single IC or a collection of multiple ICs. The chip can include, but is not limited to, the following types: GPU (Graphics Processing Unit), CPU (Central Processing Unit), FPGA (Field Programmable Gate Array), DSP (Digital Signal Processor), ASIC (Application Specific Integrated Circuit), and SoC (System on Chip). The aforementioned integrated circuit or chip can be used to execute executable instructions (or code) to implement the aforementioned component control method. The executable instructions can be stored in the integrated circuit or chip or obtained from other devices or equipment. For example, the integrated circuit or chip includes a processor, memory, and an interface for communicating with other devices. The executable instructions can be stored in the memory, and when the executable instructions are executed by the processor, the above-described component control method is implemented; or, the integrated circuit or chip can receive the executable instructions through the interface and transmit them to the processor for execution to implement the above-described component control method.

[0127] In another exemplary embodiment, a computer program product is also provided, the computer program product comprising a computer program executable by a programmable device, the computer program having a code portion for performing the component control method described above when executed by the programmable device.

[0128] Please see Figure 10 This is a schematic diagram illustrating the structure of a chip according to an exemplary embodiment. For example... Figure 10 The chip 600 shown includes a processor 601 and an interface 602. Optionally, it may also include a memory 603. The number of processors 601 can be one or more, and the number of interfaces 602 can be multiple.

[0129] In one embodiment, for the case where the chip is used to implement the method embodiments described in this disclosure: The interface 602 is used to receive or output signals; The processor 601 is used to execute some or all of the contents of the above-described component control method embodiments.

[0130] Understandably, the processor in this embodiment of the disclosure can be an integrated circuit chip with signal processing capabilities. In implementation, each step of the above method embodiment can be completed by integrated logic circuits in the processor's hardware or by instructions in software form. The processor described above can be a general-purpose processor, a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a field-programmable gate array (FPGA), or other programmable logic devices, discrete gate or transistor logic devices, or discrete hardware components.

[0131] Understandably, the memory in the embodiments of this disclosure may be volatile memory or non-volatile memory, or may include both volatile and non-volatile memory. The non-volatile memory may be read-only memory (ROM), programmable read-only memory (PROM), erasable programmable read-only memory (EPROM), electrically erasable programmable read-only memory (EEPROM), or flash memory. The volatile memory may be random access memory (RAM), which serves as an external cache. By way of example, but not limitation, many forms of RAM are available, such as static random access memory (SRAM), dynamic random access memory (DRAM), synchronous dynamic random access memory (SDRAM), double data rate synchronous dynamic random access memory (DDRSDRAM), enhanced synchronous dynamic random access memory (ESDRAM), synchronous linked dynamic random access memory (SLDRAM), and direct rambus RAM (DR RAM). It should be noted that the memory used in the systems and methods described herein is intended to include, but is not limited to, these and any other suitable types of memory.

[0132] It should be noted that the descriptions of the storage media, devices, and chip embodiments above are similar to the descriptions of the method embodiments above, and have similar beneficial effects. For technical details not disclosed in the storage media, storage media, and device embodiments of this disclosure, please refer to the descriptions of the method embodiments of this disclosure for understanding.

[0133] Other embodiments of this disclosure will readily occur to those skilled in the art upon consideration of the specification and practice of this disclosure. This disclosure is intended to cover any variations, uses, or adaptations of this disclosure that follow the general principles of this disclosure and include common knowledge or customary techniques in the art not disclosed herein. The specification and examples are to be considered exemplary only, and the true scope and spirit of this disclosure are indicated by the following claims.

[0134] It should be understood that this disclosure is not limited to the precise structures described above and shown in the accompanying drawings, and various modifications and changes can be made without departing from its scope. The scope of this disclosure is limited only by the appended claims.

Claims

1. A component control method, characterized in that, include: Obtain the sensor parameters of the component to be controlled; The sensor parameters of the component to be controlled are input into the reinforcement learning model, and the action control parameters of the component to be controlled are output. The reinforcement learning model is obtained by training based on the optimization function, preset constraints and reward function corresponding to each of the m time slots. The optimization function is used to reflect the total power consumption of n components in the electronic device in the corresponding time slot. The preset constraints are determined based on the operating parameters of each of the n components. The reward function is related to the optimization function. Both m and n are positive integers.

2. The method according to claim 1, characterized in that, The method further includes: A target function is determined, which is used to indicate the sum of the energy consumption gains of each of the n components within a preset time period, wherein the preset time period is the time period corresponding to the m time slots; The objective function is transformed into a single time slot to obtain the optimization functions corresponding to each of the m time slots.

3. The method according to claim 2, characterized in that, The determination of the objective function includes: Determine the energy consumption gain corresponding to each of the n components in the m time slots. The energy consumption gain is used to indicate the difference between the power consumption of the corresponding component when it operates at a preset frequency and the power consumption when it operates at the current operating frequency in the time slot. The energy consumption gain for the preset time period is calculated based on the energy consumption gain of each of the n components within the m time slots, thus obtaining the objective function.

4. The method according to claim 2, characterized in that, The step of performing single-slot transformation on the objective function to obtain the optimization functions corresponding to each of the m slots includes: Based on the total power consumption of each of the n components in the m time slots and the average power consumption of the n components in the preset time period, a corresponding virtual power consumption queue is determined. The virtual power consumption queue is used to reflect the deviation between the total power consumption of the n components in the corresponding time slot and the average power consumption. Based on the corresponding virtual power consumption queue, the sum of the energy consumption gains of each of the n components within the m time slots is transformed to obtain the optimization function corresponding to each of the m time slots.

5. The method according to any one of claims 1-4, characterized in that, The preset constraints include at least one of the following: The operating parameters of each of the n components are within their respective preset parameter ranges; The total power consumption of each of the n components within the m time slots is less than or equal to the corresponding preset power consumption threshold. The average power consumption of the n components during the preset time period is less than or equal to the corresponding preset power threshold.

6. The method according to any one of claims 1-4, characterized in that, The reward function includes individual reward functions for each of the n components and a collective reward function for the n components. The individual reward functions are determined based at least on the motion control parameters of the corresponding components, and the collective reward function is determined based at least on the optimization functions corresponding to each of the m time slots.

7. The method according to claim 6, characterized in that, The reward function further includes a preset penalty term; wherein, when the corresponding preset constraint condition is met, the reward function includes individual reward functions for each of the n components and a collective reward function for the n components; or, when the corresponding preset constraint condition is not met, the reward function includes the preset penalty term.

8. The method according to any one of claims 1-4, characterized in that, The method further includes: The control operation of the component to be controlled is performed based on the motion control parameters of the component to be controlled.

9. A component control device, characterized in that, include: The acquisition module is configured to acquire sensor parameters of the component to be controlled; The processing module is configured to input the sensor parameters of the component to be controlled into the reinforcement learning model and output the motion control parameters of the component to be controlled. The reinforcement learning model is obtained by training based on the optimization function, preset constraints and reward function corresponding to each of the m time slots. The optimization function is used to reflect the total power consumption of n components in the electronic device in the corresponding time slot. The preset constraints are determined based on the operating parameters of each of the n components. The reward function is related to the optimization function. Both m and n are positive integers.

10. A household appliance, characterized in that, include: processor; Memory used to store processor-executable instructions; The processor is configured to execute the executable instructions to implement the steps of the method according to any one of claims 1 to 8.

11. A computer-readable storage medium storing computer program instructions thereon, characterized in that, When the computer program instructions are executed by the processor, they implement the steps of the method according to any one of claims 1 to 8.

12. A chip, characterized in that, It includes a processor and an interface; the processor is used to read instructions to execute the method of any one of claims 1 to 8.

Images