Control method and device of photovoltaic energy storage system, electronic equipment and photovoltaic energy storage system

Abstract

The invention relates to the field of electric energy adjustment, and provides a control method and device of a photovoltaic energy storage system, electronic equipment and the photovoltaic energy storage system. The control method comprises the steps that input parameters are acquired, the input parameters comprise the current power grid state of the photovoltaic energy storage system and the power utilization habit of a user, and the current power grid state comprises the current power grid load condition and the current electricity price; selecting a target power supply strategy from a plurality of preset power supply strategies of the photovoltaic energy storage system according to the input parameters and an energy management model; according to the target power supply strategy and the current power grid state, predicting an updated power grid state after the target power supply strategy is adopted; and in response to user feedback, selectively executing the target power supply strategy or updating the target power supply strategy. By adopting the embodiment of the invention, the real demand of a user side is solved, and a power supply strategy conforming to the objective condition of a user home power grid is provided.

Description

Technical Field

[0001] The present application relates to the field of electric energy regulation, and in particular to a control method, device, electronic equipment and photovoltaic energy storage system. Background Art

[0002] In recent years, with the widespread popularity of electric vehicles, the construction of charging stations, battery swap stations and battery swap cabinets has also been fully rolled out. Due to the reduction in the cost of solar photovoltaic power generation and the shortening of its construction payback period, solar photovoltaic power generation technology has been widely applied and promoted, and the construction of photovoltaic storage systems using solar photovoltaic power generation technology is also increasing. The load of the power grid is constantly changing, sometimes with peak power consumption and sometimes with trough power consumption. Under the condition of a certain capacity of power generation and supply equipment, when the power grid load is at its peak, the power load is greater than the output of the generator, resulting in a decrease in the frequency of the power grid, and even endangering the safe operation of the system; when the power grid load is at its low point, the output of the generator is significantly reduced, so that the capacity of the power generation and supply equipment cannot be fully utilized, and the system operation is very uneconomical.

[0003] Now there is another method, which is to increase the selling price of electricity during peak load period and reduce the selling price of electricity during low load period, that is, to implement time-of-use pricing, which economically encourages more electricity consumption during low load period and encourages non-continuous production departments to avoid peak load electricity consumption and fill valley.

[0004] However, the above solutions cannot effectively solve the real needs of users and it is difficult to provide a power supply strategy that meets the objective conditions of the user's home power grid. Summary of the invention

[0005] In order to solve the problem that it is difficult for the prior art to provide a power supply strategy that meets the objective pollution conditions of user households, the embodiments of the present application provide a control method, device, electronic equipment and photovoltaic energy storage system for a photovoltaic energy storage system.

[0006] A first aspect of an embodiment of the present application provides an air conditioning control method, the method comprising:

[0007] Acquire input parameters, wherein the input parameters include the current grid state of the photovoltaic energy storage system and the user's electricity usage habits, wherein the current grid state includes the current grid load and the current electricity price;

[0008] According to the input parameters and the energy management model, a target power supply strategy is selected from a plurality of power supply strategies preset for the photovoltaic energy storage system;

[0009] According to the target power supply strategy and the current power grid state, predicting an updated power grid state after adopting the target power supply strategy;

[0010] In response to user feedback, the target power supply strategy is selectively executed or updated.

[0011] In combination with the first aspect, in an implementation of the first aspect, the current grid load condition includes at least one of the current total power consumption, the accumulated power consumption in the current period, the electrical appliances working in the current period, and the operating time of the photovoltaic energy storage system.

[0012] In combination with the first aspect, in an implementation of the first aspect, the energy management model is a deep reinforcement learning model, and generating a target power supply strategy according to the input parameters and the energy management model includes:

[0013] The input parameters are input into the deep reinforcement learning model to obtain the target power supply strategy output by the deep reinforcement learning model, wherein the deep reinforcement learning model takes the input parameters as input and takes at least one of the multiple power supply strategies as output.

[0014] In combination with the first aspect, in an implementation of the first aspect, the deep reinforcement learning model is trained in the following manner:

[0015] Obtaining a first value estimate for executing a first target power supply strategy under a power grid state at a first moment;

[0016] Obtain a second value estimate for executing a second target power supply strategy under a power grid state at a second moment;

[0017] Based on the first value estimate and the second value estimate, updating model parameters of the deep reinforcement learning model;

[0018] Among them, the first target power supply strategy is the target power supply strategy obtained by the deep reinforcement learning model with the power grid state at the first moment and the user's power usage habits as input, the environmental state at the second moment is the power grid state predicted based on the power grid state at the first moment and the first target power supply strategy, and the second target power supply strategy is the target power supply strategy obtained by the deep reinforcement learning model with the power grid state at the second moment and the user's power usage habits as input.

[0019] In combination with the first aspect, in an implementation of the first aspect, predicting, according to the target power supply strategy and the current power grid state, an updated power grid state after adopting the target power supply strategy includes:

[0020] The target power supply strategy and the current power grid state are input into a state prediction model to obtain the updated power grid state output by the state prediction model, wherein the state prediction model takes the target power supply strategy and the current power grid state as input and takes the updated power grid state as output.

[0021] In combination with the first aspect, in an implementation of the first aspect, the training data used by the state prediction model during the training process includes:

[0022] Grid load conditions and electricity prices during the target time period.

[0023] In combination with the first aspect, in an implementation of the first aspect, selectively executing the target power supply strategy or updating the target power supply strategy in response to user feedback includes:

[0024] In case of feedback from the first user, executing the target power supply strategy; and / or,

[0025] In the case of feedback from the second user, reselecting a target power supply strategy from a plurality of power supply strategies preset for the photovoltaic energy storage system according to the input parameters and the energy management model, or,

[0026] In the case of second user feedback and the input parameters also including user needs, the user needs in the input parameters are updated according to the updated user needs, and then the target power supply strategy is re-selected from the preset multiple power supply strategies of the photovoltaic energy storage system according to the input parameters and the energy management model, and the second user feedback is different from the first user feedback.

[0027] In combination with the first aspect, in an implementation manner of the first aspect, the input parameters also include user requirements.

[0028] In combination with the first aspect, in an implementation manner of the first aspect, the method further includes:

[0029] The user demand is extracted from the demand input by the user based on the language model, and the form of the user demand meets the format requirement of the input parameter of the energy management model.

[0030] In combination with the first aspect, in an implementation manner of the first aspect, the method further includes:

[0031] Sending recommendation information to the user, the recommendation information including the target power supply strategy and the updated power grid state corresponding to the target power supply strategy, or the recommendation information being power consumption information (power consumption, total power price) determined according to the updated power grid state;

[0032] receiving user feedback regarding the recommendation information;

[0033] Among them, the number of the energy management models is at least one, and the number of the target power supply strategies is at least one.

[0034] A second aspect of an embodiment of the present application provides a control device for a photovoltaic energy storage system, the control device comprising:

[0035] An input parameter acquisition module is used to acquire input parameters, wherein the input parameters include the current grid state of the photovoltaic energy storage system and the user's electricity usage habits, wherein the current grid state includes the current grid load situation and the current electricity price;

[0036] An energy management model, used for selecting a target power supply strategy from a plurality of power supply strategies preset for the photovoltaic energy storage system according to the input parameters;

[0037] A state prediction model, used to predict an updated power grid state after adopting the target power supply strategy according to the target power supply strategy and the current power grid state;

[0038] The policy processing module is used to selectively execute the target power supply policy or update the target power supply policy in response to user feedback.

[0039] In conjunction with the second aspect, in an implementation of the second aspect, the method further includes:

[0040] A recommendation module, configured to send recommendation information to a user, wherein the recommendation information includes the target power supply strategy and the updated power grid state corresponding to the target power supply strategy, or the recommendation information is power consumption information determined according to the updated power grid state;

[0041] The user feedback module is used to obtain the user feedback on the recommended information.

[0042] A third aspect of an embodiment of the present application provides an electronic device, the electronic device comprising: a memory for storing computer instructions;

[0043] A processor is used to call the computer instruction to implement the control method according to the first aspect of the embodiment of the present application.

[0044] A fourth aspect of the embodiments of the present application provides a photovoltaic energy storage system, which adopts the control method described in the first aspect of the embodiments of the present application, or includes the control device described in the second aspect of the embodiments of the present application, or includes the electronic device described in the third aspect of the embodiments of the present application.

[0045] By adopting the method provided in this embodiment, on the one hand, determining the target power supply strategy according to the power grid status and the user's power usage habits is conducive to obtaining a power supply strategy that takes into account or balances the actual power usage status of the home power grid and the user's habits; on the other hand, predicting the updated power grid status after adopting the target power supply strategy is conducive to presenting the possible power grid status corresponding to the target power supply strategy to the user, thereby providing the user with objective expectations; on the other hand, in response to user feedback, the target power supply strategy is selectively executed or updated, so that the target power supply strategy can meet the user's personalized needs. BRIEF DESCRIPTION OF THE DRAWINGS

[0046] Figure 1 It is a structural schematic diagram of a household power grid (including a photovoltaic energy storage system);

[0047] Figure 2 A schematic diagram of a control method for a photovoltaic energy storage system according to an embodiment of the present application;

[0048] Figure 3 A schematic diagram of a photovoltaic energy storage system control method according to an embodiment of the present application;

[0049] Figure 4 A schematic diagram of a training process of a deep reinforcement learning model according to an embodiment of the present application;

[0050] Figure 5 4 is a block diagram of a control device for a photovoltaic energy storage system according to an embodiment of the present application. DETAILED DESCRIPTION

[0051] Embodiments of the present invention are described in detail below, examples of which are shown in the accompanying drawings, wherein the same or similar reference numerals throughout represent the same or similar elements or elements having the same or similar functions. The embodiments described below with reference to the accompanying drawings are exemplary and are intended to be used to explain the present invention, and should not be construed as limiting the present invention.

[0052] Throughout the specification and claims, the following terms have at least the meanings explicitly associated herein, unless the context dictates otherwise. The meanings identified below do not necessarily limit the terms, but merely provide illustrative examples of the terms.

[0053] In the description of the present invention, the phrase "in one embodiment" does not necessarily refer to the same embodiment, although it may refer to the same embodiment. Similarly, the phrase "in some embodiments" as used herein, when used multiple times, does not necessarily refer to the same embodiment, although it may refer to the same embodiment. As used herein, the term "or" is an inclusive "or" operator and is equivalent to the term "and / or", unless the context clearly dictates otherwise. The term "based on" is not exclusive and allows for additional factors not described, unless the context clearly dictates otherwise. The word "exemplary" means "used as an example, instance or illustration" in this article. Any embodiment described as "exemplary" herein is not necessarily interpreted as being superior or superior to other embodiments. The scope of the present invention is limited only by the scope of the appended claims, and any examples set forth in this specification are not intended to be limiting, but only to set forth some of the many possible embodiments of the claimed invention. The various embodiments provided by the present invention should not be interpreted as limiting the scope of protection of the present invention.

[0054] In the description of the present invention, it should be understood that the terms "center", "longitudinal", "lateral", "length", "width", "thickness", "up", "down", "front", "back", "left", "right", "vertical", "horizontal", "top", "bottom", "inside", "outside", "clockwise", "counterclockwise" and the like indicate orientations or positional relationships based on the orientations or positional relationships shown in the accompanying drawings, and are only for the convenience of describing the present invention and simplifying the description, rather than indicating or implying that the referred device or element must have a specific orientation, be constructed and operated in a specific orientation, and therefore should not be understood as limiting the present invention.

[0055] In addition, the terms "first" and "second" are used for descriptive purposes only and should not be understood as indicating or implying relative importance or implicitly indicating the number of the indicated technical features. Therefore, the features defined as "first" and "second" may explicitly or implicitly include one or more of the features. In the description of the present invention, the meaning of "plurality" is two or more, unless otherwise clearly and specifically defined.

[0056] In the present invention, unless otherwise clearly specified and limited, the terms "installed", "connected", "connected", "fixed" and the like should be understood in a broad sense, for example, it can be a fixed connection, a detachable connection, or an integral connection; it can be a mechanical connection or an electrical connection; it can be a direct connection or an indirect connection through an intermediate medium, it can be the internal connection of two elements or the interaction relationship between two elements. For ordinary technicians in this field, the specific meanings of the above terms in the present invention can be understood according to specific circumstances.

[0057] In the present invention, unless otherwise clearly specified and limited, a first feature being "above" or "below" a second feature may include that the first and second features are in direct contact, or may include that the first and second features are not in direct contact but are in contact through another feature between them. Moreover, a first feature being "above", "above" and "above" a second feature includes that the first feature is directly above and obliquely above the second feature, or simply indicates that the first feature is higher in level than the second feature. A first feature being "below", "below" and "below" a second feature includes that the first feature is directly below and obliquely below the second feature, or simply indicates that the first feature is lower in level than the second feature.

[0058] First, the terms and names involved in the present application are explained as follows. These explanations are intended to facilitate those skilled in the art to understand the embodiments of the present application, but are not intended to limit the understanding of the embodiments of the present application.

[0059] Deep reinforcement learning: Deep reinforcement learning (DRL) is a technology that combines reinforcement learning (RL) with deep learning (DL), aiming to enable the agent to learn the optimal behavior strategy through interaction with the environment.

[0060] The working principle of the deep reinforcement learning model can be summarized as the following steps: Perceiving the environment: The agent obtains the state information of the environment through sensors or interfaces; State representation: The state information usually needs to be preprocessed (such as image scaling, feature extraction, etc.) before being input into the deep neural network; Decision making: The neural network outputs the probability distribution of one or more actions or directly outputs an action based on the input state information; Executing actions: The agent performs corresponding behaviors according to the actions or action probability distributions output by the network; Receiving feedback: After the agent performs an action, it receives immediate rewards from the environment and enters a new state; Updating the model: According to the results of the agent's behavior (rewards and new states), the parameters of the neural network are updated through a certain learning algorithm (such as Q-learning, policy gradient, etc.) to make better decisions in the future.

[0061] The main components of deep reinforcement learning include: Agent: the subject that learns and executes strategies; Environment: the external world where the agent is located, which can provide state information and rewards; State: describes the current status of the environment; Action: the behavior that the agent can perform; Reward: the feedback obtained from the environment after the agent performs an action, which is used to evaluate the quality of the action; Policy: the principle of the agent's selection of actions, which can be deterministic or random; Value Function: evaluates the value of a state or state-action pair, such as Q function or V function; Deep Neural Network: a model used to learn policies or value functions.

[0062] LSTM (Long Short-Term Memory): is a special recurrent neural network (RNN) designed to solve the gradient vanishing or gradient exploding problems encountered by traditional RNNs when processing long sequence data. LSTM introduces a gate mechanism to enable the network to learn long-term dependencies and effectively retain important information.

[0063] Home photovoltaic energy storage system: Figure 1The following is a schematic diagram of the structure of a household power grid (including a photovoltaic energy storage system).

[0064] Public power grid: It can supply power to the home and also receive excess power from the home. Energy storage battery: Energy storage battery is used to store excess power and release it for home use when needed. Photovoltaic module: Photovoltaic module is a solar panel that converts sunlight into direct current (DC). MPPT (maximum power point tracking): The MPPT controller is responsible for obtaining the maximum power output from the photovoltaic module and converting it into a voltage and current suitable for the inverter. Energy management controller: The energy management controller is the hub of the entire system. It monitors and controls the operating status of all devices to ensure the efficient distribution and use of energy. INVERTER: The inverter converts DC power from the MPPT or the power grid into alternating current (AC) so that it can be supplied to the home grid or other loads. Home grid: The home grid refers to the circuit network to which various electrical appliances and devices in the home are connected. AC / DC and DC / DC converters: AC / DC converters convert AC power into DC power; DC / DC converters convert DC power between different voltage levels.

[0065] By selecting different control strategies, the electricity generated by photovoltaic panels can be used directly in the household power grid, or supplied by the public power grid, or stored in energy storage batteries as a backup power source.

[0066] The present application embodiment provides a control method for a photovoltaic energy storage system, which can provide users with a power supply method that is more in line with actual power consumption conditions. Figure 2 FIG. 1 is a flow chart of a control method for a photovoltaic energy storage system provided in an embodiment of the present application. Figure 2 , the method includes the following processing steps.

[0067] S100: Obtain input parameters.

[0068] Among them, the input parameters include the current grid status of the photovoltaic energy storage system and the user's electricity usage habits.

[0069] The grid status refers to the status of the household grid, including grid compliance and current electricity prices. For example, real-time electricity price information can be obtained by communicating with the Internet.

[0070] Optionally, in an implementation of this embodiment, the current grid load includes at least one of the current total power consumption, the accumulated power consumption in the current period, the appliances working in the current period, and the operating time of the photovoltaic energy storage system. For example, it may include any combination of two or more of them, or include all of the above information.

[0071] S102: Selecting a target power supply strategy from a plurality of power supply strategies of a preset photovoltaic energy storage system according to input parameters and an energy management model.

[0072] Exemplarily, the power supply strategy includes energy-saving strategy, backup power strategy, peak-shifting strategy, etc. The details will be described below.

[0073] The energy management model takes at least the input parameters as input and the target power supply strategy as output. The target power supply strategy may be an optimal strategy or a plurality of power supply strategies ranked according to the degree of preference.

[0074] S104: According to the target power supply strategy and the current power grid state, predict an updated power grid state after adopting the target power supply strategy.

[0075] For example, a state prediction model is used to predict the state of the power grid at the next moment (ie, update the power grid state) based on the target power supply strategy at the previous moment and the current power grid state.

[0076] S106: In response to user feedback, selectively execute the target power supply strategy or update the target power supply strategy.

[0077] For example, if the first user gives feedback (for example, the user gives feedback that he is satisfied), the target power supply strategy is executed; if the second user gives feedback (for example, the user is dissatisfied), the target power supply strategy is updated. The first user and the second user feedback may both contain more specific opinions of the user, such as the target power supply strategy, the policy content that the user likes and / or the policy content that the user does not like.

[0078] By adopting the method provided in this embodiment, on the one hand, determining the target power supply strategy according to the power grid status and the user's power usage habits is conducive to obtaining a power supply strategy that takes into account or balances the actual power usage status of the home power grid and the user's habits; on the other hand, predicting the updated power grid status after adopting the target power supply strategy is conducive to presenting the possible power grid status corresponding to the target power supply strategy to the user, thereby providing the user with objective expectations; on the other hand, in response to user feedback, the target power supply strategy is selectively executed or updated, so that the target power supply strategy can meet the user's personalized needs.

[0079] In addition, in other implementations of this embodiment, the input parameters may also include user requirements. In this case, it is more conducive to the embodiment of the present application to provide a target power supply strategy that meets the user requirements, thereby meeting the requirements of different users.

[0080] Optionally, in one implementation of this embodiment, the energy control manager in the photovoltaic energy storage system can be used to obtain the high-frequency transient characteristics of the user's home power grid in history (for example, the power and voltage of electrical appliances over a period of time), and analyze the correlation between the power grid load size and time to characterize the user's electricity usage habits.

[0081] For example, the user's electricity usage habits may include: the time of day when the electricity usage is high, the time of day when the electricity usage is medium, and the time of day when the electricity usage is low; more specifically, each electricity load may be specified. For example, the time of day when the air conditioner's electricity usage is high, medium, or low, the time of day when the television's electricity usage is high, medium, or low, etc. The classification of high, medium, or low can be determined as needed and is not limited here.

[0082] Optionally, in an implementation of this embodiment, in S102, the energy management model is a deep reinforcement learning model, and a target power supply strategy is generated according to input parameters and the energy management model, including: inputting the input parameters into the deep reinforcement learning model to obtain the target power supply strategy output by the deep reinforcement learning model, wherein the deep reinforcement learning model takes the input parameters as input and uses the target power supply strategy as output.

[0083] By adopting this implementation method, a target power supply strategy corresponding to the current power grid state (or the current power grid state and user demand) can be obtained based on the deep reinforcement learning model.

[0084] It should be noted that the deep reinforcement learning model is part of the Markov Decision Process (MDP). Therefore, in other embodiments, other MDPs may be used to replace the deep reinforcement learning model.

[0085] Exemplarily, the deep reinforcement learning model is trained in the following manner.

[0086] First, obtain the first value valuation of executing the first target power supply strategy under the power grid state at the first moment; then, obtain the second value valuation of executing the second target power supply strategy under the power grid state at the second moment; then, based on the first value valuation and the second value valuation, update the model parameters of the deep reinforcement learning model. Among them, the first target power supply strategy is the target power supply strategy obtained by the deep reinforcement learning model with the power grid state at the first moment and the user's power usage habits ("the first moment" is the historical moment corresponding to the training data, and the power grid state at the first moment and the user's power usage habits are all historical data) as input, the environmental state at the second moment is the power grid state predicted based on the power grid state at the first moment and the first target power supply strategy, and the second target power supply strategy is the target power supply strategy obtained by the deep reinforcement learning model with the power grid state at the second moment and the user's power usage habits ("the second moment" is the historical moment corresponding to the training data, and the power grid state at the second moment and the user's power usage habits are all historical data) as input. Among them, the second moment is the next moment of the first moment.

[0087] In this way, the grid status and user habits are taken into account during the model training process, so that the trained deep reinforcement learning model can output a target power supply strategy that meets the user's power usage habits and grid characteristics with a high probability. Among them, illustratively, the LSTM model mentioned below can be used to predict the grid status at the second moment.

[0088] It should be noted that in other implementations, the input parameters may include user needs, and the first target power supply strategy may be a target power supply strategy obtained by the deep reinforcement learning model with the power grid state at the first moment, the user's power usage habits and the user's needs as inputs; the second target power supply strategy may be a target power supply strategy obtained by the deep reinforcement learning model with the power grid state at the second moment, the user's power usage habits and the user's needs as inputs. The deep reinforcement learning model trained based on such data can output a target power supply strategy that satisfies the user's power usage habits and power grid characteristics with a high probability, while meeting the user's current needs.

[0089] Optionally, in an implementation method of the embodiment of the present application, in S104, based on the target power supply strategy and the current power grid state, an updated power grid state after adopting the target power supply strategy is predicted, including: inputting the target power supply strategy and the current power grid state into a state prediction model, and obtaining an updated power grid state output by the state prediction model, wherein the state prediction model takes the target power supply strategy and the current power grid state as input, and takes the updated power grid state as output.

[0090] Exemplarily, the inventors consider that the operating state of the household photovoltaic energy storage system is a time-continuous transient state, so they can collect relevant data of the household photovoltaic energy storage system and use the LSTM (Long Short-Term Memory Recursive Neural Network) model to predict the state of the power grid. It should be noted that LSTM is an optimization algorithm of a recurrent neural network (RNN). Optionally, other RNNs can be used as substitutes in other embodiments. For example: basic RNN, RNN variant, LSTM variant, etc.

[0091] By adopting this implementation method, the grid state after executing the target power supply strategy under the current grid state can be relatively accurately predicted based on the target power supply strategy and the current grid state, which is conducive to providing intuitive information reference for users.

[0092] Exemplarily, the training data used in the state prediction model during the training process include: grid load conditions and electricity prices in the target time period. The grid load conditions include at least one of the total power consumption, cumulative power consumption, working electrical appliances, and the operating time of the photovoltaic energy storage system. The electricity price refers to the electricity price corresponding to the current time period, which can be obtained through communication with the Internet. Of course, in other implementations, the training data may also include user demand in the current time period.

[0093] By using the training data provided by this implementation and the conventional LSTM model training method, a state prediction model that can relatively accurately predict the state of the power grid at the next moment can be obtained.

[0094] In other implementations, the training data used by the state prediction model during the training process may also include user needs and / or user electricity usage habits.

[0095] Optionally, in an implementation of this embodiment, in S106, the target power supply strategy may be executed when the first user provides feedback (for example, user satisfaction), so that the power supply strategy can meet the actual needs of the user.

[0096] Optionally, in one implementation of this embodiment, in S106, in the case of feedback from the second user (for example, the user is dissatisfied), the target power supply strategy can be reselected from the multiple power supply strategies of the preset photovoltaic energy storage system. In this way, through cyclic iteration, it is helpful to generate a target power supply strategy that meets the user's needs.

[0097] In one embodiment of the present application, as described above, in S100, the input parameters may include not only the grid status and the user's electricity usage habits, but also the user's needs. The user's needs refer to the user's needs at present or in the future, which may be specific to the power load to be used, and may be more specific to the power load to be used and its usage time and usage duration.

[0098] Optionally, in this embodiment, user needs can be extracted from user needs input by users based on a language model (for example, a large language model, GPT, Wenxin Yiyan, Doubao, Tongyi Qianwen, or a specially trained language model), and the form of the extracted user needs meets the format requirements of the input parameters of the energy management model.

[0099] Exemplarily, the GPT model can be used to analyze the user input needs and user habits, extract the corresponding logical information, and convert the information into training parameters for deep reinforcement learning. The user input needs can be voice or text, and the content is natural language. For example, the user inputs: "I plan to turn on the water heater to take a shower in 1 hour, go to bed after the shower, turn the air conditioner to sleep mode before going to bed, and turn off the air conditioner when I go out at 8 o'clock tomorrow morning." The corresponding user needs can be: water heater-temperature-air conditioner-sleep, go out-turn off the air conditioner; or, it can also include corresponding time information, such as: water heater and usage time and usage duration-temperature-air conditioner and usage time and usage duration-sleep, go out (out time)-turn off the air conditioner and turn off time and downtime.

[0100] In the case where the input parameters include user requirements, before S106 , the method may further include the following processing.

[0101] First, the recommendation information is sent to the user, and the recommendation information includes the target power supply strategy and the updated power grid state corresponding to the target power supply strategy, or the recommendation information is the power consumption information determined according to the updated power grid state. Among them, the power consumption information includes power consumption and the total electricity price. The power consumption may include: the total power consumption within a set time period, the specific power load and its power consumption (for example, the power consumption of the air conditioner) and the duration of power consumption. The total electricity price can be determined based on the power consumption and the electricity price.

[0102] Then, user feedback on the recommended information is obtained. In the case of the first user feedback, the target power supply strategy is executed; and / or, in the case of the second user feedback, the user requirements in the input parameters are updated according to the updated user requirements, and the target power supply strategy is re-selected from the multiple power supply strategies of the preset photovoltaic energy storage system according to the input parameters and the energy management model. Thus, a power supply strategy that meets the updated user requirements is obtained.

[0103] In various embodiments of the present application, the number of energy management models is at least one, and the number of target power supply strategies is at least one. In other words, input parameters can be input into multiple different energy management models to obtain different target power supply strategies, and then the different target power supply strategies can be input into the same state prediction model to obtain power grid states corresponding to the different target power supply strategies, and then the target power supply strategies and power grid states can be provided to the user for selection at the same time.

[0104] In various embodiments of the present application, for a home photovoltaic energy storage system, a user can manually select a power supply mode through an energy controller, or can use the method provided in the embodiments of the present application to select a power supply strategy from energy-saving strategy, backup power strategy, and peak-shifting strategy.

[0105] Among them, the energy-saving strategy focuses on energy saving, with photovoltaic power supply as the highest priority, battery power supply as the second priority, and grid power supply as the lowest priority. When photovoltaic power is strong, the photovoltaic power generation power can meet the consumption of household appliances and there is a surplus to charge the energy storage battery; when photovoltaic power is weakened and the power generation power does not meet the consumption of household appliances, the battery power supply can be started to start mixed power supply; when there is no photovoltaic power, battery power supply is prioritized. When the remaining SOC (State of Charge) of the battery is too low, the grid power supply is enabled to ensure that household appliances are not out of power.

[0106] The backup power strategy gives top priority to fully charging the energy storage battery. First, under the condition of photovoltaic power generation, when the photovoltaic power generation power is sufficient, photovoltaic power generation supplies power to household appliances while charging the energy storage battery; when photovoltaic power generation is insufficient, the grid mixed power supply starts, supplying power to household appliances and charging the energy storage battery at the same time; when there is no photovoltaic power generation, only the grid power supply is used, supplying power to household appliances and charging the energy storage battery at the same time; when the battery is fully charged, photovoltaic power generation continues if there is photovoltaic power generation, and grid power generation is used if there is no photovoltaic power generation, and the power in the energy storage battery is used as a backup power source. In the event of an emergency, when there is no photovoltaic power generation and no grid power generation, the energy storage battery can be turned on for power supply.

[0107] Peak-shifting strategy: if users often use electricity during peak hours, photovoltaic power supply will be turned on. When photovoltaic power is insufficient, battery power supply will be turned on to stagger the peak hours and save money. At the same time, it is also necessary to ensure that there is always backup power in the battery to avoid power outages in the user's home due to lack of photovoltaic power or city power.

[0108] Figure 3 is a flow chart of a photovoltaic energy storage system control method according to an embodiment of the present application, more specifically, a flow chart of a photovoltaic energy storage system control method based on deep reinforcement learning, with reference to Figure 3 , the method includes the following processing steps.

[0109] 1) Information collection.

[0110] The energy control manager can be used to obtain the high-frequency transient characteristics of the user's home power grid in the past: by identifying the power and voltage of the appliances over a period of time, the load status of the power grid and the relationship between the load size and time can be analyzed to obtain the user's power consumption habits. The energy manager can communicate with the Internet to obtain real-time electricity price information.

[0111] 2) The LSTM model creates the environment state.

[0112] Based on the grid load status, usage time and electricity price information obtained by the information collection part in the past period of time, the trained LSTM model is obtained. The household grid load status and electricity price information contained in a certain time t constitute the environment state st and are further input into the LSTM model, and the household grid load status and electricity price information at the time t+1 constitute the environment st+1. The information at the time t+1 is virtual information predicted by LSTM and will not affect the real environment.

[0113] 3) GPT model

[0114] The GPT model extracts corresponding logical information by analyzing the needs and habits of the user input, and converts the information into training parameters for deep reinforcement learning. For example, if a user inputs "I plan to turn on the water heater to take a shower in one hour, go to bed after taking a shower, turn the air conditioner to sleep mode before going to bed, and turn off the air conditioner when I go out at 8 o'clock tomorrow morning", the GPT model obtains the user's target needs through language analysis and user habits: water heater-temperature-air conditioner-sleep, go out-turn off the air conditioner and the corresponding usage time, and predicts the household power grid load that the user has not mentioned through LSTM: wash clothes after taking a shower-washing machine, sleep-air conditioner, which is divided into explicit needs and implicit needs. The user's needs, the corresponding appliance usage, and the current electricity price information are converted into a deep learning parameter environment.

[0115] Among them, the GPT model is a pre-trained model, and the training set may include:

[0116] Text language: Analyze and understand the needs input by users. Home grid load status: Analyze the home grid load, as well as the energy management strategy for battery and photovoltaic use, and analyze the home grid load status, usage habits and electricity price information returned by the LSTM model. Energy management strategy: Convert user needs into deep reinforcement learning training parameters for energy management strategies.

[0117] 4) Input parameters (including user requirements) into multiple deep reinforcement learning models to obtain different power supply strategies according to different deep reinforcement learning models.

[0118] 5) LSTM model predicts the results of the scheme operation. Specifically, the LSTM model predicts the scheme operation results corresponding to each power supply strategy based on the input power supply strategy and the current power grid status. The scheme operation results here can be the power grid status predicted by LSTM, or the power consumption information determined based on the power grid status predicted by LSTM.

[0119] 6) User feedback. If the user is satisfied, the deep reinforcement learning model that the user is satisfied with can be saved, the LSTM model can be updated to increase the prediction accuracy, and the home grid can be controlled according to the power supply strategy. If the user is not satisfied, the user can re-enter the demand, obtain the updated user demand, and return to the process of step 4.

[0120] For example, Figure 4 Schematic diagram of the training process of a deep reinforcement learning model according to an embodiment of the present application. Figure 4 Explain the training process of the deep reinforcement learning model.

[0121] In this embodiment, the learning and training of the deep reinforcement model can be branched by different deep reinforcement learning models with different strategies, for example: a backup power strategy model with energy saving as the behavior strategy and backup power as the strategy; a same strategy model with peak-shifting power consumption as the behavior strategy and target strategy. Figure 4 , the training process is as follows.

[0122] ① Based on user needs, a deep reinforcement learning model is used to simulate the load of the household power grid, the real-time electricity price, and the user's electricity usage habits to obtain an action a (i.e., the target power supply strategy). In this embodiment, LSTM can also be used to predict or analyze the user's electricity usage habits mentioned above.

[0123] ② Combine action a with the current environment st as the input of the LSTM model. LSTM predicts the grid load, real-time electricity price, and user electricity usage habits for a period of time t+1 in the future as state st+1.

[0124] ③ The deep reinforcement learning model scores the returned function reward R(st,a,st+1) with the target strategy π as the standard γ(γ∈[0,1]). The action value function is mainly based on the user's electricity usage habits. A deep learning model is established to grasp the user's electricity usage rules, which can be expressed as:

[0125] Qπ(st,at)=E[Rt+γ·Rt+1+γ2·Rt+2+...+γn·Rt+n|St=st,At=at]

[0126] Among them, Rt+n is the reward at time t+n predicted by the LSTM model based on the current state st and action a.

[0127] The target strategy π may be a weighted combination of one or more of the electric energy consumed by action a, the total electricity price based on the real-time electricity price, or the user's habit conformity (comfort).

[0128] ④ Then input st into the deep reinforcement learning model to execute the action at. The LSTM model predicts a value valuation qt and a new environment st+1. Then input the new environment st+1 into the deep reinforcement learning model to obtain a new action at+1 to generate a new valuation qt+1. It can be considered that the new environment st+1 is the target obtained based on the target strategy π, thus generating an error between the target and the current value:

[0129] L(w)=1 / 2[Q(st,at;w)-(rt+γ·Q(st+1,at+1;w))]2

[0130] Update the current model parameter w:

[0131]

[0132] The deep reinforcement learning model is trained based on the value score and behavior strategy of the target strategy π (which can be the same as or different from the target strategy; different deep reinforcement learning models have different target strategies and behavior rewards) to obtain multiple optimal value function action plans an.

[0133] Through the above processing, a trained deep reinforcement learning model can be obtained as the energy management model in the embodiment of the present application.

[0134] In various embodiments of the present application, the training of LSTM, GPT and deep reinforcement learning are all performed on the local energy management controller. After the training is completed, the model is saved in the local storage space.

[0135] Figure 5 is a block diagram of a control device for a photovoltaic energy storage system according to an embodiment of the present application. Figure 5 ,The control device of the photovoltaic energy storage system includes the following modules.

[0136] An input parameter acquisition module is used to acquire input parameters, which include the current grid state of the photovoltaic energy storage system and the user's electricity consumption habits, wherein the current grid state includes the current grid load and the current electricity price;

[0137] An energy management model, used to select a target power supply strategy from a plurality of power supply strategies of a preset photovoltaic energy storage system according to input parameters;

[0138] A state prediction model is used to predict the updated grid state after adopting the target power supply strategy based on the target power supply strategy and the current grid state;

[0139] The policy processing module is used to selectively execute the target power supply policy or update the target power supply policy in response to user feedback.

[0140] By adopting the control device provided in this embodiment, on the one hand, the target power supply strategy is determined according to the power grid status and the user's power usage habits, which is conducive to obtaining a power supply strategy that takes into account or balances the actual power usage status of the home power grid and the user's habits; on the other hand, the updated power grid status after adopting the target power supply strategy is predicted, which is conducive to presenting the possible power grid status corresponding to the target power supply strategy to the user, thereby providing the user with objective expectations; on the other hand, the target power supply strategy is selectively executed or updated in response to user feedback, so that the target power supply strategy can meet the user's personalized needs.

[0141] Optionally, in an implementation of this embodiment, the control device also includes: a recommendation module, used to send recommendation information to the user, the recommendation information includes a target power supply strategy and an updated power grid status corresponding to the target power supply strategy, or the recommended information is power consumption information determined based on the updated power grid status; a user feedback module, used to obtain user feedback on the recommended information.

[0142] For detailed descriptions of each module in this embodiment, please refer to the description in the previous method embodiment, which will not be repeated here.

[0143] An embodiment of the present application also provides an electronic device, which includes a memory and a processor, wherein the memory is used to store computer instructions; the processor is used to call computer instructions to implement the method provided by the embodiment of the present application (for example, a control method for a photovoltaic energy storage system, a model training method, etc.).

[0144] Exemplarily, the electronic device may be a chip, a circuit board with an integrated chip, or hardware including a circuit board. For example, the electronic device may be a user terminal device.

[0145] The embodiments of the present application also provide a photovoltaic energy storage system, which adopts the control method or model training method of the photovoltaic energy storage system provided in the embodiments of the present application, or the photovoltaic energy storage system is integrated with the electronic device provided in the embodiments of the present application, or the photovoltaic energy storage system has the control device provided in the embodiments of the present application.

[0146] The photovoltaic energy storage system provided in the embodiment of the present application can execute the control method provided in the embodiment through its energy controller.

[0147] In the above-mentioned embodiments of the present application, the description of each embodiment has its own emphasis. For parts that are not described in detail in a certain embodiment, please refer to the relevant description of other embodiments. The steps shown in the relevant flow chart can be executed in a computer system such as a set of computer executable instructions, and although the logical order is shown in the flow chart, in some cases, the steps shown or described can be executed in an order different from that here. In other words, the order of steps described in the foregoing embodiments is only an example, and it is also within the scope of protection of the embodiments of the present application to reasonably adjust the order of steps based on the content of the embodiments of the present application.

[0148] The sequence of serial numbers or introduction of the embodiments of the present application is for description only and does not represent the superiority or inferiority of the embodiments.

[0149] In the description of this specification, the description with reference to the terms "one embodiment", "some embodiments", "example", "specific example", or "some examples" etc. means that the specific features, structures, materials or characteristics described in conjunction with the embodiment or example are included in at least one embodiment or example of the present invention. In this specification, the schematic representations of the above terms do not necessarily refer to the same embodiment or example. Moreover, the specific features, structures, materials or characteristics described may be combined in any one or more embodiments or examples in a suitable manner. In addition, those skilled in the art may combine and combine different embodiments or examples described in this specification.

[0150] Although the embodiments of the present invention have been shown and described above, it is to be understood that the above embodiments are exemplary and are not to be construed as limitations of the present invention. A person skilled in the art may change, modify, replace and vary the above embodiments within the scope of the present invention.

Claims

1. A control method for a photovoltaic energy storage system, characterized in that: The method comprises: Acquire input parameters, wherein the input parameters include the current grid state of the photovoltaic energy storage system and the user's electricity usage habits, wherein the current grid state includes the current grid load and the current electricity price; According to the input parameters and the energy management model, a target power supply strategy is selected from a plurality of power supply strategies preset for the photovoltaic energy storage system; According to the target power supply strategy and the current power grid state, predicting an updated power grid state after adopting the target power supply strategy; In response to user feedback, the target power supply strategy is selectively executed or updated.

2. The control method according to claim 1, characterized in that: The current grid load condition includes at least one of the current total power consumption, the accumulated power consumption in the current period, the electrical appliances working in the current period, and the operating time of the photovoltaic energy storage system.

3. The control method according to claim 1, characterized in that: The energy management model is a deep reinforcement learning model, and generating a target power supply strategy according to the input parameters and the energy management model includes: The input parameters are input into the deep reinforcement learning model to obtain the target power supply strategy output by the deep reinforcement learning model, wherein the deep reinforcement learning model takes the input parameters as input and takes at least one of the multiple power supply strategies as output.

4. The air conditioning control method according to claim 3, characterized in that: The deep reinforcement learning model is trained in the following way: Obtaining a first value estimate for executing a first target power supply strategy under a power grid state at a first moment; Obtain a second value estimate for executing a second target power supply strategy under a power grid state at a second moment; Based on the first value estimate and the second value estimate, updating model parameters of the deep reinforcement learning model; Among them, the first target power supply strategy is the target power supply strategy obtained by the deep reinforcement learning model with the power grid state at the first moment and the user's power usage habits as input, the environmental state at the second moment is the power grid state predicted based on the power grid state at the first moment and the first target power supply strategy, and the second target power supply strategy is the target power supply strategy obtained by the deep reinforcement learning model with the power grid state at the second moment and the user's power usage habits as input.

5. The air conditioning control method according to claim 1, characterized in that: The predicting, according to the target power supply strategy and the current power grid state, an updated power grid state after adopting the target power supply strategy includes: The target power supply strategy and the current power grid state are input into a state prediction model to obtain the updated power grid state output by the state prediction model, wherein the state prediction model takes the target power supply strategy and the current power grid state as input and takes the updated power grid state as output.

6. The air conditioning control method according to claim 5, characterized in that: The training data used by the state prediction model during the training process includes: Grid load conditions and electricity prices during the target time period.

7. The air conditioning control method according to claim 1, characterized in that: The selectively executing the target power supply strategy or updating the target power supply strategy in response to user feedback includes: In case of feedback from the first user, executing the target power supply strategy; and / or, In the case of feedback from the second user, reselecting a target power supply strategy from a plurality of power supply strategies preset for the photovoltaic energy storage system according to the input parameters and the energy management model, or, In the case of second user feedback and the input parameters also including user needs, the user needs in the input parameters are updated according to the updated user needs, and then the target power supply strategy is re-selected from the preset multiple power supply strategies of the photovoltaic energy storage system according to the input parameters and the energy management model, and the second user feedback is different from the first user feedback.

8. The control method according to claim 1, characterized in that: The input parameters also include user requirements.

9. The control method according to claim 8, characterized in that: The method further comprises: The user demand is extracted from the demand input by the user based on the language model, and the form of the user demand meets the format requirement of the input parameter of the energy management model.

10. The air conditioning control method according to claim 1, characterized in that: The method further comprises: Sending recommendation information to the user, the recommendation information including the target power supply strategy and the updated power grid state corresponding to the target power supply strategy, or the recommendation information being power consumption information (power consumption, total power price) determined according to the updated power grid state; receiving user feedback regarding the recommendation information; Among them, the number of the energy management models is at least one, and the number of the target power supply strategies is at least one.

11. A control device for a photovoltaic energy storage system, characterized in that: The control device comprises: An input parameter acquisition module is used to acquire input parameters, wherein the input parameters include the current grid state of the photovoltaic energy storage system and the user's electricity usage habits, wherein the current grid state includes the current grid load situation and the current electricity price; An energy management model, used for selecting a target power supply strategy from a plurality of power supply strategies preset for the photovoltaic energy storage system according to the input parameters; A state prediction model, used to predict an updated power grid state after adopting the target power supply strategy according to the target power supply strategy and the current power grid state; The policy processing module is used to selectively execute the target power supply policy or update the target power supply policy in response to user feedback.

12. The control device according to claim 11, characterized in that: Also includes: A recommendation module, configured to send recommendation information to a user, wherein the recommendation information includes the target power supply strategy and the updated power grid state corresponding to the target power supply strategy, or the recommendation information is power consumption information determined according to the updated power grid state; The user feedback module is used to obtain the user feedback on the recommended information.

13. An electronic device, characterized in that: The electronic device comprises: Memory, for storing computer instructions; A processor, configured to call the computer instructions to implement the control method according to any one of claims 1 to 10.

14. A photovoltaic energy storage system, characterized in that: The air conditioner adopts the control method as described in any one of claims 1 to 10, or includes the air conditioner control device as described in claim 11 or 12, or includes the electronic device as described in claim 13.

Images