Light storage system energy-saving control method based on improved DT model

Abstract

The invention discloses a light storage system energy-saving control method based on an improved DT model, and belongs to the technical field of light storage system energy-saving control, and the method comprises the steps: obtaining the sequence data information of a light storage system, carrying out the preprocessing of the obtained data of the light storage system, and forming a triple with the state, return and action as the unit; introducing a return embedding mechanism into the DT model to optimize target return prediction; and inputting the generated triad taking the state, the return and the action as units into the improved DT model, and generating the optimal charge and discharge action prediction by the improved DT model in combination with the previous state, the return and the action. According to the method, the target return is embedded into the DT model, and the predicted value is optimized in the training process, so that the management performance of the photovoltaic power generation and energy storage system is improved, and the operation efficiency and stability of the system are improved.

Description

Technical Field

[0001] The present invention relates to the technical field of photovoltaic storage system energy-saving control, and in particular to a photovoltaic storage system energy-saving control method based on an improved DT model. Background Art

[0002] With the popularity of renewable energy, photovoltaic power generation systems have been widely used around the world. However, due to the intermittent and unstable nature of photovoltaic power generation, in order to ensure the continuity and stability of power supply, energy storage systems are usually required to store excess electricity and release it when needed. Traditional photovoltaic power generation and energy storage system management methods have certain limitations in prediction and decision-making, and are difficult to cope with complex and changing environmental conditions.

[0003] Sequence modeling is an important research direction in the field of machine learning and artificial intelligence, and is widely used in natural language processing, time series prediction, robot control, and reinforcement learning. The decision transformer model is an emerging method that converts sequence data into (reward, state, action) triplets and uses the Transformer architecture to model and predict future rewards and actions. However, the existing decision transformer models still need to improve their prediction accuracy and decision quality when dealing with complex sequence tasks.

[0004] Therefore, there is an urgent need for an energy-saving control method for a photovoltaic storage system based on an improved DT model to solve the above problems. Summary of the invention

[0005] The purpose of the present invention is to provide an energy-saving control method for a photovoltaic energy storage system based on an improved DT model, which improves the management performance of photovoltaic power generation and energy storage systems by embedding the target return into the DT model and optimizing its predicted value during the training process.

[0006] To achieve the above object, the present invention provides a method for energy-saving control of a photovoltaic storage system based on an improved DT model, the steps comprising:

[0007] S1, obtaining the sequence data information of the photovoltaic storage system, and preprocessing the obtained photovoltaic storage system data to form a triplet with state, feedback, and action as units;

[0008] S2. Improve the DT model by introducing a reward embedding mechanism into the DT model to optimize the target reward prediction;

[0009] S3. The generated triples in units of state, reward, and action are input into the improved DT model. The improved DT model predicts the optimal future action sequence by causally shielding the state and reward information. The action sequence dynamically adjusts the charging and discharging behavior of the system through sampling and feedback mechanisms to generate the optimal charging and discharging action prediction.

[0010] Preferably, the photovoltaic energy storage system sequence data in step S1 includes photovoltaic power generation status, energy storage status, weather data, timestamp information, current load demand and grid electricity price.

[0011] Preferably, the status information in step S1 includes photovoltaic power generation status, energy storage status, current load demand and grid electricity price; the reward is calculated by a specific energy-saving target, and the energy-saving target aims to minimize costs; the action is a charging and discharging decision, expressed as an action value, which controls the operation of the battery in the photovoltaic storage system.

[0012] Preferably, in step S2, introducing a reward embedding mechanism into the DT model to optimize the target reward prediction includes: adding the target reward as a training object to the loss function, the loss function includes a target reward prediction loss function and an action prediction loss function, the total loss function is a weighted sum of the target reward prediction loss function and the action prediction loss function, and the target reward prediction loss function is a mean square error, and its formula is:

[0013]

[0014] In the formula, R represents the return, represents a batch containing trajectories, τ represents the trajectory, represents the target return, represents the predicted return;

[0015] The action prediction loss function is the cross entropy loss function, and its formula is:

[0016]

[0017] In the formula, a represents the action, N represents the number of samples, B represents the batch size, i represents the i-th sample, k represents the k-th action category, and a true represents the real action label, a pred represents the predicted action distribution.

[0018] Preferably, step S3 specifically includes:

[0019] S31, inputting the generated triples in units of state, reward, and action into the improved DT model;

[0020] S32, improve the DT model to map data to the embedding layer through the linear layer;

[0021] S33, the embedding layer predicts the target return;

[0022] S34, improve the DT model to obtain action distribution based on the forward propagation of state and reward information;

[0023] S35, sampling to obtain action;

[0024] S36, optimizing the target return through an autoregressive process, wherein the optimization process is to subtract the current return from the target return to obtain the next target return;

[0025] S37. Repeat steps S32-S36 to dynamically adjust the charging and discharging behavior of the system until the optimal charging and discharging action is generated.

[0026] Therefore, the present invention adopts the above-mentioned energy-saving control method of the photovoltaic storage system based on the improved DT model, which has the following beneficial effects:

[0027] (1) Embed the target reward into the DT model and learn the optimal trajectory action guided by the target reward, so as to output high-level actions in the environment;

[0028] (2) The method of the present invention improves the system operation efficiency and stability by adjusting the operation of the photovoltaic power generation and energy storage system through feedback;

[0029] (3) The method of the present invention can enhance the model's ability to model long-term dependencies and has broad application prospects;

[0030] (4) The improved DT model has strong generalization performance and can be trained in one scenario when the power generation and energy storage scenarios are different, which facilitates the processing of complex sequence tasks.

[0031] The technical solution of the present invention is further described in detail below through the accompanying drawings and embodiments. BRIEF DESCRIPTION OF THE DRAWINGS

[0032] Figure 1 is a flow chart of a method according to an embodiment of the present invention;

[0033] Figure 2 This is a curve diagram of the improved DT model training loss function of an embodiment of the present invention. DETAILED DESCRIPTION

[0034] In order to make the purpose, technical solutions and advantages of the embodiments of the present invention clearer, the technical solutions in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, rather than all of the embodiments. The components of the embodiments of the present invention generally described and shown in the drawings here can be arranged and designed in various different configurations. Therefore, the following detailed description of the embodiments of the present invention provided in the drawings is not intended to limit the scope of the claimed invention, but merely represents selected embodiments of the present invention. Based on the embodiments in the present invention, all other embodiments obtained by ordinary technicians in this field without making creative work are within the scope of protection of the present invention.

[0035] Example

[0036] The present invention provides an energy-saving control method for a photovoltaic storage system based on an improved DT model. The photovoltaic storage system is controlled for energy saving through the improved DT model. The photovoltaic storage system is composed of a solar panel, a battery and a load. After obtaining electricity, the solar panel will give priority to supplying electricity to the load, and the excess electricity will be stored in the battery. The load is also connected to the power grid. When the power of the solar panel is not enough to maintain the normal operation of the load during the peak period of electricity consumption, the load will obtain electricity from the power grid or the battery. In order to minimize costs, the logic of the load obtaining electricity is set to obtain electricity from the battery during peak electricity prices and from the power grid during valley electricity prices. The specific acquisition behavior is controlled by the improved DT model, such as Figure 1 As shown, the steps include:

[0037] S1. Acquire the sequence data information of the photovoltaic storage system, and pre-process the acquired photovoltaic storage system data to form a triplet with state, feedback, and action as units.

[0038] Specifically: First, the sequence data of the photovoltaic storage system containing information such as photovoltaic power generation status, energy storage status, weather data, timestamp, current load demand and grid electricity price is obtained, and a simulation environment including photovoltaic power generation, energy storage, weather and other parameters is established for subsequent energy management and decision model control. In this environment, the power generated by the photovoltaic panels is directly supplied to the 5G base station, and the excess power will be stored in the battery and then used by the base station during the peak demand period. Then, the obtained photovoltaic storage system sequence data is preprocessed to form a triple with state, return and action as units. Among them, the state information includes photovoltaic power generation status, energy storage status, current load demand and grid electricity price; the return is calculated by the specific energy-saving target, and the energy-saving target is to minimize the cost; the action is the charging and discharging decision, which is expressed as the action value, defined as A = {1, -1}, meaning -1: the photovoltaic panel charges the battery, 1: the battery supplies power to the base station, and the improved DT model controls the operation of the battery in the photovoltaic storage system through the action value.

[0039] S2. Improve the DT model and introduce a reward embedding mechanism into the DT model to optimize the target reward prediction.

[0040] Specifically: The DT model uses the Transformer architecture to model and predict future rewards and actions. The target reward prediction is to add the target reward as an optimization item to the loss function during the pre-training of the DT model, so that the DT model does not need to manually set the target reward during reasoning, but the model automatically predicts the value according to the environment, which greatly reduces the target reward error when facing complex environments. For example, storing excess electricity in batteries during peak power generation, or obtaining electricity from batteries during peak power consumption, can well handle the management issues of photovoltaic power generation and energy storage systems in a variety of scenarios.

[0041] The loss function includes the target return prediction loss function and the action prediction loss function. The total loss function is the weighted sum of the target return prediction loss function and the action prediction loss function. The target return prediction loss function is the mean square error, and its formula is:

[0042]

[0043] In the formula, R represents the return, represents a batch containing trajectories, τ represents the trajectory, represents the target return, represents the predicted return;

[0044] The action prediction loss function is the cross entropy loss function, and its formula is:

[0045]

[0046] In the formula, a represents the action, N represents the number of samples, B represents the batch size, i represents the i-th sample, k represents the k-th action category, and a true represents the real action label, a pred represents the predicted action distribution.

[0047] S3. Input the generated triples in units of state, reward, and action into the improved DT model. The improved DT model predicts the best future action sequence by causally shielding the state and reward information. The action sequence dynamically adjusts the charging and discharging behavior of the system through sampling and feedback mechanisms to generate the best charging and discharging action prediction. The loss function results during training are as follows: Figure 2 The process of generating the optimal charge and discharge action prediction specifically includes:

[0048] S31, inputting the generated triples in units of state, reward, and action into the improved DT model;

[0049] S32, improve the DT model to map data to the embedding layer through the linear layer;

[0050] S33, the embedding layer predicts the target return;

[0051] S34, improve the DT model to obtain action distribution based on the forward propagation of state and reward information;

[0052] S35, sampling to obtain action;

[0053] S36, optimizing the target return through an autoregressive process, wherein the optimization process is to subtract the current return from the target return to obtain the next target return;

[0054] S37. Repeat steps S32-S36 to dynamically adjust the charging and discharging behavior of the system until the optimal charging and discharging action is generated.

[0055] Therefore, the present invention adopts the above-mentioned energy-saving control method of the photovoltaic storage system based on the improved DT model, by embedding the target return into the DT model and optimizing its predicted value during the training process, thereby improving the management performance of the photovoltaic power generation and energy storage system and improving the system operation efficiency and stability.

[0056] Finally, it should be noted that the above embodiments are only used to illustrate the technical solution of the present invention rather than to limit it. Although the present invention has been described in detail with reference to the preferred embodiments, those skilled in the art should understand that they can still modify or replace the technical solution of the present invention with equivalents, and these modifications or equivalent replacements cannot cause the modified technical solution to deviate from the spirit and scope of the technical solution of the present invention.

Claims

1. A method for energy-saving control of a photovoltaic storage system based on an improved DT model, characterized in that the steps include: S1, obtaining the sequence data information of the photovoltaic storage system, and preprocessing the obtained photovoltaic storage system data to form a triplet with state, feedback, and action as units; S2. Improve the DT model by introducing a reward embedding mechanism into the DT model to optimize the target reward prediction; S3. The generated triples in units of state, reward, and action are input into the improved DT model. The improved DT model predicts the optimal future action sequence by causally shielding the state and reward information. The action sequence dynamically adjusts the charging and discharging behavior of the system through sampling and feedback mechanisms to generate the optimal charging and discharging action prediction.

2. According to claim 1, a method for energy-saving control of a photovoltaic storage system based on an improved DT model is characterized in that: The photovoltaic energy storage system sequence data in step S1 includes photovoltaic power generation status, energy storage status, weather data, timestamp information, current load demand and grid electricity price.

3. The energy-saving control method of a photovoltaic storage system based on an improved DT model according to claim 2 is characterized in that: The status information in step S1 includes the photovoltaic power generation status, energy storage status, current load demand and grid electricity price; the reward is calculated by the specific energy-saving target, and the energy-saving target aims to minimize the cost; the action is the charging and discharging decision, which is expressed as the action value, which controls the operation of the battery in the photovoltaic storage system.

4. The energy-saving control method of a photovoltaic storage system based on an improved DT model according to claim 3 is characterized in that: In step S2, introducing a reward embedding mechanism into the DT model to optimize the target reward prediction includes: adding the target reward as a training object to the loss function, the loss function includes a target reward prediction loss function and an action prediction loss function, the total loss function is the weighted sum of the target reward prediction loss function and the action prediction loss function, and the target reward prediction loss function is the mean square error, and its formula is: In the formula, R represents the return, represents a batch containing trajectories, τ represents the trajectory, represents the target return, represents the predicted return; The action prediction loss function is the cross entropy loss function, and its formula is: In the formula, a represents the action, N represents the number of samples, B represents the batch size, i represents the i-th sample, k represents the k-th action category, and a true represents the real action label, a pred represents the predicted action distribution.

5. The energy-saving control method of a photovoltaic storage system based on an improved DT model according to claim 4 is characterized in that: Step S3 specifically includes: S31, inputting the generated triples in units of state, reward, and action into the improved DT model; S32, improve the DT model to map data to the embedding layer through the linear layer; S33, the embedding layer predicts the target return; S34, improve the DT model to obtain action distribution based on the forward propagation of state and reward information; S35, sampling to obtain action; S36, optimizing the target return through an autoregressive process, wherein the optimization process is to subtract the current return from the target return to obtain the next target return; S37. Repeat steps S32-S36 to dynamically adjust the charging and discharging behavior of the system until the optimal charging and discharging action is generated.

Images