Prediction and reinforcement learning multi-target-based source network load electricity storage quantity distribution method

Abstract

The invention provides a source network load electricity storage quantity distribution method based on multiple targets of prediction and reinforcement learning. The method comprises the following steps: firstly, by collecting and preprocessing multi-modal data, constructing a GRU prediction model based on an attention mechanism so as to improve the prediction precision of power demand and supply; then, designing a multi-objective optimization particle swarm resource allocation model based on reinforcement learning, and optimizing a resource allocation strategy by adaptively adjusting particle swarm parameters and introducing a reinforcement learning component; and finally, a resource allocation scheme is corrected and adjusted through a dynamic adjustment mechanism, and cost minimization and system operation efficiency maximization are realized while system constraints are ensured to be satisfied. According to the method, the response speed and the scheduling precision of the power system can be improved, the stability and the economical efficiency of power supply are ensured, the resource configuration is optimized, the operation cost is reduced, the adjustment capability of the load side can be improved, and medium-and-long-term time-phased transaction can be better carried out.

Description

Technical Field

[0001] The present invention belongs to the technical field of energy management in power trading, and in particular relates to a source-grid-load-storage power allocation method based on prediction and reinforcement learning multi-objectives. Background Art

[0002] In modern power systems, the effective allocation of source, grid, load and storage resources is crucial to ensure efficient use of energy and stable operation of the power grid. With the large-scale grid connection of renewable energy and the development of smart grid technology, the power system is becoming more and more complex. The source, grid, load and storage system involves multiple links such as power generation, transmission, power consumption and energy storage. It is necessary to comprehensively consider the volatility of the power generation side, the transmission limitations of the grid side, the demand changes on the load side and the regulation capacity of the energy storage side. Traditional power allocation methods mainly rely on empirical judgment and simple optimization algorithms. These methods often cannot accurately predict daily power demand, and it is difficult to cope with the volatility and uncertainty of renewable energy, resulting in inefficient power dispatching and waste of resources. Therefore, developing a method that can accurately predict daily power and dynamically optimize power resource allocation has become a research hotspot in the field of power systems.

[0003] In the existing technology, Chinese patent publication number CN117522054A discloses a multi-objective collaborative planning method for distribution network source, grid, load and storage. By continuously acquiring and processing data on both sides of the source and load, an optimization scheduling model is established, and load scheduling is performed using dynamic network reconstruction and real-time optimization control strategies. At the same time, the uncertainty model is converted into a deterministic expected value model, and a two-layer combined optimization algorithm is used to simplify the solution process to reduce network losses; Chinese patent publication number CN114884135A discloses a regional-level source, grid, load and storage day-ahead coordinated control method, which aims to obtain key information, build an optimization model, and solve it on the day before (D-1) to generate a resource operation curve with a granularity of 15 minutes for the next day, so as to achieve the goals of reducing the peak-to-valley difference of power grid load, optimizing the adjustment cost of adjustable resources and improving the economy of system operation. This method aims to smooth out peaks and fill valleys by coordinating various adjustable resources and conventional units, and improve the stability and economy of power grid operation; Chinese patent publication number CN114865715A discloses a real-time optimization scheduling method for source-grid-load-storage flexibility resources based on model predictive control (MPC), which minimizes the overall cost by building a real-time scheduling model and converting it into a quadratic programming problem, while taking into account the operating characteristics and constraints of source-grid-load-storage resources. This method can adapt to the dynamic update of source-load forecast information, and provide real-time scheduling strategies for thermal power units, renewable energy, demand response, real-time electricity prices, dynamic capacity expansion of transmission lines, and energy storage equipment.

[0004] The above technical solutions have three shortcomings: First, they rely on a large amount of accurate data. Complex models may lead to high consumption of computing resources and may be difficult to adapt to rapidly changing grid conditions. Second, they lack dynamic adjustment capabilities. Existing power distribution methods often lack the ability to quickly respond to real-time data and dynamically adjust, and cannot adapt to immediate changes and emergencies in the power market. Third, multi-objective optimization is insufficient. In the power distribution of source-grid-load-storage systems, multiple objectives such as cost, efficiency, reliability and environmental impact need to be considered simultaneously, while existing technologies can often only solve a single objective and it is difficult to achieve multi-objective optimization.

[0005] Based on the above shortcomings, the present invention proposes an intelligent allocation method for source-grid-load-storage power based on prediction and reinforcement learning multi-objective optimization, aiming to solve the problems of inaccurate power dispatching caused by inefficient power resource allocation and insufficient daily power planning in the existing source-grid-load-storage system, as well as the problems of power abandonment and insufficient utilization of load-side regulation capacity faced by new energy. Summary of the invention

[0006] The present invention mainly includes three parts: a multi-dimensional intelligent GRU prediction method based on attention mechanism, a multi-optimization target particle swarm resource allocation method based on reinforcement learning, and a short-term daily power allocation method.

[0007] To achieve the above object, a source-grid-load-storage power allocation method based on prediction and reinforcement learning multi-objectives of the present invention is as follows: A source-grid-load-storage power allocation method based on prediction and reinforcement learning multi-objectives, including: a multi-dimensional intelligent selection GRU prediction method based on attention mechanism, a multi-optimization objective particle swarm resource allocation method based on reinforcement learning, and a short-term daily power allocation method; The multi-dimensional intelligent GRU prediction method based on the attention mechanism realizes the re-extraction of data features and the prediction of future medium- and long-term electricity consumption for historical electricity consumption, special date data and weather data; The multi-optimization objective particle swarm resource allocation method based on reinforcement learning is to formulate an electricity allocation plan according to the monthly contract electricity of the source, grid, load and storage and the medium- and long-term electricity demand forecast, and allocate the monthly electricity demand to each day as needed to obtain the final baseline load allocation plan; The short-term daily electricity allocation method is guided by the baseline load allocation plan and takes into account the power abandonment curve data to achieve more refined load-side short-term daily electricity dispatching.

[0008] Furthermore, the multi-dimensional intelligent GRU prediction method based on the attention mechanism specifically includes the following steps: Step 1: Multimodal data collection: collecting medium- and long-term electricity demand forecast data through the power trading system; Step 2: Data preprocessing: using data preprocessing techniques to optimize multimodal data; Step 3: Data feature selection: feature selection is performed through recursive feature elimination, which belongs to the wrapping method. The wrapping method regards the feature selection process as a search problem and evaluates the performance of different feature subsets according to the objective function. Step 4: Use the attention mechanism to extract useful features again; The attention mechanism is mainly divided into two steps: Calculate the attention score, the formula is as follows: ,

[0009] in is the time step t The hidden state of and are the learnable weights and biases in the attention mechanism; is a learnable attention weight vector; Calculate the weighted context vector as follows: , Step 5: Use the GRU model to predict the future medium- and long-term electricity consumption; Update the gate, the formula is as follows: , in is the sigmoid activation function; is the input of the current time step; is the hidden state at the previous time step; and are the learnable weight and bias parameters in the update gate; Reset the gate, the formula is as follows: , in represents element-wise multiplication; and are the learnable weight parameters and bias parameters in the GRU model; represents the candidate hidden state; Combined with the context vector obtained previously To improve the model's ability to predict future electricity, the formula is as follows: , in is combined with the context vector and the final hidden state of the GRU at the last time step The new eigenvector of ; and Represents the learnable weight parameters and bias parameters in feature fusion; Indicates the final predicted medium- and long-term electricity demand data; and Represents the learnable weight parameters and bias parameters in the entire method.

[0010] Furthermore, the multi-optimization objective particle swarm resource allocation method based on reinforcement learning includes the following steps: Step 1: Initialize the particle swarm; According to the monthly contract power data and the medium- and long-term power forecast data, each particle is randomly assigned a position. and speed , set the inertia weight , learning factor And the number of iterations ; Initialization strategy, according to the monthly contract electricity, evenly distribute it to daily, adjust it in combination with the medium- and long-term forecast electricity, and generate the initial electricity allocation plan as the initial position of the particle; Monthly contract electricity data is expressed as: ,in Represents a power plant, Indicates the month; the medium and long-term electricity forecast data is expressed as: ,in Indicates a specific day; Step 2: Multi-objective optimization function; Contains the following: Cost minimization: Minimize the total operating cost of the source-grid-load-storage system, including power generation cost, power purchase cost and energy storage cost; , in and Represents the number of equipment for power generation, power purchase and energy storage respectively; and is the corresponding unit cost; and is the decision vector The power generation, power purchase and energy storage release under the above conditions, the decision vector contains all the decision variables that need to be optimized; Meeting load demand: ensuring that the source-grid-load-storage system can meet load demand at any time; , in It's time load demand; Improve energy efficiency: Maximize energy efficiency, which can be achieved by minimizing energy loss; , in is the efficiency of the generator; and are the charging and discharging efficiencies of the energy storage device, respectively; and They are the charging and discharging behaviors of the energy storage device; Reduce environmental pollution: Minimize carbon emissions from source-grid-load-storage systems; , in is the carbon emissions per unit of electricity generated; Step 3: Reinforcement learning components; It includes four steps: Stage 1: Initialization state; Step 2: Formulate actions. Actions can be defined as the strategies that particles choose when updating their speed and position. Step 3: Set rewards. The reward function is used to evaluate the effect of a particle taking an action. In multi-objective optimization, the reward is usually related to the objective function value and the degree of constraint violation. The reward function is defined as follows: , in It is The weight of each target; is the penalty factor; is the penalty function for violating the constraint; Step 4: Define the Q-table value function Q-table is used to store the value of each state-action pair, that is, the long-term benefit of taking a certain action in a certain state; the update formula of Q value is as follows: , in and Respectively represent the current state and the actions that can be taken in the current state; and represent the next state and the next state or possible actions to take considering future rewards, respectively; represents the learning rate, which determines how quickly new information overwrites the original estimate; represents the discount factor, which is used to measure the importance of future rewards relative to current rewards; It means to select an action that maximizes the Q value from all possible future actions; Step 4: Fitness function construction; The formula is as follows: , in and is the weight coefficient, which indicates the importance of different optimization objectives; Step 5: Particle update rules; Includes the following: Maintaining external archives: ; Update individual optimality and global optimality: The individual optimal update formula is as follows: ; The global optimal update formula is as follows: , in are all individual optimal solutions in the external archive; Third, leadership selection strategy; The formula is as follows: ; Fourth, speed and position update: The formula for speed update is as follows: , in It is a particle In time speed; The particle In time location; It is a particle The individual optimal position of is the global optimal position; and Is the interval Random numbers within The position update formula is as follows: ; Step 6: Dynamic adjustment of inertia weight; The inertia weight method adopts a dynamic adjustment method combined with reinforcement learning to optimize the inertia weight according to the historical behavior of particles and reward feedback; The formula is as follows: , in and Represents future iterations round and current iteration The inertia weight of the wheel; Indicates the current iteration Round of rewards; represents the average reward of all iterations; Step 7: Constraint processing; Check whether the current particle violates the constraints of the system. If the constraints are violated, optimize the particle according to the degree of violation. The constraints involving the physical constraints on the power generation side, the transmission constraints on the power grid, the physical constraints on the power consumption side, and the energy storage capacity are defined as follows: , in It's time Generator The output power, and are the minimum and maximum output constraints of the motor; , in It's time In the power grid The transmission power on and are the maximum and minimum transmission constraints of the line; , in It's time Load demand, and are the minimum and maximum demand constraints for the load; , in It's time Energy storage equipment The energy storage state, and are the lower and upper limits of the energy storage of the device; There are four penalty functions for the four constraints, and the formulas are as follows: , Total penalty function is the sum of all individual penalties, expressed as: ; Step 8: Iteration termination condition; If the preset number of iterations is reached The algorithm terminates; if the global optimal position of the particle swarm changes little or does not change in multiple consecutive iterations, the algorithm is considered to have converged and the iteration is terminated; Step 9: Output the baseline load distribution plan corresponding to the global optimum.

[0011] Furthermore, the short-term daily electricity allocation method adds the following objective function: First, minimize the deviation between daily power allocation and baseline load allocation. This goal is to ensure that the actual power allocation is as close as possible to the baseline load demand to improve the accuracy of scheduling. , in Indicates the time in the baseline load The formula ensures the accuracy of power distribution by calculating the absolute deviation between actual power supply and baseline load demand. Second, maximize the utilization of renewable energy and reduce power abandonment. This goal aims to increase the consumption of renewable energy and reduce power abandonment caused by insufficient load demand. , in Represents the time obtained according to the power curtailment curve of electricity generated from renewable energy sources; is the power curtailment penalty coefficient, which is used to encourage the reduction of power curtailment that may be caused by energy storage regulation; Third, additional constraints: ; Ensure that at any time, the result of subtracting the baseline load demand and the change in energy storage from the power generation of renewable energy is not less than zero, that is, no power abandonment is allowed; Fourth, the added penalty function: .

[0012] Furthermore, the data preprocessing includes three steps: Step 1: Data cleaning; It includes the processing of missing values ​​and outliers in numerical data, as well as the format unification and conversion of text data into numerical data; Step 2: Data standardization: Use maximum and minimum normalization to standardize data. The formula is as follows: ; in Represents the standardized data; Represents the multimodal data collected; express The minimum and maximum values ​​of Step 3: Data encoding; Convert non-numeric features to numeric features using One-Hot Encoding.

[0013] The source-grid-load-storage power intelligent allocation method based on prediction and reinforcement learning multi-objective optimization of the present invention has the following advantages: (1) Solve the problems of inaccurate power dispatching caused by inefficient power resource allocation and insufficient daily power planning in the existing source-grid-load-storage system, as well as the problems faced by new energy sources such as power abandonment and underutilization of load-side regulation capacity.

[0014] (2) It can improve the response speed and dispatch accuracy of the power system, realize the refinement and efficiency of power distribution, and facilitate the purchase and sale of electricity more reasonably in the process of source-grid-load-storage transactions between power generation and consumption. BRIEF DESCRIPTION OF THE DRAWINGS

[0015] Figure 1 This is a flowchart of the multi-dimensional intelligent GRU prediction method based on the attention mechanism.

[0016] Figure 2 The computational process of the particle swarm resource allocation method for multi-objective optimization based on reinforcement learning. DETAILED DESCRIPTION

[0017] In order to more clearly illustrate the purpose, structure and function of the present invention, the following is a further detailed description of a source-grid-load-storage power allocation method based on prediction and reinforcement learning multi-objectives of the present invention in conjunction with the accompanying drawings.

[0018] First, by introducing a multi-dimensional intelligent GRU prediction method based on an attention mechanism, the present invention aims to improve the prediction accuracy of power demand and energy supply, thereby providing more accurate data support for power distribution. Then, combined with reinforcement learning technology, the present invention can dynamically adjust the power distribution strategy according to real-time data and market changes, and improve the response speed and adaptability of the system. Then, the present invention adopts a multi-objective optimization particle swarm resource allocation method based on reinforcement learning, which can find the optimal balance between multiple goals of power system operation, achieve comprehensive optimization of cost, efficiency, reliability and environmental impact, and obtain baseline load data with guiding significance. Finally, the multi-objective optimization particle swarm resource allocation method based on reinforcement learning is used again, and the new energy abandonment curve data and baseline load data are used as new targets, and corresponding constraints are formulated to obtain the final short-term daily power distribution plan. The load side gives full play to its own regulation ability according to the daily power distribution plan.

[0019] The multi-dimensional smart GRU prediction method based on the attention mechanism mainly targets multimodal data such as historical electricity consumption, special date data, and weather data. First, the data is preprocessed, and then the most useful features for the medium- and long-term electricity forecasting task are retained to ensure data quality. Finally, a GRU prediction model based on the attention mechanism is constructed to achieve the re-extraction of data features and the prediction of future medium- and long-term electricity. The overall process of the multi-dimensional smart GRU prediction method based on the attention mechanism is as follows: Figure 1 shown.

[0020] Step 1. Multimodal data collection The power trading system collects medium- and long-term electricity demand forecast data. This data includes historical power generation, historical electricity consumption, and special dates of each month. Weather data is particularly important for predicting electricity demand because factors such as temperature, precipitation, wind speed, and sunlight can affect electricity consumption. Weather data can be obtained through meteorological service providers.

[0021] Step 2. Data preprocessing In order to ensure the availability and quality of collected data and improve the prediction ability, data preprocessing technology is used to optimize multimodal data. Data preprocessing is mainly divided into three steps: data cleaning, data standardization and data encoding.

[0022] 1) Data cleaning It mainly includes the processing of missing values ​​and outliers in numerical data, as well as the format unification and conversion of text data into numerical data, which can ensure the quality and consistency of data and provide a solid foundation for subsequent data prediction tasks.

[0023] 2) Data Standardization Since different features may have different dimensions and numerical ranges, data normalization can not only eliminate the influence of these dimensions, so that different features can be compared fairly, but also improve the prediction effect and convergence speed of the model. In this invention, maximum and minimum normalization is used to perform data normalization, and the formula is as follows: , in Represents the standardized data; Represents the multimodal data collected; express The minimum and maximum values ​​of .

[0024] 3) Data encoding The present invention converts non-numeric features into numerical features using One-Hot Encoding to facilitate model processing.

[0025] Step 3. Data feature selection Multimodal data has many features, not all of which are forward propagated for subsequent prediction tasks. It is often necessary to select from a large number of features and pick out the features that are most helpful for model learning from the original feature set to improve the model's predictive ability.

[0026] The significance of feature selection is to improve the performance of the model. By screening out the features most relevant to the prediction target, it can reduce the complexity of the model and avoid overfitting, thereby improving the generalization ability of the model. In addition, feature selection can also help reduce the computational cost of model training and prediction, improve the operating efficiency of the model, and enhance the interpretability of the model, making the decision-making process of the model more transparent and easy to understand. Feature selection is particularly important in high-dimensional data sets. It can help identify and eliminate noise and redundant information, ensuring that the model focuses on the most valuable data.

[0027] The present invention selects recursive feature elimination (RFE) belonging to the wrapper method for feature selection. The wrapper method regards the feature selection process as a search problem and evaluates the performance of different feature subsets according to the objective function (such as model accuracy). The core idea of ​​RFE is to use a base model (such as support vector machine, decision tree, neural network, etc.) to evaluate the importance of each feature and gradually remove the least important features until the predetermined number of features is reached or a certain stopping condition is met.

[0028] Step 4. Use the attention mechanism to extract useful features again Since the GRU model may still ignore some important short-term patterns when processing very long sequences, the use of the attention mechanism allows the model to dynamically adjust the degree of attention to historical power information at different time steps. In this way, the model can not only remember long-term trends, but also respond quickly to recent fluctuations, thereby improving the model's ability to predict future power. The attention mechanism is mainly divided into two steps.

[0029] Calculate the attention score: First, the model needs to calculate a score for each time step in the sequence, which indicates the importance of the time step to the current power prediction. This is usually achieved through a learnable weight vector and a score function, as follows: , in is the time step The hidden state of are the learnable weights and biases in the attention mechanism; is a learnable attention weight vector.

[0030] Calculate weighted context vector: The model then uses these attention scores to calculate a weighted context vector, which is a weighted sum of the hidden states at all time steps in the sequence, using the following formula: , Step 5. Use the GRU model to predict future mid- to long-term electricity consumption GRU can effectively capture long-term dependencies in multimodal data, which is particularly important for predicting future mid- to long-term power demand. Through its unique gating mechanism, GRU can learn which information is important and should be retained, and which information is outdated and should be forgotten.

[0031] Update gate: determines how much past information needs to be retained and how much new information needs to be incorporated into the current state. The formula is as follows: , in is the sigmoid activation function; is the input of the current time step; is the hidden state at the previous time step; are the learnable weight and bias parameters in the update gate.

[0032] Reset gate: determines how much past information needs to be forgotten. The formula is as follows: , in are the learnable weight and bias parameters in the reset gate.

[0033] Final hidden state: Combined with the result of the update gate, the candidate hidden state (the result adjusted by the reset gate combined with the current input and the hidden state of the previous time step) is weighted averaged with the hidden state of the previous time step. The formula is as follows: , in represents element-wise multiplication; are the learnable weight parameters and bias parameters in the GRU model; represents a candidate hidden state.

[0034] Since the attention mechanism is added to the GRU model to obtain a weighted context vector, in order to enhance the GRU model's ability to capture key information in the column data, the final hidden state of the last time step is obtained. After that, combined with the previously obtained context vector To improve the model's ability to predict future electricity, the formula is as follows: , in is combined with the context vector and the final hidden state of the GRU at the last time step The new eigenvector of ; and Represents the learnable weight parameters and bias parameters in feature fusion; It is the final forecast of medium and long-term electricity demand data; Represents the learnable weight parameters and bias parameters in the entire method.

[0035] Taking into account the volatility of electricity demand and the actual execution situation, the present invention proposes a multi-objective particle swarm optimization (MOPSO) resource allocation method based on reinforcement learning to formulate an electricity allocation plan according to the monthly contract electricity of source, grid, load and storage and the medium- and long-term electricity demand forecast, and allocate the monthly electricity demand to each day as needed, so as to obtain a baseline load allocation plan with guiding significance.

[0036] First, the definition of multi-objective problem is established, and a quantitative standard is provided for each objective by designing a fitness function. Then, by initializing the particle swarm and ensuring its diversity, the foundation is laid for exploring a wide range of potential solutions. Further, reinforcement learning components are introduced, including state representation, action space and reward function, which provide particles with decision support based on environmental feedback. In the particle swarm optimization process, not only the update rule of MOPSO is followed, but also the reinforcement learning strategy is integrated to dynamically adjust the search behavior of particles. In terms of multi-objective processing, the Pareto front is used to identify and store effective non-inferior solutions, and the leader selection strategy is adopted to promote the group's exploration of new potential areas. By iteratively updating the position, velocity and Pareto front of the particles, the solution set is continuously optimized until the preset termination condition is met, and the optimal baseline load distribution plan is obtained. The parameter adjustment in the present invention adopts adaptive adjustment technology.

[0037] The calculation process of the multi-objective optimization particle swarm resource allocation method based on reinforcement learning is shown in Figure 2 .

[0038] Step 1. Initialize the particle swarm Before initializing the particle swarm, it is necessary to obtain monthly contract power data and medium- and long-term power forecast data. Monthly contract power data is expressed as: ( Represents a power plant, Indicates month); medium- and long-term electricity forecast data is expressed as: ( Indicates a specific day).

[0039] Then each particle is randomly assigned a position and speed , locations represent different potential baseline load distribution plans. Set the inertia weight (controls the conservatism of particle speed), learning factor (controls the tendency of particles to move toward individual optimal and global optimal) and the number of iterations Finally, the strategy is initialized, and the monthly contract electricity is evenly distributed to each day, and the medium- and long-term forecast electricity is adjusted to generate the initial electricity allocation plan as the initial position of the particle.

[0040] Step 2. Multi-objective optimization function Multi-Objective Optimization Problems (MOPs) refer to the consideration of multiple objective functions during the optimization process, which are often conflicting or contradictory. For example, in the coordinated optimization of sources, grids, loads and storage, it is necessary to balance the goals of power generation costs, meet load demand, improve energy efficiency, and reduce environmental pollution in order to achieve the overall optimization of the power system.

[0041] 1) Cost minimization: Minimize the total operating cost of the source-grid-load-storage system, including power generation cost, power purchase cost and energy storage cost.

[0042] (9) in and Represents the number of equipment for power generation, power purchase and energy storage respectively; and is the corresponding unit cost; is the decision vector The decision vector contains all the decision variables that need to be optimized, including the power generation, power purchase and energy storage release under the current situation.

[0043] 2) Meet load demand: Ensure that the source-grid-load-storage system can meet load demand at any time.

[0044] (10) in It's time load demand.

[0045] 3) Improve energy efficiency: Maximizing energy efficiency can be achieved by minimizing energy loss.

[0046] (11) in is the efficiency of the generator; are the charging and discharging efficiencies of the energy storage device, respectively; They are the charging and discharging behaviors of the energy storage device, respectively.

[0047] 4) Reduce environmental pollution: Minimize carbon emissions from source-grid-load-storage systems.

[0048] (12) in It is the carbon emission corresponding to unit electricity generation.

[0049] Step 3. Reinforcement Learning Component The reinforcement learning component is responsible for dynamically adjusting the behavior strategy of particles to better explore the solution space and meet actual constraints. It is divided into the following four steps: 1) Initialization state The initialization of the state is the same as the initialization of the particle swarm mentioned above, which is about the position and velocity of the particles, as well as other environmental information that may affect the decision, such as the current number of iterations, the discovered Pareto front, etc.

[0050] 2) Formulate actions Actions can be defined as the strategies chosen by particles when updating their velocities and positions, such as choosing which individual optimum or global optimum to use as a reference point, or the optimization measures to take when a constraint is violated.

[0051] 3) Set up rewards The reward function is used to evaluate the effect of a particle taking an action. In multi-objective optimization, the reward is usually related to the objective function value and the degree of constraint violation. The reward function is defined as follows: (13) in It is The weight of each target; is the penalty factor; is the penalty function for violating the constraint.

[0052] 4) Define the Q-table value function Q-table is used to store the value of each state-action pair (expected utility Q value), that is, the long-term benefit of taking a certain action in a certain state. The update formula of Q value is as follows: (14) in Respectively represent the current state and the actions that can be taken in the current state; represent the next state and the next state or possible actions to take considering future rewards, respectively; represents the learning rate, which determines how quickly new information overwrites the original estimate; represents the discount factor, which is used to measure the importance of future rewards relative to current rewards; represents the selection of a possible future action from all possible future actions such that The action with the highest value.

[0053] Step 4. Fitness function construction The fitness function of the particle is closely related to the above optimization objective and is mainly used to evaluate the quality of the particle. Combined with the optimization objective function of the source-grid-load-storage system, the comfort function of the present invention is equivalent to a multi-objective optimization problem, which is expressed as follows: (15) in and is the weight coefficient, which indicates the importance of different optimization objectives.

[0054] Step 5. Particle update rules The update rules of particles are mainly for multi-objective processing, including maintaining external archives, updating individual optimal and global optimal, leader selection strategy, and updating particle speed and position. The update process is accompanied by the aforementioned reinforcement learning to optimize the leader selection strategy or update rules, so that particles can explore and utilize the Pareto frontier more effectively.

[0055] 1) Maintain an external archive: usually an archive that stores the currently found non-dominated solutions. This archive simulates the Pareto front of the entire population. Pareto dominance is defined as follows: For two solutions , if for all targets No worse than the Good, then Dominate .

[0056] Dominate (16) 2) Update individual optimal and global optimal solutions: Each particle updates its individual optimal solution based on its own experience , and the global optimal solution Can select from external archives.

[0057] The individual optimal update formula is as follows: (17) The global optimal update formula is as follows: (18) in are all the individual optimal solutions in the external archive. The global optimal update process involves adding the newly discovered non-dominated solutions to the external archive and removing the dominated solutions.

[0058] 3) Leader selection strategy: It determines the global optimal solution that particles refer to when updating speed and position. It can be one or more. The present invention selects a leader selection strategy based on fitness. The formula is as follows: (19).

[0059] 4) Speed ​​and position update: The speed and position update of particles takes into account the individual optimum and the selected leader (global optimal solution).

[0060] The formula for velocity update is as follows: (20) in It is a particle In time speed; The particle In time location; It is a particle The individual optimal position of is the global optimal position; Is the interval The random numbers in the algorithm introduce randomness into the algorithm, thereby helping particles to escape from the local optimal solution and increasing the algorithm's exploration and global search capabilities.

[0061] The position update formula is as follows: (twenty one) Step 6. Dynamic adjustment of inertia weight Inertia weight can measure the degree to which particles maintain their original speed. A larger inertia weight is conducive to global search, but the search efficiency is low; a smaller inertia weight is conducive to local search, but it is easy to fall into the local optimal solution. In order to make the population always maintain the diversity of inertia weights, avoid falling into the local optimum, and improve the search efficiency, the inertia weight method of the present invention adopts a dynamic adjustment method combined with reinforcement learning to optimize the inertia weight according to the historical behavior of particles and reward feedback. The formula is as follows: (twenty two) in Represents future iterations round and current iteration The inertia weight of the wheel; Indicates the current iteration Round of rewards; represents the average reward over all iterations.

[0062] Step 7. Constraint processing Check whether the current particle violates the constraints of the system. If the constraints are violated, optimize the particle according to the degree of violation. It is mainly about the physical limitations of the power generation side, the transmission limitations of the power grid, the physical limitations of the power consumption side and the energy storage capacity limitations. The constraints of the four parts are defined as follows: (twenty three) in It's time Generator The output power, are the minimum and maximum output constraints of the motor.

[0063] (twenty four) in It's time In the power grid The transmission power on are the maximum and minimum transmission constraints of the line.

[0064] (25) in It's time load demand, are the minimum and maximum demand constraints for this load.

[0065] (26) in It's time Energy storage equipment The energy storage state, are the lower and upper limits of the energy storage of the device.

[0066] According to the constraints described above, the penalty function in reinforcement learning is designed to guide the search process to avoid infeasible solutions. First, there are four penalty functions for the four constraints, and the formulas are as follows:

[0067] in , ensuring that penalties are only calculated for constraint violations, and then squaring that value to amplify the impact of violations.

[0068] Total penalty function is the sum of all individual penalty points and can be expressed as: (31) Step 8. Iteration termination condition If the preset number of iterations is reached , the algorithm terminates; if the global optimal position of the particle swarm changes little or does not change in multiple consecutive iterations, the algorithm is considered to have converged and the iteration is terminated.

[0069] Step 9. Output the baseline load distribution plan corresponding to the global optimum.

[0070] The short-term daily electricity allocation guidance method for the load side in the source-grid-load-storage system mainly serves four core goals: improving the level of new energy consumption, reducing the electricity cost of adjustable loads, stimulating the flexible adjustment ability of the user side, and promoting the coordinated efforts of the source-grid-load-storage system. By effectively utilizing the abandoned wind and solar resources, this method not only improves the utilization rate of new energy, but also provides users participating in the market with the opportunity to obtain low-priced electricity, thereby reducing their electricity costs. At the same time, it encourages users to flexibly adjust their electricity consumption behavior according to market signals, enhancing the overall flexibility of the power grid. This mechanism closely links the power generation side and the user side, achieving the synergistic effect of "ensuring supply and promoting consumption", and making an important contribution to building a more efficient and flexible power system.

[0071] Specifically, the core of this method is to use a multi-objective optimization particle swarm resource allocation method based on reinforcement learning to achieve more refined short-term daily power dispatch on the load side, based on the guidance of the baseline load distribution plan and taking into account the power abandonment curve data. Because the main purpose of this section is to achieve short-term daily power distribution, the multiple objectives, constraints and penalty functions in the particle swarm optimization algorithm are increased compared to the multi-objective optimization particle swarm resource allocation method based on reinforcement learning (short-term multi-objective functions, constraints and penalty functions should include medium- and long-term multi-objective functions, constraints and penalty functions), and the corresponding settings are as follows.

[0072] Added objective function: (1) Minimize the deviation between daily power allocation and baseline load allocation. This goal aims to ensure that the actual power allocation is as close as possible to the baseline load demand to improve the accuracy of scheduling.

[0073] (32) in Indicates the time in the baseline load The formula ensures the accuracy of power distribution by calculating the absolute deviation between actual power supply and baseline load demand.

[0074] (2) Maximize the utilization rate of renewable energy and reduce power curtailment. This goal aims to increase the consumption of renewable energy and reduce power curtailment caused by insufficient load demand.

[0075] (33) in Represents the time obtained according to the power curtailment curve of electricity generated from renewable energy sources; It is the power curtailment penalty coefficient, which is used to encourage the reduction of power curtailment that may be caused by energy storage regulation.

[0076] In both objective functions, the two consecutive states of the energy storage device are considered. and , the purpose is to cope with the intermittent and uncertain nature of renewable energy, such as fluctuations in wind and solar power. By analyzing the energy storage status at consecutive moments, it is possible to better dispatch energy storage resources and develop more accurate charging and discharging plans to minimize operating costs and improve energy efficiency, smooth the output of renewable energy, and reduce power abandonment.

[0077] Added constraints: (34) Ensure that at any time, the result of subtracting the baseline load demand and the change in energy storage from the power generation of renewable energy is not less than zero, that is, no power abandonment is allowed.

[0078] Added penalty function: (35) In the short-term daily electricity allocation process, this method not only considers the optimization objectives in Section 4.2, such as minimizing costs, meeting load demands, improving energy efficiency, and reducing environmental pollution, but also specifically adds the goal of minimizing the deviation between daily electricity allocation and baseline load allocation, as well as the goal of maximizing the utilization rate of renewable energy. The addition of these objectives makes the short-term daily electricity allocation plan more accurate, better adapts to the volatility and uncertainty of renewable energy, and makes full use of the power abandonment curve data.

[0079] The daily electricity allocation plan of the load side is finally obtained through the multi-objective optimization particle swarm algorithm based on reinforcement learning. The load side can adjust its own electricity consumption capacity, transaction contracts, etc., and when low electricity prices appear during power abandonment, it can also adjust according to the allocation plan in time. This will not only help to better carry out medium- and long-term time-sharing transactions and form time-sharing quantity and price contracts, but also reduce transaction costs and less deviation assessment.

[0080] Aiming at the problems of inaccurate power dispatching caused by low efficiency of source-grid-load-storage resource allocation and insufficient daily power planning in power systems, as well as the problems of power abandonment and underutilization of load-side regulation capacity faced by new energy, a source-grid-load-storage power allocation method based on prediction and reinforcement learning multi-objective is proposed. Firstly, by collecting and preprocessing multimodal data, a GRU prediction model based on attention mechanism is constructed to improve the prediction accuracy of power demand and supply; then, a multi-objective optimization particle swarm resource allocation model based on reinforcement learning is designed, and the resource allocation strategy is optimized by adaptively adjusting the particle swarm parameters and introducing reinforcement learning components; finally, the resource allocation scheme is corrected and adjusted through a dynamic adjustment mechanism to ensure that the cost is minimized and the system operation efficiency is maximized while meeting the system constraints. This method can not only improve the response speed and dispatching accuracy of the power system, ensure the stability and economy of power supply, but also optimize resource allocation, reduce operating costs, and improve the regulation capacity of the load side, and assist in better medium- and long-term time-sharing transactions.

[0081] The above contents are further detailed descriptions of the present invention in combination with specific preferred embodiments, and it cannot be determined that the specific implementation of the present invention is limited to these descriptions. For ordinary technicians in the field, corresponding changes and substitutions can be made based on the technical methods and inventive concepts of the present invention, and the same performance or use should be regarded as the protection scope of the present invention.

Claims

1. A source-grid-load-storage power allocation method based on prediction and reinforcement learning multi-objectives, characterized in that: include: Multi-dimensional intelligent GRU prediction method based on attention mechanism, multi-optimization target particle swarm resource allocation method based on reinforcement learning, and short-term daily power allocation method; The multi-dimensional intelligent GRU prediction method based on the attention mechanism realizes the re-extraction of data features and the prediction of future medium- and long-term electricity consumption for historical electricity consumption, special date data and weather data; The multi-optimization objective particle swarm resource allocation method based on reinforcement learning is to formulate an electricity allocation plan according to the monthly contract electricity of the source, grid, load and storage and the medium- and long-term electricity demand forecast, and allocate the monthly electricity demand to each day as needed to obtain the final baseline load allocation plan; The short-term daily electricity allocation method is guided by the baseline load allocation plan and takes into account the power abandonment curve data to achieve more refined load-side short-term daily electricity dispatching.

2. The source-grid-load-storage power allocation method based on prediction and reinforcement learning multi-objectives according to claim 1 is characterized in that: The multi-dimensional intelligent GRU prediction method based on the attention mechanism specifically includes the following steps: Step 1: Multimodal data collection: collecting medium- and long-term electricity demand forecast data through the power trading system; Step 2: Data preprocessing: using data preprocessing techniques to optimize multimodal data; Step 3: Data feature selection: feature selection is performed through recursive feature elimination, which belongs to the wrapping method. The wrapping method regards the feature selection process as a search problem and evaluates the performance of different feature subsets according to the objective function. Step 4: Use the attention mechanism to extract useful features again; The attention mechanism is mainly divided into two steps: Calculate the attention score, the formula is as follows: , in is the time step The hidden state of are the learnable weights and biases in the attention mechanism; is a learnable attention weight vector; Calculate the weighted context vector as follows: , Step 5: Use the GRU model to predict the future medium and long-term electricity consumption; Update the gate, the formula is as follows: , in is the sigmoid activation function; is the input of the current time step; is the hidden state at the previous time step; , are the learnable weight and bias parameters in the update gate; Reset the gate, the formula is as follows: , in represents element-wise multiplication; are the learnable weight parameters and bias parameters in the GRU model; represents the candidate hidden state; Combined with the context vector obtained previously To improve the model's ability to predict future electricity, the formula is as follows: , in is combined with the context vector and the final hidden state of the GRU at the last time step The new eigenvector of ; Represents the learnable weight parameters and bias parameters in feature fusion; Indicates the final predicted medium- and long-term electricity demand data; Represents the learnable weight parameters and bias parameters in the entire method.

3. The source-grid-load-storage power allocation method based on prediction and reinforcement learning multi-objectives according to claim 2 is characterized in that: The multi-optimization target particle swarm resource allocation method based on reinforcement learning comprises the following steps: Step 1: Initialize the particle swarm; According to the monthly contract power data and the medium- and long-term power forecast data, each particle is randomly assigned a position. and speed , set the inertia weight , learning factor And the number of iterations ; Initialization strategy, according to the monthly contract electricity, evenly distribute it to daily, adjust it in combination with the medium- and long-term forecast electricity, and generate the initial electricity allocation plan as the initial position of the particle; Monthly contract electricity data is expressed as: in Represents a power plant, Indicates the month; the medium and long-term electricity forecast data is expressed as: in Indicates a specific day; Step 2: Multi-objective optimization function; Contains the following: Cost minimization: Minimize the total operating cost of the source-grid-load-storage system, including power generation cost, power purchase cost and energy storage cost; , in Represents the number of equipment for power generation, power purchase and energy storage respectively; is the corresponding unit cost; is the decision vector The power generation, power purchase and energy storage release under the above conditions, the decision vector contains all the decision variables that need to be optimized; Meeting load demand: ensuring that the source-grid-load-storage system can meet load demand at any time; , in It's time load demand; Improve energy efficiency: Maximize energy efficiency, which can be achieved by minimizing energy loss; , in is the efficiency of the generator; are the charging and discharging efficiencies of the energy storage device, respectively; and They are the charging and discharging behaviors of the energy storage device; Reduce environmental pollution: Minimize carbon emissions from source-grid-load-storage systems; , in is the carbon emissions per unit of electricity generated; Step 3: Reinforcement learning components; It includes four steps: Stage 1: Initialization state; Step 2: Formulate actions. Actions can be defined as the strategies that particles choose when updating their speed and position. Step 3: Set rewards. The reward function is used to evaluate the effect of a particle taking an action. In multi-objective optimization, the reward is usually related to the objective function value and the degree of constraint violation. The reward function is defined as follows: , in It is The weight of each target; is the penalty factor; is the penalty function for violating the constraint; Step 4: Define the Q-table value function Q-table is used to store the value of each state-action pair, that is, the long-term benefit of taking a certain action in a certain state; the update formula of Q value is as follows: , in Respectively represent the current state and the actions that can be taken in the current state; represent the next state and the next state or possible actions to take considering future rewards, respectively; represents the learning rate, which determines how quickly new information overwrites the original estimate; represents the discount factor, which is used to measure the importance of future rewards relative to current rewards; It means to select an action that maximizes the Q value from all possible future actions; Step 4: Fitness function construction; The formula is as follows: , in and is the weight coefficient, which indicates the importance of different optimization objectives; Step 5: Particle update rules; Includes the following: Maintaining external archives: , Update individual optimality and global optimality: The individual optimal update formula is as follows: , The global optimal update formula is as follows: , in are all individual optimal solutions in the external archive; Third, leadership selection strategy; The formula is as follows: ， Fourth, speed and position update: The formula for speed update is as follows: ， in It is a particle In time speed; It is a particle In time location; It is a particle The individual optimal position of is the global optimal position; Is the interval Random numbers within The position update formula is as follows: ； Step 6: Dynamic adjustment of inertia weight; The inertia weight method adopts a dynamic adjustment method combined with reinforcement learning to optimize the inertia weight according to the historical behavior of particles and reward feedback; The formula is as follows: ， in and Represents future iterations round and current iteration The inertia weight of the wheel; Indicates the current iteration Round of rewards; represents the average reward of all iterations; Step 7: Constraint processing; Check whether the current particle violates the constraints of the system. If the constraints are violated, optimize the particle according to the degree of violation. The constraints involving the physical constraints on the power generation side, the transmission constraints on the power grid, the physical constraints on the power consumption side, and the energy storage capacity are defined as follows: ， in It's time Generator The output power, are the minimum and maximum output constraints of the motor; ， in It's time In the power grid The transmission power, are the maximum and minimum transmission constraints of the line; ， in It's time Load demand, are the minimum and maximum demand constraints for the load; ， in It's time Energy storage equipment The energy storage state, are the lower and upper limits of the energy storage of the device; There are four penalty functions for the four constraints, and the formulas are as follows: ， Total penalty function is the sum of all individual penalties, expressed as: ； Step 8: Iteration termination condition; If the preset number of iterations is reached , the algorithm terminates; if the global optimal position of the particle swarm changes little or does not change in multiple consecutive iterations, the algorithm is considered to have converged and the iteration is terminated; Step 9: Output the baseline load distribution plan corresponding to the global optimum.

4. The source-grid-load-storage power allocation method based on prediction and reinforcement learning multi-objectives according to claim 3 is characterized in that: The short-term daily power allocation method adds the following objective function: First, minimize the deviation between daily power allocation and baseline load allocation. This goal is to ensure that the actual power allocation is as close as possible to the baseline load demand to improve the accuracy of scheduling. ， in Indicates the time in the baseline load The formula ensures the accuracy of power distribution by calculating the absolute deviation between actual power supply and baseline load demand. Second, maximize the utilization of renewable energy and reduce power abandonment. This goal aims to increase the consumption of renewable energy and reduce power abandonment caused by insufficient load demand. ， in Represents the time obtained according to the power curtailment curve of electricity generated from renewable energy sources; is the power curtailment penalty coefficient, which is used to encourage the reduction of power curtailment that may be caused by energy storage regulation; Third, additional constraints: ; Ensure that at any time, the result of subtracting the baseline load demand and the change in energy storage from the power generation of renewable energy is not less than zero, that is, no power abandonment is allowed; Fourth, the added penalty function: 。 5. The source-grid-load-storage power allocation method based on prediction and reinforcement learning multi-objectives according to claim 2 is characterized in that: The data preprocessing includes three steps: Step 1: Data cleaning; It includes the processing of missing values ​​and outliers in numerical data, as well as the format unification and conversion of text data into numerical data; Step 2: Data standardization: Use maximum and minimum normalization to standardize data. The formula is as follows: ， in Represents the standardized data; Represents the multimodal data collected; express The minimum and maximum values ​​of Step 3: Data encoding; Convert non-numeric features into numeric features using one-hot encoding.

Images