Power distribution edge intelligent agent, intelligent fusion terminal, energy storage and load regulation method

Abstract

This invention relates to the field of power distribution technology, providing a power distribution edge intelligent agent, an intelligent fusion terminal, energy storage, and load regulation method. The power distribution edge intelligent agent includes an intelligent algorithm model, which comprises a shared layer and a dedicated layer. The shared layer includes multiple gated recurrent units for acquiring common temporal features among multi-dimensional data. The shared layer simultaneously learns the common temporal features of multiple tasks using a multi-task learning algorithm. The dedicated layer includes multiple task branch modules. The prediction task branch module includes a gated recurrent unit for obtaining the predicted value of the corresponding task based on the common temporal features of multiple tasks. The regulation task branch module includes a gated recurrent unit and a reinforcement learning policy network for obtaining the regulation strategy of the corresponding task based on the common temporal features of multiple tasks. The reinforcement learning policy network is updated using a near-end policy optimization algorithm. This invention enables the coordinated operation of power distribution system sources, grid, load, and storage.

Description

Technical Field

[0001] This invention relates to the field of power distribution technology, specifically to a power distribution edge intelligent agent, an intelligent fusion terminal, a power distribution system energy storage regulation method, and a power distribution system load regulation method. Background Technology

[0002] With the construction of new power systems primarily based on new energy sources, the distribution network is gradually transforming from a simple power network that receives and distributes electricity to users into a power network that integrates and interacts with power sources, grids, loads, and storage, and is flexibly coupled with microgrids. Its functions in promoting the local consumption of distributed power sources from microgrids and supporting new types of loads are becoming increasingly significant. The new distribution system connects various power resources within a certain area, including traditional rigid loads, adjustable loads, photovoltaic power generation, and energy storage. It integrates and distributes power resources safely and quickly, making energy utilization more efficient and reliable. Simultaneously, the deep integration of artificial intelligence and edge computing, and the deployment of edge intelligence in distribution substations to achieve source-grid-load-storage coordination, has become an important development trend. Therefore, it is urgent to construct distribution edge intelligent agents oriented towards source-grid-load-storage, fully leveraging the technological advantages of the integration of edge computing and artificial intelligence algorithms to achieve intelligent scheduling of the new distribution system, thereby meeting the development needs of efficient source-grid-load-storage coordination.

[0003] Coordinated operation of power generation, grid, load, and storage in a power distribution system is essential for achieving optimal efficiency of a high proportion of distributed power sources. Existing operational technologies for power generation, grid, load, and storage systems largely focus on constructing multi-objective optimization models and designing optimization algorithms. While multi-objective optimization is a comprehensive approach, it suffers from high complexity, large computational load, and insufficient intelligence. With the development of edge computing and artificial intelligence, current mainstream optimization algorithms are struggling to meet the needs of deploying and operating edge intelligent agents. Summary of the Invention

[0004] This invention provides a power distribution edge intelligent agent, intelligent fusion terminal, energy storage and load regulation method. It uses an intelligent algorithm model based on MTL-GRU (Multi-task Learning-Gated Cyclic Unit) for photovoltaic, load and energy storage to perform multi-task comprehensive prediction, and combines it with PPO (Proximity Policy Optimization) algorithm to realize policy learning and updating, generating the optimal operation strategy for power distribution area resources, and realizing energy storage regulation and load regulation of the power distribution system, thereby achieving coordinated operation of power generation, grid, load and storage in the power distribution system.

[0005] The first aspect of the present invention provides a power distribution edge intelligent agent, the power distribution edge intelligent agent including an intelligent algorithm model, the intelligent algorithm model including: a shared layer and a dedicated layer; The shared layer includes multiple gated loop units for feature extraction of multidimensional data from the distribution radio area, obtaining common temporal features among the multidimensional data; wherein, the shared layer learns common temporal features of multiple tasks simultaneously through a multi-task learning algorithm; The dedicated layer includes at least one prediction task branch module and at least one regulation task branch module; each prediction task branch module includes a gated recurrent unit, used to obtain the prediction value of the corresponding task based on the common temporal features of multiple tasks obtained by the shared layer; each regulation task branch module includes a gated recurrent unit and a reinforcement learning policy network, used to obtain the regulation policy of the corresponding task based on the common temporal features of multiple tasks obtained by the shared layer; wherein, the reinforcement learning policy network is updated using a proximal policy optimization algorithm.

[0006] In this embodiment of the invention, the multidimensional data of the distribution substation includes: photovoltaic data, electrical load data, and energy storage data; The dedicated layer includes: a photovoltaic forecasting task branch module, a load forecasting task branch module, an energy storage regulation task branch module, and a load regulation task branch module; The shared layer learns the common time-series features of photovoltaic forecasting tasks, load forecasting tasks, energy storage regulation tasks, and load regulation tasks simultaneously through a multi-task learning algorithm.

[0007] In this embodiment of the invention, the shared layer includes multiple layers of gated loop units, each layer of gated loop units includes multiple simplified gated loop units, and each simplified gated loop unit retains only the update gate and cancels the reset gate.

[0008] In this embodiment of the invention, the photovoltaic prediction task branch module includes a single-layer gated loop unit, which retains only the update gate and cancels the reset gate; The update gate of the gated loop unit incorporates solar irradiance and its weight.

[0009] In this embodiment of the invention, the load prediction task branch module includes a single-layer gated loop unit, which retains only the update gate and cancels the reset gate; The update gate of the gated loop unit incorporates the real-time electricity price and its weight.

[0010] In this embodiment of the invention, the reinforcement learning policy network of the energy storage regulation task branch module includes: a policy network and a value network; The policy network is used to output actions in the action space based on the state space; The value network is used to output a state value based on the state, and the state value is used to update the policy network. The state space of the strategy network includes: real-time electricity price, the interaction power between energy storage and distribution network in the previous period, the total demand of rigid load and adjustable load, new energy output, energy storage battery SOC, and historical charge and discharge curve values. The action space of the strategy network is the energy storage charging / discharging power.

[0011] In this embodiment of the invention, the reinforcement learning policy network of the load regulation task branch module includes: a policy network and a value network; The policy network is used to output actions in the action space based on the state space; The value network is used to output a state value based on the state, and the state value is used to update the policy network. The state space of the strategy network includes: real-time electricity price, adjustable load response rate, adjustable load demand, renewable energy output, and historical load curve values. The action space of the policy network is an adjustable load.

[0012] In this embodiment of the invention, the reinforcement learning policy network uses a near-end policy optimization algorithm for learning and updating. During the policy decision update process, the ratio of the new and old policies is dynamically pruned by pruning the objective function, and the update step size is limited, so that the policy network can gradually approach the optimal control policy in the interaction with the distribution network environment.

[0013] In this embodiment of the invention, the reinforcement learning policy network is updated using a proximal policy optimization algorithm, including: After extracting common temporal features from the current multidimensional data of the distribution area through the shared layer, the data, together with the specific state variables of the control task branch of the dedicated layer, constitute the state space, which serves as the input to the policy network. Select and execute actions based on the current control strategy of the strategy network; Receive the reward and next-moment status returned by the distribution network environment; The experience trajectory, including state, action, reward, and next moment state, is stored in the experience pool. Sample batch data from the experience pool and calculate the advantage estimate; Based on the advantage estimate, the ratio of new to old policies is dynamically pruned using the pruning objective function of the near-end policy optimization algorithm, and the policy network is updated. The value network is updated by minimizing the mean squared error of the value objective function.

[0014] A second aspect of the present invention provides an intelligent fusion terminal deployed in a power distribution area, the intelligent fusion terminal including the aforementioned power distribution edge intelligent agent; The power distribution edge agent includes multiple containers, each of which runs an application for an intelligent algorithm model, a prediction task branch application, and a regulation task branch application, respectively.

[0015] A third aspect of the present invention provides a method for energy storage regulation in a power distribution system, comprising: The multi-dimensional data of the acquired distribution substation is input into the intelligent algorithm model, and the intelligent algorithm model outputs an energy storage regulation strategy. Energy storage regulation is then performed according to the energy storage regulation strategy. The intelligent algorithm model includes a shared layer and a dedicated layer. The shared layer comprises multiple gated recurrent units for extracting features from multidimensional data of the distribution substation and obtaining common temporal features among the multidimensional data. The shared layer simultaneously learns the common temporal features of multiple tasks using a multi-task learning algorithm. The dedicated layer includes an energy storage regulation task branch module, which comprises a single-layer gated recurrent unit and a reinforcement learning policy network for obtaining an energy storage regulation strategy based on the common temporal features of multiple tasks obtained by the shared layer. The reinforcement learning policy network is updated using a proximal policy optimization algorithm.

[0016] In this embodiment of the invention, the reinforcement learning policy network of the energy storage regulation task branch module includes: a policy network and a value network; The policy network is used to output actions in the action space based on the state space; The value network is used to output a state value based on the state, and the state value is used to update the policy network. The state space of the strategy network includes: real-time electricity price, the interaction power between energy storage and distribution network in the previous period, the total demand of rigid load and adjustable load, new energy output, energy storage battery SOC, and historical charge and discharge curve values. The action space of the strategy network is the energy storage charging / discharging power.

[0017] In this embodiment of the invention, the reinforcement learning policy network uses a near-end policy optimization algorithm for learning and updating. During the policy decision update process, the ratio of the new and old policies is dynamically pruned by pruning the objective function, and the update step size is limited, so that the policy network can gradually approach the optimal energy storage control strategy in the interaction with the distribution network environment.

[0018] In this embodiment of the invention, the reinforcement learning policy network is updated using a proximal policy optimization algorithm, including: After extracting common time-series features from the current multidimensional data of the distribution area through the shared layer, the data, together with the specific state variables of the energy storage regulation task branch of the dedicated layer, constitute the state space, which serves as the input to the policy network. Select and execute actions based on the current control strategy of the strategy network; Receive the reward and next-moment status returned by the distribution network environment; The experience trajectory, including state, action, reward, and next moment state, is stored in the experience pool. Sample batch data from the experience pool and calculate the advantage estimate; Based on the advantage estimate, the ratio of new to old policies is dynamically pruned using the pruning objective function of the near-end policy optimization algorithm, and the policy network is updated. The value network is updated by minimizing the mean squared error of the value objective function.

[0019] A fourth aspect of the present invention provides a load control method for a power distribution system, comprising: The multidimensional data of the distribution substation is input into the intelligent algorithm model, the intelligent algorithm model outputs the load control strategy, and the load control is performed according to the load control strategy. The intelligent algorithm model includes a shared layer and a dedicated layer. The shared layer comprises multiple gated recurrent units for extracting features from multidimensional data of the distribution substation and obtaining common temporal features among the multidimensional data. The shared layer simultaneously learns the common temporal features of multiple tasks using a multi-task learning algorithm. The dedicated layer includes a load control task branch module, which comprises a single-layer gated recurrent unit and a reinforcement learning policy network for obtaining a load control policy based on the common temporal features of multiple tasks obtained by the shared layer. The reinforcement learning policy network is updated using a proximal policy optimization algorithm.

[0020] In this embodiment of the invention, the reinforcement learning policy network of the load regulation task branch module includes: a policy network and a value network; The policy network is used to output actions in the action space based on the state space; The value network is used to output a state value based on the state, and the state value is used to update the policy network. The state space of the strategy network includes: real-time electricity price, adjustable load response rate, adjustable load demand, renewable energy output, and historical load curve values. The action space of the policy network is an adjustable load.

[0021] In this embodiment of the invention, the reinforcement learning policy network uses a near-end policy optimization algorithm for learning and updating. During the policy decision update process, the ratio of the new and old policies is dynamically pruned by pruning the objective function, and the update step size is limited, so that the policy network can gradually approach the optimal load control policy in the interaction with the distribution network environment.

[0022] In this embodiment of the invention, the reinforcement learning policy network is updated using a proximal policy optimization algorithm, including: After extracting common time-series features from the current multidimensional data of the distribution area through the sharing layer, the data, together with the specific state variables of the load control task branch of the dedicated layer, constitute the state space, which serves as the input to the policy network. Select and execute actions based on the current control strategy of the strategy network; Receive the reward and next-moment status returned by the distribution network environment; The experience trajectory, including state, action, reward, and next moment state, is stored in the experience pool. Sample batch data from the experience pool and calculate the advantage estimate; Based on the advantage estimate, the ratio of new to old policies is dynamically pruned using the pruning objective function of the near-end policy optimization algorithm, and the policy network is updated. The value network is updated by minimizing the mean squared error of the value objective function.

[0023] The above technical solution, by employing multi-layer gated recurrent units and integrating a multi-task learning mechanism in the shared layer, can efficiently extract common features from multi-dimensional time-series data such as photovoltaics, loads, and energy storage, significantly reducing the computational resource consumption and model parameter quantity of the edge agent, while improving the generalization ability of each related task. On this basis, each independent prediction branch in the dedicated layer uses gated recurrent units to achieve high-precision prediction of photovoltaic output and load demand, and the regulation branch introduces a reinforcement learning policy network and uses a near-end policy optimization algorithm to dynamically update network parameters, enabling the agent to adaptively learn the optimal regulation strategy for energy storage charging and discharging and adjustable loads during the operation of the distribution area, effectively balancing economy, safety, and source-grid-load-storage synergy, and meeting the requirements of real-time deployment and efficient operation of the edge agent in a resource-constrained environment.

[0024] Other features and advantages of the technical solution of the present invention will be described in detail in the following detailed embodiments section. Attached Figure Description

[0025] The accompanying drawings, which are included to provide a further understanding of the invention and form part of this invention, illustrate exemplary embodiments of the invention and are used to explain the invention, but do not constitute an undue limitation of the invention. In the drawings: Figure 1 This is a block diagram of the power distribution edge intelligent agent provided in an embodiment of the present invention; Figure 2 This is an architecture diagram of the MTL-GRU algorithm model for the power distribution edge intelligence agent provided in this embodiment of the invention; Figure 3 This is a schematic diagram of the shared layer structure of the MTL-GRU algorithm model provided in this embodiment of the invention; Figure 4 This is a block diagram of the intelligent fusion terminal provided in an embodiment of the present invention. Detailed Implementation

[0026] To make the technical solutions and advantages of the embodiments of the present invention clearer, the exemplary embodiments of the present invention will be further described in detail below with reference to the accompanying drawings. Obviously, the described embodiments are only a part of the embodiments of the present invention, and not an exhaustive list of all embodiments. It should be noted that, unless otherwise specified, the embodiments and features in the embodiments of the present invention can be combined with each other.

[0027] This invention provides a distribution edge intelligent agent, including an intelligent algorithm model comprising a shared layer and a dedicated layer. The shared layer includes multiple layers of gated recurrent units (GRUs) for feature extraction from multi-dimensional data of distribution substations, obtaining common temporal features among the multi-dimensional data. The shared layer simultaneously learns the common temporal features of multiple tasks using a multi-task learning (MTL) algorithm. The dedicated layer includes at least one prediction task branch module and at least one regulation task branch module. Each prediction task branch module includes a gated recurrent unit (S-GRU) for obtaining the predicted value of the corresponding task based on the common temporal features of multiple tasks obtained from the shared layer. Each regulation task branch module includes a gated recurrent unit (S-GRU) and a reinforcement learning policy network for obtaining the regulation policy of the corresponding task based on the common temporal features of multiple tasks obtained from the shared layer. The reinforcement learning policy network is updated using a proximal policy optimization (PPO) algorithm, enabling the policy network to gradually approach the optimal policy during interaction with the distribution network environment, while avoiding operational risks caused by policy mutations.

[0028] This invention employs a multi-layer gated recurrent unit in the shared layer and integrates a multi-task learning mechanism to efficiently extract common features from multi-dimensional time-series data such as photovoltaics, loads, and energy storage. This significantly reduces the computational resource consumption and model parameter quantity of the edge agent, while improving the generalization ability of each related task. Furthermore, in the dedicated layer, each independent prediction branch utilizes a gated recurrent unit to achieve high-precision prediction of photovoltaic output and load demand. The regulation branch introduces a reinforcement learning policy network and uses a near-end policy optimization algorithm to dynamically update network parameters. This enables the agent to adaptively learn the optimal regulation strategies for energy storage charging and discharging and adjustable loads during distribution network operation, effectively balancing economy, safety, and source-grid-load-storage synergy. This overcomes the shortcomings of existing multi-objective optimization methods, such as computational complexity and insufficient intelligence, and meets the requirements for real-time deployment and efficient operation of edge agents in resource-constrained environments.

[0029] The distribution edge intelligent agent of this invention is mainly deployed in distribution transformer areas, wherein the source-grid-load-storage collaborative system of the distribution transformer area mainly includes: traditional rigid loads, adjustable loads, photovoltaic power generation, and energy storage resources. The distribution edge intelligent agent mainly operates in intelligent converged terminals.

[0030] In a specific embodiment, the multidimensional data of the distribution substation includes: photovoltaic data, electrical load data, and energy storage data. The dedicated layer includes: a photovoltaic forecasting task branch module, a load forecasting task branch module, an energy storage regulation task branch module, and a load regulation task branch module. The shared layer simultaneously learns the common temporal characteristics of the photovoltaic forecasting task, load forecasting task, energy storage regulation task, and load regulation task through a multi-task learning algorithm.

[0031] Figure 1 This is a block diagram of the power distribution edge intelligence agent provided in an embodiment of the present invention. Figure 1 As shown, the distribution edge agent employs container technology, running multiple functional apps within the container, including a photovoltaic data acquisition app, an electricity consumption information acquisition app, and a meteorological data acquisition app. These apps collect electrical parameters from photovoltaic, load, and energy storage devices, as well as meteorological sensor parameters. The intelligent algorithm model of the distribution edge agent is the MTL-GRU algorithm model. The MTL-GRU algorithm model app is the core module for compiling the agent, extracting common temporal features across multiple tasks through the shared underlying GRU, and is responsible for training and running the shared layer of the MTL-GRU algorithm model. The application branches of the MTL-GRU algorithm model are photovoltaic forecasting, load forecasting, load regulation, and energy storage regulation, each running in its corresponding app. The load regulation app and the energy storage regulation app generate commands and issue them to the load-side and energy storage-side devices to execute corresponding controls.

[0032] like Figure 2 As shown, the MTL-GRU algorithm model mainly consists of three layers: a data layer, an MTL-GRU shared layer, and a dedicated layer. The data layer preprocesses the multidimensional data of the distribution substation (e.g., normalization), and the preprocessed data is input into the shared layer. The shared layer uses simplified GRU units (S-GRU) and extracts common characteristic parameters of the distribution substation operation through the shared GRU network, generating dedicated outputs through a fully connected layer. The dedicated layer is divided into multiple independent branch modules, such as load forecasting, photovoltaic forecasting, energy storage regulation, and load regulation, with each branch module using a customized GRU variant.

[0033] The MTL-GRU algorithm model employs a multi-task learning (MTL) structure. MTL contains a set of tasks. y t (t∈T) and data sets {x i ,y 1,i ,y 2,i , ，y t,i , ,y T,i } Where T is the number of subtasks and N is the number of data samples. y t,i Let be the label of the t-th task for the i-th data point. Define the prediction function as: ; in, The feature vector of the input sample refers to the multi-dimensional data collected by the distribution radio station area, such as the time-series features of photovoltaic output, load power, energy storage status, meteorological data (such as irradiance), and electricity price after normalization preprocessing. θ sh Parameters shared across different tasks. θ t Parameters related to specific tasks, Tags for tasks.

[0034] The overall optimization loss function for MTL is: ; in, This represents the total number of sub-tasks (e.g., photovoltaic forecasting, load forecasting, energy storage regulation, load regulation, etc.). For task indexing, start from 2; For the first The weight coefficients of each subtask are used to balance the importance of different tasks in joint training; The loss function, such as mean squared error (MSE), is used to measure the difference between the model's predicted values ​​and the true labels; For the first The prediction function for each task, input Then output the predicted value for the corresponding task; For the first The sample at the th The true label (actual observation) for each task; These are model parameters shared among multiple tasks (e.g., the weights of the GRU in a shared layer), used to extract common features; For the first Task-specific parameters (such as parameters for each branch in a dedicated layer) are used to capture task-specific patterns.

[0035] The overall loss function achieves joint optimization of multiple tasks by weighted summation of the losses of all subtasks, thereby improving the model's generalization ability and resource utilization efficiency.

[0036] This invention employs a combination of MTL (Multi-Task Learning) and GRU (Gated Recurrent Unit) to construct an MTL-GRU algorithm model. The specific steps are as follows: Step 1: Data Preprocessing. The input information of the model is cleaned and normalized, and then used as the training set, validation set, and test set of the proposed model.

[0037] Step 2: Construction of the shared layer. A shared layer of MTL is constructed using GRU to extract coupling information between photovoltaic, electrical load, and energy storage charging and discharging power.

[0038] Step 3: Output Layer Construction. A combined approach of MTL, GRU, and PPO is used to construct the task branches for the dedicated layer. First, the GRU unit is simplified by removing the original reset gates and retaining only the update gates. The computation process of the simplified GRU (Simplified-GRU, or S-GRU) unit is represented as follows: ; in, The input vector at time t is the multidimensional time-series data after normalization preprocessing, such as photovoltaic output, load power, electricity price, irradiance, etc. The hidden state at time t-1 (the output of the previous time step) carries historical time sequence information; The hidden state at time t (the output of the current time step) is obtained by linear interpolation of the historical state and the candidate state using the update gate; To update the gate, the value range is (0,1), which determines the previous hidden state. The degree of retention and candidate status The higher the degree of fusion, the higher the proportion of the current candidate state. The candidate hidden states are generated using the tanh activation function, representing the state based on the current input. The temporary state calculated from the reset historical information; The sigmoid activation function maps the linear output to (0,1) and is used to control the degree of opening and closing of the gate. The hyperbolic tangent activation function maps the linear output to (-1, 1) to generate candidate states; , To update the gates corresponding to the inputs respectively and hidden state The weight coefficient matrix; , The calculation of candidate hidden states corresponds to the input respectively. and hidden state The weight coefficient matrix; This indicates element-wise multiplication.

[0039] In this embodiment, the shared layer of the MTL-GRU includes multiple stacked gated loop units, each of which includes multiple simplified gated loop units (S-GRUs). Each simplified gated loop unit (S-GRU) retains only the update gate and cancels the reset gate.

[0040] In a specific embodiment, the shared layer consists of n×t S-GRU units, forming an n-layer S-GRU shared learning layer. For example... Figure 3 As shown, the shared layer consists of n shared S-GRU layers, each with t S-GRU units.

[0041] During model training, multidimensional feature vectors are used as input, and photovoltaic output and load prediction values ​​are used as output to train the shared-layer GRU network until a preset number of iterations are reached. Low-dimensional features in the original dataset are transformed into high-dimensional features layer by layer through multiple hidden layers, enabling the MTL-GRU shared-layer model to learn the implicit mapping relationship. The Adam optimization algorithm is used to input the features from the validation set into the trained MTL-GRU network. The output photovoltaic output and load prediction results are compared with the actual values, and the loss function is calculated. The network parameters are then adjusted generation by generation based on the loss function.

[0042] In a specific embodiment, the MTL-GRU dedicated layer of the edge agent is divided into four independent branch modules: photovoltaic prediction task branch module, load prediction task branch module, energy storage regulation task branch module, and load regulation task branch module.

[0043] The photovoltaic prediction task branch module adopts a single-layer S-GRU network, and introduces the weights of solar irradiance and solar irradiance into the update gate of the S-GRU. The calculation expression of the update gate is as follows: ; in, To update the output value of the gate, the value range is (0,1), controlling the previous hidden state. With candidate state The fusion ratio; The sigmoid activation function compresses the linear result to the (0,1) interval. To update the gate weight matrix, it is applied to the concatenated input vector. ; This represents the original input vector at time t. (such as historical photovoltaic power, temperature, etc.) and solar irradiance scalar Concatenate along the feature dimension; To update the hidden state of the door in the previous step The weight matrix; It represents the hidden state of the previous time step (t-1), carrying historical time sequence information; Let be the solar irradiance (scalar) at time t, which is directly input as prior physical information; Solar irradiance Weights (scalar or analog) (A vector of the same dimension) is used to adjust the direct effect of irradiance on the update gate.

[0044] The complete computational expression for the S-GRU network of the photovoltaic prediction task branch module is as follows: ; The S-GRU network outputs a predicted photovoltaic power value. Its expression is: ; in, Let be the predicted photovoltaic power at time t; The weight matrix of the output layer contains the hidden states. Mapped to predicted values; The hidden state calculated by the S-GRU network at the current time. For the bias term of the output layer; This is the activation function.

[0045] In the photovoltaic forecasting task branch module, solar irradiance is... Introducing the update gate separately as prior knowledge can effectively improve the sensitivity of photovoltaic prediction to changes in illumination.

[0046] In a specific embodiment, the load forecasting task branch module adopts a single-layer S-GRU network, and introduces real-time electricity price and its weight into the S-GRU update gate. The calculation expression for the update gate is as follows: ; in, For real-time electricity prices, This is the weight of the real-time electricity price (the definitions of other parameters in this formula can be found in the parameter definitions of the photovoltaic forecasting task branch module).

[0047] The complete calculation expression for the S-GRU network of the load forecasting task branch module is as follows: ; The predicted output load power of the S-GRU network Its expression is: ; in, This represents the predicted load power at time t; The weight matrix of the output layer contains the hidden states. Mapped to load forecast values; The hidden state calculated by the S-GRU network at the current time. For the bias term of the output layer; This is the activation function.

[0048] In the load forecasting task branch module, real-time electricity prices will be included. Introducing the update gate separately as prior knowledge allows the load forecasting model to perceive the potential impact of electricity prices on users' electricity consumption behavior, thereby improving forecast accuracy.

[0049] In this embodiment, the energy storage regulation task branch module and the load regulation task branch module combine S-GRU with the Proximal Policy Optimization (PPO) algorithm to generate energy storage regulation strategies. The PPO algorithm is based on an "Actor-Critic" structure and incorporates an importance sampling mechanism, using a deep neural network to construct a policy network (Actor) and a value network (Critic). The policy network outputs the probability distribution of actions based on the current state, and the value network evaluates the value of the current state. The algorithm calculates the ratio of the old and new strategies through importance sampling and uses a clip function to limit this ratio to the interval (1-c, 1+c).

[0050] Energy storage regulation first needs to be formalized as a Markov decision process, defining the state space, action space, state transition space, and reward function.

[0051] 1) State Space S t The expression is: ; in, For real-time electricity prices, This represents the interaction power between energy storage and the distribution network in the previous time period. For the total demand of rigid loads and adjustable loads, Contribute to new energy sources Et For energy storage battery SOC, These are historical charge / discharge curve values.

[0052] 2) Action Space Define the action space as the energy storage charging / discharging power. ∈[ -P MAX ,P MAX ], P MAX Indicates the maximum charge and discharge power limit of the energy storage system; when When the energy storage system discharges, the maximum discharge power shall not exceed P MAX ;when When the energy storage system is charging, the absolute value of the charging power will not exceed [a certain value]. P MAX .

[0053] 3) Define the expression for the state transition function as follows: ; in, Let t be the state of charge (SOC) of the energy storage battery. Let SOC of the energy storage battery be at time t+1. , These are charging efficiency and discharging efficiency, respectively. This refers to the battery's rated capacity. The time step (sampling / control cycle) is determined by the data acquisition frequency and control command issuance cycle of the edge agent.

[0054] The state transition function recursively calculates the SOC value at the next moment based on the current state of charge (SOC) of the energy storage and the action (charging / discharging power) performed, combined with the charging / discharging efficiency and rated capacity. This provides the agent with accurate dynamic evolution laws of energy storage, enabling the policy network to predict the long-term impact of actions.

[0055] 4) Define the expression for the reward function as follows: ; in, , , These are the weighting coefficients. This is the SOC reference value for energy storage batteries (typically 0.5). For safe power limits, For real-time electricity prices, Power for charging / discharging energy storage.

[0056] The reward function weights and sums the economic benefits of energy storage (such as arbitrage between low storage and high generation), the penalty for SOC deviation from the reference value, and the cost of power exceeding the limit. It evaluates the quality of each charging and discharging action in real time and guides the policy network to learn the optimal control strategy that can both smooth out peaks and valleys and ensure battery safety.

[0057] The PPO (Proximal Policy Optimization) algorithm model uses a value network to gradually improve the decision-making performance of the policy network, explores and learns control strategies under different energy storage scenarios, and the policy network eventually masters the optimal strategy under different energy storage scenarios through continuous interactive learning.

[0058] In a specific embodiment, the reinforcement learning policy network of the energy storage regulation task branch module includes a policy network (Actor) and a value network (Critic). The policy network layer outputs actions in the action space based on the state space. The value network layer outputs state values ​​based on the state, and these state values ​​are used to update the policy network. State space Including: real-time electricity price Rigid load demand, adjustable load demand, and renewable energy output. Response rate of adjustable load a Historical charge / discharge curve values Action space The energy storage charging and discharging power has a range of values. .

[0059] The reinforcement learning policy network adopts a near-end policy optimization algorithm. During the policy decision update process, the ratio of new and old policies is dynamically pruned by pruning the objective function, and the update step size is limited to ensure the stability and sample efficiency of the energy storage regulation policy learning process. This allows the policy network to gradually approach the optimal energy storage charging and discharging policy in interaction with the distribution network environment, while avoiding operational risks caused by policy mutations.

[0060] The value objective function of a value network (Critic) is expressed as follows: ; in, Representing state The value objective under the current state is equal to the advantage estimate of the current state plus the value estimate of the current state. The generalized advantage estimate (GAE) value represents the value in state . Take action below How good or bad it is relative to the average level; For the Critic network (parameters are) The output state value, i.e., the value derived from the state. The expected cumulative reward for starting.

[0061] Generalized dominance estimate (GAE) value The calculation expression is: ; in, The number of future steps considered when calculating advantage. As a discount factor, The smoothing coefficient for GAE; , The temporal difference error represents the difference between the current reward plus the value of the next state and the value of the current state. In the state Execute action Immediate rewards returned after the environment is restored; For the next state The output value of the Critic network.

[0062] The value objective function adds the generalized advantage estimate (GAE) to the current state value to form the true target value that the Critic network needs to fit, thereby accurately assessing the long-term return potential of the current state of energy storage (such as high SOC and high electricity price period) and providing the strategy network with a low variance and low bias advantage signal.

[0063] The expression for the pruning objective function of the PPO algorithm is: ; in, This represents the objective function that the policy network (Actor) needs to maximize in the PPO algorithm. The pruning constraint on the update step size is achieved by taking the minimum term. This represents the expectation of the samples collected from the experience pool; This indicates taking the minimum value of the two items within the parentheses; The ratio of new to old strategies; The current policy network is in state. Select action The probability of; It represents the probability of the old strategy used before the update (when collecting experience); Measure the magnitude of the strategy update; The advantage estimate serves as the "weight" for gradient updates; Indicates the ratio Clip to range Inside, among which It is a clipping hyperparameter; The PPO trimming radius limits the maximum proportion of policy changes in each update step. The larger the value of c, the more aggressive the changes are allowed, and the smaller the value of c, the more conservative the updates are.

[0064] In the update of energy storage charging and discharging strategies, the pruning objective function limits the probability ratio of the new and old strategies to no more than a preset range, thus avoiding sudden changes in charging and discharging power (such as from full charging to full discharging) caused by a single update, thereby improving sample efficiency while ensuring training stability.

[0065] In a specific embodiment, the reinforcement learning policy network of the load regulation task branch module includes a policy network (Actor) and a value network (Critic). The policy network layer outputs actions in the action space based on the state space. The value network layer outputs state values ​​based on the state, and these state values ​​are used to update the policy network. State space Including: real-time electricity price Response rate of adjustable load The demand for adjustable load New energy power output Historical load curve values ; state space Represented as: .

[0066] Define the adjustable load as the action space. A mechanism for generating adjustable load control strategies based on PPO. ∈ [0,L MAX ] ,in L MAX This refers to the total adjustable load.

[0067] Define the state transition function of the near-end policy optimization algorithm as follows: ; in, This represents the adjustable load demand value at the next time step (t+1); The action taken at the current time (t) is the adjustment amount (power reduction) of the adjustable load, with a range of values. . This represents the adjustable load demand value at the current time (t). This is the upper limit (maximum value) of the total adjustable load, used to normalize the current load demand; The time step (sampling / control cycle) converts the power regulation into energy or load changes.

[0068] This state transition function describes the change in adjustable load demand at the next moment after a load reduction action is performed.

[0069] Define the reward function of the policy network (Actor) as follows: ; in, This represents the instantaneous reward value at time t, used to evaluate the current action. Good or bad; The weighting coefficient for the first item (positive reward item), usually a positive number, is used to encourage load-cutting behaviors that are economically or efficiently beneficial. The weighting coefficient for the second item (penalty item) is usually a positive number and is used to suppress undesirable conditions (such as excessively high demand for adjustable load or excessive reduction). The adjustable load demand at the current moment; This refers to the action taken at the current moment (load reduction amount).

[0070] This reward function makes the agent tend to reduce load demand appropriately to gain benefits during the learning process, while avoiding prolonged periods of high load.

[0071] In a specific embodiment, the Proximal Policy Optimization (PPO) algorithm is used to update the reinforcement learning policy network, including the following steps: Step 1: Extract common time-series features from the current data of the distribution substation through the MTL-GRU shared layer, and combine them with specific state variables (such as real-time electricity price, SOC, load demand, etc.) of the dedicated control branch to form the state space. , as input to the Actor network; Step 2: Select and execute actions based on the current control strategy of the strategy network (such as adjusting energy storage output and adjustable load response). Step 3: The environment returns the reward and the state for the next time step; Step 4: Store the experience trajectory (state, action, reward, next state) into the experience pool; Step 5: Sample a small batch of data from the experience pool and calculate the dominance estimate. ; Step 6: Based on the advantage estimate The ratio of new to old strategies is dynamically adjusted using the pruning objective function of PPO, and the Actor network is updated accordingly. Step 7: Update the Critic network by minimizing the mean squared error value objective function.

[0072] Step 8: Repeat steps 1-7 above until the reward function converges.

[0073] This invention provides an intelligent converged terminal, which includes the aforementioned power distribution edge intelligent agent. For example... Figure 4 As shown, the intelligent converged terminal is deployed in the distribution substation area, and it contains a distribution edge intelligent agent. This edge intelligent agent is built based on container technology and has an internal data bus and multiple functional containers (containers 1 to 4). Each container runs different types of applications. Container 1 runs data acquisition applications, including: electricity consumption information acquisition application, photovoltaic data acquisition application, and meteorological data acquisition application; Container 2 runs data preprocessing applications, including: data preprocessing application and MTL-GRU algorithm model application; Container 3 runs dedicated forecasting task applications, including: photovoltaic forecasting application and load forecasting application; Container 4 runs dedicated control task applications, including: load control application and energy storage control application.

[0074] The intelligent fusion terminal connects to the physical equipment in the distribution area (photovoltaic power generation equipment, rigid loads, adjustable loads, energy storage equipment, etc.) via a southbound communication interface and communicates with the upper-level dispatch system or cloud platform via a northbound communication interface. The southbound communication interface collects electrical parameters (voltage, current, power) and meteorological sensor data from photovoltaic, load, and energy storage equipment. After entering the intelligent fusion terminal, the data is distributed to various acquisition apps in container_1 via an internal data bus. The acquisition apps push the raw data to the internal data bus, where it is read by the data preprocessing app in container_2. The preprocessing app performs normalization, cleaning, and alignment operations to generate standardized time-series feature vectors. The preprocessed data is then sent to the MTL-GRU algorithm model app in container_3. This app runs a multi-task learning model. The shared layer extracts common time-series features, while the dedicated layers output predicted values ​​and control strategies: the photovoltaic prediction app outputs photovoltaic predicted values, the load prediction app outputs load predicted values, the energy storage control app outputs energy storage control strategies, and the load control app outputs load control strategies.

[0075] The load control APP and energy storage control APP convert the strategy into specific control commands (such as adjustable load reduction amount and energy storage charging and discharging power), and send them to the adjustable load device and energy storage converter for execution through the internal data bus and southbound communication interface.

[0076] Edge agents can report their operating status, prediction results, and control logs to the power distribution master station or cloud platform through the northbound communication interface, while also receiving dispatch instructions from higher authorities.

[0077] The intelligent converged terminal provided in this invention operates with each application running in an independent container, facilitating deployment, upgrades, and resource isolation. All modules exchange data via a bus, achieving decoupling and low-latency communication. The MTL-GRU algorithm model simultaneously handles prediction and control tasks, sharing the GRU layer to reduce computational overhead. The intelligent converged terminal adopts a local edge autonomous control architecture encompassing data acquisition, preprocessing, inference, control, and execution, realizing intelligent and real-time coordination of power generation, grid, load, and storage, meeting the deployment requirements of new power distribution systems for edge intelligent agents.

[0078] This invention provides a method for energy storage regulation in a power distribution system. The method includes: inputting multi-dimensional data of a power distribution substation into an intelligent algorithm model; outputting an energy storage regulation strategy through the intelligent algorithm model; and performing energy storage regulation according to the energy storage regulation strategy. The intelligent algorithm model includes a shared layer and a dedicated layer. The shared layer includes multiple gated recurrent units for extracting features from the multi-dimensional data of the power distribution substation and obtaining common temporal features among the multi-dimensional data. The shared layer simultaneously learns the common temporal features of multiple tasks using a multi-task learning algorithm. The dedicated layer includes an energy storage regulation task branch module, which includes a single-layer gated recurrent unit and a reinforcement learning policy network, for obtaining the energy storage regulation strategy based on the common temporal features of multiple tasks obtained from the shared layer. The reinforcement learning policy network is updated using a proximal policy optimization algorithm.

[0079] The reinforcement learning policy network of the energy storage regulation task branch module includes a policy network and a value network. The policy network outputs actions in the action space based on the state space. The value network outputs state values ​​based on the state, and these state values ​​are used to update the policy network. The state space of the policy network includes: real-time electricity price, the interaction power between energy storage and the distribution network in the previous time period, the total demand of rigid and adjustable loads, renewable energy output, energy storage battery SOC, and historical charge / discharge curve values. The action space of the policy network is the energy storage charging / discharging power.

[0080] The reinforcement learning policy network uses a near-end policy optimization algorithm for learning and updating. During the policy decision update process, the ratio of new and old policies is dynamically pruned by pruning the objective function, and the update step size is limited, so that the policy network can gradually approach the optimal energy storage control strategy in the interaction with the distribution network environment.

[0081] In this embodiment, a near-end policy optimization algorithm is used to update the reinforcement learning policy network. The specific process includes: After extracting common time-series features from the current multidimensional data of the distribution area through the shared layer, the data, together with the specific state variables of the energy storage regulation task branch of the dedicated layer, constitute the state space, which serves as the input to the policy network. Select and execute actions based on the current control strategy of the strategy network; Receive the reward and next-moment status returned by the distribution network environment; The experience trajectory, including state, action, reward, and next moment state, is stored in the experience pool. Sample batch data from the experience pool and calculate the advantage estimate; Based on the advantage estimate, the ratio of new to old policies is dynamically pruned using the pruning objective function of the near-end policy optimization algorithm, and the policy network is updated. The value network is updated by minimizing the mean squared error of the value objective function.

[0082] This invention provides a load control method for a power distribution system. The method includes: inputting multi-dimensional data of a power distribution substation into an intelligent algorithm model; outputting a load control strategy through the intelligent algorithm model; and performing load control according to the load control strategy. The intelligent algorithm model includes a shared layer and a dedicated layer. The shared layer includes multiple gated recurrent units for extracting features from the multi-dimensional data of the power distribution substation and obtaining common temporal features among the multi-dimensional data. The shared layer simultaneously learns the common temporal features of multiple tasks using a multi-task learning algorithm. The dedicated layer includes a load control task branch module, which includes a single-layer gated recurrent unit and a reinforcement learning policy network, for obtaining the load control strategy based on the common temporal features of multiple tasks obtained from the shared layer. The reinforcement learning policy network is updated using a proximal policy optimization algorithm.

[0083] The reinforcement learning policy network of the load regulation task branch module includes a policy network and a value network. The policy network outputs actions in the action space based on the state space. The value network outputs state values ​​based on the state, and these state values ​​are used to update the policy network. The state space of the policy network includes: real-time electricity price, adjustable load response rate, adjustable load demand, renewable energy output, and historical load curve values. The action space of the policy network is the adjustable load quantity.

[0084] The reinforcement learning policy network uses a near-end policy optimization algorithm for learning and updating. During the policy decision-making and updating process, the ratio of new and old policies is dynamically pruned by pruning the objective function, which limits the update step size. This allows the policy network to gradually approach the optimal load control strategy in interaction with the distribution network environment.

[0085] In this embodiment, a near-end policy optimization algorithm is used to update the reinforcement learning policy network. The specific process includes: After extracting common time-series features from the current multidimensional data of the distribution area through the sharing layer, the data, together with the specific state variables of the load control task branch of the dedicated layer, constitute the state space, which serves as the input to the policy network. Select and execute actions based on the current control strategy of the strategy network; Receive the reward and next-moment status returned by the distribution network environment; The experience trajectory, including state, action, reward, and next moment state, is stored in the experience pool. Sample batch data from the experience pool and calculate the advantage estimate; Based on the advantage estimate, the ratio of new to old policies is dynamically pruned using the pruning objective function of the near-end policy optimization algorithm, and the policy network is updated. The value network is updated by minimizing the mean squared error of the value objective function.

[0086] The technical details of the power distribution system energy storage regulation method and load regulation method provided in this embodiment of the invention (pruning objective function, state transition function, reward function, value objective function, etc.) can be referred to the specific implementation details of the power distribution edge intelligent agent mentioned above, and will not be repeated here.

[0087] Those skilled in the art will understand that embodiments of the present invention can be provided as methods, systems, or computer program products. Therefore, the present invention can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, the present invention can take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code. The solutions in the embodiments of the present invention can be implemented using various computer languages, such as the object-oriented programming language Java and the interpreted scripting language JavaScript.

[0088] This invention is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of the invention. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, generate instructions for implementing the flowchart illustrations and / or block diagrams. Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.

[0089] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and / or boxes Figure 1 The function specified in one or more boxes.

[0090] These computer program instructions may also be loaded onto a computer or other programmable data processing equipment to cause a series of operational steps to be performed on the computer or other programmable equipment to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable equipment for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.

[0091] The optional embodiments of the present invention have been described in detail above with reference to the accompanying drawings. However, the embodiments of the present invention are not limited to the specific details in the above embodiments. Within the scope of the technical concept of the embodiments of the present invention, various simple modifications can be made to the technical solutions of the embodiments of the present invention, and these simple modifications all fall within the protection scope of the embodiments of the present invention. Furthermore, it should be noted that the various specific technical features described in the above specific embodiments can be combined in any suitable manner without contradiction. As long as such combination does not violate the spirit of the embodiments of the present invention, it should also be considered as the content disclosed in the embodiments of the present invention.

Claims

1. A power distribution edge intelligent agent, characterized in that, The power distribution edge intelligent agent includes an intelligent algorithm model, which includes a shared layer and a dedicated layer. The shared layer includes multiple gated loop units for feature extraction of multidimensional data from the distribution radio area, obtaining common temporal features among the multidimensional data; wherein, the shared layer learns common temporal features of multiple tasks simultaneously through a multi-task learning algorithm; The dedicated layer includes at least one prediction task branch module and at least one regulation task branch module; each prediction task branch module includes a gated recurrent unit, used to obtain the prediction value of the corresponding task based on the common temporal features of multiple tasks obtained by the shared layer; each regulation task branch module includes a gated recurrent unit and a reinforcement learning policy network, used to obtain the regulation policy of the corresponding task based on the common temporal features of multiple tasks obtained by the shared layer; wherein, the reinforcement learning policy network is updated using a proximal policy optimization algorithm.

2. The power distribution edge intelligent agent according to claim 1, characterized in that, The multidimensional data of the distribution area includes: photovoltaic data, electrical load data, and energy storage data; The dedicated layer includes: a photovoltaic forecasting task branch module, a load forecasting task branch module, an energy storage regulation task branch module, and a load regulation task branch module; The shared layer learns the common time-series features of photovoltaic forecasting tasks, load forecasting tasks, energy storage regulation tasks, and load regulation tasks simultaneously through a multi-task learning algorithm.

3. The power distribution edge intelligent agent according to claim 1, characterized in that, The shared layer includes multiple layers of gated loop units, each of which includes multiple simplified gated loop units. Each simplified gated loop unit retains only the update gate and cancels the reset gate.

4. The power distribution edge intelligent agent according to claim 2, characterized in that, The photovoltaic prediction task branch module includes a single-layer gated loop unit, which retains only the update gate and cancels the reset gate; The update gate of the gated loop unit incorporates solar irradiance and its weight.

5. The power distribution edge intelligent agent according to claim 2, characterized in that, The load forecasting task branch module includes a single-layer gated loop unit, which retains only the update gate and cancels the reset gate; The update gate of the gated loop unit incorporates the real-time electricity price and its weight.

6. The power distribution edge intelligent agent according to claim 2, characterized in that, The reinforcement learning policy network of the energy storage regulation task branch module includes: a policy network and a value network; The policy network is used to output actions in the action space based on the state space; The value network is used to output a state value based on the state, and the state value is used to update the policy network. The state space of the strategy network includes: real-time electricity price, the interaction power between energy storage and distribution network in the previous period, the total demand of rigid load and adjustable load, new energy output, energy storage battery SOC, and historical charge and discharge curve values. The action space of the strategy network is the energy storage charging / discharging power.

7. The power distribution edge intelligent agent according to claim 2, characterized in that, The reinforcement learning policy network of the load regulation task branch module includes: a policy network and a value network; The policy network is used to output actions in the action space based on the state space; The value network is used to output a state value based on the state, and the state value is used to update the policy network. The state space of the strategy network includes: real-time electricity price, adjustable load response rate, adjustable load demand, renewable energy output, and historical load curve values. The action space of the policy network is an adjustable load.

8. The power distribution edge intelligent agent according to claim 6 or 7, characterized in that, The reinforcement learning policy network uses a near-end policy optimization algorithm for learning and updating. During the policy decision update process, the ratio of new and old policies is dynamically pruned by pruning the objective function, and the update step size is limited, so that the policy network can gradually approach the optimal control policy in the interaction with the distribution network environment.

9. The power distribution edge intelligent agent according to claim 1, characterized in that, The reinforcement learning policy network is updated using a proximal policy optimization algorithm, including: After extracting common temporal features from the current multidimensional data of the distribution area through the shared layer, the data, together with the specific state variables of the control task branch of the dedicated layer, constitute the state space, which serves as the input to the policy network. Select and execute actions based on the current control strategy of the strategy network; Receive the reward and next-moment status returned by the distribution network environment; The experience trajectory, including state, action, reward, and next moment state, is stored in the experience pool. Sample batch data from the experience pool and calculate the advantage estimate; Based on the advantage estimate, the ratio of new to old policies is dynamically pruned using the pruning objective function of the near-end policy optimization algorithm, and the policy network is updated. The value network is updated by minimizing the mean squared error of the value objective function.

10. An intelligent converged terminal, deployed in a distribution radio area, characterized in that, The intelligent fusion terminal includes: the power distribution edge intelligent agent as described in any one of claims 1-9; The power distribution edge agent includes multiple containers, each of which runs an application for an intelligent algorithm model, a prediction task branch application, and a regulation task branch application, respectively.

11. A method for energy storage regulation in a power distribution system, characterized in that, include: The multi-dimensional data of the acquired distribution substation is input into the intelligent algorithm model, and the intelligent algorithm model outputs an energy storage regulation strategy. Energy storage regulation is then performed according to the energy storage regulation strategy. The intelligent algorithm model includes a shared layer and a dedicated layer. The shared layer includes multiple gated loop units for feature extraction of multidimensional data from the distribution radio area, obtaining common temporal features among the multidimensional data; the shared layer learns common temporal features of multiple tasks simultaneously through a multi-task learning algorithm; The dedicated layer includes an energy storage regulation task branch module, which comprises a single-layer gated recurrent unit and a reinforcement learning policy network, used to obtain an energy storage regulation policy based on the common temporal features of multiple tasks obtained from the shared layer; the reinforcement learning policy network is updated using a proximal policy optimization algorithm.

12. The power distribution system energy storage regulation method according to claim 11, characterized in that, The reinforcement learning policy network of the energy storage regulation task branch module includes: a policy network and a value network; The policy network is used to output actions in the action space based on the state space; The value network is used to output a state value based on the state, and the state value is used to update the policy network. The state space of the strategy network includes: real-time electricity price, the interaction power between energy storage and distribution network in the previous period, the total demand of rigid load and adjustable load, new energy output, energy storage battery SOC, and historical charge and discharge curve values. The action space of the strategy network is the energy storage charging / discharging power.

13. The power distribution system energy storage regulation method according to claim 12, characterized in that, The reinforcement learning policy network uses a near-end policy optimization algorithm for learning and updating. During the policy decision update process, the ratio of new and old policies is dynamically pruned by pruning the objective function, and the update step size is limited, so that the policy network can gradually approach the optimal energy storage control strategy in the interaction with the distribution network environment.

14. The power distribution system energy storage regulation method according to claim 12, characterized in that, The reinforcement learning policy network is updated using a proximal policy optimization algorithm, including: After extracting common time-series features from the current multidimensional data of the distribution area through the shared layer, the data, together with the specific state variables of the energy storage regulation task branch of the dedicated layer, constitute the state space, which serves as the input to the policy network. Select and execute actions based on the current control strategy of the strategy network; Receive the reward and next-moment status returned by the distribution network environment; The experience trajectory, including state, action, reward, and next moment state, is stored in the experience pool. Sample batch data from the experience pool and calculate the advantage estimate; Based on the advantage estimate, the ratio of new to old policies is dynamically pruned using the pruning objective function of the near-end policy optimization algorithm, and the policy network is updated. The value network is updated by minimizing the mean squared error of the value objective function.

15. A load control method for a power distribution system, characterized in that, include: The multidimensional data of the distribution substation is input into the intelligent algorithm model, the intelligent algorithm model outputs the load control strategy, and the load control is performed according to the load control strategy. The intelligent algorithm model includes a shared layer and a dedicated layer. The shared layer includes multiple gated loop units for feature extraction of multidimensional data from the distribution radio area, obtaining common temporal features among the multidimensional data; the shared layer learns common temporal features of multiple tasks simultaneously through a multi-task learning algorithm; The dedicated layer includes a load regulation task branch module, which comprises a single-layer gated recurrent unit and a reinforcement learning policy network, used to obtain a load regulation policy based on the common temporal features of multiple tasks obtained from the shared layer; the reinforcement learning policy network is updated using a proximal policy optimization algorithm.

16. The power distribution system load control method according to claim 15, characterized in that, The reinforcement learning policy network of the load regulation task branch module includes: a policy network and a value network; The policy network is used to output actions in the action space based on the state space; The value network is used to output a state value based on the state, and the state value is used to update the policy network. The state space of the strategy network includes: real-time electricity price, adjustable load response rate, adjustable load demand, renewable energy output, and historical load curve values. The action space of the policy network is an adjustable load.

17. The power distribution system load control method according to claim 16, characterized in that, The reinforcement learning policy network uses a near-end policy optimization algorithm for learning and updating. During the policy decision update process, the ratio of new and old policies is dynamically pruned by pruning the objective function, and the update step size is limited, so that the policy network can gradually approach the optimal load control policy in the interaction with the distribution network environment.

18. The power distribution system load control method according to claim 16, characterized in that, The reinforcement learning policy network is updated using a proximal policy optimization algorithm, including: After extracting common time-series features from the current multidimensional data of the distribution area through the sharing layer, the data, together with the specific state variables of the load control task branch of the dedicated layer, constitute the state space, which serves as the input to the policy network. Select and execute actions based on the current control strategy of the strategy network; Receive the reward and next-moment status returned by the distribution network environment; The experience trajectory, including state, action, reward, and next moment state, is stored in the experience pool. Sample batch data from the experience pool and calculate the advantage estimate; Based on the advantage estimate, the ratio of new to old policies is dynamically pruned using the pruning objective function of the near-end policy optimization algorithm, and the policy network is updated. The value network is updated by minimizing the mean squared error of the value objective function.

Images