Source network load storage intelligent collaborative optimization method

Abstract

The invention belongs to the technical field of power system optimization scheduling, and provides a source network load storage intelligent collaborative optimization method, which comprises the following steps of: deploying sensors at four ends of a source network load storage respectively, collecting in real time by utilizing a cloud data center, enabling data of the four ends to be consistent in time sequence through a PTP protocol, constructing a topological graph according to parameters and data, and establishing a source network load storage intelligent collaborative optimization system. Selecting a model in a digital twinning environment for simulation; dividing independent agents at four ends of a source network load storage, setting observation data, an execution space and excitation feedback, forming an excitation item by economy, stability and environmental protection, interactively circulating actual data, a prediction instruction and an excitation value, recording into a sequence, inputting the sequence into a strategy network, and calculating and outputting logarithmic probability gradient to update the parameters of the strategy network; and the intelligent agent completes interactive circulation according to the strategy network, generates a local scheduling instruction, aggregates the instruction to perform weighted calculation, generates a global scheduling scheme, issues the global scheduling scheme to execution equipment, updates parameters by using an average deviation calculated by a deviation vector, resolves the global scheduling scheme and issues the global scheduling scheme to form a closed-loop mechanism.

Description

Technical Field

[0001] The present invention relates to the technical field of power system optimization and dispatching, and in particular to a method for intelligent collaborative optimization of sources, networks, loads and storage. Background Art

[0002] Source-grid-load-storage refers to a new power system operation mode that is based on the overall planning of power source, grid, load and energy storage. In the field of power system management, data analysis and collaborative algorithms of the four ends of source-grid-load-storage have become the key to power system optimization.

[0003] Publication number CN118659391B in China describes a method for intelligent collaborative optimization of sources, grids, loads, and storage. This configuration method includes: reading collaborative optimization basic data; obtaining a data processing unit based on a pre-set pattern partitioning algorithm; sequentially performing power collaborative optimization calculations on the data processing unit to obtain active power data sequences and reactive power data sequences; and then obtaining a collaborative optimization data combination; which is then output to an intelligent collaborative controller for sources, grids, loads, and storage. This automatically generates power control data for bidirectionally adjustable sources, loads, and storage, achieving collaborative optimization control.

[0004] In the field of source-grid-load-storage coordination, although there is a solution to the problem of coordinated optimization control of bidirectional power-adjustable source, load, and storage power under the condition of given load access time requirements, in the township-level multi-energy complementary microgrid scenario, when debugging the coordinated work of the four ends of source, grid, load, and storage, due to the instability of the on-site equipment of the township microgrid, frequent online and offline push strategies are likely to cause equipment damage, the cost of debugging instructions and iterative solutions is high, and there is a lack of safe and effective debugging methods; on the other hand, the global power scheduling in the rural microgrid scenario is greatly affected by the season and lifestyle, and the global system cannot make real-time scheduling according to changes in the situation. There is a lack of mechanisms that can strengthen learning and closed-loop optimization scheduling, resulting in low participation of some systems and energy efficiency waste. Summary of the Invention

[0005] The present invention provides a method for intelligent collaborative optimization of power generation, grid, load, and storage. The purpose is to solve the problems of high equipment depreciation rate, low fault tolerance, high equipment cost, and the inability of the global system to make real-time scheduling according to changing circumstances during the coordinated testing of the four terminals of the power generation, grid, load, and storage. The technical solution adopted by the present invention to solve the above technical problems is to provide a method for intelligent collaborative optimization of power generation, grid, load, and storage: collecting data from the four terminals of the power generation, grid, load, and storage, constructing a digital twin model of the microgrid, deploying multiple intelligent agents for reinforcement learning, calculating and generating local policy gradients, solving the global scheduling allocation plan, and issuing the strategy to the units for iterative optimization.

[0006] As a preferred embodiment, the specific steps of collecting the source, network, load and storage four-terminal data are as follows: installing sensors in the power generation unit of the township-level microgrid to collect active power, reactive power, voltage and current in real time; deploying smart voltmeters in the distribution network unit of the township-level microgrid to collect the voltage, line status, power and alarm information of the distribution network in real time; installing smart meters in the user-side load units of the township-level microgrid, such as agricultural machinery irrigation pumps and electric vehicle charging piles, to collect power consumption, short-term power and load switch status in real time; configuring energy storage units such as supercapacitor cabinets in the township microgrid, integrating high-precision current transformers, to collect charge status in real time , single-cell voltage, and temperature conditions during charging and discharging; a cloud data center is deployed in the control center of the township-level microgrid, and the sensor data collected at the four ends are transmitted to the cloud data center through the MQTT protocol. The cloud data center synchronizes the clock with the sensor equipment through the PTP protocol to ensure that the time series of the four-end data of source, grid, load and storage are consistent. The sensor data is detected for abnormal values and missing items, aggregated according to a time window of uniform size to form data with multiple sampling frequencies, and the mean and variance of the data in the same time window are calculated for rapid trend monitoring. PCA dimensionality reduction is performed on high-dimensional data to generate low-dimensional feature vectors, which are written into the database and stored in tables according to different sampling frequencies.

[0007] As a preferred embodiment, the specific steps of constructing the microgrid digital twin model are as follows: deploying edge digital twin nodes, collecting parameters and sensor data of each end unit, and uploading them to the edge digital twin nodes; constructing a township-level microgrid electrical topology diagram in the digital twin environment according to the association and hierarchy of each end unit, and selecting the average value inverter model for the power generation unit, the π-type equivalent circuit model for the distribution unit, the constant resistance model for the controllable load unit, and the second-order RC battery model for the energy storage unit. The collected parameters are mapped to the model library parameter interface, and simulation instances of each unit are generated. The sensor data is used for verification, and the least squares fitting is used to correct the deviation between the data of the simulation instance and the actual data. The parameter vector Store the parameters of the simulation instance, where Represents the parameters of different units, T is the matrix transpose symbol, and the actual data obtained at time t is recorded as , the error obtained at time t is ,in Represents the simulation data at time t, collects the square errors at j different times to construct the least squares objective function, and the formula is: , linearize the parameter vector to obtain the regression vector, merge the regression vectors at j different times to obtain the regression matrix, and obtain the parameter increment of each simulation instance according to the influence of the error value in the regression matrix on the simulation results, which is applied to the parameter correction of the simulation instance.

[0008] As a preferred implementation method, the specific steps of deploying multiple agents for reinforcement learning are as follows: calling the digital twin model parameter interface, dividing the four terminals of source, grid, load and storage into independent agents according to their responsibilities, defining the data and status to be collected by each terminal, setting the execution space of each terminal, and setting the incentive function. The incentive items are composed of economy, stability and environmental protection. The formula of economic incentive item is: , where k represents the kth decision during scheduling, represents the normalization coefficient, Indicates the preset value of consumption cost, represents the consumption cost of the kth decision; the stability incentive formula is ,in represents the normalization coefficient of voltage deviation, represents the normalization coefficient of frequency deviation, represents the voltage value of the kth decision, Indicates the rated voltage value, represents the frequency value of the k-th decision, Indicates the rated frequency value; the environmental incentive formula is ,in represents the normalization coefficient, represents the environmental energy output of the k-th decision, Represents the total output of the k-th decision, and the above three items are combined into the overall incentive according to the weights: ,in , , are the weights of each item, and The weights in the excitation function of each terminal unit are adjusted according to the actual application; MADDPG is selected as the reinforcement algorithm to initialize the policy network, and the parameter vector in the digital twin model is normalized and input into the policy network. The next decision is output according to the probability, and the incentive value is calculated after execution. The actual data and predicted instructions interact with the incentive value to form a loop.

[0009] As a preferred embodiment, the specific steps of calculating and generating the local policy gradient are as follows: the local agent collects data at a certain time u, records the interaction sequence of actual data, predicted instructions, and incentive values, performs outlier and normalization processing locally, generates an observation vector input into the policy network, the agent outputs the instruction distribution, maps the instructions into an instruction format that can be recognized and executed by the device, and calculates the real-time incentive value after the decision is executed at the current moment. , calculate the future incentive value of the decision execution for all subsequent moments , according to the attenuation coefficient Weighted summation gives weighted cumulative return , the formula is: , calculate the logarithmic probability gradient of the policy network output at this time, multiply the logarithmic probability gradient by the corresponding weighted cumulative reward to obtain the single-step gradient contribution. The single-step gradient contribution is regarded as a signal that the parameter adjustment direction can improve the reward. After collecting the single-step gradient contributions at different times, the average gradient is obtained and used to update the policy network parameters. The data of the intelligent agent at the next moment is collected to repeat the training.

[0010] As a preferred implementation method, the specific steps for solving the global scheduling allocation scheme are as follows: the four-terminal intelligent entities of source, grid, load and storage complete an interactive cycle according to the latest strategy network, calculate the local policy gradient and generate local scheduling instructions, the township-level microgrid cloud data center pre-defines the structure, and transmits the scheduling instructions to the cloud data center according to the predefined structure. At the same time, the cloud data center presets the weight of each unit, collects the operating data and parameters of each unit, and after normalization, determines the relative importance of each unit in the overall system through the hierarchical analysis method, normalizes the calculated comprehensive score of each unit, assigns weights to each unit according to the comprehensive score, performs offline simulation in the digital twin model, analyzes sensitivity and global fluctuations, and adjusts the previous weights according to the actual impact; the cloud data center aggregates all scheduling instructions and assigns unique identifiers, performs weighted average calculation, and linearly combines them to obtain global scheduling instructions. , the formula is , where i represents unit i and n represents the total number of units, Represents the weight of the unit, the sum of the weights of each unit is 1, Represents the scheduling instructions for the unit.

[0011] As a preferred embodiment, the specific steps of sending the strategy to the unit for iterative optimization are as follows: sending the calculated global scheduling instruction to each end unit, each end unit takes out its own sub-scheduling instruction according to the scheduling instruction dimension and unique identifier, splits the scheduling instruction according to the predefined structure, and sends it to the specific execution device in real time. After the instruction is executed, the real-time data and execution status are collected and sent back to the upper unit. The upper unit uniformly feedbacks the results to the cloud data center. The cloud data center identifies the unit's scheduling instruction according to the unique identifier, compares it with the actual result of the feedback, verifies the timing consistency, and calculates the deviation vector. The formula is ,in Indicates the dispatch instruction issued. Represents the actual result of the feedback, i represents unit i, and all deviation vectors are integrated to calculate the average deviation. The average deviation is passed to the digital twin model, and the parameters are updated using least squares fitting. The global scheduling instructions are then re-solved and sent to the units for execution, forming a closed-loop scheduling mechanism of "feedback-correction-optimization-solution-send-feedback".

[0012] The beneficial effects of the present invention are: The present invention constructs a digital twin model through four-end collection, realizes standardized data aggregation and sharing, calculates the global scheduling plan in the cloud, ensures efficient coordination and cooperation between various energy units, and significantly reduces the risk and cost of on-site debugging.

[0013] By introducing multi-objective incentives, reinforcement learning continuously optimizes the policy network, enabling each terminal device to make optimal scheduling when facing different scenarios. The local and cloud are combined to form a closed-loop mechanism, improving the robustness and efficiency of the overall system. BRIEF DESCRIPTION OF THE DRAWINGS

[0014] Figure 1 The present invention is a flow chart of a method for intelligent collaborative optimization of source, grid, load and storage. DETAILED DESCRIPTION

[0015] In order to make the technical means, creative features, objectives and effects achieved by the present invention easier to understand, the present invention is further described below in conjunction with specific embodiments. However, the following embodiments are only preferred embodiments of the present invention and are not exhaustive. Based on the embodiments in the embodiments, other embodiments obtained by those skilled in the art without creative work are all within the scope of protection of the present invention.

[0016] Example 1, as Figure 1It is an intelligent collaborative optimization method for source, grid, load and storage, which includes collecting data from four terminals: source, grid, load and storage, building a digital twin model of the microgrid, deploying multiple intelligent agents for reinforcement learning, calculating and generating local policy gradients, solving the global scheduling allocation plan, and issuing the strategy to the unit for iterative optimization. The following are the specific implementation steps: a method for intelligent collaborative optimization of source, grid, load and storage, which collects source, grid, load and storage data and builds a digital twin model of the microgrid. The specific steps for collecting data from the four terminals of source, grid, load and storage are as follows: install sensors in the power generation unit of the township-level microgrid to collect active power, reactive power, voltage and current in real time; deploy smart voltmeters in the distribution network unit of the township-level microgrid to collect voltage, line conditions, power and alarm information of the distribution network in real time; install smart meters in the user-side load units of the township-level microgrid, such as agricultural machinery irrigation pumps and electric vehicle charging piles, to collect power consumption, short-term power and load switch status in real time; configure energy storage units such as supercapacitor cabinets in the township microgrid, integrate high-precision current transformers, and collect charge status, single cell voltage, and temperature conditions during charging and discharging in real time; deploy a cloud data center in the control center of the township-level microgrid, and transmit the sensor data collected from the four terminals to the cloud data center through the MQTT protocol. The cloud data center synchronizes the clock with the sensor equipment through the PTP protocol to make the four terminals of source, grid, load and storage The data time series is consistent, and abnormal values and missing items are detected for sensor data. The data are aggregated according to a uniform size time window to form data with multiple sampling frequencies. The mean and variance of the data in the same time window are calculated for rapid trend monitoring. PCA dimensionality reduction is performed on high-dimensional data to generate low-dimensional feature vectors, which are written into the database and stored in tables according to different sampling frequencies. The specific steps for building a microgrid digital twin model are as follows: deploy edge digital twin nodes, collect parameters and sensor data of each end unit, and upload them to the edge digital twin nodes; according to the association and hierarchy of each end unit, a township-level microgrid electrical topology is constructed in the digital twin environment. At the same time, the average value inverter model is selected for the power generation unit, the π-type equivalent circuit model is selected for the distribution unit, the constant resistance model is selected for the controllable load unit, and the second-order RC battery model is selected for the energy storage unit. The collected parameters are mapped to the model library parameter interface, and simulation instances of each unit are generated. The sensor data is used for verification, and the least squares fitting is used to correct the deviation between the data of the simulation instance and the actual data. The parameter vector Store the parameters of the simulation instance, where Represents the parameters of different units, T is the matrix transpose symbol, and the actual data obtained at time t is recorded as , the error obtained at time t is ,in Represents the simulation data at time t, collects the square errors at j different times to construct the least squares objective function, and the formula is: , linearize the parameter vector to obtain the regression vector, merge the regression vectors at j different times to obtain the regression matrix, and obtain the parameter increment of each simulation instance according to the influence of the error value in the regression matrix on the simulation results, which is applied to the parameter correction of the simulation instance.

[0017] Based on the above steps, the specific steps of deploying multi-agents for reinforcement learning are as follows: call the digital twin model parameter interface, divide the four terminals of source, grid, load and storage into independent agents according to their responsibilities, define the data and status that need to be collected at each terminal, set the execution space of each terminal, set the incentive function, and the incentive items are composed of economy, stability and environmental protection. The formula of economic incentive item is: , where k represents the kth decision during scheduling, represents the normalization coefficient, Indicates the preset value of consumption cost, represents the consumption cost of the kth decision; the stability incentive formula is ,in represents the normalization coefficient of voltage deviation, represents the normalization coefficient of frequency deviation, represents the voltage value of the kth decision, Indicates the rated voltage value, represents the frequency value of the k-th decision, Indicates the rated frequency value; the environmental incentive formula is ,in represents the normalization coefficient, represents the environmental energy output of the k-th decision, Represents the total output of the k-th decision, and the above three items are combined into the overall incentive according to the weights: ,in , , are the weights of each item, and , the weights in the excitation function of each terminal unit are adjusted according to the actual application; MADDPG is selected as the reinforcement algorithm, the policy network is initialized, the parameter vector in the digital twin model is normalized and input into the policy network, the next decision is output according to the probability, the incentive value is calculated after execution, and the actual data and predicted instructions are interacted with the incentive value to form a loop; the specific steps for calculating and generating the local policy gradient are: the local agent collects data at a certain time u, records the interaction sequence of actual data, predicted instructions, and incentive values, performs outlier and normalization processing locally, generates an observation vector and inputs it into the policy network, the agent outputs the instruction distribution, maps the instructions into an instruction format that the device can recognize and execute, and calculates the real-time incentive value after the decision is executed at the current moment. , calculate the future incentive value of the decision execution for all subsequent moments , according to the attenuation coefficient Weighted summation gives weighted cumulative return , the formula is: , calculate the logarithmic probability gradient of the policy network output at this time, multiply the logarithmic probability gradient by the corresponding weighted cumulative reward to obtain the single-step gradient contribution. The single-step gradient contribution is regarded as a signal that the parameter adjustment direction can improve the reward. After collecting the single-step gradient contributions at different times, the average gradient is obtained and used to update the policy network parameters. The data of the intelligent agent at the next moment is collected to repeat the training.

[0018] Based on the above steps, the specific steps for solving the global dispatch allocation plan are as follows: the four-terminal intelligent entities of source, grid, load and storage complete an interactive cycle according to the latest strategy network, calculate the local policy gradient and generate local dispatch instructions. The township-level microgrid cloud data center pre-defines the structure, and transmits the dispatch instructions to the cloud data center according to the predefined structure. At the same time, the cloud data center presets the weight of each unit, collects the operating data and parameters of each unit, and after normalization, determines the relative importance of each unit in the overall system through the hierarchical analysis method, normalizes the calculated comprehensive score of each unit, assigns weights to each unit according to the comprehensive score, performs offline simulation in the digital twin model, analyzes sensitivity and global fluctuations, and adjusts the previous weights according to the actual impact; the cloud data center aggregates all dispatch instructions and assigns unique identifiers, performs weighted average calculations, and linearly combines them to obtain global dispatch instructions. , the formula is , where i represents unit i and n represents the total number of units, Represents the weight of the unit, the sum of the weights of each unit is 1, Represents the scheduling instructions of the unit; the specific steps of issuing the strategy to the unit for iterative optimization are: issuing the calculated global scheduling instructions to each end unit, each end unit takes out its own sub-scheduling instructions according to the scheduling instruction dimension and unique identifier, splits the scheduling instructions according to the predefined structure, and issues them to the specific execution device in real time. After executing the instructions, the real-time data and execution status are collected and sent back to the upper unit. The upper unit uniformly feedbacks the results to the cloud data center. The cloud data center identifies the unit's scheduling instructions according to the unique identifier, compares them with the actual results of the feedback, verifies the timing consistency, and calculates the deviation vector. The formula is ,in Indicates the dispatch instruction issued. Represents the actual result of the feedback, i represents unit i, and all deviation vectors are integrated to calculate the average deviation. The average deviation is passed to the digital twin model, and the parameters are updated using least squares fitting. The global scheduling instructions are then re-solved and sent to the units for execution, forming a closed-loop scheduling mechanism of "feedback-correction-optimization-solution-send-feedback".

[0019] Example 2, based on the above Example 1, is specifically the following solution in the actual application scenario of the source-grid-load-storage intelligent coordination method: Step 1: In the distribution room of the township-level microgrid, sensors are installed at each end of the photovoltaic inverter, wind turbine, energy storage cabinet, and pump station. They are connected to the edge gateway via Modbus-TCP. Optical fiber is used between the edge gateway and the cloud data center to achieve clock synchronization, controlling the latency to 5ms. The data is packaged into JSON format and reported to the cloud. The cloud generates a node path table based on the sensor data and the actual operating parameters of the photovoltaic inverter, wind turbine, energy storage cabinet, and pump station. The table is imported into the simulation platform to complete the electrical topology diagram and deploy the digital twin environment to each end device.

[0020] In the second step, the voltage and frequency values of the equipment are observed by the four-terminal intelligent agent to obtain the relevant values of the stability incentive item, the power and efficiency of each device when using different energy sources are counted to obtain the relevant values of the environmental incentive item, and the electricity and fuel consumed during the operation of the equipment are calculated to obtain the relevant values of the economic incentive item; the weight of the incentive function is set according to the different types of equipment and their importance in the system, and the incentive function is comprehensively evaluated from the three aspects of economy, stability and environmental protection to obtain the optimal solution. Each intelligent agent runs more than 1,000 interactions in the local twin environment, collects the average deviation, and updates the parameters in the policy network.

[0021] Step three: Every 10 minutes, the intelligent agent at each end will form an average instruction based on the latest 200 interactions running locally and send it to the cloud for aggregation. The cloud will weightedly fuse the instructions according to the different weights of the four ends, generate a global solution and send it to each end node. After executing the instruction, the edge device calculates the deviation value and reports the deviation vector to the cloud. The cloud uses least squares fitting to adjust the weight of the LSTM model in the digital twin and generates a global solution again in 10 minutes to form a feedback loop.

[0022] The above describes the embodiments of the present invention. Without departing from the embodiments of the present invention and its broader aspects, those skilled in the art may make data modifications and method changes based on the above in specific operations. The attached claims are intended to include all such data modifications and method changes in the implementation examples that do not depart from the present invention.

Claims

1. A method for intelligent collaborative optimization of sources, grids, loads, and storage, characterized by: Collecting data from the four terminals of the power source, grid, load, and storage system to build a digital twin model of the microgrid involves deploying sensors at each terminal, using a cloud data center to collect data in real time, aligning the time sequence of the data from the four terminals through the PTP protocol, building a topology based on the parameters and data, and selecting a model for simulation in the digital twin environment. Deploying multiple agents for reinforcement learning to calculate and generate local policy gradients involves dividing the four terminals of the source, grid, load, and storage into independent agents, setting observation data, execution space, and incentive feedback. Incentives are composed of economy, stability, and environmental protection. Actual data, predicted instructions, and incentive values are interactively looped and recorded as sequences for input into the policy network. The output logarithmic probability gradient is calculated to update the policy network parameters. Solving the global scheduling allocation plan and sending the strategy to the unit for iterative optimization means that the four-end intelligent entities of source, grid, load and storage complete the interactive cycle according to the strategy network, calculate and generate local scheduling instructions, and the cloud data center aggregates the instructions for weighted calculation. The global scheduling plan is generated and sent to the execution device, and the average deviation calculated by the deviation vector is passed to the digital twin model to update the parameters, and the global scheduling plan is re-solved and sent to form a closed-loop mechanism.

2. The method for intelligent collaborative optimization of source, grid, load and storage according to claim 1, characterized in that: The specific steps of collecting data from the four terminals of source, grid, load and storage are as follows: installing and deploying sensors at the four terminals of source, grid, load and storage, deploying a cloud data center in the control center, transmitting the sensor data collected by the four terminals to the cloud data center through the MQTT protocol, synchronizing the clocks of the cloud data center with the four-terminal devices of source, grid, load and storage through the PTP protocol to ensure that the time sequence of the data from the four terminals of source, grid, load and storage is consistent, detecting abnormal values and missing items in the sensor data, aggregating them according to a unified time window to form data with multiple sampling frequencies, calculating the mean and variance of the time window data, performing rapid trend monitoring, performing PCA dimensionality reduction on the high-dimensional data, generating low-dimensional feature vectors, and writing them into the database for storage in tables divided into different sampling frequencies; The specific steps of constructing the microgrid digital twin model are as follows: deploying edge digital twin nodes, collecting parameters and sensor data of each end unit, and uploading them to the edge digital twin nodes; constructing a microgrid electrical topology diagram in the digital twin environment based on the association and hierarchy of each end unit, selecting corresponding models for the four-end units, mapping the collected parameters to the model library parameter interface, generating simulation instances of each unit, verifying them using sensor data, and using least squares fitting to correct the deviation between the data of the simulation instance and the actual data.

3. The method for intelligent collaborative optimization of source, grid, load and storage according to claim 2, characterized in that: The specific steps of constructing the microgrid digital twin model also include: using parameter vector Store the parameters of the simulation instance, where Represents the parameters of different units, T is the matrix transpose symbol, and the actual data obtained at time t is recorded as , the error obtained at time t is ,in Represents the simulation data at time t, collects the square errors at j different times to construct the least squares objective function, and the formula is: , linearize the parameter vector to obtain the regression vector, merge the regression vectors at j different times to obtain the regression matrix, and obtain the parameter increment of each simulation instance according to the influence of the error value in the regression matrix on the simulation results, which is applied to the parameter correction of the simulation instance.

4. The method for intelligent collaborative optimization of source, grid, load and storage according to claim 1, characterized in that: The specific steps of deploying multiple agents for reinforcement learning are as follows: calling the digital twin model parameter interface, dividing the four terminals of source, grid, load and storage into independent agents according to their responsibilities, defining the observation data of each terminal respectively, setting the execution space of each terminal, setting the incentive function, and forming the incentive items by economy, stability and environmental protection. The formula of economic incentive item is: , where k represents the kth decision during scheduling, represents the normalization coefficient, Indicates the preset value of consumption cost, Represents the consumption cost of the k-th decision.

5. The method for intelligent collaborative optimization of source, grid, load and storage according to claim 4, characterized in that: The specific steps of deploying multiple agents for reinforcement learning also include: the stability incentive formula is ,in represents the normalization coefficient of voltage deviation, represents the normalization coefficient of frequency deviation, represents the voltage value of the kth decision, Indicates the rated voltage value, represents the frequency value of the k-th decision, Indicates the rated frequency value.

6. The method for intelligent collaborative optimization of source, grid, load and storage according to claim 4, characterized in that: The specific steps of deploying multiple agents for reinforcement learning also include: the environmental incentive formula is ,in represents the normalization coefficient, represents the environmental energy output of the k-th decision, It represents the total output of the k-th decision, and combines the economy, stability, and environmental protection into an overall incentive according to their weights: ,in , , are the weights of each item, and .

7. The method for intelligent collaborative optimization of source, grid, load and storage according to claim 4, characterized in that: The specific steps of deploying multiple agents for reinforcement learning also include: selecting MADDPG as the reinforcement algorithm, initializing the policy network, normalizing the parameter vector in the digital twin model and inputting it into the policy network, outputting the next decision based on the probability, calculating the incentive value after execution, and interacting the actual data and predicted instructions with the incentive value to form a loop.

8. The method for intelligent collaborative optimization of source, grid, load and storage according to claim 1, characterized in that: The specific steps for calculating and generating local policy gradients are as follows: the local agent collects data at a certain time u, records the interaction sequence of actual data, predicted instructions, and incentive values, performs outlier and normalization processing locally, generates observation vectors and inputs them into the policy network, the agent outputs the instruction distribution, maps the instructions into an instruction format that the device can recognize and execute, and calculates the real-time incentive value after the decision is executed at the current moment. , calculate the future incentive value of the decision execution for all subsequent moments , according to the attenuation coefficient Weighted summation gives weighted cumulative return , the formula is: , calculate the logarithmic probability gradient of the policy network output at this time, multiply the logarithmic probability gradient by the corresponding weighted cumulative reward to obtain the single-step gradient contribution, and after collecting the single-step gradient contributions at different times, calculate the average gradient to update the policy network parameters, and collect the data of the intelligent agent at the next moment to repeat the training.

9. The method for intelligent collaborative optimization of source, grid, load and storage according to claim 1, characterized in that: The specific steps for solving the global scheduling allocation plan are as follows: the four-terminal intelligent agents of source, grid, load and storage complete an interactive cycle according to the latest policy network, calculate the local policy gradient and generate local scheduling instructions. The cloud data center predefines the structure and transmits the scheduling instructions to the cloud data center according to the predefined structure. The cloud data center presets the weight of each unit, collects the operating data and parameters of each unit, and after normalization, determines the relative importance of each unit in the overall system through the hierarchical analysis method. The normalized comprehensive score of each unit is calculated, and each unit is weighted according to the comprehensive score. Offline simulation is performed in the digital twin model to analyze sensitivity and global fluctuations, and the previous weights are adjusted according to the actual impact. The cloud data center aggregates all scheduling instructions and assigns unique identifiers, performs weighted average calculations, and linearly combines them to obtain global scheduling instructions. , the formula is , where i represents unit i and n represents the total number of units, Represents the weight of the unit, the sum of the weights of each unit is 1, Represents the scheduling instructions for the unit.

10. The method for intelligent collaborative optimization of source, grid, load and storage according to claim 1, characterized in that: The specific steps of issuing the strategy to the unit for iterative optimization are as follows: issuing the calculated global scheduling instruction to each end unit, each end unit takes out its own sub-scheduling instruction according to the scheduling instruction dimension and unique identifier, splits the scheduling instruction according to the predefined structure, and issues it to the specific execution device in real time. After executing the instruction, the real-time data and execution status are collected and sent back to the upper unit. The upper unit uniformly feedbacks the results to the cloud data center. The cloud data center identifies the unit's scheduling instruction according to the unique identifier, compares it with the actual result of the feedback, verifies the timing consistency, and calculates the deviation vector. The formula is ,in Indicates the dispatch instruction issued. It represents the actual result of the feedback, i represents unit i, and all deviation vectors are integrated to calculate the average deviation. The average deviation is passed to the digital twin model, and the parameters are updated using least squares fitting. The global scheduling instructions are then re-solved and sent to the units for execution, forming a closed-loop scheduling mechanism.

Images