A multi-agent-based microgrid group cooperative control method and system

Abstract

The application belongs to the technical field of micro-grid operation control, and provides a micro-grid group cooperative control method and system based on multiple agents. Through preprocessing and truncation grading processing of the energy storage charge quantity range and the net load range of each micro-grid, the target energy storage state space corresponding to each micro-grid is obtained, and then the individual Q table and the common Q table can be constructed based on the relationship between the state index of the target energy storage state space and the target action state space. Through comprehensive instant rewards, the relationship between the buying and selling electricity income of the micro-grid and other power grids under different net loads, and the energy storage charge quantity deviation degree when different state actions are executed are quantified, and then the training and iteration process of the individual Q table can be constrained in cooperation with the common Q table, so as to obtain the individual Q table which can represent the optimal action state that can be executed by the micro-grid under different energy storage states, so that the cooperative regulation scheme which can consider the rationality of resource allocation and the stability requirement of system operation can be obtained.

Description

Technical Field

[0001] This application belongs to the field of microgrid operation and control technology, and more specifically, relates to a microgrid group cooperative control method and system based on multi-agent systems. Background Technology

[0002] With the large-scale integration of distributed generation equipment utilizing renewable energy, microgrids have been widely used as an efficient and flexible form of power generation and distribution organization. Microgrid clusters formed by interconnecting multiple microgrids can achieve power sharing, effectively improving the system's power supply reliability and renewable energy absorption capacity. However, the strong randomness and intermittency of renewable energy power generation mean that microgrid clusters need to simultaneously address the issues of rational resource allocation and system operational stability during operation.

[0003] Currently, collaborative control methods for microgrid clusters mainly include centralized optimization, game theory-based distributed scheduling, and multi-agent system-based collaborative control. Centralized methods rely on global information and high-speed communication between microgrids, resulting in high computational complexity, single-point failures, and information privacy issues. While game theory-based methods are beneficial for distributed decision-making, they suffer from long adjustment times and poor dynamic anti-interference capabilities. Furthermore, existing multi-agent-based methods, when performing resource allocation and system operation collaborative control, are essentially based on centralized optimization's high-speed communication and game theory-based distributed scheduling, using agents to replace pre-set complex scheduling strategies. However, they still have the following problems: 1. Real-time acquisition of high-speed communication between microgrids is required; single-point failures in some microgrids can hinder overall control, easily leading to an imbalance between autonomous decision-making and group collaborative decision-making in individual microgrids; 2. The distributed processing of resource allocation and system operation stability objectives can easily result in a serious bias in the final collaborative control scheme. Summary of the Invention

[0004] To address the aforementioned deficiencies in existing technologies, this application provides a microgrid group collaborative control method and system based on multi-agent systems, aiming to solve the following problems in existing microgrid group collaborative control methods: 1. Imbalance between autonomous decision-making and group collaborative decision-making of individual microgrids caused by single-point failures during communication; 2. The control scheme cannot simultaneously meet the requirements of rational resource allocation and system operational stability due to the distributed processing of multiple control objectives to generate control schemes.

[0005] Firstly, this application provides a microgrid group cooperative control method based on multi-agent systems, including: S1. Preprocess and truncate the energy storage capacity range and net load range of each microgrid to obtain the target energy storage state space corresponding to each microgrid. Based on the types of action states corresponding to each microgrid, construct the target action state space corresponding to each microgrid. Both the target energy storage state space and the target action state space include multiple state indices. S2. Based on the energy storage state index and the action state index of the target action state space corresponding to each microgrid, construct the individual Q table and the common Q table corresponding to each microgrid, and obtain the comprehensive real-time reward of each microgrid under different energy storage state index and action state index. The initial value of each Q value in the individual Q table and the common Q table is 0. The two axes of the individual Q table and the common Q table are respectively composed of the action state index and the energy storage state index. S3. Based on the comprehensive real-time reward and common Q-table corresponding to each microgrid, the individual Q-table of each microgrid is iteratively trained through the target action selection algorithm; S4. Obtain the target energy storage state index corresponding to the energy storage load and net load of each microgrid in the current time period, and select the target action state index corresponding to the target energy storage state index from each body Q table after training. S5. Generate a coordinated control scheme based on the action state corresponding to each target action state index, and perform coordinated control on the microgrid corresponding to each target energy storage state index.

[0006] Furthermore, the energy storage capacity range and net load range of each microgrid are preprocessed and truncated in a hierarchical manner to obtain the target energy storage state space corresponding to each microgrid, including: The energy storage capacity range of the microgrid is mapped to the target range, and the mapped energy storage capacity range is truncated, rounded, and graded to obtain the first target level of multiple levels corresponding to the energy storage capacity range. The net load range of the microgrid is normalized, and the normalized net load range is mapped to the target range. The mapped net load range is then truncated, rounded, and graded to obtain the second target level corresponding to multiple levels of the net load range. Each first target level and second target level is paired one by one according to its level to obtain a target energy storage state space including multiple energy storage state indices. The total number of energy storage state indices is the product of the highest level number corresponding to the first target level and the second target level.

[0007] Furthermore, the types of operating states include full-power discharge, low-rated-power discharge, holding, low-rated-power charging, full-power charging, selling electricity to a neighboring grid, and purchasing electricity from a neighboring grid.

[0008] Furthermore, the function expression for the comprehensive instant reward is: ; in, To provide comprehensive instant rewards, This represents the revenue from electricity purchases and sales between the microgrid and the main grid, corresponding to the net load of the microgrid under the current energy storage state index. This represents the revenue from electricity purchases and sales with the neighboring grid corresponding to the net load of the microgrid under the current energy storage state index. This represents the penalty coefficient for the microgrid's actions under the current action state index. This represents the energy storage capacity of the microgrid under the current energy storage status index.

[0009] Furthermore, the target action selection algorithm is the epsilon-greedy algorithm.

[0010] Furthermore, the individual Q-tables of each microgrid are iteratively trained using a target action selection algorithm, including: S31. The comprehensive instant rewards corresponding to the microgrid are used as the Q values ​​of the corresponding items of the comprehensive instant rewards in the individual Q table and the common Q table; S32. Randomly select an energy storage state index from the target energy storage state space as the first energy storage state index, and match the first action state index with the first energy storage state index using the target action selection algorithm; S33. Obtain the microgrid in the energy storage state corresponding to the first energy storage state index, the first energy storage charge and the first net load after the first action state corresponding to the first action state index are executed, and obtain the second energy storage state index corresponding to the first energy storage charge and the first net load. S34. Obtain the maximum Q value corresponding to the second energy storage state index and the second action state index corresponding to the maximum Q value from the individual Q table, and obtain the I2Q target value based on the Q values ​​of the individual Q table and the common Q table corresponding to the second energy storage state index and the second action state index. S35. Update the Q values ​​of the individual Q table and the common Q table corresponding to the first energy storage state index and the first action state index based on the I2Q target value; S36. Repeat step S32 until the preset number of iterations is reached.

[0011] Furthermore, the formula for obtaining the I2Q target value is expressed as follows: ; Where target is the I2Q target value. To provide comprehensive instant rewards, The discount factor for the individual Q table. This is the index for the second action state. This is the index for the second energy storage state. For the first i microgrids and The Q-values ​​of the common Q-table. For the first i The maximum Q value corresponding to the individual Q table of each microgrid under the second energy storage state index.

[0012] Secondly, this application also provides a multi-agent-based microgrid group cooperative control system for executing any of the methods in this aspect, including: The energy storage state space construction module is used to preprocess and truncate the energy storage capacity range and net load range of each microgrid to obtain the target energy storage state space corresponding to each microgrid. The action state space construction module is used to construct the target action state space for each microgrid based on the types of action states corresponding to each microgrid. The Q-table construction module is used to construct individual Q-tables and common Q-tables for each microgrid based on the energy storage state indexes of the target energy storage state space and the action state indexes of the target action state space corresponding to each microgrid, and to obtain the comprehensive real-time rewards of each microgrid under different energy storage state indexes and action state indexes. The Q-table training module is used to iteratively train the individual Q-tables of each microgrid based on the comprehensive real-time reward and common Q-table corresponding to each microgrid, through the target action selection algorithm. The coordinated control module is used to obtain the target energy storage state index corresponding to the energy storage load and net load of each microgrid in the current period, select the target action state index corresponding to the target energy storage state index from the trained Q table of each body, and finally generate a coordinated control scheme based on the action state corresponding to each target action state index, and perform coordinated control on the microgrid corresponding to each target energy storage state index.

[0013] Thirdly, this application also provides an electronic device, characterized in that it comprises: At least one memory for storing computer programs; At least one processor is configured to execute a program stored in the memory, such that, when the program is executed, the processor performs the method described in the first aspect or any possible implementation thereof.

[0014] Overall, the technical solutions conceived in this application have the following beneficial effects compared with the prior art: This application preprocesses and truncates the energy storage load range and net load range of each microgrid to obtain the target energy storage state space for each microgrid. Then, based on the relationship between the state indices of the target energy storage state space and the target action state space, individual Q-tables and a common Q-table can be constructed, effectively quantifying the correlation between the energy storage state and the action state of the microgrid. By comprehensively considering real-time rewards, the application quantifies the electricity purchase and sale revenue relationship between the microgrid and other grids under different net loads, as well as the deviation of the energy storage load when performing different state actions. This, combined with the common Q-table, constrains the training and iteration process of the individual Q-tables, resulting in individual Q-tables that characterize the optimal action states that the microgrid can execute under different energy storage states. This leads to a collaborative control scheme that balances the rationality of resource allocation and the stability of system operation.

[0015] Furthermore, by using a common Q-table and comprehensive instant rewards to constrain the training iteration process of individual Q-tables, the impact of neighboring microgrids and the main grid on the system stability and resource allocation of the current microgrid can be taken into account even without communication. This is beneficial to improving the overall resource allocation rationality and system operation stability of the microgrid group, while also solving the imbalance between autonomous decision-making and group collaborative decision-making of individual microgrids caused by single-point failures during communication. Attached Figure Description

[0016] To more clearly illustrate the technical solutions in this application or related technologies, the accompanying drawings used in the description of the embodiments or related technologies will be briefly introduced below. Obviously, the accompanying drawings described below are some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0017] Figure 1 This is a flowchart illustrating the microgrid group collaborative control method provided in an embodiment of this application.

[0018] Figure 2 This is a schematic diagram of the structure of a microgrid group provided in an embodiment of this application.

[0019] Figure 3 This is a schematic diagram of the structure of the microgrid group collaborative control system provided in the embodiments of this application.

[0020] Figure 4 This is a schematic diagram of the structure of an electronic device provided in an embodiment of this application. Detailed Implementation

[0021] The technical solutions in the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings.

[0022] In the following description, the terms "second" and "second" are used for descriptive purposes only and should not be construed as indicating or implying relative importance. The following description provides multiple embodiments of this application, which can be substituted or combined with each other. Therefore, this application can also be considered to include all possible combinations of the same and / or different embodiments described. Thus, if one embodiment includes features A, B, and C, and another embodiment includes features B and D, then this application should also be considered to include embodiments containing one or more other possible combinations of A, B, C, and D, even if such embodiments are not explicitly described in the following text.

[0023] The following description provides examples and does not limit the scope, applicability, or examples set forth in the claims. Changes may be made to the function and arrangement of the described elements without departing from the scope of this application. Various processes or components may be appropriately omitted, substituted, or added to the examples. For example, the described methods may be performed in a different order than described, and various steps may be added, omitted, or combined. Furthermore, features described with respect to some examples may be combined into other examples.

[0024] Figure 1 This is a flowchart illustrating a microgrid group cooperative control method based on multi-agent systems provided in this application. The method includes the following steps: S1. Preprocess and truncate the energy storage capacity range and net load range of each microgrid to obtain the target energy storage state space corresponding to each microgrid. Based on the types of action states corresponding to each microgrid, construct the target action state space corresponding to each microgrid.

[0025] In one possible implementation, the energy storage capacity range and net load range of each microgrid are preprocessed and truncated in a hierarchical manner to obtain the target energy storage state space corresponding to each microgrid, including: The energy storage capacity range of the microgrid is mapped to the target range, and the mapped energy storage capacity range is truncated, rounded, and graded to obtain the first target level of multiple levels corresponding to the energy storage capacity range. The net load range of the microgrid is normalized, and the normalized net load range is mapped to the target range. The mapped net load range is then truncated, rounded, and graded to obtain the second target level corresponding to multiple levels of the net load range. Each first target level and second target level is paired one by one according to its level to obtain a target energy storage state space including multiple energy storage state indices. The total number of energy storage state indices is the product of the highest level number corresponding to the first target level and the second target level.

[0026] In this embodiment of the application, the implementing entity of the method is an intelligent agent used to control the operation of each microgrid. Each microgrid is configured with an independent intelligent agent, such as... Figure 2 As shown, a microgrid cluster consists of multiple microgrids, each with an independent intelligent agent for data processing and control of operational states. Each microgrid integrates photovoltaic (PV) power generation equipment, wind power generation equipment, local loads, and energy storage equipment. These devices are connected to the external power grid's PCC via a busbar, allowing for electricity purchase and sale between microgrids. The energy storage load of a microgrid is represented by the SOC (State of Charge) of the energy storage equipment, while the net load is the total load of the entire microgrid minus the output of each power generation device (PV and wind power). Due to the microgrid structure and parameters, both the net load and the energy storage load have corresponding fixed ranges, which can be mapped to the same target range to construct a target energy storage state space. This facilitates the construction of a Q-table to characterize the quantitative relationship between the energy storage state and the operational states to be executed.

[0027] An example of the process for constructing the energy storage state index of the target energy storage state space is as follows: Map the SOC of the energy storage device from [0, 1] to integers [1, 5] (SOC = 0.1 → 1, SOC = 0.9 → 5). Then, truncate using min(5, max(1, ...)) to ensure the range, thus dividing the SOC into five levels (first target levels). Next, linearly transform the net load from [-1, 1] to [0, 1], then multiply by (net load + 1) / 2, and round up to obtain integers [1, 5]. This divides the net load into five levels (second target levels). Finally, pair each first target level with a second target level according to their level to obtain 25 energy storage state indices.

[0028] The types of action states in the target action state space include full-power discharge, low-rated-power discharge, holding, low-rated-power charging, full-power charging, selling electricity to a neighboring grid, and purchasing electricity from a neighboring grid. The microgrid's intelligent agents control the energy storage power of energy storage devices to complete the corresponding action states, thereby adjusting the microgrid according to actual needs. Specific adjustment schemes can be autonomously planned by the intelligent agents. The method in this application is only used to obtain the energy storage state index corresponding to the microgrid in the current time period, and to obtain the optimal target action state index from the individual Q-table trained by this method based on the energy storage state index, finally obtaining the optimal combination of energy storage state index and action state index.

[0029] For examples of action state indices and corresponding action states, please refer to the table below: Table 1

[0030] For ease of understanding, this application provides an individual Q-table consisting of 25 energy storage state indices and 7 action state indices: Table 2

[0031] S2. Based on the energy storage state index and the action state index of the target action state space corresponding to each microgrid, construct the individual Q table and the common Q table corresponding to each microgrid, and obtain the comprehensive real-time reward of each microgrid under different energy storage state index and action state index.

[0032] In this embodiment, the initial value of each Q value in the individual Q table and the common Q table is 0. The two axes of the individual Q table and the common Q table are respectively composed of the action state index and the energy storage state index. Each item in the Q table represents the Q value corresponding to a combination of action and energy storage state index.

[0033] The function expression for the comprehensive instant reward is: ; in, To provide comprehensive instant rewards, This represents the revenue from electricity purchases and sales between the microgrid and the main grid, corresponding to the net load of the microgrid under the current energy storage state index. This represents the revenue from electricity purchases and sales with the neighboring grid corresponding to the net load of the microgrid under the current energy storage state index. This represents the penalty coefficient for the microgrid's actions under the current action state index. This represents the energy storage capacity of the microgrid under the current energy storage status index.

[0034] Action penalty coefficient It is directly related to the current action state index, and its expression is: ; in, ∈{1,2,…,7} represents the weight factor corresponding to different action state indices. As the baseline penalty coefficient, different action state indices correspond to different weighting factors. For example, when the action state index corresponds to full-power discharge or charging, its weighting factor is 1.5 (full power will aggravate the SOC deviation and damage the battery of the energy storage device, so a larger weighting factor is selected); when the action state is low-rated power discharge or charging, its weighting factor is 0.8; the specific weighting factor correspondence can be preset according to the specific situation of the microgrid, which is conducive to improving the system operation stability of the microgrid.

[0035] S3. Based on the comprehensive real-time reward and common Q-table corresponding to each microgrid, the individual Q-table of each microgrid is iteratively trained through the target action selection algorithm.

[0036] In one possible implementation, the individual Q-tables of each microgrid are iteratively trained using a target action selection algorithm, including: S31. The comprehensive instant rewards corresponding to the microgrid are used as the Q values ​​of the corresponding items of the comprehensive instant rewards in the individual Q table and the common Q table; S32. Randomly select an energy storage state index from the target energy storage state space as the first energy storage state index, and match the first action state index with the first energy storage state index using the target action selection algorithm; S33. Obtain the microgrid in the energy storage state corresponding to the first energy storage state index, the first energy storage charge and the first net load after the first action state index is executed, and obtain the second energy storage state index corresponding to the first energy storage charge and the first net load. S34. Obtain the maximum Q value corresponding to the second energy storage state index and the second action state index corresponding to the maximum Q value from the individual Q table, and obtain the I2Q target value based on the Q values ​​of the individual Q table and the common Q table corresponding to the second energy storage state index and the second action state index. S35. Update the Q values ​​of the individual Q table and the common Q table corresponding to the first energy storage state index and the first action state index based on the I2Q target value; S36. Repeat step S32 until the preset number of iterations is reached.

[0037] In this embodiment, the target action selection algorithm is the epsilon-greedy algorithm. During each training iteration, the agent randomly selects an action state index with probability ε based on the current energy storage state index and executes the corresponding action state. The purpose of using the epsilon-greedy algorithm is to allow the agent to try different scheduled actions while gradually favoring actions with higher overall immediate rewards during reinforcement learning training of the individual Q-table.

[0038] The formula for obtaining the I2Q target value is expressed as follows: ; Where target is the I2Q target value. To provide comprehensive instant rewards, The discount factor for the individual Q table. This is the index for the second action state. This is the index for the second energy storage state. For the first i microgrids and The Q-values ​​of the common Q-table. For the first i The maximum Q value corresponding to the individual Q table of each microgrid under the second energy storage state index.

[0039] During one iteration, the formula for updating the individual Q-table is: ; The common Q-table update formula is: ; in, and The learning rates are for the individual Q-tables and the shared Q-table, respectively. The learning rate of the shared Q-table is much smaller than that of the individual Q-table. The individual Q-table exhibits slow mean convergence in its updates, providing a stable reference for each iteration. Due to its slow updates, it does not change drastically due to fluctuations in single-step rewards, thus guiding individual Q-table updates in a long-term consistent direction. Furthermore, since there is no communication between microgrids, the shared Q-table, combined with integrated real-time rewards, incorporates the constraints of neighboring microgrids and the overall grid on the system stability and resource allocation of the current microgrid into the training process, which is beneficial for improving the overall resource allocation rationality and system operational stability of the microgrid group.

[0040] S4. Obtain the target energy storage state index corresponding to the energy storage load and net load of each microgrid in the current time period, and select the target action state index corresponding to the target energy storage state index from each body Q table after training.

[0041] S5. Generate a coordinated control scheme based on the action state corresponding to each target action state index, and perform coordinated control on the microgrid corresponding to each target energy storage state index.

[0042] In this embodiment of the application, the intelligent system obtains the energy storage load and net load according to the energy storage load and net load of the corresponding microgrid, obtains the first target level corresponding to the energy storage load and the second target level corresponding to the net load, and obtains the target energy storage state index by matching from the target energy storage state space based on the first target level and the second target level.

[0043] Then, the largest Q-value item corresponding to the target energy storage state index is selected from the trained individual Q-table, and the action state index corresponding to the largest Q-value item is used as the target action state index. Finally, each agent controls each microgrid to execute the action state corresponding to the target action state index, thereby realizing the coordinated control of the microgrid group.

[0044] Figure 3 This is a schematic diagram of the structure of a microgrid group cooperative control system based on multi-agent systems provided in an embodiment of this application, as shown below. Figure 3 As shown, the system includes at least: The energy storage state space construction module is used to preprocess and truncate the energy storage capacity range and net load range of each microgrid to obtain the target energy storage state space corresponding to each microgrid. The action state space construction module is used to construct the target action state space for each microgrid based on the types of action states corresponding to each microgrid. The Q-table construction module is used to construct individual Q-tables and common Q-tables for each microgrid based on the energy storage state indexes of the target energy storage state space and the action state indexes of the target action state space corresponding to each microgrid, and to obtain the comprehensive real-time rewards of each microgrid under different energy storage state indexes and action state indexes. The Q-table training module is used to iteratively train the individual Q-tables of each microgrid based on the comprehensive real-time reward and common Q-table corresponding to each microgrid, through the target action selection algorithm. The coordinated control module is used to obtain the target energy storage state index corresponding to the energy storage load and net load of each microgrid in the current period, select the target action state index corresponding to the target energy storage state index from the trained Q table of each body, and finally generate a coordinated control scheme based on the action state corresponding to each target action state index, and perform coordinated control on the microgrid corresponding to each target energy storage state index.

[0045] like Figure 4 As shown, Figure 4 This is a schematic diagram of the structure of an electronic device provided in an embodiment of this application. The electronic device may include: a processor 401, a communications interface 402, a memory 403, and a communication bus 404. The processor 401, communications interface 402, and memory 403 communicate with each other via the communication bus 404. The processor 401 can call software instructions in the memory 403 to execute the methods described in the above embodiments.

[0046] Furthermore, the logical instructions in the aforementioned memory 403 can be implemented as a software functional device and sold or used as an independent product, and can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, or part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, a server, or a network device, etc.) to execute all or part of the steps of the methods of the various embodiments of this application.

[0047] Based on the methods in the above embodiments, this application provides a computer-readable storage medium storing a computer program that, when run on a processor, causes the processor to execute the methods in the above embodiments.

[0048] Based on the methods in the above embodiments, this application provides a computer program product that, when run on a processor, causes the processor to execute the methods in the above embodiments.

[0049] It is understood that the processor in the embodiments of this application can be a CPU (Central Processing Unit), or other general-purpose processors, DSPs (Digital Signal Processors), ASICs (Application Specific Integrated Circuits), FPGAs (Field Programmable Gate Arrays), or other programmable logic devices, transistor logic devices, hardware components, or any combination thereof. A general-purpose processor can be a microprocessor or any conventional processor.

[0050] The method steps in this application embodiment can be implemented in hardware or by a processor executing software instructions. The software instructions can consist of corresponding software modules, which can be stored in random access memory (RAM), flash memory, ROM (Read-only Memory), PROM (Programmable ROM), EPROM (Erasable PROM), EEPROM (Electrically Erasable EPROM), registers, hard disks, portable hard disks, CD-ROMs, or any other form of storage medium known in the art. An exemplary storage medium is coupled to the processor, enabling the processor to read information from and write information to the storage medium. Of course, the storage medium can also be a component of the processor. The processor and the storage medium can reside in an ASIC.

[0051] In the above embodiments, implementation can be achieved, in whole or in part, through software, hardware, firmware, or any combination thereof. When implemented in software, it can be implemented, in whole or in part, as a computer program product. A computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, all or part of the flow or function according to the embodiments of this application is generated. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions can be stored in or transmitted through a computer-readable storage medium. The computer instructions can be transmitted from one website, computer, server, or data center to another website, computer, server, or data center via wired (e.g., coaxial cable, fiber optic, digital subscriber line DSL) or wireless (e.g., infrared, wireless, microwave, etc.). The computer-readable storage medium can be any available medium that a computer can access or a data storage device such as a server or data center that integrates one or more available media. The available medium can be a magnetic medium (e.g., floppy disk, hard disk, magnetic tape), an optical medium (e.g., DVD), or a semiconductor medium (e.g., SSD (Solid State Disk)).

[0052] It is understood that the various numerical designations used in the embodiments of this application are merely for the convenience of description and are not intended to limit the scope of the embodiments of this application.

[0053] Those skilled in the art will readily understand that the above are merely preferred embodiments of this application and are not intended to limit this application. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of this application should be included within the scope of protection of this application.

Claims

1. A microgrid group cooperative control method based on multi-agent systems, characterized in that, include: S1. The energy storage capacity range and net load range of each microgrid are preprocessed and truncated and graded to obtain the target energy storage state space corresponding to each microgrid. Based on the types of action states corresponding to each microgrid, the target action state space corresponding to each microgrid is constructed. Both the target energy storage state space and the target action state space include multiple state indices. S2. Based on the energy storage state index and the action state index of the target action state space corresponding to each microgrid, construct the individual Q table and the common Q table corresponding to each microgrid, and obtain the comprehensive real-time reward of each microgrid under different energy storage state indexes and action state indices. The initial value of each Q value in the individual Q table and the common Q table is 0. The two axes of the individual Q table and the common Q table are respectively composed of the action state index and the energy storage state index. S3. Based on the comprehensive real-time reward and common Q-table corresponding to each microgrid, the individual Q-table of each microgrid is iteratively trained using a target action selection algorithm; S4. Obtain the target energy storage state index corresponding to the energy storage load and net load of each microgrid in the current time period, and select the target action state index corresponding to the target energy storage state index from the individual Q table after training. S5. Generate a coordinated control scheme based on the action state corresponding to each of the target action state indices, and perform coordinated control on the microgrid corresponding to each of the target energy storage state indices.

2. The microgrid group cooperative control method according to claim 1, characterized in that, The process of preprocessing and truncation / grading the energy storage capacity range and net load range of each microgrid to obtain the target energy storage state space corresponding to each microgrid includes: The energy storage capacity range of the microgrid is mapped to the target range, and the mapped energy storage capacity range is truncated, rounded, and graded to obtain the first target level of multiple levels corresponding to the energy storage capacity range. The net load range of the microgrid is normalized, and the normalized net load range is mapped to the target range. The mapped net load range is then truncated, rounded, and graded to obtain the second target level corresponding to the multiple levels of the net load range. Each of the first target gears and the second target gears is paired one by one according to the level to obtain a target energy storage state space including multiple energy storage state indices. The total number of energy storage state indices is the product of the highest level number corresponding to the first target gear and the second target gear.

3. The microgrid group cooperative control method according to claim 1, characterized in that, The types of operating states include full-power discharge, low-rated-power discharge, holding, low-rated-power charging, full-power charging, selling electricity to a neighboring grid, and purchasing electricity from a neighboring grid.

4. The microgrid group cooperative control method according to claim 2, characterized in that, The functional expression for the comprehensive instant reward is: ； in, To provide comprehensive instant rewards, This represents the revenue from electricity purchases and sales between the microgrid and the main grid, corresponding to the net load of the microgrid under the current energy storage state index. This represents the revenue from electricity purchases and sales with the neighboring grid corresponding to the net load of the microgrid under the current energy storage state index. This represents the penalty coefficient for the microgrid's actions under the current action state index. This represents the energy storage capacity of the microgrid under the current energy storage status index.

5. The microgrid group cooperative control method according to claim 1, characterized in that, The target action selection algorithm is the epsilon-greedy algorithm.

6. The microgrid group cooperative control method according to claim 1, characterized in that, The iterative training of the individual Q-tables of each microgrid using a target action selection algorithm includes: S31. The comprehensive instant reward corresponding to each microgrid is used as the Q value of the item corresponding to the comprehensive instant reward in the individual Q table and the common Q table; S32. Randomly select an energy storage state index from the target energy storage state space as the first energy storage state index, and match the first action state index with the first energy storage state index using the target action selection algorithm; S33. Obtain the microgrid in the energy storage state corresponding to the first energy storage state index, the first energy storage charge and the first net load after the first action state index is executed, and obtain the second energy storage state index corresponding to the first energy storage charge and the first net load. S34. Obtain the maximum Q value corresponding to the second energy storage state index and the second action state index corresponding to the maximum Q value from the individual Q table, and obtain the I2Q target value based on the Q values ​​of the individual Q table and the common Q table corresponding to the second energy storage state index and the second action state index. S35. Update the Q values ​​of the individual Q table and the common Q table corresponding to the first energy storage state index and the first action state index based on the I2Q target value; S36. Repeat step S32 until the preset number of iterations is reached.

7. The microgrid group cooperative control method according to claim 1, characterized in that, The formula for obtaining the I2Q target value is expressed as follows: ； Where target is the I2Q target value. To provide comprehensive instant rewards, The discount factor for the individual Q table. This is the index for the second action state. This is the index for the second energy storage state. For the first i microgrids and The Q-values ​​of the common Q-table. For the first i The maximum Q value corresponding to the individual Q table of each microgrid under the second energy storage state index.

8. A multi-agent-based microgrid group cooperative control system, used to execute the method as described in any one of claims 1-7, characterized in that, include: The energy storage state space construction module is used to preprocess and truncate the energy storage capacity range and net load range of each microgrid to obtain the target energy storage state space corresponding to each microgrid. An action state space construction module is used to construct a target action state space for each microgrid based on the types of action states corresponding to each microgrid. The Q-table construction module is used to construct individual Q-tables and common Q-tables for each microgrid based on the energy storage state indexes of the target energy storage state space and the action state indexes of the target action state space corresponding to each microgrid, and to obtain the comprehensive real-time rewards of each microgrid under different energy storage state indexes and action state indexes. The Q-table training module is used to iteratively train the individual Q-table of each microgrid based on the comprehensive instant reward and common Q-table corresponding to each microgrid, using a target action selection algorithm. The coordinated control module is used to obtain the target energy storage state index corresponding to the energy storage load and net load of each microgrid in the current time period, select the target action state index corresponding to the target energy storage state index from the trained individual Q table, and finally generate a coordinated control scheme based on the action state corresponding to each target action state index, and perform coordinated control on the microgrid corresponding to each target energy storage state index.

9. An electronic device, characterized in that, include: At least one memory for storing computer programs; At least one processor is configured to execute a program stored in the memory, wherein when the program stored in the memory is executed, the processor is configured to perform the method as described in any one of claims 1-7.

10. A computer-readable storage medium having a computer program stored thereon, the computer-readable storage medium storing instructions that, when executed on a computer or processor, cause the computer or processor to perform the method as described in any one of claims 1-7.

Images