Micro-grid dynamic cooperative scheduling method based on multi-mode reinforcement learning

Abstract

The invention discloses a micro-grid dynamic collaborative scheduling method based on multi-modal reinforcement learning, and relates to the technical field of micro-grid control, and the method comprises the steps: constructing a multi-modal state space of a micro-grid, designing a layered reward function, and dynamically adjusting the reward weight of each layer of the layered reward function through employing a self-adaptive weight algorithm; dynamically identifying a node connection relationship by using a graph neural network, constructing a microgrid node connection graph to identify an operation mode of the microgrid, updating the node connection relationship in real time, and optimizing a distributed power supply output distribution strategy of the microgrid based on an attention mechanism of the graph neural network; and updating the overall model of the micro-grid by using a cloud edge coordination mechanism. According to the invention, the capability of coping with uncertainty of the micro-grid can be enhanced, and the intelligent operation level of the micro-grid is improved.

Description

Technical Field

[0001] The present invention relates to the field of microgrid control technology, and in particular to a microgrid dynamic collaborative scheduling method based on multimodal reinforcement learning. Background Art

[0002] With the large-scale integration of distributed energy resources, microgrids, as a key vehicle for integrating multiple resources, face multiple uncertainties in their operation. Distributed energy output is significantly affected by meteorological conditions, making traditional predictive control based on physical models difficult to cope with sudden disturbances such as extreme weather. At the same time, microgrids must balance multiple objectives, including power balance, operational economy, and carbon emission constraints. However, traditional hierarchical optimization methods are prone to falling into local optimality. Furthermore, the switching between island and grid-connected modes in microgrids leads to dynamic changes in network topology, making it difficult for traditional centralized control strategies to perceive node connectivity in real time. The need to integrate heterogeneous data from multiple sources (meteorological conditions, electricity prices, and user behavior) further complicates model construction. Therefore, there is an urgent need to enhance the intelligent operation of microgrids.

[0003] Dynamic collaborative scheduling of microgrids can improve the intelligent operation level of microgrids. Through multimodal learning reinforcement, hierarchical reward functions and graph neural networks, it can enhance the robustness of microgrids to deal with uncertainty, adapt to changes in microgrid network topology, and optimize the output of distributed power sources, thereby achieving multi-objective optimization of microgrid power balance, operating costs and low-carbon indicators.

[0004] Current microgrid collaborative control technology has the following problems: (1) Insufficient model generalization ability. Traditional rule-based control strategies rely on manual experience. Data-driven models require a large amount of labeled data and have weak cross-domain migration capabilities. (2) Low multi-objective collaborative efficiency. Fixed-weight linear weighting methods cannot dynamically balance target conflicts. There is a lack of a unified framework to integrate technical, economic, and environmental indicators. (3) Dynamic network adaptation lags. Traditional algorithms need to be reinitialized when switching between islanding and grid connection, and distributed collaborative control protocols converge slowly. (4) Communication and privacy challenges. Centralized control relies on high-bandwidth communication and has the risk of single point failure. User privacy protection mechanisms are imperfect during data sharing. These problems have restricted the development of various types of microgrids and limited their reliability, economy, and low carbon. Therefore, it is necessary to overcome the above technical problems, improve the intelligent operation level of microgrids, and achieve their multi-objective collaborative optimization. Summary of the Invention

[0005] The present invention is proposed in view of the problems existing in the existing microgrid dynamic collaborative scheduling method based on multimodal reinforcement learning. Therefore, the problem to be solved by the present invention is how to provide a microgrid dynamic collaborative scheduling method based on multimodal reinforcement learning.

[0006] In order to solve the above technical problems, the present invention provides the following technical solutions:

[0007] In a first aspect, the present invention provides a microgrid dynamic collaborative scheduling method based on multimodal reinforcement learning, which includes constructing a multimodal state space of the microgrid based on multi-source heterogeneous data, designing a hierarchical reward function, and dynamically adjusting the reward weights of each layer of the hierarchical reward function using an adaptive weight algorithm;

[0008] Utilize graph neural networks to dynamically identify node connection relationships, construct a microgrid node connection graph to identify the microgrid's operating mode, and update the node connection relationships in real time. At the same time, the distributed power output allocation strategy of the microgrid is optimized based on the graph neural network's attention mechanism.

[0009] The cloud-edge collaborative mechanism is used to update the overall model of the microgrid. Real-time data is collected and processed on the edge side, and the cloud side uses historical data for training and iteration to complete the dynamic collaborative scheduling of the microgrid based on multimodal reinforcement learning.

[0010] As a preferred solution of the microgrid dynamic collaborative scheduling method based on multimodal reinforcement learning described in the present invention, the multi-source heterogeneous data includes meteorological data, electricity market prices and user behavior patterns; the multimodal state space includes physical quantities, economic parameters and environmental variables; the physical quantities include photovoltaic output, wind power output, load, voltage, frequency and energy storage SOC; the economic parameters are real-time electricity prices; and the environmental variables include temperature, wind speed, light intensity, user behavior patterns and microgrid operation modes.

[0011] As a preferred solution of the microgrid dynamic collaborative scheduling method based on multimodal reinforcement learning described in the present invention, the hierarchical reward function includes three layers of rewards, the bottom layer rewards optimizing power balance, the middle layer rewards reducing operating costs, and the high layer rewards constraining carbon emission intensity. The rewards for the bottom, middle, and high layers are calculated as follows:

[0012] R1=-λ1*(ΔSOC 2 +ΔV 2 +Δf 2 )

[0013] R2=λ2*(Profit peak -Cost valley )

[0014] R3=-λ3*(ΔEmis carb -Quota carb )

[0015] R total =R1+R2+R3

[0016] Among them, R totalis the total reward, R1, R2, and R3 are the bottom-level power balance reward, the middle-level economic benefit reward, and the high-level carbon emission constraint reward, respectively. λ1, λ2, and λ3 are the weight coefficients of the bottom, middle, and high levels, respectively. ΔSOC is the energy storage SOC deviation, ΔV is the voltage deviation, Δf is the frequency deviation, and Profit peak Cost is the revenue from selling electricity during peak hours. valley is the cost of purchasing electricity during off-peak hours, ΔEmis carb For carbon emissions, Quota carb is the carbon quota value.

[0017] As a preferred solution of the microgrid dynamic collaborative scheduling method based on multimodal reinforcement learning described in the present invention, the method of dynamically adjusting the reward weights of each layer of the hierarchical reward function using an adaptive weight algorithm includes:

[0018] The weight vectors of rewards for the bottom, middle, and top layers are expressed as λ = [λ1,λ2,λ3], and the initial normalized weight vector is z = [z1,z2,z3]. The calculation formula for the weight vector of rewards for each layer is:

[0019]

[0020] Among them, λ i is the weight of each layer’s reward, z i is the initial weight of each layer’s reward, j is the reward level; exp(·) is the exponential function;

[0021] To maximize the total reward, the objective function of weight optimization is constructed as:

[0022] λ new =argmax λ (Ε[R total ]-L reg )

[0023]

[0024] L reg =β·KL(λ old ||λ new )

[0025] Among them, λ old ,λ new is the weight vector of each layer’s reward before and after the update, Indicates the gradient of the weight vector, E[·] is the expected operation, E[R total ] is the expected value of the total reward, L reg is the regularization loss, π θ (a|s) is the action probability distribution output by the policy network, β is the regularization coefficient, KL(λold ||λ new ) is the KL divergence of the new and old weights, T represents the total number of time steps in a time series or a round, and t represents each time step.

[0026] As a preferred solution of the microgrid dynamic collaborative scheduling method based on multimodal reinforcement learning described in the present invention, the construction of the microgrid node connection graph includes:

[0027] Based on the topology recognition module and attention mechanism of graph neural network, the GraphSAGE algorithm is used to construct the microgrid node connection graph. The nodes of the microgrid node connection graph represent the various components in the microgrid; the edges represent the electrical connection relationship between the components. The status information of each node is input during the graph construction process.

[0028] The topology identification module is used to identify the operation mode of the microgrid in real time.

[0029] As a preferred solution of the microgrid dynamic collaborative scheduling method based on multimodal reinforcement learning described in the present invention, the distributed power output allocation strategy of the microgrid optimized by the attention mechanism based on the graph neural network includes:

[0030] The influence weights between nodes in the microgrid node connection graph are learned through the attention mechanism, and the feature space of each node is established based on the multimodal state space.

[0031] For each node, calculate the attention coefficient between it and its neighboring nodes, which indicates the importance of the neighboring nodes to the current node.

[0032] The node v i With node v j The feature space of is concatenated to form a joint feature vector, and the joint feature vector is mapped to a scalar score;

[0033] Using LeakyReLU as the activation function, Sofxmax is used to ensure that the weights are non-negative and sum to 1, and the normalized attention coefficient is obtained, which is expressed as:

[0034] e ij =Leaky Re LU(a T [h i ||h j ])

[0035]

[0036] Among them, e ij is the scalar score, h i For node v i The feature space, h j For node v jThe feature space of a T is the scalar score mapping matrix, α ij is the normalized attention coefficient;

[0037] The features of neighboring nodes are weighted and aggregated according to the calculated attention coefficient to obtain the new features of the current node, which is expressed as:

[0038]

[0039] Among them, h i 'For node v i The new feature space, σ() is the activation function;

[0040] The obtained new feature space is spliced ​​and fused to obtain the multi-dimensional node interaction relationship in the microgrid;

[0041] Optimize the output allocation strategy of distributed generation according to importance weights to conduct multi-objective coordinated scheduling;

[0042] The basic allocation strategy is implemented to allocate the output upper limit of each power source according to the normalized weight, which is expressed as:

[0043]

[0044] in, To optimize the output of power supply i, is the maximum technical output of power source i, α i is the attention weight of power source i, DG is the set of all distributed power sources;

[0045] The extension of multi-objective constraints based on the weights of the hierarchical reward function is expressed as:

[0046]

[0047] Among them, k=1 means the bottom layer rewards optimizing power balance, k=2 means the middle layer rewards reducing operating costs, and k=3 means the top layer rewards constraining carbon emission intensity; is the attention weight of the power source under the k-th reward function, λ k is the priority weight of the k-th reward function;

[0048] The distributed generation output allocation strategy of the microgrid is optimized based on the weights of the extended hierarchical reward function with multi-objective constraints.

[0049] As a preferred solution of the microgrid dynamic collaborative scheduling method based on multimodal reinforcement learning described in the present invention, the method of updating the overall model of the microgrid using the cloud-edge collaborative mechanism includes:

[0050] Devices on the edge side are configured to collect real-time data and local status, and are also configured with edge computing devices and lightweight machine learning models. Some of the processed real-time data, calculation results, and model parameters are regularly encrypted and uploaded to the cloud. The cloud aggregates parameters through federated learning, updates the global strategy, and then pushes it back to the edge. The edge-side devices fine-tune the strategy based on the local scenario and then execute it.

[0051] In a second aspect, the present invention provides a computer device comprising a memory and a processor, wherein the memory stores a computer program, wherein: when the processor executes the computer program, it implements the steps of a microgrid dynamic collaborative scheduling method based on multimodal reinforcement learning.

[0052] In a third aspect, the present invention provides a computer-readable storage medium having a computer program stored thereon, wherein: when the computer program is executed by a processor, the steps of a microgrid dynamic collaborative scheduling method based on multimodal reinforcement learning are implemented.

[0053] The beneficial effects of the present invention are: by fusing multi-source heterogeneous data, constructing a multimodal state space, designing a hierarchical reward function, and realizing the optimization and coordination of multiple objectives at each layer through adaptive weight adjustment, and based on the topology recognition module and attention mechanism of the graph neural network, constructing a microgrid node connection diagram, optimizing the distributed power output of the microgrid, and combining the cloud-edge collaborative mechanism to realize multi-objective collaborative scheduling of power balance, operating cost and low-carbon indicators, enhancing the ability of the microgrid to cope with uncertainty and improving the intelligent operation level of the microgrid. BRIEF DESCRIPTION OF THE DRAWINGS

[0054] In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the following briefly introduces the drawings required for use in the description of the embodiments. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0055] Figure 1 This is a flow chart of a dynamic collaborative scheduling method for microgrids based on multimodal reinforcement learning.

[0056] Figure 2 This is the basic structural diagram of the microgrid. DETAILED DESCRIPTION

[0057] To make the above-mentioned objects, features, and advantages of the present invention more easily understood, the following detailed description of the specific embodiments of the present invention is given in conjunction with the accompanying drawings. It is obvious that the described embodiments are only part of the embodiments of the present invention, not all of them. Based on the embodiments of the present invention, all other embodiments obtained by ordinary persons in this field without creative work should fall within the scope of protection of the present invention.

[0058] In the following description, many specific details are set forth to facilitate a full understanding of the present invention. However, the present invention may also be implemented in other ways different from those described herein. Those skilled in the art may make similar generalizations without violating the connotation of the present invention. Therefore, the present invention is not limited to the specific embodiments disclosed below.

[0059] Secondly, the term "one embodiment" or "embodiment" herein refers to a specific feature, structure, or characteristic that may be included in at least one implementation of the present invention. The phrase "in one embodiment" appearing in various places throughout this specification does not necessarily refer to the same embodiment, nor does it constitute a separate or selective embodiment that is mutually exclusive with other embodiments.

[0060] Example 1

[0061] Reference Figure 1 , which is the first embodiment of the present invention, provides a microgrid dynamic collaborative scheduling method based on multimodal reinforcement learning, including:

[0062] S1: Construct a multimodal state space for the microgrid based on multi-source heterogeneous data, design a hierarchical reward function, and use an adaptive weighting algorithm to dynamically adjust the reward weights of each layer of the hierarchical reward function;

[0063] Specifically, it integrates multi-source heterogeneous data such as meteorological data, electricity market prices, and user behavior patterns to construct a multimodal state space that includes physical quantities, economic parameters, and environmental variables;

[0064] The physical quantities of the multimodal state space include: photovoltaic output, wind power output, load, voltage, frequency, and energy storage SOC; the economic parameter is the real-time electricity price; and the environmental variables include: temperature, wind speed, light intensity, user behavior patterns, and microgrid operation mode (grid-connected or islanded).

[0065] S2: Utilize graph neural networks to dynamically identify node connectivity, construct a microgrid node connectivity graph to identify the microgrid's operating mode, and update the node connectivity in real time. Simultaneously, the distributed power output allocation strategy of the microgrid is optimized based on the graph neural network's attention mechanism.

[0066] Specifically, a hierarchical reward function is designed, with bottom-level rewards optimizing power balance, middle-level rewards reducing operating costs, and high-level rewards constraining carbon emission intensity. The adaptive weight algorithm is used to dynamically adjust the reward weights of each layer to achieve optimized coordination of multiple objectives at each layer.

[0067] The layered reward function is divided into three layers. The bottom layer reward function optimizes power balance and sets penalties for energy storage SOC deviation, voltage, and frequency deviation; the middle layer reward function reduces operating costs and obtains revenue through peak and valley electricity prices and real-time electricity price changes; the high layer reward function constrains carbon emission intensity and sets penalties for deviations between carbon emissions and carbon quotas. The rewards for the bottom, middle, and high layers are calculated as follows:

[0068] R1=-λ1*(ΔSOC 2 +ΔV 2 +Δf 2 )

[0069] R2=λ2*(Profit peak -Cost valley )

[0070] R3=-λ3*(ΔEmis carb -Quota carb )

[0071] The total reward R total For: R total =R1+R2+R3

[0072] Among them, R1, R2, and R3 are the bottom-level power balance reward, the middle-level economic benefit reward, and the high-level carbon emission constraint reward respectively; λ1, λ2, and λ3 are the weight coefficients of the bottom, middle, and high levels respectively; ΔSOC is the energy storage SOC deviation, ΔV is the voltage deviation, Δf is the frequency deviation, and Profit peak Cost is the revenue from selling electricity during peak hours. valley is the cost of purchasing electricity during off-peak hours, ΔEmis carb For carbon emissions, Quota carb is the carbon quota value.

[0073] The adaptive weight algorithm is a dynamic weight allocation technology for multi-objective reinforcement learning, which dynamically adjusts the weight coefficients of rewards at each layer through an online learning mechanism.

[0074] The weight vectors of rewards for the bottom, middle, and top layers are expressed as λ = [λ1,λ2,λ3], and the initial normalized weight vector is z = [z1,z2,z3]. The calculation formula for the weight vector of rewards for each layer is:

[0075]

[0076] Among them, λ i is the weight of each layer’s reward, z i is the initial weight of each layer reward, j is the reward level; exp(·) is the exponential function; and λ i ≥0,∑λ i =1;

[0077] In order to maximize the total reward, the objective function of weight optimization is:

[0078] λ new =argmax λ (Ε[R total ]-L reg )

[0079]

[0080] L reg =β·KL(λ old ||λ new )

[0081] Among them, λ old ,λ new That is, λ=[λ1,λ2,λ3] before and after the update, Indicates the gradient of the weight vector, E[·] is the expected operation, E[R total ] is the expected value of the total reward, L reg is the regularization loss, π θ (a|s) is the action probability distribution output by the policy network, β is the regularization coefficient (usually 0.1-0.5), KL(λ old ||λ new ) is the KL divergence of the new and old weights, T represents the total number of time steps in a time series or a round, and t represents each time step.

[0082] Based on the topology recognition module of the graph neural network, a microgrid node connection diagram is constructed, the island / grid-connected mode is identified in real time, the connection relationship is updated in real time, and the distributed power output allocation strategy of the microgrid is optimized based on the attention mechanism of the graph neural network.

[0083] Based on a graph neural network topology recognition module and an attention mechanism, the module uses the GraphSAGE algorithm to construct a microgrid node connection graph. The nodes in the microgrid node connection graph represent various components in the microgrid, such as distributed power sources (PV, wind power, etc.), energy storage devices, and load nodes; the edges represent the electrical connections between these components. During the graph construction process, state information such as voltage and power at each node is input. This allows the module to identify the microgrid's operating mode in real time, particularly the transition between island and grid-connected modes.

[0084] Use the GraphSAGE algorithm to construct a microgrid node connection graph:

[0085] (1) Node sampling: The components of the microgrid (wind and solar equipment, energy storage, loads, etc.) are considered as nodes of the graph, and the electrical connections between nodes are considered as edges of the graph. A certain number of key nodes are selected from the neighboring nodes of each node through uniform loss calculation. This method can focus on the interaction between key nodes and reduce the computational complexity, which is beneficial to improving the efficiency of the algorithm, especially in large-scale microgrids.

[0086] (2) Feature aggregation: Aggregate the features of the sampled neighbor nodes with the features of the current node to generate new node features.

[0087] (3) Multi-layer feature update: The neighbor features of directly electrically connected devices are integrated to obtain local interactions, such as real-time power matching between photovoltaics and energy storage. Based on the first output, the neighbor features of indirectly connected devices in the second layer are integrated to obtain cross-regional influences, such as the indirect regulation requirements of local energy storage due to remote load fluctuations. By stacking multiple layers of GraphSAGE, node features are gradually integrated with multiple layers of neighbor information. Through multiple iterations, microgrid nodes can gradually integrate the information of their neighbor nodes, thereby better reflecting their position and role in the entire microgrid network.

[0088] The attention mechanism dynamically learns the influence weights of connections between nodes in the microgrid's node connection graph, specifically the importance of different nodes to other nodes. This determines the importance of each distributed generation (DG) to the overall microgrid operation. These weights are then used to optimize the DG output allocation strategy, enabling the microgrid to achieve coordinated scheduling across multiple objectives, including power balance, cost reduction, and low-carbon targets, across all operating modes.

[0089] The attention mechanism learns the dynamic influence weights of the connection:

[0090] (1) Establishing feature space: Establishing the feature space h of each node based on the multimodal state space (including physical quantities, economic parameters and environmental variables) i .

[0091] (2) Calculate the attention coefficient: For each node, calculate the attention coefficient between it and its neighboring nodes, which indicates the importance of the neighboring nodes to the current node.

[0092] 1) Set node v i With node v j The feature space is spliced ​​to form a joint feature vector;

[0093] 2) Through a T Map the joint feature vector to a scalar score e ij , a higher score indicates a stronger correlation;

[0094] 3) Use the LeakyReLU activation function to retain negative correlation information, and then use Sofxmax to ensure that the weights are non-negative and sum to 1, and obtain the normalized attention coefficient α ij ;

[0095] e ij =Leaky Re LU(a T [h i ||h j ])

[0096]

[0097] Among them, e ij is the scalar score, h i For node v i The feature space, h j For node v j The feature space of a T is the scalar score mapping matrix, α ij is the normalized attention coefficient;

[0098] (3) Weighted aggregation: Based on the calculated attention coefficient α ij Perform weighted aggregation on the features of neighboring nodes to obtain the new features of the current node. Neighbors with higher weights contribute more to the feature update of the current node. This weighted aggregation method can highlight the influence of important neighboring nodes and improve the model's sensitivity to network topology changes. It can be expressed as:

[0099]

[0100] Among them, h i 'For node v i The new feature space, σ() is the activation function;

[0101] (4) Multi-dimensional interaction: Through multiple independent attention parallel calculations, we can obtain multi-dimensional node interaction relationships in the microgrid. For example, by focusing on power balance, we can learn the electrical coupling between nodes; by focusing on economic costs, we can learn the relationship between electricity prices and equipment output; by focusing on carbon emissions, we can learn the relationship between the proportion of renewable energy and carbon quotas. Achieve multi-objective optimization and improve model robustness.

[0102] Optimizing the output allocation strategy of distributed generation based on importance weights:

[0103] The weight α learned by the attention mechanism iIt reflects the contribution of node i to multi-objective optimization at the current moment. For example, for the bottom-level reward, it optimizes power balance. When the frequency fluctuates, the weight of the synchronous generator node is increased to enhance the inertia support of the system. For the middle-level reward, it reduces operating costs. During periods of high electricity prices, the weight of the energy storage node is increased, and electricity is released first to reduce the cost of purchasing electricity. For the high-level reward, it constrains carbon emission intensity. The weight of the photovoltaic / wind power node is increased to reduce the use of fossil energy.

[0104] The basic allocation strategy is implemented to allocate the output upper limit of each power source according to the normalized weight, which is expressed as:

[0105]

[0106] in, is the optimized output of power source i (kW), is the maximum technical output of power source i (kW), α i is the attention weight of power source i, and DG is the set of all distributed power sources.

[0107] The extension of multi-objective constraints based on the weights of the hierarchical reward function is expressed as:

[0108]

[0109] Among them, k=1 means the bottom layer rewards optimizing power balance, k=2 means the middle layer rewards reducing operating costs, and k=3 means the top layer rewards constraining carbon emission intensity; is the attention weight of the power source under the k-th reward function, λ k is the priority weight of the k-th reward function.

[0110] Finally, the output command of each DG is calculated based on the corrected weight and sent to the controller.

[0111] S3: Use the cloud-edge collaborative mechanism to update the overall model of the microgrid. The edge side collects and processes real-time data, and the cloud side uses historical data for training and iteration to complete the dynamic collaborative scheduling of the microgrid based on multimodal reinforcement learning.

[0112] Specifically, in the cloud-edge collaboration mechanism, edge-side devices are configured to collect real-time data and local status, and edge computing devices and lightweight machine learning models are configured. Some of the processed real-time data, calculation results and model parameters are regularly encrypted and uploaded to the cloud. The cloud aggregates parameters through federated learning, updates the global strategy and then pushes it back to the edge. The edge-side devices fine-tune the strategy based on the local scenario and then execute it.

[0113] Example 2

[0114] Taking a microgrid in an industrial park as an example, the microgrid dynamic collaborative scheduling method based on multimodal reinforcement learning proposed in this invention is adopted.

[0115] The industrial park features a 1MW photovoltaic array, 500kWh lithium-ion battery storage, a 200kW charging station cluster, and 300kW of industrial loads. It also deploys smart meters to collect voltage and current data, a weather station with light and wind speed sensors, and ambient temperature and humidity sensors. On the edge, PV output is collected in real time to create load curves. This includes training an LSTM prediction model using 30 days of historical data, recording real-time electricity prices and updating them every 15 minutes. Simultaneously, an industrial gateway runs a lightweight PPO reinforcement learning model. In the cloud, three years of historical meteorological data, electricity market transaction records, and equipment operation logs are stored. A federated learning platform is also built to aggregate data from 10 similar industrial parks to train global strategies.

[0116] Based on the basic situation of this industrial park, a multimodal state space is constructed, including:

[0117] Physical quantities: photovoltaic output (0-1MW), energy storage SOC (0.1-0.9), load power (0-500kW), bus voltage (380V±5%).

[0118] Economic parameters: real-time electricity price (0.5-2.0 yuan / kWh), carbon quota (100 tons / month).

[0119] Environmental variables: Light intensity (0-1000W / m 2 ), temperature (-10℃-40℃), user charging behavior pattern (clustered into three categories: fast charging / slow charging / standby charging)

[0120] Operation mode: Grid-connected (0) / Island (1).

[0121] At the same time, the photovoltaic prediction error rate is calculated = (actual output - predicted output) / predicted output, and the peak and valley electricity price period characteristics are constructed = 1 (peak time 9:00-12:00 / 18:00-21:00), = 0 (other periods).

[0122] The hierarchical reward function is designed as follows:

[0123] (1) Bottom-level reward: R1 = -0.5 × (ΔSOC 2 +ΔV 2 +Δf 2 ), where ΔSOC exceeding ±0.1 triggers a penalty, and ΔV exceeding ±2% triggers a penalty.

[0124] (2) Middle-level reward: R2 = 0.3 × (P sell × Peak electricity price-P buy × off-peak electricity price), of which the peak electricity price is 1.5 yuan / kWh and the off-peak electricity price is 0.7 yuan / kWh.

[0125] (3) Senior management reward: R3 = -0.2 × max (0, carbon emissions - 100 tons), where carbon emissions = 0.8 kg / kWh × fossil energy consumption.

[0126] The initial weight of the adaptive weight algorithm is: λ = [0.5, 0.3, 0.2]; the weight is updated every 15 minutes through federated learning, and the normalized calculation λ is i ; and set the regularization coefficient β = 0.3; the gradient clipping threshold is 5.0.

[0127] In the graph neural network module, GraphSAGE is used to build a three-layer graph structure, with each layer aggregating the features of five neighboring nodes. The attention mechanism dynamically calculates node weights, and the weight of energy storage nodes is increased by 30% in island mode.

[0128] The cloud-edge collaboration process can be described as follows: the edge executes local policies every five minutes, generating local scheduling instructions, such as energy storage charging and discharging power. Model parameters are encrypted and uploaded to the cloud every hour. Every morning, the cloud updates the global policy through federated learning, optimizing for reducing the total operating cost of all campuses. A carbon emissions optimization strategy is published monthly, adjusting the weight λ3 to 0.4 when carbon quotas are tight.

[0129] Through the design of the present invention, the economic efficiency of the industrial park microgrid can be improved, the peak-valley arbitrage income can be increased and the annual operating costs can be reduced; the stability of the industrial park microgrid can be guaranteed, the island switching response time can be shortened, and the voltage deviation can be controlled within ±1.2%; at the same time, the photovoltaic absorption rate of the park can be improved, and carbon emissions can be effectively reduced compared with traditional microgrid control solutions.

[0130] This embodiment also provides a computer device, which is applicable to a microgrid dynamic collaborative scheduling method based on multimodal reinforcement learning, including: a memory and a processor; the memory is used to store computer-executable instructions, and the processor is used to execute computer-executable instructions to implement all or part of the steps of the method described in the embodiment of the present invention as proposed in the above embodiment.

[0131] This embodiment further provides a storage medium having a computer program stored thereon, which, when executed by a processor, performs the method of any optional implementation of the above embodiment. The storage medium can be implemented by any type of volatile or non-volatile storage device or a combination thereof, such as static random access memory (SRAM), electrically erasable programmable read-only memory (EEPROM), erasable programmable read-only memory (EPROM), programmable read-only memory (PROM), read-only memory (ROM), magnetic memory, flash memory, magnetic disk, or optical disk.

[0132] The storage medium proposed in this embodiment and the data storage method proposed in the above embodiment belong to the same inventive concept. Technical details not fully described in this embodiment can be found in the above embodiment, and this embodiment has the same beneficial effects as the above embodiment.

[0133] In summary, this method integrates multi-source heterogeneous data, constructs a multimodal state space, designs a hierarchical reward function, and realizes the optimization and coordination of multiple objectives at each layer through adaptive weight adjustment. Based on the topology recognition module and attention mechanism of the graph neural network, it constructs a microgrid node connection diagram, optimizes the distributed power output of the microgrid, and combines the cloud-edge collaboration mechanism to achieve multi-objective coordinated scheduling of power balance, operating cost and low-carbon indicators, enhance the microgrid's ability to cope with uncertainty, and improve the intelligent operation level of the microgrid.

[0134] It should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention and are not intended to limit the present invention. Although the present invention has been described in detail with reference to the preferred embodiments, those skilled in the art should understand that the technical solutions of the present invention may be modified or replaced by equivalents without departing from the spirit and scope of the technical solutions of the present invention, which should all be included in the scope of the claims of the present invention.

Claims

1. A microgrid dynamic collaborative scheduling method based on multimodal reinforcement learning, characterized by: include, Based on multi-source heterogeneous data, a multimodal state space of the microgrid is constructed, a hierarchical reward function is designed, and an adaptive weight algorithm is used to dynamically adjust the reward weights of each layer of the hierarchical reward function. Utilize graph neural networks to dynamically identify node connection relationships, construct a microgrid node connection graph to identify the microgrid's operating mode, and update the node connection relationships in real time. At the same time, the distributed power output allocation strategy of the microgrid is optimized based on the graph neural network's attention mechanism. The cloud-edge collaborative mechanism is used to update the overall model of the microgrid. Real-time data is collected and processed on the edge side, and the cloud side uses historical data for training and iteration to complete the dynamic collaborative scheduling of the microgrid based on multimodal reinforcement learning.

2. The microgrid dynamic collaborative scheduling method based on multimodal reinforcement learning according to claim 1, characterized in that: The multi-source heterogeneous data includes meteorological data, electricity market prices and user behavior patterns; the multimodal state space includes physical quantities, economic parameters and environmental variables; the physical quantities include photovoltaic output, wind power output, load, voltage, frequency and energy storage SOC; the economic parameters are real-time electricity prices; the environmental variables include temperature, wind speed, light intensity, user behavior patterns and microgrid operation mode.

3. The microgrid dynamic collaborative scheduling method based on multimodal reinforcement learning according to claim 2, characterized in that: The hierarchical reward function includes three layers of rewards: the bottom layer rewards optimizing power balance, the middle layer rewards reducing operating costs, and the top layer rewards constraining carbon emission intensity. The rewards for the bottom, middle, and top layers are calculated as follows: R1=-λ1*(ΔSOC 2 +ΔV 2 +Δf 2 ) R2=λ2*(Profit peak -Cost valley ) R3=-λ3*(ΔEmis carb -Quota carb ) R total =R1+R2+R3 Among them, R total is the total reward, R1, R2, and R3 are the bottom-level power balance reward, the middle-level economic benefit reward, and the high-level carbon emission constraint reward, respectively. λ1, λ2, and λ3 are the weight coefficients of the bottom, middle, and high levels, respectively. ΔSOC is the energy storage SOC deviation, ΔV is the voltage deviation, Δf is the frequency deviation, and Profit peak Cost is the revenue from selling electricity during peak hours. valley is the cost of purchasing electricity during off-peak hours, ΔEmis carb For carbon emissions, Quota carb is the carbon quota value.

4. The microgrid dynamic collaborative scheduling method based on multimodal reinforcement learning according to claim 3, characterized in that: The method of dynamically adjusting the reward weights of each layer of the hierarchical reward function using an adaptive weight algorithm includes: The weight vectors of rewards for the bottom, middle, and top layers are expressed as λ = [λ1,λ2,λ3], and the initial normalized weight vector is z = [z1,z2,z3]. The calculation formula for the weight vector of rewards for each layer is: Among them, λ i is the weight of each layer’s reward, z i is the initial weight of each layer’s reward, j is the reward level; exp(·) is the exponential function; To maximize the total reward, the objective function of weight optimization is constructed as: l new =argmax λ (E[R total ]-L reg ) L reg =β·KL(λ old ||λ new ) Among them, λ old ,λ new is the weight vector of each layer’s reward before and after the update, Indicates the gradient of the weight vector, E[·] is the expected operation, E[R total ] is the expected value of the total reward, L reg is the regularization loss, π θ (a|s) is the action probability distribution output by the policy network, β is the regularization coefficient, KL(λ old ||λ new ) is the KL divergence of the new and old weights, T represents the total number of time steps in a time series or a round, and t represents each time step.

5. The microgrid dynamic collaborative scheduling method based on multimodal reinforcement learning according to claim 4, characterized in that: The constructing of the microgrid node connection diagram includes: Based on the topology recognition module and attention mechanism of graph neural network, the GraphSAGE algorithm is used to construct the microgrid node connection graph. The nodes of the microgrid node connection graph represent the various components in the microgrid; the edges represent the electrical connection relationship between the components. The status information of each node is input during the graph construction process. The topology identification module is used to identify the operation mode of the microgrid in real time.

6. The microgrid dynamic collaborative scheduling method based on multimodal reinforcement learning according to claim 5, characterized in that: The distributed power output allocation strategy for optimizing microgrids based on the graph neural network attention mechanism includes: The influence weights between nodes in the microgrid node connection graph are learned through the attention mechanism, and the feature space of each node is established based on the multimodal state space. For each node, calculate the attention coefficient between it and its neighboring nodes, which indicates the importance of the neighboring nodes to the current node. The node v i With node v j The feature space of is concatenated to form a joint feature vector, and the joint feature vector is mapped to a scalar score; Using LeakyReLU as the activation function, Sofxmax is used to ensure that the weights are non-negative and sum to 1, and the normalized attention coefficient is obtained, which is expressed as: e ij =Leaky Re L U(a T [h i ||h j ]) Among them, e ij is the scalar score, h i For node v i The feature space, h j For node v j The feature space of a T is the scalar score mapping matrix, α ij is the normalized attention coefficient; The features of neighboring nodes are weighted and aggregated according to the calculated attention coefficient to obtain the new features of the current node, which is expressed as: Among them, h i 'For node v i The new feature space, σ() is the activation function; The obtained new feature space is spliced ​​and fused to obtain the multi-dimensional node interaction relationship in the microgrid; Optimize the output allocation strategy of distributed generation according to importance weights to conduct multi-objective coordinated scheduling; The basic allocation strategy is implemented to allocate the output upper limit of each power source according to the normalized weight, which is expressed as: Among them, P i opt For the optimized output of power supply i, P i max is the maximum technical output of power source i, α i is the attention weight of power source i, DG is the set of all distributed power sources; The extension of multi-objective constraints based on the weights of the hierarchical reward function is expressed as: Among them, k=1 means the bottom layer rewards optimizing power balance, k=2 means the middle layer rewards reducing operating costs, and k=3 means the top layer rewards constraining carbon emission intensity; is the attention weight of the power source under the k-th reward function, λ k is the priority weight of the k-th reward function; The distributed generation output allocation strategy of the microgrid is optimized based on the weights of the extended hierarchical reward function with multi-objective constraints.

7. The microgrid dynamic collaborative scheduling method based on multimodal reinforcement learning according to claim 6, characterized in that: The overall model for updating the microgrid using the cloud-edge collaborative mechanism includes: Devices on the edge side are configured to collect real-time data and local status, and are also configured with edge computing devices and lightweight machine learning models. Some of the processed real-time data, calculation results, and model parameters are regularly encrypted and uploaded to the cloud. The cloud aggregates parameters through federated learning, updates the global strategy, and then pushes it back to the edge. The edge-side devices fine-tune the strategy based on the local scenario and then execute it.

8. A computer device comprising a memory and a processor, wherein the memory stores a computer program, wherein: When the processor executes the computer program, the steps of the microgrid dynamic collaborative scheduling method based on multimodal reinforcement learning described in any one of claims 1 to 7 are implemented.

9. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of a microgrid dynamic collaborative scheduling method based on multimodal reinforcement learning according to any one of claims 1 to 7 are implemented.

Images