Control method, device and medium for direct-current microgrid based on dynamic electric carbon factor

Abstract

This invention relates to the field of microgrid technology, specifically providing a control method, device, and medium for a DC microgrid based on a dynamic carbon factor. The method includes: acquiring the current state information of the DC microgrid system; inputting the state information into an improved SAC controller capable of deep reinforcement learning; the improved SAC controller comprising a policy network, two evaluation networks, a dual-mode hybrid control module, an adaptive weighted reward module, and a hierarchical experience pool module; outputting the operating condition determination result through the dual-mode hybrid control module; adaptively outputting the droop coefficient of each node in the DC microgrid based on the operating condition determination result; calculating the control voltage and power allocation of each node in the DC microgrid at the next moment, and controlling the DC microgrid to allocate power according to the dynamic carbon factor. This addresses the problems of related technologies where DC microgrids only consider power balance, resulting in high carbon emissions and poor control accuracy.

Description

Technical Field

[0001] This invention relates to the field of microgrid technology, and in particular to a control method, device and medium for a DC microgrid based on dynamic carbon factor. Background Technology

[0002] DC microgrids, as an important component of future smart power distribution systems, are of great significance in promoting energy conservation and emission reduction and achieving sustainable energy development. Compared with AC microgrids, DC microgrids have higher efficiency and reliability when accommodating distributed renewable energy generation systems (such as photovoltaic and wind power), energy storage units, and DC power loads (such as electric vehicle charging piles and some electronic devices), and do not require complex control of voltage, frequency, and phase.

[0003] The dynamic carbon factor refers to the carbon emissions per unit of electricity consumption, reflecting the dynamic characteristics of power system carbon emissions as they change over time, space, and technology. It continuously varies based on factors such as time, power generation energy structure, and power generation efficiency. To meet the requirements of low-carbon transformation, DC microgrids need to rationally allocate internal power based on the dynamic changes in the carbon factor, thereby reducing overall carbon emissions. However, adjustments to power allocation must not adversely affect the voltage control accuracy of the DC microgrid; otherwise, the stability and power quality of the entire microgrid system will be jeopardized. Existing power allocation methods often focus on ensuring the basic operational needs of the microgrid, such as meeting load power requirements and maintaining power balance, neglecting the carbon emission factor. Furthermore, in some studies that attempt to consider carbon emissions, it has been found that maintaining voltage control accuracy is difficult after reducing carbon emissions.

[0004] In recent years, although some studies have introduced deep reinforcement learning algorithms to achieve adaptive control of microgrids, three major technical defects still exist: First, the commonly used random sampling experience playback mechanism does not take into account the strong temporal sequence and scenario differences of DC microgrid operating conditions, resulting in low coverage of extreme condition samples and insufficient model generalization ability and robustness; Second, most adopt a single online control mode, and continue to perform neural network inference under steady-state conditions, causing unnecessary consumption of computing resources and making it difficult to balance control accuracy and operating efficiency; Third, the reward function is not related to the microgrid operating conditions and the dynamic characteristics of the carbon factor, and the fixed weight design cannot adapt to the control priority requirements under different operating conditions.

[0005] There is currently no effective solution to the problems of high carbon emissions and poor control accuracy in DC microgrids that only consider power balance in related technologies. Summary of the Invention

[0006] The present invention provides a control method, device and medium for DC microgrid based on dynamic carbon factor, which at least solves the problems of DC microgrids in related technologies that only consider power balance, have high carbon emissions and poor control accuracy.

[0007] According to one aspect of the present invention, a control method for a DC microgrid based on a dynamic carbon factor is provided, comprising: acquiring the current state information of the DC microgrid system, wherein the state information includes the voltage of each node of the DC microgrid system, the power of each node, the dynamic carbon factor of each node, and the bus voltage of the DC microgrid system; and inputting the state information into an improved SAC (Soft Actor-Critic Algorithm) capable of deep reinforcement learning. The improved SAC controller (Actor-Critic controller) includes a policy network, two evaluation networks, a dual-mode hybrid control module, an adaptive weighted reward module, and a hierarchical experience pool module. The adaptive weighted reward module updates the controller's parameter weights for reinforcement learning, and the hierarchical experience pool module stores training data. The dual-mode hybrid control module performs real-time determination of steady-state / dynamic operating conditions based on the state information and outputs the operating condition determination result. Based on the operating condition determination result, the droop coefficient of each node in the DC microgrid is adaptively output. The policy network and evaluation network are used to output the droop coefficient under at least one operating condition determination result. The control voltage and power allocation of each node in the DC microgrid at the next time step are calculated based on the droop coefficient. The DC microgrid is controlled according to the control voltage and power allocation to allocate power according to the dynamic carbon factor, and the node voltage and bus voltage meet the accuracy requirements.

[0008] As an optional solution, the dual-mode hybrid control module performs real-time determination of steady-state / dynamic operating conditions based on the state information, including: setting voltage fluctuation thresholds, power fluctuation thresholds, and dynamic carbon factor fluctuation thresholds. A steady-state operating condition is determined when all three of the following conditions are met simultaneously; otherwise, a dynamic operating condition is determined. Specifically, the relative deviation percentage between the voltage of each node and the bus voltage is less than the voltage fluctuation threshold for multiple consecutive control cycles; wherein the control cycle is the real-time control sampling cycle of the DC microgrid, and the relative deviation percentage between the node voltage and the bus voltage = |node voltage - bus voltage| / bus voltage × 100%; the percentage of the relative fluctuation amplitude of the system's rated power and the output power of each node in a single cycle is less than the power fluctuation threshold, wherein the percentage of the relative fluctuation amplitude of the power in a single cycle = |current power - previous cycle power| / system rated power × 100%; and the dynamic carbon factor e of each node... iThe percentage of the single-period relative fluctuation amplitude of the electric carbon factor is less than the dynamic electric carbon factor fluctuation threshold, wherein the percentage of the single-period relative fluctuation amplitude of the electric carbon factor = |current e i -Previous period e i | / Previous cycle e i ×100%.

[0009] As an optional approach, based on the operating condition determination result, the droop coefficient of each node of the DC microgrid is adaptively output, including: when the operating condition determination result is a steady-state operating condition, switching to a fixed droop optimization control mode and outputting a fixed droop coefficient optimized offline by an improved SAC controller, wherein the fixed droop coefficient is the droop coefficient value converged by the improved SAC controller after training on steady-state operating condition samples; when the operating condition determination result is a dynamic operating condition, switching to a SAC dynamic control mode, wherein the strategy network outputs a dynamically optimized droop coefficient based on real-time state information; wherein the strategy network maps the state information through multiple serial fully connected layers, generates action probability parameters through parallel mean and standard deviation layers, and then samples the action probability parameters through a reparameterization method to obtain the dynamically optimized droop coefficient.

[0010] As an optional approach, the dynamic operating conditions include: high-carbon low-load operating conditions, low-carbon high-load operating conditions, and extreme dynamic operating conditions. The policy network outputs dynamically optimized droop coefficients based on real-time state information, including: when the operating condition is determined to be high-carbon low-load, outputting a dynamic droop coefficient with a preference for carbon emission optimization; when the operating condition is determined to be low-carbon high-load, outputting a dynamic droop coefficient with a preference for voltage stability; and when the operating condition is determined to be extreme dynamic, outputting a dynamic droop coefficient with a balanced preference for both carbon emission optimization and voltage stability. All dynamic droop coefficients are generated by the policy network based on real-time state information through fully connected layers, mean layers, standard deviation layers, and reparameterization methods. Policy networks with different preferences receive different reward weights during training.

[0011] As an optional approach, after calculating the control voltage and power distribution of each node in the DC microgrid at the next time step based on the droop coefficient, and controlling the DC microgrid according to the control voltage and power distribution, the method further includes: generating a system reward through the operating condition adaptive weighted reward module of the improved SAC controller to adjust the parameters of the strategy network and the evaluation network, wherein the system reward is an operating condition adaptive weighted reward, including a basic reward item and an operating condition weight coefficient, the basic reward item including voltage control reward and power-carbon emission control reward; the calculation formula for the operating condition adaptive weighted reward is as follows: In the formula, For adaptive weighted rewards based on operating conditions, For global weights, For voltage weighting, For carbon emission weighting, As a reward for voltage control, The reward is for power-carbon emission control; the operating condition weighting coefficient includes the global weight, voltage weight, and carbon emission weight, and different weighting coefficients are selected under different operating conditions. The operating conditions include at least: steady-state operating condition, high carbon low load operating condition, low carbon high load operating condition, and extreme dynamic operating condition.

[0012] As an optional approach, the formula for calculating the voltage control reward is as follows: In the formula, Let be the voltage control reward at time t, n be the number of nodes, and i be the node number. Let v be the actual voltage value at time t of the i-th node. ref This is the system voltage reference value, v bus,t It is the actual value of the system bus voltage at time t.

[0013] As an optional approach, the calculation formula for the power-carbon emission control reward is as follows: In the formula, The power-carbon emission control reward at time t. For the power allocation of the i-th node at time t, Let be the power reference value of the i-th node at time t, calculated using the following formula: In the formula, Let e ​​be the total power of the system at time t. i Let be the dynamic electric carbon factor of the i-th node at time t.

[0014] As an optional solution, the operating condition stratified experience pool module includes four independent stratified sub-experience pools: a steady-state operating condition sub-experience pool, a high-carbon low-load operating condition sub-experience pool, a low-carbon high-load operating condition sub-experience pool, and an extreme dynamic operating condition sub-experience pool. Each sub-experience pool stores sample data for its corresponding operating condition. The sample data includes state information, droop coefficient, system reward, and the system's state information at the next moment. The sample data is stored using a first-in, first-out (FIFO) principle; when the number of samples in each stratified sub-experience pool reaches a preset capacity limit, the oldest stored sample data is replaced. During training of the improved SAC controller, a fixed-ratio stratified sampling method is used to extract training samples from each sub-experience pool. The sampling ratio for the extreme dynamic operating condition sub-experience pool is 30%–40%, and the sampling ratios for the steady-state operating condition sub-experience pool, the high-carbon low-load operating condition sub-experience pool, and the low-carbon high-load operating condition sub-experience pool differ by no more than 10%.

[0015] As an optional approach, the parameter update process of the improved SAC controller is as follows: based on the training samples obtained by hierarchical sampling, the parameters of the policy network and the two evaluation networks are updated using the mini-batch gradient descent method; the two evaluation networks complete the parameter update by minimizing the temporal difference residual, the calculation of which incorporates the working condition adaptive weighted reward value; the policy network completes the parameter update by minimizing the expected KL divergence, maximizing the cumulative weighted reward with entropy regularization as the objective; and the target network parameters are updated using a soft update method to improve the stability of the parameter update process.

[0016] As an optional approach, the dynamic carbon factor of each node is determined by the grid average emission factor method, sensitivity analysis method, pricing mechanism method or carbon emission flow tracking method. The dynamic carbon factor represents the amount of carbon emissions generated by a user consuming a unit of electricity at the current location and time.

[0017] As an optional approach, the step of calculating the control voltage and power distribution of each node of the DC microgrid at the next moment using the droop coefficient includes: inputting the droop coefficient into the droop equation to calculate the control voltage of each node; and calculating the power distribution of the corresponding node based on the ratio of the droop coefficients of different nodes, wherein the ratio of the droop coefficients of any two nodes is inversely proportional to the ratio of the power distribution.

[0018] According to another aspect of the present invention, an electronic device is also provided, comprising: a processor, and a memory storing a program, the program including instructions that, when executed by the processor, cause the processor to perform the method described in any of the preceding claims.

[0019] According to another aspect of the invention, a non-transitory machine-readable medium storing computer instructions for causing the computer to perform any of the methods described above is also provided.

[0020] The proposed control method for DC microgrids based on dynamic carbon factor introduces dynamic carbon factor as the core optimization parameter and combines an improved SAC deep reinforcement learning framework with a dual-mode hybrid control mechanism to achieve adaptive low-carbon control of DC microgrids under steady-state and dynamic operating conditions.

[0021] On the one hand, this invention utilizes a working condition stratified experience pool module to divide the running samples into various working condition types and adopts differentiated sampling ratios for stratified sampling training, effectively solving the problems of low coverage of extreme working condition samples and poor model generalization ability caused by conventional random sampling. On the other hand, through the working condition adaptive weighted reward module, the voltage control weight and carbon emission control weight are dynamically adjusted according to the real-time operating conditions, and the global weight coefficient guides the model training direction, realizing dynamic adaptation of control priority under different working conditions.

[0022] Ultimately, while ensuring that the deviation between node voltage and bus voltage meets the control accuracy requirements, the power distribution ratio of each node is made close to the inverse relationship of the dynamic carbon factor, effectively reducing the overall carbon emission intensity of the DC microgrid. At the same time, it has strong robustness to load disturbances and new energy fluctuations, thus solving the problem that DC microgrids in related technologies only consider power balance, resulting in high carbon emissions and poor control accuracy. Attached Figure Description

[0023] To more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the accompanying drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are merely some embodiments of the present invention, and those skilled in the art can obtain other embodiments based on these drawings without creative effort.

[0024] Figure 1 This is a flowchart of a control method for a DC microgrid based on a dynamic carbon factor, according to an embodiment of the present invention.

[0025] Figure 2 This is a schematic diagram of the topology of a DC microgrid system according to an embodiment of the present invention.

[0026] Figure 3 This is a schematic diagram of the voltage-current characteristic curve of the DC microgrid droop control according to an embodiment of the present invention.

[0027] Figure 4 This is a schematic diagram of the power control results of a DC microgrid system according to an embodiment of the present invention.

[0028] Figure 5 This is a schematic diagram of the voltage control results of a DC microgrid system according to an embodiment of the present invention.

[0029] Figure 6 This is a schematic diagram of the structure of the electronic device created by this invention. Detailed Implementation

[0030] Embodiments of the present invention will now be described in more detail with reference to the accompanying drawings. While some embodiments of the present invention are shown in the drawings, it should be understood that the present invention can be implemented in various forms and should not be construed as limited to the embodiments set forth herein. Rather, these embodiments are provided to provide a more thorough and complete understanding of the present invention. It should be understood that the drawings and embodiments of the present invention are for illustrative purposes only and are not intended to limit the scope of protection of the present invention.

[0031] To balance the internal power of a DC microgrid system, reduce carbon emissions, and precisely control the voltage of the microgrid, embodiments of this invention provide a control method for a DC microgrid based on a dynamic carbon factor, such as... Figure 1 As shown, it includes:

[0032] Step S101: Obtain the current state information of the DC microgrid system, wherein the state information includes the voltage of each node of the DC microgrid system, the power of each node, the dynamic carbon factor of each node, and the bus voltage of the DC microgrid system.

[0033] Step S102: Input the state information into the improved SAC controller that can perform deep reinforcement learning. The improved SAC controller includes a policy network, two evaluation networks, a dual-mode hybrid control module, a working condition adaptive weighted reward module, and a working condition hierarchical experience pool module. The working condition adaptive weighted reward module is used to update the parameter weights of the controller for reinforcement learning, and the working condition hierarchical experience pool module is used to store training data.

[0034] Step S103: The dual-mode hybrid control module performs real-time determination of steady-state / dynamic operating conditions based on state information and outputs the operating condition determination result.

[0035] Step S104: Based on the operating condition determination result, adaptively output the droop coefficient of each node of the DC microgrid, wherein the strategy network and the evaluation network are used to output the droop coefficient under at least one operating condition determination result.

[0036] Step S105: Calculate the control voltage and power distribution of each node of the DC microgrid at the next moment based on the droop coefficient, and control the DC microgrid according to the control voltage and power distribution to distribute power according to the dynamic carbon factor, and ensure that the node voltage and bus voltage meet the accuracy requirements.

[0037] The reinforcement learning model provided in the embodiments of this invention can be a hetero-policy or homo-policy deep reinforcement learning algorithm, including deep deterministic policy gradient, double-delay deep deterministic policy gradient, and proximal policy optimization. The SAC (Soft Actor-Critic) algorithm used in this embodiment is a hetero-policy action-evaluation algorithm based on the maximum entropy reinforcement learning framework, designed to address the high sample complexity and low stability issues of model-free deep reinforcement learning methods. Leveraging the advantages of offline updates and the maximum entropy framework, SAC exhibits superior control performance compared to other hetero-policy or homo-policy deep reinforcement learning algorithms, including deep deterministic policy gradient, double-delay deep deterministic policy gradient, and proximal policy optimization.

[0038] State information refers to the set of data that characterizes the real-time operating conditions and energy status of the microgrid, and serves as the input to the control algorithm of this invention. Specifically, the voltage of each node refers to the real-time voltage amplitude at each distributed power source, energy storage device, or load connection point in the DC microgrid; the power of each node includes the active and reactive power of each node, reflecting the power flow distribution of the system; and the bus voltage is the voltage value of the common DC bus in the DC microgrid, serving as a key reference indicator for stable system operation.

[0039] The dynamic carbon factor of each node is the real-time carbon emission intensity corresponding to the unit electricity consumption of the node. The dynamic carbon factor is dynamically updated with the penetration rate of renewable energy in the power grid, the start-up and shutdown status of thermal power units and load changes, and is used to introduce carbon emission costs or low-carbon constraints into power allocation.

[0040] Real-time acquisition of multi-dimensional operational data from the system provides comprehensive and accurate information support for subsequent operating condition identification, strategy calculation, and low-carbon control. The real-time state of the physical system is converted into digital signals that the controller can process, ensuring that control decisions are based on the current actual operating conditions.

[0041] The improved SAC controller is an intelligent control model based on the maximum entropy reinforcement learning framework. SAC can simultaneously maximize the expected cumulative reward and policy entropy to enhance exploration ability and robustness.

[0042] The policy network is responsible for directly outputting control actions based on the input state information, i.e., the probability distribution of the droop coefficient. Two evaluation networks are used to assess the value of the actions output by the policy network, suppressing overestimation by taking the minimum of the two evaluation networks.

[0043] The dual-mode hybrid control module is used for determining the operating condition type in subsequent steps. The operating condition adaptive weighted reward module can dynamically adjust the weighting coefficients of different optimization objectives in the reward function according to the current operating condition, such as voltage deviation, power distribution accuracy, and carbon emissions, so that the controller can adaptively focus on the key control objectives under the current operating condition.

[0044] The working condition stratified experience pool module classifies and stores historical interaction data according to steady-state working conditions and dynamic working conditions, and performs stratified sampling during training to avoid strategy bias caused by uneven sample distribution.

[0045] This step inputs the collected state information into the improved SAC controller. Through a deep reinforcement learning framework, the improved SAC controller is able to learn the optimal strategy autonomously from historical control experience. This overcomes the problems of control methods in related technologies being highly dependent on precise mathematical models and having difficulty handling multi-objective dynamic optimization, thus laying the model foundation for subsequent adaptive output control parameters.

[0046] The dual-mode hybrid control module can quickly classify the current operating conditions of the DC microgrid system. The operating condition determination includes two main categories: steady-state operating conditions and dynamic operating conditions. The steady-state operating condition refers to the continuous operating state in which key parameters such as system voltage and power are within the allowable range and fluctuate with small fluctuations and no significant disturbances. The dynamic operating condition refers to the process in which system parameters change rapidly due to events such as load changes, fluctuations in renewable energy output, and grid-connected / off-grid switching.

[0047] The dual-mode hybrid control module calculates indicators such as bus voltage change rate, node power deviation and its trend in real time, and sets corresponding threshold logic or sliding window-based discrimination criteria to achieve accurate identification and rapid output of operating conditions.

[0048] Because DC microgrids have fundamentally different control strategy requirements under steady-state and dynamic conditions—steady-state requirements emphasize accuracy and energy efficiency, while dynamic requirements emphasize response speed and stability—the improved SAC controller can select a matching control mode based on the operating condition determination, avoiding the problem of insufficient applicability of a single control strategy under complex conditions.

[0049] The droop factor is a control parameter used in DC microgrids to realize distributed power distribution and voltage regulation. The droop factor is the proportional coefficient between voltage deviation and power change, which determines the response weight of each node when it undertakes power changes.

[0050] The dual-mode hybrid control module is used in the online inference phase of the controller. Based on real-time state information, it determines the operating condition and adaptively selects the generation mode for the droop coefficient. Specifically, when the system is in a dynamic operating condition, it switches to the SAC dynamic control mode, where the policy network outputs dynamically optimized droop coefficients based on the real-time system state, achieving adaptive and precise control of power and voltage. This adaptive output mechanism enables the controller to fully utilize the nonlinear mapping capabilities of deep reinforcement learning to quickly generate adaptive parameters under dynamic operating conditions.

[0051] When the system is in steady-state operation, it switches to fixed droop optimization control mode, that is, the fixed droop coefficients for steady-state operation are fixed and saved after training with the SAC algorithm for multiple operating conditions. This eliminates the need for real-time neural network inference and greatly reduces the amount of calculation required by the controller.

[0052] The adaptive output mechanism of the droop coefficient enables the improved SAC controller to fully utilize the nonlinear mapping capability of deep reinforcement learning to quickly generate adaptive parameters under dynamic conditions, while balancing computational efficiency and low-carbon optimization goals under steady-state conditions.

[0053] By converting the operating condition judgment results into specific control parameters, the problem of fixed coefficients and inability to adaptively adjust with operating conditions in traditional droop control is solved. This enables a direct response to dynamic carbon factors, allowing power distribution to actively follow changes in carbon emission intensity, thereby reducing overall system carbon emissions while maintaining voltage stability.

[0054] Based on the droop coefficient of each node and combined with the power-voltage droop characteristic equation of the DC microgrid, the reference voltage value that each node needs to output in the next control cycle is calculated, and then the power command value of each distributed power source or energy storage unit is determined, forming a complete voltage control and power distribution scheme.

[0055] Power allocation is performed based on a dynamic carbon factor, meaning that through the coupling design of the droop coefficient and the dynamic carbon factor, nodes with lower carbon emission intensity can undertake more power output, achieving low-carbon dispatch from the source side. Node voltages and bus voltages meet accuracy requirements, ensuring that the voltages of each node and the common bus remain stable within the preset allowable deviation range after control, guaranteeing the power supply quality and operational safety of the system.

[0056] The proposed control method for DC microgrids based on dynamic carbon factor introduces dynamic carbon factor as the core optimization parameter and combines an improved SAC deep reinforcement learning framework with a dual-mode hybrid control mechanism to achieve adaptive low-carbon control of DC microgrids under steady-state and dynamic operating conditions.

[0057] This embodiment utilizes a stratified operating condition experience pool module to classify operating samples into various operating condition types and employs differentiated sampling ratios for stratified sampling training, effectively solving the problems of low coverage of extreme operating condition samples and poor model generalization ability caused by conventional random sampling. On the other hand, through an adaptive weighted reward module, the voltage control weight and carbon emission control weight are dynamically adjusted according to the real-time operating conditions, and the global weight coefficient guides the model training direction, achieving dynamic adaptation of control priorities under different operating conditions.

[0058] Ultimately, while ensuring that the deviation between node voltage and bus voltage meets the control accuracy requirements, the power distribution ratio of each node is made close to the inverse relationship of the dynamic carbon factor, effectively reducing the overall carbon emission intensity of the DC microgrid. At the same time, it has strong robustness to load disturbances and new energy fluctuations, thus solving the problem that DC microgrids in related technologies only consider power balance, resulting in high carbon emissions and poor control accuracy.

[0059] As an optional solution, a dual-mode hybrid control module is used to determine steady-state / dynamic operating conditions in real time based on state information. This includes setting voltage fluctuation thresholds, power fluctuation thresholds, and dynamic carbon factor fluctuation thresholds. A steady-state condition is determined when all three of the following conditions are met; otherwise, a dynamic condition is determined. Specifically: The percentage relative deviation between the voltage of each node and the bus voltage is less than the voltage fluctuation threshold for multiple consecutive control cycles. The control cycle is the real-time control sampling cycle of the DC microgrid, where the percentage relative deviation between node voltage and bus voltage = |node voltage - bus voltage| / bus voltage × 100%; The percentage relative fluctuation of the system's rated power and the output power of each node in a single cycle is less than the power fluctuation threshold, where the percentage relative fluctuation of the power in a single cycle = |current power - previous cycle power| / system rated power × 100%; The dynamic carbon factor e of each node... i The percentage of the single-period relative fluctuation of the electric carbon factor is less than the dynamic electric carbon factor fluctuation threshold, where the percentage of the single-period relative fluctuation of the electric carbon factor = |current e i -Previous period e i | / Previous cycle e i ×100%.

[0060] The dual-mode hybrid control module is used to quickly and accurately classify and identify the current operating status of the DC microgrid. The dual-mode hybrid control module includes an operating condition determination unit, which is responsible for performing the operating condition identification function in this step.

[0061] Specifically, three types of quantitative judgment indicators are pre-defined: voltage fluctuation threshold, power fluctuation threshold, and dynamic electric carbon factor fluctuation threshold. The voltage fluctuation threshold is the allowable percentage deviation of each node's voltage relative to the bus voltage, used to quantify the degree of voltage stability. The power fluctuation threshold is the allowable percentage fluctuation based on the system's rated power, used to quantify the severity of power changes. The dynamic electric carbon factor fluctuation threshold is the allowable percentage fluctuation of the dynamic electric carbon factor itself, used to quantify the degree of change in carbon emission intensity.

[0062] The system's voltage stability can be verified when the percentage relative deviation between the voltage at each node and the bus voltage is less than the voltage fluctuation threshold for several consecutive control cycles. The percentage relative deviation of the node voltage is calculated using the formula |node voltage - bus voltage| / bus voltage × 100%.

[0063] The percentage of the relative fluctuation of the system's rated power and the output power of each node within a single cycle is less than the power fluctuation threshold. This condition verifies the power balance stability of the system. The percentage of the relative fluctuation of the power within a single cycle is calculated using the formula |current power - previous cycle power| / system rated power × 100%.

[0064] The percentage of the single-period relative fluctuation of the dynamic electrocarbon factor at each node is less than the dynamic electrocarbon factor fluctuation threshold. This condition verifies the stability of the system's carbon emission characteristics. The percentage of the single-period relative fluctuation of the electrocarbon factor is expressed by the formula |current e i -Previous period e i | / Previous cycle e i The control period is calculated as ×100%; the control period is the real-time control sampling period of the DC microgrid.

[0065] When all three conditions are met, the current operating condition is determined to be a steady-state condition; if any one condition is not met, it is determined to be a dynamic condition.

[0066] This embodiment effectively classifies the complex and ever-changing operating states of a DC microgrid by constructing multi-dimensional and quantifiable operating condition determination rules, solving the problems of ambiguous operating condition identification and single control strategies in related technologies. The operating condition determination results provide precise trigger signals for the adaptive switching of dual-mode hybrid control in subsequent steps. This allows for the use of a low-computation fixed droop control mode to improve operating efficiency under steady-state conditions, while activating the SAC dynamic control mode under dynamic conditions to ensure response speed and control accuracy. This provides a crucial decision-making basis for balancing low-carbon system operation, voltage stability, and computational efficiency.

[0067] As an optional approach, based on the operating condition determination result, the droop coefficient of each node of the DC microgrid is adaptively output, including: when the operating condition determination result is a steady-state condition, switching to a fixed droop optimization control mode, and outputting the fixed droop coefficient after offline optimization by the improved SAC controller, wherein the fixed droop coefficient is the droop coefficient value converged by the improved SAC controller after training on steady-state operating condition samples; when the operating condition determination result is a dynamic condition, switching to the SAC dynamic control mode, and the policy network outputs the dynamically optimized droop coefficient based on real-time state information; wherein the policy network maps the state information through multiple serial fully connected layers, generates action probability parameters through parallel mean and standard deviation layers, and then samples the action probability parameters through a reparameterization method to obtain the dynamically optimized droop coefficient.

[0068] The droop factor is a proportionality coefficient between voltage deviation and power change. It determines the response weight of each distributed power source or energy storage unit when handling power changes and is a key control variable for power distribution and voltage regulation. This embodiment adaptively selects the droop factor generation method based on the operating condition determination results, thereby achieving synergistic optimization of control efficiency and control accuracy.

[0069] The dual-mode hybrid control module also includes a fixed droop optimization unit and a SAC dynamic control unit. The fixed droop optimization unit is used for low computational control under steady-state conditions, while the SAC dynamic control unit is used for adaptive and precise control under dynamic conditions.

[0070] When the operating condition is determined to be a steady-state condition, the control mode of the fixed droop optimization unit is executed. The droop coefficient is not calculated in real time, but is directly called from the fixed droop coefficient saved after offline optimization by the improved SAC controller. This fixed droop coefficient is saved as the mean of the action probability distribution output by the policy network under steady-state conditions after the controller completes multi-condition training, without the need for real-time neural network inference during the control cycle.

[0071] Offline optimization refers to the process of training the policy network and evaluation network parameters using a deep reinforcement learning algorithm during the controller training phase, utilizing a large number of steady-state operating condition samples stored in the hierarchical experience pool. After the network parameters converge, the droop coefficient output by the policy network is saved as the optimal fixed parameter for that operating condition.

[0072] Under steady-state conditions, the fixed droop optimization control mode avoids real-time forward inference of the neural network in each control cycle, which greatly reduces the computational load and hardware resource consumption of the controller. At the same time, since the fixed droop coefficient has been fully optimized through reinforcement learning, it can still ensure voltage accuracy and low carbon distribution effect during steady-state operation, thus achieving the control goal of low power consumption and high efficiency.

[0073] When the operating condition is determined to be a dynamic operating condition, the SAC dynamic control unit's control mode is used. Based on the real-time collected system status information, the strategy network in the improved SAC controller outputs the dynamically optimized droop coefficient online.

[0074] Specifically, the current state information is input into the policy network. The policy network consists of multiple cascaded fully connected layers, with each fully connected layer followed by a modified linear unit as the activation function for nonlinear mapping. Subsequently, the policy network structure is split into parallel mean and standard deviation layers, which output the mean and standard deviation parameters of the action probability distribution, respectively, jointly representing the Gaussian distribution of the action probability distribution.

[0075] Finally, the probability distribution is sampled using a reparameterization method, which involves sampling noise from a standard normal distribution and performing a linear transformation of the mean and standard deviation to obtain a dynamically optimized droop coefficient. The reparameterization method makes the sampling process differentiable, ensuring the smoothness and control effectiveness of the final output droop coefficient.

[0076] Under dynamic operating conditions, the SAC dynamic control mode is adopted. The strategy network can quickly adjust the droop coefficient according to the real-time changes in the system state, so as to achieve rapid response and precise control to voltage fluctuations, power disturbances and changes in carbon factor. This solves the problems of droop control in dynamic operating conditions with slow response and poor adaptability in related technologies.

[0077] This embodiment organically combines the nonlinear mapping capability of deep reinforcement learning with the simplicity and efficiency of droop control through a dual-mode droop coefficient output mechanism that adapts to different operating conditions. On the one hand, under dynamic operating conditions, it fully leverages the adaptive and precise control advantages of the SAC algorithm to ensure that power allocation can follow the changing trend of the dynamic carbon factor in real time, achieving low-carbon optimization while ensuring voltage stability. On the other hand, under steady-state operating conditions, the fixed droop coefficient optimized offline significantly reduces computational resource consumption, avoiding the resource consumption of continuous inference of the neural network.

[0078] As an optional approach, dynamic operating conditions include: high-carbon low-load operating conditions, low-carbon high-load operating conditions, and extreme dynamic operating conditions. The policy network outputs dynamically optimized droop coefficients based on real-time state information, including: when the operating condition is determined to be high-carbon low-load, outputting a dynamic droop coefficient with a preference for carbon emission optimization; when the operating condition is determined to be low-carbon high-load, outputting a dynamic droop coefficient with a preference for voltage stability; and when the operating condition is determined to be extreme dynamic, outputting a dynamic droop coefficient with a balanced preference for both carbon emission optimization and voltage stability. The dynamic droop coefficients are all generated by the policy network based on real-time state information through fully connected layers, mean layers, standard deviation layers, and reparameterization methods. Policy networks with different preferences receive different reward weights during training.

[0079] When the operating condition determination result is a dynamic operating condition, in order to further achieve refined and differentiated control, this embodiment subdivides the dynamic operating condition into three categories: high carbon low load operating condition, low carbon high load operating condition, and extreme dynamic operating condition. For different dynamic operating conditions, a strategy network is used to output dynamic droop coefficients with different optimization preferences.

[0080] Under high-carbon, low-load conditions, the system load demand is low and the voltage deviation is small. At this time, the control objective is mainly to reduce carbon emissions. Therefore, the strategy network outputs a dynamic droop coefficient with carbon emission optimization as the main preference, prioritizing low-carbon and high-efficiency operation.

[0081] Under low-carbon, high-load conditions, the system load is heavy and the risks of power and voltage fluctuations are significant. At this time, the control focus shifts to maintaining system stability. Therefore, the strategy network outputs a dynamic droop coefficient with voltage stability as the primary preference, prioritizing the accuracy requirements of bus voltage and node voltage.

[0082] Under extreme dynamic conditions, system power, voltage, and carbon emission factor all fluctuate drastically. A single optimization objective is insufficient to balance system performance. Therefore, the strategy network outputs a dynamic droop coefficient that balances carbon emission optimization and voltage stability, thereby maximizing the level of low-carbon operation while ensuring the safe and stable operation of the system.

[0083] All the aforementioned dynamic droop coefficients are uniformly generated by the policy network of the improved SAC controller. The policy network collects real-time system state information, such as node voltage, power, dynamic carbon factor, and bus voltage, and performs feature mapping through multiple fully connected layers. Then, it outputs action probability distribution parameters through parallel mean and standard deviation layers. Finally, it uses a reparameterization method to complete action sampling, ultimately obtaining dynamically optimized droop coefficients adapted to the current operating condition. During training, the policy networks corresponding to different dynamic operating conditions are configured with differentiated reward weights through an adaptive weighted reward module. This allows the network to develop optimization preferences for the corresponding operating conditions during learning, achieving adaptive and accurate output of droop coefficients under dynamic operating conditions.

[0084] As an optional approach, the method calculates the control voltage and power distribution of each node in the DC microgrid at the next time step based on the droop coefficient. After controlling the DC microgrid according to the control voltage and power distribution, the method further includes: generating a system reward through the operating condition adaptive weighted reward module of the improved SAC controller to adjust the parameters of the strategy network and the evaluation network. The system reward is an operating condition adaptive weighted reward, including a basic reward item and an operating condition weight coefficient. The basic reward item includes voltage control reward and power-carbon emission control reward. The calculation formula for the operating condition adaptive weighted reward is as follows: In the formula, For adaptive weighted rewards based on operating conditions, For global weights, For voltage weighting, For carbon emission weighting, As a reward for voltage control, The reward is for power-carbon emission control; the operating condition weighting coefficient includes global weight, voltage weight, and carbon emission weight. Different weighting coefficients are selected under different operating conditions. The operating conditions include at least: steady-state operating condition, high carbon low load operating condition, low carbon high load operating condition, and extreme dynamic operating condition.

[0085] The adaptive weighted reward module is used to generate system reward signals that can guide the training direction of deep reinforcement learning, and adapts the control priority requirements under different operating conditions by dynamically adjusting the reward weights.

[0086] Voltage control reward characterizes the deviation between the voltage of each node and the bus voltage in the system and the reference value. The smaller the deviation, the higher the reward, which is used to guide the controller to maintain voltage stability.

[0087] The power-carbon emission control reward characterizes the difference between the proportional relationship between the output power of each node in the system and the inverse proportional relationship between the dynamic carbon factor of each node. The smaller the difference, the higher the reward, which is used to guide the controller to allocate power according to the low-carbon principle.

[0088] The global weight is used to adjust the overall reward scale of the samples under different operating conditions, while the voltage weight and carbon emission weight are used to adjust the relative importance of the voltage control target and the carbon emission control target in the reward function, respectively.

[0089] The formula for calculating the adaptive weighted reward based on operating conditions integrates the weighting coefficients of operating conditions with the basic reward items, enabling adaptive scaling of the reward value and adjustment of target preferences under different operating conditions. Operating conditions include at least: steady-state operating conditions, high-carbon low-load operating conditions, low-carbon high-load operating conditions, and extreme dynamic operating conditions. Extreme dynamic operating conditions may include extreme situations such as sudden changes in renewable energy output, sudden changes in the carbon factor, and sudden load changes.

[0090] Under steady-state conditions, voltage weighting Lower, carbon emission weight The control priority is relatively high, favoring carbon emission optimization and operational efficiency, with a fixed global weight coefficient. The sample reward weight is relatively low under this operating condition, which avoids a large number of steady-state samples dominating model training. The moderate sampling ratio of steady-state samples and the low global reward weight also prevent a large number of steady-state samples from dominating the training direction.

[0091] Voltage weighting under high carbon and low load conditions Lower, carbon emission weight The control priority is relatively high, biased towards carbon emission optimization, and the global weight coefficient is fixed. Relatively high.

[0092] Voltage weighting under low-carbon, high-load conditions High, carbon emission weight The control priority is relatively low, favoring voltage stability and power supply safety, with a fixed global weighting coefficient. The sampling ratio and global weight of high-carbon low-load and low-carbon high-load operating conditions are equal, which can balance the training contribution of the two typical operating conditions.

[0093] Voltage weighting under extreme dynamic conditions Moderate, carbon emission weight The control priority is moderate, balancing voltage stability and carbon emission control, with a fixed global weighting coefficient. The highest weight is awarded to samples under extreme operating conditions. Higher global weights result in heavier penalties for control deviations, thus reinforcing the learning of extreme operating conditions. Extreme dynamic operating condition samples have the highest sampling ratio and the highest global weight in the experience pool of the operating condition hierarchical experience pool module; both together strengthen the model's learning of this type of scenario.

[0094] This embodiment uses an adaptive weighted reward mechanism to strongly correlate the optimization objective of deep reinforcement learning with the real-time operating conditions and dynamic carbon factor characteristics of the microgrid. This solves the problem of untargeted adaptation of reward functions in related technologies, enabling the controller to automatically adjust its optimization focus according to different operating conditions during training. Under steady-state conditions, it focuses on low-carbon operation and efficiency improvement; under high-carbon and low-load conditions, it strengthens carbon emission control; under low-carbon and high-load conditions, it prioritizes voltage stability; and under extreme dynamic conditions, it balances the dual control objectives of voltage and carbon emissions, thereby achieving a dynamic balance of multi-objective control under different operating conditions.

[0095] As an optional solution, the formula for calculating the voltage control bonus is as follows: In the formula, Let be the voltage control reward at time t, n be the number of nodes, and i be the node number. Let v be the actual voltage value at time t of the i-th node. ref This is the system voltage reference value, v bus,t It is the actual value of the system bus voltage at time t.

[0096] The embodiments of the present invention provide The term measures the squared deviation of the output voltage of each node from the reference value, and the summation reflects the overall deviation across multiple nodes; the summation term Monitor the voltage deviation of each distributed node, such as distributed power sources and load nodes, to ensure local node voltage stability.

[0097] The embodiments of the present invention provide This item measures the squared deviation of the bus voltage from the reference value separately, emphasizing the global indicator of bus voltage to avoid situations where local stability exists but global instability occurs, such as when some node voltages meet the standard but the bus voltage deviates too much.

[0098] In the formula, the negative sign indicates that the smaller the deviation, the higher the reward. The reward is maximized when the deviation is 0, and decreases or even becomes negative when the deviation increases.

[0099] This design directly transforms the voltage stability objective into an optimizable reward signal, forcing the model to learn strategies to reduce voltage deviations across all nodes and the bus, ensuring that the microgrid voltage remains close to the reference value and preventing equipment failures or power quality issues caused by overvoltage or undervoltage. The combination of these two approaches allows the reward function to both constrain voltage consistency at local nodes and guarantee system-level voltage stability, aligning with the characteristics of distributed control and centralized coordination in DC microgrids.

[0100] Using the squared deviation form has a non-linear increasing characteristic. The penalty is small for small deviations and increases sharply for large deviations. This design makes the model more sensitive to large voltage deviations and prioritizes learning to correct significant deviations, such as sudden loads causing a sudden drop in voltage at a node. At the same time, it allows for small deviations to exist, avoiding energy loss caused by over-control, and achieving a balance between control accuracy and control cost.

[0101] The squared deviation function is continuous and differentiable, and its gradient is easy to calculate. It can be directly used to evaluate the parameter updates of the network and policy network, and minimize the loss through gradient descent.

[0102] When the policy network updates by minimizing the expected KL divergence, the smooth reward gradient can be effectively propagated to the optimization process of policy entropy, enabling the controller to achieve a balance between maximizing the cumulative weighted reward and maintaining the policy's exploratory nature. When the evaluation network updates by minimizing the temporal difference residual, the continuous reward surface ensures the stable convergence of the value estimation. This mathematical property ensures that the reward signal can be effectively propagated to the model parameters, aligning the optimization direction of the policy network with the objective of reducing voltage deviation. Thus, under the constraint of the entropy regularization term, the model converges to a stable and exploratory control policy.

[0103] The voltage control reward function provided by the embodiments of this invention can effectively guide the model to learn a control strategy for stabilizing DC microgrid voltage by accurately quantifying local and global voltage deviations, enhancing sensitivity by using a squared form, and adapting to the gradient optimization mechanism of reinforcement learning, thus laying the foundation for the synergistic effect of voltage stability and low-carbon optimization.

[0104] As an optional option, the formula for calculating the power-carbon emission control bonus is as follows: In the formula, The power-carbon emission control reward at time t. For the power allocation of the i-th node at time t, Let be the power reference value of the i-th node at time t, calculated using the following formula: In the formula, Let e ​​be the total power of the system at time t. i Let be the dynamic electric carbon factor of the i-th node at time t.

[0105] In the power reference value formula provided by the embodiments of the present invention, This reflects the principle that the lower the electric carbon factor, the higher the weight. If the dynamic electric carbon factor e of a certain node... i Small, then Larger values ​​have a higher proportion in the denominator, corresponding to a higher reference power. Large, that is, the total power P of the system tIt will favor low-carbon nodes. This design directly transforms the dynamic carbon factor into a quantitative basis for power allocation, making the goal of prioritizing the use of low-carbon energy a calculable reference value instead of a qualitative requirement, and ensuring that the reward function is strictly aligned with the carbon emission reduction target.

[0106] Dynamic carbon factor e i It will change with time and node status, such as when wind power output is high at night, its e i It may decrease; during peak shaving of thermal power, its e i It may rise. Reference value. Will follow e i Real-time updates ensure that the low-carbon baseline for power allocation is always matched to the current operating conditions. This dynamism ensures that the model does not adhere to a fixed power allocation ratio, but can flexibly adjust its strategy according to real-time changes in carbon emission intensity, maximizing carbon reduction benefits throughout the entire period.

[0107] In this invention, the power-carbon emission control reward formula provided by the embodiment adopts the form of squared deviation plus a negative sign, consistent with the voltage reward, when the actual power The closer to the reference value The smaller the sum of squared deviations, the higher the reward; the larger the deviation, such as excessive power allocation to high-carbon nodes, the lower the reward, or even negative. The nonlinear characteristics of the squared terms impose a stronger penalty on significant deviations, forcing the model to prioritize correcting the problem of excessive power output from high-carbon nodes or insufficient power output from low-carbon nodes, ensuring that power allocation strictly follows the low-carbon orientation.

[0108] The squared deviation of the reward function has a continuously differentiable mathematical form, and its gradient is simple to calculate, allowing it to be directly used for parameter updates in evaluation and policy networks. Meanwhile, the power reference value... Based on the total system power P t and the carbon factor e i The calculations are closely integrated with the power balance of the microgrid to avoid the reward signal becoming disconnected from the actual system operation, ensuring that the strategies learned by the model can be implemented in engineering.

[0109] The power-carbon emission control reward mechanism provided by the embodiments of this invention determines the low-carbon power allocation benchmark through dynamic weighted electric carbon factors and quantifies the allocation accuracy through squared deviation penalties. This achieves the core objective of prioritizing the use of low-carbon energy while dynamically adapting to changes in operating conditions, and also meets the gradient optimization requirements of reinforcement learning. Combined with voltage control rewards, it guides the model to maximize the carbon emission reduction benefits of the microgrid while ensuring voltage stability, perfectly aligning with the dual demands of power balance and low-carbon efficiency in new energy microgrids.

[0110] As an optional solution, the working condition stratified experience pool module includes four independent stratified sub-experience pools: steady-state working condition sub-experience pool, high-carbon low-load working condition sub-experience pool, low-carbon high-load working condition sub-experience pool, and extreme dynamic working condition sub-experience pool. Each sub-experience pool stores sample data for the corresponding working condition. The sample data includes state information, droop coefficient, system reward, and the system state information at the next moment. The sample data is stored according to the first-in, first-out principle. When the number of samples in each stratified sub-experience pool reaches the preset capacity limit, the oldest stored sample data is replaced. During the training of the improved SAC controller, a fixed-ratio stratified sampling method is used to extract training samples from each sub-experience pool. The sampling ratio of the extreme dynamic working condition sub-experience pool is 30%~40%, and the sampling ratios of the steady-state working condition sub-experience pool, the high-carbon low-load working condition sub-experience pool, and the low-carbon high-load working condition sub-experience pool do not differ by more than 10%.

[0111] The working condition hierarchical experience pool module is used to store and sample training data in the improved SAC controller. Its core function is to solve the problems of low coverage of extreme working condition samples and poor model generalization ability caused by random sampling using a single experience pool in related technologies.

[0112] When the operating condition determination result is a steady-state operating condition, the corresponding sample data will be stored in the steady-state operating condition sub-experience pool.

[0113] Dynamic operating conditions are further subdivided into high-carbon low-load operating conditions, low-carbon high-load operating conditions, and extreme dynamic operating conditions. When the operating condition is determined to be a dynamic operating condition, it is classified according to the carbon emission level and the load level, and the corresponding sample data is stored in the high-carbon low-load operating condition sub-experience pool, the low-carbon high-load operating condition sub-experience pool, and the extreme dynamic operating condition sub-experience pool.

[0114] The droop coefficient represents the action output by the controller; the system reward is the adaptive weighted reward value. The four-tuple structure, consisting of sample data including state information, droop coefficient, system reward, and the system state information at the next time step, satisfies the data requirements for temporal difference updates in deep reinforcement learning.

[0115] In terms of sample data storage and management, a first-in, first-out (FIFO) principle is adopted. That is, when the number of samples in each sub-experience pool reaches the preset capacity limit, newly collected samples replace the oldest stored sample data. This ensures that the experience pool always retains the latest and most timely working condition samples, avoiding the impact of outdated samples on model training.

[0116] The improved SAC controller uses a fixed-ratio hierarchical sampling method to extract training samples from each sub-experience pool during training. The sampling ratio of the extreme dynamic condition sub-experience pool is set to 30%~40%, giving priority to ensuring the coverage of extreme condition samples and enhancing the model's adaptability to dynamic disturbances such as sudden changes in renewable energy output, sudden changes in carbon factor, and sudden load changes.

[0117] The sampling ratios of the steady-state operating condition sub-experience pool, the high-carbon low-load operating condition sub-experience pool, and the low-carbon high-load operating condition sub-experience pool do not differ by more than 10%. This ensures the balance of training for the three types of operating condition samples while taking into account the synergistic improvement of steady-state control accuracy, carbon emission optimization effect, and voltage stability capability under high-load scenarios, thus avoiding the excessive dominance of one type of operating condition sample in the model training direction.

[0118] The operating condition hierarchical experience pool module proposed in this embodiment achieves classified storage and dynamic updating of multi-operating condition operating samples of DC microgrids through four independent hierarchical sub-experience pools and adopts a first-in-first-out (FIFO) principle for sample timeliness management. Based on this, a differentiated and targeted sample sampling strategy is formed through a fixed-ratio hierarchical sampling method. This effectively solves the problems of low coverage of extreme operating condition samples and poor model generalization ability to dynamic disturbance scenarios caused by random sampling in conventional deep reinforcement learning.

[0119] As an optional approach, the parameter update process of the improved SAC controller is as follows: based on the training samples obtained by hierarchical sampling, the parameters of the policy network and two evaluation networks are updated using the mini-batch gradient descent method; the two evaluation networks complete the parameter update by minimizing the temporal difference residuals, and the calculation of the temporal difference residuals incorporates the working condition adaptive weighted reward value; the policy network completes the parameter update by minimizing the expected KL divergence (Kullback-Leibler divergence), maximizing the cumulative weighted reward with entropy regularization as the objective; the target network parameters are updated using a soft update method to improve the stability of the parameter update process.

[0120] Mini-batch gradient descent is a deep neural network training method that involves randomly selecting a fixed number of samples from a large-scale training sample to calculate the gradient and update the parameters each time they are updated. Compared to full-batch gradient descent, this method significantly improves computational efficiency while ensuring the correctness of the convergence direction.

[0121] Temporal difference residuals are the loss function for evaluating network updates in reinforcement learning. They are defined as the difference between the target Q-value and the current Q-value, reflecting the degree of deviation of the evaluation network in assessing the value of state-action pairs. Minimizing temporal difference residuals can make the evaluation network's estimate gradually approach the true cumulative reward.

[0122] The calculation of time-series differential residuals incorporates the adaptive weighted reward value under different operating conditions, which ensures that the value assessment of the evaluation network can accurately reflect the control priority requirements under different operating conditions.

[0123] KL divergence is used to measure the difference between two probability distributions. Minimizing the expected KL divergence in policy network updates aims to make the updated policy network output distribution as close as possible to the current optimal policy distribution. At the same time, by introducing an entropy regularization term, the policy is encouraged to maintain a certain degree of randomness to enhance the exploration capability.

[0124] The policy network updates parameters by minimizing the expected KL divergence. Its maximization objective is the cumulative weighted reward with entropy regularization. That is, while pursuing high cumulative rewards, the policy's exploration capability is maintained by entropy regularization, which can avoid premature convergence to a suboptimal policy.

[0125] Soft update is a method for updating target network parameters. It refers to taking a weighted average of the current network parameters and the target network parameters in a certain proportion during each iteration, instead of directly assigning values. This ensures that the changes in the target network are smooth and significantly improves the stability of the training process.

[0126] This embodiment integrates the adaptive weighted reward value based on operating conditions into the temporal difference residual calculation, thus linking the evaluation network's value assessment with operating condition priority. This solves the problem in the conventional SAC algorithm where the reward function cannot adapt to the control requirements of different operating conditions. Simultaneously, by minimizing the KL divergence and introducing an entropy regularization term to update the policy network, necessary exploration capabilities are maintained while ensuring the convergence accuracy of the control policy. Furthermore, a soft update mechanism smooths changes in the target network, avoiding training instability caused by drastic fluctuations in target values ​​during training.

[0127] As an alternative approach, the dynamic carbon factor of each node can be determined using the grid average emission factor method, sensitivity analysis method, pricing mechanism method, or carbon emission flow tracking method. The dynamic carbon factor represents the amount of carbon emissions generated by a user consuming a unit of electricity at the current location and time.

[0128] The dynamic carbon factor represents the carbon emissions generated by a user consuming a unit of electricity at the current location and time. It reflects the spatiotemporal changes in carbon emission intensity during power generation in real time, specifically varying with factors such as the power generation energy structure, renewable energy penetration rate, start-up and shutdown status of thermal power units, load levels, and grid operation modes. The introduction of the dynamic carbon factor allows the power allocation of DC microgrids to be linked to real-time carbon emission intensity, providing a quantifiable optimization target for achieving low-carbon operation.

[0129] The grid average emission factor method calculates the average carbon emission factor based on the overall power generation structure and total power generation of the power grid. It is suitable for system-level macro carbon measurement. The data source for the grid average emission factor method is stable and the calculation is simple.

[0130] Sensitivity analysis determines the node carbon factor by calculating the marginal impact of changes in the power of each node on the total carbon emissions of the system. It can reflect the marginal contribution of node power adjustments to overall carbon emissions and is suitable for local optimization scenarios.

[0131] The pricing mechanism method converts carbon emission costs into electricity costs based on carbon market trading prices or carbon tax policies, and indirectly reflects carbon factors through economic signals. It is suitable for low-carbon dispatching in a market environment.

[0132] Carbon emission tracing is based on power flow analysis. It tracks the flow path of electrical energy from the generation side to the load side, accurately calculates the actual carbon emission responsibility borne by each load node, and can accurately reflect the spatiotemporal distribution characteristics of dynamic electrical carbon factors. It is suitable for refined low-carbon control.

[0133] This embodiment obtains real-time and accurate dynamic carbon factors for each node by selecting a calculation method that adapts to the structure and operating characteristics of the DC microgrid. This provides key parameters for the state input of the deep reinforcement learning controller in subsequent steps, enabling the controller to make optimized power allocation decisions based on real-time carbon emission intensity information.

[0134] As an optional approach, the control voltage and power distribution of each node in a DC microgrid at the next moment are calculated using the droop coefficient. This includes: inputting the droop coefficient into the droop equation to calculate the control voltage of each node; and calculating the power distribution of the corresponding node based on the ratio of the droop coefficients of different nodes, wherein the ratio of the droop coefficients of any two nodes is inversely proportional to the ratio of the power distribution.

[0135] Droop control is a traditional distributed control method in DC microgrids, used to achieve voltage control and power distribution in each unit, such as... Figure 2 As shown. Where V ref1 V ref2 This is the system output voltage under no-load conditions, i.e., the voltage reference value, which is generally set to V. ref1 =V ref2 d1 and d2 are the droop coefficients, u o1, u o2 Indicates the output voltage of the first and second nodes, i o1 i o2 Represents the output current of nodes 1 and 2; R linei U is the line resistance of the i-th branch. dc R represents the DC bus voltage. L Indicates the load of the DC microgrid system, i L This is the load current. (By...) Figure 2 The basic control relationship for droop control can be obtained as follows:

[0136]

[0137] When the downward coefficient d i Once determined, the node voltage decreases proportionally as the output power increases. This design allows each node to automatically adjust its voltage based on its own power output, avoiding the communication dependence of centralized control and achieving autonomous power balance among distributed power sources. Voltage adjustment is directly linked to power output, resulting in a fast response and rapid suppression of the impact of power fluctuations on voltage. For example, during sudden load changes, nodes can redistribute power through minor voltage adjustments.

[0138] According to the above formula, we can obtain... Figure 3 The voltage-current characteristic curve shown is given. Where u... o_s u o_l These represent the system output voltages when the droop factor is small and large, respectively. The horizontal axis I represents the output current of the DC microgrid nodes, used to reflect the load size. , It is the output current of branch 1 and branch 2 when the droop coefficient is small; , This represents the output current of branch 1 and branch 2 when the droop factor is large. The vertical axis V represents the output voltage of the node. ref This is the voltage reference value under no-load conditions. At this time, regardless of the droop factor, the node output voltage is equal to V. ref .

[0139] Sag coefficient d i The absolute value of the slope of the corresponding curve. Figure 3 A smaller droop coefficient corresponds to a flatter curve with a smaller absolute slope; a larger droop coefficient corresponds to a steeper curve with a larger absolute slope. At the same current, a smaller droop coefficient corresponds to a higher output voltage, while a larger droop coefficient corresponds to a lower output voltage. Therefore, the larger the droop coefficient, the more significant the impact of load changes on the voltage.

[0140] Due to line impedance and the fact that the internal voltages of each distributed power source are not entirely identical, the system output power characteristics are not completely consistent even with the same droop coefficient. The power p of each branch varies. i =u oi ×i oi The voltage is allocated based on the droop coefficient, and there is also a steady-state voltage deviation.

[0141] In a DC microgrid under droop control, the ratio of the droop coefficients of any two nodes is inversely proportional to the ratio of the distributed power. The power distribution ratio can be directly controlled by the droop coefficient. Power can be automatically distributed to each node according to the inverse relationship of the droop coefficient. No centralized command is required, which improves the scalability and reliability of the system. The failure of a single node does not affect the overall power distribution logic.

[0142] The control method provided by the embodiments of the present invention requires only local droop coefficients for each node to calculate control voltage and power distribution, without the need for real-time communication with other nodes, thus reducing communication delay and failure risk. The droop coefficient simultaneously determines the voltage adjustment range and the power distribution ratio, forming a closed-loop relationship between the two. Changes in power distribution will be automatically reflected in voltage adjustment through the droop equation, and small differences in voltage will drive power redistribution, ultimately achieving a balanced state where the voltage is stable within the allowable range and the power is distributed proportionally.

[0143] It should be noted that this embodiment also provides an optional implementation method, which will be described in detail below.

[0144] This embodiment provides a control method for a DC microgrid based on a dynamic carbon factor. It employs a reinforcement learning model for the policy network and evaluation network. The evaluation network determines the weight parameters of the policy network based on historical outputs and the evaluation value of the dynamic carbon factor. The current state information of the DC microgrid system is input into the policy network, and feature values ​​are extracted layer by layer through fully connected layers. These extracted feature values ​​are then input into the action distribution parameter layer of the policy network to generate action probability parameters. The probability parameters are sampled using a reparameterization method to obtain the final control command from the controller. The control command calculates the control voltage and power distribution of each node in the DC microgrid at the next moment, and precisely controls the DC microgrid according to the corresponding parameters. This solves the problems of DC microgrids that only consider power balance, resulting in high carbon emissions and poor control accuracy. It achieves the technical effect of rationally allocating internal power based on the dynamic carbon factor, reducing overall carbon emissions, and precisely controlling the maintenance voltage of the microgrid.

[0145] The model considered in this embodiment of the invention is a DC microgrid system consisting of 5 nodes, and the structural topology is shown in the figure below. Figure 2 As shown in the figure. u oi Let i represent the output voltage of the i-th node. oi d represents the output current of the i-th node. i This represents the droop coefficient of the i-th node. Figure 2 In the diagram, i=1, 2. The total system power is 50MW, and the voltage reference value V... ref =420V.

[0146] Working condition classification threshold setting:

[0147] Voltage fluctuation threshold: The maximum deviation between the voltage at each node and the bus voltage is less than 10% for five consecutive control cycles;

[0148] Power fluctuation threshold: The single-cycle fluctuation amplitude of the total system load power and the output power of each node is less than 10% of the rated power;

[0149] Fluctuation threshold of electric carbon factor: The single-period fluctuation amplitude of the dynamic electric carbon factor at each node is less than 10%;

[0150] High carbon threshold: average system carbon electrochemical factor ≥ 0.35 kg CO2 / kW; low carbon threshold: average system carbon electrochemical factor ≤ 0.15 kg CO2 / kW.

[0151] High load threshold: Total system load ≥ 35kW (70% of rated power); Low load threshold: Total system load ≤ 15kW (30% of rated power).

[0152] Improved SAC algorithm network and training parameters:

[0153] Design a controller based on an improved SAC deep reinforcement learning algorithm, which includes a policy network Actor and two evaluation networks Critic1 and Critic2. Both the policy and evaluation networks consist of four cascaded fully connected layers, each with 128 neurons, using ReLU as the activation function.

[0154] Experience pool setup: 4 stratified sub-experience pools, each with a maximum sample size of 10,000 groups; the sampling ratio during training is as follows: extreme dynamic operating condition sub-experience pool 40%, high carbon low load operating condition sub-experience pool 20%, low carbon high load operating condition sub-experience pool 20%, and steady-state operating condition sub-experience pool 20%.

[0155] Training parameters: discount factor γ=0.99, soft update coefficient τ=0.005, learning rate 3e-4, batch size 256, maximum training epochs 1000.

[0156] Parameters of the adaptive reward function for operating conditions:

[0157] Steady-state condition: Voltage weighting =0.4 is relatively low, indicating a low carbon emission weight. =0.6 is relatively high, indicating a high global weighting coefficient. =0.8;

[0158] High carbon, low load conditions: voltage weighting =0.3 is relatively low, indicating a low carbon emission weight. =0.7 is relatively high, global weight coefficient =1.2;

[0159] Low-carbon, high-load operating conditions: voltage weighting =0.7 is relatively high, indicating a high carbon emission weight. =0.3 is relatively low, global weight coefficient =1.2;

[0160] Extreme dynamic operating conditions: voltage weighting =0.5 is moderate, carbon emission weight =0.5 is moderate, global weight coefficient =1.5;

[0161] Get Figure 2 As shown, the current state information of the DC microgrid system includes the output voltage u of each node. oi The unit is V; the output power p of each node i The unit is kW; the dynamic electrocarbon factor e at each node i The unit is kgCO2 / kW. The node output power p... i =u oi ×i oi The node dynamic carbon factor represents the carbon emissions of each node's input power source, published by the power sources of each node. The model sets e... i =[0.4,0.2,0.1,0.2,0.4]kgCO2 / kW, the system average dynamic carbon dioxide factor is 0.26 kgCO2 / kW, and the operating condition is judged to be a steady state condition.

[0162] The controller's input is the system state s t Output a t It represents the droop coefficient of each node in the system. Where: system state s t The voltage value u of each node o1 u o2 u on Power value p 1,t p 2,t p n,t and system bus voltage value u bus,t The system consists of 11 dimensions, and its state can be represented by the following formula: .

[0163] System output a t It is the droop coefficient d of each node in the system. 1,t d 2,t d n,t This is used to control the input power and voltage of each node. It has a dimension of 5, and the system state can be represented by the following formula: .

[0164] The controller's input-output relationship is determined by the policy network and evaluation network through maximizing the control reward r. t Decision. System reward r t Characterizing the controller's control effect, including voltage control reward r vol,t Power-carbon emission control reward r pow,t Voltage control reward r vol,t The voltage v at each node of the system oi,t and bus voltage v bus,t Compared with reference value vref The smaller the deviation, the higher the reward. The voltage control reward formula is: .

[0165] Power-carbon emission control reward r pow,t The output power p of each node in the system i,t The smaller the difference between the proportional relationship between the nodes and the dynamic electric carbon factor proportional relationship at each node, the higher the reward. Assume the current total system power is P. t The desired power p of each node is obtained according to the dynamic carbon factor ratio of the nodes. refi,t They are [3MW, 6MW, 12MW, 6MW, 3MW].

[0166] Then the power-carbon emission control reward r is obtained. pow,t The formula is as follows: .

[0167] Combining voltage control rewards, power-carbon emission control rewards, and operating condition weighting coefficients, the final system reward function is obtained, resulting in the system reward: Among them, voltage weight =0.4, carbon emission weight =0.6 is relatively high, indicating a high global weighting coefficient. =0.8.

[0168] The controller and the system continuously interact to train the controller parameters, update the network parameters based on the experience playback mechanism of hierarchical sampling under working conditions, adjust the parameter values ​​of the action network and evaluation network based on system rewards until the training termination condition is met, and save the trained network parameters and steady-state optimized fixed droop coefficient.

[0169] Real-time control is implemented based on a trained action network and a dual-mode hybrid control module. The control mode is adaptively switched according to the real-time operating condition judgment results, and the corresponding droop coefficient is output to each node of the microgrid, ensuring that the node voltage and power changes reach the optimal control state. The power and voltage control effects are as follows: Figure 4 and Figure 5 As shown.

[0170] The power values ​​of the five nodes converged to 3.4MW, 6MW, 12MW, 6.2MW, and 3.1MW. The obtained power control results are close to the required ratio. The ratio of the five nodes is close to 1:2:4:2:1, and the overall power allocation deviation is 4%, indicating that the proposed method can achieve adjustment of each node according to the dynamic electric carbon factor ratio to minimize carbon emissions. The power allocation deviation index p... e The calculation formula is: , where p e As a power distribution deviation index, To obtain the desired power at each node according to the dynamic carbon factor ratio at each node, p i The power control results are as follows: The voltage values ​​at the five nodes converge to the bus voltage values ​​of 392V, 395V, 402V, 394V, 391V, and 387V. The results are close to the required voltage control. The deviations between the five nodes and the reference voltage are all less than 7%, indicating good voltage control performance. The formula for calculating the voltage deviation index is as follows: , where v ei The voltage deviation index, v refi The bus voltage value, v oi The output voltage values ​​for each node are shown. The voltage deviations for the five nodes are [6.67%, 5.95%, 4.29%, 6.19%, 6.90%].

[0171] Embodiments of the present invention also provide a computer program product, including a computer program, wherein the computer program, when executed by a computer's processor, is used to cause the computer to perform the method of an embodiment of the present invention.

[0172] An embodiment of the present invention also provides an electronic device, including: at least one processor; and a memory communicatively connected to the at least one processor. The memory stores a computer program executable by the at least one processor, which, when executed by the at least one processor, causes the electronic device to perform the method of the embodiment of the present invention.

[0173] refer to Figure 6 The present invention will now describe a structural block diagram of an electronic device that can serve as an embodiment of the present invention, serving as an example of a hardware device applicable to various aspects of the present invention. The electronic device is intended to represent various forms of digital electronic computer devices, such as laptop computers, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The electronic device can also represent various forms of mobile devices, such as personal digital processors, cellular phones, smartphones, wearable devices, and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely illustrative and are not intended to limit the implementation of the present invention described and / or claimed herein.

[0174] like Figure 6As shown, the electronic device includes a computing unit 601, which can perform various appropriate actions and processes based on a computer program stored in a read-only memory (ROM) 602 or a computer program loaded from a storage unit 608 into a random access memory (RAM) 603. The RAM 603 may also store various programs and data required for the operation of the electronic device. The computing unit 601, ROM 602, and RAM 603 are interconnected via a bus 604. An input / output (I / O) interface 605 is also connected to the bus 604.

[0175] Multiple components in the electronic device are connected to I / O interface 605, including: input unit 606, output unit 607, storage unit 608, and communication unit 609. Input unit 606 can be any type of device capable of inputting information into the electronic device. Input unit 606 can receive input digital or character information and generate key signal inputs related to user settings and / or function control of the electronic device. Output unit 607 can be any type of device capable of presenting information and may include, but is not limited to, a display, speaker, video / audio output terminal, vibrator, and / or printer. Storage unit 608 may include, but is not limited to, disks and optical discs. Communication unit 609 allows the electronic device to exchange information / data with other devices through computer networks such as the Internet and / or various telecommunications networks, and may include, but is not limited to, modems, network cards, infrared communication devices, and / or wireless communication transceivers, such as Bluetooth devices, WiFi devices, WiMax devices, cellular communication devices, and / or the like.

[0176] The computing unit 601 can be a variety of general-purpose and / or special-purpose processing components with processing and computing capabilities. Some examples of the computing unit 601 include, but are not limited to, CPUs, graphics processing units (GPUs), various special-purpose artificial intelligence (AI) computing units, various computing units running machine learning model algorithms, digital signal processors (DSPs), and any suitable processor, controller, microcontroller, etc. The computing unit 601 performs the various methods and processes described above. For example, in some embodiments, the method embodiments of the present invention can be implemented as computer programs tangibly contained in a machine-readable medium, such as storage unit 608. In some embodiments, part or all of the computer program can be loaded and / or installed on an electronic device via ROM 602 and / or communication unit 609. In some embodiments, the computing unit 601 can be configured to perform the methods described above by any other suitable means (e.g., by means of firmware).

[0177] Computer programs for implementing the methods of embodiments of the present invention may be written in any combination of one or more programming languages. These computer programs may be provided to a processor or controller of a general-purpose computer, special-purpose computer, or other programmable data processing apparatus, such that when executed by the processor or controller, the computer programs cause the functions / operations specified in the flowcharts and / or block diagrams to be performed. The computer programs may be executed entirely on a machine, partially on a machine, or as a standalone software package, partially on a machine and partially on a remote machine, or entirely on a remote machine or server.

[0178] In the context of embodiments of this invention, a machine-readable medium can be a tangible medium that may contain or store a program for use by or in conjunction with an instruction execution system, apparatus, or device. A machine-readable medium can be a machine-readable signal medium or a machine-readable storage medium. A machine-readable signal medium may include, but is not limited to, electronic, magnetic, optical, electromagnetic, or infrared systems, apparatus, or devices, or any suitable combination of the foregoing. More specific examples of machine-readable storage media include electrical connections based on one or more wires, portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fibers, portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination of the foregoing.

[0179] It should be noted that the term "comprising" and its variations used in the embodiments of this invention are open-ended, meaning "including but not limited to". The term "based on" means "at least partially based on". The term "one embodiment" means "at least one embodiment"; the term "another embodiment" means "at least one additional embodiment"; the term "some embodiments" means "at least some embodiments". The modifications of "one" and "a plurality" mentioned in the embodiments of this invention are illustrative and not restrictive, and those skilled in the art should understand that unless explicitly indicated otherwise in the context, they should be understood as "one or more".

[0180] The user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, stored data, displayed data, etc.) involved in the embodiments of this invention are subject to strict compliance with relevant laws, regulations, and regulatory requirements in their collection, storage, use, processing, transmission, provision, and disclosure, and adhere to the principles of legality, legitimacy, necessity, and good faith. The acquisition of relevant information and data is premised on the user's explicit consent or other legitimate reasons, and a clear and convenient authorization management approach is provided to the user, allowing the user to independently choose to consent, withdraw consent, or refuse to provide relevant information. For functions that rely on user information, if the user does not authorize or withdraws authorization, the corresponding technical function cannot be implemented, and the technical solution of this invention is not applicable in this scenario. The steps described in the method implementation embodiments provided by the embodiments of this invention can be executed in different orders and / or in parallel. Furthermore, the method implementation embodiments may include additional steps and / or omit the steps shown. The scope of protection of this invention is not limited in this respect.

[0181] The steps described in the method embodiments provided by the present invention can be performed in different orders and / or in parallel. Furthermore, the method embodiments may include additional steps and / or omit the steps shown. The scope of protection of the present invention is not limited in this respect.

[0182] The term "embodiment" in this specification refers to a specific feature, structure, or characteristic described in connection with an embodiment that may be included in at least one embodiment of the invention. The appearance of this phrase in various places throughout the specification does not necessarily imply the same embodiment, nor does it imply independence or alternativeity from other embodiments. The various embodiments in this specification are described in a related manner, with reference to each other for similar or identical parts. In particular, for apparatus, device, and system embodiments, since they are substantially similar to method embodiments, the description is relatively simple, and relevant details are referred to in the description of the method embodiments.

[0183] The above embodiments merely illustrate several implementation methods of the present invention, and while the descriptions are relatively specific and detailed, they should not be construed as limiting the scope of protection. It should be noted that those skilled in the art can make various modifications and improvements without departing from the inventive concept of the present invention, and these all fall within the scope of protection of the present invention. Therefore, the scope of protection of the present invention should be determined by the appended claims.

Claims

1. A control method for a DC microgrid based on dynamic carbon factor, characterized in that, include: Obtain the current state information of the DC microgrid system, wherein the state information includes the voltage of each node of the DC microgrid system, the power of each node, the dynamic carbon factor of each node, and the bus voltage of the DC microgrid system. The state information is input into an improved SAC controller capable of deep reinforcement learning. The improved SAC controller includes a policy network, two evaluation networks, a dual-mode hybrid control module, an adaptive weighted reward module, and a hierarchical experience pool module. The adaptive weighted reward module is used to update the parameter weights of the controller for reinforcement learning, and the hierarchical experience pool module is used to store training data. The dual-mode hybrid control module performs real-time determination of steady-state / dynamic operating conditions based on the state information and outputs the operating condition determination result. Based on the operating condition determination result, the droop coefficient of each node of the DC microgrid is adaptively output, wherein the strategy network and the evaluation network are used to output the droop coefficient under at least one operating condition determination result; The control voltage and power distribution of each node of the DC microgrid at the next moment are calculated based on the droop coefficient. The DC microgrid is controlled according to the control voltage and power distribution to distribute power according to the dynamic carbon factor, and the node voltage and bus voltage meet the accuracy requirements.

2. The method according to claim 1, characterized in that, The dual-mode hybrid control module performs real-time determination of steady-state / dynamic operating conditions based on the state information, including: A voltage fluctuation threshold, a power fluctuation threshold, and a dynamic carbon factor fluctuation threshold are set. The voltage fluctuation threshold is the percentage deviation of each node voltage from the bus voltage reference value. The power fluctuation threshold is the percentage of the system's rated power. The dynamic carbon factor fluctuation threshold is the percentage of the dynamic carbon factor. A steady-state operating condition is determined when all three of the following conditions are met simultaneously; if any one condition is not met, a dynamic operating condition is determined: The percentage relative deviation between the voltage of each node and the bus voltage is less than the voltage fluctuation threshold for several consecutive control cycles; wherein, the control cycle is the real-time control sampling cycle of the DC microgrid, and the percentage relative deviation between the node voltage and the bus voltage = |node voltage - bus voltage| / bus voltage × 100%; The percentage of the relative fluctuation of the system's rated power and the output power of each node in a single cycle is less than the power fluctuation threshold, wherein the percentage of the relative fluctuation of the power in a single cycle = |current power - previous cycle power| / system rated power × 100%; Dynamic electrocarbon factor e at each node i The percentage of the single-period relative fluctuation amplitude of the electric carbon factor is less than the dynamic electric carbon factor fluctuation threshold, wherein the percentage of the single-period relative fluctuation amplitude of the electric carbon factor = |current e i -Previous period e i | / Previous cycle e i ×100%.

3. The method according to claim 1, characterized in that, Based on the operating condition determination results, the droop coefficient of each node of the DC microgrid is adaptively output, including: When the working condition determination result is a steady-state working condition, switch to fixed droop optimization control mode and output the fixed droop coefficient after offline optimization by the improved SAC controller. The fixed droop coefficient is the droop coefficient value converged by the improved SAC controller after training on steady-state working condition samples. When the working condition determination result is a dynamic working condition, the system switches to SAC dynamic control mode, and the policy network outputs a dynamically optimized droop coefficient based on real-time state information. The policy network maps the state information through multiple serial fully connected layers, generates action probability parameters through parallel mean and standard deviation layers, and then samples the action probability parameters through a reparameterization method to obtain the dynamically optimized droop coefficient.

4. The method according to claim 3, characterized in that, The dynamic operating conditions include: high-carbon low-load operating conditions, low-carbon high-load operating conditions, and extreme dynamic operating conditions; the droop coefficient, dynamically optimized by the strategy network based on real-time state information, includes: When the operating condition is determined to be a high-carbon, low-load condition, a dynamic droop coefficient with a preference for carbon emission optimization is output. When the operating condition is determined to be a low-carbon, high-load condition, the output dynamic droop coefficient is preferred to maintain voltage stability. When the operating condition determination result is an extreme dynamic operating condition, the output dynamic droop coefficient is given as a balanced preference between carbon emission optimization and voltage stability. The dynamic droop coefficients are generated by the policy network based on real-time state information through a fully connected layer, a mean layer, a standard deviation layer, and a reparameterization method. Policy networks with different preferences have different reward weights during training.

5. The method according to claim 1, characterized in that, The method further includes: calculating the control voltage and power distribution of each node in the DC microgrid at the next time step based on the droop coefficient; controlling the DC microgrid according to the control voltage and power distribution; and then further including: The improved SAC controller uses an adaptive weighted reward module to generate system rewards to adjust the parameters of the policy network and the evaluation network. The system rewards are adaptive weighted rewards, which include basic reward items and operating condition weight coefficients. The basic reward items include voltage control rewards and power-carbon emission control rewards. The formula for calculating the adaptive weighted reward under the specified working conditions is as follows: ； In the formula, For adaptive weighted rewards based on operating conditions, For global weights, For voltage weighting, For carbon emission weighting, As a reward for voltage control, Rewards for power-carbon emission control; The operating condition weighting coefficients include the global weight, voltage weight, and carbon emission weight. Different weighting coefficients are selected under different operating conditions. The operating conditions include at least: steady-state operating condition, high-carbon low-load operating condition, low-carbon high-load operating condition, and extreme dynamic operating condition.

6. The method according to claim 5, characterized in that, The formula for calculating the voltage control reward is as follows: ； In the formula, Let be the voltage control reward at time t, n be the number of nodes, and i be the node number. Let v be the actual voltage value at time t of the i-th node. ref This is the system voltage reference value, v bus,t It is the actual value of the system bus voltage at time t.

7. The method according to claim 5, characterized in that, The calculation formula for the power-carbon emission control incentive is as follows: ； In the formula, The power-carbon emission control reward at time t. For the power allocation of the i-th node at time t, p refi,t Let be the power reference value of the i-th node at time t, calculated using the following formula: In the formula, Let e ​​be the total power of the system at time t. i Let be the dynamic electric carbon factor of the i-th node at time t.

8. The method according to claim 1, characterized in that, The working condition stratified experience pool module includes four independent stratified sub-experience pools: steady-state working condition sub-experience pool, high-carbon low-load working condition sub-experience pool, low-carbon high-load working condition sub-experience pool, and extreme dynamic working condition sub-experience pool. Each sub-experience pool stores sample data for the corresponding working condition. The sample data includes state information, droop coefficient, system reward, and the system state information at the next moment. The sample data is stored according to the first-in, first-out principle. When the number of samples in each stratified sub-experience pool reaches the preset capacity limit, the earliest stored sample data is replaced. During training, the improved SAC controller uses a fixed-ratio hierarchical sampling method to extract training samples from each sub-experience pool. The sampling ratio of the extreme dynamic operating condition sub-experience pool is 30%~40%, and the sampling ratios of the steady-state operating condition sub-experience pool, the high-carbon low-load operating condition sub-experience pool, and the low-carbon high-load operating condition sub-experience pool do not differ by more than 10%.

9. The method according to claim 8, characterized in that, The parameter update process of the improved SAC controller is as follows: Based on the training samples obtained by hierarchical sampling, the parameters of the policy network and the two evaluation networks are updated using the mini-batch gradient descent method. The two evaluation networks update parameters by minimizing the temporal difference residual, and the calculation of the temporal difference residual incorporates the working condition adaptive weighted reward value; The policy network updates parameters by minimizing the expected KL divergence, and maximizes the cumulative weighted reward with entropy regularization. The target network parameters are updated using a soft update method, which improves the stability of the parameter update process.

10. The method according to claim 1, characterized in that, The dynamic carbon factor of each node is determined by the grid average emission factor method, sensitivity analysis method, pricing mechanism method or carbon emission flow tracking method. The dynamic carbon factor represents the amount of carbon emissions generated by a user consuming a unit of electricity at the current location and time.

11. The method according to claim 1, characterized in that, The calculation of the control voltage and power distribution of each node of the DC microgrid at the next moment using the droop coefficient includes: Input the droop coefficient into the droop equation to calculate the control voltage of each node; The power allocation of the corresponding nodes is calculated based on the ratio of the droop coefficients of different nodes. The ratio of the droop coefficients of any two nodes is inversely proportional to the ratio of the allocated power.

12. An electronic device, comprising: A processor and a memory storing a program, characterized in that the program includes instructions that, when executed by the processor, cause the processor to perform the method according to any one of claims 1 to 11.

13. A non-transitory machine-readable medium storing computer instructions, characterized in that, The computer instructions are used to cause the computer to perform the method according to any one of claims 1 to 11.

Images