Distributed rooftop photovoltaic device

Abstract

This invention relates to the field of photovoltaic equipment technology, specifically disclosing a distributed rooftop photovoltaic system, including a photovoltaic array, an energy storage unit, a local load unit, a grid-connected inverter, and an energy management unit. The energy management unit comprises a data sensing module, a collaborative decision-making module, and an instruction distribution and execution module. The collaborative decision-making module employs a trained deep reinforcement learning network. This network takes real-time collected system state data as input, interacts with the environment, and maximizes long-term cumulative rewards to autonomously learn and generate collaborative control instructions. The instruction distribution and execution module parses the control instructions into specific control signals for the grid-connected inverter, energy storage unit, and adjustable load. This invention achieves adaptive real-time collaborative optimization between power generation, energy storage, power consumption, and grid connection through a data-driven approach, effectively improving the system's economy, safety, and operational robustness under conditions of random fluctuations in photovoltaic power and load, and time-varying electricity prices.

Description

Technical Field

[0001] This invention relates to the field of photovoltaic equipment technology, and in particular to a distributed rooftop photovoltaic device. Background Technology

[0002] With the increasing prevalence of distributed rooftop photovoltaic (PV) systems, energy management is crucial for system economy and stability. Existing energy management methods are mainly divided into two categories: rule-based control and model predictive control (MMCC) optimization. Rule-based methods, such as the "self-consumption with surplus power fed into the grid" model, have rigid strategies that cannot adapt to real-time fluctuations in PV output and load demand, as well as the time-varying characteristics of electricity market prices, limiting their potential for economic optimization. While MMCC optimization methods can perform forward-looking dispatch, their optimization effectiveness heavily relies on precise mathematical models of PV, load, and energy storage. However, in actual operation, PV power generation exhibits strong randomness and intermittency, and load demand is uncertain, making it difficult to establish accurate predictive models. This leads to model mismatch and deviations of optimization results from the true optimal solution in practical applications. Summary of the Invention

[0003] To address the problems of weak adaptability to environmental changes and reliance on precise mathematical models in the energy management of existing distributed rooftop photovoltaic systems, this invention provides a distributed rooftop photovoltaic device.

[0004] The technical solution adopted in this invention is:

[0005] A distributed rooftop photovoltaic device includes a photovoltaic array, an energy storage unit, a local load unit, a grid-connected inverter, and an energy management unit communicatively connected to the energy storage unit, the local load unit, and the grid-connected inverter.

[0006] The energy management unit includes;

[0007] The data sensing module is used to collect data in real time at the current moment. The power generation of the photovoltaic array The power consumption of the local load unit The state of charge of the energy storage unit and charging / discharging power limits, and electricity price signals ;

[0008] The collaborative decision-making module is used to run a trained deep reinforcement learning network. The deep reinforcement learning network takes the real-time data and historical data collected by the data sensing module as state input and outputs collaborative control actions for the energy storage unit, the adjustable load in the local load unit, and the grid-connected inverter.

[0009] The instruction distribution and execution module is used to parse the control actions output by the collaborative decision-making module into charging and discharging power instructions for the energy storage unit. Switching commands for specified adjustable loads and grid-connected power commands for grid-connected inverters. And issue it to the corresponding device for execution;

[0010] The deep reinforcement learning network is configured to learn strategies for dynamic and collaborative optimization of self-consumption, peak shaving and valley filling, and participation in grid demand response, while satisfying equipment operation constraints, by maximizing long-term cumulative rewards.

[0011] Preferably, the deep reinforcement learning network is constructed based on a proximal policy optimization algorithm, and its state space... Action space and reward function The definition is as follows:

[0012] state space For at any time Observed system state vector:

[0013] ;

[0014] in, A cyclic encoding vector for each time of day. A cyclic encoding vector for the days of the week;

[0015] Action space To enable deep reinforcement learning networks to perform at any time Output continuous action vector:

[0016] ;

[0017] in, This is a normalized reference value for grid-connected power. This is a normalized reference value for energy storage charging and discharging power. This serves as a reference value for the switching probability of adjustable loads.

[0018] reward function for:

[0019] ;

[0020] Among them, economic incentives , This represents the actual grid-connected power; purchased electricity is positive, and sold electricity is negative. To control the time interval between decisions;

[0021] Safety Rewards Used to penalize out-of-bounds states of charge in energy storage systems. and These are the upper and lower limits of the energy storage state of charge. This is the penalty coefficient;

[0022] Power Grid Friendly Rewards Used to smooth grid-connected power fluctuations. This is the moving average of the grid-connected power in the previous period. For smoothing coefficients;

[0023] , and These are the weighting coefficients for each reward item.

[0024] Preferably, the deep reinforcement learning network includes a policy network. and a value network ,in and These are network parameters;

[0025] The policy network It is implemented using an Actor network, and its network architecture is as follows:

[0026] Input layer: Number of neurons and state vector The dimensions are the same;

[0027] First hidden layer: Fully connected layer, containing 256 neurons, using the ReLU activation function;

[0028] The second hidden layer is a fully connected layer containing 128 neurons and using the ReLU activation function.

[0029] Output layer: contains action vectors Neurons of the same dimension, for and Use the Tanh activation function to limit the output to 0. ,for Use the Sigmoid activation function to limit the output to 0. ;

[0030] The value network A Critic network is used to evaluate state value, and its network architecture is as follows:

[0031] Input layer: Number of neurons and state vector The dimensions are the same;

[0032] First hidden layer: Fully connected layer, containing 256 neurons, using the ReLU activation function;

[0033] The second hidden layer is a fully connected layer containing 128 neurons and using the ReLU activation function.

[0034] Output layer: Single neuron, linear output, representing the state. The estimated cumulative return;

[0035] The training process of the deep reinforcement learning network includes:

[0036] Collect experience data through interaction with the environment. The data is then stored in an experience replay buffer; small batches of data are sampled from the buffer, and the advantage function is calculated using generalized advantage estimation. The policy network parameters are updated by maximizing the pruning objective function of the proximal policy optimization. :

[0037] ;

[0038] in, For time Expected estimate This represents the probability ratio between the old and new strategies. For the old policy network before the update, For clipping hyperparameters;

[0039] The value network parameters are updated by minimizing the mean squared error loss of the value network. :

[0040] ;

[0041] in, The target value for updating the value network.

[0042] Preferably, the energy management unit further includes a photovoltaic power ultra-short-term probabilistic prediction module, used to provide prediction information to the collaborative decision-making module;

[0043] The photovoltaic power ultra-short-term probabilistic prediction module is built based on long short-term memory networks and Monte Carlo Dropout, and its execution steps include:

[0044] S1, Input the most recent Historical photovoltaic power sequence for hours and the weather feature vector at the corresponding time. Among them, weather feature vector Includes irradiance, temperature, and humidity;

[0045] S2. Input the sequence into a two-layer stacked Long Short-Term Memory (LSTM) network, and apply a Dropout rate to each layer of the LSM network. Monte Carlo Dropout;

[0046] S3. During the prediction phase, the Long Short-Term Memory network is... Each forward propagation inference produces a different output due to the randomness of Dropout, thus yielding the future... Photovoltaic power within a time period Predicted trajectory ;

[0047] S4, based on the above Given a predicted trajectory, calculate the probability distribution of the predicted power at each future time point, extract its quantiles, and generate the future... Power prediction interval within the time period The predicted interval is then input into the collaborative decision-making module as an extended state.

[0048] Preferably, during the decision-making process, the collaborative decision-making module includes the photovoltaic power prediction range. The state representation is incorporated using a weighted expectation method, specifically by calculating the future... Weighted expected value of photovoltaic power over time ;

[0049] ;

[0050] in, This is the risk preference coefficient, whose value is based on real-time electricity price signals. and the state of charge of energy storage units Dynamic adjustments and more proactive decision-making are employed to pursue profitability.

[0051] The calculated weighted expectation sequence It is concatenated with the current state vector and together they serve as the extended state input of the deep reinforcement learning network.

[0052] Preferably, the adjustable load in the local load unit is aggregated and represented using a virtual energy storage model;

[0053] For the An adjustable load, whose virtual energy storage model is based on a virtual state of charge. Virtual charging and discharging power limit , Self-loss rate and charge / discharge efficiency The state update equation is defined as follows:

[0054] ;

[0055] in, This is the virtual energy storage rated capacity. and They are respectively The virtual charging power and virtual discharging power allocated to the load at all times must meet operational constraints. , , ;

[0056] The energy management unit includes a virtual energy storage aggregation module, which is used to connect and equate multiple adjustable load virtual energy storage models in parallel according to power and capacity to obtain the total virtual state of charge of the aggregated virtual energy storage. Total virtual charging and discharging power limit , This information is then provided to the collaborative decision-making module as status information of the system's controllable resources.

[0057] Preferably, the instruction distribution and execution module determines the action output by the collaborative decision-making module. ,Sure The total virtual energy storage power that needs to be adjusted at all times Based on a preset load priority list, the specific actions for each adjustable load are determined using the following steps:

[0058] like If it indicates that virtual energy storage charging power needs to be increased, then the loads in the activating state are traversed in order of priority from low to high, and the power required to activate them is calculated. until the cumulative power meets the requirement. And generate the corresponding load start command;

[0059] like This indicates a need to increase the virtual energy storage discharge power. Therefore, the loads in operation are traversed in descending order of priority, and the power that can be released when they are turned off is calculated. until the cumulative power meets the requirement. And generate the corresponding load shutdown command.

[0060] Preferably, the energy management unit further includes a multi-timescale rolling optimization framework, which includes:

[0061] Long-cycle optimization layer: based on duration The time interval is defined as the time period, and the running period is defined as the future time. Internally, based on photovoltaic power forecasting, load forecasting, and time-of-use pricing, and with the objective of minimizing daily operating costs, a mixed-integer linear programming approach is used to obtain the future... Reference planning curves for internal energy storage units and adjustable loads, including energy storage reference power. and adjustable load switching status plan;

[0062] Short-cycle execution layer: based on duration The time interval is controlled in real time by the collaborative decision-making module, and the economic reward item in its reward function is... Revised to:

[0063] ;

[0064] in, This is obtained by interpolating the long-cycle optimization layer planning curve. Real-time energy storage reference power, To track the weighting coefficients.

[0065] The beneficial effects of this invention are:

[0066] 1. Enhanced adaptive optimization capabilities in complex and uncertain environments. Deep reinforcement learning networks autonomously learn optimal strategies through continuous interaction with the environment. They do not rely on mathematical models of photovoltaics and loads that are difficult to establish accurately. They can directly handle multiple uncertainties such as sudden changes in illumination, random fluctuations in load, and time-varying electricity prices, achieving near-optimal control in unknown or time-varying dynamic environments. This overcomes the performance degradation problem caused by model mismatch in traditional model predictive control.

[0067] 2. It achieves automatic coordination across multiple time scales and objectives. Through the designed multi-objective composite reward function, it can automatically learn during training how to dynamically balance and coordinate multiple coupled or even conflicting objectives such as maximizing economic benefits, ensuring equipment safety, and enhancing grid friendliness. Its decision-making dimensions and flexibility far exceed those of rule control based on fixed thresholds.

[0068] 3. Possesses end-to-end optimization and continuous evolution potential. Adopting an end-to-end framework of "perception-decision-execution," raw operational data can directly drive strategy optimization. As system operational data accumulates, strategies are continuously updated through online or offline learning, enabling the system's control performance to continuously evolve and improve over time, possessing a continuous optimization capability not found in traditional preset strategy systems. Attached Figure Description

[0069] Figure 1 This is a schematic diagram of a distributed rooftop photovoltaic device in an embodiment of the present invention;

[0070] Figure 2 This is a schematic diagram of the prediction process of the photovoltaic power ultra-short-term probabilistic prediction module in an embodiment of the present invention. Detailed Implementation

[0071] The present invention will now be described in detail with reference to the accompanying drawings and embodiments.

[0072] Example

[0073] A distributed rooftop photovoltaic system, such as Figure 1 As shown, it includes a photovoltaic array (comprising multiple rooftop photovoltaic panels), an energy storage unit, a local load unit, a grid-connected inverter, and an energy management unit that is communicatively connected to the energy storage unit, the local load unit, and the grid-connected inverter.

[0074] The energy management unit includes;

[0075] The data sensing module is used to collect data in real time at the current moment. The power generation of the photovoltaic array (Positive values ​​indicate power generation), the power consumption of the local load unit. (Positive values ​​indicate consumption), the state of charge of the energy storage unit. (The range of values ​​is) These correspond to the limits for full discharge and full charge, respectively, as well as the charge / discharge power limits and the electricity price signal. (The price is positive when purchasing electricity, and negative or another price when selling electricity);

[0076] The collaborative decision-making module is used to run a trained deep reinforcement learning network. The deep reinforcement learning network takes the real-time data and historical data collected by the data sensing module as state input and outputs collaborative control actions for the energy storage unit, the adjustable load in the local load unit, and the grid-connected inverter.

[0077] The instruction distribution and execution module is used to parse the control actions output by the collaborative decision-making module into charging and discharging power instructions for the energy storage unit. Switching commands for specified adjustable loads and grid-connected power commands for grid-connected inverters. And issue it to the corresponding device for execution;

[0078] The deep reinforcement learning network is configured to learn strategies for dynamic and collaborative optimization of self-consumption, peak shaving and valley filling, and participation in grid demand response, while satisfying equipment operation constraints, by maximizing long-term cumulative rewards.

[0079] Existing distributed photovoltaic (PV) energy management systems mostly employ control methods based on fixed rules or simple model predictions. These methods struggle to cope with the randomness and volatility of PV output and load demand, as well as the time-varying nature of electricity market prices. This results in insufficient overall system economic optimization and limited flexibility. In contrast, this invention uses data-driven deep reinforcement learning as a collaborative decision-making module, replacing traditional physical model-based optimization controllers. Through continuous interaction with the system environment (including PV, load, energy storage, and the grid), it autonomously learns the optimal collaborative control strategy from historical data and a high-dimensional state space, without relying on precise and difficult-to-establish system mathematical models. The system can adaptively handle multiple uncertainties (such as sudden changes in sunlight and load fluctuations), achieving complex and dynamic real-time collaborative optimization among power generation, energy storage, electricity consumption, and grid connection. This maximizes the system's full lifecycle economic benefits (including electricity cost savings, demand response benefits, and delayed expansion investment) under the constraint of safe equipment operation. Its optimization dimensionality, adaptive capability, and robustness in handling uncertainties are significantly superior to traditional methods.

[0080] To ensure effective learning of complex policies that balance economy, security, and grid friendliness, and to address issues such as reward sparsity, unstable training, or policies deviating from actual physical constraints, the deep reinforcement learning network is constructed based on a proximal policy optimization algorithm, and its state space... Action space and reward function The definition is as follows:

[0081] state space For at any time Observed system state vector:

[0082] ;

[0083] in, This is a cyclic encoding vector for times of day; specifically, it maps the 24 hours of a day to a cycle, using sine and cosine functions for encoding. To capture the periodic characteristics of time, It is a cyclic encoding vector of the days of the week. For example, one-hot encoding can be used, which is a 7-dimensional vector corresponding to Monday to Sunday.

[0084] Action space To enable deep reinforcement learning networks to perform at any time Output continuous action vector:

[0085] ;

[0086] in, This is a normalized grid-connected power reference value, which differs from the actual grid-connected power command. The conversion relationship is as follows , This is the maximum allowable power of the grid-connected inverter. This is a normalized reference value for energy storage charge and discharge power, with positive values ​​for charging and negative values ​​for discharging. It is related to the actual charge and discharge power command. The conversion relationship is as follows , This is the maximum charging and discharging power of the energy storage unit. This is a reference value for the switching probability of adjustable loads. The higher the value, the more adjustable loads the system tends to switch on or off at the current moment.

[0087] reward function for:

[0088] ;

[0089] Among them, economic incentives , This represents the actual grid-connected power; purchased electricity is positive, and sold electricity is negative. To control and make decisions over time intervals, the unit is usually minutes (min) or hours (h), for example... ;

[0090] Safety Rewards Used to penalize out-of-bounds states of charge in energy storage systems. and These are the upper and lower limits of the energy storage state of charge, for example... , , The penalty coefficient is a large positive scalar, such as... This is a strategy used to ensure that the network's rigorous learning does not violate energy storage safety constraints;

[0091] Power Grid Friendly Rewards Used to smooth grid-connected power fluctuations. The formula is: (This is the sliding average of the grid-connected power in the previous period.) , The length of the sliding window (e.g., the past 15 control cycles). For smoothing coefficients, if possible... ;

[0092] , and Here, represents the weighting coefficient for each reward item, and is a preset positive scalar used to balance the importance of the three optimization objectives: economy, safety, and grid friendliness. It can be set to... , , .

[0093] The state space integrates real-time power, state of charge, electricity price, and time period information, providing ample decision-making basis for the deep reinforcement learning network. The action space directly maps to continuous adjustment commands for key controllable equipment, achieving fine-grained control. The reward function is crucial for guiding the deep reinforcement learning network; here, a multi-objective composite reward function is designed:

[0094] 1. This directly transforms economic costs into negative rewards, driving deep reinforcement learning networks to minimize electricity costs or maximize electricity sales revenue.

[0095] 2. A quadratic penalty term is used to provide strong negative feedback when the state of charge is close to the boundary, forcing the deep reinforcement learning network to adjust the charging and discharging strategy in advance, ensuring energy storage safety and avoiding damage to the device by the trained strategy;

[0096] 3. Encourage smooth grid connection of power to reduce the impact on the distribution network and improve the friendliness of system access.

[0097] By adjusting the weight coefficients, the priority of optimization objectives can be flexibly set, enabling deep reinforcement learning networks to learn complex balance strategies of "making money," "ensuring safety," and "protecting the power grid" simultaneously during exploration. Its learning objectives are highly consistent with the actual operating objectives of the system.

[0098] The deep reinforcement learning network includes a policy network. and a value network ,in and These are the network parameters, representing the sets of parameters (including weights and biases) to be trained for the policy network and the value network, respectively. The parameter is The policy network, input state Output action The probability distribution, The parameter is Value network, input state Output the estimated cumulative return for this state;

[0099] The policy network It is implemented using an Actor network, and its network architecture is as follows:

[0100] Input layer: Number of neurons and state vector The dimensions are the same;

[0101] First hidden layer: Fully connected layer, containing 256 neurons, using the ReLU activation function;

[0102] The second hidden layer is a fully connected layer containing 128 neurons and using the ReLU activation function.

[0103] Output layer: contains action vectors Neurons of the same dimension, for and Use the Tanh activation function to limit the output to 0. ,for Use the Sigmoid activation function to limit the output to 0. ;

[0104] The value network A Critic network is used to evaluate state value, and its network architecture is as follows:

[0105] Input layer: Number of neurons and state vector The dimensions are the same;

[0106] First hidden layer: Fully connected layer, containing 256 neurons, using the ReLU activation function;

[0107] The second hidden layer is a fully connected layer containing 128 neurons and using the ReLU activation function.

[0108] Output layer: Single neuron, linear output, representing the state. The estimated cumulative return;

[0109] The training process of the deep reinforcement learning network includes:

[0110] Collect experience data through interaction with the environment. The data is then stored in an experience replay buffer; small batches of data are sampled from the buffer, and the advantage function is calculated using generalized advantage estimation. The pruning objective function is optimized by maximizing the proximal strategy. To update the policy network parameters :

[0111] ;

[0112] in, For time The expected value is approximated in actual training by averaging mini-batch samples. This represents the probability ratio between the old and new strategies. For the old policy network before the update, For a moment The advantage function estimate represents the state... Next action Advantage relative to the average level is typically calculated using generalized advantage estimation. The clipping hyperparameter is a small positive number (e.g., ...). ), used to limit the magnitude of each policy update;

[0113] By minimizing the mean squared error loss of the value network To update value network parameters :

[0114] ;

[0115] in, The target value for updating the value network is typically calculated using the following formula: , This is the discount factor (usually 0.99). These are parameters of the target value network, periodically obtained from... synchronous.

[0116] The Actor network (policy network) proposes actions based on the current state. Its two-layer hidden structure provides sufficient nonlinear fitting capability to learn complex policies. The output layer uses different activation functions for different actions based on their physical meaning (continuous adjustment / discrete probability), ensuring the output conforms to the actual control command range. The Critic network (value network) evaluates the quality of the current state, providing direction for the Actor network's updates. Training is performed using a proximal policy optimization algorithm, the core of which is to prune probability ratios... This limits the magnitude of each policy update, avoiding drastic fluctuations during training and significantly improving training stability. The experience replay buffer breaks down temporal correlations between data points, improving data utilization. Generalized advantage estimation can more accurately assess the value of actions. This specific and complete architecture and training process ensures feasibility and repeatability.

[0117] To address the inherent intermittency and strong randomness of photovoltaic power generation, and to provide the collaborative decision-making module with uncertainty information on power in future time periods, enabling decisions to balance optimality and robustness and avoid decision failure due to prediction errors, the energy management unit also includes a photovoltaic power ultra-short-term probabilistic prediction module, which provides prediction information to the collaborative decision-making engine.

[0118] The photovoltaic power ultra-short-term probabilistic prediction module is built based on long short-term memory networks and Monte Carlo Dropout, such as... Figure 2 As shown, its execution steps include:

[0119] S1, Input the most recent Historical photovoltaic power sequence for hours and the weather feature vector at the corresponding time. Among them, weather feature vector Includes irradiance, temperature, and humidity;

[0120] S2. Input the sequence into a two-layer stacked Long Short-Term Memory (LSTM) network, and apply a Dropout rate to each layer of the LSM network. Monte Carlo Dropout;

[0121] S3. During the prediction phase, the Long Short-Term Memory network is... Each forward propagation inference produces a different output due to the randomness of Dropout, thus yielding the future... Photovoltaic power within a time period Predicted trajectory ,in, For the first The future time obtained from the Monte Carlo sampling The predicted photovoltaic power value, and so on for the rest;

[0122] S4, based on the above Given a predicted trajectory, calculate the probability distribution of the predicted power at each future time point, extract its quantiles, and generate the future... Power prediction interval within the time period ,in, and Based on Future time calculated from the second sampling result The lower and upper limits of the photovoltaic power prediction range are set, and this prediction range is input into the collaborative decision engine as an extended state.

[0123] Traditional energy management often employs deterministic prediction, which fails to quantify the risk of prediction errors. This module introduces probabilistic prediction; Long Short-Term Memory (LSTM) networks excel at handling time series data and can effectively capture the temporal dependencies of photovoltaic power. The key lies in the application of Monte Carlo Dropout, enabling Dropout during both training and prediction. This introduces randomness into single predictions and allows for repeated predictions (…). By performing multiple iterations, a distribution of prediction results can be obtained, reflecting the model's perceived uncertainty about the future. By calculating the quantiles of this distribution (such as the 10th and 90th quantiles), a power prediction range can be obtained. Feeding this range (rather than a single value) to a deep reinforcement learning network allows it to learn decision-making strategies that balance the most likely scenario with the worst-case scenario (such as a sudden drop in power generation). For example, when the lower limit of the prediction range is low, the deep reinforcement learning network may tend to store more energy in advance for unforeseen circumstances, thereby enhancing the system's robustness to power fluctuations and improving the foresight and safety of its decisions.

[0124] To enable the decision-making strategy to flexibly adjust risk preferences based on real-time system conditions (electricity price, energy storage level) and achieve a dynamic balance between robustness and economy, the collaborative decision-making engine incorporates the photovoltaic power prediction range during the decision-making process. The state representation is incorporated using a weighted expectation method, specifically by calculating the future... Weighted expected value of photovoltaic power over time ;

[0125] ;

[0126] Among them, For the future The weighted expected value of photovoltaic power is used as a reference point for decision-making. This is the risk preference coefficient, whose value is based on real-time electricity price signals. and the state of charge of energy storage units Dynamic adjustment, the calculation formula is as follows , For the Sigmoid function, and This is the gain coefficient. and For reference value, when Above the threshold and At lower levels, Taking a larger value means that decisions rely more on the lower bound of the forecast to avoid risks; conversely, taking a smaller value means that decisions are more aggressive in order to pursue returns.

[0127] The calculated weighted expectation sequence It is concatenated with the current state vector and together they serve as the extended state input of the deep reinforcement learning network.

[0128] By introducing a dynamic risk preference coefficient This method aggregates the forecast intervals into a single-point weighted expected value. This value is not a simple average, but rather a reflection of the system's risk preference. During periods of high electricity prices and insufficient energy storage—that is, when the risk of power shortages is high and costs are high—the system tends to be conservative, placing more faith in the more pessimistic lower bound of the forecast. Larger energy prices allow for greater reserves of energy storage or reduced electricity purchases at higher prices; when electricity prices are low or energy storage is sufficient (i.e., low risk of power shortages and low costs), the system tends to be more aggressive, placing greater faith in the upper limit of optimistic forecasts. Smaller, thus more proactive in charging or selling electricity. This dynamic aggregation method intelligently couples uncertain prediction intervals with certain system states (electricity price, state of charge), generating features rich in decision-making information. This enables deep reinforcement learning networks to learn to adopt different strategies under different risk scenarios, significantly improving the system's adaptive optimization capabilities under different market conditions and operating states.

[0129] To address the problem that load-side resources are difficult to schedule directly and flexibly by optimization algorithms in traditional methods, discrete and heterogeneous adjustable loads (such as air conditioners, water heaters, and charging piles) are uniformly and quantitatively incorporated into a continuous optimization energy management framework. The adjustable loads in the local load unit are aggregated and represented through a virtual energy storage model.

[0130] For the An adjustable load, whose virtual energy storage model is based on a virtual state of charge. Virtual charging and discharging power limit , Self-loss rate and charge / discharge efficiency The state update equation is defined as follows:

[0131] ;

[0132] in, For the first The virtual energy storage corresponding to each adjustable load at time The virtual state of charge, ranging from Simulate the energy satisfaction of this load, for example, the heat output corresponding to the water temperature in the water heater. and The first The maximum virtual charging power (corresponding to the load activation power) and the maximum virtual discharging power (corresponding to the power that can be reduced when the load is turned off) of each virtual energy storage unit. For the first The virtual self-loss rate of virtual energy storage simulates the natural energy dissipation of the load in standby or off state (such as hot water cooling, room heat loss). For the first The virtual charge / discharge efficiency of a virtual energy storage system simulates the efficiency of converting electrical energy into the required energy (such as heat) when a load is turned on. For the first A virtual energy storage at a time The virtual state of charge, For the first The rated capacity of a virtual energy storage unit corresponds to the total energy demand or capacity that the adjustable load needs to meet (such as the total heat capacity of a hot water tank or the heat capacity of a room). and They are respectively Time assigned to the first The virtual charging power (corresponding to load activation) and virtual discharging power (corresponding to load deactivation) of each virtual energy storage unit are determined by the instruction distribution and execution module based on the overall instruction. The allocation must be obtained and must meet operational constraints. , , ;

[0133] The energy management unit includes a virtual energy storage aggregation module, which is used to connect and equate multiple adjustable load virtual energy storage models in parallel according to power and capacity to obtain the total virtual state of charge of the aggregated virtual energy storage. (defined as each virtual energy storage) According to its rated capacity (weighted average) and the upper limit of total virtual charge and discharge power , (The sum of the power of all currently operable loads and the sum of the power of all currently operable loads) is provided to the collaborative decision engine as the status information of the system's controllable resources.

[0134] The core of this approach lies in proposing a virtual energy storage model. This model analogizes loads with time-transfer or interruptible characteristics to a battery with specific capacity, charging / discharging power, and losses. For example, the thermal inertia of an air conditioner at a set temperature can be considered its energy storage capacity, and its switching states correspond to charging and discharging. Through this model, discrete load switching commands are transformed into continuous virtual charging / discharging power commands. The aggregation module further equates multiple such virtual energy storage units to a single large, time-varying virtual energy storage unit. This design allows deep reinforcement learning networks to process real energy storage in a unified manner (through continuous actions). This is used to handle complex load clusters. The collaborative decision-making module outputs... After being parsed by the instruction distribution module, it will be based on the information of each virtual energy storage device. Prioritization determines which load actions to take. This approach greatly expands the system's flexible adjustment resources, upgrading energy management from traditional source-storage coordination to deep source-storage-load coordination. It can tap the adjustment potential of the load side without affecting user comfort, participate in more refined peak shaving and valley filling and demand response, and further improve economic efficiency.

[0135] To accurately and reliably decompose the continuous, abstract load regulation commands output by the deep reinforcement learning network into discrete control commands for specific physical devices, and to ensure a good user experience while avoiding frequent load switching or improper disconnection of critical loads, the command distribution and execution module determines the actions output by the collaborative decision engine. ,Sure The total virtual energy storage power that needs to be adjusted at all times Based on a preset load priority list, the specific actions for each adjustable load are determined using the following steps:

[0136] like If it indicates that virtual energy storage charging power needs to be increased (i.e., the load needs to be activated), then the loads in the activatable state are traversed in order of priority from low to high, and the power required to activate them is calculated. until the cumulative power meets the requirement. And generate the corresponding load start command;

[0137] like This indicates that it is necessary to increase the virtual energy storage discharge power (i.e., shut down the load). Therefore, the loads in operation are traversed in descending order of priority to calculate the power that can be released upon shutdown. until the cumulative power meets the requirement. And generate the corresponding load shutdown command.

[0138] The collaborative decision-making module outputs a general, normalized load adjustment intention. The instruction distribution and execution module is responsible for translating these into executable commands. The key is allocation based on the state of the virtual energy storage model and preset load priorities. Priorities can be set based on load importance, interruptibility, and user comfort preferences. When power consumption needs to increase (corresponding to virtual charging), loads with lower importance that can be turned on at any time are prioritized; when power consumption needs to decrease (corresponding to virtual discharging), loads with higher importance but that can be briefly interrupted are prioritized for shutdown. This priority-based allocation strategy ensures that while meeting overall power regulation targets, it also maximizes the power supply to critical loads and the user's power experience. Combined with the virtual energy storage model, it forms a complete and implementable flexible load control mechanism, enabling precise execution of optimization decisions.

[0139] To address the contradiction between achieving global economic optimization and real-time volatility mitigation in single-timescale optimization, the energy management unit further includes a multi-timescale rolling optimization framework, which comprises:

[0140] Long-cycle optimization layer: based on duration The time interval is typically set to 15 minutes, and the running cycle is future time. Within a given period, typically set to 24 hours, based on photovoltaic power forecasting, load forecasting, and time-of-use pricing, with the objective of minimizing daily operating costs, a mixed-integer linear programming approach is used to solve the problem and obtain future... Reference planning curves for internal energy storage units and adjustable loads, including energy storage reference power. and adjustable load switching status plan, in which, For long-cycle optimization layers, the first layer is... indivual The planned power of the energy storage unit obtained by solving the time period is positive for charging and negative for discharging;

[0141] Short-cycle execution layer: based on duration The time interval, typically set to 1 minute, is controlled in real-time by the collaborative decision-making engine, and its reward function includes an economic reward item. Revised to:

[0142] ;

[0143] in, This is obtained by interpolating the long-cycle optimization layer planning curve. Real-time energy storage reference power, To track the weight coefficients, the short-cycle execution layer tracks the weight coefficients of the long-cycle plan, taking a positive value, such as 0.01, to balance the weights between real-time economic optimization and plan tracking.

[0144] The control objective of the short-cycle execution layer is to track the reference plan formulated by the long-cycle optimization layer in real-time operation and make dynamic adjustments based on ultra-short-term photovoltaic power fluctuations and real-time load deviations.

[0145] Long-cycle optimization, based on prediction, can formulate globally optimal plans but cannot respond to minute-level fluctuations; short-cycle optimization responds quickly but may get trapped in local optima. This approach combines the advantages of traditional optimization and artificial intelligence. The long-cycle optimization layer (using mixed-integer linear programming) is based on relatively accurate day-ahead forecasts and can solve a globally approximate optimal plan on a 24-hour scale, considering time-of-use pricing and equipment constraints, providing the system with a macro-level, economically optimal operational reference. The short-cycle execution layer operates at a minute-level frequency, and its objective is no longer simply to minimize real-time electricity costs, but rather to minimize real-time electricity costs and track the long-cycle plan. This is achieved by introducing a tracking penalty term. This approach guides deep reinforcement learning networks to make real-time decisions that respond to momentary fluctuations in photovoltaic power and load while minimizing deviations from the long-term plan for optimal economic benefits. For example, when a short-term cloud cover causes a sudden drop in photovoltaic power, the deep reinforcement learning network can quickly call upon energy storage to discharge and smooth the power output. However, it will subsequently adjust its charging and discharging strategy to return to the long-term planned curve, avoiding overreaction that could disrupt the arbitrage strategy for the entire day. This framework, where long-term planning guides short-term execution, effectively combines the global optimization capabilities of mixed-integer linear programming with the real-time adaptive capabilities of deep reinforcement learning. While ensuring real-time performance, it significantly improves the long-term, overall economic efficiency of the system.

[0146] The embodiments described above are merely illustrative of specific implementations of the present invention, and while the descriptions are detailed, they should not be construed as limiting the scope of the present invention. It should be noted that those skilled in the art can make various modifications and improvements without departing from the concept of the present invention, and these modifications and improvements all fall within the scope of protection of the present invention.

Claims

1. A distributed rooftop photovoltaic apparatus, characterized by, It includes a photovoltaic array, an energy storage unit, a local load unit, a grid-connected inverter, and an energy management unit that is communicatively connected to the energy storage unit, the local load unit, and the grid-connected inverter; The energy management unit includes; a data awareness module for collecting in real time the power generation of the photovoltaic array the power consumption of the local load unit the state of charge of the energy storage unit and the charge and discharge power limits, and the electricity price signal ; The collaborative decision-making module is used to run a trained deep reinforcement learning network. The deep reinforcement learning network takes the real-time data and historical data collected by the data sensing module as state input and outputs collaborative control actions for the energy storage unit, the adjustable load in the local load unit, and the grid-connected inverter. The instruction distribution and execution module is used to parse the control actions output by the collaborative decision-making module into charging and discharging power instructions for the energy storage unit. Switching commands for specified adjustable loads and grid-connected power commands for grid-connected inverters. And issue it to the corresponding device for execution; The deep reinforcement learning network is configured to learn strategies for dynamic and collaborative optimization of self-consumption, peak shaving and valley filling, and participation in grid demand response, while satisfying equipment operation constraints, by maximizing long-term cumulative rewards.

2. The distributed rooftop photovoltaic plant of claim 1, wherein, The deep reinforcement learning network is constructed based on the proximal policy optimization algorithm, and its state space... Action space and reward function The definition is as follows: State space is the state space of the system at time the observed system state vector: ； wherein, is a cyclic encoding vector of the time of day, is a cyclic encoding vector of the day of the week; Action space To enable deep reinforcement learning networks to perform at any time Output continuous action vector: ； in, This is a normalized reference value for grid-connected power. This is a normalized reference value for energy storage charging and discharging power. This serves as a reference value for the switching probability of adjustable loads. Reward function is: ； Among them, economic reward items , is the actual grid-connected power, and the electricity purchase is positive and the electricity sale is negative, is the control and decision time interval; Safety reward item , for punishing the energy storage state of charge out of bounds, and are the upper and lower limits of the energy storage state of charge, respectively, is the punishment coefficient; grid-friendly reward term for smoothing the grid power fluctuation, is the moving average of the grid power in the previous time period, is the smoothing coefficient; , and These are the weighting coefficients for each reward item.

3. The distributed rooftop PV plant of claim 2, wherein, The deep reinforcement learning network comprises a policy network and a value network wherein and are network parameters, respectively. The policy network An Actor network is adopted, and the network architecture is as follows: Input layer: number of neurons is the same as the dimension of the state vector ; First hidden layer: Fully connected layer, containing 256 neurons, using the ReLU activation function; The second hidden layer is a fully connected layer containing 128 neurons and using the ReLU activation function. Output layer: contains action vectors Neurons of the same dimension, for and Use the Tanh activation function to limit the output to 0. ,for Use the Sigmoid activation function to limit the output to 0. ; The value network A Critic network is used to evaluate state value, and its network architecture is as follows: Input layer: number of neurons is the same as the dimension of the state vector ; First hidden layer: Fully connected layer, containing 256 neurons, using the ReLU activation function; The second hidden layer is a fully connected layer containing 128 neurons and using the ReLU activation function. Output layer: single neuron, linear output, represents state expected cumulative return estimate The training process of the deep reinforcement learning network includes: Collecting experience data by interacting with the environment and storing into an experience replay buffer; sampling mini-batches of data from the buffer, computing advantage functions using generalized advantage estimation ; updating the policy network parameters by maximizing a clipped target function of proximal policy optimization : ； wherein, is the expected estimate of the time , is the probability ratio of the new and old policies, is the old policy network before update, is the clipping hyperparameter; updating the value network parameters by minimizing a mean squared error loss of the value network : ； in, The target value for updating the value network.

4. The distributed rooftop photovoltaic plant of claim 1, wherein, The energy management unit also includes a photovoltaic power ultra-short-term probabilistic prediction module, which is used to provide prediction information to the collaborative decision-making module; The photovoltaic power ultra-short-term probabilistic prediction module is built based on long short-term memory networks and Monte Carlo Dropout, and its execution steps include: S1, input recent hourly historical photovoltaic power sequence and weather feature vector for the corresponding time instant wherein the weather feature vector contains irradiance, temperature, humidity; S2, inputting the sequence into a two-layer stacked long short-term memory network, wherein a Dropout rate of Monte Carlo Dropout is applied after each layer of the long short-term memory network; S3, in the prediction phase, performing a next forward pass inference, each inference producing a different output due to the randomness of Dropout, thereby obtaining future ;​​ S4, based on the above Given a predicted trajectory, calculate the probability distribution of the predicted power at each future time point, extract its quantiles, and generate the future... Power prediction interval within the time period The predicted interval is then input into the collaborative decision-making module as an extended state.

5. The distributed rooftop photovoltaic plant of claim 4, wherein, During the decision-making process, the collaborative decision-making module will use the photovoltaic power prediction range... The state representation is incorporated using a weighted expectation method, specifically by calculating the future... Weighted expected value of photovoltaic power over time ; ； wherein, is a risk preference coefficient, whose value is determined according to real-time electricity price signal and state of charge of the energy storage unit is dynamically adjusted, the decision is more aggressive to pursue the revenue; The calculated weighted expected sequence is spliced with the current time state vector and is jointly used as the extended state input of the deep reinforcement learning network.

6. The distributed rooftop photovoltaic plant of claim 1, wherein, The adjustable load in the local load unit is aggregated and represented by a virtual energy storage model. For the An adjustable load, whose virtual energy storage model is based on a virtual state of charge. Virtual charging and discharging power limit , Self-loss rate and charge / discharge efficiency The state update equation is defined as follows: ； in, This is the virtual energy storage rated capacity. and They are respectively The virtual charging power and virtual discharging power allocated to the load at all times must meet operational constraints. , , ; The energy management unit includes a virtual energy storage aggregation module, which is used to connect and equate multiple adjustable load virtual energy storage models in parallel according to power and capacity to obtain the total virtual state of charge of the aggregated virtual energy storage. Total virtual charging and discharging power limit , This information is then provided to the collaborative decision-making module as status information of the system's controllable resources.

7. The distributed rooftop photovoltaic plant of claim 6, wherein, The instruction distribution and execution module determines the actions output by the collaborative decision-making module. ,Sure The total virtual energy storage power that needs to be adjusted at all times Based on a preset load priority list, the specific actions for each adjustable load are determined using the following steps: like If it indicates that virtual energy storage charging power needs to be increased, then the loads in the activating state are traversed in order of priority from low to high, and the power required to activate them is calculated. until the cumulative power meets the requirement. And generate the corresponding load start command; like This indicates a need to increase the virtual energy storage discharge power. Therefore, the loads in operation are traversed in descending order of priority, and the power that can be released when they are turned off is calculated. until the cumulative power meets the requirement. And generate the corresponding load shutdown command.

8. The distributed rooftop photovoltaic plant of claim 1, wherein, The energy management unit also includes a multi-timescale rolling optimization framework, which includes: Long-cycle optimization layer: based on duration The time interval is defined as the time period, and the running period is in the future. Internally, based on photovoltaic power forecasting, load forecasting, and time-of-use pricing, and with the objective of minimizing daily operating costs, a mixed-integer linear programming approach is used to obtain the future... Reference planning curves for internal energy storage units and adjustable loads, including energy storage reference power. and adjustable load switching status plan; Short cycle execution layer: real-time control by the collaborative decision module with time interval of the economic reward term in the reward function is modified as: ​ ； wherein, is the long period optimized layer plan curve the moment energy storage reference power, is the tracking weight coefficient.

Images