Multi-energy complementary building energy supply system based on solar energy

Abstract

The invention relates to a multi-energy complementary building energy supply system based on solar energy, and relates to the field of building energy supply. Joint features are extracted, and the Q value of each alternative scheduling strategy parameter in the current state is calculated; a state risk factor is calculated according to the deviation degree of the current energy storage equipment SOC and a preset safety threshold value and the power grid electricity price, the exploration probability inversely proportional to the state risk factor is determined, and selection is carried out between random exploration and utilization of scheduling strategy parameters with the maximum Q value according to the exploration probability; the selected scheduling strategy parameters are used as optimization boundaries, a mixed integer linear programming model is constructed and solved, and equipment control actions are obtained; the building operation cost of the current time step is calculated according to the equipment control action, and the opposite number of the operation cost serves as a reward value; storing an experience sample containing the current state, the selected scheduling strategy parameter, the reward value and the new state into an experience playback pool; and updating the evaluation network.

Description

Technical Field

[0001] This application belongs to the field of building energy supply, and in particular relates to a multi-energy complementary building energy supply system based on solar energy. Background Technology

[0002] Currently, the penetration rate of distributed renewable energy, represented by photovoltaics, in the building sector is increasing. Constructing multi-energy complementary building systems that incorporate multiple energy forms such as photovoltaics, energy storage, and the power grid, and achieving multi-energy coordinated and optimized scheduling through advanced energy management technologies, is a key approach to reducing building operating costs and improving renewable energy absorption rates. One mainstream method for optimized scheduling is Model Predictive Control (MPC) based on predictive information, which typically models the problem as a mixed-integer linear programming (MILP) problem and solves it in a rolling manner. However, this heavily relies on accurate modeling of the physical characteristics of each device within the system, and the computational cost of solving the problem increases dramatically with the increase in system complexity and prediction time domain, making it difficult to meet the requirements of real-time control. Furthermore, the accumulation of prediction errors severely affects its long-term optimization performance, resulting in scheduling strategies that only have local optima. Deep reinforcement learning, especially deep Q-networks (DQNs), can form a direct mapping from environmental states to control actions through continuous interaction and learning between the agent and the environment, without requiring an accurate system model, and can effectively handle uncertainties in long-term decision-making problems. However, to handle continuous control variables such as energy storage charging and discharging, the action space is often discretized, which sacrifices control accuracy and faces the curse of dimensionality in high-dimensional action spaces. Furthermore, traditional ε-greedy exploration strategies are blind, failing to consider risks associated with the current system state, such as excessively low energy storage SOC, potentially leading to costly or even unsafe exploratory actions at critical moments. The inherent problem of overestimating Q-values ​​in DQN can also affect the stability and final performance of the learning strategy. Overcoming these shortcomings and improving the application of deep reinforcement learning in building energy management is an urgent problem to be solved. Summary of the Invention

[0003] To improve the synergy of solar-powered multi-energy complementary building energy supply, this application proposes a solar-based multi-energy complementary building energy supply system, comprising: The data acquisition module is used to acquire a state vector composed of a photovoltaic power generation prediction sequence, a load power prediction sequence, the grid electricity price at the current moment, and the SOC of the energy storage device at the current moment; the state vector is input into an evaluation network that integrates a long short-term memory network and a feedforward network to extract joint features and calculate the Q value of each alternative scheduling strategy parameter in the current state; The scheduling strategy selection module is used to calculate the state risk factor based on the deviation of the current energy storage device's SOC from the preset safety threshold and the grid electricity price, determine the exploration probability that is inversely proportional to the state risk factor, and select between random exploration and using the scheduling strategy parameter with the maximum Q value based on the exploration probability. The control action acquisition module uses the selected scheduling strategy parameters as the optimization boundary to construct and solve a mixed-integer linear programming model to obtain the equipment control actions. The scheduling strategy parameters are the upper and lower limits of the charging and discharging power of the energy storage equipment or the upper and lower limits of the interaction power with the power grid. Based on the equipment control actions, the module calculates the building operation cost at the current time step and uses the negative of the operation cost as the reward value. The module stores the experience sample containing the current state, the selected scheduling strategy parameters, the reward value, and the new state into the experience replay pool. An update module is used to perform priority sampling from the experience replay pool to update the evaluation network, wherein the sampling priority is determined by the absolute value of the time-series differential error, the weighted sum of the grid electricity price and the power imbalance, the update process uses the target network to calculate the target Q value and minimizes the asymmetric loss between the evaluation network Q value and the target Q value, the asymmetric loss imposing a greater penalty on overestimation of the evaluation network Q value than on underestimation.

[0004] Preferably, the step of calculating the state risk factor based on the deviation of the current energy storage device's SOC from a preset safety threshold and the grid electricity price includes: The preset security threshold is set as the upper security threshold. With safety lower limit threshold ; Calculate the SOC deviation as ; The state risk factor R is obtained by weighting and summing the SOC deviation and the normalized grid electricity price according to preset weights.

[0005] Preferably, the step of constructing and solving a mixed-integer linear programming model using the selected scheduling strategy parameters as the optimization boundary includes: The optimization objective is to minimize the building operating cost within the scheduling cycle. The upper and lower limits of the charging and discharging power of the energy storage device selected by the deep Q network or the upper and lower limits of the interaction power with the grid are used as inequality constraints. Combined with the power balance constraints within the building, the state change constraints of the energy storage device, and the actual interaction power constraints with the grid, the mixed integer linear programming model is constructed.

[0006] Preferably, the step of calculating the building operating cost at the current time step based on the device control action and using its inverse as the reward value includes: Obtain the power purchased from the grid as determined by the control actions of the device. and the power sold to the grid And the current grid purchase price of electricity. Electricity sales price ; Calculate building operating costs ; Use -C(t) as the reward value for the current time step.

[0007] Preferably, the step of prioritizing sampling from the experience replay pool to update the evaluation network, wherein the sampling priority is determined by a weighted sum of the absolute value of the time-series differential error, the grid electricity price, and the power imbalance, including: Calculate the absolute value of the error for sample i in the empirical replay pool. ; Calculate power imbalance ; Obtaining grid electricity price ; Sampling priority of sample i From the above Influence value of power imbalance The result is obtained by weighted summation based on preset weights.

[0008] Preferably, minimizing the asymmetric loss between the evaluation network Q-value and the target Q-value includes: Calculate timing difference error ; Define an asymmetric loss function L, where δ is less than 0. When δ is greater than or equal to 0, , where α and β are preset penalty coefficients, and α>β; the loss function L is minimized by gradient descent to update the evaluation network.

[0009] Compared with the prior art, the present invention has the following advantages: 1) By combining deep reinforcement learning to set optimization boundaries and mixed-integer linear programming for solving, the accuracy loss problem caused by traditional discretization methods is solved, and more precise control of energy storage and other equipment is achieved. 2) The evaluation network, which integrates long short-term memory network and feedforward network, can fully extract the joint features of temporal prediction information and current state, thereby improving the accuracy of decision-making; 3) By introducing a state-risk-based exploration mechanism, high-cost blind exploration during peak electricity prices or critical energy storage conditions is avoided, thereby enhancing the system's operational economy and security. 4) The use of an asymmetric loss function suppresses the problem of overestimation of Q value, enhances convergence stability and strategy reliability, and can significantly reduce the overall energy supply cost of multi-energy complementary buildings. Attached Figure Description

[0010] Figure 1 To evaluate the network structure diagram; Figure 2 To explore the schematic diagram of the probability calculation process; Figure 3 This is a schematic diagram of the sampling priority calculation process; Figure 4 A schematic diagram of the asymmetric loss function is shown. Detailed Implementation

[0011] To facilitate understanding of the technical solution of this application, the application will be further described below with reference to the accompanying drawings.

[0012] The terms "first" and "second," etc., used in the specification, claims, and drawings of this application are used only to distinguish different objects and not to describe a specific order. Furthermore, the terms "comprising" and "having," and any variations thereof, are intended to cover non-exclusive inclusion. For example, a process, method, system, product, or apparatus that includes a series of steps or units is not limited to the listed steps or units, but may optionally include steps or units not listed, or may optionally include other steps or units inherent to these processes, methods, products, or apparatuses.

[0013] The term "embodiment" as used herein means that a particular feature, structure, or characteristic described in connection with an embodiment may be included in at least one embodiment of this application. The appearance of this phrase in various places throughout the specification does not necessarily refer to the same embodiment, nor is it a separate or alternative embodiment mutually exclusive with other embodiments. It will be explicitly and implicitly understood by those skilled in the art that the embodiments described herein can be combined with other embodiments.

[0014] In this application, "at least one (item)" refers to one or more, "more than one" refers to two or more, "at least two (items)" refers to two or three or more, and "and / or" is used to describe the relationship between related objects, indicating that there can be three relationships. For example, "A and / or B" can mean: only A exists, only B exists, and both A and B exist simultaneously, where A and B can be singular or plural. "Or" indicates that there can be two relationships, such as only A exists and only B exists; when A and B are not mutually exclusive, it can also mean that there are three relationships, such as only A exists, only B exists, and both A and B exist simultaneously. The character " / " generally indicates that the preceding and following related objects are in an "or" relationship. "At least one (item) of the following" or similar expressions refer to any combination of these items. For example, at least one (item) of a, b, or c can mean: a, b, c, "a and b", "a and c", "b and c", or "a and b and c".

[0015] The present invention will now be described with reference to specific embodiments. However, those skilled in the art should understand that the specific data in the following embodiments do not constitute a limitation on the present invention. Specifically, a multi-energy complementary building energy supply system based on solar energy includes: The data acquisition module is used to acquire a state vector composed of a photovoltaic power generation prediction sequence, a load power prediction sequence, the grid electricity price at the current moment, and the SOC of the energy storage device at the current moment; the state vector is input into an evaluation network that integrates a long short-term memory network and a feedforward network to extract joint features and calculate the Q value of each alternative scheduling strategy parameter in the current state; In multi-energy complementary buildings, the building is considered a miniature energy center, integrating solar energy with other energy forms such as geothermal, wind, or biomass energy through intelligent control and energy storage technologies to achieve multi-energy complementarity and tiered utilization. For example, during the day, when solar photovoltaic power generation is sufficient, in addition to meeting the building's electricity needs, excess electricity can be stored through energy storage devices such as battery packs for nighttime use. Multi-energy complementary building functional systems are primarily used to optimize the utilization and scheduling of different energy sources in real time based on factors such as solar power generation, the status of the energy storage system, the building's load demand, and grid electricity prices. For example, when sunlight is abundant, solar energy is prioritized for use and charging of energy storage; when sunlight is insufficient, energy storage is prioritized for discharge, with any remaining energy purchased from the grid or utilized from other auxiliary energy sources.

[0016] Photovoltaic power generation can be predicted using, for example, a long short-term memory (LSM) network model. Inputting historical photovoltaic (PV) and load data, it outputs a power prediction sequence for the next 24 hours. These two prediction sequences are then concatenated with the current time-of-use (TOU) electricity price obtained from the grid company and the current State of Charge (SOC) value of the energy storage device read from the battery management system to obtain a one-dimensional state vector. The PV and load prediction sequence portions of the state vector are input into the LSM network module of the evaluation network to extract time-series features, while the electricity price and SOC portions are input into the feedforward network module to extract cross-sectional features. In one embodiment, the feature vectors output from the two modules are concatenated and then passed through several fully connected layers to obtain a vector where each element corresponds to the Q-value of an alternative scheduling strategy parameter, such as... Figure 1 As shown.

[0017] The scheduling strategy selection module is used to calculate the state risk factor based on the deviation of the current energy storage device's SOC from the preset safety threshold and the grid electricity price, determine the exploration probability that is inversely proportional to the state risk factor, and select between random exploration and using the scheduling strategy parameter with the maximum Q value based on the exploration probability. The state risk factor is calculated using a preset weighted formula, taking the difference between the current State of Charge (SOC) and a preset safe SOC range (e.g., 0.2 to 0.9), along with the current grid electricity price. A decreasing function, such as an inverse proportional function or a sigmoid function, is then used to map the risk factor to an exploratory probability between 0 and 1. Figure 2 As shown, the higher the risk, the lower the probability. A random number is generated; if it is less than the exploration probability, a scheduling strategy parameter is randomly selected; otherwise, the scheduling strategy parameter with the highest Q-value output by the evaluation network is selected. When the risk is high, for example, when the energy storage SOC is too low / too high, or when the electricity price is at its peak, the state risk factor will be very high, and the exploration probability will become very low. The system will be more inclined to utilize, i.e., execute the strategy with the highest and most reliable Q-value. When the risk is low, for example, when the energy storage SOC is within a safe range and the electricity price is stable or at its lowest, the state risk factor will be very low, and the exploration probability will become very high. The system will be more inclined to explore, i.e., randomly try new strategies, because even if the attempt fails, the loss will be small.

[0018] The control action acquisition module uses the selected scheduling strategy parameters as the optimization boundary to construct and solve a mixed-integer linear programming model to obtain the equipment control actions. The scheduling strategy parameters are the upper and lower limits of the charging and discharging power of the energy storage equipment or the upper and lower limits of the interaction power with the power grid. Based on the equipment control actions, the module calculates the building operation cost at the current time step and uses the negative of the operation cost as the reward value. The module stores the experience sample containing the current state, the selected scheduling strategy parameters, the reward value, and the new state into the experience replay pool. A mixed-integer linear programming model is established with the objective of minimizing the operating cost over the next hour. Constraints include the building's power balance equation, the energy storage SOC dynamic equation, and physical operating limitations of each device. The scheduling strategy parameters selected in the previous step, such as "medium-power charging for energy storage," are updated into the model with their corresponding values ​​(e.g., a lower limit of 5 kW and an upper limit of 10 kW for charging power). A commercial solver, such as Gurobi or CPLEX, is used to solve the model, obtaining the specific power settings for energy storage, air conditioning, and other devices at the next time step. The actual power purchased from the grid, obtained from solving the mixed-integer linear programming model, is multiplied by the current grid electricity price to obtain the operating cost at the current time step. This cost value is negativeened and used as the reward value. The state vector before the action, the selected scheduling strategy parameters, the reward value, and the new state vector observed after the action are combined into an experience tuple and stored in a first-in-first-out data queue, i.e., the experience replay pool.

[0019] An update module is used to perform priority sampling from the experience replay pool to update the evaluation network, wherein the sampling priority is determined by the absolute value of the time-series differential error, the weighted sum of the grid electricity price and the power imbalance, the update process uses the target network to calculate the target Q value and minimizes the asymmetric loss between the evaluation network Q value and the target Q value, the asymmetric loss imposing a greater penalty on overestimation of the evaluation network Q value than on underestimation.

[0020] For each experience in the experience replay pool, a priority is calculated. Preferably, the priority is obtained by weighted summing of the absolute value of the time-series differential TD error of the experience, the grid electricity price at the time the experience occurred, and the power imbalance between photovoltaics and load. A batch of experience samples is extracted from the pool according to the priority for training. A dual-depth Q-network algorithm is used to calculate the target Q-value through the target network. Then, a quantile regression loss function or a custom weighted mean square error loss function is used as the asymmetric loss, where the loss weight coefficient applied when the evaluated Q-value is greater than the weight coefficient applied when it is less than the target Q-value. The Adam optimizer is used to minimize this loss, completing one update of the evaluation network parameters.

[0021] In yet another embodiment, the step of calculating the state risk factor based on the deviation of the current energy storage device's SOC from a preset safety threshold and the grid electricity price includes: The preset security threshold is set as the upper security threshold. With safety lower limit threshold ; Calculate the SOC deviation as ; The state risk factor R is obtained by weighting and summing the SOC deviation and the normalized grid electricity price according to preset weights.

[0022] For example, a safety upper limit threshold of 95% and a safety lower limit threshold of 15% can be set for energy storage devices. When the current state of charge (SOC) of the energy storage device is 98%, its deviation is 3%. When the SOC is 10%, its deviation is 5. When the SOC is within the safe range of 15% to 95%, the deviation is zero. The deviation reflects the degree of physical risk caused by overcharging or over-discharging of the energy storage device. The state risk factor R is calculated by combining physical risk and economic risk. Assuming the weight of SOC deviation is 0.6 and the weight of normalized grid price is 0.4, at a certain moment, the energy storage SOC deviation is 5, and it is during peak electricity consumption with a normalized electricity price as high as 0.9. At this time, the state risk factor R is 3.36, which is a high risk value. Since the exploration probability ε is inversely proportional to the state risk factor R, a high risk value will lead to a very low exploration probability. The agent will tend to execute known safe and economical strategies rather than randomly exploring unknown actions that may lead to device damage or high electricity costs.

[0023] To combine the macro-level policy guidance of reinforcement learning with the precise solution of traditional optimization methods, in another embodiment, the construction and solution of a mixed-integer linear programming model using the selected scheduling policy parameters as the optimization boundary includes: The optimization objective is to minimize the building operating cost within the scheduling cycle. The upper and lower limits of the charging and discharging power of the energy storage device selected by the deep Q network or the upper and lower limits of the interaction power with the grid are used as inequality constraints. Combined with the power balance constraints within the building, the state change constraints of the energy storage device, and the actual interaction power constraints with the grid, the mixed integer linear programming model is constructed.

[0024] As a high-level decision-maker, the Deep Q-Network does not directly output a precise power value. Instead, it outputs an optimal power adjustment range based on the current state, such as electricity prices and load forecasts. For example, during periods of low electricity prices, the Deep Q-Network might decide to implement a charging strategy, providing a charging power range of 10 kW to 20 kW. This range serves as an inequality constraint—that is, the energy storage charging power must be greater than or equal to 10 kW and less than or equal to 20 kW—and is input into the Mixed Integer Linear Programming (MILP) model. The MILP model performs a refined solution within the boundaries given by the Deep Q-Network. MILP is an optimization model that mainly includes an objective function, constraints, and decision variables. The specific solution process is common knowledge in the field and will not be elaborated here. The objective function of the model is to minimize the total operating cost, i.e., the cost of purchasing electricity minus the revenue from selling electricity. In addition to the power range constraint given by the Deep Q-Network, the model also includes other physical constraints. For example, the power balance constraint within the building requires that all power generation, including photovoltaic power generation, energy storage discharge, and grid purchases, must equal all power consumption, including building load and energy storage charging. The energy storage device's state change constraints are updated in real time based on the charging and discharging power. The solver will find an optimal charging power value in the range of 10 kW to 20 kW, such as 17.5 kW, which can minimize the operating cost in the current scheduling cycle while satisfying all physical constraints.

[0025] In yet another embodiment, the step of calculating the building operating cost at the current time step based on the device control action and using its inverse as the reward value includes: Obtain the power purchased from the grid as determined by the control actions of the device. and the power sold to the grid And the current grid purchase price of electricity. Electricity sales price ; Calculate building operating costs ; Use -C(t) as the reward value for the current time step.

[0026] Reward values ​​guide the learning direction of the reinforcement agent, namely, maximizing cumulative rewards. By using the negative of the operating cost as the reward, the agent's goal is aligned with minimizing the operating cost. For example, during the peak electricity consumption period at 2 PM, a control action causes the building to purchase 20 kilowatts of electricity from the grid at a price of 1.2 yuan per kilowatt-hour. Assuming a time step of 1 hour, the operating cost C for this time step is 24 yuan. The reward value received by the agent is -24. If the control action occurs at noon when photovoltaic power generation is abundant and the building load is low, the agent decides to sell the excess 10 kilowatts of electricity back to the grid at a price of 0.8 yuan per kilowatt-hour. Then, the operating cost C for this time step is -8 yuan, representing a gain of 8 yuan. In this case, the reward value received by the agent is negative -8, or 8. By continuously receiving such positive or negative reward signals, the agent can learn to reduce electricity purchases or increase electricity sales when electricity prices are high, and to purchase electricity or store energy when electricity prices are low, thus forming an economically optimal operating strategy.

[0027] In yet another embodiment, the step of prioritizing sampling from the experience replay pool to update the evaluation network, wherein the sampling priority is determined by a weighted sum of the absolute value of the time-series differential error, the grid electricity price, and the power imbalance, including: Calculate the absolute value of the error for sample i in the empirical replay pool. ; Calculate power imbalance ; Obtaining grid electricity price ; Sampling priority of sample i From the above Influence value of power imbalance The result is obtained by weighted summation based on preset weights.

[0028] To optimize the learning efficiency of the agent, it prioritizes learning experiences with the greatest information content or the most significant economic impact. The absolute value of the temporal difference (TD) error reflects the accuracy of the agent's prediction of the value of an action in a given state. A large TD error, such as 0.8, indicates a significant deviation between the agent's prediction and the actual result, suggesting that the sample contains new knowledge that the agent has not yet fully understood, thus possessing high learning value. The economic impact value of the power imbalance assesses the importance of the sample from an economic perspective, such as... Figure 3As shown in the diagram, suppose the power imbalance ΔP of sample i is 30 kW, indicating a need for a 30 kW power exchange with the grid. If the grid electricity price is as high as 1.5 yuan per kilowatt-hour, its economic impact value is 45. Another sample j, with the same power imbalance of 30 kW, occurs during a period when the electricity price is only 0.3 yuan per kilowatt-hour, and its economic impact value is only 9. Clearly, the decision-making error of sample i will have more severe economic consequences. By weighting the TD error and the economic impact value, for example, with a TD error weight of 0.5 and an economic impact value weight of 0.5, sample i has a much higher priority than sample j, and is therefore sampled more frequently for training, enabling the agent to learn to make correct decisions more quickly during periods of high electricity prices.

[0029] In yet another embodiment, minimizing the asymmetric loss between the evaluation network Q-value and the target Q-value includes: Calculate timing difference error ; Define an asymmetric loss function L, where δ is less than 0. When δ is greater than or equal to 0, , where α and β are preset penalty coefficients, and α>β; the loss function L is minimized by gradient descent to update the evaluation network.

[0030] A common problem in standard deep Q-networks is the overestimation of the Q-value. This overestimation leads the agent to make overly optimistic judgments about the future rewards of certain actions, thus getting trapped in local optima. Asymmetric loss functions address or mitigate this problem by imposing a heavier penalty on overestimation than on underestimation. Figure 4 A schematic diagram of the asymmetric loss function is shown. The temporal difference (TD) error δ is less than zero, meaning the target Q-value is less than the evaluation network Q-value, indicating overestimation. More specifically, assume a penalty coefficient α of 0.9 and β of 0.1, satisfying the condition that α is greater than β. Consider two cases: first, the evaluation network Q-value is 10 units higher than the target Q-value, i.e., δ equals -10, in which case the loss L equals 90. Second, the evaluation network Q-value is the same 10 units lower than the target Q-value, i.e., δ equals +10, in which case the loss L equals 10. It can be seen that for the same error, overestimation results in a loss nine times greater than underestimation. During network parameter updates, a larger loss generates a larger gradient, leading to a more significant correction of the network parameters. This forces the evaluation network to be more conservative during updates, tending towards slight underestimation rather than overestimation, thus obtaining a more robust and reliable energy scheduling strategy.

[0031] As can be seen from the above description of the embodiments, those skilled in the art can clearly understand that this application can be implemented by means of software plus necessary general-purpose hardware platforms. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product can be stored in a storage medium, such as ROM / RAM, magnetic disk, optical disk, etc., and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute the methods described in various embodiments or some parts of the embodiments of this application.

[0032] The various embodiments in this specification are described in a progressive manner. Similar or identical parts between embodiments can be referred to mutually. Each embodiment focuses on describing the differences from other embodiments. In particular, for system or system embodiments, since they are basically similar to method embodiments, the description is relatively simple, and relevant parts can be referred to the descriptions in the method embodiments. The systems and system embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to achieve the purpose of this embodiment according to actual needs. Those skilled in the art can understand and implement this without creative effort.

[0033] The method and electronic device for providing product object information provided in this application have been described in detail above. Specific examples have been used to illustrate the principles and implementation methods of this application. The descriptions of the above embodiments are only for the purpose of helping to understand the method and its core ideas. Furthermore, those skilled in the art will recognize that, based on the ideas of this application, there will be changes in the specific implementation methods and application scope. Therefore, the content of this specification should not be construed as a limitation of this application.

Claims

1. A solar-based multi-energy complementary building energy supply system, characterized in that, include: The data acquisition module is used to acquire a state vector composed of a photovoltaic power generation prediction sequence, a load power prediction sequence, the grid electricity price at the current moment, and the SOC of the energy storage device at the current moment; the state vector is input into an evaluation network that integrates a long short-term memory network and a feedforward network to extract joint features and calculate the Q value of each alternative scheduling strategy parameter in the current state; The scheduling strategy selection module is used to calculate the state risk factor based on the deviation of the current energy storage device's SOC from the preset safety threshold and the grid electricity price, determine the exploration probability that is inversely proportional to the state risk factor, and select between random exploration and using the scheduling strategy parameter with the maximum Q value based on the exploration probability. The control action acquisition module uses the selected scheduling strategy parameters as the optimization boundary to construct and solve a mixed integer linear programming model to obtain the equipment control actions. The scheduling strategy parameters are the upper and lower limits of the charging and discharging power of the energy storage equipment or the upper and lower limits of the interaction power with the power grid. Based on the equipment control actions, the module calculates the building operation cost at the current time step and uses the negative of the operation cost as the reward value. Store the experience sample containing the current state, selected scheduling strategy parameters, reward value and new state into the experience replay pool; An update module is used to perform priority sampling from the experience replay pool to update the evaluation network, wherein the sampling priority is determined by the absolute value of the time-series differential error, the weighted sum of the grid electricity price and the power imbalance, the update process uses the target network to calculate the target Q value and minimizes the asymmetric loss between the evaluation network Q value and the target Q value, the asymmetric loss imposing a greater penalty on overestimation of the evaluation network Q value than on underestimation.

2. The system according to claim 1, characterized in that, The calculation of the state risk factor based on the deviation of the current energy storage device's SOC from a preset safety threshold and the grid electricity price includes: The preset security threshold is set as the upper security threshold. With safety lower limit threshold ; Calculate the SOC deviation as ; The state risk factor R is obtained by weighting and summing the SOC deviation and the normalized grid electricity price according to preset weights.

3. The system according to claim 1, characterized in that, The step of constructing and solving a mixed-integer linear programming model using the selected scheduling strategy parameters as the optimization boundary includes: The optimization objective is to minimize the building operating cost within the scheduling cycle. The upper and lower limits of the charging and discharging power of the energy storage device selected by the deep Q network or the upper and lower limits of the interaction power with the grid are used as inequality constraints. Combined with the power balance constraints within the building, the state change constraints of the energy storage device, and the actual interaction power constraints with the grid, the mixed integer linear programming model is constructed.

4. The system according to claim 1, characterized in that, The step of calculating the building operating cost at the current time step based on the device control actions and using its inverse as the reward value includes: Obtain the power purchased from the grid as determined by the control actions of the device. and the power sold to the grid And the current grid purchase price of electricity. Electricity sales price ; Calculate building operating costs ; Use -C(t) as the reward value for the current time step.

5. The system according to claim 1, characterized in that, The step of prioritizing sampling from the experience replay pool to update the evaluation network, wherein the sampling priority is determined by a weighted sum of the absolute value of the time-series differential error, the grid electricity price, and the power imbalance, including: Calculate the absolute value of the error for sample i in the empirical replay pool. ; Calculate power imbalance ; Obtaining grid electricity price ; Sampling priority of sample i From the above Influence value of power imbalance The result is obtained by weighted summation based on preset weights.

6. The system according to claim 1, characterized in that, The minimization of the asymmetric loss between the evaluation network's Q-value and the target Q-value includes: Calculate timing difference error ; Define an asymmetric loss function L, where δ is less than 0. When δ is greater than or equal to 0, , where α and β are preset penalty coefficients, and α>β; the loss function L is minimized by gradient descent to update the evaluation network.

Images