Scheduling control method for optical storage direct flexible system

Abstract

The invention relates to the technical field of optical storage direct-flexible systems, in particular to an optical storage direct-flexible system scheduling control method, which comprises the following steps of: establishing an optical storage direct-flexible system operation architecture model; taking the direct current bus voltage deviation as an adjusting signal, and formulating an electric power control strategy of the optical storage direct-flexible system; based on the optical storage direct-flexible system operation architecture model and the electric power control strategy, an optical storage direct-flexible system optimization scheduling model is established; and optimizing the optimal scheduling model of the optical storage direct-flexible system based on a pulse experience playback reinforcement learning algorithm to obtain an optimal scheduling reinforcement learning model of the optical storage direct-flexible system, and scheduling the optical storage direct-flexible system through the optimal scheduling reinforcement learning model of the optical storage direct-flexible system. According to the method, the pulse experience playback TD3 algorithm is adopted for solving and making a decision, the dependence of the optical storage direct-flexible system on an external network can be effectively reduced, and the operation cost and the light abandoning rate of the optical storage direct-flexible system are reduced.

Description

Technical Field

[0001] The present invention relates to the technical field of photovoltaic (PV)-storage direct-flexible systems, and in particular to a scheduling control method for photovoltaic (PV)-storage direct-flexible systems. Background Art

[0002] As one of the main energy consumers, buildings have strong load regulation potential. Therefore, tapping the flexibility of building electricity consumption plays an important role in improving the regulation capacity of the power system.

[0003] The photovoltaic storage direct-flexible system is a new concept of intelligent buildings. At present, some domestic and foreign scholars have conducted research on the optimization and scheduling of intelligent buildings. Using flexible resources in multi-energy buildings to alleviate power supply pressure can effectively reduce the system operation cost. Studies have shown that orderly charging and discharging management of electric vehicles can effectively improve the economy and renewable energy utilization rate of buildings containing distributed power sources. With the high proportion of new energy sources such as wind and light and DC equipment connected to the distribution system, DC distribution has the advantages of small transmission loss and easy load power adjustment compared with AC distribution. The photovoltaic storage direct-flexible system was proposed in this context to coordinate and optimize the control of the source, storage and load components in the system. Through deep reinforcement learning to control energy consumption, it effectively improves photovoltaic absorption while ensuring user satisfaction and reduces dependence on energy storage batteries. Designing a control strategy to guide the scheduling of various flexible resources in the photovoltaic storage direct-flexible system is the key to achieving its zero-carbon economic operation.

[0004] In recent years, reinforcement learning, as an efficient method, has been widely used to deal with complex sequential decision-making problems. At present, some reinforcement learning techniques have been successfully applied in this field. However, some related studies can only deal with discrete action spaces, which are prone to dimensional explosion problems for complex systems, resulting in reduced accuracy of training results. Therefore, more and more scholars are using reinforcement learning algorithms with continuous decision-making capabilities to optimize scheduling. With the increasing complexity of energy systems, the Deep Deterministic Policy Gradient (DDPG) algorithm with problems such as overestimation of state-action function values ​​(Q function values), unstable policy updates, and insufficient exploration can no longer meet the requirements. Therefore, some scholars have applied the Twin Delayed Deep Deterministic Policy Gradient (TD3) algorithm with dual Q functions and target policy smoothing to low-carbon economic scheduling. However, traditional deep neural networks often consume a lot of computing resources when dealing with complex high-dimensional state spaces, and it is difficult to make full use of sparse information. As the third generation of neural networks, Spiking Neural Networks (SNN) can utilize event-driven mechanisms and group coding characteristics to significantly reduce computational complexity and energy consumption, making them particularly suitable for real-time decision-making tasks.

[0005] In order to solve the problem of low energy efficiency of the traditional TD3 algorithm in dealing with complex continuous control problems and improve the robustness of sample distribution in experience replay (ER), it is necessary to design a scheduling control method for a PV-storage direct-flexible system. Summary of the invention

[0006] The purpose of the present invention is to solve at least one of the technical problems existing in the prior art and to provide a scheduling control method for a PV-storage direct-flexible system.

[0007] To achieve the above object, the technical solution adopted by the present invention is as follows: a scheduling control method for a PV-storage direct-flexible system, comprising the following steps: Step 1: Establish an operation architecture model of the PV-storage direct-flexible system; Step 2, using the DC bus voltage deviation as a regulation signal, formulate a power control strategy for the solar-storage direct-flexible system; Step 3: Based on the operation architecture model of the PV-storage direct-flexible system and the power consumption control strategy, an optimization scheduling model of the PV-storage direct-flexible system is established; Step 4, combine the pulse neural network and the priority mechanism of experience replay, propose a pulse experience replay reinforcement learning algorithm, optimize the photovoltaic storage direct and flexible system optimization scheduling model based on the pulse experience replay reinforcement learning algorithm, and schedule the photovoltaic storage direct and flexible system through the optimized photovoltaic storage direct and flexible system optimization scheduling model.

[0008] Furthermore, in step 1, it specifically includes: In the PV-storage DC-flexible system, the flexible adjustable load model responds to the DC bus voltage signal as follows: (1); (2); In the formula, , T represents the total period of power adjustment, , N represents the total number of load types; is the load flexibility coefficient; Solar storage direct and flexible system Time period The regulated power and unregulated power of the type of load; For the solar storage direct and flexible system DC bus voltage deviation during the time period; For the Power of type load; Respectively The lower and upper limits of power for the type of load; It is expressed by the following formula: (3); Where: It is the rated power when no power adjustment is performed for the corresponding flexible load; Respectively represent the maximum value, minimum value and rated voltage of the DC bus voltage; In the solar-storage direct-flexible system, the energy storage battery BES uses the state of charge SOC to describe: (4); In the formula, For BES The state of charge at the moment; For BES Output at the moment, a value greater than 0 indicates discharging, and a value less than 0 indicates charging; is the rated capacity of BES; is the self-loss coefficient of BES; and are the BES charge and discharge coefficients, respectively; The corresponding model of BES charging and discharging power and DC bus voltage signal is as follows: (5); BES charging and discharging power constraints are as follows: (6); BES charging and discharging ramp constraints are as follows: (7); The BES state of charge constraints are as follows: (8); In the formula They are The adjusted charging and discharging power and rated charging and discharging power of the time period energy storage battery; is the flexibility coefficient of the energy storage battery; They are the landslide limit and climbing limit of the energy storage battery respectively; are the maximum discharge power, maximum charging power and Working power of time storage battery; The lower and upper capacity limits are set for the energy storage battery respectively; In the EV charging model, the charging load of the electric vehicle EV at the building charging station is fitted. The charging load is related to the time when the electric vehicle arrives at the building charging station, the daily mileage of the electric vehicle and the charging time. The time it takes for electric vehicles to arrive at a building charging station follows a normal distribution, and its probability density function is: (9); Where: The time it takes for electric vehicles to reach the charging station; is the standard deviation of the fitted function of the electric vehicle arrival time; is the mean of the fitting function of the electric vehicle arrival time; The daily mileage of electric vehicles also follows a normal distribution, and its probability density function is: (10); Where: is the daily mileage; is its standard deviation; is its mean value; Charging time for electric vehicles , determined by the state of charge of the electric vehicle before and after charging: (11); Where: They are the state of charge of the battery before and after charging; Indicates the battery capacity of an electric vehicle; is the charging pile power; For charging efficiency; The actual state of charge after charging is related to the mileage and initial state of charge of the electric vehicle: (12); Where: The power consumption per 100 kilometers of the EV; The Monte Carlo method is used to simulate the charging load under disordered charging state in the solar-storage direct-flexible system. In the solar-storage direct-flexible system, when the state of charge of the electric vehicle does not reach the minimum capacity set by the EV user, the charging pile charges the electric vehicle at the rated power; when it is higher than the set capacity, the charging pile and the electric vehicle are regarded as a storage battery, and the minimum capacity set by the EV user is determined by the Monte Carlo simulation results, that is: (13); In the formula, For the Electric vehicles in The state of charge at the moment; for Moment The charging and discharging power of an electric vehicle. A value greater than 0 indicates discharging, and a value less than 0 indicates charging. is the rated capacity of the EV; is the self-loss coefficient of EV; and are the EV charging and discharging coefficients respectively; , M Expressed as the total number of electric vehicles; The charging and discharging power of electric vehicles responds to the DC bus voltage signal as follows: (14); In the formula, For the The rated charging and discharging power of an electric vehicle, For the The required state of charge set by the user of an electric vehicle; The EV charging and discharging power constraints are as follows: (15); EV charging and discharging ramp constraints: (16); The EV state of charge constraints are as follows: (17); In the formula, Respectively The maximum charging and discharging power of the vehicle and Working power at all times; Respectively The landslide limit and climbing limit of electric vehicle batteries; Respectively The vehicle's minimum and maximum state of charge and State of charge at the moment.

[0009] Furthermore, in step 2, it specifically includes: The DC bus voltage deviation As a regulating signal for the power of adjustable devices in the PV-storage direct-flexible system: When the power on the energy supply side is greater than the power on the demand side, the power consumption of each device increases; when When the power on the supply side is insufficient, the power consumption of each device is reduced; therefore, the economic operation of the photovoltaic storage direct-flexible system is achieved by solving a suitable DC bus voltage signal; The power of various types of loads, batteries and electric vehicle charging and discharging powers adjust their own power according to the DC bus signal and the corresponding equipment operation strategy; The power balance equation of the PV-storage direct-flexible system is as follows: (18); In the formula, for The total power of all loads at the moment, for Photovoltaic power at all times, for The solar-storage direct-flexible system draws power from the grid at all times. for Always discard optical power. regulating power for flexible devices; The regulated power of the flexible device is: (19); In the formula, for Parameters determined by the load connected to the DC bus of the solar-storage-DC-flexible system at any given moment; In order to improve photovoltaic consumption and reduce dependence on external power grid, the regulated power should be equal to the required power of the photovoltaic storage direct-flexible system, that is: (20); In the formula, For the solar storage direct and flexible system The power required for the time period, when the operating conditions of the PV-storage direct-flexible system meet equation (20), that is, the total regulated power of various flexible loads and flexible equipment is equal to the required regulated power of the PV-storage direct-flexible system, then the combined equations (19) and (20) are: (twenty one); (twenty two); In the formula, is the system voltage gain coefficient. The building during this period is considered a zero-carbon building, and the abandoned solar power is 0; When the PV-storage direct-flexible system cannot adjust the power to meet equation (20), equation (22) will not hold. At this time, it is necessary to find a voltage difference signal to guide the flexible equipment of the PV-storage direct-flexible system to adjust the power and achieve the optimal economic operation of the PV-storage direct-flexible system. The voltage difference signal of the PV-storage direct-flexible system is given by equation (21). By optimizing the , and obtain the power adjustment scheme of the flexible equipment of the photovoltaic storage direct-flexible system to achieve the economic operation of the photovoltaic storage direct-flexible system.

[0010] Furthermore, in step 3, it specifically includes: The economic dispatch objective function of the optimal dispatch model of the PV-storage direct-flexible system is as follows: (twenty three); (twenty four); Where: for Total operating cost of the solar-storage direct-flexible system during the time period; Solar storage direct and flexible system Period operating costs; For the solar storage direct and flexible system Power dispatch cost of time-period flexible equipment; Forsaken Light Time period penalty cost; To purchase electricity Period cost; They are Time period The dispatch cost coefficient and regulation power of each load; They are Maintenance cost coefficient and regulation power of time-slot energy storage batteries; They are The dispatch cost coefficient of electric vehicles in the time period and the The charging and discharging power of electric vehicles; is the penalty coefficient for abandoning wind and solar power; for Amount of abandoned light during the time period; for The electricity purchase price during the period; For Architecture Time-period electricity purchase: Time-period electricity purchase; The constraints of the optimal scheduling model of the PV-storage-direct-flexible system include the flexible load in the PV-storage-direct-flexible system operation architecture model, the equipment adjustment power constraints, and the power balance equation of the PV-storage-direct-flexible system in the power consumption control strategy.

[0011] Furthermore, in step 4, it specifically includes: Step 4.1, action space design: The action space represents the set of changes in the action variables at each time step; (25); is the action space, which satisfies the constraints of the optimal scheduling model of the PV-storage direct-flexible system; Step 4.2, state space design: The state space is as follows: (26); is the state space, They are electric load, photovoltaic forecast value, electricity price, power purchase amount, and action amount at the previous moment; Step 4.3, reward function design: Reinforcement learning is to maximize the expected reward value by continuously learning and updating the agent's own strategy; therefore, the setting of the reward function should be related to the objective function in the scheduling model, and its reward function is as follows: (27); Where: They are The bonus value at the time and the weight coefficient of the total operating cost of the PV-storage direct-flexible system; It is a parameter that regresses the reward function to a positive value; Step 4.4, reinforcement learning algorithm: The pulse experience replay TD3 algorithm is used to solve the reinforcement learning problem in the continuous action space; the strategy update formula of TD3 is: (28); Where: It is the TD3 algorithm strategy; is the state quantity; Yes Status The strategy of the next agent; is the action-value function; It is the experience replay buffer; The historical data is input into the spiking neural network SNN, which encodes the historical data through LIF neurons to simulate the changes in the membrane potential of neurons after receiving stimulation. In this way, the spiking neural network can capture the time correlation of state changes in continuous control problems, so that historical data can be better utilized in the experience playback process. The formula for experience playback is: (29); Where: The time of corresponds to the time of the scheduling model, It is the state of the next moment; It is the time constant, which is used to control the decay speed of the pulse signal. The larger the time constant, the longer the pulse influence time. is the initial moment; yes Input pulse signal at the moment; After the experience replay mechanism is introduced into the spiking neural network, the experience replay is weighted according to the frequency and intensity of the pulse emission; the priority calculation formula is: (30); Where: Indicates The priority of a sample, which determines how often the sample is selected during playback; Indicates The TD error of a sample. The larger the TD error, the more important the sample is. is the control parameter of priority sampling, which controls the influence of priority. Represents a normalization factor, used to ensure that the sum of priorities is 1; Step 4.5, the algorithm flow of TD3 based on pulse experience playback: Step 4.51, Initialization: Initialize the Actor network and Critic network of the pulse neural network, set the strategy network and value function network required by the TD3 algorithm, and initialize the pulse experience playback buffer; Step 4.52, data preprocessing: pulse population encoding is performed on the state features in the offline data set, and normalized feature transformation is used to reduce data distribution differences; Step 4.53, pulse experience playback: sample a batch of experience from the pulse experience playback buffer, and weight the experience data according to the Q value error output by the pulse neural network; Step 4.54, update the value function: use the sampled pulse coding experience to update the value function based on the TD3 Critic network and calculate the Q value of the state-action pair; Step 4.55, update strategy: decode the pulse signal through the Actor network to generate actions, and use the loss function with pulse regularization term to update the policy network to minimize the behavior error; Step 4.56, update the target network: periodically update the target value function and the target policy network, and synchronize the pulse neural network parameters; Step 4.57, evaluate performance: evaluate the performance of the strategy at fixed intervals and test the control effect of the pulse experience playback TD3 algorithm through the simulation environment; Step 4.58, save the training model: save the trained pulse experience playback TD3 algorithm model for subsequent testing and application; Step 4.6, optimized scheduling process of TD3 based on pulse experience playback: Step 4.61, inputting photovoltaic historical data and load historical data into the reinforcement learning environment, and initializing various reinforcement learning network parameters; Step 4.62, conduct offline scheduling training, and obtain the current state quantity from the reinforcement learning environment: power load, photovoltaic forecast value, electricity price, power purchase amount, and action quantity at the previous moment; Step 4.63, the agent takes an action based on the input state; Step 4.64, perform actions in the reinforcement learning environment to obtain the state at the next moment; Step 4.65, when the set number of training times is reached, the optimization scheduling model of the PV-storage-direct-flexible system is saved; Step 4.66, input the real-time data into the saved training model, and the solar-storage direct-flexible system optimization scheduling model gives a real-time scheduling strategy.

[0012] From the above description of the present invention, it can be seen that compared with the prior art, the present invention includes the following beneficial effects: the present invention is based on the direct coupling relationship between the bus voltage and the load power in the DC system, and guides the DC bus changes to schedule the flexible loads and equipment in the system through the source-load power difference in the system; and the pulse experience replay TD3 algorithm proposed in the present invention is used to realize the economic scheduling of the photovoltaic storage direct and flexible system. The present invention first establishes a mathematical model of the photovoltaic storage direct and flexible system considering photovoltaics, energy storage, electric vehicles, and various flexible loads, and at the same time converts the mathematical model into a reinforcement learning environment, and uses the pulse experience replay TD3 algorithm to solve and make decisions, which can effectively reduce the dependence of the photovoltaic storage direct and flexible system on the external network, and reduce the system operating cost and the abandonment rate. BRIEF DESCRIPTION OF THE DRAWINGS

[0013] Figure 1 The present invention is a flowchart of the steps of a scheduling control method for a PV-storage direct-flexible system in a preferred embodiment of the present invention. DETAILED DESCRIPTION

[0014] The technical scheme in the embodiment of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiment of the present invention. Obviously, the described embodiment is only a part of the embodiment of the present invention, not all of the embodiments. Based on the embodiment of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative work are within the scope of protection of the present invention.

[0015] Reference Figure 1 As shown, a preferred embodiment of the present invention is a scheduling control method for a PV-storage direct-flexible system, comprising the following steps: Step 1: Establish an operation architecture model of the PV-storage direct-flexible system; Step 2, using the DC bus voltage deviation as a regulation signal, formulate a power control strategy for the solar-storage direct-flexible system; Step 3: Based on the operation architecture model of the PV-storage direct-flexible system and the power consumption control strategy, an optimization scheduling model of the PV-storage direct-flexible system is established; Step 4, combine the pulse neural network and the priority mechanism of experience replay, propose a pulse experience replay reinforcement learning algorithm, optimize the photovoltaic storage direct and flexible system optimization scheduling model based on the pulse experience replay reinforcement learning algorithm, and schedule the photovoltaic storage direct and flexible system through the optimized photovoltaic storage direct and flexible system optimization scheduling model.

[0016] As a preferred embodiment of the present invention, it may also have the following additional technical features: In this embodiment, step 1 specifically includes: In the PV-storage DC-flexible system, the flexible adjustable load model responds to the DC bus voltage signal as follows: (1); (2); In the formula, , T represents the total period of power adjustment, , N represents the total number of load types; is the load flexibility coefficient; Solar storage direct and flexible system Time period The regulated power and unregulated power of the type of load; For the solar storage direct and flexible system DC bus voltage deviation during the time period; For the Power of type load; Respectively The lower and upper limits of power for the type of load; It is expressed by the following formula: (3); Where: It is the rated power when no power adjustment is performed for the corresponding flexible load; Respectively represent the maximum value, minimum value and rated voltage of the DC bus voltage; In the solar-storage direct-flexible system, the energy storage battery BES uses the state of charge SOC to describe: (4); In the formula, For BES The state of charge at the moment; For BES Output at the moment, a value greater than 0 indicates discharging, and a value less than 0 indicates charging; is the rated capacity of BES; is the self-loss coefficient of BES; and are the BES charge and discharge coefficients, respectively; The corresponding model of BES charging and discharging power and DC bus voltage signal is as follows: (5); BES charging and discharging power constraints are as follows: (6); BES charging and discharging ramp constraints are as follows: (7); The BES state of charge constraints are as follows: (8); In the formula They are The adjusted charging and discharging power and rated charging and discharging power of the time period energy storage battery; is the flexibility coefficient of the energy storage battery; They are the landslide limit and climbing limit of the energy storage battery respectively; are the maximum discharge power, maximum charging power and Working power of time storage battery; The lower and upper capacity limits are set for the energy storage battery respectively; In the EV charging model, the charging load of the electric vehicle EV at the building charging station is fitted. The charging load is related to the time when the electric vehicle arrives at the building charging station, the daily mileage of the electric vehicle and the charging time. The time it takes for electric vehicles to arrive at a building charging station follows a normal distribution, and its probability density function is: (9); Where: The time it takes for electric vehicles to reach the charging station; is the standard deviation of the fitted function of the electric vehicle arrival time; is the mean of the fitting function of the electric vehicle arrival time; The daily mileage of electric vehicles also follows a normal distribution, and its probability density function is: (10); Where: is the daily mileage; is its standard deviation; is its mean value; Charging time for electric vehicles , determined by the state of charge of the electric vehicle before and after charging: (11); Where: They are the state of charge of the battery before and after charging; Indicates the battery capacity of an electric vehicle; is the charging pile power; For charging efficiency; The actual state of charge after charging is related to the mileage and initial state of charge of the electric vehicle: (12); Where: The power consumption per 100 kilometers of the EV; The Monte Carlo method is used to simulate the charging load under disordered charging state in the solar-storage direct-flexible system. In the solar-storage direct-flexible system, when the state of charge of the electric vehicle does not reach the minimum capacity set by the EV user, the charging pile charges the electric vehicle at the rated power; when it is higher than the set capacity, the charging pile and the electric vehicle are regarded as a storage battery, and the minimum capacity set by the EV user is determined by the Monte Carlo simulation results, that is: (13); In the formula, For the Electric vehicles in The state of charge at the moment; for Moment The charging and discharging power of an electric vehicle. A value greater than 0 indicates discharging, and a value less than 0 indicates charging. is the rated capacity of the EV; is the self-loss coefficient of EV; and are the EV charging and discharging coefficients respectively; , M Expressed as the total number of electric vehicles; The charging and discharging power of electric vehicles responds to the DC bus voltage signal as follows: (14); In the formula, For the The rated charging and discharging power of an electric vehicle, For the The required state of charge set by the user of an electric vehicle; The EV charging and discharging power constraints are as follows: (15); EV charging and discharging ramp constraints: (16); The EV state of charge constraints are as follows: (17); In the formula, Respectively The maximum charging and discharging power of the vehicle and Working power at all times; Respectively The landslide limit and climbing limit of electric vehicle batteries; Respectively The vehicle's minimum and maximum state of charge and State of charge at the moment.

[0017] In this embodiment, step 2 specifically includes: The DC bus voltage deviation As a regulating signal for the power of adjustable devices in the PV-storage direct-flexible system: When the power on the energy supply side is greater than the power on the demand side, the power consumption of each device increases; when When the power on the supply side is insufficient, the power consumption of each device is reduced; therefore, the economic operation of the photovoltaic storage direct-flexible system is achieved by solving a suitable DC bus voltage signal; The power of various types of loads, batteries and electric vehicle charging and discharging powers adjust their own power according to the DC bus signal and the corresponding equipment operation strategy; The power balance equation of the PV-storage direct-flexible system is as follows: (18); In the formula, for The total power of all loads at the moment, for Photovoltaic power at all times, for The solar-storage direct-flexible system draws power from the grid at all times. for Always discard optical power. regulating power for flexible devices; The regulated power of the flexible device is: (19); In the formula, for Parameters determined by the load connected to the DC bus of the solar-storage-DC-flexible system at any given moment; In order to improve photovoltaic consumption and reduce dependence on external power grid, the regulated power should be equal to the required power of the photovoltaic storage direct-flexible system, that is: (20); In the formula, For the solar storage direct and flexible system The power required for the time period is: when the operating conditions of the PV-storage direct-flexible system meet equation (20), that is, the total regulated power of various flexible loads and flexible equipment is equal to the required regulated power of the PV-storage direct-flexible system, then the combined equations (19) and (20) yield: (twenty one); (twenty two); In the formula, is the system voltage gain coefficient. The building during this period is considered a zero-carbon building, and the abandoned solar power is 0; When the PV-storage direct-flexible system cannot adjust the power to meet equation (20), equation (22) will not hold. At this time, it is necessary to find a voltage difference signal to guide the flexible equipment of the PV-storage direct-flexible system to adjust the power and achieve the optimal economic operation of the PV-storage direct-flexible system. The voltage difference signal of the PV-storage direct-flexible system is given by equation (21). By optimizing the , and obtain the power adjustment scheme of the flexible equipment of the photovoltaic storage direct-flexible system to achieve the economic operation of the photovoltaic storage direct-flexible system.

[0018] In this embodiment, step 3 specifically includes: The economic dispatch objective function of the optimal dispatch model of the PV-storage direct-flexible system is as follows: (twenty three); (twenty four); Where: for Total operating cost of the solar-storage direct-flexible system during the time period; Solar storage direct and flexible system Period operating costs; For the solar storage direct and flexible system Power dispatch cost of time-period flexible equipment; Forsaken Light Time period penalty cost; To purchase electricity Period cost; They are Time period The dispatch cost coefficient and regulation power of each load; They are Maintenance cost coefficient and regulation power of time-slot energy storage batteries; They are The dispatch cost coefficient of electric vehicles in the time period and the The charging and discharging power of an electric vehicle; is the penalty coefficient for abandoning wind and solar power; for Amount of abandoned light during the time period; for The electricity purchase price during the period; For Architecture Purchase electricity by time period; The constraints of the optimal scheduling model of the PV-storage-direct-flexible system include the flexible load in the PV-storage-direct-flexible system operation architecture model, the equipment adjustment power constraints, and the power balance equation of the PV-storage-direct-flexible system in the power consumption control strategy.

[0019] In this embodiment, step 4 specifically includes: Step 4.1, action space design: The action space represents the set of changes in the action variables at each time step; (25); is the action space, which satisfies the constraints of the optimal scheduling model of the PV-storage direct-flexible system; Step 4.2, state space design: The state space is as follows: (26); is the state space, They are electric load, photovoltaic forecast value, electricity price, power purchase amount, and action amount at the previous moment; Step 4.3, reward function design: Reinforcement learning is to maximize the expected reward value by continuously learning and updating the agent's own strategy; therefore, the setting of the reward function should be related to the objective function in the scheduling model, and its reward function is as follows: (27); Where: They are The bonus value at the time and the weight coefficient of the total operating cost of the PV-storage direct-flexible system; It is a parameter that regresses the reward function to a positive value; Step 4.4, reinforcement learning algorithm: The pulse experience replay TD3 algorithm is used to solve the reinforcement learning problem in the continuous action space; the strategy update formula of TD3 is: (28); Where: It is the TD3 algorithm strategy; is the state quantity; Yes Status The strategy of the next agent; is the action-value function; It is the experience replay buffer; The historical data is input into the spiking neural network SNN, which encodes the historical data through LIF neurons to simulate the changes in the membrane potential of neurons after receiving stimulation. In this way, the spiking neural network can capture the time correlation of state changes in continuous control problems, so that historical data can be better utilized in the experience playback process. The formula for experience playback is: (29); Where: The time of corresponds to the time of the scheduling model, It is the state of the next moment; It is the time constant, which is used to control the decay speed of the pulse signal. The larger the time constant, the longer the pulse influence time. is the initial moment; yes Input pulse signal at the moment; After the experience replay mechanism is introduced into the spiking neural network, the experience replay is weighted according to the frequency and intensity of the pulse emission; the priority calculation formula is: (30); Where: Indicates The priority of a sample, which determines how often the sample is selected during playback; Indicates The TD error of a sample. The larger the TD error, the more important the sample is. is the control parameter of priority sampling, which controls the influence of priority. Represents a normalization factor, used to ensure that the sum of priorities is 1; Step 4.5, the algorithm flow of TD3 based on pulse experience playback: Step 4.51, Initialization: Initialize the Actor network and Critic network of the pulse neural network, set the strategy network and value function network required by the TD3 algorithm, and initialize the pulse experience playback buffer; Step 4.52, data preprocessing: pulse population encoding is performed on the state features in the offline data set, and normalized feature transformation is used to reduce data distribution differences; Step 4.53, pulse experience playback: sample a batch of experience from the pulse experience playback buffer, and weight the experience data according to the Q value error output by the pulse neural network; Step 4.54, update the value function: use the sampled pulse coding experience to update the value function based on the TD3 Critic network and calculate the Q value of the state-action pair; Step 4.55, update strategy: decode the pulse signal through the Actor network to generate actions, and use the loss function with pulse regularization term to update the policy network to minimize the behavior error; Step 4.56, update the target network: periodically update the target value function and the target policy network, and synchronize the pulse neural network parameters; Step 4.57, evaluate performance: evaluate the performance of the strategy at fixed intervals and test the control effect of the pulse experience playback TD3 algorithm through the simulation environment; Step 4.58, save the training model: save the trained pulse experience playback TD3 algorithm model for subsequent testing and application; Step 4.6, optimized scheduling process of TD3 based on pulse experience playback: Step 4.61, inputting photovoltaic historical data and load historical data into the reinforcement learning environment, and initializing various reinforcement learning network parameters; Step 4.62, conduct offline scheduling training, and obtain the current state quantity from the reinforcement learning environment: power load, photovoltaic forecast value, electricity price, power purchase amount, and action quantity at the previous moment; Step 4.63, the agent takes an action based on the input state; Step 4.64, perform actions in the reinforcement learning environment to obtain the state at the next moment; Step 4.65, when the set number of training times is reached, the optimization scheduling model of the PV-storage-direct-flexible system is saved; Step 4.66, input the real-time data into the saved training model, and the solar-storage direct-flexible system optimization scheduling model gives a real-time scheduling strategy.

[0020] The above description is only a preferred specific implementation manner of the present invention, but the protection scope of the present invention is not limited thereto. Any technician familiar with the technical field can make equivalent replacements or changes according to the technical scheme and improved concepts of the present invention within the technical scope disclosed by the present invention, and they should be covered by the protection scope of the present invention.

Claims

1. A scheduling control method for a PV-storage direct-flexible system, characterized in that: The following steps are involved: Step 1: Establish an operation architecture model of the PV-storage direct-flexible system; Step 2, using the DC bus voltage deviation as a regulation signal, formulate a power control strategy for the solar-storage direct-flexible system; Step 3: Based on the operation architecture model of the PV-storage direct-flexible system and the power consumption control strategy, an optimization scheduling model of the PV-storage direct-flexible system is established; Step 4, combine the pulse neural network and the priority mechanism of experience replay, propose a pulse experience replay reinforcement learning algorithm, optimize the photovoltaic storage direct and flexible system optimization scheduling model based on the pulse experience replay reinforcement learning algorithm, and schedule the photovoltaic storage direct and flexible system through the optimized photovoltaic storage direct and flexible system optimization scheduling model.

2. The scheduling control method of a PV-storage direct-flexible system according to claim 1, characterized in that: In step 1, specifically include: In the PV-storage DC-flexible system, the flexible adjustable load model responds to the DC bus voltage signal as follows: （1）； （2）； In the formula, , T represents the total period of power adjustment, , N represents the total number of load types; is the load flexibility coefficient; Solar storage direct and flexible system Time period The regulated power and unregulated power of the type of load; For the solar storage direct and flexible system DC bus voltage deviation during the time period; For the Power of type load; Respectively The lower and upper limits of power for the type of load; It is expressed by the following formula: （3）； Where: It is the rated power when no power adjustment is performed for the corresponding flexible load; Respectively represent the maximum value, minimum value and rated voltage of the DC bus voltage; In the solar-storage direct-flexible system, the energy storage battery BES uses the state of charge SOC to describe: （4）； In the formula, For BES The state of charge at the moment; For BES Output at the moment, a value greater than 0 indicates discharging, and a value less than 0 indicates charging; is the rated capacity of BES; is the self-loss coefficient of BES; and are the BES charge and discharge coefficients, respectively; The corresponding model of BES charging and discharging power and DC bus voltage signal is as follows: （5）； BES charging and discharging power constraints are as follows: （6）； BES charging and discharging ramp constraints are as follows: （7）； The BES state of charge constraints are as follows: （8）； In the formula They are The adjusted charging and discharging power and rated charging and discharging power of the time period energy storage battery; is the flexibility coefficient of the energy storage battery; They are the landslide limit and climbing limit of the energy storage battery respectively; are the maximum discharge power, maximum charging power and Working power of time storage battery; The lower and upper capacity limits are set for the energy storage battery respectively; In the EV charging model, the charging load of the electric vehicle EV at the building charging station is fitted. The charging load is related to the time when the electric vehicle arrives at the building charging station, the daily mileage of the electric vehicle and the charging time. The time it takes for electric vehicles to arrive at a building charging station follows a normal distribution, and its probability density function is: (9)； Where: The time it takes for electric vehicles to reach the charging station; is the standard deviation of the fitted function of the electric vehicle arrival time; is the mean of the fitting function of the electric vehicle arrival time; The daily mileage of electric vehicles also follows a normal distribution, and its probability density function is: (10)； Where: is the daily mileage; is its standard deviation; is its mean value; Charging time for electric vehicles , determined by the state of charge of the electric vehicle before and after charging: (11)； Where: They are the state of charge of the battery before and after charging; Indicates the battery capacity of an electric vehicle; is the charging pile power; For charging efficiency; The actual state of charge after charging is related to the mileage and initial state of charge of the electric vehicle: (12)； Where: The power consumption per 100 kilometers of the EV; The Monte Carlo method is used to simulate the charging load under disordered charging state in the solar-storage direct-flexible system. In the solar-storage direct-flexible system, when the state of charge of the electric vehicle does not reach the minimum capacity set by the EV user, the charging pile charges the electric vehicle at the rated power; when it is higher than the set capacity, the charging pile and the electric vehicle are regarded as a storage battery, and the minimum capacity set by the EV user is determined by the Monte Carlo simulation results, that is: （13）； In the formula, For the Electric vehicles in The state of charge at the moment; for Moment The charging and discharging power of an electric vehicle. A value greater than 0 indicates discharging, and a value less than 0 indicates charging. is the rated capacity of the EV; is the self-loss coefficient of EV; and are the EV charging and discharging coefficients respectively; , M Expressed as the total number of electric vehicles; The charging and discharging power of electric vehicles responds to the DC bus voltage signal as follows: （14）； In the formula, For the The rated charging and discharging power of an electric vehicle, For the The required state of charge set by the user of an electric vehicle; The EV charging and discharging power constraints are as follows: （15）； EV charging and discharging ramp constraints: （16）； The EV state of charge constraints are as follows: （17）； In the formula, Respectively The maximum charging and discharging power of the vehicle and Working power at all times; Respectively The landslide limit and climbing limit of electric vehicle batteries; Respectively The vehicle's minimum and maximum state of charge and State of charge at the moment.

3. The scheduling control method of a PV-storage direct-flexible system according to claim 2 is characterized in that: In step 2, specifically including: The DC bus voltage deviation As a regulating signal for the power of adjustable devices in the PV-storage direct-flexible system: When the power on the energy supply side is greater than the power on the demand side, the power consumption of each device increases; when When the power on the supply side is insufficient, the power consumption of each device is reduced; therefore, the economic operation of the photovoltaic storage direct-flexible system is achieved by solving a suitable DC bus voltage signal; The power of various types of loads, batteries and electric vehicle charging and discharging powers adjust their own power according to the DC bus signal and the corresponding equipment operation strategy; The power balance equation of the PV-storage direct-flexible system is as follows: （18）； In the formula, for The total power of all loads at the moment, for Photovoltaic power at all times, for The solar-storage direct-flexible system draws power from the grid at all times. for Always discard optical power. regulating power for flexible devices; The regulated power of the flexible device is: （19）； In the formula, for Parameters determined by the load connected to the DC bus of the solar-storage-DC-flexible system at any given moment; In order to improve photovoltaic consumption and reduce dependence on external power grid, the regulated power should be equal to the required power of the photovoltaic storage direct-flexible system, that is: （20）； In the formula, For the solar storage direct and flexible system The power required for the time period, when the operating conditions of the PV-storage direct-flexible system meet equation (20), that is, the total regulated power of various flexible loads and flexible equipment is equal to the required regulated power of the PV-storage direct-flexible system, then the combined equations (19) and (20) are: （21）； （22）； In the formula, is the system voltage gain coefficient. The building during this period is considered a zero-carbon building, and the abandoned solar power is 0; When the PV-storage direct-flexible system cannot adjust the power to meet equation (20), equation (22) will not hold. At this time, it is necessary to find a voltage difference signal to guide the flexible equipment of the PV-storage direct-flexible system to adjust the power and achieve the optimal economic operation of the PV-storage direct-flexible system. The voltage difference signal of the PV-storage direct-flexible system is given by equation (21). By optimizing the , and obtain the power adjustment scheme of the flexible equipment of the photovoltaic storage direct-flexible system to achieve the economic operation of the photovoltaic storage direct-flexible system.

4. The scheduling control method of a PV-storage direct-flexible system according to claim 3 is characterized in that: In step 3, it specifically includes: The economic dispatch objective function of the optimal dispatch model of the PV-storage direct-flexible system is as follows: （23）； （24）； Where: for Total operating cost of the solar-storage direct-flexible system during the time period; Solar storage direct and flexible system Period operating costs; For the solar storage direct and flexible system Power dispatch cost of time-period flexible equipment; Forsaken Light Time period penalty cost; To purchase electricity Period cost; They are Time period The dispatch cost coefficient and regulation power of each load; They are Maintenance cost coefficient and regulation power of time-slot energy storage batteries; They are The dispatch cost coefficient of electric vehicles in the time period and the The charging and discharging power of an electric vehicle; is the penalty coefficient for abandoning wind and solar power; for Amount of abandoned light during the time period; for The electricity purchase price during the period; For Architecture Purchase electricity by time period; The constraints of the optimal scheduling model of the PV-storage-direct-flexible system include the flexible load in the PV-storage-direct-flexible system operation architecture model, the equipment adjustment power constraints, and the power balance equation of the PV-storage-direct-flexible system in the power consumption control strategy.

5. The scheduling control method of a PV-storage direct-flexible system according to claim 4, characterized in that: In step 4, specifically including: Step 4.1, action space design: The action space represents the set of changes in the action variables at each time step; （25）； is the action space, which satisfies the constraints of the optimal scheduling model of the PV-storage direct-flexible system; Step 4.2, state space design: The state space is as follows: （26）； is the state space, They are electric load, photovoltaic forecast value, electricity price, power purchase amount, and action amount at the previous moment; Step 4.3, reward function design: Reinforcement learning is to maximize the expected reward value by continuously learning and updating the agent's own strategy; therefore, the setting of the reward function should be related to the objective function in the scheduling model, and its reward function is as follows: （27）； Where: They are The bonus value at the time and the weight coefficient of the total operating cost of the PV-storage direct-flexible system; It is a parameter that regresses the reward function to a positive value; Step 4.4, reinforcement learning algorithm: The pulse experience replay TD3 algorithm is used to solve the reinforcement learning problem in the continuous action space; the strategy update formula of TD3 is: （28）； Where: It is the TD3 algorithm strategy; is the state quantity; Yes Status The strategy of the next agent; is the action-value function; It is the experience replay buffer; The historical data is input into the spiking neural network SNN, which encodes the historical data through LIF neurons to simulate the changes in the membrane potential of neurons after receiving stimulation. In this way, the spiking neural network can capture the time correlation of state changes in continuous control problems, so that historical data can be better utilized in the experience playback process. The formula for experience playback is: （29）； Where: The time of corresponds to the time of the scheduling model, It is the state of the next moment; It is the time constant, which is used to control the decay speed of the pulse signal. The larger the time constant, the longer the pulse influence time. is the initial moment; yes Input pulse signal at the moment; After the experience replay mechanism is introduced into the spiking neural network, the experience replay is weighted according to the frequency and intensity of the pulse emission; the priority calculation formula is: （30）； Where: Indicates The priority of a sample, which determines how often the sample is selected during playback; Indicates The TD error of a sample. The larger the TD error, the more important the sample is. is the control parameter of priority sampling, which controls the influence of priority. Represents a normalization factor, used to ensure that the sum of priorities is 1; Step 4.5, the algorithm flow of TD3 based on pulse experience playback: Step 4.51, Initialization: Initialize the Actor network and Critic network of the pulse neural network, set the strategy network and value function network required by the TD3 algorithm, and initialize the pulse experience playback buffer; Step 4.52, data preprocessing: pulse population encoding is performed on the state features in the offline data set, and normalized feature transformation is used to reduce data distribution differences; Step 4.53, pulse experience playback: sample a batch of experience from the pulse experience playback buffer, and weight the experience data according to the Q value error output by the pulse neural network; Step 4.54, update the value function: use the sampled pulse coding experience to update the value function based on the TD3 Critic network and calculate the Q value of the state-action pair; Step 4.55, update strategy: decode the pulse signal through the Actor network to generate actions, and use the loss function with pulse regularization term to update the policy network to minimize the behavior error; Step 4.56, update the target network: periodically update the target value function and the target policy network, and synchronize the pulse neural network parameters; Step 4.57, evaluate performance: evaluate the performance of the strategy at fixed intervals and test the control effect of the pulse experience playback TD3 algorithm through the simulation environment; Step 4.58, save the training model: save the trained pulse experience playback TD3 algorithm model for subsequent testing and application; Step 4.6, optimized scheduling process of TD3 based on pulse experience playback: Step 4.61, inputting the photovoltaic historical data and load historical data into the reinforcement learning environment, and initializing the reinforcement learning network parameters; Step 4.62, conduct offline scheduling training, and obtain the current state quantity from the reinforcement learning environment: power load, photovoltaic forecast value, electricity price, power purchase amount, and action quantity at the previous moment; Step 4.63, the agent takes an action based on the input state; Step 4.64, perform actions in the reinforcement learning environment to obtain the state at the next moment; Step 4.65, when the set number of training times is reached, the optimization scheduling model of the PV-storage-direct-flexible system is saved; Step 4.66, input the real-time data into the saved training model, and the solar-storage direct-flexible system optimization scheduling model gives a real-time scheduling strategy.

Images