Energy scheduling method of wind and light storage, charging and discharging integrated system and related equipment

Abstract

The invention discloses an energy scheduling method of a wind and light storage, charging and discharging integrated system and related equipment. The method comprises the following steps: constructing a simulation model of the wind and light storage, charging and discharging integrated system; wherein the simulation model comprises a photovoltaic power generation model, a wind power generation model, an energy storage battery model, a vehicle model and an energy storage air conditioner model; according to the simulation model, based on a reinforcement learning algorithm, constructing a first energy scheduling model of the wind and light storage charging and discharging integrated system; in combination with the moment when the vehicle leaves the parking lot, a second energy scheduling model of the vehicle is constructed by taking the minimum difference value between the preset target SOC and the SOC after vehicle scheduling as a target; and carrying out joint training on the first energy scheduling model and the second energy scheduling model so as to carry out energy scheduling on the wind and light storage charging and discharging integrated system according to the trained first energy scheduling model and the second energy scheduling model. By adopting the embodiment of the invention, the modeling process can be simplified, and the computing power required for solving the algorithm is saved.

Description

Technical Field

[0001] The present application relates to the technical field of power supply systems, and in particular to energy scheduling and related equipment of a wind-solar-storage-charging-discharging integrated system. Background Art

[0002] The integrated wind, solar, storage, charging and discharging system is a power supply system that comprehensively utilizes renewable energy. It generates electricity through wind turbines and solar photovoltaic panels, and stores the generated electricity in a variety of energy storage devices, such as energy storage batteries, electric vehicles, etc.

[0003] Currently, there are two energy (i.e., electric energy) scheduling schemes for wind, solar, storage, charging and discharge integrated systems: (1) using traditional optimization schemes for energy scheduling, but this scheme requires repeated and complex modeling of the wind, solar, storage, charging and discharge integrated system; (2) using artificial intelligence schemes for energy scheduling, but this scheme consumes a lot of computing power. Summary of the invention

[0004] The present application provides an energy scheduling and related equipment for a wind-solar-storage-charging-discharging integrated system to solve the problem in the prior art of requiring repeated and complex modeling of the wind-solar-storage-charging-discharging integrated system and consuming a large amount of computing power.

[0005] To achieve the above objectives, the present application provides an energy scheduling method for a wind-solar-storage-charging-discharging integrated system, including: Construct a simulation model of a wind-solar-storage-charging-discharging integrated system; wherein the simulation model includes: a photovoltaic power generation model, a wind power generation model, an energy storage battery model, a vehicle model, and an energy storage air conditioning model; According to the simulation model and based on the reinforcement learning algorithm, a first energy scheduling model of the wind-solar-storage-charging-discharging integrated system is constructed; Combined with the moment when the vehicle leaves the parking lot, the second energy scheduling model of the vehicle is constructed with the goal of minimizing the difference between the preset target SOC and the SOC after the vehicle is scheduled; The first energy scheduling model and the second energy scheduling model are jointly trained to perform energy scheduling on the wind-solar-storage-charging-discharging integrated system according to the trained first energy scheduling model and the second energy scheduling model.

[0006] As an improvement of the above solution, the photovoltaic power generation model is specifically expressed as: in, Represents the power generated by the photovoltaic panel. represents solar radiation, represents the reference radiation, Represents the rated power of the solar cell array, represents the temperature coefficient, represents the reference temperature, represents the temperature of the solar cell array, , Indicates the ambient temperature of the photovoltaic panel. Represents the rated operating temperature of the solar cell array, Indicates the air temperature when the solar array is rated to operate. Indicates the light radiation on the surface of the solar cell array during rated operation; The wind power generation model is specifically expressed as: in, Represents the output power of the wind turbine generator set. Indicates the rated power of the wind turbine generator set; Indicates wind speed; Indicates the cut-in wind speed, Indicates the rated wind speed, represents the cut-out wind speed; , , They are: The energy storage battery model is specifically expressed as: in, Indicates that the energy storage battery is The amount of electricity stored at any time, Indicates that the energy storage battery is The amount of electricity stored at any time, Indicates that the energy storage battery is The charging capacity of the time period, Indicates the charging efficiency of the energy storage battery. Indicates that the energy storage battery is The discharge capacity of the time period, represents the discharge efficiency of the energy storage battery; the energy storage battery model satisfies the following constraints: in, and is a binary variable, when Indicates that the energy storage battery is Charging during the period, Indicates that the energy storage battery is Discharge during the period, Indicates that the energy storage battery is The charging capacity of the time period, Indicates the maximum charging rate of the energy storage battery, Indicates that the energy storage battery is The discharge capacity of the time period, Indicates the maximum discharge rate of the energy storage battery, Indicates the maximum battery capacity of the energy storage battery; The vehicle model is specifically expressed as: in, Indicates the current SOC of the vehicle. represents the initial SOC of the vehicle, Indicates that the vehicle is in the time slot If the charging / discharging power at is a positive value, Indicates that the vehicle is in the time slot If the charging power at is a negative value, Indicates that the vehicle is in the time slot The discharge power at Indicates the length of the time slot, Indicates the vehicle battery capacity, represents a set of available charging time slots; wherein the vehicle model satisfies the following constraints: in, Indicates the maximum chargeable power of the vehicle, Indicates the maximum dischargeable power of the vehicle; The energy storage air conditioning model is specifically expressed as: in, express The indoor temperature of the building at all times; express The outdoor temperature of the building at all times; Indicates that energy storage air conditioning Cooling / heating capacity at the moment, if is a positive value, Indicates that energy storage air conditioning The cooling capacity at the time is a negative value, Indicates that energy storage air conditioning Heating capacity at all times; It represents the equivalent thermal resistance of the building; represents the equivalent heat capacity of the building, represents the differential, where the cooling / heating capacity of the energy storage air conditioner Power consumption of energy storage air conditioner and thermal-electric conversion coefficient of energy storage air conditioner Relevant, namely: (1) When the set temperature Equal to room temperature When the energy storage air conditioner consumes power for: (2) When the outdoor temperature Less than the set temperature Refrigeration time for: After a period of time, the indoor temperature And energy storage air conditioning power consumption for: (3) When the outdoor temperature Greater than the set temperature Refrigeration time for: After a period of time, the indoor temperature And energy storage air conditioning power consumption for: in, Indicates the maximum power of the energy storage air conditioner. represents a natural constant, Represents the natural logarithm.

[0007] As an improvement of the above solution, the first energy scheduling model of the wind-solar-storage-charging-discharging integrated system is constructed based on the simulation model and the reinforcement learning algorithm, including: The energy scheduling problem of the simulation model is converted into a Markov decision problem, and the observation space, action space, reward function, strategy network and action value network of the first energy scheduling model are constructed.

[0008] As an improvement of the above solution, the reward function is: in, express The electricity selling price at the time, express The amount of electricity sold by the wind-solar-storage-charging-discharging integrated system at all times, express The electricity purchase price at the time; express The amount of electricity purchased by the wind-solar-storage-charging-discharging integrated system at all times; express Moment A penalty is imposed if the SOC of a vehicle does not meet travel demand when it leaves the parking lot.

[0009] As an improvement of the above scheme, the loss function of the action value network is: in, Representing Action-Value Network The loss function is represents the computational expectation, represents the state distribution and action distribution sampled by the previous agent in the environment exploration, Representing the target action value network The output value, Indicates status Adopt an action Instant rewards, represents the discount rate, represents the state transition probability, Indicates status The value function of: express The state of the moment; express The action of the moment, express The state of the moment; in, is the maximum entropy objective The temperature parameters, is a strategic network, Representation Policy Network In Status Output action probability; express The state of the moment, express Momentary action; The loss function of the policy network is: in, Representation Policy Network The loss function is represents the computational expectation, represents the input noise, The noise of the input It follows a normal distribution. Indicates action resampling of; The temperature parameter The loss function is: in, is the preset minimum expected entropy.

[0010] As an improvement of the above solution, the second energy scheduling model is specifically expressed as: In the formula, Is an identifier indicating whether the vehicle has arrived at the parking lot. When the car arrives at the parking lot, =1, when If the car does not arrive at the parking lot, ; Indicates the preset target SOC; Indicates the time when the vehicle leaves the parking lot. Indicates the current moment; Indicates The maximum charge capacity of the vehicle, Indicates the charging efficiency of the vehicle, Indicates The maximum power of the vehicle; Indicates The SOC after vehicle dispatching, ; Indicates The current SOC of the vehicle; Indicates The charging capacity of the vehicle, Indicates The discharge capacity of the vehicle; Represents the discharge efficiency of the vehicle, where the charging and discharging power needs to meet the following constraints: In the formula, is the total charging and discharging quota allocated by the agent to the parking lot.

[0011] To achieve the above objectives, the present application also provides an energy dispatching device for a wind-solar-storage-charging-discharging integrated system, including: The first construction module is used to construct a simulation model of a wind-solar-storage-charging-discharging integrated system; wherein the simulation model includes: a photovoltaic power generation model, a wind power generation model, an energy storage battery model, a vehicle model and an energy storage air conditioning model; A second construction module is used to construct a first energy scheduling model of the wind-solar-storage-charging-discharging integrated system based on the simulation model and a reinforcement learning algorithm; The third building module is used to build a second energy scheduling model for the vehicle based on the moment when the vehicle leaves the parking lot and takes the minimum difference between the preset target SOC and the SOC after the vehicle is scheduled as the goal; An energy scheduling module is used to jointly train the first energy scheduling model and the second energy scheduling model to perform energy scheduling on the wind-solar storage-charging-discharging integrated system according to the trained first energy scheduling model and the second energy scheduling model.

[0012] To achieve the above-mentioned objectives, an embodiment of the present application also provides an energy scheduling device for a wind-solar-storage-charging-discharging integrated system, comprising a processor, a memory, and a computer program stored in the memory and configured to be executed by the processor, wherein the processor implements the energy scheduling method for the wind-solar-storage-charging-discharging integrated system as described above when executing the computer program.

[0013] To achieve the above-mentioned purpose, an embodiment of the present application also provides a computer-readable storage medium, which includes a stored computer program; wherein, when the computer program is running, it controls the device where the computer-readable storage medium is located to execute the energy scheduling method of the integrated wind, solar, storage, charging and discharge system as mentioned above.

[0014] To achieve the above objectives, an embodiment of the present application also provides a computer program product, including a computer program / instruction, which, when executed by a processor, implements the energy scheduling method of the wind-solar-storage-charging-discharging integrated system as described above.

[0015] Compared with the prior art, the energy scheduling method and related equipment of a wind-solar storage-charging-discharging integrated system provided in the embodiment of the present application, by considering photovoltaic power generation, wind power generation, energy storage battery, vehicle and energy storage air conditioner, constructs a simulation model of the wind-solar storage-charging-discharging integrated system, and according to the simulation model, based on the reinforcement learning algorithm, constructs the first energy scheduling model of the wind-solar storage-charging-discharging integrated system, and at the same time, combined with the moment when the vehicle leaves the parking lot, with the goal of minimizing the difference between the preset target SOC and the SOC after vehicle scheduling, constructs the second energy scheduling model of the vehicle; finally, the first energy scheduling model and the second energy scheduling model are jointly trained to perform energy scheduling on the wind-solar storage-charging-discharging integrated system according to the trained first energy scheduling model and the second energy scheduling model. It can be seen that the embodiment of the present application does not need to repeatedly and complexly model the wind-solar storage-charging-discharging integrated system through the reinforcement learning algorithm, and at the same time, the second energy scheduling model constructed in combination with the traditional optimization scheme can solve the problem of power distribution of electric vehicles, avoid the problem of too large action space caused by a large number of electric vehicles, greatly reduce the computing power required for training, and make energy scheduling more comprehensive and accurate. The embodiment of the present invention combines traditional optimization solutions with artificial intelligence solutions to propose a new energy scheduling method, which not only simplifies the modeling process, but also saves the computing power required for solving the algorithm, and also makes energy scheduling more comprehensive and accurate. BRIEF DESCRIPTION OF THE DRAWINGS

[0016] Figure 1 It is a flow chart of an energy scheduling method for a wind-solar-storage-charging-discharging integrated system provided in an embodiment of the present application; Figure 2 This is a structural block diagram of an energy dispatching device for a wind-solar-storage-charging-discharging integrated system provided in an embodiment of the present application; Figure 3 It is a structural block diagram of an energy dispatching device of a wind-solar-storage-charging-discharging integrated system provided in an embodiment of the present application. DETAILED DESCRIPTION

[0017] The following will be combined with the drawings in the embodiments of the present application to clearly and completely describe the technical solutions in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.

[0018] See also Figure 1 , Figure 1 1 is a flow chart of an energy dispatching method for a wind-solar-storage-charging-discharging integrated system provided in an embodiment of the present application. The energy dispatching method for the wind-solar-storage-charging-discharging integrated system includes: S1. Construct a simulation model of a wind-solar-storage-charging-discharging integrated system; wherein the simulation model includes: a photovoltaic power generation model, a wind power generation model, an energy storage battery model, a vehicle model, and an energy storage air conditioning model; S2. According to the simulation model and based on a reinforcement learning algorithm, a first energy scheduling model of the wind-solar-storage-charging-discharging integrated system is constructed; S3, combining the moment when the vehicle leaves the parking lot, with the goal of minimizing the difference between the preset target SOC and the SOC after vehicle scheduling, constructing a second energy scheduling model for the vehicle; S4. Jointly train the first energy scheduling model and the second energy scheduling model to perform energy scheduling on the wind-solar-storage-charging-discharging integrated system according to the trained first energy scheduling model and the second energy scheduling model.

[0019] In the embodiment of the present application, firstly, the photovoltaic, wind power, energy storage battery (ESS), vehicle (i.e. electric vehicle) and energy storage air conditioner of the wind-solar-storage-charging-discharging integrated system are modeled, and in particular, the energy storage air conditioner is modeled based on the preset temperature, so as to construct a simulation model of the wind-solar-storage-charging-discharging integrated system. With respect to the global energy scheduling of the wind-solar-storage-charging-discharging integrated system, a first energy scheduling model of the wind-solar-storage-charging-discharging integrated system is constructed based on the reinforcement learning algorithm. With respect to the local energy scheduling of the vehicle, in combination with the moment when the vehicle leaves the parking lot, a second energy scheduling model of the vehicle is constructed with the goal of minimizing the difference between the preset target SOC and the SOC after the vehicle is scheduled, so as to realize the charging and discharging energy distribution of the electric vehicle.

[0020] Next, the first energy scheduling model and the second energy scheduling model are jointly trained. Specifically, the second energy scheduling model is integrated into the environment of the first energy scheduling model for training. Finally, the trained first energy scheduling model and the second energy scheduling model are used to schedule the energy of the wind-solar storage-charging-discharging integrated system. In other words, the traditional optimization algorithm is integrated into the reinforcement learning environment for training to form an improved reinforcement learning algorithm, so as to use the improved reinforcement learning algorithm to optimize the global energy scheduling of the wind-solar storage-charging-discharging integrated system; the traditional optimization algorithm is used to optimize the charging and discharging scheduling of the vehicle, and finally the optimal energy scheduling scheme of the wind-solar storage-charging-discharging integrated system is obtained to realize the energy scheduling of the wind-solar storage-charging-discharging integrated system.

[0021] Among them, SOC refers to the charge state of the battery, that is, the remaining power, and its value range is 0~1. When SOC=0, it means that the battery is fully discharged, and when SOC=1, it means that the battery is fully charged.

[0022] Specifically, the photovoltaic power generation model is specifically expressed as: in, Represents the power generated by the photovoltaic panel. represents solar radiation, represents the reference radiation, Represents the rated power of the solar cell array, represents the temperature coefficient, represents the reference temperature, represents the temperature of the solar cell array, , Indicates the ambient temperature of the photovoltaic panel. Represents the rated operating temperature of the solar cell array, Indicates the air temperature when the solar array is rated to operate. Indicates the light radiation on the surface of the solar cell array during rated operation; It is understandable that the power generation of a photovoltaic panel depends on the ambient temperature and solar radiation. The embodiment of the present application constructs a photovoltaic power generation model based on the ambient temperature and solar radiation.

[0023] Specifically, the wind power generation model is specifically expressed as: in, Represents the output power of the wind turbine generator set. Indicates the rated power of the wind turbine generator set; Indicates wind speed; Indicates the cut-in wind speed, Indicates the rated wind speed, represents the cut-out wind speed; , , They are: It can be understood that the output power of the wind turbine generator set can be determined by its power curve. The embodiment of the present application constructs a wind power generation model through the power curve.

[0024] Specifically, the energy storage battery model is specifically expressed as: in, Indicates that the energy storage battery is The amount of electricity stored at any time, Indicates that the energy storage battery is The amount of electricity stored at any time, Indicates that the energy storage battery is The charging capacity of the time period, Indicates the charging efficiency of the energy storage battery. Indicates that the energy storage battery is The discharge capacity of the time period, represents the discharge efficiency of the energy storage battery; the energy storage battery model satisfies the following constraints: in, and is a binary variable, when Indicates that the energy storage battery is Charging during the period, Indicates that the energy storage battery is Discharge during the period, Indicates that the energy storage battery is The charging capacity of the time period, Indicates the maximum charging rate of the energy storage battery, Indicates that the energy storage battery is The discharge capacity of the time period, Indicates the maximum discharge rate of the energy storage battery, Indicates the maximum battery capacity of the energy storage battery; Understandably, It means that the energy storage battery is not charged and discharged at the same time in a period of time; Indicates that the amount of electricity that can be charged in each time interval of the energy storage battery is affected by its maximum charging efficiency; It means that the amount of electricity that can be discharged by the energy storage battery in each time interval is affected by its maximum discharge efficiency; It means that the amount of electricity stored in the energy storage battery should not be greater than the maximum battery capacity of the energy storage battery. At the same time, considering the life of the energy storage battery, the energy storage battery should not be over-discharged.

[0025] The vehicle model is specifically expressed as: in, Indicates the current SOC of the vehicle. represents the initial SOC of the vehicle, Indicates that the vehicle is in the time slot If the charging / discharging power at is a positive value, Indicates that the vehicle is in the time slot If the charging power at is a negative value, Indicates that the vehicle is in the time slot The discharge power at Indicates the length of the time slot, Indicates the vehicle battery capacity, represents a set of available charging time slots; wherein the vehicle model satisfies the following constraints: in, Indicates the maximum chargeable power of the vehicle, Indicates the maximum dischargeable power of the vehicle; Understandably, Indicates that the vehicle's charging and discharging power is not greater than its maximum charge / discharge power; in addition, in order to protect the life of the vehicle battery, the vehicle battery should not be over-discharged, that is .

[0026] Specifically, the energy storage air conditioning model is specifically expressed as: in, express The indoor temperature of the building at all times; express The outdoor temperature of the building at all times; Indicates that energy storage air conditioning Cooling / heating capacity at the moment, if is a positive value, Indicates that energy storage air conditioning The cooling capacity at the time is a negative value, Indicates that energy storage air conditioning Heating capacity at all times; It represents the equivalent thermal resistance of the building; represents the equivalent heat capacity of the building, represents the differential, where the cooling / heating capacity of the energy storage air conditioner Power consumption of energy storage air conditioner and thermal-electric conversion coefficient of energy storage air conditioner Relevant, namely: (1) When the set temperature Equal to room temperature When the energy storage air conditioner consumes power for: (2) When the outdoor temperature Less than the set temperature Refrigeration time for: After a period of time, the indoor temperature And energy storage air conditioning power consumption for: (3) When the outdoor temperature Greater than the set temperature Refrigeration time for: After a period of time, the indoor temperature And energy storage air conditioning power consumption for: in, Indicates the maximum power of the energy storage air conditioner. represents a natural constant, Represents the natural logarithm.

[0027] The embodiment of the present application adopts an equivalent thermal parameter model to describe the thermodynamic process of the energy storage air conditioner to construct an energy storage air conditioner model.

[0028] In an optional embodiment, the first energy scheduling model of the wind-solar-storage-charging-discharging integrated system is constructed based on the simulation model and a reinforcement learning algorithm, including: The energy scheduling problem of the simulation model is converted into a Markov decision problem, and the observation space, action space, reward function, strategy network and action value network of the first energy scheduling model are constructed.

[0029] Specifically, the observation space includes: time (e.g., month, day, hour, minute, second), outdoor temperature, indoor temperature, energy storage air conditioning power, park electricity consumption, wind power generation, photovoltaic power generation, vehicle battery capacity, number of vehicles, number of vehicles not met (i.e., less than The number of vehicles), vehicles that do not meet capacity (i.e. less than The sum of the battery capacities of vehicles with The number of vehicles), vehicle overflow capacity (i.e., greater than the sum of the vehicle battery capacities), electricity prices.

[0030] The action space includes: energy storage battery charging and discharging, vehicle charging and discharging amount, i.e., the total charging and discharging quota of the parking lot, and energy storage air conditioner set temperature.

[0031] The goal of the embodiment of this application is to minimize the electricity cost of the wind-solar-storage-charging-discharging integrated system (park) and maximize the benefits. Therefore, it only focuses on the electricity cost of the wind-solar-storage-charging-discharging integrated system, the income from selling electricity, and the compensation for the parking lot. Specifically, the reward function is: in, express The electricity selling price at the time, express The amount of electricity sold by the wind-solar-storage-charging-discharging integrated system at all times, express The electricity purchase price at the time; express The amount of electricity purchased by the wind-solar-storage-charging-discharging integrated system at all times; express Moment A penalty is imposed if the SOC of a vehicle does not meet travel demand when it leaves the parking lot.

[0032] In order to meet the needs of users, it is necessary to ensure that the SOC of each vehicle (electric vehicle) when leaving the parking lot is greater than the preset target SOC. , The preferred value is 0.8. The difference between the preset target SOC and the SOC when the vehicle leaves the parking lot is calculated. Moment Mileage compensation for vehicles , the formula is as follows: in, Indicates the SOC of the vehicle when it leaves the parking lot. Indicates the preset target SOC, It is the number of kilometers that the vehicle can travel per kilowatt-hour of electricity; is the vehicle battery capacity. It is The departure time of the vehicle.

[0033] Compensation based on mileage Combined with the taxi fare billing method of a certain city, the compensation amount for the user's insufficient power is calculated as a penalty. The taxi billing method adopts a segmented billing method, and combines the driving time and mileage for calculation. For the convenience of calculation, this application converts the driving time into a unified distance calculation based on the average driving speed of a certain city. The billing formula is as follows: in, for Moment The penalty for the SOC of a vehicle not meeting the travel demand when it leaves the parking lot is for Moment Mileage compensation for vehicles, For the starting price, The mileage price is is the time price, For the first long distance charge, For the second long distance charge, For the third long distance fee; is the starting distance for mileage, is the time starting distance, For the first long distance, The second longest distance, The third longest distance; The conversion factor representing time and distance may be preset or calculated based on the average speed of vehicles traveling in a city.

[0034] Based on this formula, the penalty for the vehicle's SOC not meeting travel demand when leaving the parking lot can be calculated.

[0035] In the embodiment of the present application, a policy network (whose parameters are ) and two action-value networks (whose parameters are and , unified use ), complete the training of the energy scheduling model.

[0036] Specifically, the loss function of the action value network is: in, Representing Action-Value Network The loss function is represents the computational expectation, represents the state distribution and action distribution sampled by the previous agent in the environment exploration, Representing the target action value network The output value, Indicates status Adopt an action Instant rewards, represents the discount rate, represents the state transition probability, Indicates status The value function of: express The state of the moment; express The action of the moment, express The state of the moment.

[0037] This value function can be obtained through the target action value network To estimate, each The network corresponds to a Network, specifically, ;in, is the maximum entropy objective The temperature parameters, is a strategic network, Representation Policy Network In Status Output action probability; express The state of the moment, express Finally, the policy network is learned by minimizing the expected KL divergence.

[0038] Specifically, the loss function of the strategy network is: in, Representation Policy Network The loss function is represents the computational expectation, represents the input noise, The noise of the input It follows a normal distribution. Indicates action resampling.

[0039] Specifically, the temperature parameter The loss function is: in, Indicates the preset minimum expected entropy.

[0040] In an optional embodiment, the second energy scheduling model is specifically expressed as: In the formula, Is an identifier indicating whether the vehicle has arrived at the parking lot. When the car arrives at the parking lot, =1, when If the car does not arrive at the parking lot, ; Indicates the preset target SOC; Indicates the time when the vehicle leaves the parking lot. Indicates the current moment; Indicates The maximum charge capacity of the vehicle, Indicates the charging efficiency of the vehicle, Indicates The maximum power of the vehicle; Indicates The SOC after vehicle dispatching, ; Indicates The current SOC of the vehicle; Indicates The charging capacity of the vehicle, Indicates The discharge capacity of the vehicle; Represents the discharge efficiency of the vehicle, where the charging and discharging power needs to meet the following constraints: In the formula, is the total charging and discharging quota allocated by the agent to the parking lot.

[0041] In the embodiment of the present application, the intelligent agent trained by reinforcement learning determines the total charge and discharge quota of the parking lot, and intelligently dispatches multiple energy storages (energy storage batteries, vehicles, energy storage air conditioners) to store the energy generated by photovoltaic and wind power. Then, according to the second energy dispatch model of the vehicle, the total charge and discharge quota is subdivided into each vehicle, so that the SOC of all vehicles when leaving the parking lot meets , further realizing the energy scheduling optimization of vehicles.

[0042] The embodiment of the present application uses a reinforcement learning algorithm, which eliminates the need for repeated and complex modeling of the wind-solar storage-charging-discharging integrated system. At the same time, the second energy scheduling model constructed in combination with the traditional optimization scheme can solve the problem of power distribution of electric vehicles, avoid the problem of too large action space due to the large number of electric vehicles, greatly reduce the computing power required for training, and make energy scheduling more comprehensive and accurate. The embodiment of the present invention combines traditional optimization schemes and artificial intelligence schemes to propose a new energy scheduling method, which not only simplifies the modeling process, but also saves the computing power required to solve the algorithm, and also makes energy scheduling more comprehensive and accurate.

[0043] See also Figure 2 , Figure 2 1 is a structural block diagram of an energy dispatching device 10 for a wind-solar storage-charging-discharging integrated system provided in an embodiment of the present application. The energy dispatching device 10 for the wind-solar storage-charging-discharging integrated system includes: The first construction module 11 is used to construct a simulation model of a wind-solar-storage-charging-discharging integrated system; wherein the simulation model includes: a photovoltaic power generation model, a wind power generation model, an energy storage battery model, a vehicle model and an energy storage air conditioning model; A second construction module 12 is used to construct a first energy scheduling model of the wind-solar-storage-charging-discharging integrated system according to the simulation model and based on a reinforcement learning algorithm; The third construction module 13 is used to construct a second energy scheduling model for the vehicle based on the moment when the vehicle leaves the parking lot and with the goal of minimizing the difference between the preset target SOC and the SOC after the vehicle is scheduled; An energy scheduling module is used to jointly train the first energy scheduling model and the second energy scheduling model to perform energy scheduling on the wind-solar storage-charging-discharging integrated system according to the trained first energy scheduling model and the second energy scheduling model.

[0044] Optionally, the photovoltaic power generation model is specifically expressed as: in, Represents the power generated by the photovoltaic panel. represents solar radiation, represents the reference radiation, Represents the rated power of the solar cell array, represents the temperature coefficient, represents the reference temperature, represents the temperature of the solar cell array, , Indicates the ambient temperature of the photovoltaic panel. Represents the rated operating temperature of the solar cell array, Indicates the air temperature when the solar array is rated to operate. Indicates the light radiation on the surface of the solar cell array during rated operation; The wind power generation model is specifically expressed as: in, Represents the output power of the wind turbine generator set. Indicates the rated power of the wind turbine generator set; Indicates wind speed; Indicates the cut-in wind speed, Indicates the rated wind speed, represents the cut-out wind speed; , , They are: The energy storage battery model is specifically expressed as: in, Indicates that the energy storage battery is The amount of electricity stored at any time, Indicates that the energy storage battery is The amount of electricity stored at any time, Indicates that the energy storage battery is The charging capacity of the time period, Indicates the charging efficiency of the energy storage battery. Indicates that the energy storage battery is The discharge capacity of the time period, represents the discharge efficiency of the energy storage battery; the energy storage battery model satisfies the following constraints: in, and is a binary variable, when Indicates that the energy storage battery is Charging during the period, Indicates that the energy storage battery is Discharge during the period, Indicates that the energy storage battery is The charging capacity of the time period, Indicates the maximum charging rate of the energy storage battery, Indicates that the energy storage battery is The discharge capacity of the time period, Indicates the maximum discharge rate of the energy storage battery, Indicates the maximum battery capacity of the energy storage battery; The vehicle model is specifically expressed as: in, Indicates the current SOC of the vehicle. represents the initial SOC of the vehicle, Indicates that the vehicle is in the time slot If the charging / discharging power at is a positive value, Indicates that the vehicle is in the time slot If the charging power at is a negative value, Indicates that the vehicle is in the time slot The discharge power at Indicates the length of the time slot, Indicates the vehicle battery capacity, represents a set of available charging time slots; wherein the vehicle model satisfies the following constraints: in, Indicates the maximum chargeable power of the vehicle, Indicates the maximum dischargeable power of the vehicle; The energy storage air conditioning model is specifically expressed as: in, express The indoor temperature of the building at all times; express The outdoor temperature of the building at all times; Indicates that energy storage air conditioning Cooling / heating capacity at the moment, if is a positive value, Indicates that energy storage air conditioning The cooling capacity at the time is a negative value, Indicates that energy storage air conditioning Heating capacity at all times; It represents the equivalent thermal resistance of the building; represents the equivalent heat capacity of the building, represents the differential, where the cooling / heating capacity of the energy storage air conditioner Power consumption of energy storage air conditioner and thermal-electric conversion coefficient of energy storage air conditioner Relevant, namely: (1) When the set temperature Equal to room temperature When the energy storage air conditioner consumes power for: (2) When the outdoor temperature Less than the set temperature Refrigeration time for: After a period of time, the indoor temperature And energy storage air conditioning power consumption for: (3) When the outdoor temperature Greater than the set temperature Refrigeration time for: After a period of time, the indoor temperature And energy storage air conditioning power consumption for: in, Indicates the maximum power of the energy storage air conditioner. represents a natural constant, Represents the natural logarithm.

[0045] Optionally, the first energy scheduling model of the wind-solar-storage-charging-discharging integrated system is constructed based on the simulation model and a reinforcement learning algorithm, including: The energy scheduling problem of the simulation model is converted into a Markov decision problem, and the observation space, action space, reward function, strategy network and action value network of the first energy scheduling model are constructed.

[0046] Optionally, the reward function is: in, express The electricity selling price at the time, express The amount of electricity sold by the wind-solar-storage-charging-discharging integrated system at all times, express The electricity purchase price at the time; express The amount of electricity purchased by the wind-solar-storage-charging-discharging integrated system at all times; express Moment A penalty is imposed if the SOC of a vehicle does not meet travel demand when it leaves the parking lot.

[0047] Optionally, the loss function of the action value network is: in, Representing Action-Value Network The loss function is represents the computational expectation, represents the state distribution and action distribution sampled by the previous agent in the environment exploration, Representing the target action value network The output value, Indicates status Adopt an action Instant rewards, represents the discount rate, represents the state transition probability, Indicates status The value function of: express The state of the moment; express The action of the moment, express The state of the moment; in, is the maximum entropy objective The temperature parameters, is a strategic network, Representation Policy Network In Status Output action probability; express The state of the moment, express Momentary action; The loss function of the policy network is: in, Representation Policy Network The loss function is represents the computational expectation, represents the input noise, The noise of the input It follows a normal distribution. Indicates action resampling of; The temperature parameter The loss function is: in, Indicates the preset minimum expected entropy.

[0048] Optionally, the second energy scheduling model is specifically expressed as: In the formula, Is an identifier indicating whether the vehicle has arrived at the parking lot. When the car arrives at the parking lot, =1, when If the car does not arrive at the parking lot, ; Indicates the preset target SOC; Indicates the time when the vehicle leaves the parking lot. Indicates the current moment; Indicates The maximum charge capacity of the vehicle, Indicates the charging efficiency of the vehicle, Indicates The maximum power of the vehicle; Indicates The SOC after vehicle dispatching, ; Indicates The current SOC of the vehicle; Indicates The charging capacity of the vehicle, Indicates The discharge capacity of the vehicle; Represents the discharge efficiency of the vehicle, where the charging and discharging power needs to meet the following constraints: In the formula, is the total charging and discharging quota allocated by the agent to the parking lot.

[0049] It is worth noting that the working process of each module in the energy dispatching device 10 of the wind-solar-storage-charging-discharging integrated system described in the embodiment of the present application can refer to the working process of the energy dispatching method of the wind-solar-storage-charging-discharging integrated system described in the above embodiment, and will not be repeated here.

[0050] The energy dispatching device 10 of a wind-solar storage-charging-discharging integrated system provided in the embodiment of the present application constructs a simulation model of the wind-solar storage-charging-discharging integrated system by considering photovoltaic power generation, wind power generation, energy storage batteries, vehicles and energy storage air conditioners. According to the simulation model, based on the reinforcement learning algorithm, the first energy dispatching model of the wind-solar storage-charging-discharging integrated system is constructed. At the same time, combined with the moment when the vehicle leaves the parking lot, the second energy dispatching model of the vehicle is constructed with the goal of minimizing the difference between the preset target SOC and the SOC after the vehicle is dispatched; finally, the first energy dispatching model and the second energy dispatching model are jointly trained to dispatch the wind-solar storage-charging-discharging integrated system according to the trained first energy dispatching model and the second energy dispatching model. It can be seen that the embodiment of the present application does not need to repeatedly and complexly model the wind-solar storage-charging-discharging integrated system through the reinforcement learning algorithm. At the same time, the second energy dispatching model constructed in combination with the traditional optimization scheme can solve the problem of power distribution of electric vehicles, avoid the problem of too large action space caused by a large number of electric vehicles, greatly reduce the computing power required for training, and make energy dispatch more comprehensive and accurate. The embodiment of the present invention combines traditional optimization solutions with artificial intelligence solutions to propose a new energy scheduling method, which not only simplifies the modeling process, but also saves the computing power required for solving the algorithm, and also makes energy scheduling more comprehensive and accurate.

[0051] In addition, an embodiment of the present application further provides a computer-readable storage medium, which includes a stored computer program; wherein, when the computer program is running, it controls the device where the computer-readable storage medium is located to execute the energy scheduling method of the wind-solar-storage-charging-discharging integrated system as described in any of the above embodiments.

[0052] In addition, an embodiment of the present application also provides a computer program product, including a computer program / instruction, which, when executed by a processor, implements the energy scheduling method of the wind-solar-storage-charging-discharging integrated system as described in any of the above embodiments.

[0053] See also Figure 3 , Figure 3 It is a structural block diagram of an energy dispatching device 20 of a wind-solar storage-charging-discharging integrated system provided in an embodiment of the present application. The energy dispatching device 20 of the wind-solar storage-charging-discharging integrated system includes: a processor 21, a memory 22, and a computer program stored in the memory 22 and executable on the processor 21. When the processor 21 executes the computer program, the steps in the above-mentioned energy dispatching method embodiment of the wind-solar storage-charging-discharging integrated system are implemented. Alternatively, when the processor 21 executes the computer program, the functions of each module / unit in the above-mentioned device embodiments are implemented.

[0054] Exemplarily, the computer program may be divided into one or more modules / units, which are stored in the memory 22 and executed by the processor 21 to complete the present application. The one or more modules / units may be a series of computer program instruction segments capable of completing specific functions, which are used to describe the execution process of the computer program in the energy dispatching device 20 of the wind-solar-storage-charging-discharging integrated system.

[0055] The energy dispatching device 20 of the wind-solar-storage-charging-discharging integrated system may include, but is not limited to, a processor 21 and a memory 22. Those skilled in the art will understand that the schematic diagram is only an example of the energy dispatching device 20 of the wind-solar-storage-charging-discharging integrated system, and does not constitute a limitation on the energy dispatching device 20 of the wind-solar-storage-charging-discharging integrated system, and may include more or fewer components than shown in the figure, or a combination of certain components, or different components. For example, the energy dispatching device 20 of the wind-solar-storage-charging-discharging integrated system may also include input and output devices, network access devices, buses, etc.

[0056] The processor 21 may be a central processing unit (CPU), or other general-purpose processors, digital signal processors (DSP), application-specific integrated circuits (ASIC), field-programmable gate arrays (FPGA) or other programmable logic devices, discrete gates or transistor logic devices, discrete hardware components, etc. A general-purpose processor may be a microprocessor or any conventional processor, etc. The processor 21 is the control center of the energy dispatching device 20 of the wind-solar-storage-charging-discharging integrated system, and uses various interfaces and lines to connect various parts of the energy dispatching device 20 of the entire wind-solar-storage-charging-discharging integrated system.

[0057] The memory 22 can be used to store the computer program and / or module. The processor 21 realizes various functions of the energy dispatching device 20 of the wind-solar storage-charging-discharging integrated system by running or executing the computer program and / or module stored in the memory 22 and calling the data stored in the memory 22. The memory 22 can mainly include a program storage area and a data storage area, wherein the program storage area can store an operating system, an application required for at least one function (such as a sound playback function, an image playback function, etc.); the data storage area can store data created according to the use of the mobile phone (such as audio data, a phone book, etc.). In addition, the memory 22 can include a high-speed random access memory, and can also include a non-volatile memory, such as a hard disk, a memory, a plug-in hard disk, a smart memory card (Smart Media Card, SMC), a secure digital (Secure Digital, SD) card, a flash card (Flash Card), at least one disk storage device, a flash memory device, or other volatile solid-state storage devices.

[0058] Wherein, if the module / unit integrated in the energy dispatching device 20 of the wind-solar storage-charging-discharging integrated system is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the present application implements all or part of the processes in the above-mentioned embodiment method, and can also be completed by instructing the relevant hardware through a computer program. The computer program can be stored in a computer-readable storage medium, and the computer program can implement the steps of the above-mentioned various method embodiments when executed by the processor 21. Wherein, the computer program includes computer program code, and the computer program code can be in source code form, object code form, executable file or some intermediate form. The computer-readable medium may include: any entity or device capable of carrying the computer program code, recording medium, U disk, mobile hard disk, disk, optical disk, computer memory, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), electric carrier signal, telecommunication signal and software distribution medium, etc.

[0059] It should be noted that the device embodiments described above are merely schematic, wherein the units described as separate components may or may not be physically separated, and the components displayed as units may or may not be physical units, that is, they may be located in one place, or they may be distributed on multiple network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the scheme of this embodiment. In addition, in the accompanying drawings of the device embodiments provided by the present application, the connection relationship between the modules indicates that there is a communication connection between them, which may be specifically implemented as one or more communication buses or signal lines. A person of ordinary skill in the art can understand and implement it without paying any creative work.

[0060] The above is a preferred embodiment of the present application. It should be pointed out that for ordinary technicians in this technical field, several improvements and modifications can be made without departing from the principles of the present application. These improvements and modifications are also considered to be within the scope of protection of the present application.

Claims

1. An energy dispatching method for a wind-solar-storage-charging-discharging integrated system, characterized in that: include: Construct a simulation model of a wind-solar-storage-charging-discharging integrated system; wherein the simulation model includes: a photovoltaic power generation model, a wind power generation model, an energy storage battery model, a vehicle model, and an energy storage air conditioning model; According to the simulation model and based on the reinforcement learning algorithm, a first energy scheduling model of the wind-solar-storage-charging-discharging integrated system is constructed; Combined with the moment when the vehicle leaves the parking lot, the second energy scheduling model of the vehicle is constructed with the goal of minimizing the difference between the preset target SOC and the SOC after the vehicle is scheduled; The first energy scheduling model and the second energy scheduling model are jointly trained to perform energy scheduling on the wind-solar-storage-charging-discharging integrated system according to the trained first energy scheduling model and the second energy scheduling model.

2. The energy dispatching method for the wind-solar-storage-charging-discharging integrated system according to claim 1, wherein the photovoltaic power generation model is specifically expressed as: in, Represents the power generated by the photovoltaic panel. represents solar radiation, represents the reference radiation, Represents the rated power of the solar cell array, represents the temperature coefficient, represents the reference temperature, represents the temperature of the solar cell array, , Indicates the ambient temperature of the photovoltaic panel. Represents the rated operating temperature of the solar cell array, Indicates the air temperature when the solar array is rated to operate. Indicates the light radiation on the surface of the solar cell array during rated operation; The wind power generation model is specifically expressed as: in, Represents the output power of the wind turbine generator set. Indicates the rated power of the wind turbine generator set; Indicates wind speed; Indicates the cut-in wind speed, Indicates the rated wind speed, represents the cut-out wind speed; , , They are: The energy storage battery model is specifically expressed as: in, Indicates that the energy storage battery is The amount of electricity stored at any time, Indicates that the energy storage battery is The amount of electricity stored at any time, Indicates that the energy storage battery is The charging capacity of the time period, Indicates the charging efficiency of the energy storage battery. Indicates that the energy storage battery is The discharge capacity of the time period, represents the discharge efficiency of the energy storage battery; the energy storage battery model satisfies the following constraints: in, and is a binary variable, when Indicates that the energy storage battery is Charging during the period, Indicates that the energy storage battery is Discharge during the period, Indicates that the energy storage battery is The charging capacity of the time period, Indicates the maximum charging rate of the energy storage battery, Indicates that the energy storage battery is The discharge capacity of the time period, Indicates the maximum discharge rate of the energy storage battery, Indicates the maximum battery capacity of the energy storage battery; The vehicle model is specifically expressed as: in, Indicates the current SOC of the vehicle. represents the initial SOC of the vehicle, Indicates that the vehicle is in the time slot If the charging / discharging power at is a positive value, Indicates that the vehicle is in the time slot If the charging power at is a negative value, Indicates that the vehicle is in the time slot The discharge power at Indicates the length of the time slot, Indicates the vehicle battery capacity, represents a set of available charging time slots; wherein the vehicle model satisfies the following constraints: in, Indicates the maximum chargeable power of the vehicle, Indicates the maximum dischargeable power of the vehicle; The energy storage air conditioning model is specifically expressed as: in, express The indoor temperature of the building at all times; express The outdoor temperature of the building at all times; Indicates that energy storage air conditioning Cooling / heating capacity at the moment, if is a positive value, Indicates that energy storage air conditioning The cooling capacity at the time is a negative value, Indicates that energy storage air conditioning Heating capacity at all times; It represents the equivalent thermal resistance of the building; represents the equivalent heat capacity of the building, represents the differential, where the cooling / heating capacity of the energy storage air conditioner Power consumption of energy storage air conditioner and thermal-electric conversion coefficient of energy storage air conditioner Relevant, namely: (1) When the set temperature Equal to room temperature When the energy storage air conditioner consumes power for: (2) When the outdoor temperature Less than the set temperature Refrigeration time for: After a period of time, the indoor temperature And energy storage air conditioning power consumption for: (3) When the outdoor temperature Greater than the set temperature Refrigeration time for: After a period of time, the indoor temperature And energy storage air conditioning power consumption for: in, Indicates the maximum power of the energy storage air conditioner. represents a natural constant, Represents the natural logarithm.

3. The energy dispatching method of the wind-solar-storage-charging-discharging integrated system according to claim 1, characterized in that: According to the simulation model, based on the reinforcement learning algorithm, a first energy scheduling model of the wind-solar storage-charging-discharging integrated system is constructed, including: The energy scheduling problem of the simulation model is converted into a Markov decision problem, and the observation space, action space, reward function, strategy network and action value network of the first energy scheduling model are constructed.

4. The energy dispatching method of the wind-solar-storage-charging-discharging integrated system according to claim 3 is characterized in that: The reward function is: in, express The electricity selling price at the time, express The amount of electricity sold by the wind-solar-storage-charging-discharging integrated system at all times, express The electricity purchase price at the time; express The amount of electricity purchased by the wind-solar-storage-charging-discharging integrated system at all times; express Moment A penalty is imposed if the SOC of a vehicle does not meet travel demand when it leaves the parking lot.

5. The energy dispatching method of the wind-solar-storage-charging-discharging integrated system according to claim 4, characterized in that: The loss function of the action value network is: in, Representing Action-Value Network The loss function is represents the computational expectation, represents the state distribution and action distribution sampled by the previous agent in the environment exploration, Representing the target action value network The output value, Indicates status Adopt an action Instant rewards, represents the discount rate, represents the state transition probability, Indicates status The value function of: express The state of the moment; express The action of the moment, express The state of the moment; in, is the maximum entropy objective The temperature parameters, is a strategic network, Representation Policy Network In Status Output action probability; express The state of the moment, express Momentary action; The loss function of the policy network is: in, Representation Policy Network The loss function is represents the computational expectation, represents the input noise, The noise of the input It follows a normal distribution. Indicates action resampling of; The temperature parameter The loss function is: in, Indicates the preset minimum expected entropy.

6. The energy dispatching method of the wind-solar-storage-charging-discharging integrated system according to claim 1, characterized in that: The second energy scheduling model is specifically expressed as: In the formula, Is an identifier indicating whether the vehicle has arrived at the parking lot. When the car arrives at the parking lot, =1, when If the car does not arrive at the parking lot, ; Indicates the preset target SOC; Indicates the time when the vehicle leaves the parking lot. Indicates the current moment; Indicates The maximum charge capacity of the vehicle, Indicates the charging efficiency of the vehicle, Indicates The maximum power of the vehicle; Indicates The SOC after vehicle dispatching, ; Indicates The current SOC of the vehicle; Indicates The charging capacity of the vehicle, Indicates The discharge capacity of the vehicle; Represents the discharge efficiency of the vehicle, where the charging and discharging power needs to meet the following constraints: In the formula, is the total charging and discharging quota allocated by the agent to the parking lot.

7. An energy dispatching device for a wind-solar-storage-charging-discharging integrated system, characterized in that: include: The first construction module is used to construct a simulation model of a wind-solar-storage-charging-discharging integrated system; wherein the simulation model includes: a photovoltaic power generation model, a wind power generation model, an energy storage battery model, a vehicle model and an energy storage air conditioning model; A second construction module is used to construct a first energy scheduling model of the wind-solar-storage-charging-discharging integrated system according to the simulation model and based on a reinforcement learning algorithm; The third building module is used to build a second energy scheduling model for the vehicle based on the moment when the vehicle leaves the parking lot and takes the minimum difference between the preset target SOC and the SOC after the vehicle is scheduled as the goal; An energy scheduling module is used to jointly train the first energy scheduling model and the second energy scheduling model to perform energy scheduling on the wind-solar storage-charging-discharging integrated system according to the trained first energy scheduling model and the second energy scheduling model.

8. An energy dispatching device for a wind-solar-storage-charging-discharging integrated system, characterized in that: It includes a processor, a memory, and a computer program stored in the memory and configured to be executed by the processor. When the processor executes the computer program, it implements the energy scheduling method of the wind-solar-storage-charging-discharging integrated system as described in any one of claims 1 to 6.

9. A computer-readable storage medium, characterized in that: The computer-readable storage medium includes a stored computer program; wherein, when the computer program is running, it controls the device where the computer-readable storage medium is located to execute the energy scheduling method of the wind-solar-storage-charging-discharging integrated system as described in any one of claims 1 to 6.

10. A computer program product, characterized in that It includes a computer program / instruction, which, when executed by a processor, implements the energy scheduling method for the wind-solar-storage-charging-discharging integrated system as described in any one of claims 1 to 6.

Images