Incentive price determination method, device and medium for distributed resource aggregation

Abstract

The present application discloses an incentive price determination method, device and medium for distributed resource aggregation, the method comprising: obtaining power consumption data corresponding to each distributed resource in a current training period, and training a deep deterministic policy gradient model based on the power consumption data; obtaining a first power consumption change corresponding to each distributed resource in the current period; inputting the first power consumption change corresponding to each distributed resource in the current period into the trained deep deterministic policy gradient model to obtain an incentive price corresponding to each distributed resource in the current period, and being able to accurately determine the incentive price of bidirectional distributed resources based on the power consumption of the distributed resources, providing an accurate reward mechanism for each distributed resource, ensuring the effective scheduling of distributed aggregation resources, and attracting more distributed resources from the demand side to join the distributed aggregation system.

Description

Technical Field

[0001] The present invention relates to the technical field of power demand side control, and in particular to a method, device, medium and equipment for determining an incentive price for distributed resource aggregation. Background Art

[0002] With the continuous development and utilization of power resources, load aggregators are becoming the most important participants in the power demand response market. Load aggregators can integrate the demand response resources of a large number of dispersed power users and participate in the demand-side response of the power market on behalf of these users. Through the demand response management platform, they gather users with adjustable load capacity to form large-scale response resources, and then invest them in the power demand response market. This not only provides small and medium-sized load users with the opportunity to participate in market regulation, but also helps the power market operate more efficiently and competitively.

[0003] Load aggregators wholesale a large amount of electricity from the power system, and power users purchase electricity from load aggregators. Load aggregators are not only agents of user demand response resources, but also retailers or power sellers that meet user electricity needs. Load aggregators sign contracts with power grid companies on behalf of users, stipulating the time and capacity of load reduction and load transfer. Users who reduce electricity consumption or change electricity consumption time within the time period specified in the contract can obtain incentive compensation issued by load aggregators. However, load aggregators cannot send incentive compensation blindly. While ensuring the profits of load aggregators, they should issue the largest reward compensation to power users, so that more power users will join the management of load aggregators.

[0004] The existing calculation method of the reward compensation issued by the load aggregator to the power user is obtained through optimization methods such as mixed integer linear programming (MILP) and particle swarm optimization (PSO). These methods can effectively solve the allocation problem of traditional resources. Traditional resources refer to power users who obtain electricity from the load aggregator in a unidirectional manner. However, with the development of new energy, many new controllable demand-side resources have emerged, which are bidirectional resources that can both receive electricity from the power grid and provide electricity, such as residential energy storage systems, electric vehicles with V2G (vehicle to grid) functions, and mobile battery power stations. Aggregates that include these new controllable demand-side resources are also called distributed aggregations. The unidirectional resources or bidirectional resources included in distributed aggregations are all called distributed resources. The power users targeted by the existing load aggregator reward compensation calculation method do not include distributed resources, that is, the existing load aggregator reward calculation method cannot calculate the reward compensation for distributed resources. Summary of the invention

[0005] In view of this, the present invention provides a method, device, medium and equipment for determining an incentive price for distributed resource aggregation, the main purpose of which is to solve the problem that the current load aggregator reward calculation method cannot calculate the reward compensation of distributed resources.

[0006] According to one aspect of the present application, a method for determining an incentive price for distributed resource aggregation is provided, the method comprising:

[0007] Obtaining power consumption data corresponding to each distributed resource in the current training period, and training a deep deterministic policy gradient model based on the power consumption data;

[0008] Obtaining a first electric energy consumption corresponding to a current time period and a second electric energy consumption corresponding to a previous time period of each distributed resource;

[0009] Based on the first electric energy consumption and the second electric energy consumption, calculating a first electric energy consumption change corresponding to each distributed resource in a current time period;

[0010] The first electric energy consumption change corresponding to each distributed resource in the current time period is input into the trained deep deterministic policy gradient model to obtain the incentive price corresponding to each distributed resource in the current time period.

[0011] Optionally, the training of a deep deterministic policy gradient model based on the power consumption data includes:

[0012] Constructing a current policy network, a target policy network, a current evaluation network, a target evaluation network and a replay buffer pool, and obtaining a third power consumption amount corresponding to a current training period and a fourth power consumption amount corresponding to a plurality of periods before the current training period for each distributed resource;

[0013] Based on the current strategy network, the third electric energy consumption corresponding to each distributed resource in the current training period, and the fourth electric energy consumption corresponding to each period before the current training period, a plurality of training samples are obtained, and the plurality of training samples are stored in the replay buffer pool, wherein the training samples include the second electric energy consumption change corresponding to each distributed resource in the current training period, the incentive price, the third electric energy consumption change corresponding to the next period, and the reward corresponding to the aggregation system in the current training period;

[0014] Obtain any training sample from the replay buffer, input the second electric energy consumption change and the incentive price corresponding to each distributed resource in the any training sample in the current training period into the current evaluation network to obtain a current evaluation value, input the third electric energy consumption change corresponding to each distributed resource in the next period into the target strategy model to obtain the incentive price corresponding to each distributed resource in the next period, input the third electric energy consumption change and the incentive price corresponding to each distributed resource in the next period into the target evaluation network to obtain a target evaluation value;

[0015] Based on the current evaluation value, a first policy parameter in the current policy network is updated, and based on the current evaluation value and the target evaluation value, a first evaluation parameter in the current evaluation network is updated;

[0016] Based on the updated first policy parameter, the current policy network and the target policy network are updated, and based on the updated first evaluation parameter, the current evaluation network and the target evaluation network are updated;

[0017] Other training samples are obtained from the replay buffer pool, and the policy parameters and evaluation parameters are updated again based on the updated current policy network, target policy network, current evaluation network, and target evaluation network until the number of iterations is reached to obtain a trained deep deterministic policy gradient model.

[0018] Optionally, the obtaining of multiple training samples based on the current strategy network, the third power consumption corresponding to each distributed resource in the current training period, and the fourth power consumption corresponding to each period before the current training period includes:

[0019] Calculate a second electric energy consumption change corresponding to each distributed resource in the current training period based on a third electric energy consumption corresponding to each distributed resource in the current training period and a fourth electric energy consumption corresponding to a previous period of the current training period;

[0020] Input the second electric energy consumption change corresponding to each distributed resource in the current training period into the current strategy network, obtain the action determined by the current strategy network, the action is the incentive price corresponding to each distributed resource in the current training period, and calculate the reward corresponding to the aggregation system where the distributed resource is located in the current training period according to the incentive price corresponding to each distributed resource in the current training period, the third electric energy consumption, the second electric energy consumption change and the fourth electric energy consumption corresponding to multiple periods before the current training period;

[0021] Outputting the incentive price corresponding to each distributed resource in the current training period to the control terminal of each distributed resource, and obtaining the fifth electric energy consumption corresponding to each distributed resource in the next period;

[0022] Calculate a third electric energy consumption change corresponding to each distributed resource in the next training period based on the third electric energy consumption corresponding to each distributed resource in the current training period and the fifth electric energy consumption corresponding to the next training period;

[0023] The second power consumption change corresponding to each distributed resource in the current training period, the incentive price, the third power consumption change corresponding to the next period and the reward corresponding to the aggregation system in the current training period are taken as a training sample.

[0024] Optionally, after updating the first parameter in the current policy network based on the current evaluation value and updating the first evaluation parameter in the current evaluation network based on the current evaluation value and the target evaluation value, the method further comprises:

[0025] The second electric energy consumption change corresponding to each distributed resource in the current training period is used as the state of the updated current policy network input to obtain a new incentive price corresponding to the current training period, and based on the new incentive price corresponding to each distributed resource in the current training period, a new reward corresponding to the aggregation system where the distributed resource is located in the current training period is calculated;

[0026] Outputting the new incentive price corresponding to the current training period to the control terminal of each distributed resource, and obtaining the new fifth electric energy consumption corresponding to each distributed resource in the next period;

[0027] Calculate a new third electric energy consumption change corresponding to each distributed resource in the next time period based on the third electric energy consumption corresponding to each distributed resource in the current training time period and the new fifth electric energy consumption corresponding to the next time period;

[0028] The second energy consumption change corresponding to each distributed resource in the current training period, the new incentive price, the new third energy consumption change corresponding to the next period and the reward corresponding to the aggregation system in the current training period are taken as a new training sample.

[0029] Optionally, updating the target policy network based on the updated first policy parameter includes:

[0030] Based on the updated first strategy parameters, obtaining updated second strategy parameters;

[0031] Based on the updated second policy parameters, updating the target policy network;

[0032] Based on the updated first evaluation parameter, the target evaluation network is updated, including:

[0033] Based on the updated first evaluation parameter, obtaining an updated second evaluation parameter;

[0034] The target evaluation network is updated based on the updated second evaluation parameter.

[0035] Optionally, the current evaluation value is used as a loss function, and a gradient ascent method is adopted to update the first policy parameter in the current policy network.

[0036] Optionally, the following method is used to calculate the reward corresponding to the aggregation system where the distributed resources are located in the current training period:

[0037]

[0038]

[0039] μ1+μ2+μ3=1

[0040]

[0041]

[0042]

[0043]

[0044] 1≤j≤n uni

[0045]

[0046]

[0047] 1≤k≤n bi

[0048] Among them, Obj is the reward, μ1, μ2, μ3 are weight coefficients, P Agg is the total operating cost of the distributed aggregator, P uni is the total profit of all unidirectional distributed resources, P bi is the total profit of all bidirectional distributed resources, Δp ref (t) is the amount of electricity that the distributed aggregator obtains from the power company per hour, n dr is the number of distributed resources, is the power consumption of distributed resource n in period t, n uni is the number of unidirectional distributed resources, P uni (j) is the profit of distributed resource j, nbi is the number of bidirectional distributed resources, P bi (k) is the profit of distributed resource k, is the power consumption of distributed resource i in period t, and are the incentive price and energy consumption change of distributed resource i in period t, C(t) is the scheduling cost of each distributed resource in period t, and are the incentive price and energy consumption change of distributed resource j in period t, ξ t is the elasticity coefficient of the sensitivity of power consumption changes to rewards, is the energy consumption of distributed resource j in period t, α1 and α2 are coefficients, and are the incentive price and energy consumption change of distributed resource k in period t, is the power consumption of distributed resource k in period t, η k and γ k are the line loss and efficiency of charge / discharge respectively, and β1 and β are coefficients.

[0049] According to another aspect of the present application, there is provided an incentive price determination device for distributed resource aggregation, comprising:

[0050] A model training module, used to obtain the power consumption data corresponding to each distributed resource in the current training period, and train a deep deterministic policy gradient model based on the power consumption data;

[0051] An electric energy consumption acquisition module, used to acquire a first electric energy consumption corresponding to a current time period and a second electric energy consumption corresponding to a previous time period of each distributed resource;

[0052] An electric energy consumption change acquisition module, configured to calculate a first electric energy consumption change corresponding to each distributed resource in a current period based on the first electric energy consumption and the second electric energy consumption;

[0053] An incentive price acquisition module is used to input the first electric energy consumption change corresponding to each distributed resource in the current time period into the trained deep deterministic policy gradient model to obtain the incentive price corresponding to each distributed resource in the current time period.

[0054] Optionally, the model training module is also used to:

[0055] Constructing a current policy network, a target policy network, a current evaluation network, a target evaluation network and a replay buffer pool, and obtaining a third power consumption amount corresponding to a current training period and a fourth power consumption amount corresponding to a plurality of periods before the current training period for each distributed resource;

[0056] Based on the current strategy network, the third electric energy consumption corresponding to each distributed resource in the current training period, and the fourth electric energy consumption corresponding to each period before the current training period, a plurality of training samples are obtained, and the plurality of training samples are stored in the replay buffer pool, wherein the training samples include the second electric energy consumption change corresponding to each distributed resource in the current training period, the incentive price, the third electric energy consumption change corresponding to the next period, and the reward corresponding to the aggregation system in the current training period;

[0057] Obtain any training sample from the replay buffer, input the second electric energy consumption change and the incentive price corresponding to each distributed resource in the any training sample in the current training period into the current evaluation network to obtain a current evaluation value, input the third electric energy consumption change corresponding to each distributed resource in the next period into the target strategy model to obtain the incentive price corresponding to each distributed resource in the next period, input the third electric energy consumption change and the incentive price corresponding to each distributed resource in the next period into the target evaluation network to obtain a target evaluation value;

[0058] Based on the current evaluation value, a first policy parameter in the current policy network is updated, and based on the current evaluation value and the target evaluation value, a first evaluation parameter in the current evaluation network is updated;

[0059] Based on the updated first policy parameter, the current policy network and the target policy network are updated, and based on the updated first evaluation parameter, the current evaluation network and the target evaluation network are updated;

[0060] Other training samples are obtained from the replay buffer pool, and the policy parameters and evaluation parameters are updated again based on the updated current policy network, target policy network, current evaluation network, and target evaluation network until the number of iterations is reached to obtain a trained deep deterministic policy gradient model.

[0061] Optionally, the obtaining of multiple training samples based on the current strategy network, the third power consumption corresponding to each distributed resource in the current training period, and the fourth power consumption corresponding to each period before the current training period includes:

[0062] Calculate a second electric energy consumption change corresponding to each distributed resource in the current training period based on a third electric energy consumption corresponding to each distributed resource in the current training period and a fourth electric energy consumption corresponding to a previous period of the current training period;

[0063] Input the second electric energy consumption change corresponding to each distributed resource in the current training period into the current strategy network, obtain the action determined by the current strategy network, the action is the incentive price corresponding to each distributed resource in the current training period, and calculate the reward corresponding to the aggregation system where the distributed resource is located in the current training period according to the incentive price corresponding to each distributed resource in the current training period, the third electric energy consumption, the second electric energy consumption change and the fourth electric energy consumption corresponding to multiple periods before the current training period;

[0064] Outputting the incentive price corresponding to each distributed resource in the current training period to the control terminal of each distributed resource, and obtaining the fifth electric energy consumption corresponding to each distributed resource in the next period;

[0065] Calculate a third electric energy consumption change corresponding to each distributed resource in the next training period based on the third electric energy consumption corresponding to each distributed resource in the current training period and the fifth electric energy consumption corresponding to the next training period;

[0066] The second power consumption change corresponding to each distributed resource in the current training period, the incentive price, the third power consumption change corresponding to the next period and the reward corresponding to the aggregation system in the current training period are taken as a training sample.

[0067] Optionally, after updating the first parameter in the current policy network based on the current evaluation value and updating the first evaluation parameter in the current evaluation network based on the current evaluation value and the target evaluation value, the method further comprises:

[0068] The second electric energy consumption change corresponding to each distributed resource in the current training period is used as the state of the updated current policy network input to obtain a new incentive price corresponding to the current training period, and based on the new incentive price corresponding to each distributed resource in the current training period, a new reward corresponding to the aggregation system where the distributed resource is located in the current training period is calculated;

[0069] Outputting the new incentive price corresponding to the current training period to the control terminal of each distributed resource, and obtaining the new fifth electric energy consumption corresponding to each distributed resource in the next period;

[0070] Calculate a new third electric energy consumption change corresponding to each distributed resource in the next time period based on the third electric energy consumption corresponding to each distributed resource in the current training time period and the new fifth electric energy consumption corresponding to the next time period;

[0071] The second energy consumption change corresponding to each distributed resource in the current training period, the new incentive price, the new third energy consumption change corresponding to the next period and the reward corresponding to the aggregation system in the current training period are taken as a new training sample.

[0072] Optionally, updating the target policy network based on the updated first policy parameter includes:

[0073] Based on the updated first strategy parameters, obtaining updated second strategy parameters;

[0074] Based on the updated second policy parameters, updating the target policy network;

[0075] Based on the updated first evaluation parameter, the target evaluation network is updated, including:

[0076] Based on the updated first evaluation parameter, obtaining an updated second evaluation parameter;

[0077] The target evaluation network is updated based on the updated second evaluation parameter.

[0078] Optionally, the current evaluation value is used as a loss function, and a gradient ascent method is adopted to update the first policy parameter in the current policy network.

[0079] Optionally, the following method is used to calculate the reward corresponding to the aggregation system where the distributed resources are located in the current training period:

[0080]

[0081]

[0082]

[0083] μ1+μ2+μ3=1

[0084]

[0085]

[0086]

[0087]

[0088] 1≤j≤n uni

[0089]

[0090]

[0091] 1≤k≤n bi

[0092] Among them, Obj is the reward, μ1, μ2, μ3 are weight coefficients, P Agg is the total operating cost of the distributed aggregator, P uni is the total profit of all unidirectional distributed resources, P bi is the total profit of all bidirectional distributed resources, Δp ref (t) is the amount of electricity that the distributed aggregator obtains from the power company per hour, n dr is the number of distributed resources, is the power consumption of distributed resource n in period t, n uni is the number of unidirectional distributed resources, P uni (j) is the profit of distributed resource j, n bi is the number of bidirectional distributed resources, P bi (k) is the profit of distributed resource k, is the power consumption of distributed resource i in period t, and are the incentive price and energy consumption change of distributed resource i in period t, C(t) is the scheduling cost of each distributed resource in period t, and are the incentive price and energy consumption change of distributed resource j in period t, ξ t is the elasticity coefficient of the sensitivity of power consumption changes to rewards, is the energy consumption of distributed resource j in period t, α1 and α2 are coefficients, and are the incentive price and energy consumption change of distributed resource k in period t, is the power consumption of distributed resource k in period t, η k and γ k are the line loss and efficiency of charge / discharge respectively, and β1 and β are coefficients.

[0093] According to another aspect of the present application, a storage medium is provided, in which at least one executable instruction is stored, and the executable instruction enables a processor to perform operations corresponding to the above-mentioned incentive price determination method applied to distributed resource aggregation.

[0094] According to another aspect of the present application, a computer device is provided, comprising: a processor, a memory, a communication interface and a communication bus, wherein the processor, the memory and the communication interface communicate with each other via the communication bus;

[0095] The memory is used to store at least one executable instruction, and the executable instruction enables the processor to perform operations corresponding to the above-mentioned incentive price determination method applied to distributed resource aggregation.

[0096] By means of the above technical solution, the technical solution provided by the embodiment of the present invention has at least the following advantages:

[0097] The present application provides a method, device, equipment and medium for determining an incentive price for distributed resource aggregation. Based on the power consumption data of the current training period, a deep deterministic policy gradient model is trained to obtain the power consumption of the current period and the power consumption of the previous period, and the change in power consumption of the current period is calculated. The change in power consumption of the current period is input into the trained deep deterministic policy gradient model. The deep deterministic policy gradient model outputs the incentive price of the current period. Since the bidirectional distributed resources not only obtain power from the power grid but also output power, there are many variable factors in the incentive price of the bidirectional distributed resources. However, the deep deterministic policy gradient model can determine continuous actions according to the state of the facts, and can accurately determine the incentive price of the bidirectional distributed resources according to the power consumption of the bidirectional distributed resources, providing an accurate reward mechanism for each distributed resource, ensuring the effective scheduling of distributed aggregation resources, and attracting more distributed resources from the demand side to join the distributed aggregation system.

[0098] The above description is only an overview of the technical solution of the present invention. In order to more clearly understand the technical means of the present invention, it can be implemented according to the contents of the specification. In order to make the above and other purposes, features and advantages of the present invention more obvious and easy to understand, the specific implementation methods of the present invention are listed below. BRIEF DESCRIPTION OF THE DRAWINGS

[0099] Various other advantages and benefits will become apparent to those of ordinary skill in the art by reading the detailed description of the preferred embodiments below. The accompanying drawings are only for the purpose of illustrating the preferred embodiments and are not to be considered as limiting the present invention. Also, the same reference symbols are used throughout the accompanying drawings to indicate the same components. In the accompanying drawings:

[0100] Figure 1 A flowchart of a method for determining an incentive price for distributed resource aggregation provided by an embodiment of the present application is shown;

[0101] Figure 2A schematic diagram of a distributed aggregation system for another method for determining an incentive price for distributed resource aggregation provided by an embodiment of the present application is shown;

[0102] Figure 3 A schematic diagram of reward price for another method for determining incentive price for distributed resource aggregation provided in an embodiment of the present application is shown;

[0103] Figure 4 A block diagram showing a composition of an incentive price determination device for distributed resource aggregation provided by an embodiment of the present application;

[0104] Figure 5 A schematic structural diagram of a computer device provided by an embodiment of the present invention is shown.

[0105] in,

[0106] Figure 4 Middle: 402-model training module; 404-power consumption acquisition module; 406-upper power consumption change acquisition module; 408-incentive price acquisition module;

[0107] Figure 5 In it: 502 - processor; 504 - communication interface; 506 - memory; 508 - communication bus; 510 - program. DETAILED DESCRIPTION

[0108] The present invention will be described in detail below with reference to the accompanying drawings and in combination with embodiments. It should be noted that the embodiments and features in the embodiments of the present invention can be combined with each other without conflict.

[0109] In order to further explain the technical means and effects adopted by the present invention to achieve the predetermined invention purpose, the specific implementation methods, structures, features and effects of the present invention application are described in detail below in conjunction with the accompanying drawings and preferred embodiments. In the following description, different "one embodiment" or "embodiment" does not necessarily refer to the same embodiment. In addition, specific features, structures, or characteristics in one or more embodiments may be combined in any suitable form.

[0110] In view of the problem that the current load aggregator reward calculation method cannot calculate the reward compensation of distributed resources, the embodiment of the present application provides an incentive price determination method applied to distributed resource aggregation, such as Figure 1 As shown, the method includes:

[0111] 102: Obtain power consumption data corresponding to each distributed resource in the current training period, and train a deep deterministic policy gradient model based on the power consumption data;

[0112] 104: Obtain a first power consumption corresponding to a current period and a second power consumption corresponding to a previous period of the current period for each distributed resource and a power consumption change;

[0113] 106: Calculate a first electric energy consumption change corresponding to each distributed resource in the current period based on the first electric energy consumption change and the second electric energy consumption change;

[0114] 108: Input the first energy consumption change amount corresponding to each distributed resource in the current time period into the trained deep deterministic policy gradient model to obtain the incentive price corresponding to each distributed resource in the current time period.

[0115] Specifically, a typical distributed resource aggregation network may include an electricity aggregator and many electricity users / participants who are the owners of distributed resources. The electricity users / participants include new types of resources such as mobile battery power stations, residential energy storage systems (ESS), smart home devices, and electric vehicles without considering vehicle-to-grid (V2G) technology.

[0116] Since bidirectional distributed resources not only obtain electricity from the power grid but also output electricity, there are many variable factors in the incentive price of bidirectional distributed resources. However, the deep deterministic policy gradient model can determine continuous actions based on the state of facts and can determine the incentive price of bidirectional distributed resources based on the electricity consumption of bidirectional distributed resources.

[0117] First, the power consumption data of the current training period is obtained, and the deep deterministic policy gradient model is trained based on the power consumption data of the current training period. After the training, the power consumption of the current period and the power consumption of the previous period are obtained, and the change in power consumption of the current period is calculated. The change in power consumption of the current period is input into the trained deep deterministic policy gradient model, and the deep deterministic policy gradient model outputs the incentive price of the current period.

[0118] In one embodiment of the present invention, the power consumption data corresponding to each distributed resource in the current training period is obtained, and based on the power consumption data, a deep deterministic policy gradient model is trained, including:

[0119] Constructing a current policy network, a target policy network, a current evaluation network, a target evaluation network and a replay buffer pool, and obtaining a third power consumption amount corresponding to a current training period and a fourth power consumption amount corresponding to a plurality of periods before the current training period for each distributed resource;

[0120] Based on the current strategy network, the third electric energy consumption corresponding to each distributed resource in the current training period, and the fourth electric energy consumption corresponding to each period before the current training period, a plurality of training samples are obtained, and the plurality of training samples are stored in the replay buffer pool, wherein the training samples include the second electric energy consumption change corresponding to each distributed resource in the current training period, the incentive price, the third electric energy consumption change corresponding to the next period, and the reward corresponding to the aggregation system in the current training period;

[0121] Obtain any training sample from the replay buffer, input the second electric energy consumption change and the incentive price corresponding to each distributed resource in the any training sample in the current training period into the current evaluation network to obtain a current evaluation value, input the third electric energy consumption change corresponding to each distributed resource in the next period into the target strategy model to obtain the incentive price corresponding to each distributed resource in the next period, input the third electric energy consumption change and the incentive price corresponding to each distributed resource in the next period into the target evaluation network to obtain a target evaluation value;

[0122] Based on the current evaluation value, a first policy parameter in the current policy network is updated, and based on the current evaluation value and the target evaluation value, a first evaluation parameter in the current evaluation network is updated;

[0123] Based on the updated first policy parameter, the current policy network and the target policy network are updated, and based on the updated first evaluation parameter, the current evaluation network and the target evaluation network are updated;

[0124] Other training samples are obtained from the replay buffer pool, and the policy parameters and evaluation parameters are updated again based on the updated current policy network, target policy network, current evaluation network, and target evaluation network until the number of iterations is reached to obtain a trained deep deterministic policy gradient model.

[0125] In one embodiment of the present invention, the obtaining of multiple training samples based on the current strategy network, the third power consumption corresponding to the current training period of each distributed resource, and the fourth power consumption corresponding to each period before the current training period includes:

[0126] Calculate a second electric energy consumption change corresponding to each distributed resource in the current training period based on a third electric energy consumption corresponding to each distributed resource in the current training period and a fourth electric energy consumption corresponding to the distributed resource in the previous period of the current training period;

[0127] Input the second change in electric energy consumption corresponding to each distributed resource in the current training period into the current policy network, obtain the action determined by the current policy network, the action is the incentive price corresponding to each distributed resource in the current training period, and calculate the reward corresponding to the aggregation system where the distributed resource is located in the current training period according to the incentive price corresponding to each distributed resource in the current training period, the third electric energy consumption, the second change in electric energy consumption, and the fourth electric energy consumption corresponding to the distributed resource in multiple periods before the current training period;

[0128] Outputting the incentive price corresponding to each distributed resource in the current training period to the control terminal of each distributed resource, and obtaining the fifth electric energy consumption corresponding to each distributed resource in the next period;

[0129] Calculate a third electric energy consumption change corresponding to each distributed resource in the next training period based on the third electric energy consumption corresponding to each distributed resource in the current training period and the fifth electric energy consumption corresponding to the next training period;

[0130] The second power consumption change corresponding to each distributed resource in the current training period, the incentive price, the third power consumption change corresponding to the next period and the reward corresponding to the aggregation system in the current training period are taken as a training sample.

[0131] In one embodiment, after updating the first parameter in the current policy network based on the current evaluation value and updating the first evaluation parameter in the current evaluation network based on the current evaluation value and the target evaluation value, the method further comprises:

[0132] The second electric energy consumption change corresponding to each distributed resource in the current training period is used as the state of the updated current policy network input to obtain a new incentive price corresponding to the current training period, and based on the new incentive price corresponding to each distributed resource in the current training period, a new reward corresponding to the aggregation system where the distributed resource is located in the current training period is calculated;

[0133] Outputting the new incentive price corresponding to the current training period to the control terminal of each distributed resource, and obtaining the new fifth electric energy consumption corresponding to each distributed resource in the next period;

[0134] Calculate a new third electric energy consumption change corresponding to each distributed resource in the next time period based on the third electric energy consumption corresponding to each distributed resource in the current training time period and the new fifth electric energy consumption corresponding to the next time period;

[0135] The second energy consumption change corresponding to each distributed resource in the current training period, the new incentive price, the new third energy consumption change corresponding to the next period and the reward corresponding to the aggregation system in the current training period are taken as a new training sample.

[0136] Specifically, after the first policy parameter of the current policy network is updated, the new incentive price corresponding to the current training period is re-determined according to the second change in electric energy consumption in the current training period, and the reward corresponding to the aggregation system where the distributed resources are located in the current training period is calculated according to the new incentive price corresponding to each distributed resource in the current training period, the third electric energy consumption, the second change in electric energy consumption and the fourth electric energy consumption corresponding to multiple time periods before the current training period; the new incentive price corresponding to the distributed resources in the current training period is output to the control terminal of each distributed resource, and the new fifth electric energy consumption corresponding to each distributed resource in the next time period is obtained, and the new third change in electric energy consumption corresponding to each distributed resource in the next time period is calculated according to the third electric energy consumption corresponding to each distributed resource in the current training period and the new fifth electric energy consumption corresponding to the next time period, and the second change in electric energy consumption corresponding to each distributed resource in the current training period, the new incentive price, the new third change in electric energy consumption corresponding to the next time period and the reward corresponding to the aggregation system in the current training period are used as a training sample.

[0137] In one embodiment, the current evaluation value is used as the loss function, and the gradient ascent method is used to update the first policy parameter in the current policy network.

[0138] Specifically, the deep deterministic policy gradient (DDPG) method is used to determine the optimal hourly incentive price for each participating distributed resource. Based on the power adjustment reference value of the power company and the energy consumption benchmark of each participating distributed resource, the distributed aggregator uses the DDPG model to determine the optimal hourly reward, i.e., the incentive price, for each distributed resource to adjust the power to provide accurate ancillary services for the grid. There are two goals here, one is to minimize the difference between the reference power and the actual power of the grid, and the other is to maximize the total profit of the network.

[0139] The DDPG model adopts an actor-critic architecture, which includes two networks: actor and critic. The actor network includes the current policy network and the target policy network, and the critic network includes the current evaluation network and the target evaluation network. Deep neural networks (DNNs) are used to learn these two networks. In this algorithm, for a given state s t , given the state s t Input to the current policy network, the current policy network μ generates a continuous action a t =μ φ (s t ), and the current evaluation network utilization status st and action a t To approximate the Q-value function Q θ (s t ,a t ).

[0140] During training, policy improvement and evaluation iteration are crucial steps in reinforcement learning. When improving the policy, the policy parameters are updated by gradient ascent, which ensures that the actions generated by the learned policy can maximize the value of the Q function. Policy gradient used to update the policy network As shown below:

[0141]

[0142] Updating DDPG using sequentially generated experience can lead to instability issues due to temporal correlations in the data. To mitigate this instability, the learning process is updated with mini-batches of data uniformly sampled from the replay buffer. θ aims to find the action that maximizes the Q function, while the gradient μ aims to learn a policy that makes the actor more likely to take this action. Policy evaluation is performed via a critic network that is used to Q-learn the action-value function. The action-value function is updated by minimizing the mean squared Bellman error as follows.

[0143] L(θ,D)=E s ~ D [(Q θ (s t ,a t )-(r t +γ(1-d)·Q θ (s t+1 ,μ φ (s t+1 )))) 2 ]

[0144] In this process, the Q-value function is used to evaluate the target and is updated during the training process. However, this update may lead to an unstable minimization of the mean squared Bellman error. To address this challenge and enhance the stability of learning, the parameters of the target policy network and the target evaluation network are updated. After all processing is completed, the goal of DDPG is to minimize the following mean squared Bellman error.

[0145] L(θ,D)=E s～D [(Q θ (s t ,a t )-(r t +γ(1-d)·Q θtar (s t+1 ,μφtar (s t+1 )))) 2 ]

[0146] In order to enhance the exploration performance of the DDPG agent, in addition to using the replay buffer and the target network, we also adopt the actor strategy μ φ (s t ) has a mean of zero and a standard deviation of σ t Gaussian noise, that is

[0147] In a specific embodiment, the DDPG model is trained using the following method:

[0148] Step 1: Construct four networks and a replay buffer pool. First initialize the current Actor (current policy network) and the current Critic (current evaluation network). Then copy the network parameters of the current Actor (current policy network) and the current Critic (current target network) to the target Actor (target policy network) and the target Critic (target evaluation network), respectively, and thus obtain four networks.

[0149] Step 2: Generate training samples. Use the current Actor to interact with the environment, that is, to obtain the state S to the current Actor network, get action a, apply action a to the environment, and the environment will return the state s′ and reward r at the next moment. So the data of this process can be represented by a four-tuple (S, a, r, S′). This four-tuple (S, a, r, S′) means that when in state S, action a is taken, the reward r and the next state S′ are obtained. The array composed of the change in power consumption of each distributed resource in this application is the state S, defined as The array of incentive prices obtained by each distributed resource is the action, and the maximum total profit corresponding to the aggregation system where the distributed resource is located in the current training period is the reward r.

[0150] Assume that the parameters of the Actor network and the Target Actor network are w and w′ respectively; the parameters of the Critic network and the Target Critic network are θ and θ′ respectively.

[0151] Step 3: Update the current critic network. Take a training sample (S, a, r, S′) from the replay buffer pool for training. Input S and a in (S, a, r, S′) into the current critic to get the current Q value Q(S, a).

[0152] Then input S' in (S,a,r,S') into the target Actor network to get action a'. Input S' and a' together into the target Critic to get Q(S',a'). It is hoped that the Q value Q(S,a) output by the current critic should be equal to the value of the target Q'. So it becomes a supervised learning problem. Consider Q' as a label, update the current critic so that the output Q is as close to the label Q' as possible, and use the gradient descent method to minimize and update all parameters θ of the current critic network. In this way, the current critic network is updated.

[0153] Step 4: Current Actor Network Update: Use the gradient ascent method to update the first policy parameter in the current policy network of the energy consumption change.

[0154] Step 5: Update the two target networks. At regular intervals, use the parameters of the two current networks to update the target Actor and target Critic.

[0155] In one embodiment, updating the target policy network based on the updated first policy parameter includes:

[0156] Based on the updated first strategy parameters, obtaining updated second strategy parameters;

[0157] Based on the updated second strategy parameter, updating the power consumption change target strategy network;

[0158] Based on the updated first evaluation parameter, the target evaluation network is updated, including:

[0159] Based on the updated first evaluation parameter, obtaining an updated second evaluation parameter;

[0160] Based on the updated second evaluation parameter, the electric energy consumption change target evaluation network is updated.

[0161] For example, the following formula is used to update the second evaluation parameter or the second strategy parameter:

[0162] θ tar =ρθ+(1-ρ)θ tar

[0163] φ tar =ρφ+(1-ρ)φ tar

[0164] Among them, θ tar is the second strategy parameter, θ is the first strategy parameter, φ is the first evaluation parameter, tar is the second evaluation parameter.

[0165] In one embodiment, the following method is used to calculate the reward corresponding to the aggregation system where the distributed resource is located in the current training period:

[0166]

[0167]

[0168] μ1+μ2+μ3=1

[0169]

[0170]

[0171]

[0172]

[0173] 1≤j≤n uni

[0174]

[0175]

[0176] 1≤k≤n bi

[0177] Among them, Obj is the reward, μ1, μ2, μ3 are weight coefficients, P Agg is the total operating cost of the distributed aggregator, P uni is the total profit of all unidirectional distributed resources, P bi is the total profit of all bidirectional distributed resources, Δp ref (t) is the amount of electricity that the distributed aggregator obtains from the power company per hour, n dr is the number of distributed resources, is the power consumption of distributed resource n in period t, n uni is the number of unidirectional distributed resources, P uni (j) is the profit of distributed resource j, n bi is the number of bidirectional distributed resources, P bi (k) is the profit of distributed resource k, is the power consumption of distributed resource i in period t, and are the incentive price and energy consumption change of distributed resource i in period t, C(t) is the scheduling cost of each distributed resource in period t, and are the incentive price and energy consumption change of distributed resource j in period t, ξ tis the elasticity coefficient of the sensitivity of power consumption changes to rewards, is the energy consumption of distributed resource j in period t, α1 and α2 are coefficients, and are the incentive price and energy consumption change of distributed resource k in period t, is the power consumption of distributed resource k in period t, η k and γ k are the line loss and efficiency of charge / discharge respectively, and β1 and β are coefficients.

[0178] Specifically, the participating distributed resources receive dispatch signals from the distributed aggregator every hour to provide a specific amount of power support or reduce power support. The main goal of the aggregator is to maximize the total profit within the network (both the profits of the aggregator and the participants' distributed resources are included) while providing accurate auxiliary power support to the grid. In order to describe the problem, it is necessary to determine the profit of each party. From the perspective of the aggregator, its goal is to minimize the total operating cost, which can be expressed as:

[0179]

[0180]

[0181] Among them, n dr is the total number of distributed resources. C(t) is the scheduling cost / price of each distributed resource at time t. and are the excitation rate and electric energy change of distributed resource i at time t respectively.

[0182] like Figure 2 As shown in the figure, in the distributed aggregation system, the participating distributed resources include unidirectional resources and bidirectional resources. Unidirectional load refers to the situation where electricity only flows from the power grid to the load, which is usually an interruptible load, such as smart home devices, air conditioners, electric heaters, and electric vehicle charging piles without V2G function. The owners of such loads can usually accept a certain degree of discomfort cost by adjusting their utilization in exchange for rewards from aggregators. The profit of unidirectional distributed resources is the sum of the discomfort cost and the reward received from the aggregator, expressed as follows:

[0183]

[0184]

[0185] 1≤j≤n uni

[0186] Among them, n uni is the total number of unidirectional loads in the network, ξ trepresents the elasticity coefficient of the sensitivity of power consumption change to reward. The discomfort level is expressed as is a function of , and is negatively correlated with the change in energy required by the aggregator.

[0187] Bidirectional resources can interact with the grid, both receiving and providing power to the grid. This type of load usually includes residential ESS, electric vehicles with V2G functions, and mobile battery power stations, which is an emerging source of load. Basically, these are battery-based resources with different charging / discharging rates. Its profit can be expressed as:

[0188]

[0189]

[0190] 1≤k≤n bi

[0191] Among them, n bi is the number of bidirectional loads, η j and γ j are the line loss and efficiency of charge / discharge, respectively. In the equation, the discomfort level is a function of Δptj.

[0192] Taking all these formulas and variables into account, the total profit target of the entire distributed aggregation system can be expressed as:

[0193]

[0194]

[0195]

[0196] μ1+μ2+μ3=1

[0197] Among them, μ1, μ2, μ3 are weight coefficients, Δp ref (t) is the reference power obtained by the entire aggregation network from the power company every hour.

[0198] The present application provides an incentive price determination method for distributed resource aggregation. Compared with the prior art, a deep deterministic policy gradient model is trained based on the power consumption data of the current training period, the power consumption of the current period and the power consumption of the previous period are obtained, the power consumption change of the current period is calculated, and the power consumption change of the current period is input into the trained deep deterministic policy gradient model. The deep deterministic policy gradient model outputs the incentive price of the current period. Since the bidirectional distributed resources not only obtain power from the power grid but also output power, there are many variable factors in the incentive price of the bidirectional distributed resources. However, the deep deterministic policy gradient model can determine continuous actions according to the state of facts, and can accurately determine the incentive price of the bidirectional distributed resources according to the power consumption of the bidirectional distributed resources, provide an accurate reward mechanism for each distributed resource, ensure the effective scheduling of distributed aggregation resources, and attract more distributed resources from the demand side to join the distributed aggregation system.

[0199] In one embodiment, to verify the effectiveness of the proposed method, the European Power Exchange EPEX market data is used for evaluation, and the deep Q network (DQN) is used as a benchmark for comparison. The distributed aggregation system developed for training and testing includes 650 unidirectional loads and 350 bidirectional loads. The capacity of each load ranges from 4kW to 85kW. The data is divided into a training data set and a test data set, the first 3 weeks are used for training, and the last 3 weeks are used for testing.

[0200] The actor neural network (policy network) and the critic (evaluation network) neural network adopt a three-layer structure, where the layer size of the actor network is 1000, 500 and 100, and the size of the critic network is 1004, 500 and 1. The two networks use rectified linear units (ReLU), sigmoid functions and linear activation functions. The learning rates of the actor and critic networks are 10 -4 and 10 -3 The size of the replay buffer pool is 3*10 4 , the mini-batch size is set to 64. The model performs 4.7*10 4 The average daily incentive price of the DDPG algorithm and the DQN method in all rounds is as follows: Figure 3 As shown in the figure, both the DQN method and the DDPG algorithm are able to learn control actions that achieve higher rewards. Compared with the DQN algorithm, the DDPG algorithm enhances the control strategy and consistently achieves better rewards.

[0201] Furthermore, as a response to the above Figure 1 The implementation of the method shown, such as Figure 4 As shown, an embodiment of the present invention provides an incentive price determination device applied to distributed resource aggregation, including:

[0202] A model training module 402 is used to obtain the power consumption data corresponding to each distributed resource in the current training period, and train a deep deterministic policy gradient model based on the power consumption data;

[0203] The power consumption acquisition module 404 is used to acquire the first power consumption corresponding to the current time period and the second power consumption corresponding to the previous time period of the current time period of each distributed resource;

[0204] The power consumption change acquisition module 406 is used to calculate the first power consumption change corresponding to each distributed resource in the current period based on the first power consumption change and the second power consumption change;

[0205] The incentive price acquisition module 408 is used to input the first energy consumption change corresponding to each distributed resource in the current time period into the trained deep deterministic policy gradient model to obtain the incentive price corresponding to each distributed resource in the current time period.

[0206] The present application provides an incentive price determination device for distributed resource aggregation. Compared with the prior art, a deep deterministic policy gradient model is trained based on the power consumption data of the current training period, the power consumption of the current period and the power consumption of the previous period are obtained, the power consumption change of the current period is calculated, and the power consumption change of the current period is input into the trained deep deterministic policy gradient model. The deep deterministic policy gradient model outputs the incentive price of the current period. Since the bidirectional distributed resources not only obtain power from the power grid but also output power, there are many variable factors in the incentive price of the bidirectional distributed resources. However, the deep deterministic policy gradient model can determine continuous actions according to the state of facts, and can determine the incentive price of the bidirectional distributed resources according to the power consumption of the bidirectional distributed resources, providing an accurate reward mechanism for each distributed resource, ensuring the effective scheduling of distributed aggregation resources, and attracting more distributed resources from the demand side to join the distributed aggregation system.

[0207] In one embodiment, the model training module is further used to:

[0208] Constructing a current policy network, a target policy network, a current evaluation network, a target evaluation network and a replay buffer pool, and obtaining a third power consumption amount corresponding to a current training period and a fourth power consumption amount corresponding to a plurality of periods before the current training period for each distributed resource;

[0209] Based on the current strategy network, the third electric energy consumption corresponding to each distributed resource in the current training period, and the fourth electric energy consumption corresponding to each period before the current training period, a plurality of training samples are obtained, and the plurality of training samples are stored in the replay buffer pool, wherein the training samples include the second electric energy consumption change corresponding to each distributed resource in the current training period, the incentive price, the third electric energy consumption change corresponding to the next period, and the reward corresponding to the aggregation system in the current training period;

[0210] Obtain any training sample from the replay buffer, input the second electric energy consumption change and the incentive price corresponding to each distributed resource in the any training sample in the current training period into the current evaluation network to obtain a current evaluation value, input the third electric energy consumption change corresponding to each distributed resource in the next period into the target strategy model to obtain the incentive price corresponding to each distributed resource in the next period, input the third electric energy consumption change and the incentive price corresponding to each distributed resource in the next period into the target evaluation network to obtain a target evaluation value;

[0211] Based on the current evaluation value, a first policy parameter in the current policy network is updated, and based on the current evaluation value and the target evaluation value, a first evaluation parameter in the current evaluation network is updated;

[0212] Based on the updated first policy parameter, the current policy network and the target policy network are updated, and based on the updated first evaluation parameter, the current evaluation network and the target evaluation network are updated;

[0213] Other training samples are obtained from the replay buffer pool, and the policy parameters and evaluation parameters are updated again based on the updated current policy network, target policy network, current evaluation network, and target evaluation network until the number of iterations is reached to obtain a trained deep deterministic policy gradient model.

[0214] In one embodiment, the obtaining of multiple training samples based on the current strategy network, the third power consumption corresponding to the current training period of each distributed resource, and the fourth power consumption corresponding to each period before the current training period includes:

[0215] Calculate a second electric energy consumption change corresponding to each distributed resource in the current training period based on a third electric energy consumption corresponding to each distributed resource in the current training period and a fourth electric energy consumption corresponding to the distributed resource in the previous period of the current training period;

[0216] Input the second change in electric energy consumption corresponding to each distributed resource in the current training period into the current policy network, obtain the action determined by the current policy network, the action is the incentive price corresponding to each distributed resource in the current training period, and calculate the reward corresponding to the aggregation system where the distributed resource is located in the current training period according to the incentive price corresponding to each distributed resource in the current training period, the third electric energy consumption, the second change in electric energy consumption, and the fourth electric energy consumption corresponding to the distributed resource in multiple periods before the current training period;

[0217] Outputting the incentive price corresponding to each distributed resource in the current training period to the control terminal of each distributed resource, and obtaining the fifth electric energy consumption corresponding to each distributed resource in the next period;

[0218] Calculate a third electric energy consumption change corresponding to each distributed resource in the next training period based on the third electric energy consumption corresponding to each distributed resource in the current training period and the fifth electric energy consumption corresponding to the next training period;

[0219] The second power consumption change corresponding to each distributed resource in the current training period, the incentive price, the third power consumption change corresponding to the next period and the reward corresponding to the aggregation system in the current training period are taken as a new training sample.

[0220] In one embodiment, in one embodiment,

[0221] In one embodiment, updating the target policy network based on the updated first policy parameter includes:

[0222] Based on the updated first strategy parameters, obtaining updated second strategy parameters;

[0223] Based on the updated second policy parameters, updating the target policy network;

[0224] Based on the updated first evaluation parameter, the target evaluation network is updated, including:

[0225] Based on the updated first evaluation parameter, obtaining an updated second evaluation parameter;

[0226] The target evaluation network is updated based on the updated second evaluation parameter.

[0227] In one embodiment, the current evaluation value is used as the loss function, and the gradient ascent method is used to update the first policy parameter in the current policy network.

[0228] In one embodiment, the following method is used to calculate the reward corresponding to the aggregation system where the distributed resource is located in the current training period:

[0229]

[0230]

[0231] μ1+μ2+μ3=1

[0232]

[0233]

[0234]

[0235]

[0236] 1≤j≤n uni

[0237]

[0238]

[0239] 1≤k≤n bi

[0240] Among them, Obj is the reward, μ1, μ2, μ3 are weight coefficients, P Agg is the total operating cost of the distributed aggregator, P uni is the total profit of all unidirectional distributed resources, P bi is the total profit of all bidirectional distributed resources, Δp ref (t) is the amount of electricity that the distributed aggregator obtains from the power company per hour, n dr is the number of distributed resources, is the power consumption of distributed resource n in period t, n uni is the number of unidirectional distributed resources, P uni (j) is the profit of distributed resource j, n bi is the number of bidirectional distributed resources, P bi (k) is the profit of distributed resource k, is the power consumption of distributed resource i in period t, and are the incentive price and energy consumption change of distributed resource i in period t, C(t) is the scheduling cost of each distributed resource in period t, and are the incentive price and energy consumption change of distributed resource j in period t, ξ t is the elasticity coefficient of the sensitivity of power consumption changes to rewards, is the energy consumption of distributed resource j in period t, α1 and α2 are coefficients, and are the incentive price and energy consumption change of distributed resource k in period t, is the power consumption of distributed resource k in period t, η k and γ k are the line loss and efficiency of charge / discharge respectively, and β1 and β are coefficients.

[0241] According to an embodiment of the present invention, a storage medium is provided, wherein the storage medium stores at least one executable instruction, and the computer executable instruction can execute the incentive price determination method for distributed resource aggregation in any of the above method embodiments.

[0242] Figure 5 A schematic diagram of the structure of a computer device provided according to an embodiment of the present invention is shown, and the specific embodiment of the present invention does not limit the specific implementation of the computer device.

[0243] like Figure 5 As shown, the computer device may include: a processor (processor) 502 , a communication interface (Communications Interface) 504 , a memory (memory) 506 , and a communication bus 508 .

[0244] The processor 502 , the communication interface 504 , and the memory 506 communicate with each other via a communication bus 508 .

[0245] The communication interface 504 is used to communicate with other devices such as clients or other servers.

[0246] The processor 502 is used to execute the program 510, and specifically can execute the relevant steps in the above-mentioned embodiment of the incentive price determination method applied to distributed resource aggregation.

[0247] Specifically, the program 510 may include program codes, which include computer operation instructions.

[0248] The processor 502 may be a central processing unit (CPU), or an application-specific integrated circuit (ASIC), or one or more integrated circuits configured to implement the embodiments of the present invention. The one or more processors included in the computer device may be processors of the same type, such as one or more CPUs; or processors of different types, such as one or more CPUs and one or more ASICs.

[0249] The memory 506 is used to store the program 510. The memory 506 may include a high-speed RAM memory, and may also include a non-volatile memory (non-volatile memory), such as at least one disk memory.

[0250] The program 510 may be specifically configured to enable the processor 502 to execute the incentive price determination method for distributed resource aggregation in any of the above method embodiments.

[0251] Obviously, those skilled in the art should understand that the above modules or steps of the present invention can be implemented by a general computing device, they can be concentrated on a single computing device, or distributed on a network composed of multiple computing devices. In one embodiment, they can be implemented by a program code executable by a computing device, so that they can be stored in a storage device and executed by the computing device, and in some cases, the steps shown or described can be executed in a different order than here, or they can be made into individual integrated circuit modules, or multiple modules or steps therein can be made into a single integrated circuit module for implementation. Thus, the present invention is not limited to any specific combination of hardware and software.

[0252] The above embodiments are only exemplary embodiments of the present application and are not intended to limit the present application. The protection scope of the present application is defined by the claims. Those skilled in the art may make various modifications or equivalent substitutions to the present application within the essence and protection scope of the present application, and such modifications or equivalent substitutions shall also be deemed to fall within the protection scope of the present application.

Claims

1. A method for determining incentive prices for distributed resource aggregation, characterized in that: include: Obtaining power consumption data corresponding to each distributed resource in the current training period, and training a deep deterministic policy gradient model based on the power consumption data; Obtaining a first electric energy consumption corresponding to a current time period and a second electric energy consumption corresponding to a previous time period of each distributed resource; Based on the first electric energy consumption and the second electric energy consumption, calculating a first electric energy consumption change corresponding to each distributed resource in a current time period; Inputting the first electric energy consumption change corresponding to each distributed resource in the current period into the trained deep deterministic policy gradient model to obtain the incentive price corresponding to each distributed resource in the current period; The step of training a deep deterministic policy gradient model based on the power consumption data includes: Constructing a current policy network, a target policy network, a current evaluation network, a target evaluation network and a replay buffer pool, and obtaining a third electric energy consumption corresponding to a current training period and a fourth electric energy consumption corresponding to a previous period of the current training period for each distributed resource; Based on the current strategy network, the third electric energy consumption corresponding to each distributed resource in the current training period, and the fourth electric energy consumption corresponding to the previous period of the current training period, a plurality of training samples are obtained, and the plurality of training samples are stored in a replay buffer pool, wherein the training samples include the second electric energy consumption change corresponding to each distributed resource in the current training period, the incentive price, the third electric energy consumption change corresponding to the next period, and the reward corresponding to the aggregation system in the current training period; Obtain any training sample from the replay buffer, input the second electric energy consumption change and the incentive price corresponding to each distributed resource in the any training sample in the current training period into the current evaluation network to obtain a current evaluation value, input the third electric energy consumption change corresponding to each distributed resource in the next period into the target strategy network to obtain an incentive price corresponding to each distributed resource in the next period, input the third electric energy consumption change and the incentive price corresponding to each distributed resource in the next period into the target evaluation network to obtain a target evaluation value; Based on the current evaluation value, a first policy parameter in the current policy network is updated, and based on the current evaluation value and the target evaluation value, a first evaluation parameter in the current evaluation network is updated; Based on the updated first policy parameter, the current policy network and the target policy network are updated, and based on the updated first evaluation parameter, the current evaluation network and the target evaluation network are updated; Obtaining other training samples from the replay buffer pool, and re-updating the policy parameters and evaluation parameters based on the updated current policy network, the target policy network, the current evaluation network, and the target evaluation network until the number of iterations is reached, thereby obtaining a trained deep deterministic policy gradient model; The following method is used to obtain the reward corresponding to the aggregation system in the current training period: Calculate a second electric energy consumption change corresponding to each distributed resource in the current training period based on a third electric energy consumption corresponding to each distributed resource in the current training period and a fourth electric energy consumption corresponding to a previous period of the current training period; Inputting the second electric energy consumption change corresponding to each distributed resource in the current training period into the current strategy network, obtaining an action determined by the current strategy network, wherein the action is the incentive price corresponding to each distributed resource in the current training period; The following method is used to calculate the reward corresponding to the aggregation system where the distributed resources are located in the current training period: μ1+μ2+μ3=1 1≤j≤n uni 1≤k≤n bi Among them, Obj is the reward, μ1, μ2, μ3 are weight coefficients, P Agg is the total operating cost of the distributed aggregator, P uni is the total profit of all unidirectional distributed resources, P bi is the total profit of all bidirectional distributed resources, Δp ref (t) is the amount of electricity that the distributed aggregator obtains from the power company per hour, n dr is the number of distributed resources, is the power consumption of distributed resource n in period t, n uni is the number of unidirectional distributed resources, P uni (j) is the profit of distributed resource j, n bi is the number of bidirectional distributed resources, P bi (k) is the profit of distributed resource k, is the power consumption of distributed resource i in period t, and are the incentive price and energy consumption change of distributed resource i in period t, C(t) is the scheduling cost of each distributed resource in period t, and are the incentive price and energy consumption change of distributed resource j in period t, ξ t is the elasticity coefficient of the sensitivity of power consumption changes to rewards, is the power consumption of distributed resource j in period t, α1 and α2 are coefficients, and are the incentive price and energy consumption change of distributed resource k in period t, is the power consumption of distributed resource k in period t, η k and γ k are the line loss and efficiency of charge / discharge respectively, and β1 and β are coefficients.

2. The method for determining incentive prices for distributed resource aggregation according to claim 1, characterized in that: The obtaining of multiple training samples based on the current strategy network, the third power consumption corresponding to the current training period of each distributed resource, and the fourth power consumption corresponding to the previous period of the current training period, comprises: Outputting the incentive price corresponding to each distributed resource in the current training period to the control terminal of each distributed resource, and obtaining the fifth electric energy consumption corresponding to each distributed resource in the next period; Calculate a third electric energy consumption change corresponding to each distributed resource in the next training period based on the third electric energy consumption corresponding to each distributed resource in the current training period and the fifth electric energy consumption corresponding to the next training period; The second power consumption change corresponding to each distributed resource in the current training period, the incentive price, the third power consumption change corresponding to the next period and the reward corresponding to the aggregation system in the current training period are taken as a training sample.

3. The method for determining incentive prices for distributed resource aggregation according to claim 1, characterized in that: After the first parameter in the current strategy network is updated based on the current evaluation value, and the first evaluation parameter in the current evaluation network is updated based on the current evaluation value and the target evaluation value, the method further includes: The second electric energy consumption change corresponding to each distributed resource in the current training period is used as the state of the updated current policy network input to obtain a new incentive price corresponding to the current training period, and based on the new incentive price corresponding to each distributed resource in the current training period, a new reward corresponding to the aggregation system where the distributed resource is located in the current training period is calculated; Outputting the new incentive price corresponding to the current training period to the control terminal of each distributed resource, and obtaining the new fifth electric energy consumption corresponding to each distributed resource in the next period; Calculate a new third electric energy consumption change corresponding to each distributed resource in the next time period based on the third electric energy consumption corresponding to each distributed resource in the current training time period and the new fifth electric energy consumption corresponding to the next time period; The second energy consumption change corresponding to each distributed resource in the current training period, the new incentive price, the new third energy consumption change corresponding to the next period and the reward corresponding to the aggregation system in the current training period are taken as a new training sample.

4. The method for determining incentive prices for distributed resource aggregation according to claim 1, characterized in that: Based on the updated first policy parameter, the target policy network is updated, including: Based on the updated first strategy parameters, obtaining updated second strategy parameters; The target strategy network is updated based on the updated second strategy parameter; and the target evaluation network is updated based on the updated first evaluation parameter, including: Based on the updated first evaluation parameter, obtaining an updated second evaluation parameter; The target evaluation network is updated based on the updated second evaluation parameter.

5. The method for determining incentive prices for distributed resource aggregation according to claim 1, characterized in that: The current evaluation value is used as a loss function, and a gradient ascent method is adopted to update the first policy parameter in the current policy network.

6. An incentive price determination device for distributed resource aggregation, characterized in that: include: A model training module, used to obtain the power consumption data corresponding to each distributed resource in the current training period, and train a deep deterministic policy gradient model based on the power consumption data; An electric energy consumption acquisition module, used to acquire a first electric energy consumption corresponding to a current time period and a second electric energy consumption corresponding to a previous time period of each distributed resource; An electric energy consumption change acquisition module, configured to calculate a first electric energy consumption change corresponding to each distributed resource in a current period based on the first electric energy consumption and the second electric energy consumption; An incentive price acquisition module, used to input the first electric energy consumption change corresponding to each distributed resource in the current period into the trained deep deterministic policy gradient model to obtain the incentive price corresponding to each distributed resource in the current period; The model training module is also used to: Constructing a current policy network, a target policy network, a current evaluation network, a target evaluation network and a replay buffer pool, and obtaining a third electric energy consumption corresponding to a current training period and a fourth electric energy consumption corresponding to a previous period of the current training period for each distributed resource; Based on the current strategy network, the third electric energy consumption corresponding to each distributed resource in the current training period, and the fourth electric energy consumption corresponding to the previous period of the current training period, a plurality of training samples are obtained, and the plurality of training samples are stored in the replay buffer pool, wherein the training samples include the second electric energy consumption change corresponding to each distributed resource in the current training period, the incentive price, the third electric energy consumption change corresponding to the next period, and the reward corresponding to the aggregation system in the current training period; Obtain any training sample from the replay buffer, input the second electric energy consumption change and the incentive price corresponding to each distributed resource in the any training sample in the current training period into the current evaluation network to obtain a current evaluation value, input the third electric energy consumption change corresponding to each distributed resource in the next period into the target strategy network to obtain an incentive price corresponding to each distributed resource in the next period, input the third electric energy consumption change and the incentive price corresponding to each distributed resource in the next period into the target evaluation network to obtain a target evaluation value; Based on the current evaluation value, a first policy parameter in the current policy network is updated, and based on the current evaluation value and the target evaluation value, a first evaluation parameter in the current evaluation network is updated; Based on the updated first policy parameter, the current policy network and the target policy network are updated, and based on the updated first evaluation parameter, the current evaluation network and the target evaluation network are updated; Obtaining other training samples from the replay buffer pool, and re-updating the policy parameters and evaluation parameters based on the updated current policy network, the target policy network, the current evaluation network, and the target evaluation network until the number of iterations is reached, thereby obtaining a trained deep deterministic policy gradient model; The model training module is also used to: Calculate a second electric energy consumption change corresponding to each distributed resource in the current training period based on a third electric energy consumption corresponding to each distributed resource in the current training period and a fourth electric energy consumption corresponding to a previous period of the current training period; Inputting the second electric energy consumption change corresponding to each distributed resource in the current training period into the current strategy network, obtaining an action determined by the current strategy network, wherein the action is the incentive price corresponding to each distributed resource in the current training period; The following method is used to calculate the reward corresponding to the aggregation system where the distributed resources are located in the current training period: μ1+μ2+μ3=1 1≤j≤n uni 1≤k≤n bi Among them, Obj is the reward, μ1, μ2, μ3 are weight coefficients, P Agg is the total operating cost of the distributed aggregator, P uni is the total profit of all unidirectional distributed resources, P bi is the total profit of all bidirectional distributed resources, Δp ref (t) is the amount of electricity that the distributed aggregator obtains from the power company per hour, n dr is the number of distributed resources, is the power consumption of distributed resource n in period t, n uni is the number of unidirectional distributed resources, P uni (j) is the profit of distributed resource j, n bi is the number of bidirectional distributed resources, P bi (k) is the profit of distributed resource k, is the power consumption of distributed resource i in period t, and are the incentive price and energy consumption change of distributed resource i in period t, C(t) is the scheduling cost of each distributed resource in period t, and are the incentive price and energy consumption change of distributed resource j in period t, ξ t is the elasticity coefficient of the sensitivity of power consumption changes to rewards, is the power consumption of distributed resource j in period t, α1 and α2 are coefficients, and are the incentive price and energy consumption change of distributed resource k in period t, is the power consumption of distributed resource k in period t, η k and γ k are the line loss and efficiency of charge / discharge respectively, and β1 and β are coefficients.

7. A storage medium storing at least one executable instruction, wherein the executable instruction enables a processor to execute operations corresponding to the incentive price determination method for distributed resource aggregation as described in any one of claims 1 to 5.

8. A computer device comprising: A processor, a memory, a communication interface and a communication bus, wherein the processor, the memory and the communication interface communicate with each other via the communication bus; The memory is used to store at least one executable instruction, and the executable instruction enables the processor to perform operations corresponding to the incentive price determination method applied to distributed resource aggregation as described in any one of claims 1-5.

Images