Virtual power plant-oriented intelligent bidding decision-making method and system for electric vehicle aggregator

Abstract

The invention discloses an electric vehicle aggregator intelligent bidding decision-making method and system for a virtual power plant, and relates to the technical field of power systems, and the method comprises the steps: employing an aggregation virtual energy storage model to evaluate the aggregation flexibility of an electric vehicle cluster at a first time level; converting the evaluated aggregation flexibility into an energy bidding interval of a second time level; constructing and initializing a bidding strategy model of the electric vehicle aggregator based on a reinforcement learning theory; determining a standardized energy bidding decision based on the bidding strategy model, performing affine transformation by using the energy bidding interval to obtain an energy market bidding decision, submitting the energy market bidding decision to the energy market and obtaining a decision reward; and training the bidding strategy model, and updating the bidding strategy. According to the method, time scale conversion is introduced, so that mismatching of minute-level aggregation modeling and hour-level bidding decision on the time scale is avoided; and the risk of price uncertainty faced by the electric vehicle aggregator during bidding decision making is overcome by optimizing the optimal bidding strategy.

Description

Technical Field

[0001] The present invention relates to the technical field of power systems, and in particular to an intelligent bidding decision-making method and system for electric vehicle aggregators for virtual power plants. Background Art

[0003] The electrification of transportation provides an important opportunity for energy efficiency and sustainable development. In this context, the number of electric vehicles is growing rapidly. However, large-scale EV charging poses great challenges to the operation of distribution networks, such as transformer capacity overload, voltage overlimit, and increased peak load. At the same time, EVs also demonstrate flexibility similar to battery energy storage. Utilizing and optimizing the overall flexibility of EV clusters has significant potential in mitigating the negative impacts brought about by them. Regarding the EV charging optimization problem, existing methods can be divided into individual optimization methods and aggregate optimization methods. Individual optimization methods directly optimize the charging power of a single EV. Although they can ensure the optimality of the optimization results, they face scalability issues in the charging management of large-scale EVs. When managing the charging behavior of a large number of EVs at the same time, the computational burden of the decision center is unbearable. In contrast, aggregate optimization is based on aggregate modeling of EV clusters, which reduces the decision space to one dimension. However, how to deal with the mismatch between aggregate modeling and EV aggregator bidding decisions in time scales, and how to optimize the intraday bidding decisions of EV aggregators when market prices are uncertain, remain unresolved issues. Summary of the invention

[0004] In view of the above-mentioned problems, the present invention is proposed.

[0005] Therefore, the technical problem solved by the present invention is: how to provide an intelligent bidding decision method and system for EV aggregators oriented to virtual power plants to deal with the mismatch between aggregation modeling and EV aggregators' bidding decisions in time scales, and how to optimize EV aggregators' intraday bidding decisions when there is market price uncertainty?

[0006] To solve the above technical problems, the present invention provides the following technical solutions: an intelligent bidding decision method for electric vehicle aggregators for virtual power plants, comprising: using an aggregated virtual energy storage model to conduct online evaluation of the aggregate flexibility of an electric vehicle cluster at a first time level; converting the evaluated aggregate flexibility of the electric vehicle cluster at the first time level into an energy bidding interval at a second time level; based on reinforcement learning theory, constructing and initializing a bidding strategy model for electric vehicle aggregators; determining a standardized energy bidding decision based on the bidding strategy model, and using the energy bidding interval to perform an affine transformation to obtain an energy market bidding decision, submitting the energy market bidding decision to the energy market and obtaining a decision reward; training the bidding strategy model based on the decision reward, and updating the bidding strategy of the electric vehicle aggregator.

[0007] As a preferred solution of the intelligent bidding decision method for electric vehicle aggregators facing virtual power plants described in the present invention, the online evaluation of the aggregated flexibility of electric vehicle clusters at the first time level using the aggregated virtual energy storage model includes the following steps: collecting information on vehicles currently connected to charging piles and vehicles scheduled to be connected to charging piles within the next hour, and forming a collection of these vehicles. Describing the collection using a virtual energy storage model Each electric car Charge and discharge flexibility:

[0008]

[0009]

[0010]

[0011]

[0012]

[0013] Among them, τ represents the time step at the minute scale; h represents the time step at the hour scale; D represents the number of minute-level time steps contained in each hour; e i (τ) and p i (τ) represent the battery charge and power of the i-th electric vehicle at time τ; Δe i (τ) represents the battery charge of the i-th electric vehicle when it leaves or enters the charging station; In order to meet the charging demand of electric vehicles, the minimum power that the i-th electric vehicle needs to reach at time τ; Indicates the maximum chargeable capacity of the battery; and They represent the maximum discharge power and maximum charging power of the i-th electric vehicle respectively.

[0014] For the collection The virtual energy storage model parameters of electric vehicles are aggregated to obtain the minute-scale aggregated virtual energy storage model of the cluster:

[0015]

[0016]

[0017]

[0018]

[0019]

[0020]

[0021] Where ΔE(τ) represents the change in the amount of electricity in the aggregated virtual energy storage at time τ caused by the electric vehicle leaving or entering the charging station; E + / (-) (τ) represent the maximum and minimum energy limits of the aggregated virtual energy storage at time τ; P + / (-) (τ) represent the maximum and minimum power limits of the aggregated virtual energy storage at time τ respectively.

[0022] As a preferred solution of the intelligent bidding decision method for electric vehicle aggregators facing virtual power plants described in the present invention, wherein: the conversion of the evaluated aggregate flexibility of the electric vehicle cluster at the first time level into the energy bidding interval at the second time level includes the following steps: based on the minute-scale aggregated virtual energy storage model constraints, with the goal of minimizing the charging power of the hour-scale aggregated virtual energy storage, the lower bound of the hour-level energy bidding interval is optimized:

[0023]

[0024] E(τ+1)=μE(τ)+ΔE(τ)+P h+1 Δτ,

[0025]

[0026] E - (τ)≤E(τ)≤E+(τ),

[0027]

[0028] E - (τ)≤P h+1 ≤E+(τ),

[0029]

[0030] in, represents the lower bound of the energy bidding interval of the electric vehicle aggregator at h+1 hour; E(τ) represents the storage energy of the aggregated virtual energy storage at time τ; P h+1 It represents the charging power maintained by the aggregated virtual energy storage within h+1 hour.

[0031] Based on the constraints of the minute-scale aggregated virtual energy storage model, with the goal of maximizing the charging power of the hour-scale aggregated virtual energy storage, the upper bound of the hour-scale energy bidding interval is optimized:

[0032]

[0033] E(τ+1)=μE(τ)+ΔE(τ)+P h+1 Δτ,

[0034]

[0035] E - (τ)≤E(τ)≤E+(τ),

[0036]

[0037] E - (τ)≤P h+1 ≤E+(τ),

[0038]

[0039] in, represents the upper bound of the energy bidding interval of the electric vehicle aggregator at h+1 hour.

[0040] As a preferred solution of the intelligent bidding decision-making method for electric vehicle aggregators for virtual power plants described in the present invention, wherein: the bidding strategy model includes modeling the electric vehicle aggregator as an intelligent agent; wherein the intelligent agent includes a reward evaluator network and an actor network; the reward evaluator network is used to guide the training of the actor network, and the actor network represents the energy bidding strategy of the electric vehicle aggregator, and is used to output the standardized bidding decision of the electric vehicle aggregator.

[0041] The agent's observation value s at hour h h Including the first h,D The amount of virtual energy storage E(τ h,D ), the energy market price λ at hour h h , the upper bound of the energy bidding interval for the h+1 hour and the lower bound It is expressed as:

[0042]

[0043] The action value a of the agent at hour h h represents the normalized energy bidding amount of the EV aggregator at hour h+1.

[0044] The agent's reward value r at hour h h Defined as the energy market revenue earned by the EV aggregator at hour h+1.

[0045] As a preferred solution of the intelligent bidding decision method for electric vehicle aggregators facing virtual power plants described in the present invention, wherein: determining the standardized energy bidding decision based on the bidding strategy model, and using the energy bidding interval to perform affine transformation to obtain the energy market bidding decision, submitting the energy market bidding decision to the energy market and obtaining the decision reward includes: inputting the observation value of the intelligent agent in the environment into the actor network of the intelligent agent to obtain the action value a h , i.e., standardized energy bidding decision.

[0046] Energy-based bidding interval For standardized energy bidding decision h Perform an affine transformation operation to obtain the energy market bidding decision:

[0047]

[0048] in, represents the energy bidding amount of the EV aggregator at hour h+1.

[0049] Submit energy market bidding decisions to the energy market to obtain decision rewards r h :

[0050]

[0051] As a preferred solution of the intelligent bidding decision-making method for electric vehicle aggregators for virtual power plants described in the present invention, the bidding strategy model is trained based on the decision reward, and updating the bidding strategy of the electric vehicle aggregator includes the following steps: storing the action value, observation value, and reward value generated by the intelligent agent as a sample in the experience replay pool; randomly extracting a batch of samples from the experience replay pool, and updating the evaluator network of the intelligent agent; using the evaluator network to estimate the action advantage value of the intelligent agent, and using it to update the actor network, that is, the updated bidding strategy of the electric vehicle aggregator.

[0052] As a preferred solution of the intelligent bidding decision-making method for electric vehicle aggregators for virtual power plants described in the present invention, wherein: the first time level is the minute level of the next hour; the second time level is the next hour.

[0053] To further solve the above technical problems, the present invention provides the following technical solutions: an intelligent bidding decision system for electric vehicle aggregators for virtual power plants, comprising: an evaluation module, used to use an aggregated virtual energy storage model to perform online evaluation on the aggregate flexibility of an electric vehicle cluster at a first time level; a conversion module, used to use a time scale conversion to convert the evaluated aggregate flexibility of the electric vehicle cluster at the first time level into an energy bidding interval at a second time level; a modeling module, used to construct and initialize a bidding strategy model for an electric vehicle aggregator based on reinforcement learning theory; an execution module, used to determine a standardized energy bidding decision based on the bidding strategy model, and use the energy bidding interval to perform an affine transformation to obtain an energy market bidding decision, submit the energy market bidding decision to the energy market and obtain a decision reward; an optimization module, used to use a proximal strategy optimization algorithm to train the bidding strategy model based on the decision reward and update the bidding strategy of the electric vehicle aggregator.

[0054] A computer device includes a memory and a processor, wherein the memory stores a computer program, and is characterized in that when the processor executes the computer program, the steps of the intelligent bidding decision-making method for electric vehicle aggregators for virtual power plants as described above are implemented.

[0055] A computer-readable storage medium having a computer program stored thereon, characterized in that when the computer program is executed by a processor, the steps of the intelligent bidding decision-making method for electric vehicle aggregators for virtual power plants as described above are implemented.

[0056] The beneficial effects of the present invention are as follows: First, the present invention proposes a time scale conversion method when evaluating the flexibility of electric vehicle cluster aggregation, so as to eliminate the time scale mismatch problem between the minute-level aggregation modeling of electric vehicle clusters and the hour-level bidding decision of electric vehicle aggregators; secondly, the electric vehicle aggregator bidding decision optimization algorithm proposed by the present invention is based on the proximal strategy optimization algorithm, so there is no need to rely on the prediction of intraday energy market electricity prices. The optimal bidding strategy is learned through a data-driven method, which effectively overcomes the price uncertainty risk faced by electric vehicle aggregators when making bidding decisions. BRIEF DESCRIPTION OF THE DRAWINGS

[0057] In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the accompanying drawings required for use in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other accompanying drawings can be obtained based on these accompanying drawings without paying creative work.

[0058] Figure 1 An overall flow chart of an intelligent bidding decision-making method for electric vehicle aggregators for virtual power plants provided by one embodiment of the present invention.

[0059] Figure 2 A computer device diagram of the intelligent bidding decision-making method for electric vehicle aggregators for virtual power plants of the present invention. DETAILED DESCRIPTION

[0060] In order to make the above-mentioned purposes, features and advantages of the present invention more obvious and easy to understand, the specific implementation methods of the present invention are described in detail below in conjunction with the drawings of the specification. Obviously, the described embodiments are part of the embodiments of the present invention, but not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary persons in the art without creative work should fall within the scope of protection of the present invention.

[0061] In the following description, many specific details are set forth to facilitate a full understanding of the present invention, but the present invention may also be implemented in other ways different from those described herein, and those skilled in the art may make similar generalizations without violating the connotation of the present invention. Therefore, the present invention is not limited to the specific embodiments disclosed below.

[0062] Example 1

[0063] Reference Figure 1 , as an embodiment of the present invention, provides an intelligent bidding decision method for electric vehicle aggregators oriented to virtual power plants.

[0064] The existing intelligent bidding decision-making methods for electric vehicle aggregators of virtual power plants (mainly individual optimization methods and aggregate optimization methods) have the following main problems: individual optimization methods directly optimize the charging power of a single electric vehicle. Although they can ensure the optimality of the optimization results, they face scalability issues in the charging management of large-scale electric vehicles. When managing the charging behavior of a large number of electric vehicles at the same time, the computational burden of the decision center is unbearable. In contrast, aggregate optimization is based on aggregate modeling of electric vehicle clusters, reducing the decision space to one dimension. However, how to deal with the mismatch between aggregate modeling and bidding decisions of electric vehicle aggregators in terms of time scale, and how to optimize the intraday bidding decisions of electric vehicle aggregators when market prices are uncertain, remain unresolved issues.

[0065] The present application provides an effective solution to the above-mentioned problems. Next, multiple embodiments will be combined to explain in detail how to implement the industrial data priority calculation method based on correlation relationship and sequence processing.

[0066] Figure 1 The overall flow chart of the intelligent bidding decision-making method for electric vehicle aggregators oriented to virtual power plants is shown, including:

[0067] S1: Online evaluation of the aggregated flexibility of electric vehicle clusters at the first time level using an aggregated virtual energy storage model;

[0068] S2: converting the evaluated aggregate flexibility of the electric vehicle cluster at the first time level into the energy bidding interval at the second time level;

[0069] S3: Based on reinforcement learning theory, build and initialize the bidding strategy model of electric vehicle aggregators;

[0070] S4: Determine the standardized energy bidding decision based on the bidding strategy model, and use the energy bidding interval to perform affine transformation to obtain the energy market bidding decision, submit the energy market bidding decision to the energy market and obtain the decision reward;

[0071] S5: Train the bidding strategy model based on the decision reward and update the bidding strategy.

[0072] Next, we will introduce the contents of each step one by one.

[0073] S1: Online evaluation of the aggregated flexibility of electric vehicle clusters at the first time level using an aggregated virtual energy storage model.

[0074] It should be noted that, in this embodiment, the first time level may be the next hour and minute level.

[0075] S1.1: Collect information about vehicles currently connected to charging piles and scheduled to be connected to charging piles within the next hour, and form a collection of these vehicles For each electric car in the set Use virtual energy storage model to describe its charging and discharging flexibility:

[0076]

[0077]

[0078]

[0079] Where: τ represents the time step at the minute scale; h represents the time step at the hour scale; D represents the number of minute-level time steps contained in each hour; e i (τ) and p i (τ) represent the battery charge and power of the i-th electric vehicle at time τ; Δe i (τ) represents the battery charge of the i-th electric vehicle when it leaves or enters the charging station; In order to meet the charging demand of electric vehicles, the minimum power that the i-th electric vehicle needs to reach at time τ; Indicates the maximum chargeable capacity of the battery; and They represent the maximum discharge power and maximum charging power of the i-th electric vehicle respectively.

[0080] S1.2: Pair collection The virtual energy storage model parameters of the electric vehicles in the cluster are aggregated to obtain the minute-scale aggregated virtual energy storage model of the cluster:

[0081]

[0082]

[0083]

[0084] Where: ΔE(τ) represents the change in the amount of electricity in the aggregated virtual energy storage at time τ caused by the electric vehicle leaving or entering the charging station; E + / (-) (τ) represent the maximum and minimum energy limits of the aggregated virtual energy storage at time τ; P + / (-) (τ) represent the maximum and minimum power limits of the aggregated virtual energy storage at time τ respectively.

[0085] S2: The aggregate flexibility of the electric vehicle cluster evaluated in step S1 at the first time level is converted into an energy bidding interval at the second time level by using time scale conversion.

[0086] It should be noted that, in this embodiment, the second time level may be the next hour.

[0087] S2.1: Based on the constraints of the minute-scale aggregated virtual energy storage model, with the goal of minimizing the charging power of the hour-scale aggregated virtual energy storage, the lower bound of the hour-scale energy bidding interval is optimized:

[0088]

[0089] E(τ+1)=μE(τ)+ΔE(τ)+P h+1 Δτ, 8)

[0090]

[0091]

[0092]

[0093] Where: represents the lower bound of the energy bidding interval of the electric vehicle aggregator at h+1 hour; E(τ) represents the storage energy of the aggregated virtual energy storage at time τ; P h+1 It represents the charging power maintained by the aggregated virtual energy storage within h+1 hour.

[0094] S2.2: Based on the constraints of the minute-scale aggregated virtual energy storage model, with the goal of maximizing the charging power of the hour-scale aggregated virtual energy storage, the upper bound of the hour-scale energy bidding interval is optimized:

[0095]

[0096]

[0097]

[0098]

[0099]

[0100] Where: represents the upper bound of the energy bidding interval of the electric vehicle aggregator at h+1 hour.

[0101] S3: Based on reinforcement learning theory, build and initialize the bidding strategy model of electric vehicle aggregators.

[0102] S3.1: Agent: The electric vehicle aggregator is modeled as an agent, which consists of a reward evaluator network and an actor network, wherein the reward evaluator network is used to guide the training of the actor network, and the actor network represents the energy bidding strategy of the electric vehicle aggregator and is used to output the standardized bidding decision of the electric vehicle aggregator.

[0103] S3.2: Observation value: The observation value s of the agent at hour h h Including the first h,D The amount of virtual energy storage E(τ h,D ), the energy market price λ at hour h h , the upper bound of the energy bidding interval for the h+1 hour and the lower bound It can be expressed as:

[0104]

[0105] S3.3: Action value: The action value a of the agent at hour h h represents the normalized energy bidding amount of the EV aggregator at hour h+1.

[0106] S3.4: Reward value: The reward value r of the agent at hour h h Defined as the energy market revenue earned by the EV aggregator at hour h+1.

[0107] S4: Determine the standardized energy bidding decision based on the bidding strategy model, and use the energy bidding interval to perform affine transformation to obtain the energy market bidding decision, submit the energy market bidding decision to the energy market and obtain the decision reward.

[0108] S4.1: Input the observation value of the agent in the environment into the agent's actor network to obtain the action value a h , i.e., standardized energy bidding decision.

[0109] S4.2: Energy-based bidding interval For standardized energy bidding decision h Perform an affine transformation operation to obtain the energy market bidding decision:

[0110]

[0111] Where: represents the energy bidding amount of the EV aggregator at hour h+1.

[0112] S4.3: Submit energy market bidding decisions to the energy market to obtain decision rewards r h :

[0113]

[0114] S5: Use the proximal policy optimization algorithm to train the bidding strategy model based on decision rewards and update the bidding strategy of the EV aggregator.

[0115] S5.1: The action value, observation value, and reward value generated by the agent are stored as a sample in the experience replay pool.

[0116] S5.2: Randomly extract a batch of samples from the experience replay pool and update the agent's evaluator network.

[0117] S5.3: The evaluator network is used to estimate the action advantage value of the agent and used to update the actor network, that is, the updated bidding strategy of the electric vehicle aggregator.

[0118] It should be noted that the present invention is cyclically optimized, so after updating the bidding strategy of the electric vehicle aggregator in step S5, it returns to step S1 for the next round of optimization, so that the whole process forms a closed-loop system to continuously improve the bidding strategy of the electric vehicle aggregator.

[0119] In summary, the present invention proposes a time scale conversion method when evaluating the flexibility of electric vehicle cluster aggregation, so as to eliminate the time scale mismatch between the minute-level aggregation modeling of electric vehicle clusters and the hour-level bidding decision of electric vehicle aggregators; the present invention proposes an electric vehicle aggregator bidding decision optimization algorithm based on proximal strategy optimization. This method does not need to rely on the prediction of intraday energy market electricity prices, and learns the optimal bidding strategy through a data-driven method, which effectively overcomes the price uncertainty risk faced by electric vehicle aggregators when making bidding decisions.

[0120] Example 2

[0121] As an embodiment of the present invention, an intelligent bidding decision system for electric vehicle aggregators for virtual power plants is provided, which is mainly composed of an evaluation module, a conversion module, a modeling module, an execution module, and an optimization module.

[0122] The evaluation module is used to perform online evaluation of the aggregated flexibility of the electric vehicle cluster at the first time level using an aggregated virtual energy storage model.

[0123] The conversion module is used to convert the evaluated aggregate flexibility of the electric vehicle cluster at the first time level into the energy bidding interval at the second time level by using time scale conversion.

[0124] The modeling module is used to build and initialize the bidding strategy model of electric vehicle aggregators based on reinforcement learning theory.

[0125] The execution module is used to determine the standardized energy bidding decision based on the bidding strategy model, and use the energy bidding interval to perform affine transformation to obtain the energy market bidding decision, submit the energy market bidding decision to the energy market and obtain the decision reward.

[0126] The optimization module is used to train the bidding strategy model based on decision rewards using the proximal policy optimization algorithm and update the bidding strategy of the electric vehicle aggregator.

[0127] Example 3

[0128] Reference Figure 2 , is an embodiment of the present invention, which is different from the previous embodiment in that:

[0129] If the functions are implemented in the form of software functional units and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, or the part that contributes to the prior art or the part of the technical solution, can be embodied in the form of a software product. The computer software product is stored in a storage medium, including several instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to perform all or part of the steps of the methods described in each embodiment of the present invention. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), disk or optical disk, etc., which can store program codes.

[0130] The logic and / or steps represented in the flowchart or otherwise described herein, for example, can be considered as an ordered list of executable instructions for implementing logical functions, and can be embodied in any computer-readable medium for use by an instruction execution system, device or apparatus (such as a computer-based system, a system including a processor, or other system that can fetch instructions from an instruction execution system, device or apparatus and execute instructions), or in conjunction with such instruction execution systems, devices or apparatuses. For the purposes of this specification, "computer-readable medium" can be any device that can contain, store, communicate, propagate or transmit a program for use by an instruction execution system, device or apparatus, or in conjunction with such instruction execution systems, devices or apparatuses.

[0131] More specific examples of computer-readable media (a non-exhaustive list) include the following: an electrical connection with one or more wires (electronic device), a portable computer disk case (magnetic device), a random access memory (RAM), a read-only memory (ROM), an erasable and programmable read-only memory (EPROM or flash memory), an optical fiber device, and a portable compact disk read-only memory (CDROM). In addition, the computer-readable medium may even be a paper or other suitable medium on which the program is printed, since the program may be obtained electronically, for example, by optically scanning the paper or other medium, followed by editing, deciphering or, if necessary, processing in another suitable manner, and then stored in a computer memory.

[0132] It should be understood that the various parts of the present invention can be implemented by hardware, software, firmware or a combination thereof. In the above-mentioned embodiments, a plurality of steps or methods can be implemented by software or firmware stored in a memory and executed by a suitable instruction execution system. For example, if implemented by hardware, as in another embodiment, it can be implemented by any one of the following technologies known in the art or their combination: a discrete logic circuit having a logic gate circuit for implementing a logic function for a data signal, a dedicated integrated circuit having a suitable combination of logic gate circuits, a programmable gate array (PGA), a field programmable gate array (FPGA), etc.

[0133] Example 4

[0134] As an embodiment of the present invention, an intelligent bidding decision method for electric vehicle aggregators for virtual power plants is provided. In order to verify the beneficial effects of the present invention, scientific demonstration is carried out through economic benefit calculation and simulation experiments.

[0135] Simulate a large aggregator managing 50,000 electric vehicles bidding in the intraday electricity market over a period of 180 days (about 6 months). Compare the following three scenarios:

[0136] The present invention: An intelligent bidding decision-making method for electric vehicle aggregators for virtual power plants;

[0137] Method A: Deterministic optimization method based on mixed integer linear programming (MILP);

[0138] Method B: Heuristic rule approach based on historical average prices.

[0139] Table 1: Overall performance comparison

[0140] Evaluation Metrics Method of the present invention Method A Method B Average daily bidding income (10,000 yuan) 67.89 62.45 58.23 Standard deviation of income (ten thousand yuan) 3.21 5.67 7.32 Total income for half a year (100 million yuan) 12.22 11.24 10.48

[0141] Evaluation Metrics Method of the present invention Method A Method B Average calculation time (seconds) 0.83 15.67 0.21 Price prediction accuracy (%) 94.7 87.2 80.5 Peak-to-valley difference improvement rate (%) 19.3 16.1 12.7 User satisfaction (1-10 points) 8.9 7.8 7.1 Grid dispatch friendliness (1-10 points) 9.1 8.2 7.4

[0142] As shown in Table 1, the present invention performs best in all key indicators. The average daily bidding income is 8.7% higher than that of method A and 16.6% higher than that of method B. At the same time, the standard deviation of the income is the smallest, indicating that it has the highest stability. In terms of computational efficiency, although it is not as fast as method B, it is much better than method A, while maintaining the highest price prediction accuracy and peak-to-valley difference improvement rate. User satisfaction and grid friendliness are also the highest, showing the comprehensive advantages of this method.

[0143] Table 2: Average daily bidding income in different seasons (10,000 yuan)

[0144] season Method of the present invention Method A Method B spring 65.23 60.12 56.78 summer 72.45 65.89 61.34 autumn 66.78 61.56 57.45 winter 67.10 62.23 57.35

[0145] As shown in Table 2, the present invention maintains the highest profit level in all seasons. In particular, in summer, the profit reaches the highest level (724,500 yuan), which is much higher than other methods. This may be due to the large fluctuations in electricity demand in summer, and the present invention can better adapt to such changes. It is worth noting that the profit fluctuations of the present invention between seasons are relatively small, showing its good adaptability.

[0146] It should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention rather than to limit it. Although the present invention has been described in detail with reference to the preferred embodiments, those skilled in the art should understand that the technical solutions of the present invention may be modified or replaced by equivalents without departing from the spirit and scope of the technical solutions of the present invention, which should all be included in the scope of the claims of the present invention.

Claims

1. An intelligent bidding decision-making method for electric vehicle aggregators oriented to virtual power plants, characterized in that: include: Online evaluation of the aggregated flexibility of electric vehicle clusters at the first time level using an aggregated virtual energy storage model; The evaluated aggregate flexibility of the electric vehicle cluster at the first time level is converted into an energy bidding interval at the second time level; Based on reinforcement learning theory, a bidding strategy model for electric vehicle aggregators is constructed and initialized; Determine a standardized energy bidding decision based on a bidding strategy model, perform an affine transformation using the energy bidding interval to obtain an energy market bidding decision, submit the energy market bidding decision to the energy market and obtain a decision reward; The bidding strategy model is trained based on the decision reward to update the bidding strategy of the electric vehicle aggregator.

2. The intelligent bidding decision-making method for electric vehicle aggregators facing virtual power plants as claimed in claim 1, characterized in that: The online evaluation of the aggregated flexibility of the electric vehicle cluster at the first time level using the aggregated virtual energy storage model comprises the following steps: Collect information about vehicles currently connected to charging piles and scheduled to be connected to charging piles within the next hour, and form a collection of these vehicles Describing the collection using a virtual energy storage model Each electric car Charge and discharge flexibility: Among them, τ represents the time step at the minute scale; h represents the time step at the hour scale; D represents the number of minute-level time steps contained in each hour; e i (τ) and p i (τ) represent the battery charge and power of the i-th electric vehicle at time τ; Δe i (τ) represents the battery charge of the i-th electric vehicle when it leaves or enters the charging station; In order to meet the charging demand of electric vehicles, the minimum power that the i-th electric vehicle needs to reach at time τ; Indicates the maximum chargeable capacity of the battery; and They represent the maximum discharge power and maximum charging power of the i-th electric vehicle respectively; For the collection The virtual energy storage model parameters of electric vehicles are aggregated to obtain the minute-scale aggregated virtual energy storage model of the cluster: Where ΔE(τ) represents the change in the amount of electricity in the aggregated virtual energy storage at time τ caused by the electric vehicle leaving or entering the charging station; E + / (-) (τ) represent the maximum and minimum energy limits of the aggregated virtual energy storage at time τ; P + / (-) (τ) represent the maximum and minimum power limits of the aggregated virtual energy storage at time τ respectively.

3. The intelligent bidding decision-making method for electric vehicle aggregators facing virtual power plants as claimed in claim 2 is characterized by: The step of converting the evaluated aggregate flexibility of the electric vehicle cluster at the first time level into the energy bidding interval at the second time level comprises the following steps: Based on the constraints of the minute-scale aggregated virtual energy storage model, with the goal of minimizing the charging power of the hour-scale aggregated virtual energy storage, the lower bound of the hour-scale energy bidding interval is optimized: E(z+1)= l uE(z)+nE(z)+P h+1 AT, E - (τ)≤E(τ)≤E + (t), E - (τ)≤P h+1 ≤E + (t), in, represents the lower bound of the energy bidding interval of the electric vehicle aggregator at h+1 hour; E(τ) represents the storage energy of the aggregated virtual energy storage at time τ; P h+1 It represents the charging power maintained by the aggregated virtual energy storage in h+1 hour; Based on the constraints of the minute-scale aggregated virtual energy storage model, with the goal of maximizing the charging power of the hour-scale aggregated virtual energy storage, the upper bound of the hour-scale energy bidding interval is optimized: E(τ+1)=μE(τ)+ΔE(τ)+P h+1 Dt, E - (τ)≤E(τ)≤E + (t), E - (τ)≤P h+1 ≤E + (t), in, represents the upper bound of the energy bidding interval of the electric vehicle aggregator at h+1 hour.

4. The intelligent bidding decision-making method for electric vehicle aggregators facing virtual power plants as claimed in claim 3 is characterized by: The bidding strategy model includes: Modeling EV aggregators as intelligent agents; The intelligent agent includes a reward evaluator network and an actor network; the reward evaluator network is used to guide the actor network training, and the actor network represents the energy bidding strategy of the electric vehicle aggregator and is used to output the standardized bidding decision of the electric vehicle aggregator; The agent's observation value s at hour h h Including the first h,D The amount of virtual energy storage E(τ h,D ), the energy market price λ at hour h h , the upper bound of the energy bidding interval for the h+1th hour and the lower bound It is expressed as: The action value a of the agent at hour h h represents the normalized energy bid quantity of the EV aggregator at hour h+1; The agent's reward value r at hour h h Defined as the energy market revenue earned by the EV aggregator at hour h+1.

5. The intelligent bidding decision-making method for electric vehicle aggregators facing virtual power plants as claimed in claim 4, characterized in that: Determining a standardized energy bidding decision based on a bidding strategy model, and using the energy bidding interval to perform an affine transformation to obtain an energy market bidding decision, submitting the energy market bidding decision to the energy market and obtaining a decision reward include: Input the observation value in the environment to the agent's actor network and obtain the action value a h , i.e., standardized energy bidding decisions; Energy-based bidding interval For standardized energy bidding decision h Perform an affine transformation operation to obtain the energy market bidding decision: in, represents the energy bid amount of the EV aggregator at hour h+1; Submit energy market bidding decisions to the energy market to obtain decision rewards r h :

6. The intelligent bidding decision-making method for electric vehicle aggregators facing virtual power plants as claimed in claim 5, characterized in that: The step of training the bidding strategy model based on the decision reward and updating the bidding strategy of the electric vehicle aggregator comprises the following steps: The action value, observation value, and reward value generated by the agent are stored as a sample in the experience replay pool; Randomly extract a batch of samples from the experience replay pool and update the agent's evaluator network; The evaluator network is used to estimate the action advantage value of the agent and is used to update the actor network, i.e., the updated bidding strategy of the EV aggregator.

7. The intelligent bidding decision-making method for electric vehicle aggregators facing virtual power plants as claimed in claim 6 is characterized by: The first time level is the minute level of the next hour; the second time level is the next hour.

8. A system using the intelligent bidding decision method for electric vehicle aggregators for virtual power plants as described in any one of claims 1 to 7, characterized in that: include: an evaluation module for online evaluation of the aggregated flexibility of the electric vehicle cluster at the first time level using an aggregated virtual energy storage model; A conversion module, used for converting the evaluated aggregate flexibility of the electric vehicle cluster at the first time level into an energy bidding interval at the second time level by using a time scale conversion; A modeling module, which is used to build and initialize the bidding strategy model of EV aggregators based on reinforcement learning theory; An execution module is used to determine a standardized energy bidding decision based on a bidding strategy model, and use the energy bidding interval to perform an affine transformation to obtain an energy market bidding decision, submit the energy market bidding decision to the energy market and obtain a decision reward; An optimization module is used to train the bidding strategy model based on the decision reward using a proximal strategy optimization algorithm to update the bidding strategy of the electric vehicle aggregator.

9. A computer device comprising a memory and a processor, wherein the memory stores a computer program, wherein: When the processor executes the computer program, the steps of the intelligent bidding decision-making method for electric vehicle aggregators for virtual power plants described in any one of claims 1 to 7 are implemented.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the intelligent bidding decision-making method for electric vehicle aggregators for virtual power plants described in any one of claims 1 to 7 are implemented.

Images