Electric vehicle aggregation regulation and control method and system based on multi-agent distributed control

Abstract

The invention discloses an electric vehicle aggregation regulation and control method and system based on multi-agent distributed control, and belongs to the technical field of power distribution optimization. The method comprises the following specific steps: acquiring charging station data and electric vehicle charging data, and constructing a multi-agent charging station network; and each intelligent agent communicates with the electric vehicle aggregation regulation and control model to implement the solved aggregation regulation and control strategy. Meanwhile, the invention discloses a system based on the method, the electric vehicle aggregation regulation and control method and system based on multi-agent distributed control are adopted, unified aggregation regulation and control are carried out on electric vehicle charging behaviors in a region by constructing a multi-agent charging station network, and a cost reward value is introduced into a reward function; and meanwhile, a charging completion reward value, a fair reward value and a power grid stability maintaining reward value are introduced, so that the fairness and efficiency of electric vehicle access are ensured, and the stability of the power grid is considered.

Description

Technical Field

[0001] The present invention relates to the technical field of power distribution optimization, and in particular to an electric vehicle aggregation control method and system based on multi-agent distributed control. Background Art

[0002] The EV aggregation control method refers to the centralized management and coordination of multiple EVs and their related resources, such as charging stations, battery management systems, etc. The main attraction of EVs is their zero-emission characteristics, which greatly helps promote environmental sustainability, so the number of EVs is increasing year by year. As the demand for EVs increases, the demand for intelligent solutions to manage the dynamic characteristics of EV charging infrastructure is also increasing. Traditional EV charging management methods often have difficulty adapting to the dynamic nature of EV usage patterns, fluctuating electricity prices, and changing user behavior. This leads to inefficient resource utilization and low reliability, which poses challenges to both EV owners and charging infrastructure operators. The management of dynamic EV charging demand poses significant challenges due to the variability of factors such as time of day, location, and user behavior. Charging demand surges during peak hours, especially in the morning and evening, as commuters charge their vehicles before and after work, while commercial areas and shopping malls have higher charging demand during the day compared to residential areas. Effective management of these fluctuations is critical to preventing grid overload, minimizing waiting time, and ensuring fair distribution of charging resources. The impact of these demand fluctuations extends to grid stability, operational efficiency, and user convenience, requiring advanced technical solutions such as smart charging technology, demand response systems, and predictive analysis. These innovations enable charging stations to adjust operations in real time based on electricity prices, grid capacity, and user preferences, thereby improving overall system reliability. For example, the invention patent with publication number CN117374932A discloses a large-scale electric vehicle dual-layer aggregation control system and method for virtual power plants. Based on the large-scale electric vehicle aggregation control strategy for different charging and discharging price scenarios, a large-scale electric vehicle overall-local dual-layer aggregation control process for virtual power plants is proposed, and the optimal charging and discharging strategy for electric vehicles is obtained for 96 time periods throughout the day. However, further improvements are needed in operational efficiency to cope with the increasingly complex charging environment. Traditional methods for managing electric vehicle charging networks can be roughly divided into centralized and decentralized methods. Centralized systems are usually unable to expand effectively to adapt to the increasing number of electric vehicles, while decentralized systems are difficult to coordinate and optimize across networks. Summary of the invention

[0003] The purpose of the present invention is to provide an electric vehicle aggregation control method and system based on multi-agent distributed control to solve the above technical problems.

[0004] To achieve the above object, the present invention provides an electric vehicle aggregation control method based on multi-agent distributed control, and the specific steps are as follows:

[0005] Step S1: Obtain data of each charging station in a set area, and construct a multi-agent charging station network based on the data of each charging station;

[0006] Step S2: acquiring the charging data of electric vehicles in each charging station and determining the charging constraint function of the electric vehicles;

[0007] Step S3: The multi-agent charging station network solves the electric vehicle charging aggregation control model based on the reward function through a deep reinforcement learning algorithm, and each agent communicates with the electric vehicle aggregation control model to implement the solved aggregation control strategy.

[0008] Preferably, in step S1, the multi-agent charging station network includes an agent corresponding to each charging station and an aggregate control terminal, each agent communicates with the aggregate control terminal, and each agent communicates with each other.

[0009] Preferably, in step S1, the charging station data is as follows:

[0010]

[0011] Among them, r i is the completion charging coefficient of the i-th charging station, Q i is the queue length of the i-th charging station, Q t is the target queue length of the i-th charging station, p i is the energy price of the i-th charging station, e i is the energy consumption of the i-th charging station, ΔP i is the redundant power of the i-th charging station, P i,g is the power purchased by the i-th charging station, L i is the load of the i-th charging station, g i,max The maximum permissible load of the i-th charging station, is the maximum total discharge power of the i-th charging station, is the real-time aggregated energy value of the i-th charging station, is the lower limit of the aggregated energy of the i-th charging station, is the lower limit of the aggregated energy of the i-th charging station.

[0012] Preferably, the electric vehicle charging data is as follows:

[0013]

[0014] in, is the charging power of the jth electric vehicle in the i-th charging station, is the reactive power absorbed or released by the jth electric vehicle in the i-th charging station during charging, is the maximum reactive power absorbed or released by the jth electric vehicle in the i-th charging station during charging, is the maximum power capacity of the charger for the j-th electric vehicle in the i-th charging station.

[0015] Preferably, in step S2, the charging constraint function of the electric vehicle in the i-th charging station is constrained from the charging power, the reactive power during charging, and the energy during charging. The charging constraint function of the electric vehicle in the i-th charging station is specifically as follows:

[0016]

[0017] Among them, k is the number of electric vehicles in the i-th charging station;

[0018] is the sum of the maximum reactive power absorbed or released by electric vehicles in the i-th charging station during charging. The calculation formula is as follows:

[0019]

[0020] is the sum of reactive power absorbed or released by all electric vehicles in the i-th charging station, and the calculation formula is as follows:

[0021]

[0022] is the sum of the real-time aggregated energy values ​​of all charging stations at time t, is the sum of the real-time aggregated energy values ​​of all charging stations at time t-1, and They are the lower limit of total aggregate energy and the upper limit of total aggregate energy respectively. The lower limit of total aggregate energy is the sum of the lower limits of aggregate energy of all charging stations, and the upper limit of total aggregate energy is the sum of the upper limits of aggregate energy of all charging stations. The calculation formula is as follows:

[0023]

[0024] Preferably, the electric vehicle charging aggregation control model based on the reward function is as follows:

[0025] Y=max{αR c (s,a)+βR f (s,a)+γR s (s,a)+λR g (s,a)}

[0026] Among them, Y is the electric vehicle charging aggregation control model, max{} is the maximum value function, s is the current state, a is the action vector, R c (s,a) is the reward value for completing charging; R f (s,a) is the fair reward value, R s (s,a) is the cost reward value, R g (s,a) is the reward value for maintaining grid stability, α, β, γ and λ are the weights of the reward value for completing charging, the reward value for fairness, the reward value for cost and the reward value for maintaining grid stability, respectively.

[0027] Preferably, the calculation formula for completing the charging reward value is as follows:

[0028]

[0029] Where N is the number of charging stations, r i The values ​​are as follows:

[0030] When charging is completed, the value is 1, and when charging is not completed, the value is -1.

[0031] Preferably, the fair reward value calculation formula is as follows:

[0032]

[0033] The cost reward value calculation formula is as follows:

[0034]

[0035] The cost reward value calculation formula must also meet the following conditions:

[0036]

[0037] Where ΔP i is the redundant power of the i-th charging station, P i,g is the power purchased by the i-th charging station.

[0038] Preferably, the calculation formula for maintaining grid stability reward value is as follows:

[0039]

[0040] Among them, L i is the load of the i-th charging station, g max The maximum allowable load in the area is the sum of the maximum allowable loads of all charging stations in the area. The calculation formula is as follows:

[0041]

[0042] Among them, μ is the total load weight coefficient, and ν is the load extreme value weight coefficient.

[0043] A system based on the above-mentioned electric vehicle aggregation control method based on multi-agent distributed control includes:

[0044] A data acquisition module is used to acquire charging station data and electric vehicle charging data in each charging station;

[0045] A constraint module, used to determine the electric vehicle charging constraint function;

[0046] Several intelligent agents can realize interactive learning of intelligent agent detection and real-time aggregation and control of terminal optimization strategies;

[0047] Aggregation control terminal, which makes an aggregation control strategy based on maximizing rewards and considering the action strategy of the intelligent agent;

[0048] Several intelligent agents interact with each other, several intelligent agents interact with the aggregate control terminal, the constraint module and the intelligent agent communicate with the data acquisition module, and the constraint module communicates with the aggregate control terminal.

[0049] Therefore, the present invention adopts the above-mentioned electric vehicle aggregation control method and system based on multi-agent distributed control, which has the following beneficial effects:

[0050] (1) By building a multi-agent charging station network, the charging behavior of electric vehicles in the region is uniformly aggregated and regulated, thereby improving charging efficiency and reducing energy costs while ensuring the stability of the power grid.

[0051] (2) The reward function not only introduces the cost reward value, but also introduces the charging completion reward value, the fairness reward value, and the grid stability maintenance reward value, which ensures the fairness and efficiency of electric vehicle access and takes into account the stability of the grid.

[0052] The technical solution of the present invention is further described in detail below through the accompanying drawings and embodiments. BRIEF DESCRIPTION OF THE DRAWINGS

[0053] Figure 1 This is a flow chart of the electric vehicle aggregation control method based on multi-agent distributed control of the present invention;

[0054] Figure 2 This is a network diagram of a multi-agent charging station of the present invention;

[0055] Figure 3 It is a bar graph of the test data of the present invention. DETAILED DESCRIPTION

[0056] In the description of the present invention, it should be noted that the terms "upper", "lower", "inside", "outside", etc. indicate positions or positional relationships based on the positions or positional relationships shown in the accompanying drawings, or the positions or positional relationships in which the invented product is usually placed when in use. They are only for the convenience of describing the present invention and simplifying the description, and do not indicate or imply that the device or element referred to must have a specific orientation, be constructed and operated in a specific orientation, and therefore cannot be understood as a limitation on the present invention. In the description of the present invention, it should also be noted that, unless otherwise clearly specified and limited, the terms "setting", "installation", and "connection" should be understood in a broad sense, for example, it can be a fixed connection, a detachable connection, or an integral connection; it can be a mechanical connection or an electrical connection; it can be a direct connection, or it can be an indirect connection through an intermediate medium, or it can be a connection between the two elements. For those of ordinary skill in the art, the specific meanings of the above terms in the present invention can be understood according to specific circumstances.

[0057] The embodiments of the present invention are described in detail below in conjunction with the accompanying drawings.

[0058] Example 1

[0059] like Figure 1 As shown in FIG. 1 , an electric vehicle aggregation control method based on multi-agent distributed control is shown in FIG. 1 . The specific steps are as follows:

[0060] Step S1: Obtain the data of each charging station in a set area, and build a multi-agent charging station network based on the data of each charging station.

[0061] The charging station data is as follows:

[0062]

[0063] Among them, r i is the completion charging coefficient of the i-th charging station, Q i is the queue length of the i-th charging station, Q t is the target queue length of the i-th charging station, p i is the energy price of the i-th charging station, e i is the energy consumption of the i-th charging station, ΔP i is the redundant power of the i-th charging station, P i,g is the power purchased by the i-th charging station, L i is the load of the i-th charging station, g i,max The maximum permissible load of the i-th charging station, is the maximum total discharge power of the i-th charging station, is the real-time aggregated energy value of the i-th charging station, is the lower limit of the aggregated energy of the i-th charging station, is the lower limit of the aggregated energy of the i-th charging station.

[0064] like Figure 2 As shown, the multi-agent charging station network includes an agent corresponding to each charging station and an aggregation control terminal, each agent communicates with the aggregation control terminal, and each agent communicates with each other.

[0065] Step S2: Acquire the charging data of electric vehicles in each charging station and determine the charging constraint function of the electric vehicles.

[0066] Electric vehicle charging data is as follows:

[0067]

[0068] in, is the charging power of the jth electric vehicle in the i-th charging station, is the reactive power absorbed or released by the jth electric vehicle in the i-th charging station during charging, is the maximum reactive power absorbed or released by the jth electric vehicle in the i-th charging station during charging, is the maximum power capacity of the charger for the j-th electric vehicle in the i-th charging station.

[0069] In step S2, the charging constraint function of the electric vehicle in the i-th charging station is constrained from the charging power, reactive power during charging, and energy during charging. The charging constraint function of the electric vehicle in the i-th charging station is as follows:

[0070]

[0071] Among them, k is the number of electric vehicles in the i-th charging station;

[0072] is the sum of the maximum reactive power absorbed or released by electric vehicles in the i-th charging station during charging. The calculation formula is as follows:

[0073]

[0074] is the sum of reactive power absorbed or released by all electric vehicles in the i-th charging station, and the calculation formula is as follows:

[0075]

[0076] is the sum of the real-time aggregated energy values ​​of all charging stations at time t, is the sum of the real-time aggregated energy values ​​of all charging stations at time t-1, and They are the lower limit of total aggregate energy and the upper limit of total aggregate energy respectively. The lower limit of total aggregate energy is the sum of the lower limits of aggregate energy of all charging stations, and the upper limit of total aggregate energy is the sum of the upper limits of aggregate energy of all charging stations. The calculation formula is as follows:

[0077]

[0078] Step S3: The multi-agent charging station network solves the electric vehicle charging aggregation control model based on the reward function through a deep reinforcement learning algorithm, and each agent communicates with the electric vehicle aggregation control model to implement the solved aggregation control strategy.

[0079] The electric vehicle charging aggregation control model based on the reward function is as follows:

[0080] Y=max{αR c (s,a)+βR f (s,a)+γR s (s,a)+λR g (s,a)}

[0081] Among them, Y is the electric vehicle charging aggregation control model, max{} is the maximum value function, s is the current state, a is the action vector, R c (s,a) is the reward value for completing charging; R f (s,a) is the fair reward value, R s (s,a) is the cost reward value, R g (s,a) is the reward value for maintaining grid stability, α, β, γ and λ are the weights of the reward value for completing charging, the reward value for fairness, the reward value for cost and the reward value for maintaining grid stability, respectively.

[0082] The calculation formula for completing the charging reward value is as follows:

[0083]

[0084] Where N is the number of charging stations, r i The values ​​are as follows:

[0085] When charging is completed, the value is 1, and when charging is not completed, the value is -1.

[0086] Preferably, the fair reward value calculation formula is as follows:

[0087]

[0088] The addition of fair reward value optimizes the allocation of billing resources, maximizes the overall efficiency of the network, and ensures fair access for all users.

[0089] The cost reward value calculation formula is as follows:

[0090]

[0091] The cost reward value calculation formula must also meet the following conditions:

[0092]

[0093] Among them, ΔP i is the redundant power of the i-th charging station, P i,g is the power purchased by the i-th charging station.

[0094] The calculation formula for the reward value for maintaining grid stability is as follows:

[0095]

[0096] Among them, L i is the load of the i-th charging station, g max The maximum allowable load in the area is the sum of the maximum allowable loads of all charging stations in the area. The calculation formula is as follows:

[0097]

[0098] Among them, μ is the total load weight coefficient, and ν is the load extreme value weight coefficient.

[0099] A system based on the above-mentioned electric vehicle aggregation control method based on multi-agent distributed control includes:

[0100] A data acquisition module is used to acquire charging station data and electric vehicle charging data in each charging station;

[0101] A constraint module, used to determine the electric vehicle charging constraint function;

[0102] Several intelligent agents can realize interactive learning of intelligent agent detection and real-time aggregation and control of terminal optimization strategies;

[0103] Aggregation control terminal, which makes an aggregation control strategy based on maximizing rewards and considering the action strategy of the intelligent agent;

[0104] Several intelligent agents interact with each other, several intelligent agents interact with the aggregate control terminal, the constraint module and the intelligent agent communicate with the data acquisition module, and the constraint module communicates with the aggregate control terminal.

[0105] In order to verify the superiority of this embodiment, a comparative test was carried out, and the specific scheme is as follows:

[0106] Option 1: When the electric car arrives at the charging station, it is charged directly at maximum power.

[0107] Solution 2: When the electric vehicle arrives at the charging station, the electric vehicle is charged considering the grid load constraints.

[0108] Solution 3: adopt the aggregation control strategy of the embodiment.

[0109] Specific evaluation index data such as Figure 3 As shown, from Figure 3 It can be seen that the implementation of the aggregation control strategy of this application can significantly reduce energy costs, and improve charging efficiency while ensuring the stability of the power grid. According to actual data calculations, it can be concluded that this application reduces energy costs by 13% and improves charging efficiency by 32% compared to Solution 2 while ensuring the stability of the power grid.

[0110] Finally, it should be noted that the above embodiments are only used to illustrate the technical solution of the present invention rather than to limit it. Although the present invention has been described in detail with reference to the preferred embodiments, those skilled in the art should understand that they can still modify or replace the technical solution of the present invention with equivalents, and these modifications or equivalent replacements cannot cause the modified technical solution to deviate from the spirit and scope of the technical solution of the present invention.

Claims

1. A method for controlling electric vehicles based on multi-agent distributed control, characterized in that: The specific steps are as follows: Step S1: Obtain data of each charging station in a set area, and construct a multi-agent charging station network based on the data of each charging station; Step S2: acquiring the charging data of electric vehicles in each charging station and determining the charging constraint function of the electric vehicles; Step S3: The multi-agent charging station network solves the electric vehicle charging aggregation control model based on the reward function through a deep reinforcement learning algorithm, and each agent communicates with the electric vehicle aggregation control model to implement the solved aggregation control strategy.

2. The electric vehicle aggregation control method based on multi-agent distributed control according to claim 1 is characterized in that: In step S1, the multi-agent charging station network includes an agent corresponding to each charging station and an aggregate control terminal, each agent communicates with the aggregate control terminal, and each agent communicates with each other.

3. The electric vehicle aggregation control method based on multi-agent distributed control according to claim 2 is characterized in that: In step S1, the charging station data is as follows: Among them, r i is the completion charging coefficient of the i-th charging station, Q i is the queue length of the i-th charging station, Q t is the target queue length of the i-th charging station, p i is the energy price of the i-th charging station, e i is the energy consumption of the i-th charging station, ΔP i is the redundant power of the i-th charging station, P i,g is the power purchased by the i-th charging station, L i is the load of the i-th charging station, g i,max The maximum permissible load of the i-th charging station, is the maximum total discharge power of the i-th charging station, is the real-time aggregated energy value of the i-th charging station, is the lower limit of the aggregated energy of the i-th charging station, is the lower limit of the aggregated energy of the i-th charging station.

4. The electric vehicle aggregation control method based on multi-agent distributed control according to claim 3 is characterized in that: Electric vehicle charging data is as follows: in, is the charging power of the jth electric vehicle in the i-th charging station, is the reactive power absorbed or released by the jth electric vehicle in the i-th charging station during charging, is the maximum reactive power absorbed or released by the jth electric vehicle in the i-th charging station during charging, is the maximum power capacity of the charger for the j-th electric vehicle in the i-th charging station.

5. The electric vehicle aggregation control method based on multi-agent distributed control according to claim 4 is characterized in that: In step S2, the charging constraint function of the electric vehicle in the i-th charging station is constrained from the charging power, reactive power during charging, and energy during charging. The charging constraint function of the electric vehicle in the i-th charging station is as follows: Among them, k is the number of electric vehicles in the i-th charging station; is the sum of the maximum reactive power absorbed or released by electric vehicles in the i-th charging station during charging. The calculation formula is as follows: is the sum of reactive power absorbed or released by all electric vehicles in the i-th charging station, and the calculation formula is as follows: is the sum of the real-time aggregated energy values ​​of all charging stations at time t, is the sum of the real-time aggregated energy values ​​of all charging stations at time t-1, and They are the lower limit of total aggregate energy and the upper limit of total aggregate energy respectively. The lower limit of total aggregate energy is the sum of the lower limits of aggregate energy of all charging stations, and the upper limit of total aggregate energy is the sum of the upper limits of aggregate energy of all charging stations. The calculation formula is as follows:

6. The electric vehicle aggregation control method based on multi-agent distributed control according to claim 5 is characterized in that: The electric vehicle charging aggregation control model based on the reward function is as follows: Y=max{αR c (s,a)+βR f (s,a)+γR s (s,a)+λR g (s,a)} Among them, Y is the electric vehicle charging aggregation control model, max{} is the maximum value function, s is the current state, a is the action vector, R c (s,a) is the reward value for completing charging; R f (s,a) is the fair reward value, R s (s,a) is the cost reward value, R g (s,a) is the reward value for maintaining grid stability, α, β, γ and λ are the weights of the reward value for completing charging, the reward value for fairness, the reward value for cost and the reward value for maintaining grid stability, respectively.

7. The electric vehicle aggregation control method based on multi-agent distributed control according to claim 6 is characterized in that: The calculation formula for completing the charging reward value is as follows: Where N is the number of charging stations, r i The values ​​are as follows: When charging is completed, the value is 1, and when charging is not completed, the value is -1.

8. The electric vehicle aggregation control method based on multi-agent distributed control according to claim 7 is characterized in that: The fair reward value calculation formula is as follows: The cost reward value calculation formula is as follows: The cost reward value calculation formula must also meet the following conditions: Among them, ΔP i is the redundant power of the i-th charging station, P i,g is the power purchased by the i-th charging station.

9. The electric vehicle aggregation control method based on multi-agent distributed control according to claim 8 is characterized in that: The calculation formula for the reward value for maintaining grid stability is as follows: Among them, L i is the load of the i-th charging station, g max The maximum allowable load in the area is the sum of the maximum allowable loads of all charging stations in the area. The calculation formula is as follows: Among them, μ is the total load weight coefficient, and ν is the load extreme value weight coefficient.

10. A system for an electric vehicle aggregation control method based on multi-agent distributed control according to claim 9, characterized in that: include: A data acquisition module is used to acquire charging station data and electric vehicle charging data in each charging station; A constraint module, used to determine the electric vehicle charging constraint function; Several intelligent agents can realize interactive learning of intelligent agent detection and real-time aggregation and control of terminal optimization strategies; Aggregation control terminal, which makes an aggregation control strategy based on maximizing rewards and considering the action strategy of the intelligent agent; Several intelligent agents interact with each other, several intelligent agents interact with the aggregate control terminal, the constraint module and the intelligent agent communicate with the data acquisition module, and the constraint module communicates with the aggregate control terminal.

Images