Multi-charging-station electric vehicle charging and discharging scheduling method based on reinforcement learning

Abstract

The invention discloses a multi-charging-station electric vehicle charging and discharging scheduling method based on reinforcement learning, and aims to reduce power fluctuation of a power grid and improve the absorption capability of renewable energy sources. The method comprises the steps that a double-layer coordination control model of a micro-grid and a power distribution network is established, an electric vehicle charging station is divided into a working area and a living area, and randomness of user travel behaviors is described through a space chain and a time chain. And an electric vehicle charging and discharging model is constructed, the charging time margin and schedulability are considered, and it is ensured that the electric vehicle is fully charged on time. Under a multi-charging-station framework, a multi-agent reinforcement learning algorithm of centralized training and decentralized execution is adopted, cooperative training is carried out through a shared experience pool, and optimization of a charging and discharging strategy is achieved. And each agent independently decides to improve the expansibility and robustness of the system. The system effectively coordinates a plurality of charging stations, reduces the burden of a power grid, improves the utilization rate of wind energy and solar energy, meets the charging demands of electric vehicle users, and achieves the purpose of optimizing energy management.

Description

Technical Field

[0001] The application of the present invention belongs to the field of electric vehicle charging and discharging scheduling, the technical means of the present invention belongs to the field of machine learning, and also belongs to the field of artificial intelligence technology, and particularly relates to a multi-charging station electric vehicle charging and discharging scheduling method based on reinforcement learning. Background Art

[0002] In recent years, with the growing demand for energy and increasing awareness of environmental protection, electric vehicles, as a clean and efficient means of transportation, are being promoted by more and more countries and regions. However, the widespread use of electric vehicles also brings new challenges, mainly reflected in the increase in load on the power system and the integration of renewable energy.

[0003] The demand for charging electric vehicles is often concentrated in specific periods of time, which leads to a sudden increase in grid load and power fluctuations. Traditional grid planning and operation methods are difficult to cope with such concentrated short-term high loads, posing a potential threat to the stability and reliability of the power system.

[0004] With the global attention paid to renewable energy sources such as wind and solar energy, how to efficiently utilize these intermittent energy sources is an important topic. The fluctuating characteristics of renewable energy do not match the charging needs of electric vehicles, which limits their absorption in the power grid. If the charging and discharging behaviors of electric vehicles can be coordinated and their potential as distributed energy storage devices can be fully utilized, the utilization efficiency of renewable energy can be effectively improved.

[0005] Existing charging stations are mainly concentrated in specific areas of the city, and the layout of charging stations takes into account factors such as geographical location and user needs. However, how to arrange charging according to the dynamic and changing travel patterns of electric vehicle users to improve the utilization efficiency of charging stations and power grids is still a problem that needs to be solved.

[0006] Using electric vehicles as energy storage units to achieve two-way power flow through vehicle-to-grid (V2G) interaction is a potential solution to improve grid stability and renewable energy absorption capacity. However, existing solutions generally lack effective technical means to dispatch and optimize this two-way power interaction process.

[0007] In the field of electric vehicle charging and discharging scheduling, traditional methods are often based on deterministic optimization and heuristic algorithms, which often show poor adaptability when dealing with complex and dynamic environments. In addition, the scale and complexity of scheduling problems involving the coordination of multiple charging stations and multiple electric vehicle users are growing, and traditional centralized control schemes are difficult to deal with effectively. Reinforcement Learning (RL), as a learning-based control strategy, has made significant progress in the fields of robot control and game AI in recent years. However, in the specific field of electric vehicle charging and discharging scheduling, existing reinforcement learning research has the problem of insufficient exploration. When dealing with large-scale multi-agent systems and complex environments, it is difficult to achieve an effective balance between the gradually improved exploration mechanism and the efficient utilization strategy. In addition, for dynamic environment requirements such as electric vehicle charging, the transferability and real-time adaptability of traditional RL methods are insufficient. Due to the rapid expansion of the state space and action space in the multi-agent environment, many RL algorithms lack scalability when applied. Summary of the invention

[0008] In order to overcome the above-mentioned deficiencies of the prior art, the present invention proposes a multi-charging station electric vehicle charging and discharging scheduling method based on reinforcement learning, the purpose of which is to reduce power fluctuations in the power grid and improve the consumption of renewable energy.

[0009] A method for scheduling charging and discharging of electric vehicles at multiple charging stations based on reinforcement learning, comprising the following steps:

[0010] Step 1: Establish a two-layer coordinated control model of microgrid and distribution network;

[0011] Step 2: Divide the areas where different electric vehicle charging stations are located into different functional areas, consider the road network model, and establish the travel chain of electric vehicle users;

[0012] Step 3: Establish an electric vehicle charging and discharging model;

[0013] Step 4: Convert the charging and discharging schedule of electric vehicles into a Markov decision process, and determine the state, action and reward in the Markov decision process;

[0014] Step five: Use a multi-agent reinforcement learning algorithm for centralized training and decentralized execution of optimization decisions to achieve charging and discharging scheduling of electric vehicles at multiple charging stations.

[0015] Based on the above scheme, in step 1, the microgrid structure includes a wind power system, a photovoltaic power station and a basic load excluding the electric vehicle charging load;

[0016] The distribution network structure includes microgrid units and electric vehicle charging stations. The power exchange between the microgrid and the distribution network can achieve mutual support, thereby effectively improving the overall operation efficiency;

[0017] The microgrid can be connected to the distribution network through a tie line and exchange power with it in both directions. Wind power and photovoltaic power generation are prioritized to meet the basic load of the system. When the power generation is surplus, the electric vehicles in the system are charged. If there is still surplus, they are connected to the grid. When the power generation is insufficient to supply the basic load of the system, the electric vehicles in each charging station supply energy to the system through vehicle-grid interaction technology. If it is still insufficient, support is obtained from the grid, thereby improving the consumption of renewable energy and reducing grid power fluctuations. The power obtained from the grid at time t is obtained as ΔP t The expression is:

[0018] ΔP t =P t load +P t EV -P t WT -P t PV

[0019] Where P t load , P t EV , P t WT , P t PV are the base load at time t, the charging / discharging amount of the electric vehicle cluster, the wind power generation, and the photovoltaic power generation. t EV >0; P in V2G mode t EV <0, the electric vehicle is discharging.

[0020] Based on the above scheme, in step 2, the area where the electric vehicle charging station is located is divided into: a working area (W) and a living area (H);

[0021] The travel behavior of electric vehicle users is regarded as a dynamic process consisting of a series of moving and parking (charging) actions. This process can be described by space chain and time chain;

[0022] The spatial chain describes the user's travel behavior between different areas (work area, living area), and the temporal chain describes the driving and parking charging behavior of electric vehicles in different time periods;

[0023] By analyzing the travel patterns of electric vehicle users in a day, it is found that the driving behavior of users has a certain degree of randomness, and the travel behavior pattern can be represented by the transition probability matrix. Specifically, the behavior of electric vehicle users can be described by the state transition process, where the state represents the area where the user is located, and the transition probability matrix p(x|x-1,t) is used to represent the probability of the user transferring from the previous state x-1 to the current state x at time t, which is expressed as:

[0024]

[0025] Where z n The region number.

[0026] Based on the above scheme, in step 3, an electric vehicle charging and discharging model is constructed. The charging and discharging power of each electric vehicle i at time t is In [-P c ,P c ], P c For maximum charging power, there are three charging modes: charging, idle and discharging;

[0027] Charging time margin σ i It is expressed as the ratio of the expected stay time of electric vehicle i at the charging station to the shortest charging time:

[0028]

[0029]

[0030] In the formula are the times when electric vehicle i arrives and leaves the charging station, For the shortest charging time, are the initial and target power of the electric vehicle respectively;

[0031] Then from the charging time margin σ i Electric vehicle dispatchability classification can be determined:

[0032]

[0033] Where σ th is the charging time margin threshold. Dispatched electric vehicles can adjust the charging power, while non-dispatched electric vehicles need to charge at the maximum power until the charging task is met.

[0034] To meet the charging needs of each electric vehicle, the following conditions need to be set:

[0035] (1) Each electric vehicle needs to be Reached target power before

[0036] (2) The SOC of each electric vehicle at time t i,t Do not exceed battery capacity

[0037]

[0038] (3) Considering the user's temporary car pickup behavior, set the power threshold That is, after the electric car is connected to the charging pile, it needs to use power P c Charge to Subsequent scheduling can be carried out.

[0039] The constraints that must be followed during the charging process of electric vehicles include charging status, power threshold, and charging time restrictions:

[0040]

[0041] The charging process can be scheduled through the time Indicating EV charging flexibility:

[0042]

[0043] In the formula is the remaining power requirement of electric vehicle i at time t. If it is regular, electric vehicle i can continue to participate in charging scheduling, otherwise it needs to maintain the maximum power P c So the dispatchable state of electric vehicles is expressed as δ i,t express:

[0044]

[0045] Taking into account the battery life and discharge cost, the interests of electric vehicle users are optimized during the discharge process, and the charging demand loss of electric vehicle users is adopted. This indicator is reduced during the scheduling process Protect users' interests and charging needs.

[0046] Based on the above scheme, in step 4, the charging and discharging scheduling of electric vehicles at multiple charging stations is transformed into a Markov decision process;

[0047] Define the state in the Markov decision process: the state space S t Refers to the set of all possible states, describing the specific information of the environment at time t. In order to make the decision of the agent more efficient, it is necessary to provide sufficient information support while avoiding the interference of redundant information in the decision-making process. t Set to:

[0048]

[0049] In the formula is the remaining charging time of electric vehicle i at time t.

[0050] Define actions in the Markov decision process: action a t Refers to the state S observed by the agent in the environment t The actions that can be performed after that will affect the state S at the next moment. t+1 , a t It is the charging and discharging power of electric vehicle i at time t

[0051] Defining the reward in the Markov decision process: The optimization goal of the agent is to reduce the power fluctuation of the power grid and improve the consumption of renewable energy, so the reward function r is designed t Set to:

[0052]

[0053] Where α and β are weight factors, P t ls The sum of charging demand losses of electric vehicle users at time t.

[0054] Based on the above scheme, in step five, the multi-agent reinforcement learning algorithm is a machine learning method that can support multiple agents to interactively learn in the same environment. In this patent, in order to solve the problem of charging and discharging scheduling of electric vehicles at multiple charging stations, a multi-agent reinforcement learning framework with centralized training and decentralized execution is adopted. The design of this framework aims to effectively coordinate and optimize the charging and discharging scheduling of electric vehicles to reduce power fluctuations in the power grid and improve the absorption capacity of renewable energy. The specific implementation steps are as follows:

[0055] State space mapping and initialization: First, according to the Markov decision process defined in step 4, the state space of the entire system is mapped to each agent. Initialize the state information of each agent, including the remaining charging time of the electric vehicle, the current regional location, the current battery power, etc. Initialize the policy network and value network parameters of all agents.

[0056] Centralized training phase: Collect data on the state, action, reward, and next state generated by the interaction of all agents in the environment to form a training set. Use this data to update the centralized policy network and value network. Use the policy gradient reinforcement learning algorithm to optimize the strategy. During the training process, a shared experience pool is used to allow all agents to share experience, thereby improving training efficiency and convergence speed.

[0057] Decentralized execution stage: At each real-time decision moment, each agent makes independent decisions through its local policy network based on the currently observed state. Each agent only needs its own local state information to complete the charging and discharging power decision, and does not depend on the state of other agents. This approach makes the system have good scalability and robustness.

[0058] Feedback and update: After each decision cycle, each agent will feed back the immediate reward and state transfer results obtained by executing the action to the central controller. The central controller regularly collects all feedback information and performs centralized training parameter updates. The batch update strategy is combined with the discount factor attenuation mechanism to ensure the stable control performance of the system in an uncertain environment.

[0059] Optimization objectives and convergence: When setting the reward function, we fully consider the reduction of grid power fluctuations and the improvement of renewable energy absorption capacity, while balancing the charging needs and electricity costs of electric vehicle users. By designing reasonable weight factors and reward structures, we guide the reinforcement learning algorithm to converge to a set of optimal strategies, so that the overall charging system has good adaptability and efficiency in a dynamic environment.

[0060] Adaptive adjustment mechanism: The system adds an adaptive adjustment mechanism, which allows the agent to adaptively adjust its policy parameters when the environment or user needs change, ensuring the stability and efficiency of the system in the long run.

[0061] Through the above steps, the multi-agent reinforcement learning algorithm realizes the centralized and decentralized coordinated optimization of the charging and discharging scheduling of electric vehicles at multiple charging stations, which not only effectively reduces the power fluctuation of the power grid, but also improves the utilization rate of renewable energy, especially wind and solar energy. This decentralized execution strategy is particularly suitable for the real-time optimization needs of large-scale systems.

[0062] Beneficial effects of the present invention:

[0063] The present invention proposes a method for scheduling electric vehicle charging and discharging based on multi-agent reinforcement learning, which improves overall efficiency through power exchange between microgrid and distribution network; based on the division of work area and living area in the area where the electric vehicle charging station is located, the dynamic travel behavior of users is described through space chain and time chain, and the randomness of travel mode and behavior transfer probability are considered; the construction of electric vehicle charging and discharging model considers charging time margin, dispatchability and charging demand of electric vehicles, ensures that each electric vehicle reaches the target power on time, and optimizes the charging process by flexibly adjusting the charging power; converts charging and discharging scheduling into Markov decision process, defines state, action and reward; adopts multi-agent reinforcement learning algorithm with centralized training and decentralized execution, optimizes charging and discharging strategy through collaborative training of agents in shared experience pool, and each agent independently executes decision in real-time decision-making, thereby improving the scalability and robustness of the system. The present invention effectively coordinates multiple charging stations through this method, reduces the burden on the power grid, improves the utilization rate of wind energy and solar energy, and at the same time ensures that the charging needs of electric vehicle users are met, and optimizes the overall energy management of the system. BRIEF DESCRIPTION OF THE DRAWINGS

[0064] Figure 1 This is a microgrid-distribution network structure diagram of the present invention;

[0065] Figure 2 This is a schematic diagram of the wind-solar-electric vehicle complementary system of the present invention;

[0066] Figure 3 This is a schematic diagram of the zoning of an electric vehicle charging station of the present invention;

[0067] Figure 4 This is a diagram of the multi-agent deep reinforcement learning framework of the present invention. DETAILED DESCRIPTION

[0068] In order to make the objects, advantages and features of the present invention more obvious, the present invention is further described in detail below with reference to the accompanying drawings and specific embodiments.

[0069] like Figure 1 As shown in the figure, the microgrid structure includes a wind power system, a photovoltaic power station and a basic load excluding the electric vehicle charging load; the distribution network structure includes a microgrid unit and an electric vehicle charging station. The power exchange between the microgrid and the distribution network can achieve mutual support, thereby effectively improving the overall operating efficiency.

[0070] like Figure 2As shown in the figure, in the design of the wind-solar-electric vehicle complementary system, wind power and photovoltaic power generation are given priority to meet the use of the basic load in the system. When the power generation is surplus, the electric vehicles in the system are charged, and if there is still surplus, they are connected to the grid. When the power generation is not enough to supply the basic load in the system, the electric vehicles in each charging station supply energy to the system through the vehicle-grid interaction technology. If it is still insufficient, support is obtained from the power grid, thereby improving the consumption of renewable energy and reducing power grid power fluctuations.

[0071] like Figure 3 As shown in the figure, the area where the electric vehicle charging station is located is divided into: work area (W) and living area (H). The travel behavior of electric vehicle users is regarded as a dynamic process consisting of a series of movement and parking (charging) actions. This process can be described by space chain and time chain. The space chain describes the travel behavior of users between different areas (work area, living area), and the time chain describes the driving and parking charging behavior of electric vehicles in different time periods.

[0072] Construct an electric vehicle charging and discharging model. The charging and discharging power of each electric vehicle i at time t is In [-P c ,P c ], P c For the maximum charging power, three charging modes are set: charging, idle and discharging. The charging time margin σ i It is expressed as the ratio of the expected stay time of electric vehicle i at the charging station to the shortest charging time:

[0073]

[0074]

[0075] In the formula are the times when electric vehicle i arrives and leaves the charging station, For the shortest charging time, are the initial and target power of the electric vehicle respectively.

[0076] Then from the charging time margin σ i Electric vehicle dispatchability classification can be determined:

[0077]

[0078] Where σ th is the charging time margin threshold. Dispatched electric vehicles can adjust the charging power, while non-dispatched electric vehicles need to charge at the maximum power until the charging task is met.

[0079] To meet the charging needs of each electric vehicle, the following conditions need to be set:

[0080] (1) Each electric vehicle needs to be Reached target power before

[0081] (2) The SOC of each electric vehicle at time t i,t Do not exceed battery capacity

[0082]

[0083] (3) Considering the user's temporary car pickup behavior, set the power threshold That is, after the electric car is connected to the charging pile, it needs to use power P c Charge to Subsequent scheduling can be carried out.

[0084] The constraints that must be followed during the charging process of electric vehicles include charging status, power threshold, and charging time restrictions:

[0085]

[0086] The charging process can be scheduled through the time Indicating EV charging flexibility:

[0087]

[0088] In the formula is the remaining power requirement of electric vehicle i at time t. If it is regular, electric vehicle i can continue to participate in charging scheduling, otherwise it needs to maintain the maximum power P c So the dispatchable state of electric vehicles is expressed as δ i,t express:

[0089]

[0090] Taking into account the battery life and discharge cost, the interests of electric vehicle users are optimized during the discharge process, and the charging demand loss of electric vehicle users is adopted. This indicator is reduced during the scheduling process Protect users' interests and charging needs.

[0091] Then the charging and discharging scheduling of electric vehicles at multiple charging stations is transformed into a Markov decision process.

[0092] Define the state in the Markov decision process: the state space S t Refers to the set of all possible states, describing the specific information of the environment at time t. In order to make the decision of the agent more efficient, it is necessary to provide sufficient information support while avoiding the interference of redundant information in the decision-making process. t Set to:

[0093]

[0094] In the formula is the remaining charging time of electric vehicle i at time t.

[0095] Define actions in the Markov decision process: action a t Refers to the state S observed by the agent in the environment t The actions that can be performed after that will affect the state S at the next moment. t+1 , a t It is the charging and discharging power of electric vehicle i at time t

[0096] Defining the reward in the Markov decision process: The optimization goal of the agent is to reduce the power fluctuation of the power grid and improve the consumption of renewable energy, so the reward function r is designed t Set to:

[0097]

[0098] Where α and β are weight factors, The sum of charging demand losses of electric vehicle users at time t.

[0099] like Figure 4 As shown in the figure, a multi-agent reinforcement learning framework with centralized training and decentralized execution is adopted. The design of this framework aims to effectively coordinate and optimize the charging and discharging scheduling of electric vehicles to reduce grid power fluctuations and improve the absorption capacity of renewable energy. The specific implementation steps are as follows:

[0100] In the multi-agent deep reinforcement learning framework, the training process of the agent is divided into two main stages. First, in the offline training stage, the agent continuously collects experience trajectories through interaction with the environment and stores these experiences in the experience replay pool. In addition, the value network and policy network of the agent will be updated regularly to gradually improve its decision-making ability until the performance of the model converges. Secondly, in the online deployment stage, the agent will load the pre-trained policy network and its parameters, and make the best decision based on the observed information, so as to perform efficient task execution in the actual environment.

[0101] During the offline training phase, a series of basic parameters need to be set, including the capacity of the experience replay pool and the training cycle. The policy network parameters and value network parameters of the agent are initialized, and the initial state is randomly selected from the environment to start training. In each round of training, the agent is trained according to the current environment state S t Take appropriate action t , then the environment state is updated to S t+1 , and returns the immediate reward rt These empirical information (i.e. {S t ,a t ,S t+1 ,r t}) will be stored in the experience replay pool to provide data support for subsequent training.

[0102] During training, the agent updates its strategy by randomly sampling batches of experience samples from the replay pool. In the training of the policy network, the ratio ξ between the new and old strategies is first calculated. t (calculated by the probability ratio of the strategy), and then update the strategy based on this ratio to ensure that the update of the new strategy does not deviate too much from the original strategy and avoid unstable training. At the same time, the training of the value network uses the advantage function to evaluate the quality of the action and further adjust the network parameters to more accurately estimate the value of each state.

[0103] In order to ensure the stability of policy updates, the advantage function is clipped and a suitable clipping range ε ​​is set to limit the range of changes in the policy ratio. This policy clipping mechanism helps control the range of policy changes and prevent instability in the training process due to excessive updates. In addition, in order to improve the diversity of policies and avoid overfitting, an entropy constraint is added during the training process to control the entropy value of the policy, and the regularization coefficient is adjusted when necessary to ensure that the policy maintains an appropriate balance between exploration and exploitation. Therefore, the objective function L(θ) is obtained:

[0104] L(θ)=maxE[min(ξ t A t ,clip(ξ t ,1-ε,1+ε)A t )]

[0105]

[0106] Where θ is the policy network parameter.

[0107] Later, in the online deployment phase, each fully trained agent will only use the policy network obtained during the training process, and no longer rely on the assistance of the value network. At this time, the agent will load the trained and optimized optimal policy network parameters θ', and make decisions in the actual environment based on this policy network, thereby achieving efficient task execution and scheduling. By using the saved optimal policy parameters, the agent can quickly adapt and respond in real time in a dynamic environment to ensure the optimization of the scheduling process.

[0108] The above implementation modes are only used to illustrate the present invention, but not to limit the present invention. Ordinary technicians in the relevant technical field can make various changes and modifications without departing from the essence and scope of the present invention. Therefore, all equivalent technical solutions also belong to the scope of the present invention, and the patent protection scope of the present invention should be defined by the claims. The contents not described in detail in this specification belong to the prior art known to professional and technical personnel in this field.

Claims

1. A multi-charging station electric vehicle charging and discharging scheduling method based on reinforcement learning, characterized in that: The method comprises the following steps: Step 1: Establish a two-layer coordinated control model of microgrid and distribution network; Step 2: Divide the areas where different electric vehicle charging stations are located into different functional areas, consider the road network model, and establish the travel chain of electric vehicle users; Step 3: Establish an electric vehicle charging and discharging model; Step 4: Convert the charging and discharging schedule of electric vehicles into a Markov decision process and determine the state, action and reward in the Markov decision process; Step five: Use multi-agent reinforcement learning algorithms for centralized training and decentralized execution of optimized decisions to achieve charging and discharging scheduling of electric vehicles at multiple charging stations.

2. The method according to claim 1, wherein in step 1, the microgrid structure includes a wind power system, a photovoltaic power station and a basic load excluding an electric vehicle charging load, and the distribution network structure includes a microgrid unit and an electric vehicle charging station, and the power exchange between the microgrid and the distribution network can achieve mutual support.

3. The method according to claim 2, wherein: Wind power and photovoltaic power generation are prioritized to meet the basic load usage within the system. When there is surplus power generation, the electric vehicles in the system are charged. When the power generation is insufficient, the electric vehicles supply energy to the system through vehicle-grid interaction technology. If the power generation is still insufficient, support is obtained from the power grid, thereby improving the consumption of renewable energy and reducing power grid power fluctuations.

4. The method according to claim 1, wherein: In step 2, the area where the electric vehicle charging station is located is divided into a working area (W) and a living area (H), and the travel behavior of electric vehicle users is described by a space chain and a time chain. The transition probability matrix is ​​used to describe the probability of an electric vehicle user transferring from a previous state to a current state at a time.

5. The method according to claim 1, wherein: In step three, the electric vehicle charging and discharging model includes three charging modes: charging, idling and discharging, and the charging time margin is used to represent the ratio of the expected stay time of the electric vehicle to the shortest charging time, and the electric vehicle dispatchability classification is determined according to the charging time margin.

6. The method according to claim 1, wherein: In step 4, the state space of the Markov decision process includes information such as the remaining charging time of the electric vehicle and the current battery power, the action includes the charging and discharging power of the electric vehicle, and the reward function is designed to reduce grid power fluctuations while improving the absorption of renewable energy. The reward function includes the charging demand loss of electric vehicle users. Define the state in the Markov decision process: the state space S t Refers to the set of all possible states, describing the specific information of the environment at time t. In order to make the decision of the agent more efficient, it is necessary to provide sufficient information support while avoiding the interference of redundant information in the decision-making process. t Set to: In the formula is the remaining charging time of electric vehicle i at time t. Define actions in the Markov decision process: action a t Refers to the state S observed by the agent in the environment t The actions that can be performed after that will affect the state S at the next moment. t+1 , a t It is the charging and discharging power of electric vehicle i at time t Defining the reward in the Markov decision process: The optimization goal of the agent is to reduce the power fluctuation of the power grid and improve the consumption of renewable energy, so the reward function r is designed t Set to: r t =-αΔP t |-βP t ls Where α and β are weight factors, P t ls The sum of charging demand losses of electric vehicle users at time t.

7. The method according to claim 1, wherein: In step five, the multi-agent reinforcement learning algorithm adopts a framework of centralized training and decentralized execution. The centralized training phase includes collecting the agent's experience data and optimizing it using a policy gradient reinforcement learning algorithm. In the decentralized execution phase, the agent makes independent decisions based on its local state information.

8. The method according to claim 7, wherein: The feedback and update include that after each decision cycle, each agent will feed back the immediate reward and state transfer results obtained by executing the action to the central controller. The central controller regularly collects all feedback information, performs centralized training parameter updates, and adopts batch update strategy and discount factor attenuation mechanism to ensure the stable regulation performance of the system in an uncertain environment.

9. The method according to claim 1, wherein: The system adds an adaptive adjustment mechanism, which allows the agent to adaptively adjust its policy parameters when the environment or user needs change, thereby ensuring the stability and efficiency of the system in long-term operation.

10. A system for multi-charging station electric vehicle charging and discharging scheduling method based on reinforcement learning, characterized in that: The system comprises a microgrid, a distribution network, a plurality of electric vehicle charging stations and a plurality of intelligent agents, wherein the system is implemented based on any one of the methods of claims 1 to 9.

Images