V2G energy management method and device based on multilayer group optimization and deep reinforcement learning

Abstract

The invention belongs to the technical field of electric vehicle dispatching, and particularly relates to a V2G energy management method and device based on multilayer group optimization and deep reinforcement learning. The method comprises the following steps: constructing a comprehensive optimization objective function system oriented to a large-scale electric vehicle scene, wherein the objective function system comprises minimum discharge cost, minimum charge waiting time, maximum electric quantity state and maximum user power consumption saving cost; global optimization is carried out on the comprehensive optimization objective function based on a multilayer grouping competition group optimization algorithm, and an initial charging and discharging scheduling scheme is determined; correcting the initial charging and discharging scheduling scheme in real time based on a deep reinforcement learning method, and determining an optimized V2G energy management strategy; the optimized V2G energy management strategy is used for scheduling charging and discharging behaviors of the electric vehicle. According to the invention, photovoltaic power generation prediction errors and power grid load fluctuation can be effectively coped with, ordered charging and discharging of the electric vehicle are realized, the new energy consumption rate is improved, the power consumption cost of users is reduced, and the power grid operation stability is enhanced.

Description

TECHNICAL FIELD

[0001] The application belongs to the technical field of intelligent power grid and electric vehicle coordinated scheduling, and particularly relates to a vehicle-to-grid energy management method and device based on multi-layer group optimization and deep reinforcement learning. BACKGROUND

[0002] Promoting the use of electric vehicles (EV) is an important way to realize the electric energy replacement in the transportation field and the energy consumption revolution, and is of great significance for reducing dependence on fossil energy and environmental protection. At the same time, the proportion of renewable energy generation represented by photovoltaic and wind power continues to increase, and the power system is undergoing a profound green transformation. However, as the penetration rate of electric vehicles continues to rise and large-scale unordered access, the random and intermittent charging and discharging behavior of electric vehicles will significantly exacerbate the load fluctuation of the power grid, possibly leading to local line overload, power quality degradation, and even threatening the safe and stable operation of the power system. In addition, the high proportion of renewable energy connected to the grid, especially the inherent intermittency and volatility of photovoltaic power generation (SPG), increase the prediction error of power generation, further exacerbating the difficulty of balancing supply and demand in the power grid. At present, in the process of electric vehicle charging and discharging scheduling, the optimization method based on static mathematical programming and the traditional group intelligent optimization algorithm cannot effectively cope with dynamic uncertain factors such as prediction error of renewable energy output and randomness of electric vehicle user behavior. When the existing DRL method is applied to large-scale V2G scheduling, it often faces problems such as low efficiency of strategy exploration and unstable convergence. At the same time, the existing scheduling model often fails to fully and accurately balance the differentiated interest demands and bounded rationality decision-making behavior of power grid operators, load aggregators, charging station operators and electric vehicle users. SUMMARY

[0003] To overcome the problems in the related art, the embodiments of the present application provide a V2G energy management method and device based on multi-layer group optimization and deep reinforcement learning, which can effectively cope with photovoltaic power generation prediction error and power grid load fluctuation, realize orderly charging and discharging of electric vehicles, improve new energy consumption rate, reduce user electricity cost and enhance power grid operation stability.

[0004] The present application is achieved by the following technical solutions: In a first aspect, the embodiments of the present application provide a V2G energy management method based on multi-layer group optimization and deep reinforcement learning, comprising: constructing a comprehensive optimization objective function system for a large-scale electric vehicle scenario, the objective function system including minimizing discharging cost, minimizing charging waiting time, maximizing power state and maximizing user electricity cost saving; The comprehensive optimization objective function is globally optimized based on a multi-level group competition group optimization algorithm to determine the initial charging and discharging scheduling scheme. The initial charging and discharging scheduling scheme is corrected in real time using a deep reinforcement learning method to determine the optimized V2G energy management strategy; the optimized V2G energy management strategy is used to schedule the charging and discharging behavior of electric vehicles.

[0005] In one embodiment, constructing a comprehensive optimization objective function system for large-scale electric vehicle scenarios includes: Based on blind inverse game theory, we jointly infer the payoff functions and rationality levels of grid operators, electric vehicle users, and charging station operators from observational data. Based on the entropy regularization quantization response equilibrium, a multi-objective dynamic trade-off is made for minimizing discharge cost, minimizing charging waiting time, maximizing power status, and maximizing savings on user electricity costs, forming the comprehensive optimization objective function system.

[0006] In one embodiment, the objective function for minimizing discharge cost is... Represented as: in, and These are charge / discharge state parameters. The charging and discharging power of the vehicle, and for The charging and discharging electricity price during different time periods, and For charging and discharging power, For the cost of battery degradation, The duration of the control cycle is measured in units.

[0007] In one embodiment, the objective function f2 for minimizing the charging wait time is expressed as: in, and Representing vehicles The time when charging begins and the time when the device arrives at the charging station.

[0008] In one embodiment, the objective function f3 for maximizing the state of charge is expressed as: in, Indicates vehicle The initial SOC value, Indicates vehicle Battery capacity.

[0009] In one embodiment, the objective function f4 for maximizing savings in user electricity costs is expressed as: in, Indicates vehicle In time The benefits generated by discharge, Indicates vehicle In time The charging cost, It is the cost associated with battery degradation.

[0010] In one embodiment, the method based on blind inverse game theory to jointly infer the payoff functions and rationality levels of grid operators, electric vehicle users, and charging station operators from observation data includes: Construct a participant set N = {GO, EU, CSO}, where GO represents grid operators, EU represents electric vehicle users, and CSO represents charging station operators; Define the strategy space for each participant; An observation dataset was constructed based on historical scheduling data and real-time interactive data. By using a normalized least squares estimator, the payoff function parameter vector and rationality parameter vector of each participant are learned simultaneously. The rationality parameter is used to quantify the rationality of the participants' decision-making.

[0011] In one embodiment, the formula for entropy regularization quantization response equilibrium is expressed as: in, For participants The optimal strategy distribution, For the profit function, Quantifying participants The level of rationality.

[0012] In one embodiment, the multi-objective dynamic trade-off is achieved by solving the Pareto front, the expression of which is: in, For the first Adaptive weights for each objective function Here is the entropy regularization coefficient. Let be the entropy of the policy distribution.

[0013] Secondly, this application provides a V2G energy management device based on multi-level population optimization and deep reinforcement learning, which applies the V2G energy management method based on multi-level population optimization and deep reinforcement learning as described in the first aspect, including: The objective function construction module is used to construct a comprehensive optimization objective function system for large-scale electric vehicle scenarios; the objective function system includes minimizing discharge cost, minimizing charging waiting time, maximizing state of charge, and maximizing savings on user electricity costs; The global optimization module is used to perform global optimization of the comprehensive optimization objective function based on a multi-level group competition group optimization algorithm to determine the initial charging and discharging scheduling scheme; The real-time correction module is used to correct the initial charging and discharging scheduling scheme in real time based on a deep reinforcement learning method, and to determine the optimized V2G energy management strategy; the optimized V2G energy management strategy is used to schedule the charging and discharging behavior of electric vehicles.

[0014] In one embodiment, the objective function construction module is specifically used for: Based on blind inverse game theory, we jointly infer the payoff functions and rationality levels of grid operators, electric vehicle users, and charging station operators from observational data. Based on the aforementioned payoff function and rationality level, an incomplete information game framework is constructed, and the system is guided to converge to an entropy-regularized quantitative response equilibrium. Based on the entropy regularization quantization response equilibrium, a multi-objective dynamic trade-off is made for minimizing discharge cost, minimizing charging waiting time, maximizing power status, and maximizing savings on user electricity costs, forming the comprehensive optimization objective function system.

[0015] In one embodiment, the global optimization module is specifically used for: The complete charge and discharge scheduling scheme is transformed into a particle position vector through particle encoding, and an initial particle swarm containing multiple particles is randomly generated. The initial particle swarm is sorted according to the particle fitness value and divided into three groups according to the first, second and third levels; The first-level group is randomly divided into several four-particle units, the second-level group is randomly divided into several three-particle units, and the third-level group is randomly divided into several two-particle units. Within the basic unit of each group, competition and learning take place. The particle with the lowest fitness is marked as particle 1 and retains its own information to directly enter the next iteration. The remaining particles update their position and velocity according to the competition results. The competitive learning process described above is executed iteratively until the maximum number of iterations is reached. The particle with the optimal fitness value is output and decoded into the initial charge-discharge scheduling scheme.

[0016] In one embodiment, the real-time correction module is specifically used for: The uncertainty of photovoltaic power generation prediction error is modeled by Markov decision process, and a reinforcement learning framework with the goal of minimizing mean square error is constructed. A deep neural network is used to approximate the state-action value function, and the network is trained using experience replay and target network techniques. Based on the trained deep reinforcement learning model, the charging and discharging actions are dynamically adjusted according to the real-time environmental conditions, thereby correcting the initial charging and discharging scheduling scheme.

[0017] In one embodiment, the V2G energy management device further includes a constraint management module for ensuring that the V2G energy management strategy meets at least one of the following constraints: electric vehicle availability time constraint, charging and discharging power constraint, state of charge constraint, real-time electricity price boundary constraint, depth of discharge safety constraint, and number of charging and discharging electric vehicles constraint.

[0018] The beneficial effects of the embodiments in this application compared with the prior art are: The V2G energy management device based on multi-level population optimization and deep reinforcement learning provided in this application embodiment achieves a complete technical solution at the method level through modular design, featuring flexible deployment and high execution efficiency. This device decomposes the complex V2G energy management problem into three core stages: objective function construction, global optimization, and real-time correction. Through the collaborative work of each module, it effectively solves the scheduling optimization problem in large-scale electric vehicle access scenarios.

[0019] It should be understood that the above general description and the following detailed description are exemplary and explanatory only, and are not intended to limit this specification. Attached Figure Description

[0020] To more clearly illustrate the technical solutions in the embodiments of this application, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0021] Figure 1 This is a schematic diagram illustrating an application scenario of a V2G energy management method and device based on multi-level population optimization and deep reinforcement learning, provided in one embodiment of this application. Figure 2 This is a schematic diagram of a V2G energy management method and device based on multi-level population optimization and deep reinforcement learning, provided in one embodiment of this application. Detailed Implementation In the following description, specific details such as particular system architectures and techniques are set forth for illustrative purposes and not for limitation, in order to provide a thorough understanding of the embodiments of this application. However, those skilled in the art will understand that this application may also be implemented in other embodiments without these specific details. In other instances, detailed descriptions of well-known systems, apparatuses, circuits, and methods have been omitted so as not to obscure the description of this application with unnecessary detail.

[0022] It should be understood that, when used in this application specification and the appended claims, the term "comprising" indicates the presence of the described features, integrals, steps, operations, elements and / or components, but does not exclude the presence or addition of one or more other features, integrals, steps, operations, elements, components and / or a collection thereof.

[0023] It should also be understood that the term “and / or” as used in this application specification and the appended claims means any combination of one or more of the associated listed items and all possible combinations, and includes such combinations.

[0024] As used in this application specification and the appended claims, the term "if" may be interpreted, depending on the context, as "when," "once," "in response to determination," or "in response to detection." Similarly, the phrase "if determined" or "if detected [the described condition or event]" may be interpreted, depending on the context, as meaning "once determined," "in response to determination," "once detected [the described condition or event]," or "in response to detection [the described condition or event]."

[0025] Furthermore, in the description of this application and the appended claims, the terms "first," "second," "third," etc., are used only to distinguish descriptions and should not be construed as indicating or implying relative importance.

[0026] References to "one embodiment" or "some embodiments" as described in this specification mean that one or more embodiments of this application include a specific feature, structure, or characteristic described in connection with that embodiment. Therefore, the phrases "in one embodiment," "in some embodiments," "in other embodiments," "in still other embodiments," etc., appearing in different parts of this specification do not necessarily refer to the same embodiment, but rather mean "one or more, but not all, embodiments," unless otherwise specifically emphasized. The terms "comprising," "including," "having," and variations thereof mean "including but not limited to," unless otherwise specifically emphasized.

[0027] The present invention will be further described in detail below with reference to the accompanying drawings and specific embodiments.

[0028] ReferenceFigure 1 The diagram shows an application scenario of the V2G energy management method based on multi-layer swarm optimization and deep reinforcement learning proposed in this invention. This method addresses the practical scenario of coexisting photovoltaic power generation prediction errors and large-scale electric vehicle integration under a high proportion of renewable energy access, proposing a vehicle-grid collaborative scheduling architecture that integrates swarm intelligence optimization and deep reinforcement learning. In this architecture, the power grid control center 101 is responsible for global power balance and safe and stable operation; the load aggregator 102, as the core scheduling entity, aggregates dispersed electric vehicle clusters 103 to form a distributed flexible resource with considerable, controllable, and adjustable capabilities. Based on the method proposed in this application, the load aggregator 102 first uses the Multi-Level Group Competitive Group Optimization (MLGCSO) algorithm to perform day-ahead global optimization of the charging and discharging behavior of the electric vehicle cluster, generating an initial scheduling scheme that considers multiple objective trade-offs. Subsequently, in the real-time operation phase, a dynamic correction module based on deep reinforcement learning (DRL) adaptively adjusts the day-ahead scheme according to the error between the actual output and the predicted value of the photovoltaic power generation system 104, as well as the real-time operating status of the charging station. This enables effective management of photovoltaic power generation prediction errors, improves the level of new energy consumption, ensures the safe and stable operation of the power grid, and optimizes the economic efficiency on the user side.

[0029] Figure 2 This is a flowchart illustrating a V2G energy management method based on multi-level population optimization and deep reinforcement learning, provided in an embodiment of this application. (Refer to...) Figure 2 The method is described in detail below: This application provides a V2G energy management method based on multi-level population optimization and deep reinforcement learning, including: Step 201: Construct a comprehensive optimization objective function system for large-scale electric vehicle scenarios.

[0030] To scientifically and efficiently manage the charging and discharging behavior of large-scale electric vehicles, it is first necessary to establish a multi-objective optimization system that balances grid stability and user economics. This system, from the user's perspective, comprehensively considers four core objectives: minimizing discharging costs, minimizing charging waiting time, maximizing state of charge, and maximizing savings on user electricity costs.

[0031] Step 2011: Construct a participant model based on blind inverse game theory.

[0032] Considering the complex game-theoretic relationships among grid operators, electric vehicle users, and charging station operators in a V2G environment, this application employs blind inverse game theory to infer the true preferences of each participant from observed data. The set of participants is defined as: N = {GO, EU, CSO} In this model, GO represents the grid operator, EU represents electric vehicle users, and CSO represents the charging station operator. An observation dataset is constructed based on historical scheduling data and real-time interaction data. A normalized least squares estimator is used to simultaneously learn the revenue function parameter vector θ and rationality parameter vector λ of each participant.

[0033] Step 2012: Establish an incomplete information game framework.

[0034] Unlike traditional game theory methods, this application does not presuppose perfect rationality among participants, but rather acknowledges that their decisions may exhibit bounded rationality. The constructed strategy space includes: (1) Dispatch strategies of power grid operators; (2) Charging and discharging selection strategies for electric vehicle users; (3) Pricing and service strategies of charging station operators; By jointly learning the payoff function parameter θ and the rationality parameter λ from the observation data, the strong assumption of perfect rationality in traditional game theory is avoided.

[0035] Step 2013: Achieve entropy regularization quantization response equilibrium.

[0036] Based on the parameters inferred in step 2012, the system is guided to converge to an entropy-regularized quantized response equilibrium. This equilibrium is expressed as: in, For participants The optimal strategy distribution, For the profit function, Quantifying participants The rationality level. This equilibrium can more accurately reflect the bounded rational decision-making behavior in the V2G market.

[0037] Step 2014: Construct a multi-objective optimization function system.

[0038] Based on entropy regularization and quantization response equilibrium, a comprehensive optimization function system containing four core objectives is constructed: Step 2015: Achieve dynamic trade-offs among multiple objectives.

[0039] The dynamic trade-off between multiple objectives is achieved by solving the Pareto front, and its expression is as follows: in, For the first Adaptive weights for each objective function Here is the entropy regularization coefficient. The entropy is the policy distribution. This method, by jointly learning the reward function parameters and rationality parameters from observational data, can more accurately characterize the real decision-making behavior of participants in V2G environments.

[0040] Step 202: The comprehensive optimization objective function is globally optimized based on the Multi-Level Group Competition Group Optimization Algorithm (MLGCSO) to determine the initial charging and discharging scheduling scheme.

[0041] In the offline phase, the MLGCSO algorithm proposed in this application is used to globally optimize the comprehensive objective function constructed in step 201.

[0042] Step 2021, Particle Encoding and Initialization.

[0043] The complete charging and discharging scheduling scheme is transformed into particle position vectors using particle encoding. Specifically, the day is divided into 12 time periods, with the charging and discharging power of each electric vehicle within each time period used as the decision variable, and the particle dimension set to 12. Each encoded value is multiplied by four decimal places and decoded to represent the power exchanged between the electric vehicle and the power grid within that time period. An initial particle swarm containing multiple particles is randomly generated.

[0044] Step 2022, hierarchical grouping mechanism.

[0045] The initial particle swarm is sorted according to particle fitness values ​​and divided into three groups: Level-1, Level-2, and Level-3. This hierarchical structure not only maintains population diversity but also significantly improves global search capabilities.

[0046] Step 2023, dynamic unit partitioning.

[0047] The particle swarms at different levels are divided into basic units with different numbers of particles: Level-1 groups are randomly divided into several four-particle units; Level-2 groups are randomly divided into several three-particle units; and Level-3 groups are randomly divided into several two-particle units. This differentiated grouping strategy ensures that high-quality particles receive sufficient learning competition while avoiding the waste of computational resources on low-quality particles.

[0048] Step 2024: Competitive learning and updating.

[0049] Competition and learning take place within the basic units of each group: In a four-particle unit, the particles are sorted by fitness and labeled as particles 1, 2, 3, and 4. Particle 4 learns from particles 3, 2, and 1; particle 3 learns from particles 2 and 1; and particle 2 learns from particle 1. The particle with the best fitness, particle 1, retains its own information and directly enters the next iteration; The formulas for updating particle position and velocity are expressed as follows: Step 2025: Iterative convergence and scheme output.

[0050] The competitive learning process described above is executed iteratively until the maximum number of iterations is reached. The particle with the optimal fitness value is then output and decoded into the initial charge-discharge scheduling scheme.

[0051] Step 203: The initial charge and discharge scheduling scheme is modified in real time based on the deep reinforcement learning (DRL) method to determine the optimized V2G energy management strategy.

[0052] During the online operation phase, in order to cope with uncertainties such as photovoltaic power generation prediction errors, deep reinforcement learning is introduced to perform real-time and dynamic correction of the initial scheduling scheme generated offline.

[0053] Step 2031, Markov decision process modeling.

[0054] The uncertainty of photovoltaic power generation prediction error is modeled using a Markov decision process. State space. It contains three key elements: in, As a decision-making moment For photovoltaic power generation prediction error, To optimize charging station utilization. The operational space allows for all feasible charging and discharging of electric vehicles. .

[0055] Step 2032: Construct a reinforcement learning framework.

[0056] Construct a reinforcement learning framework with the objective of minimizing mean squared error. Instant reward. Correlation with the square of the photovoltaic power generation prediction error: Return Defined as the cumulative reward from that moment onwards: in, It serves as a discount factor, used to reduce the uncertainty risk associated with the expected value over time.

[0057] Step 2033, Deep Q-Network Design and Training.

[0058] A deep neural network is used to approximate the state-action value function, and empirical replay and target network techniques are employed to improve training stability. The update of the state-action value function is based on the Bellman optimality equation: The Q-network parameters are updated by minimizing the quadratic loss function: Step 2034: Real-time correction of strategy execution.

[0059] Based on the trained deep reinforcement learning model, the charging and discharging actions are dynamically adjusted according to the real-time environmental state. At each decision time t, the action that minimizes the state-action value function is selected. Step 2035, Constraint Management.

[0060] Throughout the real-time correction process, ensure that the V2G energy management strategy meets the following constraints: (1) Available time constraints (2) Charging and discharging power constraints (3) Charge state constraints (4) Real-time electricity price boundary constraints (5) Discharge depth safety constraints (6) Constraints on the number of charging and discharging electric vehicles By incorporating constraints into the reward function using the penalty function method, the feasibility and safety of the scheduling scheme are ensured.

[0061] In electric vehicle charging and discharging scheduling, in order to ensure that the scheduling scheme meets the global optimization objective while taking into account the real-time fluctuations of the system and the dynamic changes in user demand, this application introduces deep reinforcement learning (DRL) for real-time correction based on the initial scheduling scheme obtained by the multi-level group competitive population optimization algorithm (MLGCSO).

[0062] At this stage, the algorithm fully leverages the powerful perception and adaptive decision-making capabilities of deep reinforcement learning in handling sequential decision-making problems. Specifically, the system models the real-time operating environment as a Markov Decision Process (MDP), whose state space accurately captures key dynamic information, including the decision-making moments. Photovoltaic power generation prediction error and charging station utilization rate The action space consists of all feasible charge / discharge power adjustment commands. The reward function aims to minimize the mean square error (MSE) of photovoltaic power generation prediction errors, guiding the agent to learn the optimal policy.

[0063] Based on this framework, this application employs a deep Q-network (DQN) to approximate the complex state-action value function. This network continuously learns and optimizes its decision-making strategies by interacting extensively with the environment and stabilizing the training process through mechanisms such as experience replay and target networks. In online applications, the trained DRL model is adapted to real-time observations of the system state. Through strategy Dynamically output optimal charging and discharging action .

[0064] This real-time correction layer based on DRL can effectively address short-term fluctuations that were not accurately considered during the MLGCSO offline optimization phase, such as random changes in photovoltaic output and sudden situations involving electric vehicle access. It acts like an intelligent "fine-tuner," making precise and adaptive local adjustments to the global solution, thereby significantly improving the robustness, real-time performance, and economy of the entire V2G energy management system, achieving effective synergy between offline global optimization and online real-time decision-making.

[0065] Furthermore, although the operations of the method of this application are described in a specific order in the accompanying drawings, this does not require or imply that these operations must be performed in that specific order, or that all the operations shown must be performed to achieve the desired result. Additionally or alternatively, certain steps may be omitted, multiple steps may be combined into one step, and / or one step may be broken down into multiple steps.

[0066] This application also provides a computer-readable storage medium storing computer-executable instructions. When a processor executes the computer-executable instructions, it implements the multi-stage charging optimization scheduling method for electric vehicles provided in the above embodiments of this application.

[0067] In the above embodiments, the descriptions of each embodiment have different focuses. For parts that are not described in detail or recorded in a certain embodiment, please refer to the relevant descriptions of other embodiments.

[0068] The above embodiments are only used to illustrate the technical solutions of this application, and are not intended to limit them. Although this application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of this application, and should all be included within the protection scope of this application.

Claims

1. A V2G energy management method based on multi-level population optimization and deep reinforcement learning, characterized in that, include: A comprehensive optimization objective function system is constructed for large-scale electric vehicle scenarios. The objective function system includes minimizing discharge cost, minimizing charging waiting time, maximizing state of charge, and maximizing savings on user electricity costs. The comprehensive optimization objective function is globally optimized based on a multi-level group competition group optimization algorithm to determine the initial charging and discharging scheduling scheme. The initial charging and discharging scheduling scheme is corrected in real time using a deep reinforcement learning method to determine the optimized V2G energy management strategy; the optimized V2G energy management strategy is used to schedule the charging and discharging behavior of electric vehicles.

2. The V2G energy management method based on multi-level population optimization and deep reinforcement learning as described in claim 1, characterized in that, The construction of a comprehensive optimization objective function system for large-scale electric vehicle scenarios includes: Based on blind inverse game theory, the payoff functions and rationality levels of grid operators, electric vehicle users, and charging station operators are jointly inferred from observational data; and the constant voltage value for each stage is determined according to the battery state range. Based on the aforementioned payoff function and rationality level, an incomplete information game framework is constructed, and the system is guided to converge to an entropy-regularized quantitative response equilibrium. Based on the entropy regularization quantization response equilibrium, a multi-objective dynamic trade-off is made for minimizing discharge cost, minimizing charging waiting time, maximizing power status, and maximizing savings on user electricity costs, forming the comprehensive optimization objective function system.

3. The V2G energy management method based on multi-level population optimization and deep reinforcement learning as described in claim 1, characterized in that, The global optimization of the comprehensive objective function based on the multi-level group competition swarm optimization algorithm to determine the initial charge-discharge scheduling scheme includes: The complete charge / discharge scheduling scheme is encoded as particles, and an initial particle swarm containing multiple particles is randomly generated. The initial particle swarm is sorted according to the particle fitness value and divided into multiple levels; The particle swarms at different levels are divided into basic units with different numbers of particles. Within each basic unit, particles with poor fitness compete with particles with better fitness to learn and update their own position and velocity; the competitive learning process is executed iteratively until the maximum number of iterations is reached, and the particle with the best fitness is output and decoded into the initial charge and discharge scheduling scheme.

4. The V2G energy management method based on multi-level population optimization and deep reinforcement learning as described in claim 3, characterized in that, The division into multiple levels includes dividing the particle swarm into a first level, a second level, and a third level; the further division of the particle swarms at different levels into basic units with different numbers of particles includes: The first-level particle swarm is randomly divided into several four-particle units; The second-level particle swarm is randomly divided into several three-particle units; The third-level particle swarm is randomly divided into several two-particle units.

5. The V2G energy management method based on multi-level population optimization and deep reinforcement learning as described in claim 1, characterized in that, The real-time correction of the initial charge-discharge scheduling scheme based on deep reinforcement learning includes: The uncertainty of photovoltaic power generation prediction error is modeled by Markov decision process, and a reinforcement learning framework with the goal of minimizing mean square error is constructed. A deep neural network is used to approximate the state-action value function, and the network is trained using experience replay and target network techniques. Based on the trained deep reinforcement learning model, the charging and discharging actions are dynamically adjusted according to the real-time environmental conditions, thereby correcting the initial charging and discharging scheduling scheme.

6. The V2G energy management method based on multi-level population optimization and deep reinforcement learning as described in claim 5, characterized in that, The state space of the Markov decision process Represented as , in, For decision-making moments, For photovoltaic power generation prediction error, For charging station utilization; the action space includes all feasible charging and discharging actions. Instant rewards It is related to the square of the photovoltaic power generation prediction error.

7. The V2G energy management method based on multi-level population optimization and deep reinforcement learning as described in any one of claims 1 to 6, characterized in that, The constraints of the V2G energy management strategy include at least one of the following: electric vehicle availability time constraint, charging and discharging power constraint, state of charge constraint, real-time electricity price boundary constraint, depth of discharge safety constraint, and number of charging and discharging electric vehicles constraint.

8. The V2G energy management method based on multi-level population optimization and deep reinforcement learning as described in claim 7, characterized in that, The constraint on the number of charging and discharging electric vehicles is expressed as follows: in and They represent in The number of electric vehicles that are constantly charging and discharging. This indicates the total number of electric vehicles in the area.

9. The V2G energy management method based on multi-level population optimization and deep reinforcement learning as described in claim 1, characterized in that, The objective function that minimizes discharge cost Represented as: in, and These are charge / discharge state parameters. The charging and discharging power of the vehicle, and for The charging and discharging electricity price during different time periods, and For charging and discharging efficiency, For the cost of battery degradation, The duration of the control cycle is measured in units.

10. A V2G energy management device based on multi-level population optimization and deep reinforcement learning, characterized in that, The V2G energy management method based on multi-level population optimization and deep reinforcement learning as described in any one of claims 1 to 9 includes: The objective function construction module is used to build a comprehensive optimization objective function system for large-scale electric vehicle scenarios; The global optimization module is used to perform global optimization of the comprehensive optimization objective function based on a multi-level group competition group optimization algorithm to determine the initial charging and discharging scheduling scheme; The charging package determination module is used to construct a multi-stage modular package model based on the electric vehicle charging model and the user's personalized charging needs, and to determine the user's charging package module based on the multi-stage modular package model. The real-time correction module is used to correct the initial charge and discharge scheduling scheme in real time based on deep reinforcement learning methods, and determine the optimized V2G energy management strategy.

Images