An electric vehicle aggregated charging method assisted by reinforcement learning-based MPC
Patent Information
- Application Number
- CN202611102042.5
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2026-07-23
- Publication Date
- 2026-10-09
AI Technical Summary
但是，若直接由强化学习算法输出聚合充电功率，则难以充分保证功率边界、车辆电量状态、控制输入范围和功率变化率等运行约束，影响控制结果的可执行性和安全性
[0036]本发明提供的一种强化学习辅助MPC的电动汽车聚合充电方法，将新能源出力短时滚动预测、异质电动汽车集群聚合状态建模、强化学习算法参数修正和MPC滚动优化相结合。该方法通过Ridge多输出回归生成预测域内的新能源出力预测序列，并结合预测偏差对MPC参考功率进行修正，使滚动优化过程能够利用实时偏差信息；通过统一电量空间和电量等级聚合方式表征不同电池容量、不同初始电量状态和不同充电需求车辆的集群状态，并结合车辆接入、离开和充电迁移过程更新聚合状态向量，提高了对异质电动汽车集群可调节能力的表征能力；通过强化学习算法输出MPC参数修正动作，而不是直接输出聚合充电功率，使MPC参数能够随新能源预测信息、车辆聚合状态和运行偏差信息进行调整；通过MPC在聚合充电功率边界、功率变化率、电动汽车状态、充电设施容量和控制输入约束下滚动求解聚合充电功率，提高了聚合充电功率控制结果的可执行性和安全性。与固定参数MPC方法相比，本发明能够提高控制参数对新能源出力波动、预测偏差和车辆随机行为的适应能力；与直接由强化学习算法输出聚合充电功率的方法相比，本发明能够在保持在线自适应调节能力的同时，利用MPC显式处理运行约束。
Smart Images
Figure CN122890477A_ABST
Abstract
Description
Technical Field
[0001] This invention relates to the field of electric vehicle aggregated charging control technology, and in particular to an electric vehicle aggregated charging method using reinforcement learning-assisted model predictive control (MPC). Background Technology
[0002] With the continuous expansion of new energy power generation and the increasing number of electric vehicles, electric vehicle clusters, as flexible load resources with regulation potential, play an important role in promoting the consumption of new energy and improving the operational flexibility of the power distribution system. By aggregating and controlling charging power, it is possible to guide electric vehicle clusters to participate in grid operation regulation while meeting user charging needs and charging facility constraints.
[0003] In operational scenarios involving new energy sources, the control of aggregated charging for electric vehicles needs to consider not only the capacity of charging piles, vehicle battery status, and user charging demand, but also the aggregated charging power based on changes in new energy output. New energy output is volatile and uncertain, and short-term forecasts often deviate from actual output. Simultaneously, the access and departure behaviors of electric vehicles are influenced by user travel patterns, charging demand, and time distribution, exhibiting randomness and time-varying characteristics, causing the cluster's adjustability to change with operational status.
[0004] Currently, electric vehicle (EV) aggregated charging control methods mainly include fixed-parameter MPC methods, offline optimization tuning methods, and intelligent optimization methods. MPC can handle constraints such as aggregated charging power boundaries, power change rate, and vehicle state in the rolling time domain, but its power point tracking weights, power smoothing weights, and constraint penalty parameters usually need to be pre-set. When the output of renewable energy, prediction deviations, and vehicle aggregated state change, fixed control parameters struggle to simultaneously consider renewable energy consumption, power smoothing, and vehicle charging demand. Offline optimization tuning methods can obtain control parameters under typical operating conditions, but lack the ability to correct for real-time state changes online. Reinforcement learning algorithms can adjust strategies based on operational feedback and are suitable for control problems with strong stochasticity and time-varying characteristics. However, if the aggregated charging power is directly output by the reinforcement learning algorithm, it is difficult to fully guarantee operational constraints such as power boundaries, vehicle charge state, control input range, and power change rate, affecting the executability and safety of the control results.
[0005] In summary, existing methods still suffer from insufficient adaptability of control parameters, imprecise characterization of aggregation states, and difficulty in balancing operational constraints and online adjustment under conditions of short-term prediction bias in new energy sources, random vehicle access and departure, and dynamic changes in cluster status. Therefore, it is necessary to propose an electric vehicle aggregation charging method that combines short-term prediction information, electric vehicle aggregation states, and online parameter correction mechanisms to improve the control's adaptability to different operating conditions. Summary of the Invention
[0006] Based on the above, the purpose of this invention is to provide a reinforcement learning-assisted MPC (Multi-Processing Control) method for aggregated charging of electric vehicles. This method, after acquiring information on renewable energy output and electric vehicle operating status, performs short-term rolling prediction of renewable energy output, aggregates the electric vehicle cluster status by power level, and uses a reinforcement learning algorithm to correct MPC parameters online. The aggregated charging power is then calculated by the MPC in a rolling manner, improving the adaptability of the electric vehicle cluster to fluctuations in renewable energy output, prediction errors, and random vehicle access and departure.
[0007] To achieve the above objectives, the present invention provides a method for aggregated charging of electric vehicles using reinforcement learning-assisted MPC, the method comprising:
[0008] Step S1: Short-term rolling prediction of renewable energy output. Based on the historical output sequence, time location characteristics, and local statistical characteristics of renewable energy, the Ridge multi-output regression method is used to generate a renewable energy output prediction sequence within the prediction domain; the MPC reference power is corrected based on the deviation between the current actual output and the predicted output of renewable energy; during the rolling control process, the prediction sequence called at the current control moment is updated by combining periodic refresh and sequence shifting.
[0009] Step S2: Modeling the aggregation state of heterogeneous electric vehicle clusters. Electric vehicles with different battery capacities, different initial charge states, and different charging needs are mapped to a unified charge space and divided into multiple charge levels; the number of vehicles in each charge level is counted to form an electric vehicle cluster aggregation state vector, and the electric vehicle cluster aggregation state vector is updated according to vehicle access information, vehicle departure information, and charging migration relationships.
[0010] Step S3: MPC Parameter Correction Action Generation. Based on new energy prediction information, electric vehicle cluster aggregation status, prediction deviation information, power tracking deviation information, historical control information, and capacity margin information, a state observation of the reinforcement learning algorithm is constructed. The reinforcement learning algorithm outputs parameter correction actions, and the parameter correction actions are converted into MPC parameters through a state-related mapping function. The reinforcement learning algorithm does not directly output aggregated charging power; the parameter correction actions are used to adjust the MPC parameters after mapping.
[0011] Step S4: Solving the rolling optimization of aggregated charging power. Based on the MPC reference power, the aggregated state vector of the electric vehicle cluster, and the MPC parameters corrected by the reinforcement learning algorithm, construct the MPC rolling optimization objective; under the constraints of aggregated charging power boundary, power change rate, electric vehicle state, charging facility capacity, and control input, solve for the aggregated charging power sequence in the prediction domain.
[0012] Step S5: Aggregated charging power execution and feedback update. The aggregated charging power corresponding to the first control step in the prediction domain is selected as the current executed power. Based on the execution result, the aggregated state of the electric vehicle cluster, historical control power, prediction deviation information, and state observations of the reinforcement learning algorithm are updated before entering the next control cycle.
[0013] Optionally, the data required for the short-term rolling prediction of new energy output, the aggregation state modeling of electric vehicle clusters, and the rolling optimization of MPC include one or more of the following: historical output of new energy, current actual output, electric vehicle access information, electric vehicle departure information, electric vehicle power status, number of charging piles, rated power of a single pile, aggregated rated charging power, control step size, and prediction domain length.
[0014] Optionally, the short-term rolling forecast of new energy output specifically includes:
[0015] Step S11: Based on the historical output sequence of new energy sources, time location characteristics, and local statistical characteristics, the Ridge multi-output regression method is used to generate the predicted output sequence of new energy sources within the prediction domain;
[0016] Step S12: Based on the deviation between the current actual output of the new energy source and the predicted output at the corresponding time, obtain the prediction deviation estimate, and use the prediction deviation estimate to correct the MPC reference power;
[0017] Step S13: During the rolling control process, the predicted sequence called at the current control moment is updated by combining periodic refresh and sequence shift.
[0018] Optionally, the aggregation state modeling of the heterogeneous electric vehicle cluster specifically includes:
[0019] Step S21: Map electric vehicles with different battery capacities, different initial states of charge, and different charging needs to a unified power space, and divide the unified power space into multiple power levels;
[0020] Step S22: Count the number of vehicles in each power level to form an electric vehicle cluster aggregation state vector, which is used to characterize the distribution of vehicles in each power level.
[0021] Step S23: Update the aggregated state vector of the electric vehicle cluster based on vehicle access information, vehicle departure information, and charging migration relationship.
[0022] Optionally, the state observations of the reinforcement learning algorithm include one or more of the following: new energy prediction features, electric vehicle aggregation state features, prediction deviation features, power tracking deviation features, historical control features, and capacity margin features; wherein, the electric vehicle aggregation state features are used to characterize the distribution status of the electric vehicle cluster at each power level, the scale of connected vehicles, and the proportion of low-power vehicles.
[0023] Optionally, the generation of the MPC parameter correction action specifically includes:
[0024] Step S31: Input the state observations into the reinforcement learning algorithm to obtain parameter correction actions;
[0025] Step S32: Modify the action input state related mapping function with the parameters to obtain the MPC parameters;
[0026] The MPC parameters include one or more of the following: power tracking weights, power smoothing weights, prediction bias compensation coefficients, and prediction domain discount factors; wherein, the reinforcement learning algorithm does not directly output aggregated charging power, and the parameter correction action is used to adjust the MPC parameters after mapping.
[0027] The state observations include vehicle aggregation state characteristics, new energy prediction characteristics, prediction deviation and power tracking deviation characteristics, capacity margin characteristics, and historical control information characteristics; wherein, the vehicle aggregation state characteristics include one or more of the following: battery level distribution information, connected vehicle scale information, and low battery vehicle proportion information.
[0028] Optionally, the rolling optimization solution for the aggregated charging power specifically includes:
[0029] Step S41: Using the aggregated charging power sequence within the prediction domain as the optimization object, and constructing an MPC rolling optimization objective based on the MPC reference power, the aggregated state vector of the electric vehicle cluster, and the MPC parameters; the MPC rolling optimization objective includes one or more of the following: power tracking term, charging demand term, power smoothing term, and operational constraint term;
[0030] Step S42: Set constraints for the MPC rolling optimization process, including one or more of the following: aggregate charging power boundary constraints, power change rate constraints, electric vehicle state constraints, charging facility capacity constraints, and control input boundary constraints;
[0031] Step S43: Solve the MPC rolling optimization model to obtain the aggregated charging power sequence in the prediction domain, and execute the first control step in the aggregated charging power sequence in the current control cycle.
[0032] Optionally, the aggregated charging power is updated with feedback, specifically including:
[0033] Step S51: Select the aggregated charging power corresponding to the first control step within the prediction domain as the current execution power;
[0034] Step S52: Update the aggregated state of the electric vehicle cluster, historical control power, prediction deviation information, and state observations of the reinforcement learning algorithm based on the current execution power;
[0035] Step S53: Enter the next control cycle, re-call the short-term rolling prediction information and the updated electric vehicle cluster aggregation status, and repeat steps S1 to S5 to achieve closed-loop rolling optimization of electric vehicle aggregated charging power.
[0036] This invention provides a reinforcement learning-assisted MPC (Multi-Processing Control) method for electric vehicle (EV) aggregation charging, combining short-term rolling prediction of new energy power output, aggregation state modeling of heterogeneous EV clusters, reinforcement learning algorithm parameter correction, and MPC rolling optimization. This method generates a new energy power output prediction sequence within the prediction domain through Ridge multi-output regression and corrects the MPC reference power based on prediction bias, enabling the rolling optimization process to utilize real-time bias information. It characterizes the cluster state of vehicles with different battery capacities, initial charge states, and charging demands by using a unified energy space and energy level aggregation method, and updates the aggregation state vector based on vehicle access, departure, and charging migration processes, improving the representation of the adjustability of heterogeneous EV clusters. Instead of directly outputting aggregated charging power, the reinforcement learning algorithm outputs MPC parameter correction actions, allowing MPC parameters to be adjusted according to new energy prediction information, vehicle aggregation state, and operational bias information. By using MPC to roll-solve the aggregated charging power under constraints of aggregated charging power boundaries, power change rate, EV state, charging facility capacity, and control input, the executability and safety of the aggregated charging power control results are improved. Compared with the fixed-parameter MPC method, this invention can improve the adaptability of control parameters to fluctuations in new energy power output, prediction deviations, and random vehicle behavior; compared with the method of directly outputting aggregated charging power by reinforcement learning algorithm, this invention can explicitly handle operating constraints using MPC while maintaining online adaptive adjustment capability. Attached Figure Description
[0037] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the accompanying drawings used in the embodiments will be briefly described below. Obviously, the drawings described below are merely some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without any creative effort.
[0038] Figure 1 This is a flowchart of the electric vehicle aggregated charging method using reinforcement learning-assisted MPC according to an embodiment of the present invention;
[0039] Figure 2 This is a control structure diagram for the reinforcement learning algorithm to correct MPC parameters and perform aggregated charging and rolling optimization in an embodiment of the present invention.
[0040] Figure 3 This is a comparison chart of aggregated charging power and actual photovoltaic output under different control methods in embodiments of the present invention. Detailed Implementation
[0041] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.
[0042] The purpose of this invention is to provide a reinforcement learning-assisted MPC method for electric vehicle aggregation charging. In electric vehicle aggregation charging scenarios with new energy access, a rolling optimization input is constructed using short-term prediction information of new energy and aggregation state information of electric vehicle clusters. The MPC parameters are then corrected through a reinforcement learning algorithm, so that the aggregation charging power is solved by MPC under running constraints, thereby improving the adaptability of electric vehicle clusters to fluctuations in new energy output, prediction deviations, and random vehicle access and departure.
[0043] To make the above-mentioned objects, features and advantages of the present invention more apparent and understandable, the present invention will be further described in detail below with reference to the accompanying drawings and specific embodiments.
[0044] like Figure 1 As shown, this invention discloses a reinforcement learning-assisted MPC method for aggregated charging of electric vehicles, the method comprising:
[0045] Step S1: Short-term rolling prediction of renewable energy output. Based on the historical output sequence, time location characteristics, and local statistical characteristics of renewable energy, the Ridge multi-output regression method is used to generate a renewable energy output prediction sequence within the prediction domain; the MPC reference power is corrected based on the deviation between the current actual output and the predicted output of renewable energy; during the rolling control process, the prediction sequence called at the current control moment is updated by combining periodic refresh and sequence shifting.
[0046] Step S2: Modeling the aggregation state of heterogeneous electric vehicle clusters. Electric vehicles with different battery capacities, different initial charge states, and different charging needs are mapped to a unified charge space and divided into multiple charge levels; the number of vehicles in each charge level is counted to form an electric vehicle cluster aggregation state vector, and the electric vehicle cluster aggregation state vector is updated according to vehicle access information, vehicle departure information, and charging migration relationships.
[0047] Step S3: MPC Parameter Correction Action Generation. Based on new energy prediction information, electric vehicle cluster aggregation status, prediction deviation information, power tracking deviation information, historical control information, and capacity margin information, a state observation of the reinforcement learning algorithm is constructed. The reinforcement learning algorithm outputs parameter correction actions, and the parameter correction actions are converted into MPC parameters through a state-related mapping function. The reinforcement learning algorithm does not directly output aggregated charging power; the parameter correction actions are used to adjust the MPC parameters after mapping.
[0048] Step S4: Solve the rolling optimization problem for aggregated charging power. For example... Figure 2 Based on the MPC reference power, the aggregated state vector of the electric vehicle cluster, and the MPC parameters corrected by the reinforcement learning algorithm, an MPC rolling optimization objective is constructed. Under the constraints of aggregated charging power boundary, power change rate, electric vehicle state, charging facility capacity, and control input, the aggregated charging power sequence in the prediction domain is solved.
[0049] Step S5: Aggregated charging power execution and feedback update. The aggregated charging power corresponding to the first control step in the prediction domain is selected as the current executed power. Based on the execution result, the aggregated state of the electric vehicle cluster, historical control power, prediction deviation information, and state observations of the reinforcement learning algorithm are updated before entering the next control cycle.
[0050] The following is a detailed discussion of each step:
[0051] Step S1: Short-term rolling forecast of new energy output.
[0052] Step S11: Based on the historical output sequence, temporal location characteristics, and local statistical characteristics of the new energy source, the Ridge multi-output regression method is used to generate the predicted output sequence of the new energy source within the prediction domain. Let the current control time be t, the control step size be Δt, and the prediction domain length be N. p Then the predicted sequence of new energy output within the prediction domain is expressed as: In the formula, P pre (t) represents the predicted domain output sequence generated at time t; P pre (t+lΔt|t) is the predicted value of the new energy output at time t for the l-th future control step.
[0053] Step S12: Based on the deviation between the current actual output of the new energy source and the predicted output at the corresponding time, obtain a prediction deviation estimate, and use the prediction deviation estimate to correct the MPC reference power. The reference power is expressed as: In the formula, P ref (t+lΔt|t) is the reference power for the l-th control step within the prediction domain; κ e (t) is the prediction bias compensation coefficient; ρ is the bias compensation attenuation coefficient; e(t) is the prediction bias estimate.
[0054] Step S13: During the rolling control process, the prediction sequence called at the current control moment is updated using a combination of periodic refresh and sequence shifting. When the current control moment is the prediction refresh moment, the new energy output prediction sequence within the prediction domain is regenerated; when the current control moment is not the prediction refresh moment, the most recently generated prediction sequence is shifted to obtain the prediction sequence called at the current control moment.
[0055] Step S2: Model the aggregation state of heterogeneous electric vehicle clusters.
[0056] Step S21: Map electric vehicles with different battery capacities, different initial states of charge, and different charging requirements to a unified power space, and divide the unified power space into multiple power levels.
[0057] Step S22: Count the number of vehicles within each battery level to form an aggregated state vector for the electric vehicle cluster. In the formula, q(t) is the aggregated state vector of the electric vehicle cluster at time t; q j (t) represents the number of vehicles in the j-th power level; n represents the number of power levels.
[0058] Step S23: Update the aggregated state vector of the electric vehicle cluster based on vehicle access information, vehicle departure information, and charging migration relationship.
[0059] The aggregation state update relationship of the electric vehicle cluster is represented as follows: In the formula, A(u(t)) is the charging migration matrix related to the normalized charging control quantity u(t); q arr (t) is the vehicle access state vector; q dep (t) is the vehicle departure state vector.
[0060] Through the above steps, the electric vehicle cluster aggregation state model can describe the cluster state changes of heterogeneous electric vehicles under the combined effects of vehicle access, vehicle departure, and charging migration, providing MPC with vehicle state constraints and adjustability information.
[0061] Step S3: MPC parameter correction action generation.
[0062] Step S31: Based on new energy prediction information, electric vehicle cluster aggregation status, prediction deviation information, power tracking deviation information, historical control information, and capacity margin information, construct the state observations for the reinforcement learning algorithm. The state observations are represented as follows: In the formula, s(t) represents the state observation; s q (t) represents the vehicle aggregation state feature; s p (t) represents the predictive characteristics of new energy sources; s e (t) represents the characteristics of prediction error and power tracking error; s c (t) represents the capacity margin characteristic; s h (t) represents the characteristics of historical control information.
[0063] Step S32: Input the state observations into the reinforcement learning algorithm to obtain parameter correction actions, and input the parameter correction actions into the state correlation mapping function to obtain MPC parameters: , In the formula, π φ (·) represents the policy mapping of the reinforcement learning algorithm; φ represents the policy network parameters; z(t) represents the parameter correction action; θ(t) represents the MPC parameters; and M(·) represents the state-related mapping function. The MPC parameters include one or more of the following: power tracking weights, power smoothing weights, prediction bias compensation coefficients, and prediction domain discount factors. The reinforcement learning algorithm does not directly output the aggregated charging power; the parameter correction action, after mapping, is used to adjust the MPC parameters.
[0064] Step S4: Solve the rolling optimization problem for aggregated charging power.
[0065] Step S41: At the current control time t, using the aggregated charging power sequence within the prediction domain as the optimization object, and based on the MPC reference power, the aggregation state of the electric vehicle cluster, and the MPC parameters, construct the MPC rolling optimization objective. The aggregated charging power sequence within the prediction domain is expressed as: In the formula, P EV(t) represents the aggregated charging power sequence within the prediction domain; P EV (t+lΔt|t) represents the aggregated charging power decision for the l-th control step at time t.
[0066] The rolling optimization objective is expressed as: In the formula, J(t) is the MPC objective function; J track (t) represents the power point tracking term, used to describe the deviation between the aggregated charging power and the reference power; J demand (t) represents the charging demand term, used to characterize vehicle charging demand and the requirements for improving the condition of vehicles with low battery levels; J smooth (t) is the power smoothing term, used to suppress drastic fluctuations in aggregated charging power; J constraint (t) represents the operational constraints, which describe constraints such as aggregate charging power, vehicle battery status, and control boundaries.
[0067] Step S42: Set the constraints for the MPC rolling optimization process. The constraints include one or more of the following: aggregate charging power boundary constraints, power change rate constraints, electric vehicle state constraints, charging facility capacity constraints, and control input boundary constraints. The MPC rolling optimization process satisfies the following constraints: In the formula, Ω q Ω represents the feasible region of the aggregate state. P For the feasible domain of aggregated charging power; Ω u To control the input feasible region.
[0068] Step S43: Solve the MPC rolling optimization model to obtain the aggregated charging power sequence within the prediction domain. In each control cycle, MPC only executes the aggregated charging power corresponding to the first control step within the prediction domain, and re-solves the model in the next control cycle based on the updated new energy prediction information and the aggregated state of the electric vehicle cluster.
[0069] Step S5: Aggregated charging power execution and feedback update.
[0070] Step S51: Select the aggregated charging power corresponding to the first control step in the MPC rolling optimization results as the current execution power: In the formula, P EV,exe (t) represents the actual aggregate charging power performed at the current moment; P EV * (t|t) represents the aggregated charging power corresponding to the first control step in the MPC rolling optimization results.
[0071] Step S52: Update the aggregated state of the electric vehicle cluster, historical control power, prediction deviation information, and state observations of the reinforcement learning algorithm based on the current execution power.
[0072] Step S53: Enter the next control cycle, re-call the short-term rolling prediction information and the updated electric vehicle cluster aggregation status, and repeat steps S1 to S5 to achieve closed-loop rolling optimization of electric vehicle aggregated charging power.
[0073] In one embodiment, the method of the present invention is compared with the optimization algorithm-MPC method and the fixed parameter MPC method. The experimental results are as follows: Figure 3 As shown. Figure 3 In the diagram, the horizontal axis represents time, and the vertical axis represents power. The actual photovoltaic output curve represents the actual photovoltaic power output curve. Reinforcement learning-MPC represents the aggregated charging power curve obtained using the method of this invention. The optimization algorithm-MPC and fixed parameter MPC represent the aggregated charging power curves obtained by the corresponding comparative methods. Under the experimental conditions shown, the aggregated charging power obtained using the method of this invention is closer to the actual photovoltaic power output trend than the optimization algorithm-MPC and fixed parameter MPC in most time periods. This indicates that online correction of MPC parameters through reinforcement learning can improve the adaptability of aggregated charging control to photovoltaic power output fluctuations and changes in operating conditions.
[0074] The various embodiments in this specification are described in a progressive manner, with each embodiment focusing on the differences from other embodiments. The same or similar parts between the various embodiments can be referred to each other.
[0075] This document uses specific examples to illustrate the principles and implementation methods of the present invention. The descriptions of the above embodiments are only for the purpose of helping to understand the method and core ideas of the present invention. Furthermore, those skilled in the art will recognize that, based on the ideas of the present invention, there will be changes in the specific implementation methods and application scope. Therefore, the content of this specification should not be construed as a limitation of the present invention.
Claims
1. A reinforcement learning-assisted MPC method for aggregated charging of electric vehicles, characterized in that, The method includes: Step S1: Short-term rolling prediction of new energy power output; Based on the historical power output sequence, time location characteristics and local statistical characteristics of new energy, the Ridge multi-output regression method is used to generate a new energy power output prediction sequence within the prediction domain; Based on the deviation between the current actual power output and the predicted power output of new energy, the MPC reference power is corrected; During the rolling control process, the prediction sequence called at the current control moment is updated by combining periodic refresh and sequence shifting. Step S2: Model the aggregation state of heterogeneous electric vehicle clusters; map electric vehicles with different battery capacities, different initial charge states, and different charging needs to a unified charge space and divide them into multiple charge levels; count the number of vehicles in each charge level to form an electric vehicle cluster aggregation state vector, and update the electric vehicle cluster aggregation state vector according to vehicle access information, vehicle departure information, and charging migration relationship. Step S3: MPC parameter correction action generation; Based on new energy prediction information, electric vehicle cluster aggregation status, prediction deviation information, power tracking deviation information, historical control information, and capacity margin information, construct the state observations of the reinforcement learning algorithm; Use the reinforcement learning algorithm to output parameter correction actions, and convert the parameter correction actions into MPC parameters through a state-related mapping function; Wherein, the reinforcement learning algorithm does not directly output aggregated charging power, and the parameter correction actions are used to adjust the MPC parameters after mapping; Step S4: Rolling optimization solution for aggregated charging power; Based on the MPC reference power, the aggregated state vector of the electric vehicle cluster, and the MPC parameters corrected by the reinforcement learning algorithm, construct the MPC rolling optimization objective; Under the constraints of aggregated charging power boundary, power change rate, electric vehicle state, charging facility capacity, and control input, solve the aggregated charging power sequence in the prediction domain; Step S5: Aggregated charging power execution and feedback update; Select the aggregated charging power corresponding to the first control step in the prediction domain as the current execution power, and update the aggregated state of the electric vehicle cluster, historical control power, prediction deviation information and state observations of the reinforcement learning algorithm according to the execution result, and enter the next control cycle.
2. The electric vehicle aggregated charging method with reinforcement learning-assisted MPC according to claim 1, characterized in that, The short-term rolling forecast of new energy output specifically includes: Step S11: Based on the historical output sequence of new energy sources, time location characteristics, and local statistical characteristics, the Ridge multi-output regression method is used to generate the predicted output sequence of new energy sources within the prediction domain; Step S12: Based on the deviation between the current actual output of the new energy source and the predicted output at the corresponding time, obtain the prediction deviation estimate, and use the prediction deviation estimate to correct the MPC reference power; Step S13: During the rolling control process, the predicted sequence called at the current control moment is updated by combining periodic refresh and sequence shift.
3. The electric vehicle aggregated charging method with reinforcement learning-assisted MPC according to claim 1, characterized in that, The aggregation state modeling of the heterogeneous electric vehicle cluster specifically includes: Step S21: Map electric vehicles with different battery capacities, different initial states of charge, and different charging needs to a unified power space, and divide the unified power space into multiple power levels; Step S22: Count the number of vehicles in each power level to form an aggregated state vector of electric vehicle clusters; Step S23: Update the aggregated state vector of the electric vehicle cluster based on vehicle access information, vehicle departure information, and charging migration relationship.
4. The electric vehicle aggregated charging method with reinforcement learning-assisted MPC according to claim 1, characterized in that, The generation of the MPC parameter correction action specifically includes: Step S31: Input the state observations into the reinforcement learning algorithm to obtain parameter correction actions; Step S32: Modify the action input state related mapping function with the parameters to obtain the MPC parameters.
5. The electric vehicle aggregated charging method with reinforcement learning-assisted MPC according to claim 4, characterized in that, The state observations include vehicle aggregation state characteristics, new energy prediction characteristics, prediction deviation and power tracking deviation characteristics, capacity margin characteristics, and historical control information characteristics; wherein, the vehicle aggregation state characteristics include one or more of the following: battery level distribution information, connected vehicle scale information, and low battery vehicle proportion information.
6. The electric vehicle aggregated charging method with reinforcement learning-assisted MPC according to claim 4, characterized in that, The MPC parameters include one or more of the following: power tracking weights, power smoothing weights, prediction bias compensation coefficients, and prediction domain discount factors; wherein, the reinforcement learning algorithm does not directly output aggregated charging power, and the parameter correction action is used to adjust the MPC parameters after mapping.
7. The electric vehicle aggregated charging method with reinforcement learning-assisted MPC according to claim 1, characterized in that, The rolling optimization solution for the aggregated charging power specifically includes: Step S41: Using the aggregated charging power sequence in the prediction domain as the optimization object, and constructing the MPC rolling optimization objective based on the MPC reference power, the aggregated state vector of the electric vehicle cluster, and the MPC parameters; Step S42: Set the constraints for the MPC rolling optimization process; Step S43: Solve the MPC rolling optimization model to obtain the aggregated charging power sequence in the prediction domain.
8. The electric vehicle aggregated charging method with reinforcement learning-assisted MPC according to claim 7, characterized in that, The MPC rolling optimization objective includes one or more of the following: power tracking term, charging demand term, power smoothing term, and operational constraint term.
9. The electric vehicle aggregated charging method with reinforcement learning-assisted MPC according to claim 7, characterized in that, The constraints include one or more of the following: aggregate charging power boundary constraints, power change rate constraints, electric vehicle state constraints, charging facility capacity constraints, and control input constraints.
10. The electric vehicle aggregated charging method with reinforcement learning-assisted MPC according to claim 1, characterized in that, The aggregated charging power execution and feedback update specifically includes: Step S51: Select the aggregated charging power corresponding to the first control step within the prediction domain as the current execution power; Step S52: Update the aggregated state of the electric vehicle cluster, historical control power, prediction deviation information, and state observations of the reinforcement learning algorithm based on the current execution power; Step S53: Enter the next control cycle, re-call the short-term rolling prediction information and the updated electric vehicle cluster aggregation status, and repeat steps S1 to S5 to achieve closed-loop rolling optimization of electric vehicle aggregated charging power.