Electric vehicle aggregator assisted frequency modulation method based on action space abstraction

Abstract

The application discloses a kind of electric vehicle aggregator auxiliary frequency modulation methods based on action space abstraction, comprising: obtaining the power grid operating state information and electric vehicle cluster operating state information in current control period;Build target regulation power;Continuous state is constructed system, and discrete state is obtained by discretization processing current;Continuous charging and discharging control action is mapped to a finite number of discrete candidate scheduling strategy set, target scheduling strategy is selected, and actual control power of current time is generated;Aggregator injection error is calculated;The next time power grid frequency deviation and the next time aggregate state of charge are updated;Instantaneous reward function is constructed, and state-action value function is updated online;Electric vehicle aggregator auxiliary frequency modulation is realized.The application can consider the reward mechanism of frequency deviation cost, frequency modulation tracking error cost and battery degradation cost simultaneously, realize the coordination optimization between power grid frequency support effect, frequency modulation service quality and battery life protection.

Description

Technical Field

[0001] This invention belongs to the field of power and relates to electric vehicle aggregation dispatch control and power system ancillary service technology, specifically to an electric vehicle aggregator auxiliary frequency regulation method based on action space abstraction. Background Technology

[0002] With the continuous increase in the proportion of new energy power generation and the large-scale grid connection of renewable energy sources such as wind power and photovoltaics, the random volatility and uncertainty of the power system have significantly increased. Due to the intermittent and fluctuating nature of renewable energy output, the grid frequency stability faces higher requirements, and the demand for ancillary service resources with high response speed and high regulation accuracy is constantly increasing, especially the demand for secondary frequency regulation and automatic generation control ancillary services.

[0003] However, existing technologies have at least the following problems: First, existing centralized optimization methods are difficult to meet the requirements of real-time performance and computational efficiency for assisted frequency regulation online control; second, existing fixed rule control methods lack state awareness and policy adaptation capabilities, making it difficult to take into account the needs of multi-objective optimization; third, existing reinforcement learning methods suffer from the curse of dimensionality and difficulties in online convergence when directly processing continuous high-dimensional action spaces; and fourth, existing reward function designs are not flexible enough in coordinating frequency deviation, frequency regulation tracking error, and battery degradation, making it difficult to adapt to the dynamic changes in the state of the power grid and the fleet. Summary of the Invention

[0004] Purpose of the invention: To address the problems of insufficient real-time performance, weak adaptability, and poor overall control performance in existing technologies for electric vehicle aggregation resources to participate in grid auxiliary frequency regulation, this invention provides an electric vehicle aggregation auxiliary frequency regulation method based on action space abstraction. This method enables rapid response to grid frequency regulation needs while meeting the target state of charge constraints before vehicle disconnection, and also takes into account frequency regulation service quality, battery degradation costs, and electric vehicle cluster operation constraints, thereby improving the real-time performance, adaptability, and overall control performance of electric vehicle aggregation resources participating in grid auxiliary services.

[0005] Technical Solution: To achieve the above objectives, this invention provides an electric vehicle aggregator-assisted frequency modulation method based on action space abstraction, comprising the following steps:

[0006] S1: Obtain the grid operation status information and electric vehicle cluster operation status information within the current control cycle. The grid operation status information includes grid frequency deviation and automatic generation control commands. The electric vehicle cluster operation status information includes baseline charging power, aggregated state of charge, and aggregated charge and discharge power boundaries.

[0007] S2: Construct the target regulation power based on the baseline charging power and automatic generation control commands;

[0008] S3: Construct the continuous state of the system based on the current grid frequency deviation and aggregated state of charge, and discretize the continuous state of the system to obtain the current discrete state;

[0009] S4: Using the action space abstraction method, the continuous charging and discharging control actions are mapped to a finite set of discrete candidate scheduling strategies, and the target scheduling strategy is selected according to the current discrete state;

[0010] S5: Generate the actual control power at the current moment using the target scheduling strategy, and ensure that the actual control power satisfies the aggregated charging and discharging power boundary constraints;

[0011] S6: Calculate the aggregator injection error based on the target adjustment power and the actual control power;

[0012] S7: Update the grid frequency deviation at the next moment based on the aggregator injection error, and update the aggregated state of charge at the next moment based on the actual control power;

[0013] S8: Construct an instantaneous reward function that simultaneously considers the cost of frequency deviation, the cost of frequency modulation tracking error, and the cost of battery degradation, and use the Q-learning algorithm to update the state-action value function online;

[0014] S9: Repeat steps S1 to S8 in subsequent control cycles to achieve auxiliary frequency regulation of electric vehicle aggregator while meeting the target state of charge constraint before vehicle disconnection.

[0015] Further, step S1 includes:

[0016] A1: Obtain the power grid operating status information for the current control cycle, including power grid frequency deviation information. Automatic power generation control command information and information related to power grid disturbances ;

[0017] A2: Obtain the operating status information of the electric vehicle cluster within the current control cycle. The operating status information of the electric vehicle cluster includes the baseline charging power. Polymerized state of charge And aggregated charge and discharge power boundary information, including aggregated state of charge. satisfy:

[0018]

[0019] in, This represents the number of electric vehicles connected to the aggregator during the current control period. Indicates the first electric vehicles in the first State of charge within each control cycle;

[0020] A3: Obtain individual constraint information for each connected electric vehicle, and calculate the aggregated maximum charging power based on the individual constraint information. and maximum discharge power This makes the actual control power satisfy:

[0021]

[0022] in, For the first The actual control power of the electric vehicle cluster within each control cycle; individual constraint information includes the current state of charge of a single vehicle, the maximum charging power of a single vehicle, the maximum discharging power of a single vehicle, the expected off-grid time, and the target state of charge;

[0023] A4: The acquired power grid operation status information and electric vehicle cluster operation status information are synchronized in time, aggregated and preprocessed to form the input data required for subsequent control.

[0024] Further, step S2 includes:

[0025] B1: Obtain the baseline charging power within the current control cycle. and automatic power generation control commands ;

[0026] B2: Based on baseline charging power and automatic power generation control commands Construct target regulation power ,satisfy:

[0027]

[0028] in, Indicates the first The target regulating power within each control cycle.

[0029] Further, step S3 includes:

[0030] C1: Extract the grid frequency deviation within the current control cycle and polymer charge state and with grid frequency deviation and polymer charge state Together they constitute the continuous state of the system ,satisfy:

[0031]

[0032] in, Indicates the first The continuous state of the system within each control cycle;

[0033] C2: The power grid frequency deviation is processed according to a preset discretization rule. Divide the intervals to obtain the corresponding frequency discrete interval numbers. And according to the preset discretization rules, the aggregated charge state is... Divide the data into intervals to obtain the corresponding discrete interval numbers for the state of charge. ,satisfy:

[0034]

[0035] in, This represents a frequency deviation interval mapping function. Represents the mapping function for the aggregate charged state interval;

[0036] C3: Numbered according to the frequency discrete interval and discrete interval numbering of the state of charge For the continuous state of the system Perform a combined mapping to obtain the current discrete state. ,satisfy:

[0037]

[0038] in, Indicates the first The current discrete state within each control cycle. This represents a discrete state combination mapping function.

[0039] Further, step S4 includes:

[0040] D1: Based on the operational requirements of electric vehicle aggregators in auxiliary frequency regulation scenarios, the continuous charging and discharging control actions are abstracted into a finite number of discrete candidate scheduling strategies, and a set of discrete candidate scheduling strategies is constructed. ,satisfy:

[0041]

[0042] in, Represents a set of discrete candidate scheduling policies. Indicates the first One candidate scheduling strategy, This represents the total number of candidate scheduling strategies;

[0043] D2: Based on the current discrete state State-action value function corresponding to each candidate scheduling strategy In the discrete candidate scheduling strategy set Select target scheduling strategy ,satisfy:

[0044]

[0045] in, Indicates the first The target scheduling strategy selected within each control cycle. Represents the current discrete state Next candidate scheduling strategy The corresponding state-action value function value. In the discrete candidate scheduling policy set Select the candidate scheduling strategy that maximizes the state-action value function.

[0046] D3: During the strategy selection process, an ε-greedy mechanism is used to switch between randomly selecting candidate scheduling strategies and selecting the target scheduling strategy according to step D2. When the random number... At that time, from the discrete candidate scheduling strategy set Randomly select a candidate scheduling strategy; when At that time, select the target scheduling strategy according to step D2, where, It is a random number. The preset exploration rate.

[0047] Further, step S5 includes:

[0048] E1: Based on the target scheduling strategy selected within the current control cycle. and the current discrete state Generate the actual control power at the current moment. ,satisfy:

[0049]

[0050] in, Indicates the first The actual control power of the electric vehicle cluster within each control cycle Indicates the first The target scheduling strategy selected within each control cycle. Indicates the first The current discrete state within each control cycle;

[0051] E2: Based on the target scheduling strategy The corresponding control intent generates a value corresponding to the current discrete state. Matching charging and discharging power outputs allow different target scheduling strategies to correspond to different actual control power generation methods;

[0052] E3: Actual control power generated Perform constraint verification to ensure that the actual control power meets the aggregate charge and discharge power boundary constraints:

[0053]

[0054] in, Indicates the first The aggregate maximum discharge power of the electric vehicle cluster within each control cycle Indicates the first The aggregated maximum charging power of the electric vehicle cluster within each control cycle;

[0055] E4: When the actual control power generated by the target scheduling strategy exceeds the boundary constraint of the aggregated charging and discharging power, the actual control power is boundary corrected to limit the corrected actual control power to the range. Inside;

[0056] E5: The actual control power that satisfies the boundary constraints of aggregated charge and discharge power. This represents the actual power output of the electric vehicle cluster during the current control cycle.

[0057] Furthermore, the formula for calculating the aggregator injection error in step S6 is as follows:

[0058]

[0059] in, Indicates the first Aggregator injection error within each control cycle Indicates the first Target adjustment power within each control cycle Indicates the first The actual control power within each control cycle.

[0060] Further, step S7 includes:

[0061] F1: Get the aggregator injection error within the current control cycle. Information related to power grid disturbances Current power grid frequency deviation Current aggregate state of charge and actual control power ;

[0062] F2: Based on aggregator injection error Information related to power grid disturbances Calculate the total power imbalance of the system ,satisfy:

[0063]

[0064] in, Indicates the first Total power imbalance of the system within each control cycle Indicates the first Random net load disturbance of the power grid or equivalent external disturbance power within a control cycle Indicates the first Aggregator injection error within each control cycle;

[0065] F3: Based on the current power grid frequency deviation And the total power imbalance of the system Update the grid frequency deviation for the next time step. ,satisfy:

[0066]

[0067] in, Indicates the first Power grid frequency deviation within each control cycle Indicates the first Power grid frequency deviation within each control cycle Indicates the control cycle. Represents the system's equivalent inertia constant. This represents the system's equivalent damping coefficient. Indicates the first Total power imbalance of the system within each control cycle;

[0068] F4: Based on the actual control power within the current control cycle Update the aggregated state of charge at the next time step. ,satisfy:

[0069]

[0070] in, Indicates the first The aggregated state of charge within each control cycle Indicates the first Aggregate charge state within each control cycle This represents the charging and discharging efficiency function corresponding to the actual control power direction, when... When charging efficiency is measured, Discharge efficiency is taken at that time. Indicates the first The actual control power of the electric vehicle cluster within each control cycle Indicates the control cycle. This indicates the total capacity of the electric vehicle cluster.

[0071] Further, step S8 includes:

[0072] G1: Based on the updated grid frequency deviation at the next time step Aggregator injection error and actual control power Construct the instant reward function for the current control period. ,satisfy:

[0073]

[0074] in, Indicates the first Instant reward value within each control period This represents the frequency deviation cost term. This represents the cost term for frequency modulation tracking error. This represents the cost of battery degradation, and the negative sign indicates that the immediate reward is maximized by minimizing each cost term.

[0075] G2: Based on the power grid frequency deviation at the next moment and the current discrete state Construct a frequency deviation cost term that satisfies:

[0076]

[0077] in, Represents the current discrete state The relevant frequency deviation weights are used to adjust the influence of the frequency deviation cost term on the immediate reward function under different operating conditions. Indicates the first Power grid frequency deviation within one control cycle;

[0078] G3: Based on aggregator injection error Construct a frequency modulation tracking error cost term that satisfies:

[0079]

[0080] in, Indicates the frequency modulation tracking error weight. Indicates the first Aggregator injection error within each control cycle;

[0081] G4: Based on actual control power Construct a battery degradation cost term that satisfies:

[0082]

[0083] in, This represents the battery degradation cost function. Indicates the first The actual control power of the electric vehicle cluster within each control cycle Indicates the first Aggregate charge state within each control cycle Indicates parameters related to battery temperature or equivalent degradation.

[0084] G5: Based on the current discrete state The selected target scheduling strategy Instant reward value and the discrete state at the next moment The Q-learning algorithm is used to study the state-action value function. Perform online updates to meet the following requirements:

[0085]

[0086] in, Indicates the current discrete state Select target scheduling strategy The corresponding state-action value function value. Indicates the learning rate. Indicates the discount factor. Represents a set of discrete candidate scheduling policies. Indicates the discrete state at the next time step. The maximum value among the state-action value functions corresponding to each candidate scheduling strategy.

[0087] Further, step S9 includes:

[0088] H1: After the current control cycle ends, the next control cycle begins. The power grid operation status information and electric vehicle cluster operation status information are reacquired, and the input data for the next control cycle is generated.

[0089] H2: In the next control cycle, repeat steps S1 to S8 to reconstruct the target regulation power, system state, target scheduling strategy, actual control power, aggregator injection error, grid frequency deviation at the next moment, aggregated state of charge at the next moment, instantaneous reward function, and state-action value function to form a closed-loop online control process.

[0090] H3: During the repeated execution of steps S1 to S8, continuously monitor the current state of charge, remaining access time, expected disconnection time and target state of charge of each connected electric vehicle, and determine whether each connected electric vehicle meets the target state of charge constraint before disconnection.

[0091] H4: When the remaining access time of any connected electric vehicle is only sufficient to reach its target state of charge, restrict the connected electric vehicle from participating in discharge regulation, and / or adjust the aggregated charge and discharge power boundary of the electric vehicle cluster to ensure that each connected electric vehicle reaches its target state of charge before leaving the grid.

[0092] H5: Under the premise of meeting the target state of charge constraint before the vehicle leaves the grid, the auxiliary frequency regulation of the electric vehicle aggregator is realized by continuously repeating steps S1 to S8 in the subsequent control cycle.

[0093] This invention provides a reward mechanism that simultaneously considers the costs of frequency deviation, frequency regulation tracking error, and battery degradation. Under the premise of ensuring the target state of charge constraint before the vehicle leaves the grid, it achieves coordinated optimization between grid frequency support effect, frequency regulation service quality, and battery life protection.

[0094] Beneficial Effects: Compared with existing technologies, this invention introduces an action space abstraction method, transforming the continuous charging and discharging control problem into a selection problem of a finite number of discrete candidate scheduling strategies. This significantly reduces computational complexity and improves the real-time performance and engineering feasibility of scheduling decisions. Simultaneously, by combining the Q-learning algorithm to construct an online strategy update mechanism, the control strategy can adaptively adjust according to changes in grid frequency deviation and the state of charge of the electric vehicle cluster, overcoming the lack of flexibility in traditional fixed-rule methods. Furthermore, this invention designs a multi-objective reward mechanism that simultaneously considers the costs of frequency deviation, frequency regulation tracking error, and battery degradation, achieving coordinated optimization between grid frequency support, frequency regulation service quality, and battery life protection. It can rapidly enhance regulation capabilities to improve system stability when frequency disturbances are large, and suppress unnecessary charging and discharging behaviors during small high-frequency fluctuations, thereby effectively reducing battery degradation costs. In addition, this invention fully considers the target state of charge constraints before vehicle disconnection during the scheduling process, ensuring the fulfillment of user charging needs by dynamically adjusting the aggregated charging and discharging power boundary. Moreover, the method of this invention does not rely on an accurate model and can operate stably in complex grid environments with random disturbances and uncertainties, demonstrating good robustness and practical application value. Attached Figure Description

[0095] Figure 1 This is a schematic flowchart of the method of the present invention;

[0096] Figure 2 This is a dynamic schematic diagram of the strategy selection method of the present invention during a representative operating cycle.

[0097] Figure 3 A comparison chart of grid frequency deviation distribution and battery degradation costs under different control strategies;

[0098] Figure 4 This is a comparison chart of scheduling decisions under different control strategies during periods of high volatility. Detailed Implementation

[0099] The present invention will be further illustrated below with reference to the accompanying drawings and specific embodiments. It should be understood that these embodiments are for illustrative purposes only and are not intended to limit the scope of the invention. After reading this invention, any modifications of the invention in various equivalent forms by those skilled in the art will fall within the scope defined by the appended claims.

[0100] Example 1:

[0101] like Figure 1 As shown, this embodiment provides an electric vehicle aggregator-assisted frequency modulation method based on action space abstraction, including the following steps:

[0102] S1: Obtain the grid operation status information and electric vehicle cluster operation status information within the current control cycle. The grid operation status information includes grid frequency deviation and automatic generation control commands. The electric vehicle cluster operation status information includes baseline charging power, aggregated state of charge, and aggregated charge and discharge power boundaries.

[0103] Step S1 includes:

[0104] A1: Obtain the power grid operating status information for the current control cycle, including power grid frequency deviation information. Automatic power generation control command information and information related to power grid disturbances ;

[0105] A2: Obtain the operating status information of the electric vehicle cluster within the current control cycle. The operating status information of the electric vehicle cluster includes the baseline charging power. Polymerized state of charge And aggregated charge and discharge power boundary information, including aggregated state of charge. satisfy:

[0106]

[0107] in, This represents the number of electric vehicles connected to the aggregator during the current control period. Indicates the first electric vehicles in the first State of charge within each control cycle;

[0108] A3: Obtain individual constraint information for each connected electric vehicle, and calculate the aggregated maximum charging power based on the individual constraint information. and maximum discharge power This makes the actual control power satisfy:

[0109]

[0110] in, For the first The actual control power of the electric vehicle cluster within each control cycle; individual constraint information includes the current state of charge of a single vehicle, the maximum charging power of a single vehicle, the maximum discharging power of a single vehicle, the estimated off-grid time, and the target state of charge; when the remaining access time of any electric vehicle is only sufficient to meet its target state of charge, the dischargeable power of that electric vehicle is set to zero.

[0111] A4: The acquired power grid operation status information and electric vehicle cluster operation status information are synchronized in time, aggregated and preprocessed to form the input data required for subsequent control.

[0112] S2: Construct the target regulation power based on the baseline charging power and automatic generation control commands;

[0113] Step S2 includes:

[0114] B1: Obtain the baseline charging power within the current control cycle. and automatic power generation control commands ;

[0115] B2: Based on baseline charging power and automatic power generation control commands Construct target regulation power ,satisfy:

[0116]

[0117] in, Indicates the first Target adjustment power within each control cycle;

[0118] B3: Adjust target power This serves as a target reference for generating actual control power in subsequent candidate scheduling strategies and is used for subsequent aggregator injection error calculation.

[0119] S3: Construct the continuous state of the system based on the current grid frequency deviation and aggregated state of charge, and discretize the continuous state of the system to obtain the current discrete state;

[0120] Step S3 includes:

[0121] C1: Extract the grid frequency deviation within the current control cycle and polymer charge state and with grid frequency deviation and polymer charge state Together they constitute the continuous state of the system ,satisfy:

[0122]

[0123] in, Indicates the first The continuous state of the system within each control cycle;

[0124] C2: The power grid frequency deviation is processed according to a preset discretization rule. Divide the intervals to obtain the corresponding frequency discrete interval numbers. And according to the preset discretization rules, the aggregated charge state is... Divide the data into intervals to obtain the corresponding discrete interval numbers for the state of charge. ,satisfy:

[0125]

[0126] in, This represents a frequency deviation interval mapping function. Represents the mapping function for the aggregate charged state interval;

[0127] C3: Numbered according to the frequency discrete interval and discrete interval numbering of the state of charge For the continuous state of the system Perform a combined mapping to obtain the current discrete state. ,satisfy:

[0128]

[0129] in, Indicates the first The current discrete state within each control cycle. Represents a discrete-state combination mapping function;

[0130] C4: Set the current discrete state It serves as the state input for subsequent discrete candidate scheduling strategy selection and state-action value function update.

[0131] S4: Using the action space abstraction method, the continuous charging and discharging control actions are mapped to a finite set of discrete candidate scheduling strategies, and the target scheduling strategy is selected according to the current discrete state;

[0132] Step S4 includes:

[0133] D1: Based on the operational requirements of electric vehicle aggregators in auxiliary frequency regulation scenarios, the continuous charging and discharging control actions are abstracted into a finite number of discrete candidate scheduling strategies, and a set of discrete candidate scheduling strategies is constructed. ,satisfy:

[0134]

[0135] in, Represents a set of discrete candidate scheduling policies. Indicates the first One candidate scheduling strategy, This represents the total number of candidate scheduling strategies;

[0136] Discrete candidate scheduling strategy set It includes two or more of the following strategies: baseline tracking strategy, frequency modulation target tracking strategy, conservative up-adjustment strategy, conservative down-adjustment strategy, and reserved frequency modulation capacity strategy.

[0137] D2: Based on the current discrete state State-action value function corresponding to each candidate scheduling strategy In the discrete candidate scheduling strategy set Select target scheduling strategy ,satisfy:

[0138]

[0139] in, Indicates the first The target scheduling strategy selected within each control cycle. Represents the current discrete state Next candidate scheduling strategy The corresponding state-action value function value. In the discrete candidate scheduling policy set Select the candidate scheduling strategy that maximizes the state-action value function.

[0140] D3: During the strategy selection process, an ε-greedy mechanism is used to switch between randomly selecting candidate scheduling strategies and selecting the target scheduling strategy according to step D2. When the random number... At that time, from the discrete candidate scheduling strategy set Randomly select a candidate scheduling strategy; when At that time, select the target scheduling strategy according to step D2, where, It is a random number. The preset exploration rate.

[0141] S5: Generate the actual control power at the current moment using the target scheduling strategy, and ensure that the actual control power satisfies the aggregated charging and discharging power boundary constraints;

[0142] Step S5 includes:

[0143] E1: Based on the target scheduling strategy selected within the current control cycle. and the current discrete state Generate the actual control power at the current moment. ,satisfy:

[0144]

[0145] in, Indicates the first The actual control power of the electric vehicle cluster within each control cycle Indicates the first The target scheduling strategy selected within each control cycle. Indicates the first The current discrete state within each control cycle;

[0146] E2: Based on the target scheduling strategy The corresponding control intent generates a value corresponding to the current discrete state. Matching charging and discharging power outputs allow different target scheduling strategies to correspond to different actual control power generation methods;

[0147] E3: Actual control power generated Perform constraint verification to ensure that the actual control power meets the aggregate charge and discharge power boundary constraints:

[0148]

[0149] in, Indicates the first The aggregate maximum discharge power of the electric vehicle cluster within each control cycle Indicates the first The aggregated maximum charging power of the electric vehicle cluster within each control cycle;

[0150] E4: When the actual control power generated by the target scheduling strategy exceeds the boundary constraint of the aggregated charging and discharging power, the actual control power is boundary corrected to limit the corrected actual control power to the range. Inside;

[0151] E5: The actual control power that satisfies the boundary constraints of aggregated charge and discharge power. This serves as the actual execution power of the electric vehicle cluster within the current control cycle and is used for subsequent aggregator injection error calculation, aggregator charge state update, and reward function construction.

[0152] S6: Calculate the aggregator injection error based on the target adjustment power and the actual control power;

[0153] Obtain the target regulation power within the current control cycle and actual control power ;

[0154] The formula for calculating the aggregator injection error is:

[0155]

[0156] in, Indicates the first Aggregator injection error within each control cycle Indicates the first Target adjustment power within each control cycle Indicates the first The actual control power within each control cycle.

[0157] Based on the aggregator injection error Characterizing the tracking deviation of the electric vehicle aggregator on the target adjusted power, where, when When the absolute value of increases, it indicates that the actual control power deviates more from the target adjustment power; when When the absolute value of decreases, it indicates that the actual control power tracks the target adjustment power more effectively.

[0158] Injecting errors into aggregators This serves as the input for subsequent grid frequency deviation updates, frequency regulation tracking error cost calculations, and reward function construction.

[0159] S7: Update the grid frequency deviation at the next moment based on the aggregator injection error, and update the aggregated state of charge at the next moment based on the actual control power;

[0160] Step S7 includes:

[0161] F1: Get the aggregator injection error within the current control cycle. Information related to power grid disturbances Current power grid frequency deviation Current aggregate state of charge and actual control power ;

[0162] F2: Based on aggregator injection error Information related to power grid disturbances Calculate the total power imbalance of the system Satisfy:

[0163]

[0164] in, Indicates the first Total power imbalance of the system within each control cycle Indicates the first Random net load disturbance of the power grid or equivalent external disturbance power within a control cycle Indicates the first Aggregator injection error within each control cycle;

[0165] F3: Based on the current power grid frequency deviation And the total power imbalance of the system Update the grid frequency deviation for the next time step. ,satisfy:

[0166]

[0167] in, Indicates the first Power grid frequency deviation within each control cycle Indicates the first Power grid frequency deviation within each control cycle Indicates the control cycle. Represents the system's equivalent inertia constant. This represents the system's equivalent damping coefficient. Indicates the first Total power imbalance of the system within each control cycle;

[0168] F4: Based on the actual control power within the current control cycle Update the aggregated state of charge at the next time step. ,satisfy:

[0169]

[0170] in, Indicates the first Aggregate charge state within each control cycle Indicates the first Aggregate charge state within each control cycle This represents the charging and discharging efficiency function corresponding to the actual control power direction, when... When charging efficiency is measured, Discharge efficiency is taken at that time. Indicates the first The actual control power of the electric vehicle cluster within each control cycle Indicates the control cycle. This indicates the total capacity of the electric vehicle cluster.

[0171] F5: Update the next time-instance grid frequency deviation and the aggregated state of charge at the next moment It serves as the input for subsequent state discretization, reward function construction, and state-action value function updates.

[0172] S8: Construct an instantaneous reward function that simultaneously considers the cost of frequency deviation, the cost of frequency modulation tracking error, and the cost of battery degradation, and use the Q-learning algorithm to update the state-action value function online;

[0173] Step S8 includes:

[0174] G1: Based on the updated grid frequency deviation at the next time step Aggregator injection error and actual control power Construct the instant reward function for the current control period. ,satisfy:

[0175]

[0176] in, Indicates the first Instant reward value within each control period This represents the frequency deviation cost term. This represents the cost term for frequency modulation tracking error. This represents the cost of battery degradation, and the negative sign indicates that the immediate reward is maximized by minimizing each cost term.

[0177] G2: Based on the power grid frequency deviation at the next moment and the current discrete state Construct a frequency deviation cost term that satisfies:

[0178]

[0179] in, Represents the current discrete state The relevant frequency deviation weights are used to adjust the influence of the frequency deviation cost term on the immediate reward function under different operating conditions. Indicates the first Power grid frequency deviation within one control cycle;

[0180] G3: Based on aggregator injection error Construct a frequency modulation tracking error cost term that satisfies:

[0181]

[0182] Among them, Indicates the frequency modulation tracking error weight. Indicates the first Aggregator injection error within each control cycle;

[0183] G4: Based on actual control power Construct a battery degradation cost term that satisfies:

[0184]

[0185] in, This represents the battery degradation cost function. Indicates the first The actual control power of the electric vehicle cluster within each control cycle Indicates the first Aggregate charge state within each control cycle Indicates parameters related to battery temperature or equivalent degradation.

[0186] G5: Based on the current discrete state The selected target scheduling strategy Instant reward value and the discrete state at the next moment The Q-learning algorithm is used to study the state-action value function. Perform online updates to meet the following requirements:

[0187]

[0188] in, Indicates the current discrete state Select target scheduling strategy The corresponding state-action value function value. Indicates the learning rate. Indicates the discount factor. Represents a set of discrete candidate scheduling policies. Indicates the discrete state at the next time step. The maximum value among the state-action value functions corresponding to each candidate scheduling strategy;

[0189] G6: The updated state-action value function is used to select discrete candidate scheduling strategies in subsequent control cycles to achieve online coordinated optimization of grid frequency stability, frequency regulation tracking performance and battery degradation cost.

[0190] S9: Repeat steps S1 to S8 in subsequent control cycles to achieve auxiliary frequency regulation of electric vehicle aggregator while meeting the target state of charge constraint before vehicle disconnection.

[0191] Step S9 includes:

[0192] H1: After the current control cycle ends, the next control cycle begins. The power grid operation status information and electric vehicle cluster operation status information are reacquired, and the input data for the next control cycle is generated.

[0193] H2: In the next control cycle, repeat steps S1 to S8 to reconstruct the target regulation power, system state, target scheduling strategy, actual control power, aggregator injection error, grid frequency deviation at the next moment, aggregated state of charge at the next moment, instantaneous reward function, and state-action value function to form a closed-loop online control process.

[0194] H3: During the repeated execution of steps S1 to S8, continuously monitor the current state of charge, remaining access time, expected disconnection time and target state of charge of each connected electric vehicle, and determine whether each connected electric vehicle meets the target state of charge constraint before disconnection.

[0195] H4: When the remaining access time of any connected electric vehicle is only sufficient to reach its target state of charge, restrict the connected electric vehicle from participating in discharge regulation, and / or adjust the aggregated charge and discharge power boundary of the electric vehicle cluster to ensure that each connected electric vehicle reaches its target state of charge before leaving the grid.

[0196] H5: Under the premise of meeting the target state of charge constraint before the vehicle leaves the grid, the auxiliary frequency regulation of the electric vehicle aggregator is realized by continuously repeating steps S1 to S8 in the subsequent control cycle.

[0197] Example 2:

[0198] To verify the effectiveness, real-time performance, and adaptability of the method of the present invention, this embodiment constructs a high-fidelity simulation environment that includes a dynamic model of power grid frequency, an electric vehicle cluster aggregation model, and a random disturbance signal model. In this environment, the method of the present invention is compared and analyzed with various baseline control strategies.

[0199] In this embodiment, the simulation experiment was completed in the MATLAB R2024b environment, the control period was set to 2s, the grid frequency dynamic adopted the single-area load frequency control model, and the electric vehicle aggregation model comprehensively considered factors such as battery capacity, charging and discharging power, SOC constraints and target state of charge constraints before vehicle disconnection.

[0200] Table 1 is mainly used to compare the overall performance differences of different scheduling strategies at the annual operation level.

[0201]

[0202] As can be seen from the results in Table 1, while the full tracking strategy performs well in terms of tracking error, it has the highest battery degradation cost; the priority battery protection strategy has a lower degradation cost, but its frequency regulation performance and tracking capability are significantly reduced; the fixed compromise strategy performs moderately well in all indicators, but lacks dynamic adaptability and is difficult to achieve the optimal overall effect. In contrast, the method of this invention can achieve a better balance between frequency regulation performance, tracking error, and battery degradation cost, indicating that this invention does not simply pursue the improvement of a single indicator, but achieves a better overall scheduling effect under multiple objective constraints.

[0203] Figure 2 This is primarily used to demonstrate the dynamic selection process of different candidate scheduling strategies by the method of the present invention within a representative operating cycle. From Figure 2It can be seen that when there is a large frequency deviation in the power grid, the method of this invention tends to choose a more aggressive frequency tracking strategy to prioritize improving frequency support capabilities; while when the power grid frequency is in a relatively stable range, it tends to choose a more conservative strategy to reduce unnecessary frequent charging and discharging and reduce battery degradation. When the aggregate state of charge is low, even if there is a certain degree of frequency disturbance, the controller will appropriately suppress over-discharge behavior to prioritize ensuring the fleet's energy state and the user's off-grid needs. This indicates that this invention does not use fixed control logic, but can make online adaptive decisions by combining frequency state and SOC state.

[0204] Figure 3 This is mainly used to illustrate the comprehensive advantages of the method of this invention from the perspectives of both statistical distribution and long-term economic consequences. Figure 3 As shown in (a), the frequency deviation distribution corresponding to the method of the present invention is more concentrated and less dispersed, indicating that it has a better ability to suppress system frequency fluctuations, fewer extreme values ​​of frequency deviation occur, and the system operates more stably. In contrast, the priority protection battery strategy has a more dispersed frequency deviation distribution due to its lower frequency modulation participation, indicating poorer system stability. Figure 3 (b) shows that the method of the present invention can significantly reduce the battery degradation cost compared with the full tracking strategy while maintaining good frequency control effect. This indicates that the method of the present invention does not simply rely on more frequent charging and discharging to improve frequency regulation performance, but achieves a better balance between frequency regulation benefits and battery life protection.

[0205] Figure 4 Figure (a) illustrates the comparison between the target regulation power and the actual control power of each control strategy during a typical high-fluctuation short period. It can be seen that when the system frequency fluctuation amplitude is large or the frequency regulation demand is strong, the method of this invention can quickly adjust the scheduling strategy according to the current grid state, enabling the actual control power to promptly approach the target regulation power, thereby providing stronger frequency support capabilities at critical moments. In contrast, fixed strategies, due to their lack of state adaptation capabilities, suffer from response lag or insufficient support under large disturbances. Figure 4 Figure (b) illustrates the differences in scheduling behavior of different control strategies in the high-frequency, low-amplitude fluctuation range within the same time period. It can be seen that the method of this invention actively reduces the tracking intensity of the frequency modulation signal in the low-value, high-frequency oscillation range, avoiding frequent charging and discharging operations, thereby effectively reducing ineffective battery loss. In contrast, the fixed compromise strategy, due to its constant proportional tracking, continues charging and discharging even under small disturbances, leading to unnecessary battery degradation. This result demonstrates that the method of this invention can achieve an adaptive trade-off between frequency support and battery protection based on the operating state.

[0206] In conclusion, Figure 4The adaptive scheduling characteristics of the method of the present invention were verified from the perspective of micro-scheduling behavior, that is, the adjustment capability is enhanced under large disturbances and the ineffective response is reduced under small disturbances, thereby achieving coordinated optimization between frequency modulation performance and battery life.

Claims

1. A method for auxiliary frequency modulation of electric vehicle aggregators based on action space abstraction, characterized in that, Includes the following steps: S1: Obtain the grid operation status information and electric vehicle cluster operation status information within the current control cycle. The grid operation status information includes grid frequency deviation and automatic generation control commands. The electric vehicle cluster operation status information includes baseline charging power, aggregated state of charge, and aggregated charge and discharge power boundaries. S2: Construct the target regulation power based on the baseline charging power and automatic generation control commands; S3: Construct the continuous state of the system based on the current grid frequency deviation and aggregated state of charge, and discretize the continuous state of the system to obtain the current discrete state; S4: Using the action space abstraction method, the continuous charging and discharging control actions are mapped to a finite set of discrete candidate scheduling strategies, and the target scheduling strategy is selected according to the current discrete state; S5: Generate the actual control power at the current moment using the target scheduling strategy, and ensure that the actual control power satisfies the aggregated charging and discharging power boundary constraints; S6: Calculate the aggregator injection error based on the target adjustment power and the actual control power; S7: Update the grid frequency deviation at the next moment based on the aggregator injection error, and update the aggregated state of charge at the next moment based on the actual control power; S8: Construct an instantaneous reward function that simultaneously considers the cost of frequency deviation, the cost of frequency modulation tracking error, and the cost of battery degradation, and use the Q-learning algorithm to update the state-action value function online; S9: Repeat steps S1 to S8 in subsequent control cycles to achieve auxiliary frequency regulation of electric vehicle aggregator under the premise of meeting the target state of charge constraint before the vehicle leaves the grid; Step S4 includes: D1: Based on the operational requirements of electric vehicle aggregators in auxiliary frequency regulation scenarios, the continuous charging and discharging control actions are abstracted into a finite number of discrete candidate scheduling strategies, and a set of discrete candidate scheduling strategies is constructed. ,satisfy: ； in, Represents a set of discrete candidate scheduling policies. Indicates the first One candidate scheduling strategy, This represents the total number of candidate scheduling strategies; D2: Based on the current discrete state State-action value function corresponding to each candidate scheduling strategy In the discrete candidate scheduling strategy set Select target scheduling strategy ,satisfy: ； in, Indicates the first The target scheduling strategy selected within each control cycle. Represents the current discrete state Next candidate scheduling strategy The corresponding state-action value function value. In the discrete candidate scheduling policy set Select the candidate scheduling strategy that maximizes the state-action value function. D3: During the strategy selection process, an ε-greedy mechanism is used to switch between randomly selecting candidate scheduling strategies and selecting the target scheduling strategy according to step D2. When the random number... At that time, from the discrete candidate scheduling strategy set Randomly select a candidate scheduling strategy; when At that time, select the target scheduling strategy according to step D2, where, It is a random number. Preset exploration rate; Step S8 includes: G1: Based on the updated grid frequency deviation at the next time step Aggregator injection error and actual control power Construct the instant reward function for the current control period. ,satisfy: ； in, Indicates the first Instant reward value within each control period This represents the frequency deviation cost term. This represents the cost term for frequency modulation tracking error. This represents the cost of battery degradation, and the negative sign indicates that the immediate reward is maximized by minimizing each cost term. G2: Based on the power grid frequency deviation at the next moment and the current discrete state Construct a frequency deviation cost term that satisfies: ； in, Represents the current discrete state The relevant frequency deviation weights are used to adjust the influence of the frequency deviation cost term on the immediate reward function under different operating conditions. Indicates the first Power grid frequency deviation within one control cycle; G3: Based on aggregator injection error Construct a frequency modulation tracking error cost term that satisfies: ； in, Indicates the frequency modulation tracking error weight. Indicates the first Aggregator injection error within each control cycle; G4: Based on actual control power Construct a battery degradation cost term that satisfies: ； in, This represents the battery degradation cost function. Indicates the first The actual control power of the electric vehicle cluster within each control cycle Indicates the first Aggregate charge state within each control cycle Indicates parameters related to battery temperature or equivalent degradation. G5: Based on the current discrete state The selected target scheduling strategy Instant reward value and the discrete state at the next moment The Q-learning algorithm is used to study the state-action value function. Perform online updates to meet the following requirements: ； in, Indicates the current discrete state Select target scheduling strategy The corresponding state-action value function value. Indicates the learning rate. Indicates the discount factor. Represents a set of discrete candidate scheduling policies. Indicates the discrete state at the next time step. The maximum value among the state-action value functions corresponding to each candidate scheduling strategy.

2. The electric vehicle aggregator-assisted frequency modulation method based on action space abstraction according to claim 1, characterized in that, Step S1 includes: A1: Obtain the power grid operating status information for the current control cycle, including power grid frequency deviation information. Automatic power generation control command information and information related to power grid disturbances ; A2: Obtain the operating status information of the electric vehicle cluster within the current control cycle. The operating status information of the electric vehicle cluster includes the baseline charging power. Polymerized state of charge And aggregated charge and discharge power boundary information, including aggregated state of charge. satisfy: ； in, This represents the number of electric vehicles connected to the aggregator during the current control period. Indicates the first electric vehicles in the first State of charge within each control cycle; A3: Obtain individual constraint information for each connected electric vehicle, and calculate the aggregated maximum charging power based on the individual constraint information. and maximum discharge power This makes the actual control power satisfy: ； in, For the first The actual control power of the electric vehicle cluster within each control cycle; individual constraint information includes the current state of charge of a single vehicle, the maximum charging power of a single vehicle, the maximum discharging power of a single vehicle, the expected off-grid time, and the target state of charge; A4: The acquired power grid operation status information and electric vehicle cluster operation status information are synchronized in time, aggregated and preprocessed to form the input data required for subsequent control.

3. The electric vehicle aggregator-assisted frequency modulation method based on action space abstraction according to claim 2, characterized in that, Step S2 includes: B1: Obtain the baseline charging power within the current control cycle. and automatic power generation control commands ; B2: Based on baseline charging power and automatic power generation control commands Construct target regulation power ,satisfy: ； in, Indicates the first The target adjustment power within each control cycle.

4. The electric vehicle aggregator-assisted frequency modulation method based on action space abstraction according to claim 3, characterized in that, Step S3 includes: C1: Extract the grid frequency deviation within the current control cycle and polymer charge state and with grid frequency deviation and polymer charge state Together they constitute the continuous state of the system ,satisfy: ； in, Indicates the first The continuous state of the system within each control cycle; C2: The power grid frequency deviation is processed according to a preset discretization rule. Divide the intervals to obtain the corresponding frequency discrete interval numbers. And according to the preset discretization rules, the aggregated charge state is... Divide the data into intervals to obtain the corresponding discrete interval numbers for the state of charge. ,satisfy: ； in, This represents a frequency deviation interval mapping function. Represents the mapping function for the aggregate charged state interval; C3: Numbered according to the frequency discrete interval and discrete interval numbering of the state of charge For the continuous state of the system Perform a combined mapping to obtain the current discrete state. ,satisfy: ； in, Indicates the first The current discrete state within each control cycle. This represents a discrete state combination mapping function.

5. The electric vehicle aggregator-assisted frequency modulation method based on action space abstraction according to claim 4, characterized in that, Step S5 includes: E1: Based on the target scheduling strategy selected within the current control cycle. and the current discrete state Generate the actual control power at the current moment. ,satisfy: ； in, Indicates the first The actual control power of the electric vehicle cluster within each control cycle Indicates the first The target scheduling strategy selected within each control cycle. Indicates the first The current discrete state within each control cycle; E2: Based on the target scheduling strategy The corresponding control intent generates a value corresponding to the current discrete state. Matching charging and discharging power outputs allow different target scheduling strategies to correspond to different actual control power generation methods; E3: Actual control power generated Perform constraint verification to ensure that the actual control power meets the aggregate charge and discharge power boundary constraints: ； in, Indicates the first The aggregate maximum discharge power of the electric vehicle cluster within each control cycle Indicates the first The aggregated maximum charging power of the electric vehicle cluster within each control cycle; E4: When the actual control power generated by the target scheduling strategy exceeds the boundary constraint of the aggregated charging and discharging power, the actual control power is boundary corrected to limit the corrected actual control power to the range. Inside; E5: The actual control power that satisfies the boundary constraints of aggregated charge and discharge power. This represents the actual power output of the electric vehicle cluster during the current control cycle.

6. The electric vehicle aggregator-assisted frequency modulation method based on action space abstraction according to claim 5, characterized in that, The formula for calculating the aggregator injection error in step S6 is as follows: ； in, Indicates the first Aggregator injection error within each control cycle Indicates the first Target adjustment power within each control cycle Indicates the first The actual control power within each control cycle.

7. The electric vehicle aggregator-assisted frequency modulation method based on action space abstraction according to claim 6, characterized in that, Step S7 includes: F1: Get the aggregator injection error within the current control cycle. Information related to power grid disturbances Current power grid frequency deviation Current aggregate state of charge and actual control power ; F2: Based on aggregator injection error Information related to power grid disturbances Calculate the total power imbalance of the system ,satisfy: ； in, Indicates the first Total power imbalance of the system within each control cycle Indicates the first Random net load disturbance of the power grid or equivalent external disturbance power within a control cycle Indicates the first Aggregator injection error within each control cycle; F3: Based on the current power grid frequency deviation And the total power imbalance of the system Update the grid frequency deviation for the next time step. ,satisfy: ； in, Indicates the first Power grid frequency deviation within each control cycle Indicates the first Power grid frequency deviation within each control cycle Indicates the control cycle. Represents the system's equivalent inertia constant. This represents the system's equivalent damping coefficient. Indicates the first Total power imbalance of the system within each control cycle; F4: Based on the actual control power within the current control cycle Update the aggregated state of charge at the next time step. ,satisfy: ； in, Indicates the first Aggregate charge state within each control cycle Indicates the first Aggregate charge state within each control cycle This represents the charging and discharging efficiency function corresponding to the actual control power direction, when... When charging efficiency is measured, Discharge efficiency is taken at that time. Indicates the first The actual control power of the electric vehicle cluster within each control cycle Indicates the control cycle. This indicates the total capacity of the electric vehicle cluster.

8. The electric vehicle aggregator-assisted frequency modulation method based on action space abstraction according to claim 7, characterized in that, Step S9 includes: H1: After the current control cycle ends, the next control cycle begins. The power grid operation status information and electric vehicle cluster operation status information are reacquired, and the input data for the next control cycle is generated. H2: In the next control cycle, repeat steps S1 to S8 to reconstruct the target regulation power, system state, target scheduling strategy, actual control power, aggregator injection error, grid frequency deviation at the next moment, aggregated state of charge at the next moment, instantaneous reward function, and state-action value function to form a closed-loop online control process. H3: During the repeated execution of steps S1 to S8, continuously monitor the current state of charge, remaining access time, expected disconnection time and target state of charge of each connected electric vehicle, and determine whether each connected electric vehicle meets the target state of charge constraint before disconnection. H4: When the remaining access time of any connected electric vehicle is only sufficient to reach its target state of charge, restrict the connected electric vehicle from participating in discharge regulation, and / or adjust the aggregated charge and discharge power boundary of the electric vehicle cluster to ensure that each connected electric vehicle reaches its target state of charge before leaving the grid. H5: Under the premise of meeting the target state of charge constraint before the vehicle leaves the grid, the auxiliary frequency regulation of the electric vehicle aggregator is realized by continuously repeating steps S1 to S8 in the subsequent control cycle.

Images