Intelligent scheduling method fusing EV charging and discharging behavior clustering and deep reinforcement learning
By integrating EV charging and discharging behavior clustering with deep reinforcement learning, an intelligent scheduling method is developed. This method utilizes Mean Shift and K-means++ algorithms to identify user differences and combines DQN algorithm optimization strategies to solve the scheduling challenges when large-scale electric vehicles are connected to the power grid. It achieves load shaving and cost optimization, and improves the accuracy and fairness of scheduling.
Patent Information
- Application Number
- CN202511486670.3
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2025-10-17
- Publication Date
- 2026-01-13
AI Technical Summary
Existing electric vehicle charging and discharging scheduling methods struggle to effectively handle the uncertainty and randomness of individual users when connected to the power grid on a large scale, resulting in high information processing pressure. Furthermore, existing clustering algorithms perform poorly in cases of ambiguous boundaries and numerous transitional samples, lacking strong time-series applicability for decision-making. Additionally, reinforcement learning suffers from the dimensionality curse problem of decision variables under state space expansion.
An intelligent scheduling method integrating EV charging and discharging behavior clustering and deep reinforcement learning is adopted. The Mean Shift and K-means++ algorithms are used to identify differences in user charging and discharging behavior, construct an MDP model, and combine the DQN algorithm to optimize the strategy, compress the decision dimension, and achieve precise scheduling at the individual level.
It enables rapid peak shaving, improves the accuracy and economy of scheduling, makes scheduling results more equitable, alleviates the problem of uneven costs among users, and reduces system load peaks and user charging costs.
Smart Images
Figure CN121328831A_ABST
Abstract
Description
TECHNICAL FIELD
[0001] The application belongs to the technical field of electric vehicle charging and discharging control, and particularly relates to an intelligent scheduling method fusing EV charging and discharging behavior clustering and deep reinforcement learning. BACKGROUND
[0002] Under the background of China's "double carbon" target, electric vehicles (EV) as a green travel tool are developing rapidly and their number is rising sharply. It is estimated that the number of electric vehicles will grow to 390 million by 2040. From the perspective of energy storage, electric vehicles can be used as dispersed mobile energy storage, and have huge adjustment potential. If a single vehicle is 50kWh, the equivalent energy storage capacity can reach 195 billion kWh, which is an important adjustment resource. With the large-scale access of electric vehicles to the power grid, unorganized charging load is easy to overlap with the peak period of traditional load, forming a new load peak and increasing the pressure on the power system. To alleviate the load impact, the vehicle-to-grid (V2G) technology can be used to achieve orderly charging and discharging while meeting user travel needs, and through price incentives and intelligent control, users can be guided to participate, thereby reducing peak load and improving user benefits [7] .
[0003] The electric vehicle has strong dispersion and randomness. When a large number of individual users directly participate in the interaction and adjustment of the vehicle network, it will cause great information processing pressure to the scheduling system. The clustering method can effectively reduce the decision dimension and reduce the system burden. The current partial clustering algorithm describes the uncertainty of EV characteristics to a certain extent, and has efficiency advantage for the group with clear structure, but the effect is poor when dealing with more boundary fuzzy and transition samples, and lacks applicability in time sequence strong decision-making scenarios. At present, a large number of studies have been carried out around the price signal guiding EV to participate in V2G. The literature based on PSO algorithm of electric vehicle ordered charging and discharging two-stage optimization strategy[J]. Proceedings of the Chinese Society of Electrical Engineering; Electric vehicle ordered charging strategy based on multi-objective[J]. Modern electronic technology; The above-mentioned literature constructs a multi-objective optimization model based on time-of-use price, taking the minimum peak-valley difference and cost as the target, realizes the charging and discharging regulation and control, and most of them focus on the optimization of charging and discharging strategy under the price mechanism design, without fully considering the difference of response willingness of EV users in the actual participation of V2G process. While the existing research on electric vehicle charging behavior mainly focuses on building mathematical model and solving optimal charging and discharging strategy by combining optimization algorithm. The literature of electric vehicle charging station multi-time scale stochastic optimization scheduling considering demand response[J]. Power System Protection and Control, proposes a multi-time scale stochastic optimization model, introduces the conditional value at risk in day-ahead, and realizes rolling correction in day-ahead by using predictive control. The literature of dynamic user willingness driven electric vehicle multi-objective collaborative scheduling strategy[J / OL]. Electric Power Systems and Their Automatic Control, optimizes the time-of-use and zoning price compensation by using improved particle swarm algorithm, realizes the maximization of income and minimization of new energy consumption deviation. The literature of WANG S Y, BI S Z, ZHANG YA. Reinforcement learning for real-time pricing and scheduling control in EV charging stations[J]. IEEE Transactions on Industrial Informatics, 2021, 17(2): 849-859. proposes an online reinforcement learning method for optimizing the scheduling and pricing strategy of electric vehicle charging station, but the model-driven electric vehicle scheduling method in the above-mentioned literature depends on accurate modeling, which is easy to be affected by uncertainty in actual operation and reduce accuracy. Reinforcement learning has self-adaptability, but it faces the problem of "dimension disaster" of decision variables caused by state space expansion under large-scale EV access, and lacks clustering mechanism and uncertainty modeling to enhance scene adaptability. Based on this, an intelligent scheduling method combining EV charging and discharging behavior clustering and deep reinforcement learning is designed and proposed to realize the extremely urgent peak shaving task. SUMMARY
[0004] The application aims to provide an intelligent scheduling method combining EV charging and discharging behavior clustering and deep reinforcement learning, which identifies the difference of user charging and discharging behavior through clustering and compresses the dimension of scheduling decision.
[0005] In order to achieve the above-mentioned purpose, the application adopts the following technical solutions: An intelligent scheduling method combining EV charging and discharging behavior clustering and deep reinforcement learning, comprising: A joint clustering method is constructed to determine the number of clusters by Mean Shift and refine the clustering results by K-means++, specifically including: data preprocessing; determining the number of groups; obtaining the clustering results; An electric vehicle scheduling model is constructed: It is assumed that the charging behavior of each EV user is independent, and the minimum power requirement of the i-th EV is reached when the EV leaves, then the shortest charging time required by the i-th EV As follows: In the formula: is the rated capacity of the battery of the i-th EV; is the charging power of the i-th EV at time t; is the in-network SOC of the i-th EV; is the expected SOC of the i-th EV; An MDP model is constructed: At the cluster level, a scheduling model is constructed to minimize the load peak and EV cluster charging cost, and the load reduction is allocated to the EV cluster; at the individual level, based on the adjusted power obtained at the cluster level, the scheduling task is allocated to the individual EV user, an MDP model is constructed, and a DQN algorithm is introduced for policy learning optimization, through interaction with the environment, the agent continuously optimizes the policy under the guidance of the reward function, and the goal focuses on minimizing the individual user EV charging cost.
[0006] Further, in data preprocessing, the charging parameters of all EVs are collected and standardized. In the formula: is the original feature value; is the standardized value; is the sample point value.
[0007] Further, the MS algorithm is used to determine the number of EV groups, and the specific steps of the MS algorithm are as follows: Step1: In the normalized feature space, the kernel density estimation method is introduced to model the probability density function of sample distribution non-parametrically, and the density gradient of each point in the space is described. The estimation function is as follows: In the formula: h is the bandwidth coefficient, which balances the granularity of clustering and the smoothness of the result; d is the feature dimension; n is the number of sample points; K(•) is the kernel function; the Gaussian kernel function is used to avoid hard boundary effect and has better robustness to noise points, so as to obtain smoother density estimation; Step2: Taking the sample point as the initial position, the iterative update is performed in the direction of the density gradient defined by the kernel density function until the drift vector converges to the density maximum point; the mean drift vector is given by the following formula: In the formula: j is the iteration number; Step3: If the distance between two modal points is less than the preset merging threshold , it is considered as the same class cluster, and the modal merging operation is performed, , the loop is ended; otherwise, the iteration operation is continued, and Step2 is executed; Obtain the clustering result: all the modal points after merging form the final clustering center, denoted as ; according to the determined number of clusters k, the K-means++ algorithm is used to obtain the EV clustering result with different response characteristics.
[0008] Further, in order to ensure that the electric vehicle still has enough power after completing the discharge task to meet the basic travel demand, the SOC at the departure time should be greater than the minimum SOC required by the user, as follows: In the formula: is the state of charge of the ith EV at time t; is the minimum required state of charge of the ith EV at the departure time; , are the charging period and the discharging period, respectively; , are the start charging time and the start discharging time, respectively; , are the charging efficiency and the discharging efficiency, which are 0.95.
[0009] Further, the EV access charging pile stays for a long time as follows: Charging and discharging state constraints: The grid load value after accessing EV is updated as follows: In the formula, , respectively EV access and off-grid time; , respectively the charging and discharging state of EV; 1 is charging, -1 is discharging, and 0 is resting; is the discharging power of the ith EV at time t; N and M are the number of charging and discharging EVs at time t.
[0010] Further, in the reinforcement learning framework, the agent continuously learns the optimal strategy through interaction with the environment to maximize its reward function; the interaction process can be generally formulated as a Markov decision process, defined as a five-tuple , wherein s represents the state space, a represents the action space, and p is the state transition probability. In the DQN solving process, p is not explicitly modeled, but is learned through the state transition data sampled by interacting with the environment; is the instantaneous reward function at time t, is the discount factor of future rewards.
[0011] The beneficial effects of the present application are: the intelligent scheduling method provided by the present application can quickly realize the peak shaving task in specific use, the difference of user charging and discharging behavior is identified through clustering, the scheduling decision dimension is compressed, the multi-gear charging and discharging power is discretized into an action space at the individual level, and the power is adjusted in combination with the SOC state to improve the accuracy of response, an MDP model is further constructed with the minimum user charging cost as the target, the economic improvement is realized, the cost unevenness between users is relieved, the scheduling result is more fair and consistent, and the DQN algorithm with the introduction of the target network is used to realize the optimization of the EV charging and discharging strategy, and the simulation results verify the effectiveness of the proposed method in reducing the user charging cost and reducing the system peak load. BRIEF DESCRIPTION OF DRAWINGS
[0012] Figure 1 is the clustering number index evaluation diagram in the embodiment of the intelligent scheduling method of the present application; Figure 2 is the clustering result diagram in the embodiment of the intelligent scheduling method of the present application; Figure 3 is the load comparison diagram under different EV scheduling strategies in the embodiment of the intelligent scheduling method of the present application; Figure 4 is the EV cluster output situation diagram in the embodiment of the intelligent scheduling method of the present application; Figure 5 is the SOC curve diagram of the EV in the embodiment of the intelligent scheduling method of the present application; Figure 6 is the reward curve graph in the smart scheduling method embodiment of the present application; Figure 7 is the load peak shaving and user cost comparison graph under different scheduling strategies in the smart scheduling method embodiment of the present application; Figure 8 is the EV individual secondary cost graph under different scheduling strategies in the smart scheduling method embodiment of the present application; Figure 9 is the overall architecture and execution flow chart of the DQN algorithm in the smart scheduling method embodiment of the present application; DETAILED DESCRIPTION
[0013] Specific embodiment 1: The technical solutions of the present application will be described below in conjunction with embodiments. Obviously, the described embodiments are part of the embodiments of the present application, rather than all the embodiments. Based on the embodiments in the present application, all other embodiments obtained by those of ordinary skill in the art without creative labor fall within the scope of the present application. It should be noted that: in the present application, all implementation manners and preferred implementation methods mentioned in the present embodiment can be combined to form new technical solutions, unless otherwise specified. In the present application, all technical features and preferred features mentioned in the present embodiment can be combined to form new technical solutions, unless otherwise specified. The disclosed "range" in the present application is in the form of lower limit and upper limit, which can be one or more lower limits and one or more upper limits, respectively.
[0014] The smart scheduling method of the present application fuses EV charging and discharging behavior clustering and deep reinforcement learning, comprising: A joint clustering method is constructed to determine the number of clusters by Mean Shift and refine the clustering results by K-means++, specifically including: data preprocessing: collecting the charging parameters of all EVs and performing standardization processing thereon; In the formula: is the original feature value; is the standardized value; is the sample point value; Determine the number of groups: the number of EV groups is determined by using the MS algorithm, wherein the specific steps of the MS algorithm are as follows: Step 1: In the normalized feature space, introduce the kernel density estimation method to perform non-parametric modeling on the probability density function of sample distribution, thereby depicting the density gradient of each point in the space. The estimation function form is as follows: where h is the bandwidth coefficient, balancing the granularity of clustering and the smoothness of the result; d is the feature dimension; n is the number of sample points; K(•) is the kernel function; the Gaussian kernel function is used to avoid the hard boundary effect and has better robustness to noise points, so as to obtain a smoother density estimation; Step 2: Taking the sample point as the initial position, performing iterative update in the direction of the density gradient defined by the kernel density function until the drift vector converges to the density maximum point; the mean drift vector is given by the following formula: where j is the iteration number; Step 3: If the distance between two modal points is less than the preset merging threshold , it is considered as the same class cluster, and the modal merging operation is performed, , and the loop is ended; otherwise, the iteration operation is continued, and Step 2 is performed; Obtain the clustering result: all the modal points after merging form the final clustering center, denoted as ; according to the determined number of clusters k, the K-means++ algorithm is used to obtain the EV clustering result with different response characteristics.
[0015] Construct the electric vehicle scheduling model: Assuming that the charging behavior of each EV user is independent, and the minimum power requirement is reached when the EV leaves, the shortest charging time required by the ith EV is as follows: where is the rated capacity of the battery of the ith EV; is the charging power of the ith EV at time t; is the in-network SOC of the ith EV; is the expected SOC of the ith EV; The residence time of the EV accessing the charging pile is as follows: Charging and discharging state constraints: The load value of the power grid after accessing the EV is updated as follows: where , are the in-network and off-network times of the EV, respectively; , are the charging and discharging states of the EV, respectively; 1 is charging, -1 is discharging, and 0 is resting; Let be the discharge power of the i-th EV at time t; N and M are the number of EVs charging and discharging at time t, respectively.
[0016] To ensure that electric vehicles retain sufficient charge to meet basic travel needs after completing their discharge cycle, the State of Charge (SOC) at the time of departure must be greater than the minimum SOC required by the user, as shown in the following formula: In the formula: Let t be the state of charge of the i-th EV; This represents the minimum required state of charge when the i-th EV leaves. , These are the charging period and the discharging period, respectively. , These are the start of charging and the start of discharging, respectively. , The charging efficiency and discharging efficiency are set to 0.95.
[0017] Building an MDP model: At the cluster level, a scheduling model is constructed to allocate load reduction to the EV cluster with the goal of minimizing peak load and charging costs for the EV cluster. At the individual level, based on the adjustment power obtained at the cluster level, scheduling tasks are allocated to individual EV users. An MDP model is constructed, and the DQN algorithm is introduced for policy learning and optimization. Through interaction with the environment, the agent continuously optimizes the policy under the guidance of the reward function, with the goal of minimizing the charging costs for individual EV users.
[0018] To address the decision-making problem in electric vehicle scheduling, this embodiment employs the Deep Q-Network (DQN) algorithm. The overall architecture and execution flow of the DQN algorithm are shown in the appendix of the specification. Figure 9 As shown, at each time step t, the agent observes the current environmental state. And input it into the valuation network The network outputs the Q-values of all available actions. To balance policy superiority and exploratory nature, action selection follows an ε-greedy policy: a random action is selected with probability ε, and the action with the highest current Q-value is selected with probability 1-ε. The agent executes the selected action. Afterwards, interacting with the environment yields immediate rewards. and the next state and the state transition sample quadruple The data is stored in the experience replay pool. A Q-value approximator is constructed to guide the agent in learning optimal action sequences in dynamic environments. Combined with experience replay and the target network mechanism, the stability and convergence efficiency of training are improved. The agent can obtain its charged state at time t through the state space. Incentive electricity prices , the length of time that the electric vehicle stays , the total power of the cluster , the response intention The action set covers the charging and discharging state of the EV, and the electric vehicle can select different power stages at each decision node, the charging power kW, the discharging power kW, and a complete charging and discharging behavior sequence is recorded as a strategy v. Taking SOC as the core index, a state transition chain based on MDP is constructed.
[0019] To verify the effectiveness of the method proposed in the application, part of the sample data in a set of domestic EV charging transaction data set published by Scientific Data is selected for instance simulation analysis, including EV charging start time, off-grid time, transaction electricity, etc. A typical regional load in a city is used for simulation, and a time-of-use electricity price is used for simulation. The number of EVs is 2000, the rated battery capacity is 50kwh, the charging and discharging efficiency is 0.95, the charging power range is [11kw, 7kw, 4kw], the discharging power range is [11kw, 6kw, 3kw], the time period is 24 hours, and the scheduling is performed with a step of 15 minutes.
[0020] In this embodiment, 2000 EVs are clustered, and the MS and Kmeans++ algorithms are used for clustering. The EVs' on-grid and off-grid time, demand electricity are used as indexes. The optimal value of the cluster number K is determined according to the DBI, CH index, and SI coefficient, and the evaluation results of the three indexes are shown in Figure 5 . According to the cluster number K, the clustering results of the EV charging behavior parameter indexes are shown in the accompanying drawings Figure 2 .
[0021] Figure 1 The CH index, SI coefficient and DBI value are shown as the cluster center number K changes. The CH index reaches the maximum value when K=7, indicating that the inter-class difference of the cluster under this cluster number is the most significant, and the intra-class similarity is the highest; the SI value performs better in the range of K=6~7, indicating that the clustering attribution degree of the sample in this range is higher; the DBI value reaches a lower level around K=7, indicating that the cluster separation is stronger. According to the trend analysis of the three indexes, K=7 is the optimal choice for the current data in clustering division, and a better clustering effect can be achieved.
[0022] The clustering results of the EV charging behavior parameter indexes are shown in the accompanying drawings Figure 3It can be seen that in the unordered charging scenario, the EV charging load is mainly concentrated in the middle of the day and in the evening, forming a "peak on peak" in the 18-22 period. When EVs are not clustered to participate in V2G, some EVs choose to charge in the low electricity price period 0-6, charge in the flat electricity price period 7-9, and discharge in the high electricity price period 10-12, 16-20. The high peak load is reduced, the user charging cost is reduced, but the peak shifting effect is limited. The method of the embodiment considers charging cost and load pressure, and finely guides EVs to charge in the load valley and low electricity price period, and feedback power to the grid in the high load and high electricity price period. Compared with other strategies, the algorithm of the embodiment shifts more peak load to the load valley period, and the load reduction is 1077.73kW.
[0023] By the description attached Figure 4 It can be seen that in the early morning 0-6, in the low load valley period and low electricity price, all clusters are continuously in the charging state. Due to the charging needs of some clusters of EVs, a small amount of EVs are still charging at 8-10. At the load peak period 11-14, clusters 2, 4 and 7 discharge to reduce the load pressure of the grid under the premise of meeting the user charging demand. Clusters 1 and 3 remain in the charging state at 6-12, so at the evening peak period 16-22, clusters 1 and 3 have enough power to discharge and perform evening load peak shaving. Cluster 7 has small user vehicle demand and is more inclined to economy, so it discharges for a long time, reducing its own charging cost. Clusters 5 and 6 have large vehicle demand, and the users are inclined to charging comfort, and do not perform discharging actions, but most of the vehicles choose to charge in the low electricity price period, avoiding charging in the high electricity price and high load period, and appropriately reducing the charging cost. The method of the embodiment realizes the charging and discharging peak-shifting cooperation among different clusters, reduces the user charging cost, and at the same time relieves the load pressure of the grid.
[0024] By the description attached Figure 5 It can be seen that the vehicle demand and the network access period have a significant impact on the charging and discharging strategy of EVs. EV7 stays for a long time, and preferentially charges in the flat electricity price period. After entering the high load and high electricity price period at 11:00, it discharges to benefit, and then charges in the low price period. EV1, EV2 and EV3 arrive with low SOC, and continuously charge upon arrival. They have surplus to participate in discharging in the afternoon peak period. EV5, EV6 and EV10 arrive with SOC about 0.5 and in the peak period. They first moderately discharge to reduce the SOC to 0.2, participate in V2G while avoiding low SOC to cause battery performance loss, and then charge in the afternoon flat electricity price and load flat period to reach the target power set by the vehicle owner before leaving. EV9 stays for a short time, and does not discharge to meet the rigid demand. Part of the charging is delayed to 13:00 in the flat price period to avoid peak shaving and cost reduction. From the SOC change curves of the above EVs, it can be seen that the method of the embodiment can guide EVs to make adaptive decisions in V2G, while considering economy and battery health to perform system peak shaving.
[0025] FIG. 1 is a schematic diagram of a power grid according to an embodiment of the present application. Figure 6 As can be seen, under the joint action of the introduction of the ε-greedy strategy and the experience replay mechanism, the agent gradually converges within about 400 rounds of training, and the reward curve gradually tends to be stable from the early stage of large fluctuations. The results show that the DQN agent can effectively capture the optimal action strategy of the individual EV in the dynamic environment, and realize autonomous learning and behavior adjustment based on state perception.
[0026] FIG. 2 is a schematic diagram of a power grid according to an embodiment of the present application. Figure 8 As can be seen, compared with unordered charging and the traditional V2G strategy, the method of the embodiment exhibits better performance in the concentration and dispersion of the cost distribution, realizes economic improvement, and at the same time, alleviates the problem of uneven cost among users, and the scheduling result is more fair and consistent. If the cost difference between different users is too large, some users will bear too much scheduling task, which will affect the willingness of users to participate.
[0027] FIG. 3 is a schematic diagram of a power grid according to an embodiment of the present application. Figure 7 As can be seen, the strategy proposed in the embodiment is superior to unordered charging and the traditional V2G strategy in various indicators. In terms of load peak, the method of the embodiment reduces the load peak from 11068.74 kW under unordered charging to 9991.01 kW, which is further reduced by 5.4% compared with 10562.36 kW under the V2G strategy; in terms of peak-valley difference, the method of the embodiment is only 1369.28 kW, which is reduced by 60.4% and 47.3% respectively compared with unordered charging and the V2G strategy, and the system load curve is smoother, which effectively alleviates the operating pressure of the distribution network. In terms of user economy, the method of the embodiment controls the average charging cost to 12.75 yuan, which is significantly lower than 37.06 yuan under unordered charging and 18.53 yuan under the V2G strategy, with a reduction of 65.6% and 31.2% respectively, which further verifies the comprehensive advantages of the method of the embodiment in peak shaving and user-side economy.
[0028] The above description is only a preferred embodiment of the present application, and is not intended to limit the present application in any form. Although the present application has been disclosed as above with a preferred embodiment, it is not intended to limit the present application. Any skilled person in the art can make some changes or modifications to the above disclosed technical content without departing from the scope of the technical solution of the present application, and any simple modification, equivalent change and modification made to the above embodiment based on the technical essence of the present application shall still fall within the scope of the technical solution of the present application.
Claims
1. An intelligent scheduling method integrating EV charging and discharging behavior clustering and deep reinforcement learning, characterized in that, include: A joint clustering method is constructed, which uses Mean Shift to determine the number of clusters and K-means++ to refine the clustering results. Specifically, the method includes: data preprocessing; determining the number of groups; and obtaining the clustering results. Constructing an electric vehicle scheduling model: Assuming each EV user's charging behavior is independent, and considering that the EV reaches the minimum charge requirement upon leaving the vehicle, the shortest charging time required for the i-th EV is determined. As shown in the following formula: In the formula: Let be the rated battery capacity of the i-th EV; Let be the charging power of the i-th EV at time t; For the SOC of the i-th EV entering the network; For the expected SOC of the i-th EV; Building an MDP model: At the cluster level, a scheduling model is constructed to allocate load reduction to the EV cluster with the goal of minimizing peak load and charging costs for the EV cluster. At the individual level, based on the adjustment power obtained at the cluster level, scheduling tasks are allocated to individual EV users. An MDP model is constructed, and the DQN algorithm is introduced for policy learning and optimization. Through interaction with the environment, the agent continuously optimizes the policy under the guidance of the reward function, with the goal of minimizing the charging costs for individual EV users.
2. The intelligent scheduling method integrating EV charging and discharging behavior clustering and deep reinforcement learning according to claim 1, characterized in that, In data preprocessing, charging parameters of all EVs are collected and standardized. In the formula: These are the original eigenvalues; The value is the standardized value; These are the sample point values.
3. The intelligent scheduling method integrating EV charging and discharging behavior clustering and deep reinforcement learning according to claim 2, characterized in that, The MS algorithm is used to determine the number of EV groups. The specific steps of the MS algorithm are as follows: Step 1: In the normalized feature space, a kernel density estimation method is introduced to nonparametrically model the probability density function of the sample distribution, thereby plotting the density gradient of each point in space. The estimation function is as follows: In the formula: h is the bandwidth coefficient, which balances the fineness of clustering with the smoothness of the result; d is the feature dimension; n is the number of sample points; K(•) is the kernel function; using the Gaussian kernel function can avoid hard boundary effects and has better robustness to noisy points, so as to obtain a smoother density estimate. Step 2: Using the sample point as the initial position, perform iterative updates along the density gradient direction defined by the kernel density function until the drift vector converges to the density maximum point; the mean drift vector is given by the following formula: In the formula: j is the iteration number; Step 3: If the distance between two modal points is less than the preset merging threshold If they are not classified as the same cluster, then a modal merging operation is performed. If the loop ends, continue the iterative calculation and execute Step 2. Obtaining the clustering results: The set of all merged modal points constitutes the final cluster centers, denoted as . Based on the determined number of clusters k, the K-means++ algorithm is used to obtain EV clustering results with different response characteristics.
4. The intelligent scheduling method integrating EV charging and discharging behavior clustering and deep reinforcement learning according to claim 3, characterized in that, To ensure that electric vehicles retain sufficient charge to meet basic travel needs after completing their discharge cycle, the State of Charge (SOC) at the time of departure must be greater than the minimum SOC required by the user, as shown in the following formula: In the formula: Let t be the state of charge of the i-th EV; This represents the minimum required state of charge when the i-th EV leaves. , These are the charging period and the discharging period, respectively. , These are the start of charging and the start of discharging, respectively. , The charging efficiency and discharging efficiency are set to 0.
95.
5. The intelligent scheduling method integrating EV charging and discharging behavior clustering and deep reinforcement learning according to claim 4, characterized in that, The dwell time of an EV at a charging station is as follows: Charge / discharge state constraints: The grid load value after EV integration is updated as follows: In the formula: , These refer to the times when the EV enters and leaves the network; , These represent the charging and discharging states of the EV; 1 indicates charging, -1 indicates discharging, and 0 indicates resting. Let be the discharge power of the i-th EV at time t; N and M are the number of EVs charging and discharging at time t, respectively.
6. The intelligent scheduling method integrating EV charging and discharging behavior clustering and deep reinforcement learning according to claim 1, characterized in that, Within the reinforcement learning framework, an agent continuously learns the optimal policy through interaction with the environment to maximize the reward function; this interaction process can be transformed into a Markov decision process, defined as a quintuple. , where s represents the state space, a represents the action space, and p is the state transition probability. In the DQN solution process, it is not necessary to explicitly model p, but to learn it through the state transition data obtained by interacting with the environment and sampling. Let be the instantaneous reward function at time t. This is a discount factor for future rewards.