Dynamic allocation method of electric vehicle charging power considering voltage balance and user-friendliness

Abstract

The present invention discloses a method for dynamically allocating charging power of electric vehicles taking into account voltage balance and user friendliness, including the steps of: extracting user charging behavior data; performing data processing and data screening; performing data transformation on user charging data and statistically analyzing charging characteristics; constructing a user friendliness index system, statistically analyzing or calculating the values ​​of each index; calculating the weight of the user friendliness index; generating a multi-user multi-dimensional integrated charging scenario using a scenario generation method; determining the charging power allocation principle of electric vehicles, and building a reinforcement learning environment based on the power allocation principle; building an improved DDPG reinforcement learning model; training the reinforcement learning model, outputting the charging power allocation of each user in each scenario and saving the model. In the electric vehicle power allocation method, the present invention considers a new evaluation criterion: user friendliness, and adds a reinforcement learning method to achieve more accurate and efficient dynamic allocation of charging power.

Description

Technical Field

[0001] The invention belongs to the field of electric vehicles, and in particular relates to a method for dynamically allocating charging power of an electric vehicle taking voltage balance and user-friendliness into consideration. Background Art

[0002] As the number of electric vehicles increases, the charging load value of electric vehicles gradually increases, and the load pressure on the power grid side increases significantly. At the same time, problems such as low charging efficiency of electric vehicles and low utilization of charging piles are gradually becoming prominent. Low utilization of charging piles may not lead to problems such as charging station shutdown, but it will cause adverse effects such as loss of revenue of charging stations and reduced user experience.

[0003] In order to further improve the operating efficiency of charging piles and increase the profits of charging operators, it is imperative to coordinate and optimize the bilateral interests between "cars and piles". The charging mode of average power distribution used in the traditional charging process can no longer meet the current charging needs. Therefore, a new power allocation strategy needs to be proposed to deal with this situation.

[0004] The existing power allocation method has some defects in terms of flexibility and intelligence. First, it does not take into account the different maximum charging powers of each vehicle in different scenarios, resulting in fixed power allocation and the inability to flexibly allocate power according to user charging habits; second, it does not take into account the classification of users. Different user charging habits lead to different user categories, and power cannot be intelligently allocated according to user categories. Summary of the invention

[0005] In view of the deficiencies in the prior art, the present invention provides a method for dynamically allocating charging power for electric vehicles taking voltage balance and user-friendliness into consideration. The present invention proposes a user charging scenario based on user charging characteristics; proposes the concept of user-friendliness, and constructs a user-friendliness index system, and uses a comprehensive analysis method to calculate the user-friendliness of each user; builds an environment for a reinforcement learning model according to the charging principle, and determines the environmental variable parameters; selects an improved DDPG algorithm as a reinforcement learning agent, and builds a reinforcement learning model, and finally outputs the allocated power.

[0006] To achieve the above object, the present invention adopts the following technical solutions:

[0007] A method for dynamically allocating charging power of an electric vehicle taking voltage balance and user-friendliness into consideration comprises the following steps:

[0008] S1. extracting charging behavior data of multiple users from a database, and extracting charging characteristics after preprocessing the charging behavior data, wherein the charging characteristics include charging date, charging start time, charging end time, charging duration, charging power and charging location;

[0009] S2. After transforming the charging behavior data of each user, statistics are collected on the charging start time, charging duration, charging power and charging location;

[0010] S3. Construct a user-friendliness index system, count or calculate the value of each index based on the charging characteristics of each user, calculate the weight of the user-friendliness index using a comprehensive evaluation method, and calculate the friendliness of all users;

[0011] S4, using a scenario generation method to generate a multi-user multi-dimensional fusion charging scenario based on the transformed charging behavior data and user friendliness;

[0012] S5. Determine the charging power allocation principle for electric vehicles and build a reinforcement learning environment based on the power allocation principle;

[0013] S6. Determine the parameters of the DDPG reinforcement learning model based on the reinforcement learning environment and build the DDPG reinforcement learning model;

[0014] S7. Use the charging scenario to train the DDPG reinforcement learning model and output the charging power distribution of each user in each charging scenario, and save the trained DDPG reinforcement learning model;

[0015] S8. Use the trained DDPG reinforcement learning model to allocate electric vehicle charging power in actual charging scenarios.

[0016] To optimize the above technical solutions, the specific measures taken also include:

[0017] Further, in S3, the user-friendliness refers to the electric vehicle company's evaluation criteria for the user's charging behavior;

[0018] The user-friendliness index system includes primary indicators, secondary indicators of primary indicators, and tertiary indicators of secondary indicators;

[0019] The first-level indicator is user-friendliness.

[0020] The secondary indicators of the primary indicators include charging station safety indicators, power grid demand response indicators, user-friendly habit indicators, operator economic indicators and user service satisfaction indicators;

[0021] The third-level indicators of the second-level indicators are set as follows:

[0022] Charging station safety indicators include: maximum power of charging piles, charging station utilization rate and average charging compliance rate; grid demand response indicators include: valley charging ratio and peak charging ratio; user-friendly habit indicators include: charging time, valley charging index and low-density charging index; operator economic indicators include: charging price and charging service fee; user service satisfaction indicators include: charging power, transaction volume, transaction amount and number of devices.

[0023] Furthermore, in S3, the specific process of counting or calculating the values ​​of each indicator based on the charging characteristics of each user, calculating the weight of the user-friendliness indicator by a comprehensive evaluation method, and calculating the friendliness of all users includes:

[0024] S31. Calculate or count the values ​​of each third-level index, and unify the values ​​of each third-level index into vector Z k = {Z j}, j = 1, 2, ..., n, where k is the user number, Z j is the index value of each charging behavior, and n is the total number of charging behaviors;

[0025] S32, use the analytic hierarchy process to calculate the subjective weight vector ω′ of the three-level indicators;

[0026] S33, use the entropy weight method to calculate the objective weight ω″ of the three-level indicators;

[0027] S34, based on the subjective and objective indicators of the three-level indicators, use linear weighting to calculate the combined weight ω;

[0028] S35, select the three-level indicator value set I of each user's charging behavior k = {Z k}, use the vector product formula to calculate the friendliness of each user and conduct statistics;

[0029] S36. Use the adaptive density clustering algorithm DBSCAN to cluster the user friendliness after statistics and plan the friendliness range.

[0030] Furthermore, S4 specifically includes:

[0031] S41, using a deep autoencoder to generate a sequence of user charging dates and charging times from a user's charging date matrix;

[0032] S42, modify the user's time-related data according to the following format, wherein the time-related data includes charging start time, charging end time and charging duration:

[0033]

[0034] Where t represents time, [·] represents the rounded integer value of ·;

[0035] S43, determine whether the overall data of the charging start time satisfies the normal distribution function. If so, fit the charging start time into the normal distribution function X~N(μ,σ 2 ), transfer to S45; if not satisfied, transfer to S44;

[0036] S44, the charging start time data is divided into three segments with 2 o'clock, 12 o'clock and 19 o'clock as the dividing points, and 0-2 o'clock in the morning is divided into 19 o'clock, and the three segments of data are respectively fitted with normal distribution to obtain three-segment normal distribution functions

[0037] S45, converting the obtained normal distribution function into a probability density function to obtain a probability density function of the charging start time of a user;

[0038] S46. For charging locations with a charging probability less than 0.1, delete them and fit the remaining charging location data to the following probability distribution function:

[0039]

[0040] Among them, x m is the mth charging location, there are l charging locations in total, n m is the number of times the battery is charged at the mth charging location, N c is the total number of charging times;

[0041] S47: Determine whether the overall data of the user's initial charging SOC satisfies the normal distribution function. If so, fit the user's initial charging SOC into the normal distribution function. Go to S410; otherwise go to S49;

[0042] S48, dividing the user's initial charging SOC data into three segments of 30%, 60%, and 80%, respectively performing normal distribution fitting on the three segments of data to obtain three-segment normal distribution functions

[0043] S49, converting the obtained normal distribution function into a probability density function to obtain a probability density function of the initial SOC of a user's charging;

[0044] S410: Fitting charging time data to a normal distribution function

[0045] S411, calculate the user's charging power each time based on the historical data of the user's transaction power and charging time, and compare to obtain the user's maximum charging power P max , and make the following transformation

[0046] S412: Count the user friendliness set of the user and generate a user friendliness set F * ;

[0047] S412, counting the number of charges M in the generated charging date and charging number matrix, and using the Latin hypercube sampling method to collect the charging start time, charging location, user charging initial SOC and user friendliness set F * M groups of samples are drawn from the data and combined in order, supplemented by the user's maximum charging power P max , generate a user's charging scenario S~{T,L,SOC s ,F,P max} M , where T is the user's single charging time, L is the user's single charging location, SOC s is the initial SOC of a single charge for the user, F is the user-friendliness, P is max Maximum charging power for users.

[0048] Further, in S5, the electric vehicle charging power distribution principle includes:

[0049] Principle 1: When the sum of the maximum charging power of the two vehicles is less than the maximum power output by the charging pile, the output power of the two guns is the maximum charging power of the vehicle, and the formula is as follows:

[0050] When P 桩max >P 1max +P 2max

[0051] Where P1 is the charging power allocated by the charging pile to the first electric car, P2 is the charging power allocated by the charging pile to the second electric car, and P 桩max is the maximum power output of the charging pile, P 1max is the maximum charging power of the first vehicle, P 2max The maximum charging power for the second vehicle;

[0052] Principle 2: When charging time is at peak power consumption, the charging power needs to be reduced; when charging time is at low power consumption, the charging power should be increased as appropriate;

[0053] Principle 3: When the user-friendliness gap between two vehicles is greater than the set user-friendliness gap and Principle 1 is not met, the charging power of the vehicle with relatively lower user-friendliness is reduced, and the charging power of the vehicle with relatively higher user-friendliness is increased;

[0054] Principle 4: When the SOC gap between the two vehicles is greater than the set SOC gap and does not meet Principle 1, the charging power of the vehicle with a relatively higher SOC is reduced, and the charging power of the vehicle with a relatively lower SOC is increased;

[0055] Principle 5: When the charging voltage of two vehicles is lower than the minimum charging reference voltage, increase the charging voltage of the vehicle, reduce the charging current or increase the charging power.

[0056] Further, in S5, the specific process of building a reinforcement learning environment based on the power allocation principle is:

[0057] S51, determining parameters of the reinforcement learning environment, wherein the parameters include actions, states, and rewards; specifically comprising the following steps:

[0058] Set the action to the output power of the two muzzles of the charging pile, represented by the array [P1, P2], where P1 is the charging power allocated by the charging pile to the first electric car, and P2 is the charging power allocated by the charging pile to the second electric car;

[0059] The state is defined as the current state of the charging pile and the state of the electric vehicle collected by the charging pile, including the current SOC of the vehicle, forming the current SOC array of the two electric vehicles [SOC1, SOC2], where SOC1 is the current SOC of the first electric vehicle and SOC2 is the current SOC of the second electric vehicle;

[0060] The total reward function R consists of 4 reward functions:

[0061]

[0062] Among them, R1 is the first reward function, P emax is the maximum charging power of the e-th electric vehicle, P e is the charging power received by the e-th electric vehicle, R2 is the second reward function, P 桩max is the maximum output power of the charging pile, R3 is the third reward function, F1 and F2 are the user-friendliness of the first and second electric vehicles, R4 is the fourth reward function, and U e is the charging voltage of the e-th electric vehicle, U ref is the minimum reference voltage for charging, λ1, λ2, λ3 and λ4 are the coefficients of the four reward functions;

[0063] S52. The terminal state of reinforcement learning is defined as satisfying all of the following conditions:

[0064] Condition 1: The sum of the muzzle output power does not exceed the maximum output power of the charging pile;

[0065] Condition 2: The muzzle output power does not exceed the maximum charging power of the vehicle;

[0066] Condition 3: The vehicle SOC reaches 90% or more.

[0067] Furthermore, S6 is specifically:

[0068] S61. Define the state space as S, the action space as A, and the reward function as Where h is the number of reward functions;

[0069] S62, use s to represent the current state of the environment, use a to represent the action generated by the policy network, define the parameter of the policy network as θ, and the function of its output action as μ θ (s), define the parameter of the value function network as φ, and its output is Q φ (s,a);

[0070] S63. Determine the objective function L(φ) of the value function network and the objective function of the policy network

[0071]

[0072] where y i =r i +γQ φ' (s i+1, μ θ' (s i+1 ), N is the number of sampled experiences, r i is the reward value of the i-th stage, γ is the learning rate of reinforcement learning, Q φ' (s i+1 ,μ θ' (s i+1 )) represents the output result of the target value network at step i+1, s i+1 represents the state of step i+1, μ θ' (s i+1 ) means in s i+1 The output result of the target strategy network in the state; J(θ) is the objective function of the reinforcement learning strategy, μ θ (s i ) means in s i The decision made by the policy network in the state;

[0073] S64, introducing target strategy network parameter θ' and target value function network parameter φ' for calculating the target value.

[0074] Furthermore, S7 specifically includes:

[0075] S71. Compare all individual user charging scenarios S~{T, L, SOC s ,F,P max} M , group different user charging scenarios with charging time within 1 hour and the same charging location into one set, and randomly combine them in pairs as a comprehensive charging scenario;

[0076] S72, train the DDPG reinforcement learning model with each comprehensive charging scenario; after one round of training, observe whether the reward function curve converges and is reasonable. If so, go to step S74, otherwise go to S73;

[0077] S73, using an adaptive parameterization method to adjust the coefficients of the multi-objective function in the reward function, and training again until the stop condition is met;

[0078] S74, outputting the final power allocation result for each scenario;

[0079] S75. Save the trained DDPG reinforcement learning model.

[0080] Further, in S72, the training of the DDPG reinforcement learning model with each comprehensive charging scenario specifically includes:

[0081] S72.1. Set a time interval Dt, calculate the vehicle SOC every time interval in a scene, and fit the SOC curve in a linear proportion; set the input state space dimension and the output action space dimension;

[0082] S72.2. Initialize the policy network parameters θ, the value function network parameters φ, the target policy network parameters θ' and the target value function network parameters φ'. The neural network in the policy network adopts a fully connected neural network with two hidden layers. The dimensions of the two hidden layers are 4 and 10 respectively, and the activation function is the ReLU function.

[0083] S72.3, use DDPG algorithm to update the policy network, and the learning rate is 10 -3 , set the discount factor γ = 0.6, the sampling trajectory length is 1000, and the maximum number of iterations is 1000;

[0084] S72.4. Use the experience replay buffer D to store the experience tuple (s, a, r, s');

[0085] S72.5. Target y calculated by target value network objective function i Update the value function network parameters φ, and update the policy network parameters θ through the policy network objective function;

[0086] S72.6. Update the target strategy network parameter θ' and the target value function network parameter φ', the formula is as follows:

[0087] θ'←τθ+(1-τ)θ'

[0088] φ'←τφ+(1-τ)φ'

[0089] Where τ is the update rate of the target policy network and the target value function network.

[0090] Furthermore, S73 is specifically:

[0091] S73.1. Initialize the coefficients λ1, λ2, λ3 and λ4 of the four reward functions to

[0092] S73.2. Use the coefficients of the current reward function to calculate the values ​​of the objective function of the value function network and the objective function of the policy network, and evaluate the performance of the DDPG reinforcement learning model under the current coefficients;

[0093] S73.3. Based on the evaluation results, a genetic algorithm is used to update the coefficients of the reward function so that the objective function of the value function network converges with the objective function of the policy network.

[0094] The beneficial effects of the present invention are:

[0095] In the electric vehicle power allocation method, the present invention considers a new evaluation criterion: user-friendliness, and adds a reinforcement learning method to achieve more accurate and efficient dynamic allocation of charging power. This method can not only allocate charging power according to factors such as the user's charging needs, but also dynamically adjust the charging power in time according to the power grid load conditions, thereby achieving efficient and stable operation of the charging system. The present invention adopts reinforcement learning technology, and through continuous learning and optimization, it improves the accuracy and precision of charging power prediction and allocation, providing users with more convenient and efficient charging services. BRIEF DESCRIPTION OF THE DRAWINGS

[0096] Figure 1 It is an overall flow chart of the method for dynamically allocating charging power for electric vehicles taking voltage balance and user-friendliness into consideration, proposed by the present invention.

[0097] Figure 2 It is a user-friendliness index system constructed by the present invention.

[0098] Figure 3 It is a flow chart of the user-friendliness index system evaluation method of the present invention.

[0099] Figure 4 It is a flow chart of the multi-objective reinforcement learning training process of the present invention. DETAILED DESCRIPTION

[0100] The present invention will now be described in further detail with reference to the accompanying drawings.

[0101] like Figures 1 to 4 As shown, the present invention provides a method for dynamically allocating charging power of an electric vehicle taking voltage balance and user-friendliness into consideration, comprising the following steps:

[0102] S1. Extract charging behavior data of 2,000 users from the database, extract charging features after preprocessing the charging behavior data, the charging features include charging date, charging start time, charging end time, charging duration, charging power and charging location; the data comes from the charging behavior data of electric vehicles in the past three years provided by an electric vehicle company, wherein the data preprocessing needs to delete the incomplete data and erroneous data, the data screening removes some irrelevant data, and converts the data that cannot be counted into numbers, such as charging start time, charging end time and charging duration.

[0103] S2. After transforming the charging behavior data of each user, statistics are collected on the charging start time, charging duration, charging power and charging location;

[0104] S3. Construct a user-friendliness index system, count or calculate the value of each index based on the charging characteristics of each user, calculate the weight of the user-friendliness index using a comprehensive evaluation method, and calculate the friendliness of all users; specifically, including:

[0105] S31. Construct the index system. First-level index: user-friendliness;

[0106] S32. Construct secondary indicators in the primary indicators of the indicator system: charging station safety indicators, power grid demand response indicators, user-friendly habit indicators, operator economic indicators, and user service satisfaction indicators;

[0107] S33. Determine the third-level indicators of each second-level indicator: charging station safety indicators include: maximum power of charging piles, charging station utilization rate, average charging compliance rate; grid demand response indicators include: valley charging ratio, peak charging ratio; user-friendly habit indicators include: charging time, valley charging index; low-density charging index; operator economic indicators include: charging price, charging service fee; user service satisfaction indicators include: charging power, transaction power, transaction amount, number of devices;

[0108] S34. Count or calculate the value of each indicator: Calculate or count the value of each third-level indicator and unify it into vector Z k = {Z j}, j = 1, 2, ..., n, where k is the user number, Z j is the index value of each charging behavior, and n is the total number of charging behaviors.

[0109] S35, use the hierarchical analysis method to calculate the subjective weight vector ω′ of the three-level indicators;

[0110] S36, using the entropy weight method to calculate the objective weight ω″ of the three-level indicators;

[0111] S37, based on the subjective and objective indicators of the three-level indicators, use linear weighting to calculate the combined weight ω;

[0112] S38, select each user's charging behavior three-level indicator value set I k = {Z k}, use the vector product formula to calculate the friendliness of each user and conduct statistics;

[0113] S39. Use the adaptive density clustering algorithm DBSCAN to cluster the user friendliness after statistics and plan a reasonable friendliness range.

[0114] S4. Generate a multi-user multi-dimensional integrated charging scenario based on the transformed charging behavior data and user friendliness using a scenario generation method; specifically including:

[0115] S41, using a deep autoencoder to generate a sequence of user charging dates and charging times from a user's charging date matrix;

[0116] S42, modify the user's time-related data according to the following format, wherein the time-related data includes charging start time, charging end time and charging duration:

[0117]

[0118] Where t represents time, [·] represents the rounded integer value of ·;

[0119] S43, determine whether the overall data of the charging start time satisfies the normal distribution function. If so, fit the charging start time into the normal distribution function X~N(μ,σ 2 ), transfer to S45; if not satisfied, transfer to S44;

[0120] S44, the charging start time data is divided into three segments with 2 o'clock, 12 o'clock and 19 o'clock as the dividing points, and 0-2 o'clock in the morning is divided into 19 o'clock, and the three segments of data are respectively fitted with normal distribution to obtain three-segment normal distribution functions

[0121] S45, converting the obtained normal distribution function into a probability density function to obtain a probability density function of the charging start time of a user;

[0122] S46. For charging locations with a charging probability less than 0.1, delete them and fit the remaining charging location data to the following probability distribution function:

[0123]

[0124] Among them, x m is the mth charging location, there are l charging locations in total, n mis the number of times the battery is charged at the mth charging location, N c is the total number of charging times;

[0125] S47: Determine whether the overall data of the user's initial charging SOC satisfies the normal distribution function. If so, fit the user's initial charging SOC into the normal distribution function. Go to S410; otherwise go to S49;

[0126] S48, dividing the user's initial charging SOC data into three segments of 30%, 60%, and 80%, respectively performing normal distribution fitting on the three segments of data to obtain three-segment normal distribution functions

[0127] S49, converting the obtained normal distribution function into a probability density function to obtain a probability density function of the initial SOC of a user's charging;

[0128] S410: Fitting charging time data to a normal distribution function

[0129] S411, calculate the user's charging power each time based on the historical data of the user's transaction power and charging time, and compare to obtain the user's maximum charging power P max , and make the following transformation

[0130] S412: Count the user friendliness set of the user and generate a user friendliness set F * ;

[0131] S412, counting the number of charges M in the generated charging date and charging number matrix, and using the Latin hypercube sampling method to collect the charging start time, charging location, user charging initial SOC and user friendliness set F * M groups of samples are drawn from the data and combined in order, supplemented by the user's maximum charging power P max , generate a user's charging scenario S~{T,L,SOC s ,F,P max} M , where T is the user's single charging time, L is the user's single charging location, SOC s is the initial SOC of a single charge for the user, F is the user-friendliness, P is max Maximum charging power for users.

[0132] S5. Determine the electric vehicle charging power allocation principle and build a reinforcement learning environment based on the power allocation principle; the electric vehicle charging power allocation principle includes:

[0133] Principle 1: When the sum of the maximum charging power of the two vehicles is less than the maximum power output by the charging pile, the output power of the two guns is the maximum charging power of the vehicle, and the formula is as follows:

[0134] When P 桩max >P 1max +P 2max

[0135] Where P1 is the charging power allocated by the charging pile to the first electric car, P2 is the charging power allocated by the charging pile to the second electric car, and P 桩max is the maximum power output of the charging pile, P 1max is the maximum charging power of the first vehicle, P 2max The maximum charging power for the second vehicle;

[0136] Principle 2: When charging time is at peak power consumption, the charging power needs to be reduced; when charging time is at low power consumption, the charging power should be increased as appropriate;

[0137] Principle 3: When the user-friendliness gap between two vehicles is greater than the set user-friendliness gap and Principle 1 is not met, the charging power of the vehicle with relatively lower user-friendliness is reduced, and the charging power of the vehicle with relatively higher user-friendliness is increased;

[0138] Principle 4: When the SOC gap between the two vehicles is greater than the set SOC gap and does not meet Principle 1, the charging power of the vehicle with a relatively higher SOC is reduced, and the charging power of the vehicle with a relatively lower SOC is increased;

[0139] Principle 5: When the charging voltage of two vehicles is lower than the minimum charging reference voltage, increase the charging voltage of the vehicle, reduce the charging current or increase the charging power.

[0140] The specific process of building a reinforcement learning environment based on the power allocation principle is:

[0141] S51, determining parameters of the reinforcement learning environment, wherein the parameters include actions, states, and rewards; specifically comprising the following steps:

[0142] Set the action to the output power of the two muzzles of the charging pile, represented by the array [P1, P2], where P1 is the charging power allocated by the charging pile to the first electric car, and P2 is the charging power allocated by the charging pile to the second electric car;

[0143] The state is defined as the current state of the charging pile and the state of the electric vehicle collected by the charging pile, including the current SOC of the vehicle, forming the current SOC array of the two electric vehicles [SOC1, SOC2], where SOC1 is the current SOC of the first electric vehicle and SOC2 is the current SOC of the second electric vehicle;

[0144] The total reward function R consists of 4 reward functions:

[0145]

[0146] Among them, R1 is the first reward function, P emax is the maximum charging power of the e-th electric vehicle, P e is the charging power received by the e-th electric vehicle, R2 is the second reward function, P 桩max is the maximum output power of the charging pile, R3 is the third reward function, F1 and F2 are the user-friendliness of the first and second electric vehicles, R4 is the fourth reward function, and U e is the charging voltage of the e-th electric vehicle, U reφ is the minimum reference voltage for charging, λ1, λ2, λ3 and λ4 are the coefficients of the four reward functions;

[0147] S52. The terminal state of reinforcement learning is defined as satisfying all of the following conditions:

[0148] Condition 1: The sum of the muzzle output power does not exceed the maximum output power of the charging pile;

[0149] Condition 2: The muzzle output power does not exceed the maximum charging power of the vehicle;

[0150] Condition 3: The vehicle SOC reaches 90% or more.

[0151] S6. Determine the parameters of the Deep Deterministic Policy Gradient (DDPG) reinforcement learning model based on the reinforcement learning environment and build the DDPG reinforcement learning model; including:

[0152] S61. Define the state space as S, the action space as A, and the reward function as Where h is the number of reward functions;

[0153] S62, use s to represent the current state of the environment, use a to represent the action generated by the policy network, define the parameter of the policy network as θ, and the function of its output action as μ θ (s), define the parameter of the value function network as φ, and its output is Q φ (s,a);

[0154] S63. Determine the objective function L(φ) of the value function network and the objective function of the policy network

[0155]

[0156] where y i =r i +γQ φ' (s i+1 ,μ θ' (s i+1 ), N is the number of sampled experiences, r i is the reward value of the i-th stage, γ is the learning rate of reinforcement learning, Q φ' (s i+1 ,μ θ' (s i+1 )) represents the output result of the target value network at step i+1, s i+1 represents the state of step i+1, μ θ' (s i+1 ) means in s i+1 The output result of the target strategy network in the state; J(θ) is the objective function of the reinforcement learning strategy, μ θ (s i ) means in s i The decision made by the policy network in the state;

[0157] S64, introducing target strategy network parameter θ' and target value function network parameter φ' for calculating the target value.

[0158] S7. Use the charging scenario to train the DDPG reinforcement learning model and output the charging power distribution of each user in each charging scenario, and save the trained DDPG reinforcement learning model; specifically including:

[0159] S71. Compare all individual user charging scenarios S~{T, L, SOC s ,F,P max} M , group different user charging scenarios with charging time within 1 hour and the same charging location into one set, and randomly combine them in pairs as a comprehensive charging scenario;

[0160] S72, train the DDPG reinforcement learning model with each comprehensive charging scenario; after one round of training, observe whether the reward function curve converges and is reasonable. If so, go to step S74, otherwise go to S73;

[0161] In S72, the training of the DDPG reinforcement learning model with each comprehensive charging scenario specifically includes:

[0162] S72.1. Calculate the vehicle SOC every 5 minutes in a scenario, fit the SOC curve in a linear ratio, set the time interval Dt = 5min; set the input state space dimension and the output action space dimension to 2;

[0163] S72.2. Initialize the policy network parameters θ, the value function network parameters φ, the target policy network parameters θ' and the target value function network parameters φ'. The neural network in the policy network adopts a fully connected neural network with two hidden layers. The dimensions of the two hidden layers are 4 and 10 respectively, and the activation function is the ReLU function.

[0164] S72.3, use DDPG algorithm to update the policy network, the learning rate is 10 -3 , set the discount factor γ = 0.6, the sampling trajectory length is 1000, and the maximum number of iterations is 1000;

[0165] S72.4. Use the experience replay buffer D to store the experience tuple (s, a, r, s');

[0166] S72.5. Target y calculated by target value network objective function i Update the value function network parameters φ, and update the policy network parameters θ through the policy network objective function;

[0167] S72.6. Update the target strategy network parameter θ' and the target value function network parameter φ', the formula is as follows:

[0168] θ'←τθ+(1-τ)θ'

[0169] φ'←τφ+(1-τ)φ'

[0170] Where τ is the update rate of the target policy network and the target value function network.

[0171] S73, using an adaptive parameterization method to adjust the coefficients of the multi-objective function in the reward function, and training again until the stop condition is met; specifically:

[0172] S73.1. Initialize the coefficients λ1, λ2, λ3 and λ4 of the four reward functions to

[0173] S73.2. Use the coefficients of the current reward function to calculate the values ​​of the objective function of the value function network and the objective function of the policy network, and evaluate the performance of the DDPG reinforcement learning model under the current coefficients;

[0174] S73.3. Based on the evaluation results, a genetic algorithm is used to update the coefficients of the reward function so that the objective function of the value function network converges with the objective function of the policy network.

[0175] S74, outputting the final power allocation result for each scenario;

[0176] S75. Save the trained DDPG reinforcement learning model.

[0177] S8. Use the trained DDPG reinforcement learning model to allocate electric vehicle charging power in actual charging scenarios.

[0178] The above are only preferred embodiments of the present invention, and the protection scope of the present invention is not limited to the above embodiments. All technical solutions under the concept of the present invention belong to the protection scope of the present invention. It should be pointed out that for ordinary technicians in this technical field, some improvements and modifications without departing from the principle of the present invention should be regarded as the protection scope of the present invention.

Claims

1. A method for dynamically allocating charging power for electric vehicles taking into account voltage balance and user-friendliness, characterized in that: The steps include: S1. extracting charging behavior data of multiple users from a database, and extracting charging characteristics after preprocessing the charging behavior data, wherein the charging characteristics include charging date, charging start time, charging end time, charging duration, charging power and charging location; S2. After transforming the charging behavior data of each user, statistics are collected on the charging start time, charging duration, charging power and charging location; S3, construct a user-friendliness index system, count or calculate the value of each index based on the charging characteristics of each user, calculate the weight of the user-friendliness index using a comprehensive evaluation method, and calculate the friendliness of all users; in S3, the user-friendliness refers to the electric vehicle company's evaluation criteria for the user's charging behavior; The user-friendliness index system includes primary indicators, secondary indicators of primary indicators, and tertiary indicators of secondary indicators; The first-level indicator is user-friendliness. The secondary indicators of the primary indicators include charging station safety indicators, power grid demand response indicators, user-friendly habit indicators, operator economic indicators and user service satisfaction indicators; The third-level indicators of the second-level indicators are set as follows: The charging station safety indicators include: maximum power of charging piles, charging station utilization rate and average charging compliance rate; the grid demand response indicators include: valley charging ratio and peak charging ratio; the user-friendly habit indicators include: charging time, valley charging index and low-density charging index; the operator economic indicators include: charging price and charging service fee; the user service satisfaction indicators include: charging power, transaction power, transaction amount and number of devices; In S3, the specific process of counting or calculating the values ​​of each indicator based on the charging characteristics of each user, calculating the weight of the user-friendliness indicator by a comprehensive evaluation method, and calculating the friendliness of all users includes: S31. Calculate or count the values ​​of each third-level index, and unify the values ​​of each third-level index into vector Z k = {Z j }, j = 1, 2, ..., n, where k is the user number, Z j is the index value of each charging behavior, and n is the total number of charging behaviors; S32, use the analytic hierarchy process to calculate the subjective weight vector ω′ of the three-level indicators; S33, use the entropy weight method to calculate the objective weight ω″ of the three-level indicators; S34, based on the subjective and objective indicators of the three-level indicators, use linear weighting to calculate the combined weight ω; S35, select the three-level indicator value set I of each user's charging behavior k = {Z k }, use the vector product formula to calculate the friendliness of each user and conduct statistics; S36, clustering the user friendliness after statistics using the adaptive density clustering algorithm DBSCAN, and planning the friendliness range; S4, using a scenario generation method to generate a multi-user multi-dimensional fusion charging scenario based on the transformed charging behavior data and user friendliness; S5. Determine the charging power allocation principle for electric vehicles and build a reinforcement learning environment based on the power allocation principle; S6. Determine the parameters of the DDPG reinforcement learning model based on the reinforcement learning environment and build the DDPG reinforcement learning model; S7. Use the charging scenario to train the DDPG reinforcement learning model and output the charging power distribution of each user in each charging scenario, and save the trained DDPG reinforcement learning model; S8. Use the trained DDPG reinforcement learning model to allocate electric vehicle charging power in actual charging scenarios.

2. The method for dynamically allocating charging power of electric vehicles considering voltage balance and user-friendliness as claimed in claim 1, characterized in that: S4 specifically includes: S41, using a deep autoencoder to generate a sequence of user charging dates and charging times from a user's charging date matrix; S42, modify the user's time-related data according to the following format, wherein the time-related data includes charging start time, charging end time and charging duration: Where t represents time, [·] represents the rounded integer value of ·; S43, determine whether the overall data of the charging start time satisfies the normal distribution function. If so, fit the charging start time into the normal distribution function X~N(μ,σ 2 ), transfer to S45; if not satisfied, transfer to S44; S44, the charging start time data is divided into three segments with 2 o'clock, 12 o'clock and 19 o'clock as the dividing points, and 0-2 o'clock in the morning is divided into 19 o'clock, and the three segments of data are respectively fitted with normal distribution to obtain three-segment normal distribution functions S45, converting the obtained normal distribution function into a probability density function to obtain a probability density function of the charging start time of a user; S46. For charging locations with a charging probability less than 0.1, delete them and fit the remaining charging location data to the following probability distribution function: Among them, x m is the mth charging location, there are l charging locations in total, n m is the number of times the battery is charged at the mth charging location, N c is the total number of charging times; S47: Determine whether the overall data of the user's initial charging SOC satisfies the normal distribution function. If so, fit the user's initial charging SOC into the normal distribution function. Go to S410; otherwise go to S49; S48, dividing the user's initial charging SOC data into three segments of 30%, 60%, and 80%, respectively performing normal distribution fitting on the three segments of data to obtain three-segment normal distribution functions S49, converting the obtained normal distribution function into a probability density function to obtain a probability density function of the initial SOC of a user's charging; S410: Fitting charging time data to a normal distribution function S411, calculate the user's charging power each time based on the historical data of the user's transaction power and charging time, and compare to obtain the user's maximum charging power P max , and make the following transformation S412: Count the user friendliness set of the user and generate a user friendliness set F * ; S412, counting the number of charges M in the generated charging date and charging number matrix, and using the Latin hypercube sampling method to collect the charging start time, charging location, user charging initial SOC and user friendliness set F * M groups of samples are drawn from the data and combined in order, supplemented by the user's maximum charging power P max , generate a user's charging scenario S~{T,L,SOC s ,F,P max } M , where T is the user's single charging time, L is the user's single charging location, SOC s is the initial SOC of a single charge for the user, F is the user-friendliness, P is max Maximum charging power for users.

3. The method for dynamically allocating charging power of electric vehicles considering voltage balance and user-friendliness as claimed in claim 1, characterized in that: In S5, the electric vehicle charging power distribution principle includes: Principle 1: When the sum of the maximum charging power of the two vehicles is less than the maximum power output by the charging pile, the output power of the two guns is the maximum charging power of the vehicle, and the formula is as follows: When P 桩max >P 1max +P 2max Where P1 is the charging power allocated by the charging pile to the first electric car, P2 is the charging power allocated by the charging pile to the second electric car, and P 桩max is the maximum power output of the charging pile, P 1max is the maximum charging power of the first vehicle, P 2max The maximum charging power for the second vehicle; Principle 2: When charging time is at peak power consumption, the charging power needs to be reduced; when charging time is at low power consumption, the charging power should be increased as appropriate; Principle 3: When the user-friendliness gap between two vehicles is greater than the set user-friendliness gap and Principle 1 is not met, the charging power of the vehicle with relatively lower user-friendliness is reduced, and the charging power of the vehicle with relatively higher user-friendliness is increased; Principle 4: When the SOC gap between the two vehicles is greater than the set SOC gap and does not meet Principle 1, the charging power of the vehicle with a relatively higher SOC is reduced, and the charging power of the vehicle with a relatively lower SOC is increased; Principle 5: When the charging voltage of two vehicles is lower than the minimum charging reference voltage, increase the charging voltage of the vehicle, reduce the charging current or increase the charging power.

4. The method for dynamically allocating charging power of electric vehicles considering voltage balance and user friendliness as claimed in claim 3, characterized in that: In S5, the specific process of building a reinforcement learning environment based on the power allocation principle is: S51, determining parameters of the reinforcement learning environment, wherein the parameters include actions, states, and rewards; specifically comprising the following steps: Set the action to the output power of the two muzzles of the charging pile, represented by the array [P1, P2], where P1 is the charging power allocated by the charging pile to the first electric car, and P2 is the charging power allocated by the charging pile to the second electric car; The state is defined as the current state of the charging pile and the state of the electric vehicle collected by the charging pile, including the current SOC of the vehicle, forming the current SOC array of the two electric vehicles [SOC1, SOC2], where SOC1 is the current SOC of the first electric vehicle and SOC2 is the current SOC of the second electric vehicle; The total reward function R consists of 4 reward functions: Among them, R1 is the first reward function, P emax is the maximum charging power of the e-th electric vehicle, P e is the charging power received by the e-th electric vehicle, R2 is the second reward function, P 桩max is the maximum output power of the charging pile, R3 is the third reward function, F1 and F2 are the user-friendliness of the first and second electric vehicles, R4 is the fourth reward function, and U e is the charging voltage of the e-th electric vehicle, U ref is the minimum reference voltage for charging, λ1, λ2, λ3 and λ4 are the coefficients of the four reward functions; S52. The terminal state of reinforcement learning is defined as satisfying all of the following conditions: Condition 1: The sum of the muzzle output power does not exceed the maximum output power of the charging pile; Condition 2: The muzzle output power does not exceed the maximum charging power of the vehicle; Condition 3: The vehicle SOC reaches 90% or more.

5. The method for dynamically allocating charging power for electric vehicles taking into account voltage balance and user friendliness as claimed in claim 2, characterized in that: S6 is specifically: S61. Define the state space as S, the action space as A, and the reward function as R: Where h is the number of reward functions; S62, use s to represent the current state of the environment, use a to represent the action generated by the policy network, define the parameter of the policy network as θ, and its output action function as μ θ (s), define the parameter of the value function network as φ, and its output is Q φ (s,a); S63. Determine the objective function L(φ) of the value function network and the objective function of the policy network where y i =r i +γQ φ' (s i+1 ,μ θ' (s i+1 ), N is the number of sampled experiences, r i is the reward value of the i-th stage, γ is the learning rate of reinforcement learning, Q φ' (s i+1 ,μ θ' (s i+1 )) represents the output result of the target value network at step i+1, s i+1 represents the state of step i+1, μ θ' (s i+1 ) means in s i+1 The output result of the target strategy network in the state; J(θ) is the objective function of the reinforcement learning strategy, μ θ (s i ) means in s i The decision made by the policy network in the state; S64, introducing target strategy network parameter θ' and target value function network parameter φ' for calculating the target value.

6. The method for dynamically allocating charging power of electric vehicles taking into account voltage balance and user friendliness as claimed in claim 5, characterized in that: S7 specifically includes: S71. Compare all individual user charging scenarios S~{T, L, SOC s ,F,P max } M , group different user charging scenarios with charging time within 1 hour and the same charging location into one set, and randomly combine them in pairs as a comprehensive charging scenario; S72, train the DDPG reinforcement learning model with each comprehensive charging scenario; after one round of training, observe whether the reward function curve converges and is reasonable. If so, go to step S74, otherwise go to S73; S73, using an adaptive parameterization method to adjust the coefficients of the multi-objective function in the reward function, and training again until the stop condition is met; S74, outputting the final power allocation result for each scenario; S75. Save the trained DDPG reinforcement learning model.

7. The method for dynamically allocating charging power of electric vehicles taking into account voltage balance and user friendliness as claimed in claim 6, characterized in that: In S72, the training of the DDPG reinforcement learning model with each comprehensive charging scenario specifically includes: S72.

1. Set a time interval Δt; calculate the vehicle SOC every time interval in a scenario, fit the SOC curve in a linear proportion, and set the input state space dimension and the output action space dimension; S72.

2. Initialize the policy network parameters θ, the value function network parameters φ, the target policy network parameters θ' and the target value function network parameters φ'. The neural network in the policy network adopts a fully connected neural network with two hidden layers. The dimensions of the two hidden layers are 4 and 10 respectively, and the activation function is the ReLU function. S72.3, use DDPG algorithm to update the policy network, and the learning rate is 10 -3 , set the discount factor γ = 0.6, the sampling trajectory length is 1000, and the maximum number of iterations is 1000; S72.

4. Use the experience replay buffer D to store the experience tuple (s, a, r, s'); S72.

5. Target y calculated by target value network objective function i Update the value function network parameters φ, and update the policy network parameters θ through the policy network objective function; S72.

6. Update the target strategy network parameter θ' and the target value function network parameter φ', the formula is as follows: θ'←τθ+(1-τ)θ' φ'←τφ+(1-τ)φ' Where τ is the update rate of the target policy network and the target value function network.

8. The method for dynamically allocating charging power for electric vehicles taking into account voltage balance and user-friendliness as claimed in claim 6, characterized in that: S73 is specifically: S73.

1. Initialize the coefficients λ1, λ2, λ3 and λ4 of the four reward functions to S73.

2. Use the coefficients of the current reward function to calculate the values ​​of the objective function of the value function network and the objective function of the policy network, and evaluate the performance of the DDPG reinforcement learning model under the current coefficients; S73.

3. Based on the evaluation results, a genetic algorithm is used to update the coefficients of the reward function so that the objective function of the value function network converges with the objective function of the policy network.

Images