Large-scale charging station deep reinforcement learning ordered charging method based on disjunction graph

Abstract

The invention discloses a large-scale charging station deep reinforcement learning ordered charging method based on a disjunction graph, and the method comprises the following steps: 1) discretizing the charging tasks of an electric vehicle, and obtaining the number of discrete charging tasks and a predicted parking time demand; 3) printing an electricity price identifier for the electric vehicle; 4) constructing a charging scheduling disjunction graph based on the number of the discrete charging tasks and the electricity price identifier; 5) performing feature extraction on the disjunction graph by adopting a graph attention mechanism to obtain global features; 6) constructing a value network and a strategy network; 7) inputting the global features into a value network and a strategy network, and predicting the score of each action by using a multi-layer perceptron to obtain the probability of each charging action; and 8) selecting the charging action with the maximum probability as an electric vehicle charging strategy. According to the electric vehicle ordered charging scheduling strategy based on the charging priority model and the graph neural network reinforcement learning method, the operation efficiency and resource distribution of the charging station are remarkably optimized.

Description

Technical Field

[0001] The present invention relates to the field of electric vehicles, and in particular to a deep reinforcement learning orderly charging method for large-scale charging stations based on a disjunctive graph. Background Art

[0002] As a clean means of transportation that reduces greenhouse gas emissions and reduces dependence on fossil energy, electric vehicles have developed rapidly in recent years. By the end of 2023, China's new energy vehicle ownership has exceeded 20.41 million, and continues to grow at an annual growth rate of more than 30%. However, the rapid popularization of electric vehicles also puts higher requirements on the scientific deployment and scheduling of charging facilities. Unreasonable charging behavior may bring challenges to the stable operation of the power grid. Therefore, the study of orderly charging strategies is of great significance to the stability of the power grid and the development of low-carbon transportation.

[0003] Although the orderly charging strategy is widely regarded as an effective way to solve the problem of grid load fluctuation, the existing technology still has many defects in practical application. First, due to the high randomness of the arrival time of electric vehicles, the charging demand shows significant dynamic fluctuation characteristics. The existing optimization methods often assume that the environmental information is complete and fixed, which makes it difficult to deal with this uncertainty. This assumption leads to the low applicability of traditional optimization algorithms in actual scenarios, and the charging scheduling results often fail to meet expectations. Secondly, many current optimization methods rely on complex objective functions and large-scale iterative calculations, such as Q learning in reinforcement learning methods. Although such methods can theoretically achieve the global optimal solution, with the increase in the number of electric vehicles, the dimensions of the state and action space are significantly expanded, leading to the aggravation of the dimensionality curse problem. Especially in real-time charging scheduling, the high computational complexity and large solution delay make it difficult for these algorithms to meet the real-time requirements, thereby affecting the execution efficiency of the scheduling strategy. In addition, most of the existing methods take a single vehicle or a cluster of vehicles as the research object, and train each vehicle independently, ignoring the balance between group characteristics and global optimization, further increasing the instability of the system. Under the dual pressure of dynamic changes in grid load and high-frequency access of electric vehicles, the existing technology still has a lot of room for improvement in terms of balancing efficiency and reliability. These defects make how to design efficient and real-time scheduling optimization algorithms under the uncertainty and dynamic conditions of the charging process an important challenge in the research of orderly charging scheduling of electric vehicles. Summary of the invention

[0004] The purpose of the present invention is to provide a large-scale charging station deep reinforcement learning orderly charging method based on a disjunctive graph, comprising the following steps:

[0005] 1) Obtain charging station data, including electricity price data and charging data;

[0006] 2) Based on the charging data, the charging tasks of electric vehicles are discretized to obtain the number of discrete charging tasks and the estimated parking time requirements;

[0007] 3) Based on the expected parking time demand and electricity price data, the electric vehicle is marked with an electricity price identifier;

[0008] 4) Construct a charging scheduling disjunctive graph based on the number of discrete charging tasks and electricity price identifiers;

[0009] 5) Use the graph attention mechanism to extract features from the disjunctive graph and obtain global features;

[0010] 6) Construct value network and strategy network;

[0011] 7) Input the global features into the value network and the policy network, use the multi-layer perceptron to predict the score of each action, and obtain the probability of each charging action;

[0012] 8) Select the charging action with the highest probability as the electric vehicle charging strategy.

[0013] Furthermore, the charging station data includes electricity price data and charging data.

[0014] Furthermore, the electricity price data is the electricity price in the past t period;

[0015] The charging data includes charging demand data and estimated departure time data;

[0016] The charging demand data is the time when the electric vehicle arrives at the electric vehicle charging station and starts charging;

[0017] The estimated departure time data is the end time of charging of the electric vehicle.

[0018] Further, the number of discrete charging tasks is as follows:

[0019]

[0020] in, represents the charging demand of the i-th electric vehicle at time t, I t represents the set of electric vehicles charging at the charging station at time t, Δ t Indicates the time interval between each scheduled task; P i Represents the number of discrete charging tasks for the i-th electric vehicle.

[0021] Further, the estimated parking time demand is as follows:

[0022]

[0023] Where, d irepresents the expected parking time requirement of the i-th electric vehicle in a scheduling task; D i represents the total expected parking time demand of the i-th electric vehicle.

[0024] Further, the electricity price identifier is as follows:

[0025]

[0026] Among them, c t Represents the current electricity price, and T represents the current time. Charging task for time t; is the expected parking time demand at time t.

[0027] Furthermore, in the charging scheduling extraction diagram, the charging priority of electric vehicles with a power price identifier of 1 is higher than that of electric vehicles with a power price identifier of 0;

[0028] When the electricity price identifier is the same, the charging priority of electric vehicles with a smaller number of discrete charging tasks is higher than that of electric vehicles with a larger number of discrete charging tasks.

[0029] Furthermore, the global feature h pool As shown below:

[0030]

[0031] In the formula, w j is the weight; h j is the feature of the jth node of the charging scheduling disjunctive graph; N is the number of nodes in the charging scheduling disjunctive graph.

[0032] Further, the value network looks like this:

[0033] Q(s,a)=f(h pool ) (5)

[0034] where Q(s,a) represents the future reward of state S and action a. f(·) represents the prediction function of the neural network.

[0035] Furthermore, the policy network P(a i ) is as follows:

[0036]

[0037] The technical effect of the present invention is unquestionable. The present invention proposes an orderly charging scheduling strategy for electric vehicles based on a charging priority model and a graph neural network reinforcement learning method, which significantly optimizes the operating efficiency and resource allocation of charging stations.

[0038] The beneficial effects of the present invention are as follows:

[0039] 1) By accurately quantifying the key characteristics of each vehicle (such as the expected charge, remaining power, and parking time), combined with efficient data statistics and processing processes, the overall charging demand is converted into multiple manageable subtasks. Using the charging priority calculation model, combined with historical electricity price data and real-time load information, the task priority is scientifically calculated, and an innovative charging task sorting mechanism is proposed to give priority to time-sensitive and urgent tasks, ensuring the fairness and efficiency of scheduling.

[0040] 2) Introducing the graph neural network (GNN), by constructing a disjunctive graph structure, the charging scheduling problem is transformed into a graph problem, significantly simplifying the dimensions of the state space and action space. The graph attention network (GAT) is combined to extract features from the nodes in the graph to capture the relationships between electric vehicles, thereby optimizing the accuracy and coordination of task allocation. At the same time, this method uses pulse charging to transform the charging process into multiple discrete tasks, avoiding the complexity and shortcomings of centralized training of all vehicles in traditional methods, and effectively reducing computational complexity and training costs.

[0041] 3) By combining priority with graph neural networks, an efficient and flexible scheduling model is constructed, which not only improves the operating efficiency of electric vehicle charging stations, but also provides technical support for the expansion of charging networks in the future. BRIEF DESCRIPTION OF THE DRAWINGS

[0042] Figure 1 is a flow chart of the present invention;

[0043] Figure 2 It is the interactive display of scheduling information of the present invention;

[0044] Figure 3 is the disjunctive graph construction of the present invention;

[0045] Figure 4 It is a schematic diagram of the structure of the unified multi-modal mechanism of the present invention. DETAILED DESCRIPTION

[0046] The present invention is further described below in conjunction with the embodiments, but it should not be understood that the above subject matter of the present invention is limited to the following embodiments. Without departing from the above technical ideas of the present invention, various substitutions and changes are made according to the common technical knowledge and customary means in the art, which should all be included in the protection scope of the present invention.

[0047] Embodiment 1:

[0048] See also Figures 1 to 4 , a deep reinforcement learning orderly charging method for large-scale charging stations based on disjunctive graphs, comprising the following steps:

[0049] 1) Obtain charging station data, including electricity price data and charging data;

[0050] 2) Based on the charging data, the charging tasks of electric vehicles are discretized to obtain the number of discrete charging tasks and the estimated parking time requirements;

[0051] 3) Based on the expected parking time demand and electricity price data, the electric vehicle is marked with an electricity price identifier;

[0052] 4) Construct a charging scheduling disjunctive graph based on the number of discrete charging tasks and electricity price identifiers;

[0053] 5) Use the graph attention mechanism to extract features from the disjunctive graph and obtain global features;

[0054] 6) Construct value network and strategy network;

[0055] 7) Input the global features into the value network and the policy network, use the multi-layer perceptron to predict the score of each action, and obtain the probability of each charging action;

[0056] 8) Select the charging action with the highest probability as the electric vehicle charging strategy.

[0057] The charging station data includes electricity price data and charging data.

[0058] The electricity price data is the electricity price in the past t period;

[0059] The charging data includes charging demand data and estimated departure time data;

[0060] The charging demand data is the time when the electric vehicle arrives at the electric vehicle charging station and starts charging;

[0061] The estimated departure time data is the end time of charging of the electric vehicle.

[0062] The number of discrete charging tasks is as follows:

[0063]

[0064] in, represents the charging demand of the i-th electric vehicle at time t, I t represents the set of electric vehicles charging at the charging station at time t, Δ t Indicates the time interval between each scheduled task; P i Represents the number of discrete charging tasks for the i-th electric vehicle.

[0065] The estimated parking time requirements are as follows:

[0066]

[0067] Where, d irepresents the expected parking time requirement of the i-th electric vehicle in a scheduling task; D i represents the total expected parking time demand of the i-th electric vehicle.

[0068] The electricity price identifiers are as follows:

[0069]

[0070] Among them, c t Represents the current electricity price, and T represents the current time. Charging task for time t; is the expected parking time demand at time t.

[0071] In the charging scheduling extraction diagram, the charging priority of electric vehicles with electricity price identifier 1 is higher than that of electric vehicles with electricity price identifier 0;

[0072] When the electricity price identifier is the same, the charging priority of electric vehicles with a smaller number of discrete charging tasks is higher than that of electric vehicles with a larger number of discrete charging tasks.

[0073] Global feature h pool As shown below:

[0074]

[0075] In the formula, w j is the weight; h j is the feature of the jth node of the charging scheduling disjunctive graph; N is the number of nodes in the charging scheduling disjunctive graph.

[0076] The value network looks like this:

[0077] Q(s,a)=f(h pool ) (5)

[0078] where Q(s,a) represents the future reward of state S and action a. f(·) represents the prediction function of the neural network.

[0079] Policy network P(a i ) is as follows:

[0080]

[0081] Embodiment 2:

[0082] A deep reinforcement learning orderly charging method for large-scale charging stations based on disjunctive graphs, comprising the following steps:

[0083] 1) Obtain charging station data, including electricity price data and charging data;

[0084] 2) Based on the charging data, the charging tasks of electric vehicles are discretized to obtain the number of discrete charging tasks and the estimated parking time requirements;

[0085] 3) Based on the expected parking time demand and electricity price data, the electric vehicle is marked with an electricity price identifier;

[0086] 4) Construct a charging scheduling disjunctive graph based on the number of discrete charging tasks and electricity price identifiers;

[0087] 5) Use the graph attention mechanism to extract features from the disjunctive graph and obtain global features;

[0088] 6) Construct value network and strategy network;

[0089] 7) Input the global features into the value network and the policy network, use the multi-layer perceptron to predict the score of each action, and obtain the probability of each charging action;

[0090] 8) Select the charging action with the highest probability as the electric vehicle charging strategy.

[0091] Embodiment 3:

[0092] A method for orderly charging of large-scale charging stations through deep reinforcement learning based on a disjunctive graph, the technical content of which is the same as that of Example 2, and further, the charging station data includes electricity price data and charging data.

[0093] Embodiment 4:

[0094] A method for orderly charging based on deep reinforcement learning of large-scale charging stations based on disjunctive graphs, the technical content of which is the same as any one of Embodiments 2-3, and further, the electricity price data is the electricity price of the past t period;

[0095] The charging data includes charging demand data and estimated departure time data;

[0096] The charging demand data is the time when the electric vehicle arrives at the electric vehicle charging station and starts charging;

[0097] The estimated departure time data is the end time of charging of the electric vehicle.

[0098] Embodiment 5:

[0099] A method for orderly charging based on deep reinforcement learning of large-scale charging stations based on disjunctive graphs, the technical content of which is the same as any one of Embodiments 2-4, and further, the number of discrete charging tasks is as follows:

[0100]

[0101] in, represents the charging demand of the i-th electric vehicle at time t, I trepresents the set of electric vehicles charging at the charging station at time t, Δ t Indicates the time interval between each scheduled task; P i Represents the number of discrete charging tasks for the i-th electric vehicle.

[0102] Embodiment 6:

[0103] A method for orderly charging at a large-scale charging station based on deep reinforcement learning of a disjunctive graph, the technical content of which is the same as any one of Embodiments 2-5, and further, the estimated parking time demand is as follows:

[0104]

[0105] Where, d i represents the expected parking time requirement of the i-th electric vehicle in a scheduling task; D i represents the total expected parking time demand of the i-th electric vehicle.

[0106] Embodiment 7:

[0107] A method for orderly charging based on deep reinforcement learning of large-scale charging stations based on disjunctive graphs, the technical content of which is the same as any one of Embodiments 2-6, and further, the electricity price identifier is as follows:

[0108]

[0109] Among them, c t Represents the current electricity price, and T represents the current time.

[0110] Embodiment 8:

[0111] A method for orderly charging based on deep reinforcement learning of large-scale charging stations based on a disjunctive graph, the technical content of which is the same as any one of Embodiments 2-7, and further, in the charging scheduling disjunctive graph, an electric vehicle with an electricity price identifier of 1 has a higher charging priority than an electric vehicle with an electricity price identifier of 0;

[0112] When the electricity price identifier is the same, the charging priority of electric vehicles with a smaller number of discrete charging tasks is higher than that of electric vehicles with a larger number of discrete charging tasks.

[0113] Embodiment 9:

[0114] A method for orderly charging based on deep reinforcement learning of large-scale charging stations based on disjunctive graphs, the technical content of which is the same as any one of Embodiments 2-8, further, the global feature h pool As shown below:

[0115]

[0116] In the formula, w j is the weight; h jis the feature of the jth node of the charging scheduling disjunctive graph; N is the number of nodes in the charging scheduling disjunctive graph.

[0117] Embodiment 10:

[0118] A method for orderly charging based on deep reinforcement learning of large-scale charging stations based on disjunctive graphs, the technical content of which is the same as any one of Embodiments 2-9, and further, the value network is as follows:

[0119] Q(s,a)=f(h pool ) (5)

[0120] where Q(s,a) represents the future reward of state S and action a. f(·) represents the prediction function of the neural network.

[0121] Embodiment 11:

[0122] A method for orderly charging based on deep reinforcement learning of large-scale charging stations based on disjunctive graphs, the technical content of which is the same as any one of Embodiments 2-10, further, the strategy network P(a i ) is as follows:

[0123]

[0124] Embodiment 12:

[0125] A deep reinforcement learning orderly charging method for large-scale charging stations based on disjunctive graphs adopts a data input module, a priority determination module, a scheduling disjunctive graph construction module, and a strategy and value network construction module.

[0126] The specific implementation steps are as follows:

[0127] Step 1: Data Input

[0128] The model receives data in different modalities, including:

[0129] Electricity price data: electricity prices for the past 24 hours

[0130] Charging data:

[0131] Charging demand data: Electric vehicles inform aggregators of their charging information when they arrive at an electric vehicle charging station

[0132] Estimated time of departure data: EVs inform aggregators of their charging information when they arrive at EV charging stations

[0133] Step 2:

[0134] For the charging demand of the i-th electric vehicle, we first convert the continuous charging task of charging electric vehicles into multiple charging subtasks, discretizing the continuous problem. The number of subtasks contained in the current charging task is P i It can be obtained from formula (1):

[0135]

[0136] in, represents the charging demand of the i-th electric vehicle at time t, I t represents the set of electric vehicles charging at the charging station at time t, Δ t Indicates the time interval between each scheduled task

[0137] The estimated parking time requirement for the i-th electric vehicle can also be quantified using the above idea:

[0138]

[0139] The next step is to determine the identifier. The determination of the identifier is based on our assumption that the electric vehicle always runs at the maximum power. In this way, we know that P is required according to the above modeling. i The fastest way to complete the charging is to find the fastest charging completion time to the estimated departure time. i Is there a time when the electricity price is lower than the current electricity price during the parking subtask? If so, we will give the car an identifier.

[0140]

[0141] Among them, c t represents the current electricity price, T represents the current time

[0142] Step 3: Construction of charging scheduling disjunctive graph

[0143] like Figure 3 As shown in Figure 1, in order to enhance the mutual coupling relationship in the charging process of electric vehicles, the disjunctive graph structure is introduced to transform the charging scheduling problem into a graph problem. The charging task of each electric vehicle is divided into P i subtasks, and we focus on the priority relationship between the first subtasks of each electric vehicle. Using the identifiers generated in step 2, we believe that electric vehicles without identifiers should be charged with priority over electric vehicles with identifiers at present, and electric vehicles with equivalent identifiers should be charged with fewer charging subtasks and more charging subtasks.

[0144] Step 4: Strategy and Value Network Construction Plate Module

[0145] like Figure 4 Specifically, we extract features from the disjunctive graph constructed in step 3. We use the graph attention mechanism to extract features from the disjunctive graph to achieve comprehensive fusion of information between different nodes. Specifically, we use the attention mechanism to characterize each node, and finally pool the features extracted from each node to obtain the global feature.

[0146] Get global features

[0147]

[0148] Among them, N is the number of nodes, and F is the number of features for each node. The output of this layer is a set of new node features h′

[0149]

[0150] in F′ is the new feature number. Then we perform weighted pooling on these new node features.

[0151]

[0152] Weighted pooling is performed by adding i Apply weight W i , perform weighted summation of all node features, and finally obtain a global vector h pool . Weight W i Depending on the specific task, if a node represents the first task of charging an electric vehicle, then we will give the characteristics of this node a higher weight. In addition, we will also give a higher weight to the node without an identifier of the electric vehicle charging task.

[0153] Constructing a value network

[0154] Q(s,a)=f(h pool ) (7)

[0155] where Q(s,a) represents the future reward of state S and action a. f(·) represents the prediction function of the neural network.

[0156] Constructing a policy network

[0157]

[0158] A multi-layer perceptron (MLP) is used to predict the score of each action. Finally, all possible scores are converted into the probability of each action through a softmax function.

Claims

1. A deep reinforcement learning orderly charging method for large-scale charging stations based on disjunctive graphs, characterized in that: The following steps are involved: 1) Obtain charging station data, including electricity price data and charging data; 2) Based on the charging data, the charging tasks of electric vehicles are discretized to obtain the number of discrete charging tasks and the estimated parking time requirements; 3) Based on the expected parking time demand and electricity price data, the electric vehicle is marked with an electricity price identifier; 4) Construct a charging scheduling disjunctive graph based on the number of discrete charging tasks and electricity price identifiers; 5) Use the graph attention mechanism to extract features from the disjunctive graph and obtain global features; 6) Construct value network and strategy network; 7) Input the global features into the value network and the policy network, use the multi-layer perceptron to predict the score of each action, and obtain the probability of each charging action; 8) Select the charging action with the highest probability as the electric vehicle charging strategy.

2. According to the method of deep reinforcement learning orderly charging of large-scale charging stations based on disjunctive graphs in claim 1, it is characterized in that: The charging station data includes electricity price data and charging data.

3. According to the method of deep reinforcement learning orderly charging in large-scale charging stations based on disjunctive graphs in claim 1, it is characterized in that: The electricity price data is the electricity price in the past t period; The charging data includes charging demand data and estimated departure time data; The charging demand data is the time when the electric vehicle arrives at the electric vehicle charging station and starts charging; The estimated departure time data is the end time of charging of the electric vehicle.

4. According to claim 1, a large-scale charging station deep reinforcement learning orderly charging method based on disjunctive graph is characterized in that: The number of discrete charging tasks is as follows: in, represents the charging demand of the i-th electric vehicle at time t, I t represents the set of electric vehicles charging at the charging station at time t, Δ t Indicates the time interval between each scheduled task; P i Represents the number of discrete charging tasks for the i-th electric vehicle.

5. According to claim 1, a large-scale charging station deep reinforcement learning orderly charging method based on disjunctive graph is characterized in that: The estimated parking time requirements are as follows: Where, d i represents the expected parking time requirement of the i-th electric vehicle in a charging task; D i represents the total expected parking time demand of the i-th electric vehicle.

6. According to claim 1, a large-scale charging station deep reinforcement learning orderly charging method based on disjunctive graph is characterized in that: Electricity price identifier As shown below: Among them, c t represents the current electricity price, T represents the current time; Charging task for time t; is the expected parking time demand at time t.

7. According to claim 1, a large-scale charging station deep reinforcement learning orderly charging method based on disjunctive graph is characterized in that: In the charging scheduling extraction diagram, the charging priority of electric vehicles with electricity price identifier 1 is higher than that of electric vehicles with electricity price identifier 0; When the electricity price identifier is the same, the charging priority of electric vehicles with a smaller number of discrete charging tasks is higher than that of electric vehicles with a larger number of discrete charging tasks.

8. According to claim 1, a large-scale charging station deep reinforcement learning orderly charging method based on disjunctive graph is characterized in that: Global feature h pool As shown below: In the formula, w j is the weight; h j is the feature of the jth node of the charging scheduling disjunctive graph; N is the number of nodes in the charging scheduling disjunctive graph.

9. According to claim 1, a large-scale charging station deep reinforcement learning orderly charging method based on disjunctive graph is characterized in that: The value network looks like this: Q(s,a)=f(h pool ) (5) where Q(s,a) represents the future reward of state S and action a. f(·) represents the prediction function of the neural network.

10. According to claim 1, a large-scale charging station deep reinforcement learning orderly charging method based on disjunctive graph is characterized in that: Policy network P(a i ) is as follows:

Images