Electric power-traffic collaborative balance scheduling method and system for electric vehicle charging

Abstract

The invention discloses an electric power-traffic collaborative balance scheduling method and system for electric vehicle charging, and belongs to the technical field of electric power systems, and the method comprises the steps: obtaining the travel charging demand information, charging station information and traffic network information of each electric vehicle in real time; constructing a power-traffic coupling operation network model which comprises a power network model, a traffic network model and a coupling network model; using a multi-agent multi-layer graph convolutional learning algorithm to perform power-traffic collaborative balance scheduling, using a preposed graph convolutional network to perform coupling feature representation and environmental state perception on the coupling network model to obtain the coupling feature of each node, and using a charging station as an agent to perform power-traffic collaborative balance scheduling; and inputting the coupling characteristics of the node where the charging station is located into the state space as a part of observed quantity of the intelligent agent, and outputting a charging scheduling decision of each electric vehicle through a target strategy network and a target value network in the intelligent agent. The electric vehicle charging routing behavior can be optimized, and real-time balanced scheduling of the electric power-traffic network is realized.

Description

Technical Field

[0001] The present invention belongs to the technical field of electric power systems, and in particular relates to a method and system for coordinated and balanced dispatching of electric power and transportation for charging electric vehicles. Background Art

[0002] The statements in this section merely provide background information related to the present invention and do not necessarily constitute prior art.

[0003] With the substantial increase in the proportion of electric vehicles, the degree of mutual influence and mutual restriction between urban transportation networks and distribution networks will become more and more obvious. The mobility flexibility and randomness of charging of electric vehicles pose challenges to the operation of urban transportation networks and distribution networks. Unguided concentrated charging of electric vehicles may cause the charging load to be superimposed on the conventional power load during peak hours, resulting in a "peak on peak" phenomenon, causing node voltage over-limit, increased network loss, and reduced power quality, threatening the safe and stable operation of the power grid. In the transportation network, the superposition of charging peaks and travel peaks leads to uneven distribution of traffic flow in the transportation network, and even induces traffic congestion or fault spread. The development of electric vehicle energy replenishment flexibility and battery charging technology has also enhanced the coupling and interaction potential of transportation networks and distribution networks. More and more electric vehicles choose to flexibly replenish electricity during driving, and it is possible to guide and dispatch electric vehicle charging decisions by using differentiated charging station price signals or traffic congestion information.

[0004] At present, the existing scheduling methods do not take into account the impact of electric vehicle charging station selection and driving routes on power grid operation and traffic flow distribution at the same time, and focus more on optimization problems under deterministic conditions. Prior knowledge of the power-traffic coupling environment, such as power grid status, traffic conditions, etc., is still required; secondly, even if the existing methods take into account the impact of charging scheduling decisions on the traffic side, they still only describe traffic flow distribution through static user equilibrium and semi-dynamic user equilibrium theory, which cannot reflect the real-time dynamic change characteristics of the coupled network; in addition, although the traditional dynamic user equilibrium theory can reflect certain dynamic change characteristics, it is based on the unreasonable assumption that traffic flow can be redistributed in the time and space dimensions, and is difficult to apply to real life. Summary of the invention

[0005] In order to address the deficiencies of the above-mentioned prior art, the present invention provides a method and system for coordinated and balanced scheduling of electricity and transportation for electric vehicle charging. By constructing a power-transportation coupling network operation optimization model and adopting a multi-agent multi-layer graph convolution learning algorithm to perform real-time coordinated and balanced scheduling of electricity and transportation, it can effectively characterize the spatiotemporal dynamic characteristics of traffic flow and capture complex dynamic charging environment information, thereby solving the unreasonable assumptions of dynamic user equilibrium theory in the prior art based on redistribution in the spatiotemporal dimensions, and the inability to effectively capture changes in environmental state quantities in unstable operation scenarios such as traffic congestion, charger failure and topology changes, and further optimize the routing behavior of electric vehicle charging, and achieve better real-time balanced scheduling of the power-transportation network.

[0006] In a first aspect, the present invention provides a method for coordinated and balanced scheduling of electricity and transportation for charging of electric vehicles.

[0007] A power-traffic coordinated balanced dispatching method for electric vehicle charging, comprising: Obtain real-time information on travel charging needs, charging stations and traffic network for each electric vehicle; Construct a power-transportation coupled operation network model, including a power grid model, a transportation grid model, and a coupled network model; Based on the real-time information obtained, a multi-agent multi-layer graph convolutional learning algorithm is used to perform power-traffic coordinated balanced scheduling to obtain the charging scheduling decision for each electric vehicle; Among them, the electricity-transportation coordinated balanced scheduling strategy is: use the front-end graph convolutional network to represent the coupling characteristics of the coupling network model and perceive the environmental state to obtain the coupling characteristics of each node; take the charging station as the intelligent agent, and input the coupling characteristics of the node where the charging station is located into the state space as part of the observation quantity of the intelligent agent, and output the charging scheduling decision of each electric vehicle through the target strategy network and target value network in the intelligent agent; the charging scheduling decision includes the charging station to be selected and the corresponding driving path.

[0008] A further technical solution is that the training process of the multi-agent multi-layer graph convolution learning algorithm includes: Obtain the historical travel charging demand information, charging station information and traffic network information of each electric vehicle to build a training data set; Construct a power-transportation coupled operation network model, including a power grid model, a transportation grid model, and a coupled network model; The pre-convolutional graph network is used to represent the coupling features and perceive the environment state of the coupled network model, and the pre-convolutional graph network is embedded in multi-agent reinforcement learning to form a multi-agent multi-layer graph convolutional learning algorithm. The training dataset is fed into a multi-agent multi-layer graph convolutional learning algorithm to train charging station selection and routing decisions for electric vehicle charging requests. Determine whether the algorithm converges and whether the current path traffic is in a balanced state. If so, end the training; otherwise, continue the cyclic iterative training.

[0009] In a further technical solution, the multi-agent multi-layer graph convolution learning algorithm includes a deep neural network and a model-free environment, wherein the deep neural network includes a set of front graph convolution networks and a strategy-value network of each agent, and the model-free environment is a dynamic environment of a power-transportation coupled operation network; The training dataset is fed into a multi-agent multi-layer graph convolutional learning algorithm to train charging station selection and routing decisions for electric vehicle charging requests, including: Based on the training data set, a power-transportation coupled operation network model is constructed, and the coupled network model is represented by coupling features and environmental state perception using a pre-graph convolutional network to obtain the coupling features of each node. Taking the charging station as an agent, initialize the state of each agent; For each charging request, each agent generates an action through the policy network under a given power-traffic coupling environment state, generates a reward corresponding to the current state-action through the value network, and then combines the current state and action to update the next state; The current state, action, reward, and next state of each agent form a tuple, and the entropy value is introduced to calculate the maximum cumulative discounted reward value gradient, update the policy network parameters, and update the value network parameters in a way that minimizes the joint loss. The updated parameters are assigned to the target policy network and the target value network respectively; The next training is carried out based on the updated state, and the training is continuously iterated until the set conditions are met and the decision training is completed.

[0010] In a further technical solution, the state is the state observation of each agent, including time, node number of the transportation network where the charging station is located, expected travel time to the station, t Moment m Number of electric vehicles at charging stations t+tr Moment m The number of electric vehicles, charging power, charging price in the charging station, and the coupling characteristics of the nodes where the charging station is located after the pre-graph convolution operation; The reward function of each agent includes: waiting time cost of charging station, charging time cost, charging fee, initial travel time cost, path allocation delay reward, and traffic network section congestion cost; Among them, the delay reward and traffic network congestion cost are allocated to the path. tThe sum of the congestion costs of all congested sections in the time period is taken as the congestion cost of the traffic network section. Based on the dynamic equilibrium allocation model of the traffic network, the time cost of each OD pair driving path and the traffic flow of each section are solved. The path result of dynamic traffic allocation is compared with the time cost and congestion cost of the algorithm-allocated path to obtain the path allocation delay reward, and the reward function value is backtracked and updated. The OD pair is composed of a charging request point and a target charging station, and the traffic network model is a traffic network dynamic equilibrium allocation model.

[0011] In a second aspect, the present invention provides a power-traffic coordinated balancing dispatching system for electric vehicle charging.

[0012] A power-traffic coordinated balanced dispatching system for electric vehicle charging, comprising: An information acquisition module is used to obtain the travel charging demand information, charging station information and traffic network information of each electric vehicle in real time; Model building module, used to build a power-transportation coupled operation network model, including a power grid model, a transportation grid model, and a coupled network model; The balanced scheduling module is used to perform power-traffic coordinated balanced scheduling based on the acquired real-time information using a multi-agent multi-layer graph convolutional learning algorithm to obtain the charging scheduling decision for each electric vehicle; Among them, the electricity-transportation coordinated balanced scheduling strategy is: use the front-end graph convolutional network to represent the coupling characteristics of the coupling network model and perceive the environmental state to obtain the coupling characteristics of each node; take the charging station as the intelligent agent, and input the coupling characteristics of the node where the charging station is located into the state space as part of the observation quantity of the intelligent agent, and output the charging scheduling decision of each electric vehicle through the target strategy network and target value network in the intelligent agent; the charging scheduling decision includes the charging station to be selected and the corresponding driving path.

[0013] In a third aspect, the present invention further provides an electronic device, comprising: a memory for storing executable instructions; and a processor for implementing the above-mentioned power-traffic coordinated balanced scheduling method for electric vehicle charging when executing the executable instructions stored in the memory.

[0014] In a fourth aspect, the present invention further provides a computer-readable storage medium storing executable instructions for causing a processor to execute the executable instructions to implement the above-mentioned power-traffic coordinated balanced scheduling method for electric vehicle charging.

[0015] In a fifth aspect, the present invention also provides a computer program product, which includes executable instructions stored in a computer-readable storage medium; wherein, when the processor of the electronic device reads the executable instructions from the computer-readable storage medium and executes the executable instructions, the above-mentioned power-transportation coordinated balanced scheduling method for electric vehicle charging is implemented.

[0016] One or more of the above technical solutions have the following beneficial effects: 1. The present invention provides a method and system for coordinated and balanced scheduling of electricity and transportation for electric vehicle charging. By constructing an electricity-transportation coupling network operation optimization model and adopting a multi-agent multi-layer graph convolution learning algorithm for real-time coordinated and balanced scheduling of electricity and transportation, it can effectively characterize the spatiotemporal dynamic characteristics of traffic flow and capture complex dynamic charging environment information, thereby solving the unreasonable assumptions of dynamic user equilibrium theory in the prior art based on redistribution in the spatiotemporal dimension, and the inability to effectively capture changes in environmental state quantities in unstable operation scenarios such as traffic congestion, charger failure and topology change, and further optimize the routing behavior of electric vehicle charging to achieve better real-time balanced scheduling of the electricity-transportation network.

[0017] 2. The present invention proposes a multi-agent multi-layer graph convolution reinforcement learning algorithm, through which the charging station is regarded as a multi-agent. For each electric vehicle charging request, the algorithm makes a real-time decision on the selection of the charging station and its corresponding routing path, which can effectively handle the diversity of electric vehicle charging requests and realize real-time response to charging requests.

[0018] 3. In the multi-agent multi-layer graph convolution reinforcement learning algorithm, the present invention introduces a reward function backtracking mechanism and uses the dynamic user equilibrium theory to construct a delayed reward for path allocation, thereby overcoming the unreasonable assumption of the traditional dynamic user equilibrium model that traffic can be redistributed in the spatiotemporal dimension, thereby optimizing the routing behavior of electric vehicle charging and realizing real-time balanced scheduling of the power-transportation network.

[0019] 4. The present invention introduces a front graph convolutional network in multi-agent reinforcement learning. It describes the topology and state changes of the coupled environment by constructing a coupling network feature matrix. It uses the front graph convolutional network to mine the topology and state changes of the coupling network in traffic networks, charging station congestion or failure scenarios, and further embeds it into multi-agent reinforcement learning, thereby realizing the perception of the charging environment status of electric vehicles.

[0020] Advantages of additional aspects of the present invention will be given in part in the following description, and in part will become obvious from the following description, or will be learned through practice of the present invention. BRIEF DESCRIPTION OF THE DRAWINGS

[0021] The accompanying drawings in the specification, which constitute a part of the present invention, are used to provide a further understanding of the present invention. The exemplary embodiments of the present invention and their descriptions are used to explain the present invention and do not constitute improper limitations on the present invention.

[0022] Figure 1 A framework diagram of the power-traffic coordinated balanced scheduling method for electric vehicle charging according to an embodiment of the present invention; Figure 2 This is a schematic diagram of the power-traffic coordinated balanced scheduling method for electric vehicle charging according to an embodiment of the present invention. DETAILED DESCRIPTION

[0023] It should be noted that the following detailed descriptions are exemplary only, are intended to describe specific embodiments, are intended to provide further explanation of the present invention, and are not intended to limit exemplary embodiments according to the present invention. Unless otherwise specified, all technical and scientific terms used herein have the same meanings as those of ordinary skill in the art to which the present invention belongs. In addition, it should also be understood that when the terms "comprising" and / or "including" are used in this specification, it indicates the presence of features, steps, operations, devices, components and / or combinations thereof.

[0024] Embodiment 1 This embodiment provides a power-traffic coordinated balanced scheduling method for electric vehicle charging, which specifically includes the following steps: Obtain real-time information on travel charging needs, charging stations and traffic network for each electric vehicle; Construct a power-transportation coupled operation network model, including a power grid model, a transportation grid model, and a coupled network model; Based on the real-time information obtained, a multi-agent multi-layer graph convolutional learning algorithm is used to perform power-traffic coordinated balanced scheduling to obtain the charging scheduling decision for each electric vehicle; Among them, the electricity-transportation coordinated balanced scheduling strategy is: use the front-end graph convolutional network to represent the coupling characteristics of the coupling network model and perceive the environmental state to obtain the coupling characteristics of each node; take the charging station as the intelligent agent, and input the coupling characteristics of the node where the charging station is located into the state space as part of the observation quantity of the intelligent agent, and output the charging scheduling decision of each electric vehicle through the target strategy network and target value network in the intelligent agent; the charging scheduling decision includes the charging station to be selected and the corresponding driving path.

[0025] In the above-mentioned power-traffic coordinated balanced scheduling method for electric vehicle charging proposed in this embodiment, the proposed multi-agent multi-layer graph convolution learning algorithm is mainly used for power-traffic coordinated balanced scheduling. First, the multi-agent multi-layer graph convolution learning algorithm is introduced in detail through the following content.

[0026] Specifically, based on the multi-agent multi-layer graph convolution learning algorithm, real-time coordinated balanced dispatch of electricity and transportation is carried out. The training process includes the following steps: Step S1, obtain the historical travel charging demand information, charging station information and traffic network information of each electric vehicle, and construct a training data set. The charging demand information includes the location of the electric vehicle, SOC and battery capacity, etc.; the charging station information includes the location of the charging station, the number of chargers and the operating status, etc.; the traffic network information includes the traffic network topology structure, the number of passing vehicles, etc.

[0027] Step S2: construct a power-transportation coupled operation network model, including a power grid model, a transportation grid model, and a coupled network model. Figure 1 As shown in the figure, the basic process of model construction is as follows: at the distribution network layer, the charging demand distribution and distribution network structure will affect the power flow distribution. The distribution network solves the dynamic optimal power flow distribution model according to the conventional load and charging load forecast results of each node, and transmits the marginal electricity price signal updated every hour to the charging station. The differentiated charging prices and operating conditions of the charging station will affect the user's charging decision and path selection; at the transportation network layer, in the context of advanced intelligent network environment, the OD pair demand information composed of the charging request point and the target charging station is used for real-time path selection, and the traffic dynamic equilibrium allocation model is used to perform delayed reward backtracking of path allocation, and the corresponding transportation network tends to dynamic equilibrium.

[0028] In this embodiment, at the distribution network layer, the node marginal electricity price of the node where the charging station is located is solved, and the selected power grid model is the distribution network AC optimal power flow model, and its construction process is: First, the goal of the optimal power flow operation of the distribution network is to minimize the power cost required for conventional loads and charging station charging loads, which includes the power generation cost of the generator set and the power purchase cost from the main grid, as shown in formula (1): (1) In the formula, It is a set of distribution network nodes; and Node j Secondary and primary cost coefficients of the generator set; for t Time period node j Active output of the generator set; Through the node j Active power connected to the slack bus, ρ It is the negotiated electricity price with the external grid.

[0029] Secondly, the constraint conditions are constructed. The following equations (2) to (7) are the dynamic optimal power flow constraints of the distribution network based on the second-order cone convex relaxation method, which can be expressed as: (2) (3) (4) (5) (6) (7) In the formula, 、 for t Time period node j Active and reactive loads; 、 for t Time period node j Active and reactive output of the generator set; 、 for t Time period node j Active and reactive power purchased from the external network; , for t Time period branch ( i , j )’s active and reactive currents; , For branch ( i , j )’s resistance and reactance; express t Time Line( i , j ) of the square of the current amplitude; Indicates that all headers are j The set of end nodes of the branch; and Respectively t Time period node i ,node j The square of the voltage amplitude; represents a branch set, ( i , j ) is a node i Point to Node j Directed lines of , Node i The upper and lower limits of the voltage amplitude; Indicates the line ( i , j )’s current-carrying capacity; , For Node j The upper and lower limits of the active output of the generator set; , For Node j The upper and lower limits of reactive power output of the generator set.

[0030] Furthermore, the Guribo solver is used to solve the AC optimal power flow model of the distribution network constructed based on the above equations (1) to (7). The node marginal electricity price of the distribution network is obtained by solving the dual variable of the above equation (2). The charging price of the node where the charging station is located can be further calculated based on the node marginal electricity price. For example, a service fee can be added based on the actual situation on the basis of the node marginal electricity price to calculate the charging price of the node where the charging station is located.

[0031] In this embodiment, at the traffic network layer, a dynamic user equilibrium model is constructed to quantify the path flow distribution results. w ∈ W ,Will t The time from the starting point O to the end point D k The traffic flow allocated on the feasible paths is recorded as The dynamic user equilibrium model satisfies the constraints of node traffic conservation, the correlation between path traffic and section traffic, traffic propagation constraints, and section traffic status update, as follows: (8) (9) (10) (11) (12) (13) In the formula, A r , B r Node r The collection of outflow and inflow segments; Represents a node r The new travel demand starting from express t Time flow into the road section a traffic flow; For OD w On a feasible path k Traffic on Represents the affiliation relationship between the road segment and the path. a Belong to the path k hour ,on the contrary ; for t Time outflow section a Traffic flow; express t Time section a State traffic flow on; For road section a free flow travel time; Indicates road segment a traffic capacity.

[0032] Preferably, based on the above constraints, this embodiment adopts a discretization processing method to solve the above dynamic traffic equilibrium, and uses the Frank-Wolfe algorithm to solve it.

[0033] In addition, in this embodiment, in order to further characterize the interaction within the coupled operation network, a coupled network model including the distribution network and the transportation network is established: (14) In the formula, G , G PN , G TN , G PTN They represent the coupled network model, distribution network model, transportation network model and coupling layer model respectively. Specifically, the distribution network model and transportation network model are both represented by graph theory as follows: (15) (16) In the formula, N PN , E PN , F PN They are the node set, line set and flow set of the distribution network respectively. Specifically, Represents the distribution network i nodes; Representation Node i and nodes j The connection relationship between them; For Node i and nodes j Connected line flows; N TN , E TN , F TN , T TNThey are the node set, road section set, road section flow set and road section travel time set of the traffic network; Representing the transportation network i nodes; Represents a transportation network node i and nodes j The connection relationship between them; For road section e Tij Traffic flow; t Tij For road section The travel time.

[0034] The coupling layer model is as follows: (17) In the formula, E PTN is a collection of virtual connection lines between the distribution network and the transportation network, representing the node locations of the charging stations in the two networks; F PTN A collection of charging station operating status.

[0035] Step S3: Use the front graph convolutional network to represent the coupling features and perceive the environmental state of the coupling network model, and embed the front graph convolutional network into multi-agent reinforcement learning to form a multi-agent multi-layer graph convolutional learning algorithm.

[0036] This embodiment proposes a method for coordinated and balanced dispatching of electricity and transportation based on multi-agent multi-layer graph convolution learning, which includes two parts: one is to use the front graph convolution network to realize the coupling feature representation and environmental state perception of the coupling network in step S2, and the other is to embed the front graph convolution network into multi-agent reinforcement learning to form a multi-agent multi-layer graph convolution learning algorithm. Among them, the multi-agent multi-layer graph convolution learning algorithm consists of a deep neural network and a model-free environment. The deep neural network includes a set of front graph convolution networks and the strategy-value network structure of each agent, and the model-free environment is the dynamic environment of the electricity-transportation coupling operation network, including random and uncertain charging station operation status, user dynamic charging demand, dynamic traffic flow distribution, and traffic congestion / failure, charging station failure and other emergencies.

[0037] like Figure 2 As shown in Figure 1, the above-mentioned power-transportation coordinated balanced dispatching strategy mainly includes: (a) coupling feature extraction; (b) decision-making process, specifically: First, the coupling features are extracted using the pre-convolutional graph network. Specifically, the coupling network model is abstracted into graph structure data, which consists of two parts: the adjacency matrix and the feature matrix. The adjacency matrix is ​​used to describe the internal connection relationship of the coupling network. The traffic flow and travel time of the road sections in the traffic network have a greater impact on user decisions and system operation, rather than the traffic inflow or outflow of the nodes in the traffic network. Therefore, each road section of the traffic network is modeled as a node in the adjacency matrix. Under this premise, the coupling network adjacency matrix is ​​constructed as shown in formula (18): (18) In the formula, T, P, and S represent the topological connection relationship matrix of the transportation network, distribution network, and charging station, respectively. TS and ST indicate that there is a physical connection between the charging station and the transportation network; PS and SP indicate the electrical connection between the charging station and the distribution network. Since there is no direct physical or electrical connection between the transportation network and the distribution network, the positions of TP and PT are filled with 0; if A PTN Midpoint i and nodes j If there is a connection between A PTN Matrix elements a ij =1, otherwise a ij = 0. In particular, for the internal connection matrix T of the traffic network, and the connection matrix TS and ST between the traffic network and the charging station, if the traffic section i Traffic section j Connect end to end, then a ij =1; if passing through the traffic section j Accessible charging station k , then there exists a jk =1, otherwise a jk =0.

[0038] Further, using X PTN Represents the coupled operation network feature matrix to characterize the state information of each subject in the environment. Its basic form can be expressed as X PTN,t =[T t , P t , S t ], where T t , P t , S t They are: (19) In the formula, , Represent the traffic flow and travel time sets of traffic sections respectively; , Represents the node voltage and load set of the distribution network; , are the number of vehicles arriving at the charging station and the marginal electricity price set of the node.

[0039] In acquiring graph structure knowledge ( A PTN , X PTN , t ), the graph convolution layer performs graph convolution operation to integrate the feature information of the node neighborhood and complete the reception, extraction, aggregation and transformation of the real-time information of the coupling network. The calculation process can be expressed as: (20) In the formula, Z is the output information after being processed by the graph convolution layer; , I N is the identity matrix; is the degree matrix, which indicates how many nodes are associated with the target node; X is the coupling characteristic matrix; W is the weight parameter matrix; is a non-linear activation function.

[0040] The above process actually means: First, through the graph convolution parameter matrix Node feature matrix Performing a linear transformation, we will have n Nodes m dimensional feature vector is mapped to Dimension; secondly, the converted information Y = XW Normalized graph adjacency matrix Passed to the neighboring nodes; finally, through the nonlinear activation function Get the output result matrix Z .

[0041] Through the above-mentioned graph convolution operation, the feature information of the node neighborhood can be integrated to realize the environmental state perception and feature extraction of the coupled network, and finally the coupling characteristics of each node can be extracted.

[0042] On the basis of the above, the coupling feature information of the node where the charging station is located after the previous graph convolutional network operation is passed to the downstream multi-agent graph reinforcement learning as the agent coupling state observation.

[0043] Secondly, a multi-agent multi-layer graph reinforcement learning method is executed to make charging scheduling decisions for each electric vehicle, including the charging station to be selected and its corresponding driving path. In this embodiment, the charging station is regarded as an agent, and the electric vehicle charging guidance task in the power-traffic coupling environment is described as a random game process in a multi-agent scenario. The decision-making process is: Taking the charging station as the agent, the state of each agent is initialized. On this basis, for each charging request Each agent generates actions in a given power-traffic coupling environment through a policy network and uses the value network to determine the time period. t Generate the reward corresponding to the current state-action, and then update the next state by combining the current state and action. Among them, the state, action, reward and state transition of the agent are: Status: m The state observation of an agent o m The following formula (21) is used to express it, including time, node number of the transportation network where the charging station is located, expected travel time to reach the station, t Moment m Number of electric vehicles at charging stations t + t r Moment m The number of electric vehicles, charging power, charging price in the charging station, and the coupling characteristics of the node where the charging station is located after the pre-graph convolution operation.

[0044] (twenty one) Action: Given the state of the electric-traffic coupling environment, each agent maps the state into a multi-dimensional discrete action space based on the feedback from the environment. a m = { a m,s , a m,l},in a m,s and a m,l Respectively indicate going to m The willingness to charge at the charging station and the m The number of the corresponding routing path of the charging station.

[0045] State transfer: according to the current state of the charging environment s t and joint actions a t , determine the next state.

[0046] Reward: The reward function of each agent includes: 1) waiting time cost at the charging station; 2) charging time cost; 3) charging fee; 4) initial travel time cost; 5) path allocation delay reward; 6) traffic network congestion cost.

[0047] Furthermore, with respect to the above rewards, among which: 1) The waiting time of the charging station is obtained by solving the following equation (22): (twenty two) In the formula, The waiting time required for users to arrive at the charging station; For users to reach m When a charging station is used, the remaining charging time of electric vehicles in the station is the collection; When the user reaches m The remaining charging time required for the electric vehicles waiting at the charging station; Is the first time the user arrives m The operating status of each charging station (including charging electric vehicles joining, waiting and receiving guidance); function Indicates the remaining charging time of electric vehicles when charging at charging stations in ascending order, k Indicates k elements.

[0048] 2) The charging time required for an electric vehicle from the start of charging to the completion of charging is calculated using the following equations (23) to (24): (twenty three) (twenty four) In the formula, For the i The charging time required for an electric vehicle from the start to the completion of charging; For the i Electric vehicles (EVs) arrive in m SOC value at charging station No. Indicates the power consumption per unit mileage; Indicates the time from the point where the electric vehicle requests charging to the m The route traveled by the charging station; For path the length of the included sections; For the i The battery capacity of an electric vehicle; For the i The maximum power allowed for charging an electric vehicle; For the m The charging power of the charger at each charging station.

[0049] 3) Electric vehicles in the m The charging cost required for the charging station can be calculated by the following formula (25): (25) In the formula, For electric vehicles m The charging fee required at the charging station; For the m Charging station t The charging price at the moment.

[0050] 4) No. i Electric cars go to m The travel time to the charging station can be expressed as , which can be obtained by accumulating the travel time of each section of the route.

[0051] In addition, this embodiment also introduces 5) path allocation delay reward and 6) traffic network section congestion cost. Specifically, first define the congested (or faulty) section: t Any section a Traffic flow x a Greater than its own capacity c a (The traffic capacity of each road section is a preset fixed value), it means that the road section is congested or faulty. At this time, a certain road section congestion cost is assigned to the road section (the assigned road section congestion cost is a preset value), and then t The sum of the congestion costs of all congested sections in the time period is the congestion cost of the traffic network section. Based on the dynamic equilibrium allocation model of the traffic network constructed in step S2, the time cost of the driving path of each OD pair (composed of the charging request point and the target charging station) and the traffic flow of each section are obtained. On this basis, the path result of dynamic traffic allocation is compared with the time cost and congestion cost of the algorithm-allocated path to obtain the path allocation delay reward, and the reward function value is backtracked and updated.

[0052] Furthermore, considering that although the two allocation results also have a certain impact on the time cost, the impact is generally small, this embodiment mainly obtains the delay reward by analyzing the traffic congestion cost, that is, by subtracting the congestion cost of the road section of dynamic traffic allocation from the congestion cost of the path after algorithm allocation, the path allocation delay reward is obtained. Since the dynamic traffic allocation result is an ideal state in real life, it is the optimal allocation result, that is, the congestion cost of the road section of dynamic traffic allocation must be less than or equal to the congestion cost of the algorithm allocation result, so in fact the path allocation delay reward is a negative value, and the larger the difference between the two, the smaller the corresponding delay reward value, so as to backtrack and update the reward function value.

[0053] Furthermore, according to the state, action, reward and state transfer of the above-mentioned agents, the current state, action, reward and next state of each agent are used to form a tuple, and the entropy value is introduced to calculate the maximum cumulative discounted reward value gradient, update the policy network parameters, and update the value network parameters in a way that minimizes the joint loss. The updated parameters are assigned to the target policy network and the target value network respectively, specifically: Assume that there is N agents, which include N Policy Network , the parameters are ,as well as N A value network with parameters For the agent m , calculate the advantage function: (26) In the formula, For the m The value function of each agent; Indicates m Agents except for taking actions The average value of the value function obtained when taking actions other than is the advantage function, which represents the relative size of the current action taken by the agent compared to the average level; the larger the value of the advantage function, the better the action effect. If only the action with the largest current advantage function is taken at each step, it does not mean that the cumulative reward is the largest. On this basis, we introduce To express the randomness of the agent's strategy, the larger the entropy value, the more random the agent's strategy is, which is more conducive to the agent's exploration and can avoid falling into local extreme values ​​to a certain extent. is the temperature parameter that determines the balance between entropy maximization and reward. The goal of the policy network is to use entropy to maximize the cumulative discounted reward value, as shown in formula (27): (27) The gradient of the maximized cumulative discounted reward value is calculated by formula (28) to update the policy network, which can be expressed as: (28) In addition, during the interaction between the agent and the environment, due to parameter sharing, all value networks are updated together. Here, the joint regression loss function of the value network is defined as: (29) (30) Update the value network parameters in a way that minimizes the joint loss.

[0054] Furthermore, the parameters of the policy network and the value network are continuously adjusted and updated during training, while the target network does not participate in the training process. After each update of the policy network and the value network parameters, the target network needs to be soft-updated to stabilize the algorithm training process: (31) (32) In the formula, and are the parameters of the target value network and the target strategy network respectively, Hyperparameters for controlling soft updates of the target network.

[0055] Step S4: input the training data set into a multi-agent multi-layer graph convolutional learning algorithm to train charging station selection and routing decision-making for electric vehicle charging requests.

[0056] Specifically, according to the historical data in step S1, the method in step S3 is executed to train the charging station selection and routing decision of the electric vehicle charging request. The training is continuously iterated by continuously updating the state of the intelligent body until the set conditions are met and the decision training is completed. The training process is shown in Table 1 below.

[0057] Table 1 Training process of the multi-agent multi-layer graph convolution learning algorithm in this embodiment

[0058] Step S5: determine whether the algorithm has converged and whether the current path traffic is in a balanced state. If so, end the training; otherwise, return to step S4 to continue the cyclic iterative training.

[0059] Finally, based on the real-time information on travel charging needs, charging station information, and traffic network information of each electric vehicle, the multi-agent multi-layer graph convolutional learning algorithm proposed above is used to perform power-traffic coordinated balanced scheduling, which can obtain the optimal charging scheduling decision for each electric vehicle and optimize the routing behavior of electric vehicle charging, thereby achieving real-time balanced scheduling of the power-traffic network.

[0060] Embodiment 2 This embodiment provides a power-traffic coordinated balanced dispatching system for electric vehicle charging, including: An information acquisition module is used to obtain the travel charging demand information, charging station information and traffic network information of each electric vehicle in real time; Model building module, used to build a power-transportation coupled operation network model, including a power grid model, a transportation grid model, and a coupled network model; The balanced scheduling module is used to perform power-traffic coordinated balanced scheduling based on the acquired real-time information using a multi-agent multi-layer graph convolutional learning algorithm to obtain the charging scheduling decision for each electric vehicle; Among them, the electricity-transportation coordinated balanced scheduling strategy is: use the front-end graph convolutional network to represent the coupling characteristics of the coupling network model and perceive the environmental state to obtain the coupling characteristics of each node; take the charging station as the intelligent agent, and input the coupling characteristics of the node where the charging station is located into the state space as part of the observation quantity of the intelligent agent, and output the charging scheduling decision of each electric vehicle through the target strategy network and target value network in the intelligent agent; the charging scheduling decision includes the charging station to be selected and the corresponding driving path.

[0061] Embodiment 3 This embodiment provides an electronic device, including: a memory, used to store executable instructions; and a processor, used to implement the above method provided in this embodiment when executing the executable instructions stored in the memory.

[0062] Embodiment 4 This embodiment also provides a computer-readable storage medium storing executable instructions. When the executable instructions are executed by a processor, the processor will be caused to execute the above method provided in this embodiment.

[0063] Embodiment 5 This embodiment provides a computer program product, which includes an executable instruction, which is a computer instruction; the executable instruction is stored in a computer-readable storage medium. When a processor of an electronic device reads the executable instruction from the computer-readable storage medium and the processor executes the executable instruction, the electronic device executes the above method provided in this embodiment. The steps involved in the above embodiments 2 to 5 correspond to the method embodiment 1. For the specific implementation, please refer to the relevant description part of embodiment 1. The term "computer-readable storage medium" should be understood as a single medium or multiple media including one or more instruction sets; it should also be understood to include any medium that can store, encode or carry an instruction set for execution by a processor and enable the processor to execute any method in the present invention.

[0064] Those skilled in the art should understand that the modules or steps of the present invention described above can be implemented by a general-purpose computer device, or alternatively, they can be implemented by a program code executable by a computing device, so that they can be stored in a storage device and executed by the computing device, or they can be made into individual integrated circuit modules, or multiple modules or steps therein can be made into a single integrated circuit module for implementation. The present invention is not limited to any specific combination of hardware and software.

[0065] The above description is only a preferred embodiment of the present invention. Although the specific implementation mode of the present invention is described in conjunction with the accompanying drawings, it is not a limitation of the protection scope of the present invention. Those skilled in the art should understand that on the basis of the technical solution of the present invention, various modifications or variations that can be made by those skilled in the art without creative work are still within the protection scope of the present invention.

Claims

1. A power-traffic coordinated balanced scheduling method for electric vehicle charging, characterized in that: include: Obtain real-time information on travel charging needs, charging stations and traffic network for each electric vehicle; Construct a power-transportation coupled operation network model, including a power grid model, a transportation grid model, and a coupled network model; Based on the real-time information obtained, a multi-agent multi-layer graph convolutional learning algorithm is used to perform power-traffic coordinated balanced scheduling to obtain the charging scheduling decision for each electric vehicle; Among them, the power-traffic coordinated balanced dispatching strategy is: using the front graph convolutional network to represent the coupling characteristics of the coupling network model and perceive the environmental state, and obtain the coupling characteristics of each node; Taking the charging station as the intelligent agent, the coupling characteristics of the nodes where the charging station is located are input into the state space as part of the observation quantity of the intelligent agent. Through the target strategy network and target value network in the intelligent agent, the charging scheduling decision of each electric vehicle is output; the charging scheduling decision includes the charging station to be selected and the corresponding driving path.

2. The method for coordinated and balanced dispatching of electric vehicles for charging of electric vehicles according to claim 1, characterized in that: The training process of the multi-agent multi-layer graph convolution learning algorithm includes: Obtain the historical travel charging demand information, charging station information and traffic network information of each electric vehicle to build a training data set; Construct a power-transportation coupled operation network model, including a power grid model, a transportation grid model, and a coupled network model; The pre-convolutional graph network is used to represent the coupling features and perceive the environment state of the coupled network model, and the pre-convolutional graph network is embedded in multi-agent reinforcement learning to form a multi-agent multi-layer graph convolutional learning algorithm. The training dataset is fed into a multi-agent multi-layer graph convolutional learning algorithm to train charging station selection and routing decisions for electric vehicle charging requests. Determine whether the algorithm converges and whether the current path traffic is in a balanced state. If so, end the training; otherwise, continue the cyclic iterative training.

3. The method for coordinated and balanced dispatching of electric vehicles for charging of electric vehicles according to claim 1, characterized in that: The front-end graph convolutional network is used to represent the coupling features of the coupled network model and perceive the environmental state, including: Abstracting the coupling network model into graph structure data; the graph structure includes an adjacency matrix and a feature matrix, the adjacency matrix is ​​used to describe the connection relationship within the coupling network, and the feature matrix is ​​used to characterize the state information of each subject in the environment; The acquired graph structure data is input into the front graph convolution network, and the feature information of the node neighborhood is integrated through the graph convolution operation to obtain the coupling features of each node.

4. The method for coordinated and balanced dispatching of electric vehicles for charging of electric vehicles according to claim 2, characterized in that: The multi-agent multi-layer graph convolution learning algorithm includes a deep neural network and a model-free environment, wherein the deep neural network includes a set of front graph convolution networks and a strategy-value network of each agent, and the model-free environment is a dynamic environment of a power-transportation coupled operation network; The training dataset is fed into a multi-agent multi-layer graph convolutional learning algorithm to train charging station selection and routing decisions for electric vehicle charging requests, including: Based on the training data set, a power-transportation coupled operation network model is constructed, and the coupled network model is represented by coupling features and environmental state perception using a pre-graph convolutional network to obtain the coupling features of each node. Taking the charging station as an agent, initialize the state of each agent; For each charging request, each agent generates an action through the policy network under a given power-traffic coupling environment state, generates a reward corresponding to the current state-action through the value network, and then combines the current state and action to update the next state; The current state, action, reward, and next state of each agent form a tuple, and the entropy value is introduced to calculate the maximum cumulative discounted reward value gradient, update the policy network parameters, and update the value network parameters in a way that minimizes the joint loss. The updated parameters are assigned to the target policy network and the target value network respectively; The next training is carried out based on the updated state, and the training is continuously iterated until the set conditions are met and the decision training is completed.

5. The method for coordinated and balanced dispatching of electric vehicles for charging of electric vehicles according to claim 4, characterized in that: The state is the state observation of each agent, including time, node number of the transportation network where the charging station is located, expected travel time to the station, t Moment m Number of electric vehicles at charging stations t+tr Moment m The number of electric vehicles, charging power, charging price in the charging station, and the coupling characteristics of the nodes where the charging station is located after the pre-graph convolution operation; The acquisition of the charging price includes: With the goal of minimizing the electricity cost required for conventional loads and charging station charging loads, combined with dynamic optimal power flow constraints, the AC optimal power flow model of the distribution network, namely the power grid model, is constructed; Based on the AC optimal power flow model of the distribution network, the node marginal electricity price of the distribution network is solved, and then the charging price of the node where the charging pile is located is calculated.

6. The method for coordinated and balanced dispatching of electric vehicles for charging of electric vehicles according to claim 4, characterized in that: The reward function of each agent includes: waiting time cost of charging station, charging time cost, charging fee, initial travel time cost, path allocation delay reward, and traffic network section congestion cost; Among them, for the delayed reward of path allocation and the congestion cost of traffic network sections, the sum of the congestion costs of all congested sections in time period t is used as the congestion cost of traffic network sections, and based on the dynamic equilibrium allocation model of the traffic network, the time cost of each OD pair of driving paths and the traffic flow of each section are solved, and the path results of dynamic traffic allocation are compared with the time cost and congestion cost of the algorithm-allocated path to obtain the delayed reward of path allocation, and the reward function value is backtracked and updated; The OD pair is composed of a charging request point and a target charging station, and the traffic network model is a traffic network dynamic equilibrium allocation model.

7. A power-traffic coordinated balanced dispatching system for electric vehicle charging, characterized in that: include: An information acquisition module is used to obtain the travel charging demand information, charging station information and traffic network information of each electric vehicle in real time; Model building module, used to build a power-transportation coupled operation network model, including a power grid model, a transportation grid model, and a coupled network model; The balanced scheduling module is used to perform power-traffic coordinated balanced scheduling based on the acquired real-time information using a multi-agent multi-layer graph convolutional learning algorithm to obtain the charging scheduling decision for each electric vehicle; Among them, the power-traffic coordinated balanced dispatching strategy is: using the front graph convolutional network to represent the coupling characteristics of the coupling network model and perceive the environmental state, and obtain the coupling characteristics of each node; Taking the charging station as the intelligent agent, the coupling characteristics of the nodes where the charging station is located are input into the state space as part of the observation quantity of the intelligent agent. Through the target strategy network and target value network in the intelligent agent, the charging scheduling decision of each electric vehicle is output; the charging scheduling decision includes the charging station to be selected and the corresponding driving path.

8. An electronic device, characterized in that: include: A memory for storing executable instructions; The processor is used to implement the power-transportation coordinated balanced scheduling method for electric vehicle charging as described in any one of claims 1 to 6 when executing the executable instructions stored in the memory.

9. A computer-readable storage medium, characterized in that: Executable instructions are stored, which are used to cause the processor to execute the executable instructions to implement the power-traffic coordinated balanced scheduling method for electric vehicle charging as described in any one of claims 1-6.

10. A computer program product, characterized in that The computer program product includes executable instructions stored in a computer-readable storage medium; When the processor of the electronic device reads the executable instructions from the computer-readable storage medium and executes the executable instructions, the power-transportation coordinated balanced scheduling method for electric vehicle charging as described in any one of claims 1 to 6 is implemented.

Images