An electric vehicle charging navigation method based on electricity price prediction and electronic equipment

Abstract

The application discloses a kind of electric vehicle charging navigation method and electronic equipment based on electricity price prediction, method includes: according to distribution network, traffic network, charging station and electric vehicle model, establish power-traffic coupling network model;According to power-traffic coupling network model, establish the charging navigation economic optimization model of electric vehicle, determine the variable and constraint of model;According to variable and constraint design in multi-agent deep reinforcement learning state variable set, action variable set and reward function;Multi-agent SAC training network structure considering electricity price prediction information is constructed, the parameters of action network and evaluation network of SAC training network structure are set;Through interactive power-traffic coupling network model, train agent to learn to make optimal decision under actual traffic condition and dynamic electricity price, to maximize reward function, realize the cost optimization of electric vehicle user in driving and charging process.The application can effectively reduce electric vehicle economic driving and charging cost.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of power system and traffic network collaborative optimization, and particularly relates to an electric vehicle charging navigation method based on electricity price prediction and an electronic device. BACKGROUND

[0002] With the continuous popularity of electric vehicles, the number of charging facilities is increasing, continuously meeting the expanding market demand. This trend greatly increases the proportion of power load in the traffic network, significantly enhancing the interaction and synergy between the power distribution network and the traffic network. However, large-scale electric vehicle centralized charging will bring adverse effects such as overload to the power distribution network, threatening the safe operation of the power grid and causing an increase in additional costs such as long waiting time.

[0003] The power-traffic coupled network is an inevitable trend of the development of electric vehicles, and multi-agent deep reinforcement learning, as a high-level extension of single-agent deep reinforcement learning, enables a single agent to maximize its reward through distributed decision-making, enhanced adaptability, and collaborative intelligence, thereby effectively solving complex and large-scale problems. At present, the charging navigation technology based on multi-agent deep reinforcement learning still needs to be developed in the application of the power-traffic coupled network, especially in optimizing potential future costs, so there is an urgent need for an efficient and accurate method to provide electric vehicle users with the optimal path and the optimal charging station selection. SUMMARY

[0004] To at least partially solve one of the technical problems existing in the prior art, the purpose of the present application is to provide an electric vehicle charging navigation method based on electricity price prediction, an electronic device and a medium.

[0005] The first technical solution adopted by the present application is:

[0006] An electric vehicle charging navigation method based on electricity price prediction, comprising the following steps:

[0007] According to the power distribution network, the traffic network, the charging station and the electric vehicle model, a power-traffic coupled network model is established;

[0008] According to the power-traffic coupled network model, a charging navigation economic optimization model of the electric vehicle is established, and the variables and constraints of the model are determined; according to the variables and constraints, the state variable set X=(X1, X2, …, Xm) of the multi-agent deep reinforcement learning, the action variable set A=(a1, a2, …, am) and the reward function R=(R1, R2, …, Rm) are designed, wherein m represents the number of agents; m m m

[0009] ​​​Construct a multi-agent SAC training network structure that considers electricity price forecast information, and set the parameters of the action network and evaluation network of the SAC training network structure;

[0010] By using an interactive power-transportation coupled network model, the agent is trained to make optimal decisions under actual traffic conditions and dynamic electricity prices to maximize the reward function, thereby optimizing the cost for electric vehicle users during driving and charging, and achieving an economical operating level.

[0011] Furthermore, the establishment of a power-transportation coupled network model based on the power distribution network, transportation network, charging station, and electric vehicle model includes:

[0012] The optimal power flow model of the distribution network is shown in equation (1) below:

[0013]

[0014] Among them, the constraints for configuring the optimal power flow model of the distribution network include power balance constraints and voltage constraints;

[0015] The power balance constraints are shown in equations (2)-(10):

[0016]

[0017] In the formula, c g Indicates the cost of purchasing electricity. This represents the active power output of the node generator. Indicates the power injected from other networks, a 0,i ,a 1,i ρ represents their correlation coefficient; V i It is the AC voltage at node i. V i , V i upper and lower boundaries, These represent the generator output power. upper and lower bounds, S ij This represents the branch transmission power between node i and node j. Y represents the load of node i. ij It is the line admittance matrix. This represents the maximum value of the transmitted power in the branch. This indicates the maximum value of the phase angle difference between node i and node j. U ij The imaginary part;

[0018] Since constraint (2) is a nonconvex nonlinear constraint, the second-order cone relaxation method is used to transform constraint (2) into the standard second-order cone inequality form:

[0019]

[0020] |U ij | 2 ≤U ii U jj (14)

[0021]

[0022] By extracting the dual variables of the active power balance equation, the local marginal electricity price λ is calculated. i ;

[0023] Transportation network model As shown in equation (16):

[0024]

[0025] In the formula, ε represents the set of traffic nodes and the set of edges, respectively; w l w2 represents the edge weights between nodes v and u with respect to distance and branch travel time, respectively. Considering actual traffic congestion, the actual travel time of the branch can be calculated using the BPR model, which is shown in equation (17) below:

[0026]

[0027] In the formula, x (v,u) and c (v,u) Let (v, u) represent the traffic flow and traffic capacity of edge (v, u), respectively.

[0028] The charging station model is shown in equation (18) below:

[0029]

[0030] In the formula, L represents the set of charging stations; S(k,t) represents the state matrix of the k-th charging station; and This indicates that the k-th charging station corresponds to a node and a location in the power distribution network and the transportation network, respectively; This represents the charging demand of the k-th charging station; Indicates the real-time charging electricity price; This indicates the waiting time required for an electric vehicle to charge.

[0031] The electric vehicle model is shown in equation (19) below:

[0032]

[0033] In the formula, t represents the time step of the charging process; Represents the battery status of the i-th electric vehicle; v represents the location node of the electric vehicle in the transportation network; E ini and E max These represent the initial state of charge and the energy capacity, respectively.

[0034] The prediction network model for Long Short-Term Memory (LSTM) networks is as follows:

[0035] f t =σ(W fx x t +W fh h t-1 +b f (20)

[0036] i t =σ(W ix x t +W ih h t-1 +b i ) (twenty one)

[0037] g t =φ(W gx x t +W gh h t-1 +b g ) (twenty two)

[0038] o t =σ(W ox x t +W oh h t-1 +b o ) (twenty three)

[0039] s t =g t ⊙i t +s t-1 ⊙f t (twenty four)

[0040] h t =φ(s) t )⊙o t (25)

[0041] In the formula, {x1,x2,…,x T-1 ,x T} represents the sequence input; x t g represents the k-dimensional subsequence input at time step t; t i t ,f t ,o tThese represent candidate values, input gate, forget gate, and output gate, respectively; σ and φ represent the sigmoid and tanh activation functions, respectively; ⊙ represents element-wise multiplication; s t h represents the update state of a memory cell. t This indicates the hidden state at the current moment.

[0042] Furthermore, the expression for the economic optimization model of the electric vehicle charging navigation is as follows:

[0043]

[0044] The constraints of the electric vehicle charging navigation economic optimization model include route operation constraints and charging station selection constraints.

[0045] The constraints of the economic optimization model for configuring electric vehicle charging navigation are shown in equations (27)-(32):

[0046]

[0047] In the formula, and d a These are the actual driving time and distance; and These are the charging capacity and the waiting time, respectively; c tr and c ch These are the driving and charging costs, respectively; if the electric vehicle travels along edge a, then the binary variable x... a The value is 1 if the charging station k is chosen, and 0 otherwise; if the charging station k is chosen, then the binary variable y... k p is 1 if it is positive and 0 otherwise; tr and q t These are the cost conversion weighting factors for distance and time, respectively; E end Indicates the remaining battery power upon arrival at the charging station; o ini This indicates the initial position of the electric vehicle.

[0048] Furthermore, the state variables are designed as follows:

[0049] In the electric vehicle charging navigation task within a power-transportation coupled network, the state should be selected based on environmental indicators that best reflect the current operating status of the system and are directly related to the actions. These indicators include road node v and the electric vehicle's battery status. Electricity price information received by electric vehicles from various charging stations The electricity price prediction information for each charging station is predicted by the network output. Time step t,

[0050] The observed state variables are represented by equation (33):

[0051]

[0052] Furthermore, the action variables are selected as follows:

[0053] The action variable should be one that directly affects the reward and state. Therefore, the current state is input, and the charging station k selected by the agent is used as the action variable:

[0054] a i =k,k∈L (34)

[0055] In the formula, L represents the set of charging stations.

[0056] Furthermore, the reward function is as follows:

[0057] The agent's optimization objective is to find the economically optimal solution within the feasible domain over time steps. Therefore, the reward settings are divided into rewards for reaching the destination charging station and rewards for traveling on the road:

[0058]

[0059] In the formula, E max This indicates the maximum battery capacity of the electric vehicle.

[0060] Furthermore, the action network selects action variables based on state variables, and the policy function of the action network is updated using a stochastic policy gradient algorithm.

[0061]

[0062] In the formula, This is the output value of the Q-network; logπ(a) i |o i ;θ i ) represents an entropy term, and α is the weight value of the entropy term; Perform gradient calculation for the action network.

[0063] Furthermore, the target value Q of the Q-network target (o i ,a i ) by Critic target network V Tar (s′ i Composed of components, according to the Bellman equation, the objective value of the Q-network is as follows:

[0064]

[0065] Where γ represents the discount factor.

[0066] Furthermore, the multi-agent SAC training network structure that considers electricity price prediction information simultaneously introduces a Q-network and an evaluation network. The target value of the Q-network is calculated using the target value of the evaluation network, and then the Q-network is used... target (o i ,a i Update the parameters φ of the Q network. i To minimize the loss function Use V Tar Update the parameters of the evaluation network To minimize the loss function and As shown in the following formula:

[0067]

[0068] The system first obtains action variables based on state variables through a policy network, and then sets the system state X... i and system action variable a i As input, it is mapped through a Q-network to obtain... The network update parameters are obtained by backpropagation algorithm using the action network loss function of equation (36):

[0069]

[0070] In the formula, This is the loss function for the action network.

[0071] The second technical solution adopted in this invention is:

[0072] An electronic device includes a processor and a memory, wherein the memory stores at least one instruction, at least one program, a code set, or an instruction set, and the at least one instruction, the at least one program, the code set, or the instruction set is loaded and executed by the processor to implement an electric vehicle charging navigation method based on electricity price prediction as described above.

[0073] The third technical solution adopted in this invention is:

[0074] A computer-readable storage medium storing at least one instruction, at least one program, a code set, or an instruction set, wherein the at least one instruction, the at least one program, the code set, or the instruction set is loaded and executed by a processor to implement an electric vehicle charging navigation method based on electricity price prediction as described above.

[0075] The fourth technical solution adopted in this invention is:

[0076] A computer program product or computer program includes computer instructions stored in a computer-readable storage medium. A processor of a computer device can read the computer instructions from the computer-readable storage medium and execute the computer instructions, causing the computer device to perform the aforementioned electric vehicle charging navigation method based on electricity price prediction.

[0077] The beneficial effects of this invention include:

[0078] (1) This invention provides electric vehicle users with the optimal route and the optimal charging station selection by establishing a power-transportation coupled network model and using electric vehicle charging navigation technology with integrated electricity price prediction and multi-agent deep reinforcement learning. This helps to improve the travel efficiency of electric vehicles and reduce driving and charging costs.

[0079] (2) The intelligent agent in this invention enables the system to make automated and intelligent decisions. By interacting with the power-transportation coupled network environment, the intelligent agent continuously learns and improves its decisions to adapt to different operating conditions, which reduces the need for human intervention and improves the autonomy of the system.

[0080] (3) Through intelligent decision-making, the system can learn and optimize route and charging station selection autonomously to ensure efficient operation of the system and reduce total cost. Attached Figure Description

[0081] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the following description is provided with accompanying drawings of the relevant technical solutions in the embodiments of the present invention or the prior art. It should be understood that the accompanying drawings described below are only for the purpose of clearly illustrating some embodiments of the technical solutions of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0082] Figure 1 This is a flowchart of the steps of the electric vehicle charging navigation method based on electricity price prediction in an embodiment of the present invention;

[0083] Figure 2 This is a schematic diagram of the multi-agent SAC training network structure in an embodiment of the present invention. Detailed Implementation

[0084] The embodiments of this application are described in detail below. Examples of these embodiments are shown in the accompanying drawings, wherein the same or similar reference numerals denote the same or similar elements or elements having the same or similar functions throughout. The embodiments described below with reference to the accompanying drawings are exemplary and are only used to explain this application, and should not be construed as limiting this application. The step numbers in the following embodiments are set only for ease of explanation, and there is no limitation on the order between the steps. The execution order of each step in the embodiments can be adaptively adjusted according to the understanding of those skilled in the art.

[0085] The terminology used in the embodiments of this application is for the purpose of describing specific embodiments only and is not intended to limit the embodiments of this application. The singular forms "a," "described," and "the" used in the embodiments of this application and the appended claims are also intended to include the plural forms, unless the context clearly indicates otherwise. Furthermore, unless otherwise expressly limited, terms such as "set," "install," and "connect" should be interpreted broadly, and those skilled in the art can reasonably determine the specific meaning of the above terms in this invention in conjunction with the specific content of the technical solution.

[0086] In the description of this application, it should be understood that the orientation descriptions, such as up, down, front, back, left, right, etc., indicate the orientation or positional relationship based on the orientation or positional relationship shown in the accompanying drawings. They are only for the convenience of describing this application and simplifying the description, and do not indicate or imply that the device or element referred to must have a specific orientation, or be constructed and operated in a specific orientation. Therefore, they should not be construed as limitations on this application.

[0087] In the description of this application, "several" means one or more, "more than" means two or more, "greater than," "less than," and "exceeding" are understood to exclude the stated number, while "above," "below," and "within" are understood to include the stated number. The use of "first" and "second" in the description is merely for distinguishing technical features and should not be construed as indicating or implying relative importance, or implicitly indicating the number of indicated technical features, or implicitly indicating the order of the indicated technical features.

[0088] In the description of this application, "and / or" describes the relationship between related objects, indicating that there can be three relationships. For example, A and / or B can represent: A alone, A and B simultaneously, or B alone. The character " / " generally indicates that the related objects before and after it are in an "or" relationship.

[0089] Terminology Explanation:

[0090] SAC: an abbreviation for Soft Actor-Critic; SAC is an algorithm based on the maximum entropy reinforcement learning framework. By introducing entropy maximization during policy optimization, it balances exploration and exploitation, enabling agents to explore and learn more effectively in complex environments.

[0091] To address existing technical problems, this invention provides an electric vehicle charging navigation method for power-transport coupled networks that integrates electricity price prediction and multi-agent deep reinforcement learning. The aim is to reduce the economical driving and charging costs of electric vehicles through data intelligence. The key technical solution is the rational construction of a multi-agent deep reinforcement learning scheduling framework adapted to electric vehicle charging navigation in power-transport coupled networks. This includes selecting state variables, action variables, design constraints, and reward functions to represent the operation of electric vehicles in the power-transport coupled network. Through interaction with real-time data, the system can adapt to constantly changing environmental conditions and user needs, coping with fluctuations in electricity prices, traffic conditions, and charging demand, thereby optimizing electric vehicle scheduling and improving the overall network efficiency. The application areas of this invention cover multiple fields such as electric vehicle charging navigation, prediction networks, multi-agent reinforcement learning, and scheduling and control of power-transport coupled networks. It aims to improve the stable and economical operation of power-transport coupled networks.

[0092] Example 1

[0093] like Figure 1 As shown, this embodiment provides an electric vehicle charging navigation method integrating electricity price prediction and multi-agent deep reinforcement learning, including the following steps:

[0094] Step S1: Establish a mechanism model for the coupling of the power and transportation networks, including models of the distribution network, transportation network, charging stations, and electric vehicles;

[0095] Step S2: Based on the power-transportation coupled network model, establish an economic optimization model for charging navigation of electric vehicles, and clarify the system variables and constraints; construct a reinforcement learning training model framework based on the variables, constraints and indicators, that is, design the set of state variables, the set of action variables and the reward function in multi-agent deep reinforcement learning.

[0096] Step S3: Construct a multi-agent SAC training network structure that considers electricity price prediction information. Set the number of training rounds, number of network layers, network size, and sequence length of the electricity price prediction network. Set the hidden layer size, number of hidden layers, activation function, learning rate, entropy coefficient, batch size, discount factor, and replay buffer size of the action network and evaluation network of the SAC training network structure.

[0097] Step S4: Train the agent by using a power-transportation coupled network environment model to teach it how to make optimal decisions under different conditions to maximize the reward function, thereby optimizing the cost for electric vehicle users during driving and charging and achieving an economical operating level.

[0098] As one implementation method, in step S1, the model is established as follows:

[0099] The optimal power flow model of the distribution network is shown in equations (1)-(10):

[0100]

[0101] Among them, the constraints for configuring the optimal power flow model of the distribution network mainly include power balance constraints and voltage constraints.

[0102] The power balance constraint is shown in equation (5):

[0103]

[0104] Among them, c g [$] represents the cost of electricity purchase. This represents the active power output of the node generator. Indicates the power injected from other networks, a 0,i ,a 1,i ρ[$ / kWh] represents their correlation coefficient; V i [kV] is the AC voltage at node i, S ij [kVA] represents the branch transmission power between node i and node j. Y represents the load of node i. ij [S] is the line admittance matrix. U ij The imaginary part.

[0105] More preferably, since constraint (2) is a non-convex nonlinear constraint, constraint (2) can be made convex instead of being treated according to the principles of semidefinite programming. Using the second-order cone relaxation method, constraint (2) can be transformed into the standard second-order cone inequality form:

[0106]

[0107] |U ij | 2 ≤U ii U jj (14)

[0108]

[0109] More preferably, the local marginal electricity price λ can be calculated by extracting the dual variables of the active power balance equation. i [$ / kWh].

[0110] Transportation network model As shown in equation (16):

[0111]

[0112] in, ε represents the set of traffic nodes and the set of edges, respectively; w l [km] and w2[h] represent the edge weights between nodes v and u with respect to distance and branch travel time, respectively. The actual branch travel time can be calculated using the BPR model. The BPR model is shown in equation (17) below:

[0113]

[0114] In the formula, x (v,u) and c (v,u) Let (v, u) represent the traffic flow and traffic capacity of edge (v, u), respectively.

[0115] The charging station model is shown in equation (18) below:

[0116]

[0117] Where L represents the set of charging stations; S(k,t) represents the state matrix of the k-th charging station; and This indicates that the k-th charging station corresponds to a node and a location in the power distribution network and the transportation network, respectively; This represents the charging demand of the k-th charging station; Indicates the real-time charging electricity price; This indicates the waiting time required for an electric vehicle to charge.

[0118] The electric vehicle model is shown in equation (19) below:

[0119]

[0120] Where t[h] represents the time step of the charging process; Represents the battery status of the i-th electric vehicle; v represents the location node of the electric vehicle in the transportation network; E ini [kWh] and E max [kWh] represents the initial state of charge and the energy capacity, respectively.

[0121] The prediction network model for Long Short-Term Memory (LSTM) networks is as follows:

[0122] ft =σ(W fx x t +W fh h t-1 +b f (20)

[0123] i t =σ(W ix x t +W ih h t-1 +b i ) (twenty one)

[0124] g t =φ(W gx x t +W gh h t-1 +b g ) (twenty two)

[0125] o t =σ(W ox x t +W oh h t-1 +b o ) (twenty three)

[0126] s t =g t ⊙i t +s t-1 ⊙f t (twenty four)

[0127] h t =φ(s) t )⊙o t (25)

[0128] Where, {x1,x2,…,x T-1 ,x T} represents the sequence input; x t g represents the k-dimensional subsequence input at time step t; t i t ,f t ,o t σ and φ represent candidate values, input gate, forget gate, and output gate, respectively; σ and φ represent sigmoid and tanh activation functions, respectively; ⊙ represents element-wise multiplication.

[0129] As one implementation method, the economic optimization model for the electric vehicle charging navigation is as follows:

[0130]

[0131] The constraints of the economic optimization model for configuring electric vehicle charging navigation include route operation constraints and charging station selection constraints, which are shown in equations (27)-(32) below:

[0132]

[0133] in, and d a [km] represents the actual travel time and distance; and These are the charging capacity and the waiting time, respectively; c tr [$] and c ch [$] represent the driving and charging costs, respectively; p tr [km / h] and q t [$ / h] are the cost conversion weighting factors for distance and time, respectively; E end [kWh] represents the remaining power after arriving at the charging station.

[0134] The design of a deep reinforcement learning framework involves four key elements: state, action, reward, and environment. State represents the situation the agent is in, action is the operation the agent can perform, reward is immediate feedback, and environment is the external world. In deep reinforcement learning, the agent learns how to formulate strategies to maximize cumulative rewards by observing states, selecting actions, receiving rewards, and interacting with the environment.

[0135] 1) State Variable Design. In the electric vehicle charging navigation task within a power-transportation coupled network, the state should be selected based on environmental indicators that best reflect the current operating status of the system and are directly related to the actions. This paper selects road node v and the electric vehicle's charge state. Electricity price information received by electric vehicles from various charging stations The electricity price prediction information for each charging station is predicted by the network output. Time step t

[0136] The observation of state variables can be expressed as equation (33):

[0137]

[0138] 2) Action Variable Design. Action variables should be selected that directly affect the reward and state. Therefore, given the current state, the charging station k selected by the agent is used as the action variable:

[0139] a i =k,k∈L (34)

[0140] 3) Reward Function Design. The optimization objective of the agent is to find the economically optimal solution in the feasible region over time steps. Therefore, the reward is set as follows (13), which is divided into the reward for reaching the destination charging station and the reward for driving on the road.

[0141]

[0142] As a specific implementation method, in step S3, a multi-agent SAC training network structure that considers electricity price forecast information is constructed as follows: Figure 2 As shown in Table 1, the training parameters of the multi-agent SAC training network structure are also shown.

[0143] exist Figure 2 The text describes the set of states of the power-transportation coupled network at time step t. These states are mapped to the energy system decision variable a through a policy function. i The agent interacts with the environment in the next moment, generating a new state s′. i And the economic cost required for the coupled network in the next moment -r i This information is stored in an experience pool and randomly selected as training samples during network training.

[0144] Figure 2 The action network used in this algorithm selects action variables based on state variables, and its policy function is updated using a stochastic policy gradient algorithm.

[0145]

[0146] in, This is the output value of the Q-network; logπ(a) i |o i ;θ i ) represents an entropy term, and α is the weight value of the entropy term; Perform gradient calculation for the action network.

[0147] The target value of the Q network is Q. target (o i ,a i ) by Critic target network V Tar (s′ i Composed of components, according to the Bellman equation, the objective value of the Q-network is as follows:

[0148]

[0149]

[0150] Where γ represents the discount factor.

[0151] The multi-agent SAC training network structure, which considers electricity price forecasting information, incorporates both a Q-network and an evaluation network. The target value of the Q-network is calculated using the target value of the evaluation network, and then the Q-network's target value is used... target (o i ,a i Update the parameters φ of the Q network. i To minimize the loss function Use V Tar Update the parameters of the evaluation network To minimize the loss function and As shown in the following formula:

[0152]

[0153] The system first obtains action variables based on state variables through a policy network, and then sets the system state X... i and system action variable a i As input, it is mapped through a Q-network to obtain... The network update parameters are obtained by backpropagation algorithm using the action network loss function of equation (36):

[0154]

[0155] in, This is the loss function for the action network.

[0156] The objective function employs a soft update approach, progressively updating the parameters of the policy network to the target policy network, and simultaneously progressively updating the parameters of the evaluation network to the target evaluation network. A learning rate τ is introduced, and the update strategy is as follows:

[0157]

[0158] Table 1. Multi-Agent SAC Training Network Structure and Training Parameter Settings

[0159]

[0160]

[0161] For example, an agent is trained using the Python platform, with 3000 training epochs. The agent is saved after every 1000 training epochs. The final stopping condition is reaching the maximum number of training epochs. The experiment consists of two steps: First, the agent is trained to avoid network constraints and achieve optimal decision-making; after convergence, the updated network parameters are saved. Second, the agent's decision-making performance is tested on a test dataset after loading the network parameters.

[0162] Example 2

[0163] This invention also provides an electronic device, which includes a processor and a memory. The memory stores at least one instruction, at least one program, a code set, or an instruction set. The at least one instruction, the at least one program, the code set, or the instruction set is loaded and executed by the processor to achieve the following: Figure 1 This paper presents a method for electric vehicle charging navigation based on electricity price prediction.

[0164] It is understood that the memory may include random access memory (RAM) or read-only memory. Optionally, the memory may include non-transitory computer-readable storage medium. The memory can be used to store instructions, programs, code, code sets, or instruction sets. The memory may include a stored program area and a stored data area, wherein the stored program area may store instructions for implementing an operating system, instructions for at least one function, instructions for implementing the various method embodiments described above, etc.; the stored data area may store data created according to the use of the server, etc.

[0165] A processor may include one or more processing cores. The processor connects to various parts of the server via various interfaces and lines, executing instructions, programs, code sets, or instruction sets stored in memory, and accessing data stored in memory to perform various server functions and process data. Optionally, the processor may be implemented using at least one of the following hardware forms: Digital Signal Processing (DSP), Field-Programmable Gate Array (FPGA), and Programmable Logic Array (PLA). The processor may integrate one or more of the following: Central Processing Unit (CPU) and Modem. The CPU primarily handles the operating system and applications; the modem handles wireless communication. It is understood that the modem may also be implemented as a separate chip without being integrated into the processor.

[0166] Since this electronic device is the electronic device corresponding to the electric vehicle charging navigation method based on electricity price prediction in the embodiments of the present invention, and the principle of solving the problem by this electronic device is similar to that of the method, the implementation of this electronic device can refer to the implementation process of the above method embodiments, and the repeated parts will not be described again.

[0167] Example 3

[0168] This invention also provides a computer-readable storage medium storing at least one instruction, at least one program, a code set, or an instruction set, wherein the at least one instruction, the at least one program, the code set, or the instruction set is loaded and executed by a processor to achieve the following: Figure 1 This paper presents a method for electric vehicle charging navigation based on electricity price prediction.

[0169] Those skilled in the art will understand that all or part of the steps in the various methods of the above embodiments can be implemented by a program instructing related hardware. The program can be stored in a computer-readable storage medium, including read-only memory (ROM), random access memory (RAM), programmable read-only memory (PROM), erasable programmable read-only memory (EPROM), one-time programmable read-only memory (OTPROM), electrically-Erasable Programmable Read-Only Memory (EEPROM), compact disc read-only memory (CD-ROM) or other optical disc storage, disk storage, magnetic tape storage, or any other computer-readable medium capable of carrying or storing data.

[0170] Since the storage medium is the storage medium corresponding to the electric vehicle charging navigation method based on electricity price prediction in the embodiment of the present invention, and the principle of the storage medium in solving the problem is similar to that of the method, the implementation of the storage medium can refer to the implementation process of the above method embodiment, and the repeated parts will not be described again.

[0171] Example 4

[0172] In some possible implementations, various aspects of the methods of the embodiments of the present invention can also be implemented as a program product comprising program code that, when run on a computer device, causes the computer device to perform the steps of an electric vehicle charging navigation method based on electricity price prediction according to various exemplary embodiments of the present application described above. The executable computer program code or "code" used to perform the various embodiments can be written in high-level programming languages ​​such as C, C++, Python, Smalltalk, Java, JavaScript, Visual Basic, Structured Query Language (e.g., Transact-SQL), Perl, or in various other programming languages.

[0173] It should be understood that various parts of the present invention can be implemented in hardware, software, firmware, or a combination thereof. In the above embodiments, multiple steps or methods can be implemented in software or firmware stored in memory and executed by a suitable instruction execution system. For example, if implemented in hardware, as in another embodiment, it can be implemented using any one or a combination of the following techniques known in the art: discrete logic circuits having logic gates for implementing logical functions on data signals, application-specific integrated circuits (ASICs) having suitable combinational logic gates, programmable gate arrays (PGAs), field-programmable gate arrays (FPGAs), etc.

[0174] In the description of this specification, the references to terms such as "one embodiment," "some embodiments," "example," "specific example," or "some examples," etc., indicate that a specific feature, structure, material, or characteristic described in connection with that embodiment or example is included in at least one embodiment or example of the present invention. In this specification, the illustrative expressions of the above terms do not necessarily refer to the same embodiment or example. Furthermore, the specific features, structures, materials, or characteristics described may be combined in any suitable manner in one or more embodiments or examples. Moreover, without contradiction, those skilled in the art can combine and integrate the different embodiments or examples described in this specification, as well as the features of different embodiments or examples.

[0175] The above embodiments are merely illustrative of the technical concept and features of the present invention, and are intended to enable those skilled in the art to understand the content of the present invention and implement it accordingly. They should not be construed as limiting the scope of protection of the present invention. All equivalent changes or modifications made based on the essence of the content of the present invention should be covered within the scope of protection of the present invention.

Claims

1. A method for electric vehicle charging navigation based on electricity price prediction, characterized in that, Includes the following steps: Based on the power distribution network, transportation network, charging station and electric vehicle models, establish a power-transportation coupled network model; Based on the power-transportation coupled network model, an economic optimization model for charging navigation of electric vehicles is established, and the variables and constraints of the model are determined. Design the set of state variables in multi-agent deep reinforcement learning based on variables and constraints. Action variable set and reward function ; Construct a multi-agent SAC training network structure that considers electricity price forecast information, and set the parameters of the action network and evaluation network of the SAC training network structure; By using an interactive power-transportation coupled network model, the agent is trained to make optimal decisions under actual traffic conditions and dynamic electricity prices in order to maximize the reward function, thereby achieving cost optimization for electric vehicle users during driving and charging. The establishment of a power-transportation coupled network model based on the power distribution network, transportation network, charging station, and electric vehicle model includes: The optimal power flow model for the distribution network is as follows: As shown: Among them, the constraints for configuring the optimal power flow model of the distribution network include power balance constraints and voltage constraints; Power balance constraints are as follows: As shown: In the formula, Indicates the cost of purchasing electricity. This represents the active power output of the node generator. This represents the power injected from other networks. This represents their correlation coefficient; It is a node AC voltage, They represent upper and lower boundaries, These represent the generator output power. upper and lower boundaries, Represents a node and nodes Branch transmission power between Represents a node The load, It is the line admittance matrix. This represents the maximum value of the transmitted power in the branch. Show node and nodes The maximum value of the phase angle difference between them. express The imaginary part; Due to constraints It is a non-convex nonlinear constraint. A second-order cone relaxation method is used to relax the constraint. Transform into the standard second-order cone inequality form: The local marginal electricity price is calculated by extracting the dual variables of the active power balance equation. ; Transportation network model As shown in the following formula As shown: In the formula, Let each represent a set of traffic nodes and a set of edges, respectively. Representing nodes respectively and The edge weights between distance and branch road travel time are used; considering actual traffic congestion, the actual branch road travel time is calculated using the BPR model, as shown in the following formula. As shown: In the formula, and Representing edges respectively Traffic flow and traffic capacity; The charging station model is as follows: As shown: In the formula, Represents a collection of charging stations; Indicates the first The state matrix of each charging station; and Indicates the first Each charging station corresponds to a node and location in the power distribution network and transportation network; Indicates the first The charging demand of each charging station. Indicates the real-time charging electricity price; This indicates the waiting time required for an electric vehicle to charge. The electric vehicle model is as follows: As shown: In the formula, Indicates the time step in the charging process; Indicates the first The battery status of an electric vehicle; This indicates the location node of an electric vehicle in the transportation network; and These represent the initial state of charge and the energy capacity, respectively. The prediction network model for Long Short-Term Memory (LSTM) networks is as follows: In the formula, Input as a sequence; Indicates at time step of dimensional subsequence input; These represent candidate values, input gate, forget gate, and output gate, respectively. and They represent and the tanh activation function; Element-level multiplication; Indicates the update status of the memory cell. This indicates the hidden state at the current moment.

2. The electric vehicle charging navigation method based on electricity price prediction according to claim 1, characterized in that, The expression for the economic optimization model of the electric vehicle charging navigation is: The constraints of the electric vehicle charging navigation economic optimization model include route operation constraints and charging station selection constraints. The constraints of the economic optimization model for configuring electric vehicle charging navigation are as follows: As shown: In the formula, and These are the actual driving time and distance; and These are the charging capacity and the waiting time, respectively. and These are the costs of driving and charging; if an electric vehicle drives over a sidewalk... Then binary variables A value of 1 indicates a positive result, while a value of 0 indicates a negative result; if a charging station is selected... Then binary variables =1, otherwise =0; and These are the cost conversion weighting factors for distance and time, respectively; This indicates the remaining battery power after arriving at the charging station; This indicates the initial position of the electric vehicle.

3. The electric vehicle charging navigation method based on electricity price prediction according to claim 2, characterized in that, The state variables are designed as follows: In the electric vehicle charging navigation task within a power-transportation coupled network, the state should be selected based on environmental indicators that best reflect the current operating status of the system and are directly related to the actions, with road nodes being chosen as the selection criteria. Electric vehicle battery status Electricity price information received by electric vehicles from various charging stations Electricity price forecasts for each charging station are generated through the prediction network. Time step , The observation of the state variable is expressed as: : 。 4. The electric vehicle charging navigation method based on electricity price prediction according to claim 2, characterized in that, The action variables are selected as follows: The action variables should be those that directly affect the reward and state. Therefore, input the current state and select the charging station chosen by the agent. As a variable for the action taken: In the formula, This represents a collection of charging stations.

5. The electric vehicle charging navigation method based on electricity price prediction according to claim 2, characterized in that, The reward function is as follows: The agent's optimization objective is to find the economically optimal solution within the feasible domain over time steps. Therefore, the reward settings are divided into rewards for reaching the destination charging station and rewards for traveling on the road: In the formula, This indicates the maximum battery capacity of the electric vehicle.

6. The electric vehicle charging navigation method based on electricity price prediction according to claim 1, characterized in that, The action network selects action variables based on state variables, and the policy function of the action network is updated using a stochastic policy gradient algorithm. In the formula, It is the output value of the Q network; Represents an entropy term. It is the weight value of the entropy term; Perform gradient calculation for the action network.

7. The electric vehicle charging navigation method based on electricity price prediction according to claim 6, characterized in that, The target value of the Q network By Critic target network Based on the Bellman equation, the objective value of the Q-network is as follows: in, This represents the discount factor.

8. The electric vehicle charging navigation method based on electricity price prediction according to claim 7, characterized in that, The multi-agent SAC training network structure that considers electricity price prediction information incorporates both a Q-network and an evaluation network. The target value of the Q-network is calculated using the target value of the evaluation network, and then... Update Q network parameters To minimize the loss function ,use Update the parameters of the evaluation network To minimize the loss function , and As shown in the following formula: The system first obtains action variables based on state variables through a policy network, and then... and system action variables As input, it is mapped through a Q-network to obtain... The network update parameters are obtained through the backpropagation algorithm of the action network loss function in equation (36): In the formula, This is the loss function for the action network.

9. An electronic device, characterized in that, The electronic device includes a processor and a memory, wherein the memory stores at least one instruction, at least one program, a code set, or an instruction set, and the at least one instruction, the at least one program, the code set, or the instruction set is loaded and executed by the processor to implement the method as described in any one of claims 1 to 8.

Images