A dynamic pricing and scheduling method for electric vehicle charging stations

Abstract

The application relates to a dynamic pricing and scheduling method of an electric vehicle charging station, comprising the following steps: acquiring the global state of the electric vehicle charging station in real time, constructing a global Markov decision process based on the global state, and solving the real-time optimal three-grade charging price based on a SAC reconstruction algorithm; a user selects a target gear and a target power state in the real-time optimal three-grade charging price; the local state of each charging pile in the electric vehicle charging station is acquired in real time; a dynamic graph and a local Markov decision process are constructed based on the local state of each charging pile, and the charging power of each charging pile is solved based on a graph attention network and a multi-charging pile deep reinforcement learning algorithm; for the charging pile selected by the user, the selected charging pile is used to charge the vehicle of the user according to the charging power solved by the selected charging pile until the vehicle of the user reaches the target power state, and the method can significantly improve the charging station profit and reduce the charging queuing time without changing the hardware.

Description

Technical Field

[0001] This application relates to the field of electric vehicle charging station management technology, and in particular to a dynamic pricing and scheduling method for electric vehicle charging stations. Background Technology

[0002] Existing electric vehicle charging stations generally suffer from the following problems: First, pricing and scheduling are decoupled, making it impossible to make real-time trade-offs between user response and station congestion; second, uniform pricing or coarse-grained strategies make it difficult to balance profit and service quality (queue latency); third, scheduling often uses fully connected or rule-based heuristics, making it difficult to express local congestion relationships and topological constraints between charging piles, easily leading to persistent congestion in certain areas. Although reinforcement learning has applications in single tasks, there is a lack of joint solutions that can simultaneously consider "pricing affecting demand / arrival behavior" and "scheduling affecting service quality." Summary of the Invention

[0003] Therefore, it is necessary to provide a dynamic pricing and scheduling method for electric vehicle charging stations, including:

[0004] S1: Real-time acquisition of the global state of electric vehicle charging stations, construction of a global Markov decision process based on the global state, and solving for the real-time optimal three-tier charging price based on the SAC reconstruction algorithm.

[0005] S2: The user selects the target charging level and target battery status from the three best charging price levels in real time;

[0006] S3: Real-time acquisition of the local state of each charging pile in the electric vehicle charging station. The local state includes the total available power of the charging pile, the current battery status of the user's vehicle, the target battery status, and the charging price corresponding to the target level. Based on the local state of each charging pile, a dynamic graph and a local Markov decision process are constructed, and the charging power of each charging pile is solved based on the graph attention network and the deep reinforcement learning algorithm for multiple charging piles.

[0007] S4: For the charging pile selected by the user, charge the user's vehicle according to the charging power calculated for the selected charging pile until the user's vehicle reaches the target power level.

[0008] Beneficial effects: This method can significantly improve the profitability of charging stations and reduce charging queuing time without modifying the hardware, and it has good interpretability and engineering feasibility. Attached Figure Description

[0009] To more clearly illustrate the technical solutions in the embodiments of this application or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0010] Figure 1 This is a flowchart of the dynamic pricing and scheduling method for electric vehicle charging stations in this application embodiment. Detailed Implementation

[0011] To make the above-mentioned objectives, features, and advantages of this application more apparent and understandable, the specific embodiments of this application are described in detail below with reference to the accompanying drawings. Many specific details are set forth in the following description to provide a thorough understanding of this application. However, this application can be implemented in many other ways different from those described herein, and those skilled in the art can make similar modifications without departing from the spirit of this application. Therefore, this application is not limited to the specific embodiments disclosed below.

[0012] Furthermore, the terms "first" and "second" are used for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the number of technical features indicated. Thus, a feature defined as "first" or "second" may explicitly or implicitly include at least one of that feature. In the description of this application, "multiple" means at least two, such as two, three, etc., unless otherwise explicitly specified.

[0013] like Figure 1 As shown, this embodiment provides a dynamic pricing and scheduling method for electric vehicle charging stations, including:

[0014] S1: Obtain the global state of the electric vehicle charging station in real time, construct a global Markov decision process based on the global state, and solve for the real-time optimal three-tier charging price based on the SAC reconstruction algorithm.

[0015] The three charging options include:

[0016] Fast charging mode: charging to Suitable for time-sensitive users.

[0017] Standard setting: Charging to Balancing time and battery life.

[0018] Full charge level: charging to full capacity Suitable for users with battery anxiety.

[0019] In this embodiment, the global state includes: the load state, power state, demand forecast, time characteristics, historical statistics, and cost perception characteristics of the electric vehicle charging station;

[0020] The load status of electric vehicle charging stations includes charging pile utilization, number of charging piles currently charging, number of idle charging piles, waiting queue length, average waiting time, and maximum waiting time.

[0021] The power status of an electric vehicle charging station includes its current total power, total capacity, remaining power, and current electricity price.

[0022] Demand forecasting for electric vehicle charging stations includes current arrival rate, arrival rate within the next hour, and current departure rate.

[0023] The time characteristics of electric vehicle charging stations include time information (such as the time of day of the week).

[0024] Historical statistics for electric vehicle charging stations include the percentage of users who selected each charging price level in the past hour, the average battery shortage, revenue in the past hour, and the number of vehicles served in the past hour.

[0025] The perceived cost characteristics of electric vehicle charging stations include the ratio of the current electricity price to the average electricity price, off-peak period indicators (charging pile utilization rate less than 30%), peak period indicators (charging pile utilization rate greater than 80%), and cost advantages (positive for off-peak periods and negative for peak periods).

[0026] Furthermore, the construction of the global Markov decision process based on the global state includes:

[0027] The global state is used as the first state space, and a first action space is defined, wherein the first action space is a continuous space. The pricing includes service fees for peak, weekday, and off-peak hours, with each period having three tiers of service fees. Service fees are calculated using a mapping function, as shown in the formula:

[0028] ;

[0029] in, express The service fee for the k-th tier at time t within the time period. This indicates the minimum service fee (0.05 yuan / kWh). This indicates the maximum service fee (0.8 yuan / kWh). express The normalized output signal of the k-th time interval. This indicates the mapping factor (set to 2.0). This mapping will normalize the network's output. Linear conversion to actual service fee range ,For example: The mapping is 0.05 yuan. The mapping is 0.425 yuan. The mapping is 0.8 yuan.

[0030] The charging price is calculated based on the electricity price and service fee, using the following formula:

[0031] ;

[0032] in, express The charging price at time t in the time period is the k-th tier. Represents the time corresponding to time t Electricity price during specific time periods express The service fee for the k-th tier at time t within the time period.

[0033] Electricity prices change over time, and the corresponding relationship is as follows:

[0034] .

[0035] This design enables: (1) to perceive differences in time-of-use electricity costs; (2) to learn differentiated service fee strategies; and (3) to offer price discounts during low-cost periods to enhance competitiveness.

[0036] Example: If the user's initial SOC is ,but:

[0037] Fast charging level (1 / 3 level): (Approximately 23.3% needs to be charged).

[0038] Standard setting (2 / 3 setting): (Approximately 46.7% needs to be charged).

[0039] Full charge tier (3 / 3 tier): (Must be 70% full).

[0040] By adjusting the three service fees for different time periods, user guidance under different load conditions is achieved, as shown in Table 1.

[0041] Table 1 Examples of pricing strategies under different load conditions

[0042] ;

[0043] Specific pricing example (taking off-peak period as an example, electricity purchase cost) The figures (yuan / kWh) are shown in Table 2;

[0044] Table 2 Comparison of service fees and prices during off-peak and peak hours.

[0045] ;

[0046] Total price = Electricity purchase cost (0.35) + Service fee. The total price will increase accordingly during peak periods (electricity purchase cost 1.0 yuan / kWh) and off-peak periods (electricity purchase cost 0.65 yuan / kWh), but the service fee strategy will remain similar.

[0047] This service fee strategy allows:

[0048] During idle periods: Users tend to choose the 3 / 3 option (lowest total price of 0.65 yuan / kWh) to increase charging time, improve utilization, and increase service fee revenue.

[0049] When fully loaded: users tend to choose the 1 / 3 option (lowest total price of 0.50 yuan / kWh) to leave quickly and reduce queuing. Although the service fee is low, it increases the turnover rate.

[0050] Advantages during off-peak hours: Lower electricity purchase costs, and even with higher service fees, the total price remains competitive, attracting users to charge during off-peak hours.

[0051] Based on net profit, throughput, waiting time, and charging pile utilization, a first reward function is constructed.

[0052] A global Markov decision process is constructed based on the first state space, the first action space, the state transition probability, the first reward function, and the discount factor.

[0053] Optionally, the expression for the first reward function is:

[0054] ;

[0055] ;

[0056] ;

[0057] ;

[0058] ;

[0059] ;

[0060] in, This represents the first reward at time t. , , , These are the first weighting coefficient, the second weighting coefficient, the third weighting coefficient, and the fourth weighting coefficient (with values ​​of 1.0, 0.5, 0.3, and 0.2, respectively). This represents the net profit at time t. This represents the throughput at time t. This represents the waiting penalty at time t. This represents the utilization rate of the charging pile at time t. Optimization items, Indicates the number of charging stations. Represents the time t. Service fee for each charging station Indicates the time up to time t. The electricity provided by each charging station This represents the overload cost up to time t. This represents the utilization rate of the charging pile at time t. Time period weighting This represents the number of vehicles that have completed charging by time t. Indicates the length of the waiting queue. This indicates the number of items in the waiting queue at time t. The waiting time for the position.

[0061] Furthermore, the real-time optimal three-tier charging price is obtained based on the SAC reconstruction algorithm, which includes finding the real-time optimal three-tier service fee. The three service fees are then added to the electricity price at the corresponding time to obtain the real-time optimal three-tier charging price. Specifically:

[0062] Learn the optimal strategy from state to service fee using the SAC reconstruction algorithm. The following strategies were discovered during the training process for the charging stations:

[0063] 1. Cost-aware pricing: During off-peak periods (electricity purchase cost of 0.35 yuan / kWh), even with a higher service fee (such as 0.5 yuan / kWh), the total price (0.85 yuan / kWh) is still lower than the peak period benchmark price, attracting price-sensitive users to charge during off-peak hours.

[0064] 2. Demand Forecasting: Before peak demand (e.g., 17:00-18:00), reduce the service fee by 1 / 3 to guide some users to choose fast charging and reserve capacity for the upcoming peak; at the same time, increase the service fee by 3 / 3 to suppress the demand for long-term charging.

[0065] 3. Improved off-peak utilization: During the late-night off-peak period (2:00-5:00), the service fee for the 3 / 3 tiers is significantly reduced (e.g., 0.2 yuan / kWh), with a total price of only 0.55 yuan / kWh, attracting overnight charging users and improving equipment utilization and total revenue.

[0066] 4. Power contention mitigation: When power utilization... Even if the utilization rate is not high, the service fee for the 3 / 3 tier will be increased to avoid new users choosing to charge for a long time, which would intensify power competition.

[0067] 5. Profit maximization: During periods of high demand and off-peak electricity prices, appropriately increase the service fee cap (to approximately 0.8 yuan / kWh) to maximize revenue per unit of electricity while maintaining price competitiveness.

[0068] This dynamic adjustment capability, which combines time-of-use pricing arbitrage and demand response management, is learned through a large amount of training data and surpasses the limitations of manual rules.

[0069] The core idea of ​​the SAC reconstruction algorithm is to maximize policy entropy while optimizing cumulative reward, thus encouraging exploration.

[0070] ;

[0071] in, For policy entropy, For temperature parameters, This represents the discount factor at time t. Indicates the first action. This represents the first state at time t. This represents the first reward at time t. Representation strategy.

[0072] SAC uses the following neural network:

[0073] Policy network (Actor) :

[0074] ;

[0075] ;

[0076] ;

[0077] ;

[0078] ;

[0079] in, This represents the output of the first hidden layer of the policy network. This represents the output of the second hidden layer of the policy network. This represents the weight matrix of the first hidden layer of the policy network. This represents the bias vector of the first hidden layer of the policy network. This represents the weight matrix of the second hidden layer of the policy network. This represents the bias vector of the second hidden layer of the policy network. This represents the mean vector of the Gaussian policy. The weights representing the mean. This indicates the bias of the mean. Indicates standard deviation, The weights representing the logarithmic standard deviation, Indicates the bias of the logarithmic standard deviation;

[0080] The input is a 33-dimensional state (including cost-aware features), and the output is a 9-dimensional service fee control signal (3 time periods × 3 tiers). A Gaussian strategy is adopted, and gradient propagation is supported through the reparameterization trick.

[0081] Q Network (Critic) and :

[0082] ;

[0083] ;

[0084] ;

[0085] ;

[0086] in, This represents the concatenated vector of the first state and the first action pair. This represents the output of the first hidden layer of the Q network. This represents the weight matrix of the first hidden layer of the Q-network. This represents the bias vector of the first hidden layer of the Q network. This represents the output of the second hidden layer of the Q network. This represents the weight matrix of the second hidden layer of the Q-network. This represents the bias vector of the second hidden layer of the Q network. Represents the Q value. The weights representing the Q-values Indicates the bias of the Q value;

[0087] Input dimension is Double Q-Learning is used to reduce overestimation bias.

[0088] The key hyperparameters of the SAC reconstruction algorithm are set as follows:

[0089] State dimension: 33 (including cost-aware features); Action dimension: 9 (3 time periods × 3 gears); Hidden layer dimension: 256; Learning rate: η = 10−5; Batch size: B = 256; Update interval: U pricing = 80 time steps; soft update coefficient: τ = 0.001; initial temperature:α 0 = 0.25; Service fee range: [0.05, 0.8] yuan / kWh;

[0090] The training method for the SAC reconstruction algorithm is the conventional SAC reconstruction algorithm training algorithm.

[0091] S2: Users select their target charging level and target battery level from the three best charging price options available in real time.

[0092] S3: Real-time acquisition of the local state of each charging pile in the electric vehicle charging station. The local state includes the total available power of the charging pile, the current battery status of the user's vehicle, the target battery status, and the charging price corresponding to the target level. Based on the local state of each charging pile, a dynamic graph and a local Markov decision process are constructed, and the charging power of each charging pile is solved based on the graph attention network and the deep reinforcement learning algorithm for multiple charging piles.

[0093] In this embodiment, the local state includes: the status of the user vehicle connected to the corresponding charging pile, the current battery status of the user vehicle, the ideal battery status, the target battery status, the target charging level, the battery capacity of the user vehicle, the current power of the corresponding charging pile, the maximum power of the corresponding charging pile, the allowed power of the user vehicle, the remaining charging time, the arrival time of the user vehicle, the estimated departure time of the user vehicle, the charging price corresponding to the target charging level, the discount rate, the waiting queue length, the estimated number of vehicles arriving in the next time period, the total available power of the charging pile, and the current electricity price.

[0094] Example: Electric vehicle charging stations have The total available power of the charging stations is [number] =600kW, power shortage threshold (i.e., 420 kW). The status of each charging station at time t is shown in Table 3;

[0095] Table 3 Examples of local states of charging piles

[0096] ;

[0097] Compared to traditional methods, the second state space adds the target battery level, target charging level, and current charging pile power, enabling scheduling to perceive users' differentiated needs and price information, and achieve coordination with pricing.

[0098] Furthermore, a local Markov decision process is constructed based on the local state of the charging pile, including:

[0099] The local state of any charging pile is taken as the second state space, and a second action space is defined, which includes several power levels. The second action space of a charging pile is represented as follows: kW;

[0100] The actual set of actionable actions is determined by the action masking mechanism:

[0101] ;

[0102] in, Represents the time t. The actual feasible second action space for a charging pile Indicates the second action. Represents the time t. The maximum power of each charging station Represents the time t. Each charging station connects to a user vehicle with the permitted power output. This design avoids the charging station learning invalid actions and accelerates training convergence.

[0103] A second reward function is constructed based on the charging revenue, charging volume achievement, waiting time, and charging pile utilization rate of the charging pile.

[0104] Based on the second state space, second action space, state transition probability, second reward function, and discount factor corresponding to the charging pile, a corresponding local Markov decision process is constructed.

[0105] Optionally, the expression for the second reward function is:

[0106] ;

[0107] ;

[0108] ;

[0109] ;

[0110] ;

[0111] in, Represents the time t. The second reward for each charging station Represents the time t. The charging revenue of each charging station Represents the time t. Rewards for achieving the target number of charging stations Represents the time t. Waiting penalty for each charging station Represents the time t. One charging station, Indicates the first weight. Represents the time t. The charging price corresponds to the target charging level for each charging station, where the charging price is the sum of the current electricity price and service fee. This represents the electricity price at time t. Indicates the length of the time slice (e.g., 10 minutes). Represents the time t. The status of each charging station connected to a user's vehicle (1 for connected, 0 for not connected). Indicates the second weight. Represents the time t. The battery status of user vehicles connected to each charging station. Represents the time t. The target state of charge (target SOC) of user vehicles connected to each charging station. Indicates the third weight. Represents the time t. The maximum power of each charging station Represents the time t. The power of each charging station Indicates the length of the waiting queue. Indicates the number of charging stations. Represents the time t. The maximum power of each charging station Represents the time t. The power of each charging station Indicates the fourth weight. This represents the total available power of the charging pile at time t.

[0112] Specifically, a dynamic graph is constructed based on the local state of each charging station, including:

[0113] A dynamic graph is constructed by using each charging pile as a node, the degree of power competition between charging piles as the edge weight, and the adjacency matrix calculated based on the local state of each charging pile.

[0114] Furthermore, the adjacency matrix is ​​calculated based on the local state of each charging station, including:

[0115] Step 1: Initialize the adjacency matrix and set the power stress threshold;

[0116] Step 2: Determine whether each charging station is active based on the status of user vehicles connected to the charging station; if a user vehicle is connected to the charging station, the corresponding charging station is active; if a user vehicle is not connected to the charging station, the corresponding charging station is inactive.

[0117] Step 3: Calculate the total power utilization rate based on the current power of each active charging station. The calculation formula is:

[0118] ;

[0119] in, This represents the total power utilization rate at time t. Indicates the number of charging stations. Represents the time t. The power of each charging station This represents the total available power of the charging pile at time t;

[0120] Specifically, based on the data shown in Table 3, the total power utilization rate is calculated as follows:

[0121] ;

[0122] Step 4: When the total power utilization rate is less than or equal to the power shortage threshold, set all the element values ​​in the adjacency matrix to 0 or set the element values ​​corresponding to two physically adjacent charging piles to 1; otherwise, it is determined that there is a competition relationship between all active charging piles, and the element values ​​corresponding to two active charging piles are set to 1 to obtain the adjacency matrix.

[0123] Specifically, due to Given the current relatively abundant power, a sparse graph (or an empty graph) is constructed based on the algorithm:

[0124] ;

[0125] in, Let represent the adjacency matrix at time t. Each element with a value of 0 indicates that there is no competition between the two corresponding charging piles. Each charging pile makes independent decisions and does not need to consider the state of its neighbors.

[0126] If two more new vehicles arrive at this time, charging stations 3 and 6 will begin charging, each using 90 kW, then:

[0127] ;

[0128] Power becomes critical, at which point all active charging stations (1, 2, 3, 4, 5, 6, 7) need to be interconnected to form a fully connected diagram:

[0129] ;

[0130] An element with a value of 1 indicates a competitive relationship between the two corresponding charging piles; for charging pile 1, calculate its attention weights to the other 6 neighboring charging piles:

[0131] ;

[0132] in, Indicates charging station The influence weight on the decision of charging pile 1. For example, if the calculation yields:

[0133] ;

[0134] This indicates that the decision-making of charging pile 1 is mainly influenced by its own state. ) and charging pile 4 ( The impact of high-power use.

[0135] Optionally, this embodiment supports multiple adjacency matrix construction strategies. In addition to the power-aware binary strategy described above, the following can also be selected:

[0136] 1. Fully-Connected Mode: All charging stations are always interconnected, and the adjacency matrix remains constant.

[0137] ;

[0138] The advantage is that information flows smoothly, but the computational complexity is high. Furthermore, it introduces noise when the power is sufficient.

[0139] 2. Spatial Adjacency Mode: Connects only physically adjacent charging stations (tri-diagonal matrix):

[0140] ;

[0141] Computationally efficient (complexity) However, it overlooked the power competition between remote charging stations.

[0142] Optionally, more complex continuous graph construction strategies can be designed, such as:

[0143] Soft threshold strategy: Edge weights change smoothly with power utilization.

[0144] ;

[0145] in, Indicates the charging station at time t With charging piles Edge weights between them For the Sigmoid function, This is the steepness parameter.

[0146] Power gradient strategy: Construct edge weights based on the power differences of charging stations.

[0147] ;

[0148] in, Indicates the charging station at time t With charging piles Edge weights between them Represents the time t. The power of each charging station Represents the time t. The power of each charging station This indicates the first hyperparameter; high-power piles are more closely connected to each other, reflecting a stronger competitive relationship.

[0149] Top-K nearest neighbors: Each stake is connected only to the stake with the closest power. A neighborhood, balancing sparsity and expressiveness.

[0150] This embodiment employs a power-aware mode that adaptively switches between sparse and fully connected graphs based on power usage. This is a hard-threshold strategy, and although it only has two states, it offers the following advantages in practice:

[0151] Sparsity: When power is sufficient (charging pile utilization rate) The diagram is empty; each stake makes independent decisions, resulting in a computational complexity of [missing information]. .

[0152] Synergy: When power is scarce (charging pile utilization rate) This forms a fully connected graph, where all active piles coordinate to distribute power and avoid overload.

[0153] Semanticity: The existence of edges indicates a power competition relationship, and the GAT attention mechanism automatically learns the competition intensity.

[0154] Robustness: The binary strategy avoids frequent switching near the threshold, thus improving stability.

[0155] The reason for choosing the power-aware mode in this embodiment is that: (1) it is simple and efficient to implement; (2) its physical meaning is clear (coordination is required when power is tight); and (3) experiments show that this strategy achieves a good balance between net profit and service quality. Although more complex continuous strategies are theoretically more flexible, they introduce additional hyperparameters (such as soft threshold steepness and similarity bandwidth), which increases the difficulty of parameter tuning, and the performance improvement is limited in the scenario of this study.

[0156] Furthermore, the method for calculating the charging power of each charging station based on graph attention networks and deep reinforcement learning algorithms for multiple charging stations includes:

[0157] The dynamic graph and the second action space of each charging pile are passed through a graph attention network. The charging piles aggregate the second action spaces of neighboring charging piles that have competing relationships according to the adjacency matrix to obtain the graph attention corresponding to the charging piles.

[0158] The graph attention corresponding to each charging pile is input into the target Critic network to obtain the Q value of each charging pile.

[0159] The Q value of each charging pile is input into the policy network of the corresponding charging pile to obtain the charging power of each charging pile.

[0160] Specifically, Graph Attention Networks (GAT) adaptively aggregate neighbor node information through an attention mechanism. The computation process of a single attention head is as follows:

[0161] Step 1: Feature transformation, for nodes Features Applying linear transformations:

[0162] ;

[0163] in, express Features after linear transformation Indicates charging station exist The output features of layer GAT have a dimension of For the first layer GAT, (Encoder output dimension), for the second layer GAT, (The output dimension of the first layer GAT is obtained by splicing 32 dimensions from each of the 8 attention heads); express The learnable weight matrix of layer GAT, ; This represents the layer index of GAT. This represents the output feature dimension of a single attention head ( ), Indicates the dimension of the input features;

[0164] For the first layer GAT ( ), input features Generated by the node encoder: charging pile 18-dimensional local state With the second action in 1D Concatenate into a 19-dimensional vector Then, it is encoded into 256-dimensional features through a fully connected layer and layer normalization and ReLU activation function. For the second layer of GAT, the input features are the output features of the first layer of GAT. The dimension is 256. Node features. Essentially, it is an encoded representation of the charging pile's status and second actions, including the charging pile's physical status (SOC, power, time, etc.), user demand information (target SOC, gear selection), and interaction information with the environment.

[0165] Step 2: Attention coefficient calculation (core of attention matrix construction), calculation node For neighboring nodes raw attention score This is a crucial step in constructing the attention matrix. Specifically, the first step is to connect the nodes... and nodes Feature concatenation after transformation:

[0166] ;

[0167] in This represents a vector concatenation operation. Then, an importance score is calculated using a learnable attention mechanism.

[0168] ;

[0169] in, This is a learnable attention weight vector. This represents the LeakyReLU activation function. This indicates transpose. The physical meaning is: at a given node In the current state, neighboring nodes Its relative importance.

[0170] For all node pairs in a batch, an attention score matrix can be constructed. ,in In the charging station scenario, Reflects the charging pile Power usage status of charging piles The degree of impact of the decision (e.g., the intensity of power competition).

[0171] Step 3: Attention Normalization and Masking. To obtain normalized attention weights, we first apply an adjacency matrix mask to ensure that attention is calculated only for nodes with edges in the graph.

[0172] ;

[0173] Then, softmax normalization is used to obtain the final attention weight matrix. :

[0174] ;

[0175] in, Represents the attention weight matrix The element in the i-th row and j-th column, Indicates the existence of an edge nodes For neighboring nodes Attention score Indicates the existence of an edge nodes For neighboring nodes Attention score For nodes The set of neighbors. Due to the absence of edge nodes. ,have ,therefore This achieves hard constraints on the adjacency matrix.

[0176] Physical meaning of the attention weight matrix: the final attention weight matrix. (in and ) represents a node When aggregating neighbor information, for neighbor nodes Weighting coefficients for features. In charging pile scheduling, if... Larger size indicates that the charging station The state of the charging station (such as high power use) The impact of their decisions is significant.

[0177] Step 4: Feature aggregation, using attention weights to aggregate neighbor features:

[0178] ;

[0179] in, Represents a node In the Output features of layer GAT (features that have not yet been spliced). Represents the activation function of the exponential linear unit;

[0180] Step 5: Multi-head splicing, using Each attention point is spliced ​​together and output.

[0181] ;

[0182] in, Represents a node In the The final output features of the layer GAT (the complete features after multi-head splicing). Indicates the first Normalized attention weights for each attention head. Represents a node After the first The features after linear transformation of each attention head, with dimension . , This represents the vector concatenation operator, and the final output dimension is... .

[0183] GAT's attention mechanism enables each charging station to:

[0184] 1. Identify key competitors: by learning attention weights Charging piles Automatically identify neighboring charging stations that have the greatest impact on it. For example, when a charging station... When using near-maximum power It will be relatively large, charging pile They will give more consideration to its condition.

[0185] 2. Coordinated power allocation: In situations of power shortage ( Each pile transmits the information "how much power I need" through GAT, and negotiates a power distribution scheme that does not overload.

[0186] 3. Priority Learning: GAT can learn implicit priority rules, for example:

[0187] EVs with less remaining time receive higher power (to avoid timeout);

[0188] Prioritize EVs in the higher price range (profit optimization);

[0189] EV acceleration near the target SOC is complete (increased throughput).

[0190] The framework of Graph Attention Network and Multi-Charging-Pile Deep Reinforcement Learning Algorithm (GAT-MAAC) includes:

[0191] Training phase: The target Critic network (GAT-Critic network) has access to global information (the state and actions of all stakes) and learns an accurate Q-value function.

[0192] The mathematical expression for GAT-Critic is:

[0193] ;

[0194] ;

[0195] ;

[0196] ;

[0197] ;

[0198] in, Indicates charging station The input vector is determined by the local state. Second action obtained by piecing together Indicates charging station The local state, Indicates charging station The second action, Indicates charging station The initial embedding features, This represents the weight matrix of the node encoder. Indicates charging station The output features of layer 1 GAT This represents the first layer of the graph attention network. Indicates charging station The output characteristics of the 2-layer GAT This represents the second-layer graph attention network. Indicates charging station Q-value (state-action value function). This represents the weight matrix of the first layer of the Q-value prediction head. This represents the weight matrix of the second layer of the Q-value prediction head. This represents the weight matrix of the Q-value prediction head output layer. Represents the ReLU activation function. Representation layer normalization.

[0199] Its key design features are:

[0200] Use LayerNorm for stable training to avoid gradient explosion.

[0201] Two layers of GAT enable 2-hop information transmission, allowing each charging station to sense "neighbor's neighbor".

[0202] Multi-head attention (8 heads) enhances expressiveness and captures diverse competitive patterns.

[0203] Execution phase: Each policy network (Actor) makes independent decisions based solely on local loading, without the need for communication.

[0204] Each charging station It has an independent Actor network :

[0205] ;

[0206] ;

[0207] ;

[0208] ;

[0209] in, This represents the output feature (dimension 128) of the first hidden layer of the Actor network. This represents the output feature (dimension 128) of the second hidden layer of the Actor network. This represents the output dimension (dimension 64) of the third hidden layer of the Actor network. , , , These represent the weight matrices of the first, second, third, and output layers in the Actor network, respectively. , , , These represent the biases of the first, second, third, and output layers in the Actor network, respectively. This represents the output vector of the Actor network (containing 10 unnormalized log probabilities), and the 10-dimensional output. The vector corresponds to 10 power levels, and each logit value represents the unnormalized log probability of selecting the corresponding power level.

[0210] In this embodiment, to avoid charging piles selecting physically infeasible actions, an action mask is introduced. The process is performed to obtain the masked output vector. Specifically, for invalid actions (such as charging when there is no vehicle available, exceeding the user's vehicle's allowed power, etc.), their logit is set to negative infinity, so that their probability becomes zero after softmax.

[0211] ;

[0212] For example, when there are no cars at the charging station ( When ), only kW is active; the logit for other gears is set to [value]. When a vehicle is present but the maximum allowable power is 50kW, the logit for the 60, 70, 80, and 90kW charging levels is disabled. This design ensures that the charging station samples only from the set of valid actions, avoiding the learning of invalid strategies, significantly accelerating training convergence, and guaranteeing decision safety.

[0213] Finally, the action probability distribution is obtained through softmax normalization:

[0214] ;

[0215] Due to invalid actions ,have Therefore, its probability is 0, which achieves a hard constraint.

[0216] To reduce Actor gradient variance, a counterfactual baseline from COMA is used; for charging stations The baseline is defined as:

[0217] ;

[0218] in, Including charging piles The second action of other charging piles, Indicates charging station The counterfactual baseline value refers to the value in the global state. The second action of other charging piles In a fixed location, charging stations Choose the average Q value of all possible actions; Indicates the global state and joint actions Below, charging pile The Q value (calculated from the GAT-Critic network).

[0219] The advantage function is defined as:

[0220] ;

[0221] in, Indicates charging station The advantage function value, this baseline eliminates the conflict with charging piles. The irrelevant reward component allows the gradient to more accurately reflect the impact of individual decisions.

[0222] The key hyperparameters of the GAT-MAAC algorithm are set as follows:

[0223] Actor learning rate: Critic learning rate: Batch size: Update interval: Time step; Soft update coefficient: Entropy regularity coefficient: GAT attention head count: Number of GAT layers: 2.

[0224] This embodiment provides a phased collaborative training strategy:

[0225] Phase 1 (Episode 1-500) Pre-training on the charging station side:

[0226] Fixed three price tiers Yuan / kWh;

[0227] Only update the GAT-MAAC network;

[0228] Objective: To learn basic power allocation strategies.

[0229] Phase 2 (Episodes 501-2000) Joint Training:

[0230] Simultaneous training on both the charging station side and the charging pile side;

[0231] The update frequency of the charging station is lower than that of the charging pile (ratio 1:10).

[0232] Objective: To learn bilateral collaborative strategies.

[0233] 3. Phase 3 (Episodes 2001-3000) Fine-tuning:

[0234] Reduce the learning rate to 10% of its original value;

[0235] Reduce exploration noise;

[0236] Objective: To refine and optimize the strategy.

[0237] The update frequency design uses different update intervals between the two layers to reflect differences in time scales.

[0238] Charging station side: Updated once every 60 time steps (approximately 10 minutes of actual time).

[0239] Charging station side: Updated once every 80 time steps (approximately 13 minutes of actual time);

[0240] This design allows the charging pile side to quickly adapt to the current load, while the charging station side adjusts its pricing strategy over a longer period.

[0241] The experience replay buffer uses independent experience replay buffers on both sides:

[0242] Charging station side: Capacity 50,000, storage experience tuples , This represents the state vector of the charging station (dimension 33). This indicates the power level control signal (dimension 9, 3 time periods × 3 levels). Indicates the state Execute The immediate reward value obtained (including net profit from service fees, throughput rewards, waiting penalties, and utilization optimization). Indicates that the charging station is performing The state vector at the next time step (dimension 33);

[0243] Charging station side: Capacity 100,000, storage ;

[0244] The charging pile side buffer has a larger capacity because its sample dimension is higher ( (Each charging station) requires more experience to support training.

[0245] S4: For the charging pile selected by the user, charge the user's vehicle according to the charging power calculated for the selected charging pile until the user's vehicle reaches the target power level.

[0246] The dynamic pricing and scheduling method for electric vehicle charging stations provided in this embodiment has the following beneficial effects:

[0247] 1. A three-tiered collaborative architecture (charging station pricing - user selection - charging equipment scheduling) is proposed to optimize the entire process from demand guidance to resource allocation. The innovative three-tier pricing mechanism influences users' charging goal selection through price incentives, increasing the certainty of departure time.

[0248] 2. Charging Station Pricing: The SAC reconstruction algorithm is used to learn the mapping from 33-dimensional states (including cost-aware features) to 9-dimensional service fee output. The service fee pricing model (price = electricity purchase cost + service fee), state space design (load, power, demand, time, historical statistics, cost awareness), reward function design (service fee net profit, throughput, waiting penalty, utilization optimization), and the guiding role of the three-tier pricing strategy are elaborated. Maximum entropy reinforcement learning is used to achieve a balance between exploration and utilization, and cost arbitrage is achieved by combining time-of-use pricing.

[0249] 3. Charging Station Scheduling: The GAT-MAAC algorithm is used to optimize the coordination of multiple charging stations. The role of graph attention mechanism in modeling power competition is highlighted: (1) The dynamic adjacency matrix adaptively constructs the graph topology based on power utilization; (2) The multi-head GAT layer automatically identifies key competitors through attention weights; (3) The counterfactual baseline reduces the variance of policy gradients. The centralized training and decentralized execution framework balances training effectiveness and execution efficiency.

[0250] 4. Collaborative Training: Design phased training strategies and differentiated update frequencies to achieve effective collaboration between the two sides.

[0251] The technical features of the above embodiments can be combined in any way. For the sake of brevity, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0252] The embodiments described above are merely illustrative of several implementation methods of this application, and while the descriptions are relatively specific and detailed, they should not be construed as limiting the scope of the patent application. It should be noted that those skilled in the art can make various modifications and improvements without departing from the concept of this application, and these all fall within the protection scope of this application. Therefore, the protection scope of this patent application should be determined by the appended claims.

Claims

1. A dynamic pricing and scheduling method for electric vehicle charging stations, characterized in that, The method comprises the following steps: S1: Real-time acquisition of the global state of the electric vehicle charging station, construction of a global Markov decision process based on the global state, and solving of a real-time optimal three-tier charging price based on a SAC reconstruction algorithm; S2: User selection of a target gear and a target power state from the real-time optimal three-tier charging price; S3: Real-time acquisition of the local state of each charging pile in the electric vehicle charging station, the local state comprising the total available power of the charging pile, the current power state of the user's vehicle, the target power state, and the charging price corresponding to the target gear; construction of a dynamic graph and a local Markov decision process based on the local state of each charging pile, and solving of the charging power of each charging pile based on a graph attention network and a multi-charging pile deep reinforcement learning algorithm; Construction of a local Markov decision process based on the local state of the charging pile, comprising: Taking the local state of any charging pile as a second state space, and defining a second action space, the second action space containing a plurality of power gears; Construction of a second reward function based on the charging revenue, charging quantity achievement, waiting time, and charging pile utilization rate corresponding to the charging pile; the expression of the second reward function is: ； ； ； ； ； in, Represents the time t. The second reward for each charging station Represents the time t. The charging revenue of each charging station Represents the time t. Rewards for achieving the target number of charging stations Represents the time t. Waiting penalty for each charging station Represents the time t. Overload penalty for each charging station Indicates the first weight. Represents the time t. The charging price corresponds to the target charging level for each charging station, where the charging price is the sum of the current electricity price and service fee. This represents the electricity price at time t. Indicates the length of the time slice. Represents the time t. The situation where a charging station is connected to a user's vehicle. Indicates the second weight. Represents the time t. The battery status of user vehicles connected to each charging station. Represents the time t. The target battery level of the user vehicles connected to each charging station. Indicates the third weight. Represents the time t. The maximum power of each charging station Represents the time t. The power of each charging station Indicates the length of the waiting queue. Indicates the number of charging stations. Represents the time t. The maximum power of each charging station Represents the time t. The power of each charging station Indicates the fourth weight. This represents the total available power of the charging pile at time t; Construction of a corresponding local Markov decision process based on the second state space, the second action space, the state transition probability, the second reward function, and the discount factor corresponding to the charging pile; Construction of a dynamic graph based on the local state of each charging pile, comprising: Taking each charging pile as a node, taking the power competition degree between the charging piles as an edge weight, and taking the local state of each charging pile to calculate an adjacency matrix, thereby constructing a dynamic graph; Calculation of the adjacency matrix based on the local state of each charging pile, comprising: Step 1: Initialization of the adjacency matrix, and setting of a power shortage threshold; Step 2: Determination of whether each charging pile is active based on the connection of the charging pile to the user's vehicle; if the charging pile is connected to the user's vehicle, the corresponding charging pile is active; if the charging pile is not connected to the user's vehicle, the corresponding charging pile is not active; Step 3: Calculation of the total power usage rate based on the current power of each active charging pile, the calculation formula being: ； in, This represents the total power utilization rate at time t. Indicates the number of charging stations. Represents the time t. The power of each charging station This represents the total available power of the charging pile at time t; Step 4: When the total power usage rate is less than or equal to the power shortage threshold, the element values in the adjacency matrix are all set to 0 or the element values corresponding to two physically adjacent charging piles are set to 1; otherwise, it is determined that there is a competition relationship between all active charging piles, and the element values corresponding to two active charging piles are set to 1, thereby obtaining the adjacency matrix; S4: For the charging pile selected by the user, charging the user's vehicle according to the charging power solved corresponding to the selected charging pile until the user's vehicle reaches the target power state.

2. The dynamic pricing and dispatch method of claim 1, wherein, The global state comprises: the load state, the power state, the demand prediction, the time characteristics, the historical statistics, and the cost perception characteristics of the electric vehicle charging station; The load state of the electric vehicle charging station comprises the charging pile utilization rate, the number of charging piles being charged, the number of idle charging piles, the length of the waiting queue, the average waiting time, and the maximum waiting time; The power state of the electric vehicle charging station comprises the total power, the total capacity, the remaining power, and the current electricity price of the electric vehicle charging station; The demand prediction of the electric vehicle charging station comprises the current arrival rate, the future one-hour arrival rate, and the current departure rate. The time feature of the electric vehicle charging station includes time information; The historical statistics of the electric vehicle charging station include the proportion of users selecting each price level in the past 1 hour, the average power gap, the income in the past 1 hour, and the number of vehicles served in the past 1 hour; The cost perception feature of the electric vehicle charging station includes the ratio of the current electricity price to the average electricity price, the low valley period symbol, the peak period symbol, and the cost advantage.

3. The dynamic pricing and dispatch method of claim 2, wherein, The global Markov decision process is constructed based on the global state, including: The global state is taken as a first state space, and a first action space is defined, which includes service fee pricing of peak period, flat period and valley period, each period including service fee pricing of three levels; A first reward function is constructed based on net profit, throughput, waiting time and charging pile utilization rate; A global Markov decision process is constructed based on the first state space, the first action space, state transition probability, the first reward function and the discount factor.

4. The dynamic pricing and dispatch method of claim 3, wherein, The expression of the first reward function is: ； ； ； ； ； wherein, represents the first reward at time t, , , , are respectively a first weight coefficient, a second weight coefficient, a third weight coefficient, a fourth weight coefficient, represents the net profit at time t, represents the throughput at time t, represents the waiting penalty at time t, represents the charging pile utilization rate at time t of the optimization item, represents the number of charging piles, represents the service fee of the t th charging pile at time t, represents the amount of electricity provided by the t th charging pile to time t, represents the overload cost to time t, represents the time period weight of the charging pile utilization rate at time t, represents the number of vehicles completing charging to time t, represents the length of the waiting queue, represents the waiting time of the t th in the waiting queue at time t.

5. The dynamic pricing and dispatch method of claim 4, wherein, The local state includes: the condition of the corresponding charging pile connecting the user's vehicle, the current power state of the user's vehicle, the ideal power state, the target power state, the target level, the battery capacity of the user's vehicle, the current power of the corresponding charging pile, the maximum power of the corresponding charging pile, the allowed power of the user's vehicle, the remaining charging time, the user's vehicle arrival time, the user's vehicle expected departure time, the charging price corresponding to the target level, the discount rate, the length of the waiting queue, the number of vehicles expected to arrive in the next period, the total available power of the charging pile, and the current electricity price.

6. The dynamic pricing and dispatch method of claim 1, wherein, The charging power of each charging pile is solved based on the graph attention network and the multi-charging pile deep reinforcement learning algorithm, including: The dynamic graph and the second action space of each charging pile are input into the graph attention network, and the charging piles aggregate the second action space of the neighbor charging piles with which they have a competitive relationship according to the adjacency matrix to obtain the corresponding graph attention of the charging piles; The corresponding graph attention of each charging pile is input into the target Critic network respectively to obtain the Q value of each charging pile; The Q value of each charging pile is input into the corresponding charging pile strategy network to obtain the charging power of each charging pile.

Images