Vehicle-to-vehicle energy transaction grouping method based on dynamic cluster and deep reinforcement learning

Abstract

The invention discloses a vehicle-to-vehicle energy transaction grouping method based on dynamic cluster and deep reinforcement learning, and the method can effectively improve the vehicle-to-vehicle energy transaction efficiency through the dynamic cluster division of electric vehicles participating in the transaction according to the position, speed, driving direction and energy information. Specifically, the method uses deep reinforcement learning to dynamically select the number of cluster groups and adjust the eigenvalue weight coefficient in the clustering algorithm, and performs real-time optimal decision according to the current state to obtain the maximum long-term reward, thereby improving the total economic utility of the vehicle-to-vehicle energy transaction. Simulation results show that the method promotes the formation of transactions in a cluster, improves the energy transmission efficiency in the vehicle-to-vehicle energy transaction, and has applicability in a larger-scale vehicle-to-vehicle energy transaction scene.

Description

Technical Field

[0001] The present invention relates to the technical field of electric vehicle energy trading in a connected vehicle network, and in particular to a vehicle-to-vehicle energy trading grouping method based on dynamic clustering and deep reinforcement learning. Background Art

[0002] With the increasing global concern about energy shortages and carbon emissions, electric vehicles have become a common choice for countries to reduce greenhouse gas emissions and improve the global ecological environment. At the same time, cars, especially electric vehicles, have become more intelligent and autonomous. Using advanced information and network technologies, the Internet of Vehicles, which connects smart cars to the Internet, has become a key technology to promote the development of future intelligent transportation systems. As an organic information system, the Internet of Vehicles has achieved seamless connectivity between smart cars, roadside infrastructure and pedestrians, and provided communication paradigms such as vehicle-to-vehicle, vehicle-to-roadside unit, and vehicle-to-grid, enabling a variety of applications and services such as vehicle supervision and traffic guidance.

[0003] However, with the widespread popularity of electric vehicles, the problem of charging during driving has attracted the attention of consumers, manufacturers and the government. Due to the capacity limitation of the on-board battery, the range of electric vehicles per charge is limited, and the battery needs to be charged once or multiple times when traveling long distances. However, there may be a tricky situation where the battery power is insufficient to support the vehicle to the nearest charging station. The limitation of urban resources requires the construction of car charging stations to meet the charging needs of high-density users. Range anxiety, charging anxiety and charging queues during peak hours have troubled electric vehicle users and have become a problem that needs to be solved as a barrier to the further development of electric vehicles.

[0004] Wireless power transmission technology provides a solution to the problem of charging electric vehicles. Since Nikola Tesla proposed to use electromagnetic waves to transmit electricity from one place to another, after more than 100 years of development, wireless power transmission has become a key technology to promote the development of point-to-point energy trading in the distributed energy Internet. Therefore, in the Internet of Vehicles ecosystem, car-to-car energy trading has also become a feasible energy sharing solution for users to charge conveniently and efficiently when car-to-grid energy trading cannot meet charging needs. In this new car-to-car energy trading system, vehicles with sufficient power during driving can transmit power to other vehicles that have charging needs or are about to run out of battery power. This solution can effectively alleviate users' range anxiety, the shortage of charging pile resources, and balance the distribution of power resources in the regional electric vehicle Internet.

[0005] Reducing the energy trading distance between vehicles can reduce power loss in wireless power transmission and obtain better communication performance. Dividing the vehicles involved in the transaction into clusters and conducting energy transactions within the clusters is a method generally recognized by researchers to reduce the energy trading distance. However, the discussion on how to cluster the trading vehicles is not in-depth enough in existing research. The selection and design of the partitioning method and the appropriate selection of clustering partitioning parameters based on the mobility information and power of electric vehicles can effectively improve the transaction efficiency, maximize the satisfaction of the vehicle needs of buyers and sellers, and increase the benefits of all parties in the energy trading system. Summary of the invention

[0006] The purpose of the invention is to provide a vehicle-to-vehicle energy trading grouping method based on dynamic clustering and deep reinforcement learning for a vehicle-to-vehicle energy trading system, to perform weighted clustering according to vehicle mobility information and battery power status, and to divide a large-scale energy trading network into multiple clusters, so as to improve the efficiency of wireless power transmission between vehicles in the cluster and the accessibility of power trading. At the same time, deep reinforcement learning is used to optimize the number of clusters and the eigenvalue weight coefficients in the cluster division algorithm to maximize the economic utility of the system and obtain the best system energy trading strategy.

[0007] The vehicle-to-vehicle energy trading scenario model of the present invention is shown in Figure 1 .

[0008] The principle flow chart of the method of the present invention is shown in Figure 2 .

[0009] The total economic utility and average cluster power variance comparison diagram of the method of the present invention and the comparative method is shown in Figure 3 .

[0010] The trend diagram of the number of low-power vehicles when the number of steps increases under different electric vehicle numbers is shown in the figure. Figure 4 .

[0011] The vehicle-to-vehicle energy trading scenario model of the present invention is as follows Figure 1As shown in the figure, electric vehicles are divided into different clusters according to their mobility information and energy information. Electric vehicles in the cluster can participate in transactions as buyers and sellers of energy transactions. Electric vehicles in each cluster are equipped with smart meters to record their power. Buyers, that is, vehicles with low power, send requests to charge them by interacting with the system; sellers, that is, vehicles with sufficient power, sell their excess power (excess power is obtained directly from the power grid or other energy sources at a lower price) to other vehicles in need during driving to obtain rewards; smart meters, as an important element of the system, can track the energy transmission between buyers and sellers according to the pre-established energy purchase agreement; trusted institutions are used to publish the entity identities involved in the energy trading system and manage network infrastructure. Building cluster-based vehicle-to-vehicle energy transactions is conducive to reducing energy transaction losses. Therefore, the establishment and division of clusters need to consider the following factors: First, mobility information. When transactions occur in clusters within a certain physical distance, the Euclidean distance between energy buyers and sellers can be shortened, allowing buyers and sellers to reduce their consumption resources and improve the quality of energy transmission when sharing energy. Second, power information. In order to facilitate more transactions within the cluster, it is necessary to ensure that there are potential buyers (vehicles with low power and purchase demand) and sellers (vehicles with high power and excess power) in each cluster as much as possible. This can be achieved by following the steps below:

[0012] Step (1): A total of N electric vehicles in the energy trading system are managed by a trusted institution, represented by a set V. The trusted institution initializes the system transaction process. For the nth electric vehicle V n ,V n ∈V, obtain its account balance and assign a wallet address. Collect mobility information and energy information in the energy trading system. Electric vehicles are located on an open road in two-dimensional space, and the energy transaction loss and communication distance between the two vehicles are defined by Euclidean distance. is the nth electric car in the system, that is, V n The coordinates of V n 、V n′ (n′∈N and n′≠n) The distance between two workshops is:

[0013]

[0014] In addition, the V n The motion state is a random variable Definition. Among them, Indicates the direction of travel to the destination, d ir is a set of directions, which can be set according to the actual road conditions. For example, when the scene is at an intersection of an open road in a city, d ir ={d ir [1],dir [2],d ir [3],d ir [4]}, including the four directions of east, west, south and north. On closed roads where vehicles travel in a straight line, such as national highways and expressways, the destination exit number can be used to indicate the destination. Indicates V n Driving speed, s pe is the speed set, s pe ={s pe [1],s pe [2],s pe [3]}, where s pe [1] pe [2] and pe [3] are low speed, transition speed and high speed respectively. In summary, at time t, V n The movement information can be expressed as In addition, V n The amount of electricity used is e n (t) as its energy information.

[0015] Step (2) dynamically selects the optimal number of clusters through deep reinforcement learning to obtain the optimized electric vehicle cluster grouping result. Based on the Markov decision process, a Q learning method based on a deep neural network is used, that is, a deep Q network optimizes the number of groups, thereby determining the optimal system energy trading strategy to improve system performance. Based on the scenario system model and the deep Q network algorithm, the system state space and action space are determined. The steps are as follows:

[0016] Step (2.1), based on each stept of the deep reinforcement learning process, the state space S(t) can be expressed as the position, driving direction, speed information and energy information of all electric vehicles in the system:

[0017]

[0018] Among them, the vehicle's movement information is the location, destination direction and driving speed, and the energy information is the vehicle's power level.

[0019] In step (2.2), the deep reinforcement learning agent adjusts the number of groupings K(t) of stept and the characteristic value weight coefficient ε(t) = {ε1(t), ε2(t), ε3(t), ε4(t), ε4′(t)} in the grouping algorithm. ε1, ε2, ε3 represent the weights of the clustering algorithm for position, driving direction, and speed, respectively, and ε4 and ε4′ represent the weights of the proposed clustering algorithm for power information. Therefore, the action space A(t) can be expressed as:

[0020] A(t)=[K(t),ε(t)]

[0021] At the same time, the hyperparameters of the deep Q network, such as the number of hidden layers, the number of neurons in the hidden layers, the training batch size, the capacity of the experience replay area, the target network update frequency, the training exploration degree, the discount factor, and the learning rate, are determined.

[0022] Step (3), in order to maintain the stability of the cluster and improve transaction efficiency, the electric vehicles are clustered according to their current location, direction, speed and power. The K-means clustering algorithm divides the samples into several categories based on the internal relationship between the data under the premise of unknown sample labels, so that the samples in the same category are as similar as possible, and the samples in different categories are as different as possible. After the number of groups K(t) and the weight coefficient ε(t) = {ε1(t), ε2(t), ε3(t), ε4(t), ε4′(t)} in the grouping algorithm are determined, V n Mobile information m n (t) and power information e n (t) is used as the sample feature value in the clustering algorithm, denoted as I n (t) = {m n (t),e n (t)}, the vehicle information set of all electric vehicles in the system is I(t) = {I1(t), I2(t), ..., I N (t)}. For the mobile information, V n With V n′ Euclidean distance between eigenvalues:

[0023]

[0024] Different from the convergence goal of mobile information, energy trading needs to obtain the differentiation of electricity based on electricity information to form clusters. Different from the dependence of K-Means clustering algorithm on the center point, it gives the swinging Euclidean distance of electricity:

[0025]

[0026] In this case, j is introduced as a swing counter. By pre-classifying electric vehicles with high and low power levels and clustering them using different methods based on the energy information in their sample feature values, it is possible to ensure that there are potential buyers and sellers in each cluster. The cost function in the algorithm is:

[0027]

[0028] Among them C k , C k ∈V, is the center point of the kth cluster, which specifically includes the following steps:

[0029] Step (3.1), initialization: randomly select K initial cluster centers The feature information set corresponding to the initial cluster center point Counter j l =0,j h =0.

[0030] Step (3.2), for the non-central electric vehicle V n , if V n The amount of electricity n (t) is greater than the power threshold e θ , then it is recorded as a high-power vehicle, j h =j h +1, j =j h ; Otherwise, it is recorded as a low-battery vehicle, j l =j l +1, j=j l .

[0031] Step (3.3), calculate V n The cost function of each center point is mapped to the group corresponding to the center point with the smallest cost function. For the kth * The set of all vehicles in a cluster.

[0032] In step (3.4), after all electric vehicles are grouped, the center of the cluster is recalculated using the location information of the electric vehicles in the cluster:

[0033]

[0034] Step (3.5), at this time, if the cost function converges, the current cluster division result is output; if the cost function does not converge, the counter is reset to zero and steps (3.2)-(3.4) are repeated.

[0035] This grouping method is conducive to reducing energy transaction losses. The establishment and division of clusters need to take into account mobility information, which can enable transactions to take place in clusters within a certain physical distance, shortening the Euclidean distance between energy buyers and sellers, thereby allowing buyers and sellers to reduce their consumption of resources and improve the quality of energy transmission when sharing energy. Considering power information can facilitate more transactions and ensure that there are potential buyer vehicles and seller vehicles in each cluster as much as possible.

[0036] Step (4), after the grouping is completed, the vehicles in the cluster generate transactions, and the buyer vehicle purchases energy from the seller vehicle for opportunity charging to support long-distance travel of electric vehicles. In a transaction, in order to indicate the identity of the buyer and seller of the electric vehicle, To represent the buyer's vehicle that needs electricity, To express The seller's vehicle that sells the electricity. Send energy request, Energy is sold at a fixed price based on the dynamic market price set by the system. The system coordinates energy supply between sellers and buyers within the cluster based on factors such as charging efficiency, economic benefits, and communication quality. Once an energy transaction agreement is reached, The energy currency is transferred from its wallet to the seller based on the smart meter reading. Provided wallet address.

[0037] Step (5) calculates the total economic utility of the system after the transaction occurs:

[0038] ρ is the price per unit of energy in the system, η n (t) is the buyer's vehicle The energy purchased. Buyers and sellers also incur other costs when conducting transactions, including unit management costs ψ h To compensate for hardware losses and operating costs, the seller's utility function is expressed as:

[0039]

[0040] Among them, the total management fee is related to the transaction electricity volume during the actual transaction process.

[0041] For the buyer's electric car, it is expected that the received signal interference is smaller and the energy loss is smaller when paying a certain amount of energy currency. In the buyer's utility function, the power of energy transmission will decline due to the increase in distance. Therefore, the buyer's utility function can be modeled as:

[0042]

[0043] in, Used to characterize from The quality of energy received. Related to the vehicle-to-vehicle energy transmission method, it is a decreasing function that changes with distance. β is the coefficient that converts energy quality into monetary rewards, and ρη n (t) is the buyer's acquisition of η n (t) Cost per unit of energy. If there is no significant benefit, electric vehicles with excess power may be unwilling to share their power with other electric vehicles with less power. At the same time, buyers also hope to purchase as much power as possible at the same cost in the transaction. Buyers and sellers seek to maximize their own interests in the process of energy trading.

[0044] The reward function in step (6) aims to make adaptive decisions through deep reinforcement learning to maximize system performance and economic benefits. The objective function is expressed as:

[0045] maxQ(S(t),A(t))

[0046] Among them, Q(S(t), A(t)) is the action value function of deep reinforcement learning, and the reward function at stept is:

[0047] R(t)=R(S(t),A(t))=ω b U B (t)+ω s U S (t)

[0048] Among them, U B (t) is the total economic utility of the buyer’s electric vehicle, U S (t) is the total economic utility of the seller’s electric vehicle, ω b ,ω s ∈(0,1] is the weight coefficient, ω b +ω s =1, which reflects the emphasis on the economic utility of buyers and sellers in the system optimization goal, and can be dynamically adjusted according to the actual system needs. Therefore, the optimal action value function of the system, that is, the long-term reward of the system, can be expressed as Where γ(t)∈(0,1] is the discount factor, which reflects the current trade-off between instantaneous rewards and future rewards. The network is trained until the total reward converges, and the trained model is used to make real-time optimal decisions on the energy trading vehicle grouping strategy.

[0049] The vehicle-to-vehicle energy trading grouping method based on dynamic clustering and deep reinforcement learning effectively divides the large-scale vehicle-to-vehicle energy trading network into multiple groups by using a clustering method that considers the mobility information and energy information of electric vehicles, thereby improving the efficiency and economic benefits of energy trading. The dynamic decision on the number of cluster groups and the eigenvalue weight coefficient in the cluster division algorithm makes the grouping of vehicle-to-vehicle transactions in the present invention more flexible and intelligent. BRIEF DESCRIPTION OF THE DRAWINGS

[0050] Figure 1 ,Schematic diagram of the model including charging stations, electric vehicles, road test units and trusted institutions in the vehicle-to-vehicle energy trading scenario.

[0051] Figure 2 ,Principle flow chart of the vehicle-to-vehicle energy trading grouping method based on dynamic clustering and deep reinforcement learning.

[0052] Figure 3, the trading system performance comparison diagram of the method of the present invention and the comparative method, respectively compares the total economic utility and the average cluster power variance of the trading system, wherein the comparative method uses the K-Means clustering algorithm to perform cluster division according to the location information of the electric vehicles in the system.

[0053] Figure 4 ,The number of low battery vehicles changes in different scale energy trading scenarios as the number of system steps increases. DETAILED DESCRIPTION

[0054] The following is a further explanation of the technical solution of the vehicle-to-vehicle energy trading grouping method based on dynamic clustering and deep reinforcement learning with reference to the accompanying drawings and examples.

[0055] The principle flow chart of the method of the present invention is as follows Figure 2 As shown, the following steps are included:

[0056] Step 1: Obtain the location, speed, driving direction and other mobile information and energy information of the electric vehicles in the energy trading system through a trusted institution, assign wallet addresses to the electric vehicles participating in the transaction, and determine their account balances;

[0057] Step 2: Determine the state space and action space of deep reinforcement learning, determine the network hyperparameters, and design the network.

[0058] Step 3: According to the number of cluster groups selected by the agent, the participating electric vehicles are grouped according to their location, speed, driving direction and energy information;

[0059] Step 4: Energy trading is carried out within the cluster. The seller's electric vehicle transmits wireless power to the buyer's electric vehicle, and the buyer's electric vehicle transfers energy currency from its wallet to the wallet address provided by the seller's electric vehicle.

[0060] Step 5: Calculate the total economic utility in the energy trading system based on the traded electricity and trading conditions;

[0061] Step six: Use economic utility as the reward function of deep reinforcement learning to train the network until the total reward converges. Use the trained model to make real-time optimal decisions on the number of groups of energy trading vehicles and the eigenvalue weight coefficients in the clustering algorithm.

[0062] Figure 3 The performance comparison chart of the trading system of the present invention and the comparative method is shown in Figure 1, which compares the total economic utility and average cluster power variance of the trading system. The comparative method uses the K-Means clustering algorithm to perform cluster division based on the location information of electric vehicles in the system. Figure 3It can be seen that the total economic utility and the average cluster power variance in the method of the present invention are higher than those of the comparison method, that is, the division of trading vehicles by the method of the present invention makes the distribution of high- and low-power vehicles in each cluster more reasonable, promotes the formation of transactions within the cluster, and at the same time improves the energy transmission efficiency in vehicle-to-vehicle energy transactions.

[0063] Figure 4 The number of low-power vehicles in different energy trading scenarios increases with the number of system steps. Figure 4 It can be seen that in the energy trading scenario of 50 electric vehicles, the number of low-power electric vehicles gradually decreases over time under the method of the present invention, and eventually all low-power electric vehicles in the system are replenished. The occasional slight rise in the curve is due to the continuous consumption of electricity by electric vehicles during driving. When the number of electric vehicles increases, the number of low-power electric vehicles continues to decrease, and eventually ensures that the power of almost all low-power electric vehicles is replenished. Replenishing battery power through car-to-car energy trading can increase the range of electric vehicles and ensure their normal driving within a certain period of time. This result verifies the practical feasibility of the method of the present invention.

Claims

1. A vehicle-to-vehicle energy trading grouping method based on dynamic clustering and deep reinforcement learning, characterized by: The method comprises the following steps: Step 1: Obtain the location, speed, driving direction and energy information of electric vehicles in the energy trading system through a trusted institution, assign wallet addresses to electric vehicles participating in the transaction, and determine their account balances; Step 2: Determine the state space and action space of deep reinforcement learning, determine the network hyperparameters, and design the network; Step 3: According to the number of cluster groups selected by the agent, the participating electric vehicles are grouped according to their location, speed, driving direction and energy information; Step 4: Energy trading is carried out within the cluster. The seller's electric vehicle transmits wireless power to the buyer's electric vehicle, and the buyer's electric vehicle transfers energy currency from its wallet to the wallet address provided by the seller's electric vehicle. Step 5: Calculate the total economic utility in the energy trading system based on the traded electricity and trading conditions; Step six: Use economic utility as the reward function of deep reinforcement learning to train the network until the total reward converges. Use the trained model to make real-time optimal decisions on the number of groups of energy trading vehicles and the eigenvalue weight coefficients in the clustering algorithm.

2. The vehicle-to-vehicle energy trading grouping method based on dynamic clustering and deep reinforcement learning according to claim 1 is characterized by: In step 1, a total of N electric vehicles in the energy trading system are managed by a trusted institution, represented by a set V; the trusted institution initializes the system transaction process, and for the nth electric vehicle V n ,V n ∈V, obtain its account balance and assign a wallet address; collect mobility information and energy information in the energy trading system; the electric vehicle is located on an open road in two-dimensional space, and the energy transaction loss and communication distance between the two vehicles are defined by Euclidean distance; is the nth electric car in the system, that is, V n The coordinates of V n 、V n′ The distance between the two workshops is: n′∈N and n′≠n, V driving on an open road n The motion state is a random variable Definition; where Indicates the direction of travel to the destination, d ir It is a direction collection, which is set according to the actual situation of the road; Indicates V n Driving speed, s pe is the speed set, s pe ={s pe [1],s pe [2],s pe [3]}, where s pe [1] pe [2] and pe [3] are low speed, transition speed and high speed respectively; at time t, V n The movement information is expressed as In addition, V n The amount of electricity used is e n (t) as its energy information.

3. The vehicle-to-vehicle energy trading grouping method based on dynamic clustering and deep reinforcement learning according to claim 1 is characterized by: In step 2, based on each stept of the deep reinforcement learning process, the state space S(t) is represented by the position, driving direction, speed information, and energy information of all electric vehicles in the system: The deep reinforcement learning agent adjusts the number of groupings K(t) of step t and the eigenvalue weight coefficient ε(t) = {ε1(t), ε2(t), ε3(t), ε4(t), ε4′(t)} in the grouping algorithm. ε1, ε2, and ε3 represent the weights of the clustering algorithm for position, driving direction, and speed, respectively. ε4 and ε4′ represent the weights of the proposed clustering algorithm for power information. The action space A(t) is expressed as: A(t)=[K(t),ε(t)] At the same time, the hyperparameters of the deep Q network, such as the number of hidden layers, the number of neurons in the hidden layers, the training batch size, the capacity of the experience replay area, the target network update frequency, the training exploration degree, the discount factor, and the learning rate, are determined.

4. The vehicle-to-vehicle energy trading grouping method based on dynamic clustering and deep reinforcement learning according to claim 1 is characterized by: In step 3, after the number of groups K(t) and the weight coefficient ε(t) = {ε1(t), ε2(t), ε3(t), ε4(t), ε4′(t)} in the grouping algorithm are determined, V n Mobile information m n (t) and power information e n (t) is used as the sample feature value in the clustering algorithm, denoted as I n (t) = {m n (t),e n (t)}, the vehicle information set of all electric vehicles in the system is I(t) = {I1(t), I2(t), ..., I N (t)}; for the mobile information, give V n With V n′ Euclidean distance between eigenvalues: Different from the convergence goal of mobile information, energy trading needs to obtain the differentiation of electricity based on electricity information to form clusters. Different from the dependence of the K-means clustering algorithm on the center point, it gives the swinging Euclidean distance of electricity: The j is introduced as a swing counter. By pre-classifying electric vehicles with high and low power, and using different methods to cluster them according to the energy information in their sample feature values, it is ensured that there are potential buyers and sellers in each cluster. The cost function in the algorithm is: Among them C k , C k ∈V, is the center point of the kth cluster.

5. The vehicle-to-vehicle energy trading grouping method based on dynamic clustering and deep reinforcement learning according to claim 4 is characterized by: Step 3 specifically includes the following steps: Step (3.1), initialization: randomly select K initial cluster centers The feature information set corresponding to the initial cluster center point Counter j l =0,j h =0; Step (3.2), for the non-central electric vehicle V n , if V n The amount of electricity n (t) is greater than the power threshold e θ , then it is recorded as a high-power vehicle, j h =j h +1,j=j h ; Otherwise, it is recorded as a low-battery vehicle, j l =j l +1, j=j l ; Step (3.3), calculate V n The cost function of each center point is mapped to the group corresponding to the center point with the smallest cost function. For the kth * The set of all vehicles in a cluster; In step (3.4), after all electric vehicles are grouped, the center of the cluster is recalculated using the location information of the electric vehicles in the cluster: Step (3.5), at this time, if the cost function converges, the current cluster division result is output; if the cost function does not converge, the counter is reset to zero and steps (3.2)-(3.4) are repeated.

6. The vehicle-to-vehicle energy trading grouping method based on dynamic clustering and deep reinforcement learning according to claim 1 is characterized by: In step 4, in a transaction, to indicate the identity of the buyer and seller of the electric vehicle, To represent the buyer's vehicle that needs electricity, To express The seller vehicle that sells the electricity; Send energy request, Energy is sold at a fixed price based on the dynamic market price set by the system; energy supply between sellers and buyers is coordinated within the cluster based on charging efficiency, economic benefits, and communication quality factors; once an energy trading agreement is reached, The energy currency is transferred from its wallet to the seller based on the smart meter reading. Provided wallet address.

7. The vehicle-to-vehicle energy trading grouping method based on dynamic clustering and deep reinforcement learning according to claim 1 is characterized by: In step 5, ρ is the price per unit of energy in the system, η n (t) is the buyer's vehicle The energy purchased; buyers and sellers also incur other costs when conducting transactions, including unit management costs ψ h To compensate for hardware losses and operating costs, the seller's utility function is expressed as: Among them, the total management fee is related to the amount of electricity traded during the actual transaction process; For the buyer's electric car, it is expected that the received signal interference is smaller and the energy loss is smaller when a certain amount of energy currency is paid; in the buyer's utility function, the power of energy transmission will decline due to the increase in distance; the buyer's utility function is modeled as: in, Used to characterize from The quality of energy received at Related to the vehicle-to-vehicle energy transmission method, it is a decreasing function that changes with distance. β is the coefficient that converts energy quality into monetary rewards, and ρη n (t) is the buyer's acquisition of η n (t) Cost per unit of energy.

8. The vehicle-to-vehicle energy trading grouping method based on dynamic clustering and deep reinforcement learning according to claim 1 is characterized by: In step 6, the reward function makes adaptive decisions through deep reinforcement learning to maximize system performance and economic benefits; the objective function is expressed as: maxQ(S(t),A(t)) Among them, Q(S(t), A(t)) is the action value function of deep reinforcement learning, and the reward function at stept is: R(t)=R(S(t),A(t))=ω b U B (t)+ω s U S (t) Among them, U B (t) is the total economic utility of the buyer’s electric vehicle, U S (t) is the total economic utility of the seller’s electric vehicle, ω b ,ω s ∈(0,1] is the weight coefficient, ω b +ω s =1, reflecting the emphasis on the economic utility of buyers and sellers in the system optimization goal, and dynamically adjusted according to the actual system needs; the optimal action value function of the system, that is, the long-term reward of the system, is expressed as Where γ(t)∈(0,1] is the discount coefficient, which reflects the current trade-off between instantaneous rewards and future rewards. The network is trained until the total reward converges, and the trained model is used to make real-time optimal decisions on the energy trading vehicle grouping strategy.

Images