Electric vehicle charging scheduling method based on auction and multi-agent deep reinforcement learning

Abstract

The invention discloses an auction and multi-agent deep reinforcement learning-based electric vehicle charging scheduling method, and relates to the technical field of electric vehicle shared charging reservation scheduling. The method comprises the steps of providing an auction-based shared charging reservation system model; a sharing charging optimal user selection problem is formalized by taking maximization of social welfare as a target; an auction mechanism is introduced, and a new appointment user selection incentive mechanism is provided to calculate a winner set and the payment prices of winner; the system is further expanded, and the reservable time of the charging pile is divided into a plurality of time periods; a charging period selection algorithm based on a multi-agent deep reinforcement learning network is provided, and the objective is to maximize the utility of the charging of the electric vehicle user. According to the invention, the problems of too long waiting time and waste of charging pile resources caused by the fact that the electric vehicle users are gathered in the charging peak period in the existing charging reservation system can be solved, and meanwhile, a multi-period reservation and anti-strategy scheduling mechanism can be provided for the electric vehicle users.

Description

Technical Field

[0001] This invention relates to the technical field of electric vehicle shared charging reservation scheduling, and in particular to an electric vehicle charging scheduling method based on auction and multi-agent deep reinforcement learning. Background Technology

[0002] With the increasing number of electric vehicles, the demand for fast charging stations is growing rapidly. However, fast charging stations face challenges in resource allocation and scheduling optimization. Especially when multiple electric vehicle users are competing for charging stations, how to efficiently allocate charging stations to avoid excessively long waiting times while ensuring the efficient utilization of charging stations has become an urgent problem to be solved.

[0003] Existing research largely focuses on orderly charging, primarily utilizing intelligent queuing systems to reduce waiting times for electric vehicle users. Intelligent queuing systems employ a sliding window mechanism to collect data on vehicle traffic flow and charging pile status changes at charging stations, dynamically updating estimated waiting times. When multiple electric vehicles simultaneously request charging services, the algorithm comprehensively considers factors such as remaining battery power, reservation time, and distance from the charging station, determining the service order through a priority queue.

[0004] While charging reservation systems allow electric vehicle users to book charging times and locations in advance, most systems only support "instant reservation" mode, meaning electric vehicle users can only initiate a reservation when the charging need occurs. [3] This approach easily leads to queuing congestion during peak charging periods, failing to shorten the actual waiting time for electric vehicle users and potentially causing inefficient use of charging station resources and a decline in the user experience. A few systems that support future time slot reservations employ a rigid "unique allocation of minimum charging time slots" model, where the system assigns a unique electric vehicle user to each available minimum charging time slot. This model often results in resource waste, such as when an electric vehicle user's charging time is less than the available minimum charging time slot. Therefore, research on using auctions to solve the resource allocation problem of multiple electric vehicle users sharing charging stations under multiple reservation time slots is both practically significant and urgently needs to be addressed. Summary of the Invention

[0005] This invention addresses the technical problems existing in the prior art by providing an electric vehicle charging scheduling method based on auction and multi-agent deep reinforcement learning. This method solves the problem of excessively long waiting times and wasted charging pile resources caused by the rigid mode of "instant reservation" and "unique allocation of minimum charging time".

[0006] To solve the above-mentioned technical problems, the present invention provides the following technical solution:

[0007] This invention provides an electric vehicle charging scheduling method based on auction and multi-agent deep reinforcement learning, comprising: an auction-based electric vehicle shared charging reservation system model consisting of multiple electric vehicle users and a charging station; the charging station has several charging piles, each charging pile having an arbitrary maximum charging capacity; electric vehicle users have different requested charging amounts and unit electricity bidding prices; electric vehicle users submit bids to the charging station to reserve charging; the auction-based electric vehicle shared charging reservation system allows multiple electric vehicle users to share charging piles for serial charging.

[0008] The optimal user selection problem for shared charging is formalized with the objective function of maximizing social welfare.

[0009] Based on the optimal user selection problem of shared charging, a reservation user selection incentive mechanism is proposed to calculate the set of winners for each charging station and the price paid by electric vehicle users.

[0010] Based on the aforementioned user selection incentive mechanism, the available time for charging piles is divided into multiple time periods. Multi-agent deep reinforcement learning is used to help electric vehicle users decide on charging time periods. A charging time period selection algorithm based on a multi-agent deep reinforcement learning network is proposed, with the goal of maximizing the utility of electric vehicle users for this charging session.

[0011] As a preferred embodiment of the electric vehicle charging scheduling method based on auction and multi-agent deep reinforcement learning described in this invention, the auction-based electric vehicle shared charging reservation system model includes:

[0012] An auction-based shared charging reservation system consisting of multiple electric vehicle users and a single charging station. The charging station contains... A set of charging piles is represented as: Charging piles The maximum rechargeable capacity is And the unit electricity cost is ;have The set of electric vehicle users who have made charging reservations is denoted as . Assuming electric vehicle users The bid submitted to the charging station is ,in, Electric vehicle users The requested charging amount, Electric vehicle users unit electricity bidding price ; The set of bids from electric vehicle users is represented as The set of maximum charging capacities for charging stations is represented as follows: After bidding, the auction-based shared charging reservation system will determine the set of auction winners according to the allocation rules of this invention. and the winner Actual price per unit of electricity paid It is required to allocate charging stations. The total amount of charging requested by electric vehicle users shall not exceed the maximum charging capacity of the charging station. Charging piles The set of winners is represented as Each winner can only be assigned to one charging station. The set of winners is represented as follows: The auction results will be sent to each electric vehicle user, who will then charge their vehicle at the assigned charging station and pay the charging fee.

[0013] electric vehicle users Utility is defined as the difference between the value gained from charging and the actual value paid. For electric vehicle users... The utility can be calculated as follows:

[0014] (1)

[0015] in, Indicates electric vehicle users The value gained per unit of electricity; in particular, the utility of electric vehicle users who do not win the auction will be 0, because they neither gain the value of charging nor pay for charging.

[0016] Please note that electric vehicle users unit electricity bidding price Possibly related to electric vehicle users Value obtained per unit of electricity Different, because electric vehicle users They are selfish individuals, and The information is private and unknown to other electric vehicle users and the platform; therefore, electric vehicle users... By changing the bid price per unit of electricity To maximize one's own utility;

[0017] The utility of a charging station is expressed as:

[0018] (2)

[0019] Social welfare is expressed as:

[0020] (3)

[0021] As a preferred embodiment of the electric vehicle charging scheduling method based on auctions and multi-agent deep reinforcement learning for optimal user selection in shared charging with the objective function of maximizing social welfare, as described in this invention, the problem includes:

[0022] The formal expression of the optimal user selection problem for shared charging is:

[0023] (4)

[0024] (4-1)

[0025] (4-2)

[0026] Constraint (4-1) refers to charging piles The sum of the charging requests from the assigned electric vehicle users shall not exceed the maximum charging capacity of the charging station. Constraint (4-2) states that there is no overlap between the sets of winners of different charging stations;

[0027] Although electric vehicle users The true value of a unit of electricity Only electric vehicle users themselves know this, but this invention will prove that, in the incentive mechanism designed in this invention, electric vehicle users... Submission does not equal true value Quotation It does not increase its own utility; therefore, in maximizing total social welfare in the incentive mechanism designed in this invention, it uses... express ;

[0028] This invention addresses the optimal user selection problem in shared charging, aiming to maximize social welfare and simultaneously optimize the utility of both electric vehicle users and charging stations. Furthermore, this invention allows multiple electric vehicle users to share a charging station for sequential charging, resolving the resource waste caused by the rigid model of exclusive charging time slots in existing charging reservation systems.

[0029] As a preferred embodiment of the electric vehicle charging scheduling method based on auction and multi-agent deep reinforcement learning described in this invention, the incentive mechanism for the reservation user selection includes:

[0030] Functions selected by the user and payment decision function The incentive mechanism for users to make reservations ;function Calculate the set of winners for that reservation period. ,function Calculate the price per unit of electricity that each electric vehicle user needs to pay;

[0031] The incentive mechanism for users making reservations includes the following steps:

[0032] S31: Input parameters include the bid set Electric vehicle user group Maximum charging capacity set Charging station collection ;

[0033] S32: Based on the bid price Sort the electric vehicle users in a non-increasing order to obtain a sequence set of electric vehicle users. According to the maximum charging capacity of the charging pile The charging piles are sorted in non-decreasing order to obtain a charging pile sequence set. ;

[0034] S33: Initialize the remaining charging station set ; make charging piles Winners Collection and the provisional winner Empty, and simultaneously set the charging station Temporary power allocation =0; and initialize electric vehicle users. ;

[0035] S34: For electric vehicle users Looking for charging stations , If no charging station can be found If a charging station is found, proceed to step S36; Then proceed to the next step;

[0036] S35: If electric vehicle users Requested charging amount and charging station The sum of the temporarily allocated electricity is less than that of the charging pile. Half the capacity, that is Then it will be temporarily decided to include electric vehicle users Assigned to charging station , that is to say ;

[0037] If electric vehicle users Requested charging amount and charging station The sum of the temporary allocated electricity is greater than or equal to that of the charging pile. Half the capacity and less than or equal to the charging pile Capacity, i.e. Then determine the charging station Provisional winners With electric vehicle users The union of these sets represents the final set of winners. and charging piles From the set Remove from the middle, that is ;

[0038] If electric vehicle users Requested charging amount and charging station The sum of the temporarily allocated electricity is greater than that of the charging pile. The capacity and the remaining charging pile collection also have more charging piles than Large-capacity charging piles , Then upgrade the charging pile. The winners are a group of electric vehicle users , charging pile From the set Remove from the middle, and define the charging pile at the same time. ,Right now If charging station With charging piles The sum of the temporarily allocated electricity is less than that of the charging pile. If half the capacity is used, then the charging stations will be upgraded. The temporary allocation of electricity and the provisional set of winners, i.e. If charging station With charging piles The sum of the temporary allocated electricity is greater than or equal to that of the charging pile. If half the capacity is used, then the charging stations will be upgraded. The winners will gather and the charging stations will be available. From the set Remove from the middle, that is ;

[0039] If electric vehicle users Requested charging amount and charging station The sum of the temporarily allocated electricity is greater than that of the charging pile. The capacity and the remaining set of charging stations does not contain charging stations with the same capacity. Large-capacity charging piles , charging pile From the set Remove from the middle, and compare electric vehicle users. The total value of charging and the amount temporarily allocated to charging piles The sum of the total charging value for all electric vehicle users ultimately determines the value of the charging station. Allocate charging demand with high total charging value and upgrade charging stations. The set of winners, the total value of electric vehicle user charging is defined as the product of the bid price and the requested charging amount;

[0040] S36: Order Considering the next electric vehicle user ,like If so, proceed to step S34; otherwise, proceed to step S37.

[0041] S37: If the set of remaining charging stations is not empty at this time Then iterate through the remaining charging piles in the set. ,make equal And let If the remaining charging station set is empty at this time, then let ;

[0042] S38: Traversing electric vehicle users Initialize electric vehicle users The unit electricity price for payment is 0; initialize electric vehicle users.

[0043] S39: Order As an electric vehicle user The new bid price will include the electric vehicle user sequence set. The bid prices are assigned to the bidders in ascending order. Update electric vehicle users The bid set obtained after the new bid price And on Perform steps S31-S37 until the first electric vehicle user is found. The new bid price for becoming the auction winner as an electric vehicle user Key payment price ;

[0044] S310: Consider the next electric vehicle user ,like Then proceed to the next step; if The process then ends;

[0045] S311: If If so, proceed to step S39; otherwise, proceed to step S310.

[0046] The incentive mechanism satisfies the requirements of computational efficiency, individual rationality, authenticity, and a 1 / 2 approximate ratio. It is a strategic scheduling mechanism proposed by electric vehicle users, guiding them to truthfully disclose information. This achieves efficient allocation of resources to the highest-value users while effectively constraining users' strategic bidding behavior through bidding, preventing them from obtaining charging resources by submitting false bids. Therefore, it can guarantee the rights and interests of electric vehicle users. Quotation This equals the actual value that electric vehicle users receive per unit of electricity. .

[0047] As a preferred embodiment of the electric vehicle charging scheduling method based on auction and multi-agent deep reinforcement learning described in this invention, the available time for charging stations is divided into multiple time slots, and electric vehicle users can choose to reserve future charging time slots according to their own needs, forming a set of time slots. , For the number of time periods; to allow eligible electric vehicle users The earliest time to start charging for the trip is Charging period Estimation of unit electricity consumption For and the latest tolerable start charging time is Electric vehicle users have different valuations of a unit of electricity depending on the charging period. During charging period The estimated value per unit of electricity is known only to electric vehicle users and is expressed as:

[0048] (5)

[0049] (5-1)

[0050] electric vehicle users The bid submitted to the charging station is , Indicates electric vehicle users Scheduled charging times Electric vehicle users The amount of charge one is willing to make. Electric vehicle users During charging period The bid price that the unit of electricity is willing to pay grid;

[0051] electric vehicle users During the period The utility is expressed as:

[0052] (6)

[0053] in, Indicates electric vehicle users During the period The price per unit of electricity determined after winning the auction. Indicates time period The collection of auction winners;

[0054] electric vehicle users The user selects a unique charging time slot and sends a reservation bid to the charging station. After collecting all the bids from all electric vehicles, the charging station uses an incentive mechanism to determine whether each electric vehicle user wins the auction and the corresponding payment price. This process is modeled as a Markov decision process. In this context, the key symbol for a Markov process is... , , , respectively representing time periods Charging environment conditions at the time, electric vehicle users during the time period The actions taken, and the rewards that electric vehicle users receive after taking those actions.

[0055] The objective function for maximizing the utility of this charging session for electric vehicle users is expressed as:

[0056] (7)

[0057] (7-1)

[0058] (7-2)

[0059] in, For electric vehicle users During the period The rewards obtained Indicates time period charging pile The set of auction winners. Constraint (7-1) refers to the time period. charging pile The sum of the charging requests from the assigned electric vehicle users shall not exceed the maximum charging capacity of the charging station. Constraint (7-2) refers to the time period There is no overlap between the winning sets of different charging stations.

[0060] Treating each electric vehicle user as an agent, a charging time selection algorithm based on a multi-agent deep reinforcement learning network is proposed.

[0061] The charging time selection algorithm based on multi-agent deep reinforcement learning network includes the following steps:

[0062] S41: Input parameters include time period sets and the number of electric vehicle users Initialize electric vehicle users replay memory For capacity For electric vehicle users Randomly initialized value evaluation neural network The participants Number and target Value Network parameters and the probability related to the action selection , , The initial number of iterations is... , ;

[0063] S42 Initialization Period ;

[0064] S43: Initialize Electric Vehicle Users ;

[0065] S44: Observing Electric Vehicle Users The current state, if electric vehicle users The available time slots do not include Or, if the electric vehicle user has already submitted a bid at another time, then the electric vehicle user... Bidding is abandoned during this period; otherwise, a random number is generated arbitrarily. And according to the parameters The relationship determines how to choose the current action. If electric vehicle users Choose to bid in the current time period =1, then the auction result is obtained according to the incentive mechanism, and the utility of the electric vehicle user is calculated as the reward. A reward will be given if no bid is submitted during the current period. ;get The state of the time period, and Recorded in middle.

[0066] S45: If Calculate the cumulative reward function ,in, This is a discount factor used to balance the weights of long-term and immediate rewards, with a value range of (0,1]. , ; Utilizing value assessment networks Parameters in For loss function Perform gradient descent;

[0067] S46: ,like Then proceed to step S44; otherwise proceed to step S47.

[0068] S47: ,like Then jump to S43; if Then, based on the design, determine whether to use a value assessment network for the current time period. parameters Reset target network parameters Reset if necessary;

[0069] S48: Order ,like If, then jump to S42; The process then ends;

[0070] This invention divides the reservable time of charging stations into multiple time periods. Under the condition of maximizing social welfare, it allows electric vehicle users to choose a suitable charging time period according to their own needs, thereby improving the utility of electric vehicle users during charging. It can solve the problem of excessively long waiting times caused by electric vehicle users gathering during peak charging periods due to instant reservations in the existing charging reservation system.

[0071] An electronic device includes a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor is capable of executing the program to implement the electric vehicle charging scheduling method based on auction and multi-agent deep reinforcement learning.

[0072] A computer-readable storage medium storing computer instructions thereon, which, when executed by a processor, implement the electric vehicle charging scheduling method based on auction and multi-agent deep reinforcement learning.

[0073] Compared with the prior art, the beneficial effects of the present invention are as follows:

[0074] This invention addresses the problem in existing charging reservation systems where the rigid model of "instant reservation" and "unique allocation of the minimum charging time" leads to excessively long waiting times and wasted charging pile resources due to electric vehicle users congregating during peak charging hours. It improves the utility of electric vehicle users during charging while maximizing social welfare.

[0075] In this invention, the incentive mechanism for reservation users is a multinomial-time algorithm. The set of electric vehicle users is... Sort the bids in non-increasing order. The time complexity is... , is the set of maximum charging capacity The time complexity of sorting according to the non-decreasing order of maximum charge is O(n). .because Therefore, the time complexity of sorting is O(n log n). The process of determining whether an electric vehicle user is the auction winner runs... Next, the most complex case in this process needs to be traversed. Each charging station requires The time. Therefore, the user selects the function. The time complexity is .

[0076] The initial price paid by each electric vehicle user needs to be set. The time, after which it is determined The payment price for each electric vehicle user will be charged at most [amount]. Secondary user selection function, therefore payment decision function The time complexity is .

[0077] Therefore, the time complexity of users choosing the incentive mechanism is... In other words, the incentive mechanism for users to make reservations is a multinomial-time algorithm.

[0078] In this invention, the incentive mechanism for users making reservations satisfies individual rationality. In the payment decision function... In China, we are looking for electric vehicle users. When making a key offer, the new offer satisfy At the same time, according to the requirements in the system model Then the new quote The range is Therefore, the final price paid by the electric vehicle user Electric vehicle users The utility Then the utility of electric vehicle users is non-negative. If electric vehicle users If not the winner of the auction, then the electric vehicle user The utility Therefore, the incentive mechanism for users to make reservations satisfies individual rationality.

[0079] In this invention, the incentive mechanism for pre-booking users' selection satisfies the principle of authenticity. Before proving the authenticity of the incentive mechanism for pre-booking users' selection, we first introduce Myerson's theorem:

[0080] An auction mechanism is real if and only if:

[0081] The selection criteria are monotonous: if electric vehicle users Through quotation To win the auction, it must also be through bidding. Win the auction.

[0082] The price paid by each winner These are all key quotes: if electric vehicle users The bid was lower than this price Electric vehicle users You won't win the auction.

[0083] In the incentive mechanism for selecting reservation users, electric vehicle users are first grouped together. The selection rule is monotonic because the prices are sorted in a non-increasing order. This is because electric vehicle users... A higher price would not deter electric vehicle users. The position in the sorting is shifted backward, so as long as electric vehicle users Quotation Winning the auction means that the electric vehicle user New bid price At that time, electric vehicle users They will also become the winners of the auction.

[0084] As the algorithm's new quote gradually increases... The value obtained is the first one that can enable electric vehicle users Winning the auction means becoming the winner's bid, if Then the new set of winners This does not include electric vehicle users. electric vehicle users Unable to win the auction, so electric vehicle users The price paid The key is the timing of the quote. Therefore, the incentive mechanism for pre-booking users satisfies Myerson's theorem, meaning that the incentive mechanism for pre-booking users satisfies the requirement of authenticity.

[0085] In this invention, the incentive mechanism for selecting reservation users is an algorithm that satisfies an approximate 1 / 2 ratio. The optimal user selection problem for shared charging is essentially a multi-knapsack problem, and the incentive mechanism for selecting reservation users is an approximate algorithm based on a greedy algorithm. Assuming... This indicates the optimal solution to the problem. Social welfare, with This indicates that the incentive mechanism for pre-registered users yields an approximate solution. Social welfare.

[0086] electric vehicle users according to The electric vehicle user sequence set is obtained by non-increasing sequential sorting. ,but When a user makes a reservation and selects an incentive mechanism to determine the charging station... After the electric vehicle users are finally assigned charging stations The maximum possible loss value is used To express.

[0087] Scenario 1:

[0088] If the user selects a function middle The condition for non-empty is met, such that... Then it satisfies electric vehicle users All have been allocated, therefore ,in this case .

[0089] Scenario 2:

[0090] When backpack Final loading items By comparing electric vehicle users The total value of charging and the amount temporarily allocated to charging piles The value is determined by the sum of the total charging value for all electric vehicle users. Therefore, in this case .

[0091] Scenario 3:

[0092] When backpack If the final contents are different from those in case 2, then... Assume that the index of the next item selected by the user through the reservation incentive mechanism is... , ,in this case .

[0093] The conclusion can be drawn as follows:

[0094]

[0095] Therefore, the ratio between the approximate solution and the optimal solution is 1 / 2, meaning that the incentive mechanism for pre-booking users is an algorithm that satisfies the approximate ratio of 1 / 2. Attached Figure Description

[0096] To more clearly illustrate the technical solutions of the embodiments of the present invention, the drawings used in the description of the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort. Wherein:

[0097] Figure 1 This is a schematic diagram of the method flow of an electric vehicle charging scheduling method based on auction and multi-agent deep reinforcement learning according to an embodiment of the present invention;

[0098] Figure 2 This is a schematic diagram of the reservation user selection incentive mechanism of an electric vehicle charging scheduling method based on auction and multi-agent deep reinforcement learning, as described in one embodiment of the present invention.

[0099] Figure 3 This is a schematic diagram of the charging time slot selection algorithm based on a multi-agent deep reinforcement learning network, which is an embodiment of the electric vehicle charging scheduling method based on auction and multi-agent deep reinforcement learning according to one embodiment of the present invention. Detailed Implementation

[0100] To make the above-mentioned objects, features, and advantages of the present invention more apparent and understandable, specific embodiments of the present invention will be described in detail below with reference to the accompanying drawings. Obviously, the described embodiments are only a part of the embodiments of the present invention, and not all of them. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort should fall within the protection scope of the present invention.

[0101] Many specific details are set forth in the following description in order to provide a full understanding of the invention. However, the invention may also be practiced in other ways different from those described herein, and those skilled in the art can make similar extensions without departing from the spirit of the invention. Therefore, the invention is not limited to the specific embodiments disclosed below.

[0102] Secondly, the term "one embodiment" or "embodiment" as used herein refers to a specific feature, structure, or characteristic that may be included in at least one implementation of the present invention. The phrase "in one embodiment" appearing in different places in this specification does not necessarily refer to the same embodiment, nor is it a single or selective embodiment that is mutually exclusive with other embodiments.

[0103] Example 1

[0104] Reference Figures 1-3 This is one embodiment of the present invention, which provides a method for electric vehicle charging scheduling based on auctions and multi-agent deep reinforcement learning, such as... Figure 1 As shown, it includes the following steps:

[0105] S1: A shared electric vehicle charging reservation system model based on auction consists of multiple electric vehicle users and a charging station. The charging station has several charging piles, each with an arbitrary maximum charging capacity. Electric vehicle users have different requested charging capacities and bid prices per unit of electricity. Electric vehicle users submit bids to the charging station to make charging reservations. The shared electric vehicle charging reservation system based on auction allows multiple electric vehicle users to share charging piles for serial charging.

[0106] 2. Furthermore, the auction-based electric vehicle shared charging reservation system model includes:

[0107] An auction-based shared charging reservation system consisting of multiple electric vehicle users and a single charging station. The charging station contains... A set of charging piles is represented as: Charging piles The maximum rechargeable capacity is And the unit electricity cost is ;have The set of electric vehicle users who have made charging reservations is denoted as . Assuming electric vehicle users The bid submitted to the charging station is ,in, Electric vehicle users The requested charging amount, Electric vehicle users unit electricity bidding price ; The set of bids from electric vehicle users is represented as The set of maximum charging capacities for charging stations is represented as follows: After bidding, the auction-based shared charging reservation system will determine the set of auction winners according to the allocation rules of this invention. and the winner Actual price per unit of electricity paid It is required to allocate charging stations. The total amount of charging requested by electric vehicle users shall not exceed the maximum charging capacity of the charging station. Charging piles The set of winners is represented as Each winner can only be assigned to one charging station. The set of winners is represented as follows: The auction results will be sent to each electric vehicle user, who will then charge their vehicle at the assigned charging station and pay the charging fee.

[0108] electric vehicle users Utility is defined as the difference between the value gained from charging and the actual value paid. For electric vehicle users... The utility can be calculated as follows:

[0109] (1)

[0110] in, Indicates electric vehicle users The value gained per unit of electricity; in particular, the utility of electric vehicle users who do not win the auction will be 0, because they neither gain the value of charging nor pay for charging.

[0111] Please note that electric vehicle users unit electricity bidding price Possibly related to electric vehicle users Value obtained per unit of electricity Different, because electric vehicle users They are selfish individuals, and The information is private and unknown to other electric vehicle users and the platform; therefore, electric vehicle users... By changing the bid price per unit of electricity To maximize one's own utility;

[0112] The utility of a charging station is expressed as:

[0113] (2)

[0114] Social welfare is expressed as:

[0115] (3)

[0116] S2: Formalize the optimal user selection problem for shared charging with the objective function of maximizing social welfare;

[0117] Furthermore, the formalized shared charging optimal user selection problem, with maximizing social welfare as the objective function, includes:

[0118] The formal expression of the optimal user selection problem for shared charging is:

[0119] (4)

[0120] (4-1)

[0121] (4-2)

[0122] Constraint (4-1) refers to charging piles The sum of the charging requests from the assigned electric vehicle users shall not exceed the maximum charging capacity of the charging station. Constraint (4-2) states that there is no overlap between the sets of winners of different charging stations;

[0123] Although electric vehicle users The true value of a unit of electricity Only electric vehicle users themselves know this, but this invention will prove that, in the incentive mechanism designed in this invention, electric vehicle users... Submission does not equal true value Quotation It does not increase its own utility; therefore, in maximizing total social welfare in the incentive mechanism designed in this invention, it uses... express .

[0124] S3: Based on the optimal user selection problem of shared charging, a reservation user selection incentive mechanism is proposed to calculate the winner set of each charging pile and the price paid by electric vehicle users;

[0125] Furthermore, the incentive mechanism for users making reservations includes:

[0126] Functions selected by the user and payment decision function The incentive mechanism for users to make reservations ;function Calculate the set of winners for that reservation period. ,function Calculate the price per unit of electricity that each electric vehicle user needs to pay;

[0127] The incentive mechanism for users making reservations includes the following steps:

[0128] S31: Input parameters include the bid set Electric vehicle user group Maximum charging capacity set Charging station collection ;

[0129] S32: Based on the bid price Sort the electric vehicle users in a non-increasing order to obtain a sequence set of electric vehicle users. According to the maximum charging capacity of the charging pile The charging piles are sorted in non-decreasing order to obtain a charging pile sequence set. ;

[0130] S33: Initialize the remaining charging station set ; make charging piles Winners Collection and the provisional winner Empty, and temporarily allocate power. =0; and initialize electric vehicle users. ;

[0131] S34: For electric vehicle users Looking for charging stations , If no charging station can be found If a charging station is found, proceed to step S36; Then proceed to the next step;

[0132] S35: If electric vehicle users Requested charging amount and charging station The sum of the temporarily allocated electricity is less than that of the charging pile. Half the capacity, that is Then it will be temporarily decided to include electric vehicle users Assigned to charging station , that is to say ;

[0133] If electric vehicle users Requested charging amount and charging station The sum of the temporary allocated electricity is greater than or equal to that of the charging pile. Half the capacity and less than or equal to the charging pile Capacity, i.e. Then determine the charging station Provisional winners With electric vehicle users The union of these sets represents the final set of winners. and charging piles From the set Remove from the middle, that is ;

[0134] If electric vehicle users Requested charging amount and charging station The sum of the temporarily allocated electricity is greater than that of the charging pile. The capacity and the remaining charging pile collection also have more charging piles than Large-capacity charging piles , Then upgrade the charging pile. The winners are a group of electric vehicle users , charging pile From the set Remove from the middle, and define the charging pile at the same time. ,Right now If charging station With charging piles The sum of the temporarily allocated electricity is less than that of the charging pile. If half the capacity is used, then the charging stations will be upgraded. The temporary allocation of electricity and the provisional set of winners, i.e. If charging station With charging piles The sum of the temporary allocated electricity is greater than or equal to that of the charging pile. If half the capacity is used, then the charging stations will be upgraded. The winners will gather and the charging stations will be available. From the set Remove from the middle, that is ;

[0135] If electric vehicle users Requested charging amount and charging station The sum of the temporarily allocated electricity is greater than that of the charging pile. The capacity and the remaining set of charging stations does not contain charging stations with the same capacity. Large-capacity charging piles , charging pile From the set Remove from the middle, and compare electric vehicle users. The total value of charging and the amount temporarily allocated to charging piles The sum of the total charging value for all electric vehicle users ultimately determines the value of the charging station. Allocate charging demand with high total charging value and upgrade charging stations. The set of winners, the total value of electric vehicle user charging is defined as the product of the bid price and the requested charging amount;

[0136] S36: Order Considering the next electric vehicle user ,like If so, proceed to step S34; otherwise, proceed to step S37.

[0137] S37: If the set of remaining charging stations is not empty at this time Then iterate through the remaining charging piles in the set. ,make equal And let If the remaining charging station set is empty at this time, then let ;

[0138] S38: Traversing electric vehicle users Initialize electric vehicle users The unit electricity price for payment is 0; initialize electric vehicle users.

[0139] S39: Order As an electric vehicle user The new bid price will include the electric vehicle user sequence set. The bid prices are assigned to the bidders in ascending order. Update electric vehicle users The bid set obtained after the new bid price And on Perform steps S31-S37 until the first electric vehicle user is found. The new bid price for becoming the auction winner as an electric vehicle user Key payment price ;

[0140] S310: Consider the next electric vehicle user ,like Then proceed to the next step; if The process then ends;

[0141] S311: If If so, proceed to step S39; otherwise, proceed to step S310.

[0142] The incentive mechanism satisfies the requirements of computational validity, individual rationality, authenticity, and a 1 / 2 approximate ratio; therefore, electric vehicle users... Quotation This equals the actual value that electric vehicle users receive per unit of electricity. .

[0143] S4: Based on the aforementioned user selection incentive mechanism, the available time for charging piles is divided into multiple time periods. Multi-agent deep reinforcement learning is used to make decisions on charging time periods for electric vehicle users. A charging time period selection algorithm based on a multi-agent deep reinforcement learning network is proposed, with the goal of maximizing the utility of electric vehicle users for this charging session.

[0144] Furthermore, the available charging time slots are divided into multiple time periods, allowing electric vehicle users to choose and reserve future charging times based on their needs. , For the number of time periods; to allow eligible electric vehicle users The earliest time to start charging for the trip is Charging period Estimation of unit electricity consumption For and the latest tolerable start charging time is Electric vehicle users have different valuations of a unit of electricity depending on the charging period. During charging period The estimated value per unit of electricity is known only to electric vehicle users and is expressed as:

[0145] (5)

[0146] (5-1)

[0147] electric vehicle users The bid submitted to the charging station is , Indicates electric vehicle users Scheduled charging times Electric vehicle users The amount of charge one is willing to make. Electric vehicle users During charging period The bid price that the unit of electricity is willing to pay grid;

[0148] electric vehicle users During the period The utility is expressed as:

[0149] (6)

[0150] in, Indicates electric vehicle users During the period The price per unit of electricity determined after winning the auction. Indicates time period The collection of auction winners;

[0151] electric vehicle users The user selects a unique charging time slot and sends a reservation bid to the charging station. After collecting all the bids from all electric vehicles, the charging station uses an incentive mechanism to determine whether each electric vehicle user wins the auction and the corresponding payment price. This process is modeled as a Markov decision process. In this context, the key symbol for a Markov process is... , , , respectively representing time periods Charging environment conditions at the time, electric vehicle users during the time period The actions taken, and the rewards that electric vehicle users receive after taking those actions.

[0152] The objective function for maximizing the utility of this charging session for electric vehicle users is expressed as:

[0153] (7)

[0154] (7-1)

[0155] (7-2)

[0156] in, For electric vehicle users During the period The rewards obtained Indicates time period charging pile The set of auction winners. Constraint (7-1) refers to the time period. charging pile The sum of the charging requests from the assigned electric vehicle users shall not exceed the maximum charging capacity of the charging station. Constraint (7-2) refers to the time period There is no overlap between the winning sets of different charging stations.

[0157] Treating each electric vehicle user as an agent, a charging time selection algorithm based on a multi-agent deep reinforcement learning network is proposed.

[0158] The charging time selection algorithm based on multi-agent deep reinforcement learning network includes the following steps:

[0159] S41: Input parameters include time period sets and the number of electric vehicle users Initialize electric vehicle users replay memory For capacity For electric vehicle users Randomly initialized value evaluation neural network The participants Number and target Value Network parameters and the probability related to the action selection , , The initial number of iterations is... , ;

[0160] S42 Initialization Period ;

[0161] S43: Initialize Electric Vehicle Users ;

[0162] S44: Observing Electric Vehicle Users The current state, if electric vehicle users The available time slots do not include Or, if the electric vehicle user has already submitted a bid at another time, then the electric vehicle user... Bidding is abandoned during this period; otherwise, a random number is generated arbitrarily. And according to the parameters The relationship determines how to choose the current action. If electric vehicle users Choose to bid in the current time period =1, then the auction result is obtained according to the incentive mechanism, and the utility of the electric vehicle user is calculated as the reward. A reward will be given if no bid is submitted during the current period. ;get The state of the time period, and Recorded in middle.

[0163] S45: If Calculate the cumulative reward function ,in, This is a discount factor used to balance the weights of long-term and immediate rewards, with a value range of (0,1]. , ; Utilizing value assessment networks Parameters in For loss function Perform gradient descent;

[0164] S46: ,like Then proceed to step S44; otherwise proceed to step S47.

[0165] S47: ,like Then jump to S43; if Then, based on the design, determine whether to use a value assessment network for the current time period. parameters Reset target network parameters Reset if necessary;

[0166] S48: Order ,like If, then jump to S42; The process then ends.

[0167] Example 2

[0168] Referring to Tables 1 and 2, an embodiment of the present invention provides an electric vehicle charging scheduling method based on auction and multi-agent deep reinforcement learning. To verify its beneficial effects, a scientific demonstration is conducted through specific implementation methods and implementation results.

[0169] The specific details of this embodiment are as follows:

[0170] This embodiment takes a shared electric vehicle charging scenario as an example. The electric vehicle charging scheduling method based on auction and multi-agent deep reinforcement learning includes the following steps:

[0171] Table 1 Parameters of Charging Stations

[0172]

[0173] Table 2 Parameters for Electric Vehicle Users

[0174]

[0175] The relevant parameters are set as follows: cost per unit of electricity .

[0176] Step 1: Input parameters include the bid set Electric vehicle user group Maximum charging capacity set Charging station collection ;

[0177] Step 2: Based on the bid price Sort the electric vehicle users in a non-increasing order to obtain a sequence set of electric vehicle users. According to the maximum charging capacity of the charging pile The charging piles are sorted in non-decreasing order to obtain a charging pile sequence set. ;

[0178] Step 3: Initialize the remaining charging station set ; make charging piles Winners Collection and the provisional winner Empty, and temporarily allocate power. =0; and initialize ;

[0179] Step 4: Find a suitable charging station ;

[0180] Step 5: If electric vehicle users Requested charging amount and charging station The sum of the temporarily allocated electricity is less than that of the charging pile. Half the capacity Then, the user will be temporarily... Assigned to charging station ,Right now ;

[0181] If electric vehicle users Requested charging amount and charging station The sum of the temporary allocated electricity is greater than or equal to that of the charging pile. Half the capacity and less than or equal to the charging pile capacity Then determine the charging station Provisional winners With electric vehicle users The union of these sets represents the final set of winners. and charging piles From the set Remove from the middle, that is ;

[0182] If electric vehicle users Requested charging amount and charging station The sum of the temporarily allocated electricity is greater than that of the charging pile. The capacity and the remaining charging pile collection also have more charging piles than Large-capacity charging piles , Then upgrade the charging pile. The winners are a group of electric vehicle users , charging pile From the set Remove from the middle, and define the charging pile at the same time. ,Right now If charging station With charging piles The sum of the temporarily allocated electricity is less than that of the charging pile. If half the capacity is used, then the charging stations will be upgraded. The temporary allocation of electricity and the provisional set of winners, i.e. If charging station With charging piles The sum of the temporary allocated electricity is greater than or equal to that of the charging pile. If half the capacity is used, then the charging stations will be upgraded. The winners will gather and the charging stations will be available. From the set Remove from the middle, that is ;

[0183] If electric vehicle users Requested charging amount and charging station The sum of the temporarily allocated electricity is greater than that of the charging pile. The capacity and the remaining set of charging stations does not contain charging stations with the same capacity. Large-capacity charging piles , charging pile From the set Remove from the middle, and compare electric vehicle users. The total value of charging and the amount temporarily allocated to charging piles The sum of the total charging value for all electric vehicle users ultimately determines the value of the charging station. Allocate charging demand with high total charging value and upgrade charging stations. The set of winners, the total value of user charging is defined as the product of the bid price and the requested charging amount;

[0184] Step 6: Considering the next electric vehicle user ,like If so, proceed to step 4; otherwise, proceed to step 7.

[0185] Repeat steps 4-6 until all electric vehicle users have been visited; at this point... ;

[0186] Step 7: If the remaining charging station set is not empty at this time Then iterate through the remaining charging piles in the set. ,make equal And let If the remaining charging station set is empty at this time, then let ;

[0187] at this time ,therefore {1,2,3,4,5,6,7,8,9,10,11,12,13};

[0188] Step 8: Iterate through electric vehicle users Initialize user The unit price for electricity is 0; initialization is used. household;

[0189] Step 9: Let As an electric vehicle user The new bid price will include the electric vehicle user sequence set. The bid prices are assigned to the bidders in ascending order. Update electric vehicle users The bid set obtained after the new bid price And on Perform steps S31-S37 until the first electric vehicle user is found. The new bid price for becoming the auction winner as an electric vehicle user Key payment price ;

[0190] After calculation of electric vehicle users =1 Key payment price ;

[0191] Step 10: Consider the next electric vehicle user ,like Then proceed to the next step; if The redirection process then ends;

[0192] Step 11: If If so, proceed to step 9; otherwise, proceed to step 10.

[0193] Repeat steps 9-11 until all electric vehicle users have been visited; obtain The remaining electric vehicle users paid 0.

[0194] It should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention and are not intended to limit it. Although the present invention has been described in detail with reference to preferred embodiments, those skilled in the art should understand that modifications or equivalent substitutions can be made to the technical solutions of the present invention without departing from the spirit and scope of the technical solutions of the present invention, and all such modifications or substitutions should be covered within the scope of the claims of the present invention.

Claims

1. A method for scheduling electric vehicle charging based on auctions and multi-agent deep reinforcement learning, characterized in that, Includes the following steps: S1: A shared electric vehicle charging reservation system model based on auction consists of multiple electric vehicle users and a charging station. The charging station has several charging piles, each with an arbitrary maximum charging capacity. Electric vehicle users have different requested charging capacities and bid prices per unit of electricity. Electric vehicle users submit bids to the charging station to make charging reservations. The shared electric vehicle charging reservation system based on auction allows multiple electric vehicle users to share charging piles for serial charging. S2: Formalize the optimal user selection problem for shared charging with the objective function of maximizing social welfare; S3: Based on the optimal user selection problem of shared charging, a reservation user selection incentive mechanism is proposed to calculate the winner set of each charging pile and the price paid by electric vehicle users; S4: Based on the reservation user selection incentive mechanism, the reservation time of charging piles is divided into multiple time periods. Multi-agent deep reinforcement learning is used to make decisions on charging time periods for electric vehicle users. A charging time period selection algorithm based on multi-agent deep reinforcement learning network is proposed, with the goal of maximizing the utility of electric vehicle users for this charging.

2. The electric vehicle charging scheduling method based on auction and multi-agent deep reinforcement learning as described in claim 1, characterized in that, In S1, the auction-based electric vehicle shared charging reservation system model includes: An auction-based shared charging reservation system consisting of multiple electric vehicle users and a charging station, where the charging station has... A set of charging piles is represented as: Charging piles The maximum rechargeable capacity is And the unit electricity cost is ;have The set of electric vehicle users who have made charging reservations is denoted as . Assuming electric vehicle users The bid submitted to the charging station is ,in, Electric vehicle users The requested charging amount, Electric vehicle users unit electricity bidding price ; The set of bids from electric vehicle users is represented as The set of maximum charging capacities for charging stations is represented as follows: After bidding, the auction-based shared charging reservation system will determine the set of auction winners according to the allocation rules of this invention. and the winner Actual price per unit of electricity paid It is required to allocate charging stations. The total amount of charging requested by electric vehicle users shall not exceed the maximum charging capacity of the charging station. Charging piles The set of winners is represented as Each winner can only be assigned to one charging station, and the set of charging station winners is represented as follows: The auction results will be sent to each electric vehicle user, who will then charge their vehicle at the assigned charging station and pay the charging fee. electric vehicle users Utility is defined as the difference between the value gained from charging and the actual value paid. For electric vehicle users... The utility calculation is as follows: (1) in, Indicates electric vehicle users The value gained per unit of electricity; the utility of electric vehicle users who do not win the auction is 0, because they do not gain any value from charging or pay for charging. electric vehicle users unit electricity bidding price Possibly related to electric vehicle users Value obtained per unit of electricity Different, because electric vehicle users They are selfish individuals, and The information is private and unknown to other electric vehicle users and the platform; electric vehicle users By changing the bidding price per unit of electricity To maximize one's own utility; The utility of a charging station is expressed as: (2) Social welfare is expressed as: (3)。 3. The electric vehicle charging scheduling method based on auction and multi-agent deep reinforcement learning as described in claim 1, characterized in that, In step S2, the optimal user selection problem for shared charging is formalized with the objective function of maximizing social welfare, including: The formal expression of the optimal user selection problem for shared charging is: (4) (4-1) (4-2) Constraint (4-1) refers to charging piles The sum of the charging requests from the assigned electric vehicle users shall not exceed the maximum charging capacity of the charging station. Constraint (4-2) states that there is no overlap between the sets of winners of different charging stations.

4. The electric vehicle charging scheduling method based on auction and multi-agent deep reinforcement learning as described in claim 1, characterized in that, In S3, the incentive mechanism for users who make reservations includes: Functions selected by the user and payment decision function The incentive mechanism for users to make reservations ;function Calculate the set of winners for that reservation period. ,function Calculate the price per unit of electricity that each electric vehicle user needs to pay; The incentive mechanism for users making reservations includes the following steps: S31: Input parameters include the bid set Electric vehicle user group Maximum charging capacity set Charging station collection ; S32: Based on the bid price Sort the electric vehicle users in a non-increasing order to obtain a set of electric vehicle user sequences. According to the maximum charging capacity of the charging pile The charging piles are sorted in non-decreasing order to obtain a charging pile sequence set. ; S33: Initialize the remaining charging station set ; make charging piles Winners Collection and the provisional winner Empty, and simultaneously set the charging station Temporary power allocation =0; and initialize electric vehicle users. ; S34: For electric vehicle users Looking for charging stations , If no charging station can be found If a charging station is found, proceed to step S36; Then proceed to the next step; S35: If electric vehicle users Requested charging amount and charging station The sum of the temporarily allocated electricity is less than that of the charging pile. Half the capacity, that is Then it will be temporarily decided to include electric vehicle users Assigned to charging station , that is to say ; If electric vehicle users Requested charging amount and charging station The sum of the temporary allocated electricity is greater than or equal to that of the charging pile. Half the capacity and less than or equal to the charging pile Capacity, i.e. Then determine the charging station Provisional winners With electric vehicle users The union of these sets represents the final set of winners. and charging piles From the set Remove from the middle. Right now; If electric vehicle users Requested charging amount and charging station The sum of the temporarily allocated electricity is greater than that of the charging pile. The capacity and the remaining charging pile collection also have more charging piles than Large-capacity charging piles , Then upgrade the charging pile. The winners are a group of electric vehicle users , charging pile From the set Remove from the middle, and define the charging pile at the same time. ,Right now If charging station With charging piles The sum of the temporarily allocated electricity is less than that of the charging pile. If half the capacity is used, then the charging stations will be upgraded. The temporary allocation of electricity and the provisional set of winners, i.e. If charging station With charging piles The sum of the temporary allocated electricity is greater than or equal to that of the charging pile. If half the capacity is used, then the charging stations will be upgraded. The winners will gather and the charging stations will be available. From the set Remove from the middle, that is ; If electric vehicle users Requested charging amount and charging station The sum of the temporarily allocated electricity is greater than that of the charging pile. The capacity and the remaining set of charging stations does not contain charging stations with the same capacity. Large-capacity charging piles , charging pile From the set Remove from the middle, and compare electric vehicle users. The total value of charging and the amount temporarily allocated to charging piles The sum of the total charging value for all electric vehicle users ultimately determines the value of the charging station. Allocate charging demand with high total charging value and upgrade charging stations. The set of winners, the total value of electric vehicle user charging is defined as the product of the bid price and the requested charging amount; S36: Order Considering the next electric vehicle user ,like If so, proceed to step S34; otherwise, proceed to step S37. S37: If the set of remaining charging stations is not empty at this time Then iterate through the remaining charging piles in the set. ,make equal And let If the remaining charging station set is empty at this time, then let ; S38: Traversing electric vehicle users Initialize electric vehicle users The unit electricity price for payment is 0; initialize electric vehicle users. ; S39: Order As an electric vehicle user The new bid price will include the electric vehicle user sequence set. The bid prices are assigned to the bidders in ascending order. Update electric vehicle users The bid set obtained after the new bid price And on Perform steps S31-S37 until the first electric vehicle user is found. The new bid price for becoming the auction winner as an electric vehicle user Key payment price ; S310: Consider the next electric vehicle user ,like Then proceed to the next step; if The process then ends; S311: If If so, proceed to step S39; otherwise, proceed to step S310. The incentive mechanism satisfies the requirements of computational validity, individual rationality, authenticity, and a 1 / 2 approximate ratio; therefore, electric vehicle users... Quotation This equals the actual value that electric vehicle users receive per unit of electricity. .

5. The electric vehicle charging scheduling method based on auction and multi-agent deep reinforcement learning as described in claim 1 or 4, characterized in that: In S4, The available charging time slots are divided into multiple time periods, allowing electric vehicle users to select and reserve future charging times based on their needs. , For the number of time periods; to allow eligible electric vehicle users The earliest time to start charging for the trip is Charging period The estimated value per unit of electricity is And the latest tolerable start time for charging is Electric vehicle users have different valuations of a unit of electricity depending on the charging period. During charging period The estimated value per unit of electricity is known only to electric vehicle users and is expressed as: (5) (5-1) electric vehicle users The bid submitted to the charging station is , Indicates electric vehicle users Scheduled charging times Electric vehicle users The amount of charge one is willing to make. Electric vehicle users During charging period The bid price that the unit of electricity is willing to pay ; electric vehicle users During the period The utility is expressed as: (6) in, Indicates electric vehicle users During the period The price per unit of electricity determined after winning the auction. Indicates time period The collection of auction winners; electric vehicle users The user selects a unique charging time slot and sends a reservation bid to the charging station. After collecting all the bids from all electric vehicles, the charging station uses an incentive mechanism to determine whether each electric vehicle user wins the auction and the corresponding payment price. This process is modeled as a Markov decision process. In this context, the key symbol for a Markov process is... , , , respectively representing time periods Charging environment conditions at the time, electric vehicle users during the time period The actions taken, and the rewards that electric vehicle users receive after taking those actions; The objective function for maximizing the utility of this charging session for electric vehicle users is expressed as: (7) (7-1) (7-2) in, For electric vehicle users During the period The rewards obtained Indicates time period charging pile The set of auction winners, constraint (7-1) refers to the time period charging pile The sum of the charging requests from the assigned electric vehicle users shall not exceed the maximum charging capacity of the charging station. Constraint (7-2) refers to the time period There is no overlap between the winning sets of different charging stations.

6. The electric vehicle charging scheduling method based on auction and multi-agent deep reinforcement learning as described in claim 5, characterized in that: In S4, each electric vehicle user is treated as an intelligent agent, and a charging time selection algorithm based on a multi-agent deep reinforcement learning network is proposed. The charging time selection algorithm based on multi-agent deep reinforcement learning networks includes the following steps: S41: Input parameters include time period sets and the number of electric vehicle users Initialize electric vehicle users replay memory For capacity For electric vehicle users Randomly initialized value evaluation neural network The participants Number and target Value Network parameters and the probability related to the action selection , , The initial number of iterations is , ; S42 Initialization Period ; S43: Initialize Electric Vehicle Users ; S44: Observing Electric Vehicle Users The current state, if electric vehicle users The available time slots do not include Or, if the electric vehicle user has already submitted a bid at another time, then the electric vehicle user... Bidding is abandoned during this period; otherwise, a random number is generated arbitrarily. And according to the parameters The relationship determines how to choose the current action. If electric vehicle users Choose to bid in the current time period =1, then the auction result is obtained according to the incentive mechanism, and the utility of the electric vehicle user is calculated as the reward. A reward will be given if no bid is submitted during the current period. ;get The state of the time period, and Recorded in middle; S45: If Calculate the cumulative reward function ,in, This is a discount factor used to balance the weights of long-term and immediate rewards, with a value range of (0,1]. , ; Utilizing value assessment networks Parameters in For loss function Perform gradient descent; S46: ,like Then proceed to step S44; otherwise proceed to step S47. S47: ,like Then jump to S43; if Then, based on the design, determine whether to use a value assessment network for the current time period. parameters Reset target network parameters Reset if necessary; S48: Order ,like If, then jump to S42; The process then ends.

7. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the program, it implements the electric vehicle charging scheduling method based on auction and multi-agent deep reinforcement learning as described in any one of claims 1 to 6.

8. A computer-readable storage medium storing computer instructions thereon, characterized in that, When executed by the processor, the computer instructions implement the electric vehicle charging scheduling method based on auction and multi-agent deep reinforcement learning as described in any one of claims 1-6.

Images