A game learning-based electric vehicle bidirectional charging auxiliary decision-making method and system

Abstract

The application relates to a game learning-based electric vehicle bidirectional charging auxiliary decision method and system. In the method, after electric vehicle information is collected and electric vehicle network entry states are predicted, a game process based on belief learning between a micro-grid control center and an electric vehicle is carried out, optimal charging and discharging strategies of the vehicle are obtained, and finally charging and discharging behavior decision is carried out under the guidance of the strategies. In the game process, the micro-grid control center does not know the income function of the electric vehicle user, and the electric vehicle does not know the strategy space of the power grid; the optimal charging and discharging strategies of the electric vehicle and the corresponding power grid price of each period are calculated by taking the minimization of the total charging and discharging cost of the electric vehicle as the target. Compared with the prior art, the application can quickly and accurately guide the charging and discharging behavior decision of the electric vehicle supporting the V2G technology when entering the network, has the advantages of reducing the charging cost of the user, improving the satisfaction of the user, enhancing the reliability, accuracy, real-time performance, and realizing the peak load shifting of the power grid, etc.

Description

Technical Field

[0001] The present invention relates to the technical field of electric vehicles, and in particular to a game learning-based electric vehicle bidirectional charging auxiliary decision-making method and system. Background Art

[0002] To achieve the dual carbon goals, my country has proposed building a new power system dominated by renewable energy. However, renewable energy sources, such as wind and solar power, are subject to significant uncertainty and peak-shaving characteristics due to environmental and climate influences. Therefore, microgrids integrate various distributed power sources, loads, energy storage devices, and control devices into a miniature energy supply system, enabling smooth transitions between on-grid and off-grid modes. Simultaneously, driven by evolving trends in the power market, new business models such as integrated energy services, power aggregators, green electricity trading, and carbon trading have emerged.

[0003] Electric vehicles (EVs) have rapidly gained popularity in recent years due to their economic and low-carbon advantages. Vehicle-to-Grid (V2G) technology treats EVs as distributed energy storage devices with spatiotemporal characteristics, controls their charging and discharging behavior, and allows them to participate in grid scheduling, effectively improving the stability and economic efficiency of grid operations. As flexible loads, EVs often spend more time at charging stations than their batteries require to fully charge. By systematically regulating the charging and discharging behavior of large numbers of EVs, not only can the impact on the grid caused by large-scale, random access be avoided, but also a certain degree of peak load shifting can be achieved. However, due to the high randomness of EVs' access and exit times, and the uncertainty of their state of charge upon arrival at charging stations, current scheduling schemes cannot accurately predict EV travel patterns and demand, and therefore cannot guide EV users to make charging and discharging decisions quickly. Furthermore, existing charging and discharging guidance methods require a significant amount of time to wait for policy optimization results, increasing waiting times for EV users upon arrival at charging stations and reducing user satisfaction.

[0004] The invention patent with publication number CN110979083A discloses a bidirectional charging and discharging control system and method for electric vehicles. The method first collects images of electric vehicles by a video monitoring unit and identifies the identity information of the electric vehicles. Then, the demand information of the electric vehicles is collected by a bidirectional charging pile and transmitted to a micro-grid control center. The micro-grid control center optimizes the scheduling according to the time-of-use electricity price and consumption function in a preset charging period, and calculates the optimal charging and discharging strategy of the electric vehicles at the lowest cost through game calculation. Finally, the optimal charging and discharging strategy of the electric vehicles is transmitted to the bidirectional charging pile to guide the electric vehicles to charge or discharge in an orderly manner. However, this method only considers the interests of electric vehicle users and cannot achieve the effect of improving user satisfaction while realizing the "peak load shifting" of the power grid.

[0005] The invention patent with publication number CN117314111A discloses a cluster electric vehicle master-slave game optimization scheduling method, device and medium. The disclosed method constructs an electric vehicle charging demand prediction model based on road network and a master-slave game double-layer optimization model based on load aggregator and electric vehicle cluster. On the basis of considering the electricity use preferences of electric vehicle users, the method mobilizes the electric vehicle users to participate in demand response. However, this application only uses a single model to describe electric vehicle trip data and travel characteristics, and does not consider the influence of factors such as driver behavior, vehicle information difference, and prediction accuracy of future electric vehicle access on the prediction model. Since the time of electric vehicle access and exit from the power grid has strong randomness, and the state of charge when arriving at the charging station is uncertain, it is impossible to accurately predict the travel situation and demand of electric vehicles, and therefore it is impossible to guide electric vehicle users to make charging and discharging behavior decisions within a short period of time.

[0006] Currently, the related disclosed patents and methods have certain applications in optimizing the scheduling of electric vehicles with irregularity and randomness using support V2G technology, and realizing the peak load shifting of new energy micro-grid. However, the traditional game model requires the revenue functions of both parties as common knowledge, which increases the difficulty of information exchange. At the same time, in the face of actual large-scale complex urban road networks, there is still a lack of an efficient and reliable optimization scheduling method to assist electric vehicles in making optimal charging and discharging behavior decisions in view of the different state characteristics of electric vehicles and the randomness of participating in charging. SUMMARY

[0007] The purpose of the present invention is to overcome the shortcomings of the aforementioned prior art by providing a game-learning-based bidirectional charging decision-making assistance method and system for electric vehicles. This method addresses the characteristics of vehicle-to-grid interaction in existing urban scenarios, combines collaborative vehicle infrastructure systems with vehicle-to-grid technology, pre-collects the current initial status of electric vehicles in the coverage area, and predicts their state upon network access. Based on game learning, the method calculates the optimal charging and discharging strategy, thereby quickly and accurately guiding V2G-enabled electric vehicles to make charging and discharging decisions upon network access, reducing user waiting time and charging costs.

[0008] The purpose of the present invention can be achieved by the following technical solutions:

[0009] According to one aspect of the present invention, a method for assisting decision-making in bidirectional charging of electric vehicles based on game learning is provided, and the method steps include:

[0010] S1. The onboard device of the electric vehicle obtains the user's charging demand and determines whether the electric vehicle's stay time at the charging station exceeds the time required for its battery to be fully charged. If the judgment result is yes, step S2 is performed; if the judgment result is no, the normal charging process is entered;

[0011] S2. The onboard device of the electric vehicle collects feedback information of the user participating in the charging and discharging protocol. If the user feedback information is yes, step S3 is performed; if the user feedback information is no, the normal charging process is entered;

[0012] S3. Vehicle sensors collect electric vehicle information and transmit it to the microgrid control center. Based on the electric vehicle information, the microgrid control center predicts the electric vehicle's grid access status.

[0013] S4. Based on the predicted electric vehicle access status, determine whether the game solution is completed when the vehicle arrives at the target charging station; if so, proceed to step S5; if not, enter the normal charging process;

[0014] S5. Conduct a belief-based learning game between the microgrid control center and the electric vehicle to obtain the optimal charging and discharging strategy for the vehicle;

[0015] S6. Based on the optimal charging and discharging strategy, control the corresponding charging and discharging behavior of the electric vehicle after it arrives at the charging station.

[0016] As a preferred technical solution, the electric vehicle information in S3 includes original destination node information, vehicle location information, vehicle speed information, and vehicle charging request information.

[0017] As a preferred technical solution, the specific formula for determining whether the electric vehicle's stay time at the charging station exceeds the time required to fully charge its battery as described in S1 is as follows:

[0018]

[0019] Among them, T plug_out and T plug_in is the time when the electric vehicle leaves and arrives at the charging station, T plug_out -T plug_in Cap is the time that electric vehicles stay at charging stations. m is the battery capacity of different electric vehicles, h is the scheduling time of 24 hours a day, is the maximum charging and discharging power of the electric vehicle at time h.

[0020] As a preferred technical solution, the electric vehicle access status in S3 includes the time when the electric vehicle arrives at the target charging station and the charge state of the electric vehicle upon arrival.

[0021] As a preferred technical solution, the belief learning-based game process between the microgrid control center and the electric vehicle in S5 includes the game process of the microgrid control center and the game process of the electric vehicle on-board device;

[0022] During the microgrid control center game process, the microgrid control center does not need to know the profit function of the electric vehicle user. The game process aims to minimize the total charging and discharging costs of the electric vehicle and calculate the grid electricity price for each time period corresponding to the optimal charging and discharging strategy of the electric vehicle. During the microgrid control center game process, a time-forward scheduling strategy is adopted, that is, a day is divided into multiple fixed time periods, and a microgrid control center game process is carried out before each time period.

[0023] During the game process of the electric vehicle on-board device, the electric vehicle does not need to know the profit function of the microgrid. The game process aims to minimize the total charging and discharging costs of the electric vehicle and calculate the optimal charging and discharging strategy of the electric vehicle.

[0024] As a preferred technical solution, during the game of electric vehicle onboard devices, the goal is to minimize the total charging and discharging cost of the electric vehicle. The specific formula is:

[0025]

[0026] in, is the total charging and discharging cost of electric vehicle m; T plug_out and T plug_in is the time it takes for an electric vehicle to leave and arrive at a charging station, and ρ(h) is the price function is the charging and discharging strategy of electric vehicle m: 1 means charging, 0 means neither charging nor discharging, and -1 means discharging; is the charge and discharge power of electric vehicle m at time h; δ is the charge and discharge discount coefficient; avg(p h) is the average of all electric vehicle charging and discharging strategies at time h.

[0027] As a preferred technical solution, the specific steps of the microgrid control center game process are:

[0028] a1. The microgrid control center aggregates the status information of vehicles arriving at the same charging station during the same period;

[0029] a2. Initialize electricity price strategy;

[0030] a3. The microgrid control center extracts the relative frequency of the electric vehicle’s historical charging and discharging strategies from the aggregated vehicle status information as belief;

[0031] a4. Calculate the price signal; the microgrid control center calls the solver, learns based on the belief, and combines the price signal game to obtain the objective function and the new optimal electricity price strategy;

[0032] a5. Determine whether the mean square error between the electricity price matrix in the new optimal electricity price strategy and the current electricity price matrix is ​​less than the minimum value ε; if so, the iteration terminates; otherwise, return to a2 for a new round of game.

[0033] As a preferred technical solution, the specific formula for calculating the price signal is:

[0034] ρ(h)=0.15·r 2

[0035] Where ρ(h) is the price function, which represents the ratio of the total load demand of the power grid at time h to the power generation of the microgrid, i.e., the load demand information of the microgrid; r is the ratio of the average load to the average power generation, including the regional base load and the average power consumption of electric vehicles; 0.15 is the set reference electricity price; the total load demand of the power grid at time h includes the average base load, the total charge and discharge of electric vehicles, and the power imported from other power grids.

[0036] As a preferred technical solution, the specific steps of the game process of the electric vehicle on-board device are:

[0037] b1. The onboard device of electric vehicles aggregates historical electricity price information of the microgrid center;

[0038] b2. Initialize the electric vehicle charging and discharging strategy and the optimal charging and discharging matrix;

[0039] b3. The on-board device of the electric vehicle extracts the distribution frequency of the historical electricity price strategy from the aggregated historical electricity price information as the belief;

[0040] b4. The electric vehicle's onboard device calls the solver, learns based on the belief, and uses game theory to find the objective function and the new optimal charging and discharging strategy;

[0041] b5. Determine whether the absolute value of the mean square error between the charge-discharge matrix in the new optimal charge-discharge strategy and the current charge-discharge matrix is ​​less than the minimum value δ. If so, the iteration terminates; otherwise, return to b2 for a new round of game.

[0042] According to another aspect of the present invention, a game-learning-based electric vehicle bidirectional charging decision-making auxiliary system is provided. The system operates using the game-learning-based electric vehicle bidirectional charging decision-making auxiliary method described above. The system includes an electric vehicle onboard device, a microgrid control center, and a charging station.

[0043] The electric vehicle onboard device includes a communication module, a navigation module, a human-computer interaction module, and a game module, which are used to collect electric vehicle information and conduct a non-cooperative game process based on belief learning to obtain the optimal charging and discharging strategy for the vehicle;

[0044] The microgrid control center includes a communication module, a data calculation module, a game module, and an energy integration scheduling module. It is used to predict the grid access status of electric vehicles and conduct a non-cooperative game process based on belief learning to obtain the optimal charging and discharging strategy for the vehicles.

[0045] The charging station includes a communication module and a bidirectional interface module, which are used to receive the optimal charging and discharging strategy from the microgrid control center and use the bidirectional charging pile to perform charging and discharging.

[0046] Compared with the prior art, the present invention has the following advantages:

[0047] 1. The present invention discloses a game-learning-based decision-making method for bidirectional charging of electric vehicles. This method involves first collecting electric vehicle information via vehicle sensors and using the microgrid control center to predict the electric vehicle's grid access status based on the received electric vehicle information. This method then determines whether the game solution has been completed when the vehicle arrives at the target charging station. A belief-based learning game process then occurs between the microgrid control center and the electric vehicle to determine the vehicle's optimal charging and discharging strategy. Finally, upon arrival at the charging station, the electric vehicle makes charging and discharging behavior decisions guided by the optimal charging and discharging strategy. This method enables rapid and accurate guidance of charging and discharging behavior decisions for V2G-enabled electric vehicles upon grid access, without requiring the two players to know each other's payoff functions in advance. Game learning can effectively handle the complex decision-making processes of multiple agents in incomplete information scenarios, reducing user charging costs, improving user satisfaction, and achieving peak-shaving and valley-filling of grid load.

[0048] 2. In the game process of the present invention, the microgrid control center knows the profit function of the electric vehicle user and the strategy space of the electric vehicle. The microgrid control center and the electric vehicle on-board device calculate the optimal charging strategy of the electric vehicle and the corresponding grid electricity price for each time period with the goal of minimizing the objective function, so that the calculation of the optimal charging and discharging strategy is more accurate. In the traditional game process, the profit function and strategy space of both parties are public knowledge, that is, both parties need to know them. However, the present invention is based on the game learning method, which fully considers user needs, transmits electric vehicle demand information and current charging and discharging strategies in real time, and completes the interaction of information flows through the on-board unit. It can effectively balance the scheduling of energy flows between electric vehicles and microgrids, and accurately calculate the optimal charging and discharging strategy for electric vehicles.

[0049] 3. The microgrid control center data calculation module in this invention receives initial status information from vehicles, including original destination node information, vehicle location information, vehicle speed information, and vehicle charging request information. It then calculates and predicts the electric vehicle's grid access status, including the time it arrives at the target charging station and its state of charge upon arrival. The game-playing module uses this information to pre-determine the optimal charging and discharging strategy, iterating to determine an equilibrium before the electric vehicle enters the charging station, thus reducing waiting time.

[0050] 4. In the belief learning based on the game learning method in the present invention, the microgrid control center extracts the relative frequency of the electric vehicle's historical charging and discharging strategies from the aggregated vehicle status information as the belief; and the electric vehicle's on-board device extracts the distribution frequency of the historical electricity price strategy from the aggregated historical electricity price information as the belief; it is conducive to solving the "vehicle-grid" game model and assisting electric vehicles to obtain better charging and discharging behavior strategies, making decisions more accurate.

[0051] 5. In the electric vehicle bidirectional charging auxiliary decision-making system based on game learning of the present invention, the on-board device of the electric vehicle includes a navigation module. Combined with the application of vehicle networking technology, it can predict the time when the electric vehicle enters the network according to the starting and ending points specified by the user and the vehicle speed and other information collected by the on-board sensor, and notify the charging station at the end point to prepare for access through the communication module, thereby avoiding the situation where a large number of electric vehicles enter the charging station at once and the surge in the number of charging station users, thereby enhancing the reliability and practicality of the system.

[0052] 6. The present invention adopts a "time-ahead electricity price" mechanism in an auxiliary decision-making method for bidirectional charging of electric vehicles based on game learning. That is, a time-ahead scheduling strategy is adopted to divide a day into multiple fixed time periods. A microgrid control center game process is carried out before each time period. That is, before each charging time period, there is a process of calculating the optimal charging and discharging strategy. This ensures that even if a new electric vehicle joins the discharge process to the microgrid, the optimal charging and discharging strategy under the new situation can be proposed in a timely manner, thereby enhancing real-time performance. BRIEF DESCRIPTION OF THE DRAWINGS

[0053] Figure 1 This is an architecture diagram of an electric vehicle bidirectional charging auxiliary decision-making system based on game learning in the present invention;

[0054] Figure 2 This is a schematic diagram of the steps of a game learning-based auxiliary decision-making method for bidirectional charging of electric vehicles in the present invention;

[0055] Figure 3 This is a flowchart of an electric vehicle optimization scheduling based on vehicle networking technology in an electric vehicle bidirectional charging auxiliary decision-making based on game learning in an embodiment;

[0056] Figure 4 Flowchart of the game process of the microgrid control center in the present invention;

[0057] Figure 5 The figure is a flow chart of the game process of the on-board device of the electric vehicle in the present invention. DETAILED DESCRIPTION

[0058] The following will clearly and completely describe the technical solutions in the embodiments of the present invention in conjunction with the accompanying drawings. Obviously, the described embodiments are part of the embodiments of the present invention, not all of them. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative efforts should fall within the scope of protection of the present invention.

[0059] Example

[0060] The background of this embodiment is that currently, related patents and methods have some applications in optimizing the scheduling of irregular and random V2G-enabled electric vehicles using optimization theory and other methods, as well as implementing peak-shaving and valley-filling in new energy microgrids. However, in the face of large-scale and complex urban road networks, there is a lack of efficient and reliable optimization scheduling methods to assist electric vehicles in making optimal charging and discharging behavior decisions, taking into account the diverse characteristics of electric vehicles and the random nature of charging.

[0061] Therefore, in this embodiment, a bidirectional charging auxiliary decision-making system for electric vehicles based on game learning is constructed, and a bidirectional charging auxiliary decision-making method is used. According to the characteristics of the "vehicle-grid" interactive scenario in the existing urban scenario, combined with the collaborative vehicle infrastructure system and the vehicle network technology, the current original status of electric vehicles in the coverage area is collected in advance and the status information when joining the network is predicted. The optimal charging and discharging strategy is calculated based on the game learning method, and then the electric vehicles supporting V2G technology are quickly and accurately guided to make charging and discharging behavior decisions when joining the network, thereby reducing user waiting time and charging costs.

[0062] In this embodiment, we first build an electric vehicle bidirectional charging auxiliary decision-making system based on game learning. The system architecture is as follows: Figure 1 As shown in the figure, an architecture of an electric vehicle bidirectional charging and discharging auxiliary decision-making device based on game learning is connected to the vehicle network system. It mainly consists of three units, including a microgrid control center module, a charging station control module with bidirectional charging piles, and an electric vehicle on-board device.

[0063] In this embodiment, the on-board device A of the electric vehicle includes a communication module A1, a navigation module A2, a human-computer interaction module A3 and a game module A4.

[0064] Among them, the communication module A1 receives the original destination node information, vehicle location information, vehicle speed information, vehicle charging request information and the end point information determined by the electric vehicle user perceived by the vehicle sensor, and transmits the information to the microgrid control center.

[0065] Navigation module A2 predicts the time when electric vehicles will connect to charging stations based on the user-specified starting and ending points and the vehicle speed sensed by sensors. It notifies the charging station at the destination through communication module A1 to make preparations for access, thus avoiding a surge in users caused by the disorderly access of large numbers of electric vehicles to charging stations.

[0066] The human-computer interaction module A3 allows electric vehicle users to set parameters such as the target destination, charging station stay time, and target power before departure, and then exchange game information with the microgrid control center through the communication module. After the game is completed, the decision results are pushed to the user and feedback information on the user's participation in the charging and discharging agreement is collected through the human-computer interaction module within a preset time.

[0067] In this embodiment, the game module A4 and the game module B3 in the microgrid control center jointly participate in a non-cooperative game based on the "vehicle-grid" interaction, aggregate the information of electric vehicles connected to the grid in the same period, as well as the historical electricity prices and charging and discharging strategies, and adopt the belief learning method in game learning to predict the grid electricity price based on the beliefs formed in the early stage. Based on this, the optimal charging and discharging strategy for each electric vehicle in the next period is solved. The game iteration is completed when the electric vehicle user expenditure reaches the minimum value or the electric vehicle charging and discharging strategy is no longer updated.

[0068] In this embodiment, the micro-grid control center B includes a communication module B1, a data calculation module B2, a game module B3, and an energy integration scheduling module B4.

[0069] The communication module B1 receives the original destination node information, vehicle position information, vehicle speed information, vehicle charging request information, and end point information determined by the electric vehicle user from the electric vehicle on-board device communication module A1 in a large-scale scenario, and accepts the predicted electric vehicle grid access time from the navigation module A2. On the other hand, it receives historical electricity price information and the like from the charging station communication module C1.

[0070] The data calculation module B2 aggregates the vehicle information received by the communication module B1, and calculates the state of charge and other data of the electric vehicle when it arrives at the charging station in combination with the original vehicle data.

[0071] The game module B3 aggregates the electric vehicle grid access time, the time that can participate in bidirectional charging and discharging, the historical electricity price of the charging station, and the state of charge and other information of the electric vehicle when it arrives at the station provided by the data calculation module B2, which is received by the communication module B1. Based on the non-cooperative game model in the "vehicle-grid" interaction, the game module of the micro-grid control center aggregates the charging and discharging strategies in the historical game process, predicts the charging and discharging strategy of the electric vehicle in the next period according to the previous belief, and calculates the total load of the electric vehicle in the region according to the minimum expenditure of the electric vehicle user or the charging and discharging strategy of the electric vehicle and the power grid price no longer updated. The game iteration is completed.

[0072] The energy integration scheduling module B4 integrates various distributed power sources, loads, energy storage devices, and control devices into a miniature energy supply system, realizes smooth conversion between grid-connected and off-grid modes, and reasonably allocates power energy after obtaining the game results to achieve the goal of optimizing new energy scheduling.

[0073] In this embodiment, the charging station C includes a communication module C1 and a bidirectional interface module C2.

[0074] The communication module C1 transmits the historical electricity price and other information of the bidirectional charging pile to the micro-grid control center and accepts the optimal charging and discharging strategy and energy information from the game module B3 of the micro-grid control center.

[0075] One end of the bidirectional interface module C2 is connected to the microgrid control center, and the other end is connected to the electric vehicle arriving at the charging station. Users connect their electric vehicle to the intelligent bidirectional charging station by inserting a charging plug into the bidirectional interface on the charging station. The bidirectional interface on the intelligent bidirectional charging station can be used for both charging and discharging. The electric vehicle makes charging and discharging decisions guided by the optimal charging and discharging strategy.

[0076] In this embodiment, a game learning-based electric vehicle bidirectional charging auxiliary decision-making method is applied. The method steps are shown in the following diagram: Figure 2 As shown in the figure, the existing urban "vehicle-grid" interaction scenarios are summarized and modeled. The microgrid control center obtains the vehicle initial status information sent by the on-board units of electric vehicles within the coverage area through the vehicle networking technology, and uses the game learning method to calculate the optimal charging and discharging strategy. Then, when joining the network, it quickly and accurately guides electric vehicles that support V2G technology to make charging and discharging behavior decisions, reducing user charging costs, improving user satisfaction, and achieving "peak shaving and valley filling" of the grid load.

[0077] In this embodiment, an electric vehicle optimization scheduling process based on vehicle networking technology in an electric vehicle bidirectional charging auxiliary decision-making based on game learning is as follows: Figure 3 The specific steps are as follows:

[0078] In this embodiment, step S1 is specifically as follows: Under the vehicle networking technology, the vehicle obtains real-time information about its location, direction, travel distance, speed, acceleration, etc. through its own sensor equipment, and simultaneously receives the destination location information set by the user and the stay time at the charging station, etc.;

[0079] In this embodiment, step S2 is specifically as follows: determining whether the stay time at the charging station exceeds the time required to fully charge the battery; if not, executing the normal charging process; otherwise, executing step S3; the formula for determining whether the stay time at the charging station exceeds the time required to fully charge the battery is:

[0080]

[0081] Among them, T plug_out and T plug_in is the time when the electric vehicle leaves and arrives at the charging station, h is the scheduling time of 24 hours a day, Cap m is the battery capacity of different electric vehicles, is the maximum charging and discharging power of the electric vehicle at time h.

[0082] In this embodiment, a forward scheduling strategy is adopted, with scheduling once every hour. Each electric vehicle can be charged, discharged, or kept idle during its stay in the microgrid.

[0083] In this embodiment, step S3 is specifically as follows: the onboard device of the electric vehicle collects feedback information of the user participating in the charging and discharging agreement within a preset time. If the user feedback information is approval, step S4 is executed; if the user feedback information is rejection, the normal charging process is entered;

[0084] In this embodiment, step S4 is specifically as follows: the electric vehicle communication module receives vehicle information from the vehicle sensor, including the original node, destination node information, vehicle location information, vehicle speed information, vehicle charging request information, etc.;

[0085] In this embodiment, step S5 is specifically as follows: the microgrid control center receives vehicle information within the coverage area and uses the Internet of Vehicles technology to calculate the time it takes for the electric vehicle to reach the terminal charging station under the current traffic conditions. The formula is:

[0086] T plug_in =T0+T nav

[0087] Calculate the vehicle state of charge information at the arrival time using the following formula:

[0088]

[0089] Among them, T plug_in is the time when the electric vehicle arrives at the charging station, T0, T nav They represent the time when the electric vehicle reports information and the driving time from the initial node to the set end point provided by the navigation module, εV0 and εV arr It indicates the electric energy level when the electric starter reaches the charging station after the initial electric energy level is reached, h is the scheduling time of 24 hours a day, It represents the power of the electric vehicle during driving at time h.

[0090] In this embodiment, step S6 is specifically as follows: judging whether the game model solution is completed when the electric vehicle arrives at the charging station based on the historical game model solution time, if so, executing S7, otherwise executing the normal charging process;

[0091] In this embodiment, step S7 is specifically as follows: the microgrid control center aggregates information about vehicles arriving at the same charging station during the same period, combines historical electricity prices and historical charging and discharging strategies, and calculates the optimal charging and discharging strategy for the electric vehicle through a belief-based game learning method;

[0092] In this embodiment, the "vehicle-grid" game process is a non-cooperative dynamic game. The microgrid control center participates in the game as an intermediary. Through the communication module, it aggregates data from on-board devices to calculate the time and vehicle status of the electric vehicle's arrival at the charging station. At the same time, it collects historical charging and discharging electricity prices and historical optimal charging and discharging strategies at the target charging station. Considering that the charging and discharging behavior strategy of electric vehicles will affect the pricing strategy of the microgrid and thus the charging costs of electric vehicle users, generally speaking, electric vehicle users will choose to charge during time periods with lower electricity prices. Such user choices will further affect the strategy of the microgrid, and the two interact with each other. Based on this vehicle-grid interaction, with the goal of minimizing electric vehicle electricity costs and achieving a good grid load peak-shaving and valley-filling effect, the problem is modeled as a non-cooperative game model. In this game, the electric vehicle that supports bidirectional charging and discharging and the microgrid control center connected to the electric vehicle are the two participants.

[0093] In this embodiment, after obtaining the grid load and aggregated electric vehicle information and demand, the respective game modules of the microgrid control center and the electric vehicle calculate the optimal charging strategy of the electric vehicle and the corresponding grid electricity price for each time period with the goal of minimizing the objective function. In the repeated game, both the vehicle and the grid modify their beliefs and make behavioral decisions until the charging strategy of any electric vehicle no longer changes.

[0094] In this embodiment, the belief learning-based game process between the microgrid control center and the electric vehicle includes the game process of the microgrid control center and the game process of the electric vehicle on-board device; during the game process of the microgrid control center, the microgrid control center is unknown to the electric vehicle user's benefit function, and the game process aims to minimize the total charging and discharging costs of the electric vehicle, and calculates the grid electricity price for each time period corresponding to the optimal charging and discharging strategy of the electric vehicle; during the game process of the electric vehicle on-board device, the electric vehicle is unknown to the strategy space of the microgrid, and the game process aims to minimize the total charging and discharging costs of the electric vehicle, and calculates the optimal charging and discharging strategy of the electric vehicle.

[0095] In this embodiment, the microgrid control center game process is as follows: Figure 4 As shown, the specific steps are:

[0096] a1. The microgrid control center aggregates the status information of vehicles arriving at the same charging station during the same period;

[0097] a2. Initialize electricity price strategy;

[0098] a3. The microgrid control center extracts the relative frequency of the electric vehicle’s historical charging and discharging strategies from the aggregated vehicle status information as belief;

[0099] a4. Calculate the price signal; the microgrid control center calls the solver, learns based on the belief, and combines the price signal game to obtain the objective function and the new optimal electricity price strategy;

[0100] a5. Determine whether the mean square error between the electricity price matrix in the new optimal electricity price strategy and the current electricity price matrix is ​​less than the minimum value ε; if so, the iteration terminates; otherwise, return to a2 for a new round of game.

[0101] In this embodiment, the game process of the electric vehicle onboard device is as follows: Figure 5 As shown, the specific steps are:

[0102] b1. The onboard device of electric vehicles aggregates historical electricity price information of the microgrid center;

[0103] b2. Initialize the electric vehicle charging and discharging strategy and the optimal charging and discharging matrix;

[0104] b3. The on-board device of the electric vehicle extracts the distribution frequency of the historical electricity price strategy from the aggregated historical electricity price information as the belief;

[0105] b4. The electric vehicle's onboard device calls the solver, learns based on the belief, and uses game theory to find the objective function and the new optimal charging and discharging strategy;

[0106] b5. Determine whether the absolute value of the mean square error between the charge-discharge matrix in the new optimal charge-discharge strategy and the current charge-discharge matrix is ​​less than the minimum value δ. If so, the iteration terminates; otherwise, return to b2 for a new round of game.

[0107] In this embodiment, the microgrid optimal reflection function represents the microgrid benefit, and the specific formula is:

[0108]

[0109] Among them, Q MG (t) is the electricity price of the microgrid at time t (yuan / kwh), D(t) is the fixed load of residents at time t (kwh), P ev (t) is the electric vehicle power load at time t (kwh), Q G (t) The electricity price of the large power grid at time t (yuan / kwh), P G (t) is the electric energy purchased by the microgrid from the main grid at time t (kWh).

[0110] In this embodiment, the optimal reflection function of electric vehicles represents the benefits of electric vehicles. The specific formula is:

[0111]

[0112] Among them, Q eVS(t) is the charging and discharging price provided by the charging pile operator to electric vehicles at time t (yuan / kwh), Guiding the charging and discharging power of the i-th electric vehicle for the charging pile operator (kw), B loss is the battery loss cost of electric vehicles (yuan).

[0113] Traditional non-cooperative games typically assume that the optimal response function of the microgrid and the optimal response function of the electric vehicle are public knowledge. However, in this embodiment, a game learning method is used to solve the game equilibrium solution, in the case where the electric vehicle user does not know the optimal response function of the microgrid, and the microgrid does not know the optimal response function of the electric vehicle.

[0114] In this embodiment, the microgrid control center aggregates the status information of vehicles arriving at the same charging station during the same period, and the electric vehicle onboard device aggregates the historical electricity price information set by the microgrid;

[0115] In this embodiment, the microgrid control center's game module B3 initializes the microgrid's charging and discharging price decision. The electric vehicle's onboard device's game module A4 initializes the EV charging and discharging plan and the optimal charging and discharging matrix.

[0116] In this embodiment, the two players in a belief-learning game use the historical strategy distribution of their opponents as their beliefs about their strategies in the next round of the game. Furthermore, for the microgrid control center, these beliefs refer to the microgrid control center's game module memorizing the relative frequencies of each strategy previously adopted by electric vehicles. These frequencies serve as beliefs about the future behavior of other electric vehicles. For electric vehicles, these beliefs refer to the game module of their onboard devices using the frequency distribution of past electricity prices as their beliefs about the future electricity prices of the microgrid.

[0117] In this example, at the tth period of belief learning, the opponent formed by the participant chooses action c i The relationship between the probability and belief weight is:

[0118]

[0119] Where t is the number of belief learning periods; c i for action; b t (c i ) is formed by the player whose opponent chooses action c i the number of cases; For a player to form an opponent, his opponent chooses action c i is the total number of belief cases.

[0120] In this example, given the belief in the probabilities of other players choosing various actions, the player chooses pure strategy a in period t+1. i The probability is:

[0121]

[0122] Among them, π(a i / μ t ) Calculate each pure strategy a that the participant can choose i expected payment; a i It is a pure strategy.

[0123] In this embodiment, similarly, the participant selects the strategy with the maximum probability value as his or her optimal response.

[0124] In this embodiment, the price signal is calculated using the following formula:

[0125] ρ(h)=0.15·r 2

[0126] Where ρ(h) is the price function, which represents the ratio of the total load demand of the power grid at time h to the power generation of the microgrid, i.e., the load demand information of the microgrid; r is the ratio of the average load to the average power generation, including the regional base load and the average power consumption of electric vehicles; 0.15 is the set reference electricity price; the total load demand of the power grid at time h includes the average base load, the total charge and discharge of electric vehicles, and the power imported from other power grids.

[0127] In this embodiment, the solver in the game module A4 of the electric vehicle onboard device is called to solve the objective function and the new optimal charging strategy. Furthermore, the objective function is:

[0128]

[0129] in, is the total charging and discharging cost of electric vehicle m; T plug_out and T plug_in is the time it takes for an electric vehicle to leave and arrive at a charging station, and ρ(h) is the price function is the charging and discharging strategy of electric vehicle m: 1 means charging, 0 means neither charging nor discharging, and -1 means discharging; is the charge and discharge power of electric vehicle m at time h; δ is the charge and discharge discount coefficient; avg(p h ) is the average of all electric vehicle charging and discharging strategies at time h.

[0130] In this embodiment, after each round of game, the microgrid control center determines whether the mean square error between the new electricity price matrix and the current electricity price matrix is ​​less than a certain minimum value ε; the electric vehicle onboard device determines whether the absolute value of the mean square error between the new charge and discharge matrix and the current charge and discharge matrix is ​​less than a certain minimum value δ. If both are less than a certain minimum value, the iteration terminates. Without loss of generality, ε and δ can be set to 0.01. Otherwise, return to S701 for the next round of game.

[0131] In this embodiment, the current optimal electricity price and charging and discharging strategy are reported to the microgrid and the target charging station and executed.

[0132] In this embodiment, the B1 communication module of the microgrid control center transmits the optimal charging and discharging strategy of the electric vehicle to the bidirectional charging pile through the communication module. After the electric vehicle arrives at the pre-designated location, the electric vehicle is guided to charge or discharge in an orderly manner. The electric vehicle makes charging and discharging behavior decisions under the guidance of the optimal charging and discharging strategy.

[0133] In this embodiment, during the game-solving process, the microgrid's game module B3 only needs to know the historical charging and discharging strategies of electric vehicles in the area, while the electric vehicle's onboard device game module A4 only needs to know the historical electricity prices of the microgrid in the area. In traditional non-cooperative game methods, the microgrid and electric vehicle game modules need to know information such as each other's payoff functions. In comparison, this embodiment reduces the amount of public knowledge involved in the game process, making it more realistic.

[0134] In this embodiment, the charging and discharging process of an electric vehicle returning to a residential complex at night is taken as an example for explanation. According to information provided by the property management, there are 500 electric vehicles supporting V2G technology in the residential complex.

[0135] In this embodiment, an electric car is selected as the type that leaves early and returns late, that is, it is connected to the microgrid at around 6 pm and needs to leave at around 8 am the next day. If the electric car is located at a company 10 km away from the residence when it departs, the destination is set to the park through the human-computer interaction module in the on-board device before departure, and the preset stay time at the charging station is 14 hours (connected at 6 pm and needs to leave at 8 am the next day). The target level is required to reach 100% when leaving. The battery capacity is 60Kwh. Assuming the charge and discharge power is 13Kw, it takes 5 hours to fully charge (the charging time is an integer). Then the electric car will go through the following process:

[0136] W1, the departure time is 6 pm, the user sets the destination location as the residential park in the interactive interface, and the expected departure time is 8 am the next day;

[0137] W2. During driving, sensors in onboard devices record vehicle information: Under the Internet of Vehicles technology, through the vehicle's own sensor equipment, the vehicle can understand its own position, direction, travel distance, speed, acceleration and other vehicle information in real time;

[0138] W3. The vehicle-mounted device uses the following formula to preliminarily determine whether the stay time at the charging station exceeds the time required to fully charge the battery:

[0139]

[0140] Among them, Tplug_out and T plug_in The time when the electric vehicle leaves and arrives at the charging station, Cap m is the battery capacity of different electric vehicles, is the maximum charging and discharging power of the electric vehicle at time h.

[0141] In this embodiment, since the electric vehicle is expected to stay at the target charging station for 14 hours and the battery capacity is 60Kwh, assuming that the slow charging and discharging pile charges and discharges 13Kwh per hour, it takes 4.6 hours to fully charge from 0. If the conditions are met, the process proceeds to step W4;

[0142] W4. The on-board device collects user feedback on the charging and discharging protocol through the human-computer interaction module within a preset time. If the user feedback indicates approval, step W5 is executed to participate in the subsequent bidirectional charging and discharging protocol. If the user feedback indicates rejection, the vehicle enters the normal charging process after arriving at the charging station.

[0143] W5. Since the electric car is 10 km away from the charging station and the owner's speed is usually 60 km / h, and it is rush hour, the onboard navigation module, based on traffic conditions and historical vehicle driving data, estimates that the electric car will arrive at the charging station at 6:30 PM.

[0144] W6. After receiving the user's agreement to participate in charging and discharging, the communication module of the microgrid control center collects vehicle information within the coverage area (including original destination node information, vehicle location information, vehicle speed information, vehicle charging request information, etc.) through the communication module. Since the vehicle's state of charge is 100% when it departs, it takes 0.5 hours to reach the charging station. The motor power during the driving process is 25 kW. Therefore, according to the following formula, the battery level when arriving at the charging station should be 47.5 KWh:

[0145]

[0146] Among them, εV0 and εV arr Indicates the electric energy level when the electric starter reaches the charging station. nav The driving time from the initial node to the set destination provided by the navigation module, h is the scheduling time of 24 hours a day, It represents the power of the electric vehicle during driving at time h.

[0147] W7: The microgrid control center aggregates information about vehicles at the same charging station during the same time period and determines whether the game model solution has been completed when the electric vehicle arrives at the charging station. Since there are 500 V2G-enabled electric vehicles in the park, and the historical game model solution time is 1200 seconds, the electric vehicle's driving time exceeds the model solution time, so it can participate in the charging and discharging service during the corresponding time period, and proceeds to step W8.

[0148] W8, the microgrid control center, and the electric vehicle on-board device combine historical electricity prices and historical charging and discharging strategies to obtain the optimal charging and discharging strategy and the optimal electricity price strategy for the electric vehicle through game calculation based on belief learning.

[0149] In this embodiment, the obtained optimal charging and discharging strategy, that is, the optimal charging and discharging power of the electric vehicle at the corresponding time, is shown in Table 1; the obtained optimal electricity price strategy, that is, the real-time electricity price of the residential area microgrid at the corresponding time, is shown in Table 2.

[0150] Table 1 Optimal charging and discharging power of electric vehicles

[0151]

[0152] The obtained optimal electricity price strategy, that is, the real-time electricity price of the residential area microgrid at the corresponding time, is shown in Table 2.

[0153] Table 2 Real-time electricity prices of microgrids in residential areas

[0154]

[0155] In this example, based on real-time charging and discharging prices, the profit from discharging during peak hours is 2.55 yuan. Charging during off-peak hours, from 11 PM to 7 AM the following day, based on real-time charging and discharging prices, results in a low-peak charging cost of 9.18 yuan. Therefore, the total charging cost for the electric car that day is 6.6 yuan, and it reaches the ideal 100% state of charge by the time it leaves the residential area at 8 AM the following day. At 8 AM the following morning, the user settles the daily electricity bill via WeChat or Alipay and drives away.

[0156] In this embodiment, electric vehicles only know the historical electricity prices of the microgrid in the area, and the microgrid only knows the historical charging and discharging strategies of electric vehicles in the area. Through repeated interactions based on game learning, an equilibrium result is eventually obtained. The two parties in the game do not need to predict each other's payoff functions. Game learning can effectively handle the complex decision-making process of multiple intelligent agents in incomplete information scenarios.

[0157] As mentioned above, it is not difficult to see that by applying the game learning-based electric vehicle bidirectional charging auxiliary decision-making method and system, the efficiency of both parties can be significantly improved, and electric vehicles supporting V2G technology can be quickly and accurately guided to make charging and discharging behavior decisions when electric vehicles enter the grid, thereby reducing user costs, improving user satisfaction, and achieving "peak shaving and valley filling" of the grid load.

[0158] The above merely illustrates the specific embodiments of the present application, but the protection scope of the present application is not limited thereto, and any skilled person in the art can easily think of various equivalent modifications or replacements within the technical range disclosed by the present application, and these modifications or replacements shall be covered within the protection scope of the present application. Therefore, the protection scope of the present application shall be subject to the protection scope of the claims.

Claims

1. A game learning-based auxiliary decision-making method for electric vehicle bidirectional charging, characterized in that: The method steps include: S1. The onboard device of the electric vehicle obtains the user's charging demand and determines whether the electric vehicle's stay time at the charging station exceeds the time required for its battery to be fully charged. If the judgment result is yes, step S2 is performed; if the judgment result is no, the normal charging process is entered; S2. The onboard device of the electric vehicle collects feedback information of the user participating in the charging and discharging protocol. If the user feedback information is yes, step S3 is performed; if the user feedback information is no, the normal charging process is entered; S3. Vehicle sensors collect electric vehicle information and transmit it to the microgrid control center. Based on the electric vehicle information, the microgrid control center predicts the electric vehicle's grid access status. S4. Based on the predicted electric vehicle access status, determine whether the game solution is completed when the vehicle arrives at the target charging station; if so, proceed to step S5; if not, enter the normal charging process; S5. Conduct a belief-based learning game between the microgrid control center and the electric vehicle to obtain the optimal charging and discharging strategy for the vehicle; S6. Based on the optimal charging and discharging strategy, control the corresponding charging and discharging behavior of the electric vehicle after it arrives at the charging station; The belief-learning-based game process between the microgrid control center and the electric vehicle in S5 includes the game process of the microgrid control center and the game process of the electric vehicle on-board device; During the microgrid control center game process, the microgrid control center does not need to know the profit function of the electric vehicle user. The game process aims to minimize the total charging and discharging costs of the electric vehicle and calculate the grid electricity price for each time period corresponding to the optimal charging and discharging strategy of the electric vehicle. During the microgrid control center game process, a time-forward scheduling strategy is adopted, that is, a day is divided into multiple fixed time periods, and a microgrid control center game process is carried out before each time period. During the game process of the electric vehicle onboard device, the electric vehicle does not need to know the revenue function of the microgrid. The game process aims to minimize the total charging and discharging costs of the electric vehicle and calculate the optimal charging and discharging strategy of the electric vehicle. The specific steps of the electric vehicle on-board device game process are: b1. The onboard device of electric vehicles aggregates historical electricity price information of the microgrid center; b2. Initialize the electric vehicle charging and discharging strategy and the optimal charging and discharging matrix; b3. The on-board device of the electric vehicle extracts the distribution frequency of the historical electricity price strategy from the aggregated historical electricity price information as the belief; b4. The electric vehicle's onboard device calls the solver, learns based on the belief, and uses game theory to find the objective function and the new optimal charging and discharging strategy; b5. Determine whether the absolute value of the mean square error between the charge-discharge matrix in the new optimal charge-discharge strategy and the current charge-discharge matrix is ​​less than the minimum value δ. If so, the iteration terminates; otherwise, return to b2 for a new round of game.

2. The electric vehicle bidirectional charging auxiliary decision-making method based on game learning according to claim 1 is characterized in that: The electric vehicle information in S3 includes original destination node information, vehicle location information, vehicle speed information, and vehicle charging request information.

3. The electric vehicle bidirectional charging auxiliary decision-making method based on game learning according to claim 1 is characterized in that: The specific formula for determining whether the electric vehicle's stay time at the charging station exceeds the time required to fully charge its battery in S1 is as follows: in, and The time when the electric vehicle leaves and arrives at the charging station, It is the time that electric vehicles stay at the charging station. is the battery capacity of different electric vehicles, h is the scheduling time of 24 hours a day, is the maximum charging and discharging power of the electric vehicle at time h.

4. The electric vehicle bidirectional charging auxiliary decision-making method based on game learning according to claim 1 is characterized in that: The electric vehicle access status in S3 includes the time when the electric vehicle arrives at the target charging station and the charge state of the electric vehicle upon arrival.

5. The method for assisting decision-making in bidirectional charging of electric vehicles based on game learning according to claim 1, characterized in that: In the game process of the electric vehicle on-board device, the goal is to minimize the total charging and discharging cost of the electric vehicle, and the specific formula is: in, is the total charging and discharging cost of electric vehicle m; and The time when the electric vehicle leaves and arrives at the charging station, is the price function , is the charging and discharging strategy of electric vehicle m: 1 is charging, 0 is neither charging nor discharging, and -1 is discharging; is the charging and discharging power of electric vehicle m at time h; is the charge and discharge discount coefficient; is the mean of all electric vehicle charging and discharging strategies at time h.

6. The electric vehicle bidirectional charging auxiliary decision-making method based on game learning according to claim 1 is characterized in that: The specific steps of the microgrid control center game process are: a1. The microgrid control center aggregates the status information of vehicles arriving at the same charging station during the same period; a2. Initialize electricity price strategy; a3. The microgrid control center extracts the relative frequency of the electric vehicle’s historical charging and discharging strategies from the aggregated vehicle status information as belief; a4. Calculate the price signal; the microgrid control center calls the solver, learns based on the belief, and combines the price signal game to obtain the objective function and the new optimal electricity price strategy; a5. Determine whether the mean square error between the electricity price matrix in the new optimal electricity price strategy and the current electricity price matrix is ​​less than the minimum value ε; if so, the iteration terminates; Otherwise, return to a2 for a new round of game.

7. The method for assisting decision-making in bidirectional charging of electric vehicles based on game learning according to claim 6, characterized in that: The specific formula for calculating the price signal is: in, is a price function, which represents the ratio of the total load demand of the power grid at time h to the power generation of the microgrid, i.e., the load demand information of the microgrid; r is the ratio of the average load to the average power generation, including the regional base load and the average power consumption of electric vehicles; 0.15 is the set reference electricity price; the total load demand of the power grid at time h includes the average base load, the total charging and discharging capacity of electric vehicles, and the imported power from other power grids.

8. An electric vehicle bidirectional charging auxiliary decision-making system based on game learning, characterized in that: The system is operated by an electric vehicle bidirectional charging auxiliary decision-making method based on game learning as described in any one of claims 1 to 7, and includes an electric vehicle on-board device, a microgrid control center, and a charging station; The electric vehicle onboard device includes a communication module, a navigation module, a human-computer interaction module, and a game module, which are used to collect electric vehicle information and conduct a non-cooperative game process based on belief learning to obtain the optimal charging and discharging strategy for the vehicle; The microgrid control center includes a communication module, a data calculation module, a game module, and an energy integration scheduling module. It is used to predict the grid access status of electric vehicles and conduct a non-cooperative game process based on belief learning to obtain the optimal charging and discharging strategy for the vehicles. The charging station includes a communication module and a bidirectional interface module, which are used to receive the optimal charging and discharging strategy from the microgrid control center and use the bidirectional charging pile to perform charging and discharging.

Images