Game learning-based electric vehicle two-way charging auxiliary decision-making method and system

Abstract

The invention relates to a game learning-based electric vehicle two-way charging auxiliary decision-making method and system. According to the method, after electric vehicle information is collected and the network access state of an electric vehicle is predicted, a belief learning-based game process between a micro-grid control center and the electric vehicle is carried out to obtain an optimal charging and discharging strategy of the vehicle; and finally, carrying out charging and discharging behavior decision-making under the guidance of the strategy. In the game process, the micro-grid control center knows the revenue function of the electric vehicle user and knows the strategy space of the electric vehicle unknown power grid; and with minimization of the total charging and discharging cost of the electric vehicle as a target, an optimal charging and discharging strategy of the electric vehicle and the power grid electricity price corresponding to each time period are calculated. Compared with the prior art, the method can quickly and accurately guide the electric vehicle supporting the V2G technology to make a charging and discharging behavior decision during network access, and has the advantages of reducing the charging cost of a user, improving the satisfaction degree of the user, enhancing the reliability, accuracy and real-time performance, realizing peak load shifting of a power grid load and the like.

Description

Technical Field

[0001] The present invention relates to the technical field of electric vehicles, and in particular to a game learning-based electric vehicle bidirectional charging auxiliary decision-making method and a system thereof. Background Art

[0002] In order to achieve the dual carbon goals, my country has proposed to build a new power system with new energy as the main body. However, new energy represented by wind and solar is affected by the environment and climate, and has significant uncertainty and anti-peak characteristics. Therefore, microgrids integrate various distributed power sources, loads, energy storage devices and control devices into a micro energy supply system, which can achieve a smooth transition between grid-connected and off-grid modes. At the same time, under the influence of the trend of power market changes, new business forms such as integrated energy services, power aggregators, green power trading, and carbon trading have emerged one after another.

[0003] Electric vehicles (EV) have been rapidly promoted and popularized in recent years due to their economic and low-carbon advantages. Through the vehicle to grid (V2G) technology, electric vehicles are regarded as distributed energy storage devices with spatiotemporal characteristics, and the charging and discharging behaviors of electric vehicles are controlled to participate in the dispatching and operation of the power grid, which effectively improves the stability and economy of the power grid operation. As flexible loads, the time that electric vehicles stay at charging stations often exceeds the time required for their batteries to be fully charged. By orderly regulating the charging and discharging behaviors of a large number of electric vehicles, it is not only possible to avoid the impact on the power grid caused by large-scale random access, but also to achieve a certain degree of peak shaving and valley filling. However, since the time when electric vehicles access and leave the power grid is highly random, the charge state when arriving at the charging station is uncertain. The current dispatching scheme cannot accurately predict the travel conditions and needs of electric vehicles, and therefore cannot guide electric vehicle users to complete charging and discharging behavior decisions in a short time. At the same time, since the existing charging and discharging guidance methods require a lot of time to wait for the strategy optimization results, the waiting time of electric vehicle users after arriving at the charging station is increased, which reduces user satisfaction.

[0004] The invention patent with the publication number CN110979083A discloses a bidirectional charging and discharging control system and method for electric vehicles, wherein the method first collects electric vehicle images by a video monitoring unit and identifies the electric vehicle identity information; the bidirectional charging pile collects the electric vehicle demand information and transmits the electric vehicle demand information to the microgrid control center, and the microgrid control center performs scheduling optimization based on the time-of-use electricity price and consumption function with a preset charging period, and obtains the optimal charging and discharging strategy for electric vehicles at the lowest cost through game calculation; finally, the optimal charging and discharging strategy for electric vehicles is transmitted to the bidirectional charging pile to guide the electric vehicles to charge or discharge in an orderly manner. However, this method only considers the interests of electric vehicle users, and cannot achieve the effect of "peak shaving and valley filling" of power grid load while improving user satisfaction.

[0005] The invention patent with publication number CN117314111A discloses a master-slave game optimization scheduling method, equipment and medium for cluster electric vehicles. The disclosed method constructs an electric vehicle charging demand prediction model based on the road network and a master-slave game two-layer optimization model based on load aggregators and electric vehicle clusters, and mobilizes electric vehicle users to participate in demand response on the basis of considering the electricity consumption preferences of electric vehicle users. However, the application disclosure text uses a single model to characterize the travel data and travel characteristics of electric vehicles, without considering the influence of factors such as owner behavior, vehicle information differences, and future electric vehicle access prediction accuracy on the prediction model. Since the time when electric vehicles connect to and leave the power grid is highly random, the charge state when arriving at the charging station is uncertain, and it is impossible to accurately predict the travel situation and demand of electric vehicles, so it is impossible to guide electric vehicle users to complete charging and discharging behavior decisions in a relatively short time.

[0006] At present, the relevant public patents and methods have certain applications in optimizing the scheduling of V2G electric vehicles with irregularity and randomness using optimization theory and other methods, as well as realizing peak shaving and valley filling of new energy microgrids. However, the traditional game model requires the benefit function of both parties as common knowledge, which increases the difficulty of information exchange. At the same time, in the face of the actual large-scale and complex urban road network, there is still a lack of an efficient and reliable optimization scheduling method to assist electric vehicles in making optimal charging and discharging behavior decisions in view of the different state characteristics of electric vehicles and the randomness of charging. Summary of the invention

[0007] The purpose of the present invention is to provide a bidirectional charging auxiliary decision-making method and system for electric vehicles based on game learning in order to overcome the defects of the above-mentioned prior art. In view of the characteristics of the "vehicle-network" interactive scene in the existing urban scene, combined with the collaborative vehicle infrastructure system and the vehicle networking technology, the current original state of electric vehicles in the coverage area is collected in advance and the state information when entering the network is predicted. The optimal charging and discharging strategy is calculated based on the game learning method, and then the electric vehicles supporting V2G technology are quickly and accurately guided to make charging and discharging behavior decisions when the electric vehicles enter the network, reducing user waiting time and charging costs.

[0008] The purpose of the present invention can be achieved by the following technical solutions:

[0009] According to one aspect of the present invention, a method for assisting decision-making in bidirectional charging of electric vehicles based on game learning is provided, and the method steps include:

[0010] S1. The on-board device of the electric vehicle obtains the user's charging demand and determines whether the electric vehicle's stay time at the charging station exceeds the time required for its battery to be fully charged. If the judgment result is yes, proceed to step S2; if the judgment result is no, enter the normal charging process;

[0011] S2, the on-board device of the electric vehicle collects feedback information of the user participating in the charging and discharging protocol. If the user feedback information is yes, proceed to step S3; if the user feedback information is no, enter the conventional charging process;

[0012] S3, vehicle sensors collect electric vehicle information and transmit it to the microgrid control center. Based on the electric vehicle information, the microgrid control center predicts the electric vehicle access status;

[0013] S4. According to the predicted electric vehicle access status, determine whether the game solution is completed when the vehicle arrives at the target charging station; if so, proceed to step S5; if not, enter the conventional charging process;

[0014] S5, conducting a belief-based learning game process between the microgrid control center and the electric vehicle to obtain the optimal charging and discharging strategy for the vehicle;

[0015] S6. Based on the optimal charging and discharging strategy, control the electric vehicle to perform corresponding charging and discharging behaviors after arriving at the charging station.

[0016] As a preferred technical solution, the electric vehicle information in S3 includes original destination node information, vehicle location information, vehicle speed information, and vehicle charging request information.

[0017] As a preferred technical solution, the specific formula for determining whether the electric vehicle's stay time at the charging station exceeds the time required for its battery to be fully charged as described in S1 is as follows:

[0018]

[0019] Among them, T plug_out and T plug_in is the time when the electric vehicle leaves and arrives at the charging station, T plug_out -T plug_in Cap is the time that electric vehicles stay at charging stations. m is the battery capacity of different electric vehicles, h is the scheduling time of 24 hours a day, is the maximum charging and discharging power of the electric vehicle at time h.

[0020] As a preferred technical solution, the electric vehicle access status in S3 includes the time when the electric vehicle arrives at the target charging station and the charge state of the electric vehicle when arriving.

[0021] As a preferred technical solution, the belief learning-based game process between the microgrid control center and the electric vehicle in S5 includes the game process of the microgrid control center and the game process of the on-board device of the electric vehicle;

[0022] During the game process of the microgrid control center, the microgrid control center does not need to know the profit function of the electric vehicle user. The game process aims to minimize the total charging and discharging costs of the electric vehicle and calculate the grid electricity price for each time period corresponding to the optimal charging and discharging strategy of the electric vehicle; during the game process of the microgrid control center, a time-forward scheduling strategy is adopted, that is, a day is divided into multiple fixed time periods, and a microgrid control center game process is carried out before each time period;

[0023] During the game process of the electric vehicle on-board device, the electric vehicle does not need to know the revenue function of the microgrid. The game process aims to minimize the total charging and discharging cost of the electric vehicle and calculate the optimal charging and discharging strategy of the electric vehicle.

[0024] As a preferred technical solution, during the game of electric vehicle on-board devices, the goal is to minimize the total charging and discharging cost of the electric vehicle. The specific formula is:

[0025]

[0026] in, is the total charging and discharging cost of electric vehicle m; T plug_out and T plug_in is the time when the electric vehicle leaves and arrives at the charging station, ρ(h) is the price function is the charging and discharging strategy of electric vehicle m: 1 means charging, 0 means neither charging nor discharging, and -1 means discharging; is the charge and discharge power of electric vehicle m at time h; δ is the charge and discharge discount coefficient; avg(p h) is the average of all electric vehicle charging and discharging strategies at time h.

[0027] As a preferred technical solution, the specific steps of the game process of the microgrid control center are:

[0028] a1. The microgrid control center aggregates the status information of vehicles arriving at the same charging station during the same period;

[0029] a2. Initialize electricity price strategy;

[0030] a3. The microgrid control center extracts the relative frequency of the electric vehicle’s historical charging and discharging strategies as beliefs from the aggregated vehicle status information;

[0031] a4. Calculate the price signal; the microgrid control center calls the solver, learns based on the belief, and combines the price signal game to obtain the objective function and the new optimal electricity price strategy;

[0032] a5. Determine whether the mean square error between the electricity price matrix in the new optimal electricity price strategy and the current electricity price matrix is ​​less than the minimum value ε; if so, the iteration terminates; otherwise, return to a2 for a new round of game.

[0033] As a preferred technical solution, the specific formula for calculating the price signal is:

[0034] ρ(h)=0.15·r 2

[0035] Among them, ρ(h) is the price function, which represents the ratio of the total load demand of the power grid at time h to the power generation of the microgrid, that is, the load demand information of the microgrid, r is the ratio of the average load to the average power generation, including the regional basic load and the average power consumption of electric vehicles; 0.15 is the set reference electricity price; the total load demand of the power grid at time h includes the average basic load, the total charging and discharging capacity of electric vehicles, and the imported power from other power grids.

[0036] As a preferred technical solution, the specific steps of the game process of the electric vehicle on-board device are:

[0037] b1. The on-board device of electric vehicles aggregates the historical electricity price information of the microgrid center;

[0038] b2. Initialize the electric vehicle charging and discharging strategy and the optimal charging and discharging matrix;

[0039] b3. The on-board device of the electric vehicle extracts the distribution frequency of the historical electricity price strategy from the aggregated historical electricity price information as the belief;

[0040] b4. The on-board device of the electric vehicle calls the solver, learns according to the belief, and uses game theory to find the objective function and the new optimal charging and discharging strategy;

[0041] b5. Determine whether the absolute value of the mean square error between the charge-discharge matrix in the new optimal charge-discharge strategy and the current charge-discharge matrix is ​​less than the minimum value δ. If so, the iteration is terminated; otherwise, return to b2 for a new round of game.

[0042] According to another aspect of the present invention, there is provided an electric vehicle bidirectional charging auxiliary decision-making system based on game learning, the system is operated by the electric vehicle bidirectional charging auxiliary decision-making method based on game learning as described above, the system comprises an electric vehicle on-board device, a microgrid control center, and a charging station;

[0043] The electric vehicle onboard device includes a communication module, a navigation module, a human-computer interaction module and a game module, which are used to collect electric vehicle information and conduct a non-cooperative game process based on belief learning to obtain the optimal charging and discharging strategy of the vehicle;

[0044] The microgrid control center includes a communication module, a data calculation module, a game module, and an energy integration scheduling module; it is used to predict the grid access status of electric vehicles and conduct a non-cooperative game process based on belief learning to obtain the optimal charging and discharging strategy for the vehicle;

[0045] The charging station includes a communication module and a bidirectional interface module, which are used to receive the optimal charging and discharging strategy from the microgrid control center and use the bidirectional charging pile to perform charging and discharging.

[0046] Compared with the prior art, the present invention has the following advantages:

[0047] 1. The present invention discloses a method for assisting decision-making in bidirectional charging of electric vehicles based on game learning. The method first collects electric vehicle information through vehicle sensors, and uses the microgrid control center to predict the electric vehicle access status based on the received electric vehicle information; then determines whether the game solution is completed when the vehicle arrives at the target charging station; then conducts a belief-based game process between the microgrid control center and the electric vehicle to obtain the optimal charging and discharging strategy for the vehicle; finally, after the electric vehicle arrives at the charging station, it makes a charging and discharging behavior decision under the guidance of the optimal charging and discharging strategy. It realizes the rapid and accurate guidance of electric vehicles supporting V2G technology to make charging and discharging behavior decisions when accessing the network, and the two parties in the game do not need to know each other's benefit function in advance. Game learning can effectively handle the complex decision-making process of multiple intelligent agents in incomplete information scenarios, reduce user charging costs, improve user satisfaction, and achieve "peak shaving and valley filling" of power grid load.

[0048] 2. In the game process of the present invention, the microgrid control center has the profit function of the electric unknown car users, and the electric car has the unknown grid strategy space; the microgrid control center and the electric car on-board device aim to minimize the objective function, calculate the optimal charging strategy of the electric car and the corresponding grid electricity price for each time period, so that the calculation of the optimal charging and discharging strategy is more accurate. In the traditional game process, the profit function and strategy space of both parties are public knowledge, that is, both parties need to know them. However, the present invention is aimed at the scenario where the microgrid does not know the profit function of the electric car user, and the electric car user does not know the grid strategy space (grid electricity price setting strategy). Based on the game learning method, it fully considers the user needs, transmits the electric car demand information and the current charging and discharging strategy in real time, and completes the interaction of information flow through the on-board unit, which can effectively balance the scheduling of energy flow between each electric car and the microgrid, and accurately calculate the optimal charging and discharging strategy for the electric car.

[0049] 3. The data calculation module of the microgrid control center in the present invention receives the initial state sent by the vehicle, including the original destination node information, vehicle location information, vehicle speed information, and vehicle charging request information, thereby calculating and predicting the electric vehicle access status, which includes the time when the electric vehicle arrives at the target charging station and the charge state of the electric vehicle when it arrives. The game module completes the optimal charging and discharging strategy solution in advance based on the above information, and can iterate the equilibrium result before the electric vehicle enters the charging station, reducing the waiting time of the electric vehicle.

[0050] 4. In the belief learning based on the game learning method in the present invention, the microgrid control center extracts the relative frequency of the electric vehicle's historical charging and discharging strategies from the aggregated vehicle status information as the belief; and the electric vehicle's on-board device extracts the distribution frequency of the historical electricity price strategy from the aggregated historical electricity price information as the belief; it is conducive to solving the "vehicle-grid" game model and assisting electric vehicles to obtain better charging and discharging behavior strategies, making decisions more accurate.

[0051] 5. In the electric vehicle bidirectional charging auxiliary decision-making system based on game learning of the present invention, the on-board device of the electric vehicle includes a navigation module. Combined with the application of vehicle networking technology, the time for the electric vehicle to enter the network can be predicted according to the starting point and end point specified by the user and the vehicle speed and other information collected by the on-board sensor. The communication module can inform the charging station at the end point to be ready for access, thereby avoiding a surge in the number of charging station users caused by a large number of electric vehicles entering the charging station, thereby enhancing the reliability and practicality of the system.

[0052] 6. The present invention adopts a "time-ahead electricity price" mechanism in an auxiliary decision-making method for bidirectional charging of electric vehicles based on game learning, that is, a time-ahead scheduling strategy is adopted to divide a day into multiple fixed time periods, and a microgrid control center game process is carried out before each time period, that is, there is a process of calculating the optimal charging and discharging strategy before each charging time period. This ensures that even if new electric vehicles are added to the discharge process to the microgrid, the optimal charging and discharging strategy under the new situation can be proposed in a timely manner, thereby enhancing real-time performance. BRIEF DESCRIPTION OF THE DRAWINGS

[0053] Figure 1 This is an architecture diagram of an electric vehicle bidirectional charging auxiliary decision-making system based on game learning in the present invention;

[0054] Figure 2 A schematic diagram of the steps of a method for assisting decision-making in bidirectional charging of electric vehicles based on game learning in the present invention;

[0055] Figure 3 It is a flow chart of electric vehicle optimization scheduling based on vehicle networking technology in an electric vehicle bidirectional charging auxiliary decision-making based on game learning in an embodiment;

[0056] Figure 4 It is a flow chart of the game process of the microgrid control center in the present invention;

[0057] Figure 5 The figure is a flow chart of the game process of the on-board device of the electric vehicle in the present invention. DETAILED DESCRIPTION

[0058] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work should fall within the scope of protection of the present invention.

[0059] Example

[0060] The background of this embodiment is that: currently, the relevant disclosed patents and methods have certain applications in optimizing the scheduling of V2G electric vehicles with irregularity and randomness, and realizing peak load shifting and valley filling of new energy microgrids by using optimization theory and other methods. However, in the face of the actual large-scale and complex urban road network, there is still a lack of an efficient and reliable optimization scheduling method to assist electric vehicles in making optimal charging and discharging behavior decisions in view of the different state characteristics of electric vehicles and the randomness of charging.

[0061] Therefore, in this embodiment, a two-way charging auxiliary decision-making system for electric vehicles based on game learning is built, and a two-way charging auxiliary decision-making method is used. According to the characteristics of the "vehicle-grid" interactive scenarios in existing urban scenarios, combined with the collaborative vehicle infrastructure system and the Internet of Vehicles technology, the current original status of electric vehicles in the coverage area is collected in advance and the status information when joining the network is predicted. The optimal charging and discharging strategy is calculated based on the game learning method, and then the electric vehicles supporting V2G technology are quickly and accurately guided to make charging and discharging behavior decisions when the electric vehicles join the network, thereby reducing user waiting time and charging costs.

[0062] In this embodiment, we first build an electric vehicle bidirectional charging auxiliary decision-making system based on game learning. The system architecture is as follows: Figure 1 As shown, an architecture of an electric vehicle bidirectional charging and discharging auxiliary decision-making device connected to the Internet of Vehicles system based on game learning is mainly composed of three units, including a microgrid control center module, a charging station control module with a bidirectional charging pile, and an electric vehicle on-board device.

[0063] In this embodiment, the on-board device A of the electric vehicle includes a communication module A1, a navigation module A2, a human-computer interaction module A3 and a game module A4.

[0064] Among them, the communication module A1 receives the original destination node information, vehicle location information, vehicle speed information, vehicle charging request information and the end point information determined by the electric vehicle user perceived by the vehicle sensor, and transmits the information to the microgrid control center.

[0065] The navigation module A2 predicts the time when the electric vehicle will access the charging station based on the starting and ending points specified by the user and the vehicle speed sensed by the sensor, and informs the charging station at the end point through the communication module A1 to make corresponding preparations for access, thereby avoiding a surge in users caused by large-scale electric vehicles accessing the charging station in an unordered manner.

[0066] The human-computer interaction module A3 allows electric vehicle users to set parameters such as the target destination, charging station stay time, target power, etc. before departure, and then interact with the microgrid control center through the communication module for game information. After the game is completed, the decision results are pushed to the user and the feedback information of the user participating in the charging and discharging protocol is collected through the human-computer interaction module within a preset time.

[0067] In this embodiment, the game module A4 and the game module B3 in the microgrid control center jointly participate in the non-cooperative game based on the "vehicle-grid" interaction, aggregate the electric vehicle information connected to the power grid in the same period, as well as the historical electricity prices and charging and discharging strategies, and adopt the belief learning method in game learning to predict the power grid electricity price according to the beliefs formed in the early stage, and use this as a basis to solve the optimal charging and discharging strategy for each electric vehicle in the next period. When the electric vehicle user expenditure reaches the minimum value or the electric vehicle charging and discharging strategy is no longer updated, the game iteration is completed.

[0068] In this embodiment, the microgrid control center B includes a communication module B1, a data calculation module B2, a game module B3, and an energy integration scheduling module B4.

[0069] Among them, the communication module B1 receives the original destination node information, vehicle location information, vehicle speed information, vehicle charging request information and the end point information determined by the electric vehicle user transmitted from the communication module A1 of the electric vehicle onboard device in a large-scale scenario, and receives the electric vehicle network access time predicted by the navigation module A2. On the other hand, it receives the historical electricity price and other information statistically collected by the communication module C1 of the charging station.

[0070] The data calculation module B2 aggregates the vehicle information received by the communication module B1, and calculates the data such as the state of charge of the electric vehicle when it arrives at the charging station in combination with the original vehicle data.

[0071] The game module B3 aggregates the information received by the communication module B1, such as the time when the electric vehicle joins the network, the time when it can participate in two-way charging and discharging, the historical electricity price of the charging station, and the information provided by the data calculation module B2, such as the charge status of the electric vehicle when it arrives at the station. Based on the non-cooperative game model in the "vehicle-grid" interaction, the game module of the microgrid control center aggregates the charging and discharging strategies in the historical game process, and adopts the belief learning method to predict the charging and discharging strategy of the electric vehicle in the next period according to the previous belief, and calculates the total load of electric vehicles in the area based on this, and calculates the grid electricity price for the next period of time in combination with the basic load of residents and the daily power generation of the area. When the electric vehicle user expenditure reaches the minimum value or the electric vehicle charging and discharging strategy and the grid electricity price are no longer updated, the game iteration is completed.

[0072] The energy integration dispatch module B4 integrates various distributed power sources, loads, energy storage devices and control devices into a micro energy supply system to achieve a smooth transition between grid-connected and off-grid modes. After obtaining the game results, the electric energy is reasonably allocated to achieve the goal of optimizing the dispatch of new energy.

[0073] In this embodiment, the charging station C includes a communication module C1 and a bidirectional interface module C2.

[0074] The communication module C1 transmits the historical electricity price information of the bidirectional charging pile to the microgrid control center and receives the optimal charging and discharging strategy and power information from the game module B3 of the microgrid control center.

[0075] One end of the bidirectional interface module C2 is connected to the microgrid control center, and the other end is connected to the electric vehicle that is about to arrive at the charging station. The user connects the electric vehicle to the intelligent bidirectional charging pile by inserting the charging plug into the bidirectional interface on the charging pile. The bidirectional interface on the intelligent bidirectional charging pile can be used for both charging and discharging. The electric vehicle makes charging and discharging behavior decisions under the guidance of the optimal charging and discharging strategy.

[0076] In this embodiment, a game learning-based electric vehicle bidirectional charging auxiliary decision-making method is applied, and the method steps are shown in the following figure: Figure 2 As shown in the figure, the existing urban "vehicle-grid" interaction scenarios are summarized and modeled. The microgrid control center obtains the vehicle initial state information sent by the on-board units of electric vehicles within the coverage area through the Internet of Vehicles technology, and uses the game learning method to calculate the optimal charging and discharging strategy, so as to quickly and accurately guide electric vehicles supporting V2G technology to make charging and discharging behavior decisions when accessing the network, thereby reducing users' charging costs, improving user satisfaction, and achieving "peak shaving and valley filling" of the grid load.

[0077] In this embodiment, an electric vehicle optimization scheduling process based on vehicle networking technology in an electric vehicle bidirectional charging auxiliary decision-making based on game learning is as follows: Figure 3 As shown, the specific steps are as follows:

[0078] In this embodiment, step S1 is specifically as follows: Under the vehicle networking technology, the vehicle obtains information about its position, direction, travel distance, speed and acceleration in real time through the vehicle's own sensor equipment, and receives the end point location information set by the user and the stay time at the charging station, etc.;

[0079] In this embodiment, step S2 is specifically as follows: determine whether the stay time at the charging station exceeds the time required for the battery to be fully charged, if not, execute the normal charging process; otherwise, execute step S3; the formula for determining whether the stay time at the charging station exceeds the time required for the battery to be fully charged is:

[0080]

[0081] Among them, T plug_out and T plug_in is the time when the electric vehicle leaves and arrives at the charging station, h is the scheduling time of 24 hours a day, and Cap m is the battery capacity of different electric vehicles, is the maximum charging and discharging power of the electric vehicle at time h.

[0082] In this embodiment, a time-forward scheduling strategy is adopted, and scheduling is performed once every hour. Each electric vehicle can be charged, discharged or kept idle during its stay in the microgrid.

[0083] In this embodiment, step S3 is as follows: the on-board device of the electric vehicle collects feedback information of the user participating in the charging and discharging agreement within a preset time. If the user feedback information is agreement, step S4 is executed; if the user feedback information is rejection, the normal charging process is entered;

[0084] In this embodiment, step S4 is specifically as follows: the electric vehicle communication module receives vehicle information from the vehicle sensor, including the original node, the destination node information, the vehicle location information, the vehicle speed information, the vehicle charging request information, etc.;

[0085] In this embodiment, step S5 is as follows: the microgrid control center receives vehicle information within the coverage area, and uses the Internet of Vehicles technology to calculate the time it takes for the electric vehicle to arrive at the terminal charging station under the current traffic conditions. The formula is:

[0086] T plug_in =T0+T nav

[0087] Calculate the vehicle charge status information at the arrival time using the formula:

[0088]

[0089] Among them, T plug_in is the time when the electric vehicle arrives at the charging station, T0, T nav They represent the time when the electric vehicle reports information, the driving time from the initial node to the set end point provided by the navigation module, εV0 and εV arr It indicates the initial power level of the electric starter when it reaches the charging station. h is the scheduling time of 24 hours a day. It represents the power of the electric vehicle during driving at time h.

[0090] In this embodiment, step S6 is specifically as follows: judging whether the game model solution is completed when the electric vehicle arrives at the charging station according to the historical game model solution time, if so, executing S7, otherwise executing the conventional charging process;

[0091] In this embodiment, step S7 is specifically as follows: the microgrid control center aggregates the vehicle information arriving at the same charging station in the same period, combines the historical electricity price and the historical charging and discharging strategy, and calculates the optimal charging and discharging strategy of the electric vehicle through a game learning method based on belief learning;

[0092] In this embodiment, the "vehicle-grid" game process is a non-cooperative dynamic game. The microgrid control center participates in the game as an intermediary. The communication module aggregates the time and vehicle status of the electric vehicle arriving at the charging station from the on-board device, and collects the historical charging and discharging electricity prices and the historical optimal charging and discharging strategies at the target charging station. Considering that the charging and discharging behavior strategy of electric vehicles will affect the pricing strategy of the microgrid and thus affect the charging costs of electric vehicle users, generally speaking, electric vehicle users will choose to charge in time periods with lower electricity prices. Such user choices will further affect the strategy of the microgrid, and the two interact with each other. Based on such vehicle-grid interaction characteristics, with the goal of minimizing the electricity cost of electric vehicles and achieving a better grid load peak-shaving and valley-filling effect, the problem is modeled as a non-cooperative game model. In the game, the electric vehicle that supports bidirectional charging and discharging and the microgrid control center connected to the electric vehicle are the two participants in the game.

[0093] In this embodiment, after obtaining the grid load and aggregated electric vehicle information and demand, the respective game modules of the microgrid control center and the electric vehicle calculate the optimal charging strategy of the electric vehicle and the corresponding grid electricity price for each time period with the goal of minimizing the objective function. In the repeated game, both the vehicle and the grid modify their beliefs and make behavioral decisions until the charging strategy of any electric vehicle no longer changes.

[0094] In this embodiment, the belief learning-based game process between the microgrid control center and the electric vehicle includes the game process of the microgrid control center and the game process of the electric vehicle on-board device; in the game process of the microgrid control center, the microgrid control center is unknown to the electric vehicle user's benefit function, and the game process aims to minimize the total charging and discharging cost of the electric vehicle, and calculates the grid electricity price for each time period corresponding to the optimal charging and discharging strategy of the electric vehicle; in the game process of the electric vehicle on-board device, the electric vehicle is unknown to the strategy space of the microgrid, and the game process aims to minimize the total charging and discharging cost of the electric vehicle, and calculates the optimal charging and discharging strategy of the electric vehicle.

[0095] In this embodiment, the game process of the microgrid control center is as follows: Figure 4 As shown, the specific steps are:

[0096] a1. The microgrid control center aggregates the status information of vehicles arriving at the same charging station during the same period;

[0097] a2. Initialize electricity price strategy;

[0098] a3. The microgrid control center extracts the relative frequency of the electric vehicle’s historical charging and discharging strategies as beliefs from the aggregated vehicle status information;

[0099] a4. Calculate the price signal; the microgrid control center calls the solver, learns based on the belief, and combines the price signal game to obtain the objective function and the new optimal electricity price strategy;

[0100] a5. Determine whether the mean square error between the electricity price matrix in the new optimal electricity price strategy and the current electricity price matrix is ​​less than the minimum value ε; if so, the iteration terminates; otherwise, return to a2 for a new round of game.

[0101] In this embodiment, the game process of the electric vehicle onboard device is as follows: Figure 5 As shown, the specific steps are:

[0102] b1. The on-board device of electric vehicles aggregates the historical electricity price information of the microgrid center;

[0103] b2. Initialize the electric vehicle charging and discharging strategy and the optimal charging and discharging matrix;

[0104] b3. The on-board device of the electric vehicle extracts the distribution frequency of the historical electricity price strategy from the aggregated historical electricity price information as the belief;

[0105] b4. The on-board device of the electric vehicle calls the solver, learns according to the belief, and uses game theory to find the objective function and the new optimal charging and discharging strategy;

[0106] b5. Determine whether the absolute value of the mean square error between the charge-discharge matrix in the new optimal charge-discharge strategy and the current charge-discharge matrix is ​​less than the minimum value δ. If so, the iteration is terminated; otherwise, return to b2 for a new round of game.

[0107] In this embodiment, the microgrid optimal reflection function represents the microgrid benefit, and the specific formula is:

[0108]

[0109] Among them, Q MG (t) is the electricity price of the microgrid at time t (yuan / kwh), D(t) is the fixed load of residents at time t (kwh), P ev (t) is the electric vehicle power load at time t (kwh), Q G (t) The electricity price of the large power grid at time t (yuan / kwh), P G (t) is the electric energy purchased by the microgrid from the large grid at time t (kwh).

[0110] In this embodiment, the electric vehicle optimal response function represents the electric vehicle benefit, and the specific formula is:

[0111]

[0112] Among them, Q eVS(t) is the charging and discharging price provided by the charging pile operator to electric vehicles at time t (yuan / kwh), Guiding the charging and discharging power of the i-th electric vehicle for the charging pile operator (kw), B loss is the battery loss cost of electric vehicles (yuan).

[0113] In traditional non-cooperative games, the optimal response function of the microgrid and the optimal response function of the electric vehicle are usually assumed to be public knowledge. However, in this embodiment, in the case where the electric vehicle user does not know the optimal response function of the microgrid and the microgrid does not know the optimal response function of the electric vehicle, a game learning method is used to solve the game equilibrium solution.

[0114] In this embodiment, the microgrid control center aggregates the status information of vehicles arriving at the same charging station during the same period, and the on-board device of the electric vehicle aggregates the historical electricity price information set by the microgrid;

[0115] In this embodiment, the microgrid control center's game module B3 initializes the microgrid's charging and discharging price decision. The electric vehicle's onboard device's game module A4 initializes the EV charging and discharging plan and the optimal charging and discharging matrix;

[0116] In this embodiment, the two parties in the game based on belief learning use the historical strategy distribution of the opponent in the past as the belief about the opponent's strategy in the new round of game. Further, for the microgrid control center, the belief refers to the game module of the microgrid control center remembering the relative frequency of each strategy previously adopted by the electric vehicle, and these frequencies are the beliefs about the later behavior of other electric vehicles; and for the electric vehicle, the belief refers to the game module of its on-board device using the frequency of past electricity price distribution as the belief about the later electricity price of the microgrid.

[0117] In this example, at the tth period of belief learning, the participant forms a situation where his opponent chooses action c i The relationship between the probability and belief weight is:

[0118]

[0119] Where t is the number of belief learning periods; c i for action; b t (c i ) is formed by the participant whose opponent chooses action c i the number of cases; For a participant to form a choice of action c for his opponent i is the total number of belief cases.

[0120] In this example, given the belief that other participants have probabilities of choosing various actions, the participant chooses pure strategy a in period t+1. i The probability is:

[0121]

[0122] Among them, π(a i / μ t ) Calculate for each pure strategy a that the participant can choose i Expected payment; a i It is a pure strategy.

[0123] In this embodiment, similarly, the participant selects the strategy with the maximum probability value as his / her optimal response.

[0124] In this embodiment, the price signal is calculated, and the specific formula is:

[0125] ρ(h)=0.15·r 2

[0126] Among them, ρ(h) is the price function, which represents the ratio of the total load demand of the power grid at time h to the power generation of the microgrid, that is, the load demand information of the microgrid, r is the ratio of the average load to the average power generation, including the regional basic load and the average power consumption of electric vehicles; 0.15 is the set reference electricity price; the total load demand of the power grid at time h includes the average basic load, the total charging and discharging capacity of electric vehicles, and the imported power from other power grids.

[0127] In this embodiment, the solver in the game module A4 of the electric vehicle onboard device is called to solve the objective function and the new optimal charging strategy. Further, the objective function is:

[0128]

[0129] in, is the total charging and discharging cost of electric vehicle m; T plug_out and T plug_in is the time when the electric vehicle leaves and arrives at the charging station, ρ(h) is the price function is the charging and discharging strategy of electric vehicle m: 1 means charging, 0 means neither charging nor discharging, and -1 means discharging; is the charge and discharge power of electric vehicle m at time h; δ is the charge and discharge discount coefficient; avg(p h ) is the average of all electric vehicle charging and discharging strategies at time h.

[0130] In this embodiment, after each round of game, the microgrid control center determines whether the mean square error between the new electricity price matrix and the current electricity price matrix is ​​less than a certain minimum value ε; the electric vehicle onboard device determines whether the absolute value of the mean square error between the new charge and discharge matrix and the current charge and discharge matrix is ​​less than a certain minimum value δ. If both are less than a certain minimum value, the iteration is terminated. Without loss of generality, ε and δ can be taken as 0.01. Otherwise, return to S701 for the next round of game;

[0131] In this embodiment, the current optimal electricity price and charging and discharging strategy are reported to the microgrid and the target charging station and executed.

[0132] In this embodiment, the B1 communication module of the microgrid control center transmits the optimal charging and discharging strategy of the electric vehicle to the bidirectional charging pile through the communication module. After the electric vehicle arrives at the pre-designated location, the electric vehicle is guided to charge or discharge in an orderly manner, and the electric vehicle makes charging and discharging behavior decisions under the guidance of the optimal charging and discharging strategy.

[0133] In this embodiment, during the game solving process, the game module B3 in the microgrid only needs to know the information of the historical charging and discharging strategies of electric vehicles in the area, and the game module A4 of the electric vehicle onboard device only needs to know the information of the historical electricity price of the microgrid in the area. In the traditional non-cooperative game method, the microgrid and electric vehicle game modules need to know the information such as the other party's profit function. In comparison, in this embodiment, public knowledge is reduced in the game process, which is more in line with the actual scenario.

[0134] In this embodiment, the charging and discharging process of an electric vehicle returning to a residential area at night is taken as an example for explanation. According to the information provided by the property management, there are 500 electric vehicles supporting V2G technology in the residential area.

[0135] In this embodiment, an electric car is selected as the type that leaves early and returns late, that is, it is connected to the microgrid at around 6 pm and needs to leave at around 8 am the next day. If the electric car is located at a company 10km away from the residence when it departs, the destination is set to the park through the human-computer interaction module in the on-board device before departure, and the preset stay time at the charging station is 14 hours (connected at 6 pm and needs to leave at 8 am the next day), and the target level is required to reach 100% when leaving. The battery capacity is 60Kwh, assuming the charging and discharging power is 13Kw, it takes 5 hours to fully charge (the charging time is an integer). Then the electric car will go through the following process:

[0136] W1, the departure time is 6 pm, the end point set by the user in the interactive interface is the residential park, and it is expected to leave the charging station at 8 pm the next day;

[0137] W2. During driving, sensors in the vehicle-mounted device record vehicle information: Under the Internet of Vehicles technology, through the vehicle's own sensor equipment, the vehicle can understand its own position, direction, driving distance, speed, acceleration and other vehicle information in real time;

[0138] W3. The vehicle-mounted device uses the following formula to preliminarily determine whether the stay time at the charging station exceeds the time required to fully charge the battery:

[0139]

[0140] Among them, Tplug_out and T plug_in Cap is the time when the electric vehicle leaves and arrives at the charging station. m is the battery capacity of different electric vehicles, is the maximum charging and discharging power of the electric vehicle at time h.

[0141] In this embodiment, since the electric vehicle is expected to stay at the target charging station for 14 hours and the battery capacity is 60Kwh, assuming that the slow charging and discharging pile charges and discharges 13Kwh per hour, it takes 4.6 hours to fully charge from 0. If the conditions are met, then step W4 is entered;

[0142] W4. The vehicle-mounted device collects user feedback information on the charging and discharging protocol through the human-computer interaction module within a preset time. If the user feedback information is consent, step W5 is executed to participate in the subsequent two-way charging and discharging protocol. If the user feedback information is rejection, the vehicle enters the regular charging process after arriving at the charging station.

[0143] W5. Since the electric car is 10 km away from the charging station, the car owner usually drives at 60 km / h, and this is the rush hour after get off work, the onboard navigation module combines traffic conditions and historical vehicle driving data to estimate that the electric car will arrive at the charging station at 18:30 in the evening;

[0144] W6. After receiving the user's agreement to participate in charging and discharging, the communication module of the microgrid control center collects vehicle information within the coverage area (including original destination node information, vehicle location information, vehicle speed information, vehicle charging request information, etc.) through the communication module; since the vehicle's state of charge is 100% when it sets off, it takes 0.5 hours to reach the charging station, and the motor power during the driving process is 25kw, the power level when arriving at the charging station should be 47.5Kwh according to the following formula:

[0145]

[0146] Among them, εV0 and εV arr It represents the initial power level of the electric starter and the power level when it reaches the charging station, T nav The driving time from the initial node to the set destination provided by the navigation module, h is the scheduling time of 24 hours a day, It represents the power of the electric vehicle during driving at time h.

[0147] W7, the microgrid control center aggregates the information of vehicles at the same charging station during the same period, and determines whether the game model solution is completed when the electric vehicle arrives at the charging station. Since there are 500 electric vehicles supporting V2G technology among the permanent users of the park, the historical game model solution time is 1200s, and the driving time of the electric vehicle exceeds the model solution time, it can participate in the charging and discharging service in the corresponding time period, and proceed to step W8;

[0148] W8, the microgrid control center and the electric vehicle on-board device combine the historical electricity prices and historical charging and discharging strategies, and obtain the optimal charging and discharging strategy and the optimal electricity price strategy for the electric vehicle through game calculation based on belief learning.

[0149] In this embodiment, the optimal charging and discharging strategy obtained, that is, the optimal charging and discharging power of the electric vehicle at the corresponding time, is shown in Table 1; the optimal electricity price strategy obtained, that is, the real-time electricity price of the residential area microgrid at the corresponding time, is shown in Table 2.

[0150] Table 1 Optimal charging and discharging power of electric vehicles

[0151]

[0152] The obtained optimal electricity price strategy, that is, the real-time electricity price of the residential area microgrid at the corresponding time, is shown in Table 2.

[0153] Table 2 Real-time electricity prices of microgrids in residential areas

[0154]

[0155] In this embodiment, the profit of discharging during the peak period of electricity consumption is 2.55 yuan according to the real-time charging and discharging price; charging during the low-consumption period from 23:00 to 7:00 the next day, the charging cost during the low-consumption period is 9.18 yuan according to the real-time charging and discharging price. Therefore, the total charging cost of the electric car on that day is 6.6 yuan, and it reaches the ideal 100% charge state when leaving the residential area at 8 o'clock the next day. At 8 o'clock the next morning, the user settles the electricity cost of the day through WeChat or Alipay and drives the electric car away.

[0156] In this embodiment, electric vehicles only know the information of the historical electricity prices of the microgrid in the area, and the microgrid only knows the information of the historical charging and discharging strategies of electric vehicles in the area. Through repeated interactions based on game learning, an equilibrium result is finally obtained. The two parties in the game do not need to know each other's profit function in advance. Game learning can effectively handle the complex decision-making process of multiple intelligent agents in incomplete information scenarios.

[0157] As mentioned above, it is not difficult to see that by applying the game learning-based electric vehicle bidirectional charging auxiliary decision-making method and system, the efficiency of both parties can be significantly improved, and electric vehicles supporting V2G technology can be quickly and accurately guided to make charging and discharging behavior decisions when electric vehicles are connected to the grid, thereby reducing user costs, improving user satisfaction, and achieving "peak shaving and valley filling" of the grid load.

[0158] The above is only a specific embodiment of the present invention, but the protection scope of the present invention is not limited thereto. Any technician familiar with the technical field can easily think of various equivalent modifications or replacements within the technical scope disclosed by the present invention, and these modifications or replacements should be included in the protection scope of the present invention. Therefore, the protection scope of the present invention shall be based on the protection scope of the claims.

Claims

1. A game learning-based auxiliary decision-making method for electric vehicle bidirectional charging, characterized in that: The method steps include: S1. The on-board device of the electric vehicle obtains the user's charging demand and determines whether the electric vehicle's stay time at the charging station exceeds the time required for its battery to be fully charged. If the judgment result is yes, proceed to step S2; if the judgment result is no, enter the normal charging process; S2, the on-board device of the electric vehicle collects feedback information of the user participating in the charging and discharging protocol. If the user feedback information is yes, proceed to step S3; if the user feedback information is no, enter the conventional charging process; S3, vehicle sensors collect electric vehicle information and transmit it to the microgrid control center. Based on the electric vehicle information, the microgrid control center predicts the electric vehicle access status; S4. According to the predicted electric vehicle access status, determine whether the game solution is completed when the vehicle arrives at the target charging station; if so, proceed to step S5; if not, enter the conventional charging process; S5, conducting a belief-based learning game process between the microgrid control center and the electric vehicle to obtain the optimal charging and discharging strategy for the vehicle; S6. Based on the optimal charging and discharging strategy, control the electric vehicle to perform corresponding charging and discharging behaviors after arriving at the charging station.

2. According to the game learning-based decision-making method for electric vehicle bidirectional charging according to claim 1, it is characterized in that: The electric vehicle information in S3 includes original destination node information, vehicle location information, vehicle speed information, and vehicle charging request information.

3. The method for auxiliary decision-making of bidirectional charging of electric vehicles based on game learning according to claim 1 is characterized in that: The specific formula for determining whether the electric vehicle's stay time at the charging station exceeds the time required for its battery to be fully charged in S1 is as follows: Among them, T plug_out and T plug_in is the time when the electric vehicle leaves and arrives at the charging station, T plug_out -T plug_in Cap is the time that electric vehicles stay at charging stations. m is the battery capacity of different electric vehicles, h is the scheduling time of 24 hours a day, is the maximum charging and discharging power of the electric vehicle at time h.

4. The method for assisting decision-making in bidirectional charging of electric vehicles based on game learning according to claim 1 is characterized in that: The electric vehicle access status in S3 includes the time when the electric vehicle arrives at the target charging station and the charge state of the electric vehicle when arriving.

5. The method for assisting decision-making in bidirectional charging of electric vehicles based on game learning according to claim 1 is characterized in that: The game process based on belief learning between the microgrid control center and the electric vehicle in S5 includes the game process of the microgrid control center and the game process of the on-board device of the electric vehicle; During the game process of the microgrid control center, the microgrid control center does not need to know the profit function of the electric vehicle user. The game process aims to minimize the total charging and discharging costs of the electric vehicle and calculate the grid electricity price for each time period corresponding to the optimal charging and discharging strategy of the electric vehicle; during the game process of the microgrid control center, a time-forward scheduling strategy is adopted, that is, a day is divided into multiple fixed time periods, and a microgrid control center game process is carried out before each time period; During the game process of the electric vehicle on-board device, the electric vehicle does not need to know the revenue function of the microgrid. The game process aims to minimize the total charging and discharging cost of the electric vehicle and calculate the optimal charging and discharging strategy of the electric vehicle.

6. The method for assisting decision-making in bidirectional charging of electric vehicles based on game learning according to claim 5 is characterized in that: In the game process of the electric vehicle on-board device, the goal is to minimize the total charging and discharging cost of the electric vehicle, and the specific formula is: in, is the total charging and discharging cost of electric vehicle m; T plig_out and T plug_in is the time when the electric vehicle leaves and arrives at the charging station, ρ(h) is the price function is the charging and discharging strategy of electric vehicle m: 1 means charging, 0 means neither charging nor discharging, and -1 means discharging; is the charge and discharge power of electric vehicle m at time h; δ is the charge and discharge discount coefficient; avg(p h ) is the average of all electric vehicle charging and discharging strategies at time h.

7. The method for assisting decision-making in bidirectional charging of electric vehicles based on game learning according to claim 5 is characterized in that: The specific steps of the microgrid control center game process are: a1. The microgrid control center aggregates the status information of vehicles arriving at the same charging station during the same period; a2. Initialize electricity price strategy; a3. The microgrid control center extracts the relative frequency of the electric vehicle’s historical charging and discharging strategies as beliefs from the aggregated vehicle status information; a4. Calculate the price signal; the microgrid control center calls the solver, learns based on the belief, and combines the price signal game to obtain the objective function and the new optimal electricity price strategy; a5. Determine whether the mean square error between the electricity price matrix in the new optimal electricity price strategy and the current electricity price matrix is ​​less than the minimum value ε; if so, the iteration is terminated; Otherwise, return to a2 for a new round of game.

8. The method for assisting decision-making in bidirectional charging of electric vehicles based on game learning according to claim 7 is characterized in that: The specific formula for calculating the price signal is: ρ(h)=0.15·r 2 Among them, ρ(h) is the price function, which represents the ratio of the total load demand of the power grid at time h to the power generation of the microgrid, that is, the load demand information of the microgrid, r is the ratio of the average load to the average power generation, including the regional basic load and the average power consumption of electric vehicles; 0.15 is the set reference electricity price; the total load demand of the power grid at time h includes the average basic load, the total charging and discharging capacity of electric vehicles, and the imported power from other power grids.

9. The method for assisting decision-making in bidirectional charging of electric vehicles based on game learning according to claim 5 is characterized in that: The specific steps of the game process of electric vehicle on-board devices are: b1. The on-board device of electric vehicles aggregates the historical electricity price information of the microgrid center; b2. Initialize the electric vehicle charging and discharging strategy and the optimal charging and discharging matrix; b3. The on-board device of the electric vehicle extracts the distribution frequency of the historical electricity price strategy from the aggregated historical electricity price information as the belief; b4. The on-board device of the electric vehicle calls the solver, learns according to the belief, and uses game theory to find the objective function and the new optimal charging and discharging strategy; b5. Determine whether the absolute value of the mean square error between the charge-discharge matrix in the new optimal charge-discharge strategy and the current charge-discharge matrix is ​​less than the minimum value δ. If so, the iteration is terminated. Otherwise, return to b2 for a new round of game.

10. An electric vehicle bidirectional charging auxiliary decision-making system based on game learning, characterized in that: The system is operated by an auxiliary decision-making method for bidirectional charging of electric vehicles based on game learning as described in claims 1-9, and the system includes an electric vehicle on-board device, a microgrid control center, and a charging station; The electric vehicle onboard device includes a communication module, a navigation module, a human-computer interaction module and a game module, which are used to collect electric vehicle information and conduct a non-cooperative game process based on belief learning to obtain the optimal charging and discharging strategy of the vehicle; The microgrid control center includes a communication module, a data calculation module, a game module, and an energy integration scheduling module; it is used to predict the grid access status of electric vehicles and conduct a non-cooperative game process based on belief learning to obtain the optimal charging and discharging strategy for the vehicle; The charging station includes a communication module and a bidirectional interface module, which are used to receive the optimal charging and discharging strategy from the microgrid control center and use the bidirectional charging pile to perform charging and discharging.

Images