Real-time optimization scheduling method and device for charging and discharging of electric vehicles with multi-period demand

Abstract

The application provides a multi-period demand electric vehicle charging and discharging real-time optimization scheduling method and device, and relates to the electric vehicle charging and discharging scheduling optimization technical field.The method comprises the following steps: fitting the probability distribution of electric vehicle travel and energy use according to travel record data, and constructing a probability model of electric vehicle use mode; performing Markov chain simulation on the uncertain travel of the electric vehicle according to the probability model, and obtaining chain data of multi-period travel under different space types; screening and integrating the chain data to obtain multi-parking period uncertain demand data of the electric vehicle under the space type where the device is located, and calculating the electric vehicle schedulable flexibility constraint coupled in the multi-parking period; constructing a Markov decision process, and optimizing the charging and discharging strategy of the electric vehicle by using a deep reinforcement learning algorithm to output the real-time optimal charging and discharging strategy of the electric vehicle.The application can solve the complex constraint problem caused by the uncertain multi-period demand of the electric vehicle.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of electric vehicle charging and discharging scheduling optimization, in particular to a multi-period demand electric vehicle charging and discharging real-time optimization scheduling method and device. BACKGROUND

[0002] With the deepening of global energy transformation, the proportion of renewable energy in the power system continues to rise, but its inherent intermittency and volatility pose a significant challenge to the stable operation of the power grid, making it difficult to balance power supply and demand. It is urgent to use the load side to improve system flexibility by adjusting the time distribution and intensity of electricity consumption to match the fluctuating supply, thereby improving the overall efficiency of the system. The orderly charging and discharging of EVs (Electric Vehicles) is beneficial to the stable operation of the power grid, supply and demand matching, and peak shaving. EVs are widely recognized as flexible and controllable resources on the load side. During the parking period, the battery can be used as an energy storage system to actively participate in peak shaving of the power grid, provided that the energy needs of EV users can be met before the end of the parking period.

[0003] The prior art can schedule the charging and discharging of EVs according to the flexible electricity price formulated by the power grid by equipping the EV bidirectional charging and discharging facility with an intelligent controller, so that the EVs charge during the low electricity price period and send power back to the grid during the high electricity price period through vehicle-to-grid interaction technology. This not only helps EV users optimize charging costs, but also helps the power grid achieve peak shaving and enhance its adaptability to renewable energy output fluctuations, providing strong support for building a more stable and efficient intelligent power grid scheduling system. The randomness of EV users' travel will affect the time and energy availability of EV scheduling optimization, posing challenges to EV charging and discharging optimization. Not only is it necessary to pursue the optimal cost under flexible electricity prices, but it is also necessary to ensure that enough energy is provided for EVs to meet users' travel needs. In different spatial types of cities, the frequency of EVs, the duration of their stay, and the probability of charging are different. To address the problem of optimizing the scheduling of EV charging and discharging, a reliable EV load model is needed, and the demand uncertainty and spatial and temporal heterogeneity introduced by human random behavior should be fully considered.

[0004] Currently, some studies use data-driven methods to represent random and unpredictable travel patterns. The current deep Q network method can achieve user-oriented real-time charging scheduling, which divides users into two groups that prefer daytime and nighttime charging based on statistical data from the Pecan Street dataset. However, the above method does not consider the random departure and arrival times of EVs, but only simulates the dynamics of the vehicle fleet based on historical data or knowledge extracted from it. This method performs well in simulating group characteristics, but due to the lack of data, it is difficult to capture the response characteristics of the vehicle fleet to the control strategy.

[0005] The existing EV charging and discharging real-time optimization research often lacks consideration of the uncertainty of EV demand in the real world, and cannot achieve charging and discharging scheduling considering the spatial and temporal heterogeneity of EV load. Random optimization often needs to comprehensively model the random parking of EV load, so it is often difficult to achieve when dealing with the uncertain demand of EV in the real world. In order to solve the coupling problem of sequence decision of EV control, the existing technology proposes a forward scheduling method, which considers the load rebound effect of a single parking period, but fails to explore the flexibility potential of two-way battery swapping between vehicles and networks. In addition, traditional optimization methods such as stochastic optimization will reduce the solution speed when dealing with the increase of time resolution of EV control. Therefore, the existing technology usually sets the time resolution of the EV optimization problem to one hour, ignoring the short stay, which may lead to inaccurate estimation of the flexibility of EV. The current model-based optimization method may not perform well in dealing with various real-time uncertainty scenarios, resulting in inaccurate estimation of the flexibility potential of EV. In order to cope with the flexibility of EV and the power price changing over time in real-time scheduling, many studies have adopted deep reinforcement learning method due to its ability to interact with dynamic environment and adapt to environmental changes. However, the uncertainty of EV load may lead to unpredictable learning cost in the deep reinforcement learning framework, slowing down the process of converging to a stable strategy. In addition, the existing technology ignores the multi-period parking mode of EV in daily use, limiting the full use of the flexibility potential of EV and the optimization of user experience. Therefore, there is an urgent need for an EV charging and discharging optimization method that can consider multi-period flexibility to better balance user demand and scheduling benefits. SUMMARY

[0006] In order to solve the technical problems in the prior art that the multi-period parking mode characteristics of EV are ignored in charging and discharging, the complex constraint problem caused by the high uncertainty of EV demand cannot be solved, and the long-term impact of decision cannot be fully considered due to the limitation of computational complexity in optimization solution, and the optimality of global scheduling cannot be fully considered in real-time response, the embodiments of the present application provide a multi-period demand electric vehicle charging and discharging real-time optimization scheduling method and device. The technical solution is as follows:

[0007] On the one hand, a multi-period demand electric vehicle charging and discharging real-time optimization scheduling method is provided, which is realized by a multi-period demand electric vehicle charging and discharging real-time optimization scheduling device. The method comprises:

[0008] S1, obtaining travel record data; fitting the probability distribution of electric vehicle travel and energy use according to the travel record data; constructing a probability model of electric vehicle usage mode according to the probability distribution;

[0009] S2, according to the probability model, Markov chain simulation is carried out on the uncertain travel of the electric vehicle, and chain data of multi-period travel under different space types is obtained;

[0010] S3, the chain data is screened and integrated, and multi-parking period uncertain demand data of the electric vehicle under the space type where the device is located is obtained;

[0011] S4, according to the multi-parking period uncertain demand data of the electric vehicle under the space type where the device is located, the electric vehicle schedulable flexibility constraint coupled with the multi-parking period on the travel chain is calculated;

[0012] S5, according to the electric vehicle dynamic schedulable flexibility constraint coupled with the multi-parking period on the travel chain, a Markov decision process is constructed, a deep reinforcement learning algorithm is used to optimize the charging and discharging strategy of the electric vehicle, and the real-time charging and discharging optimal strategy of the electric vehicle is output.

[0013] On the other hand, a multi-period demand electric vehicle charging and discharging real-time optimization scheduling device is provided, which is applied to a multi-period demand electric vehicle charging and discharging real-time optimization scheduling method, and the device comprises:

[0014] The construction unit is used for obtaining travel record data; fitting the probability distribution of electric vehicle travel and energy use according to the travel record data; and constructing a probability model of electric vehicle use mode according to the probability distribution;

[0015] The first acquisition unit is used for simulating Markov chain on the uncertain travel of the electric vehicle according to the probability model, and obtaining chain data of multi-period travel under different space types;

[0016] The second acquisition unit is used for screening and integrating the chain data, and obtaining multi-parking period uncertain demand data of the electric vehicle under the space type where the device is located;

[0017] The calculation unit is used for calculating the electric vehicle schedulable flexibility constraint coupled with the multi-parking period on the travel chain according to the multi-parking period uncertain demand data of the electric vehicle under the space type where the device is located;

[0018] The optimization unit is used for constructing a Markov decision process according to the electric vehicle dynamic schedulable flexibility constraint coupled with the multi-parking period on the travel chain, using a deep reinforcement learning algorithm to optimize the charging and discharging strategy of the electric vehicle, and outputting the real-time charging and discharging optimal strategy of the electric vehicle.

[0019] In another aspect, a multi-period demand electric vehicle charging and discharging real-time optimization scheduling device is provided, comprising: a processor; a memory having computer readable instructions stored thereon, the computer readable instructions, when executed by the processor, implement any one of the above multi-period demand electric vehicle charging and discharging real-time optimization scheduling methods.

[0020] In another aspect, a computer readable storage medium is provided, the storage medium having at least one instruction stored therein, the at least one instruction being loaded and executed by a processor to implement any one of the above multi-period demand electric vehicle charging and discharging real-time optimization scheduling methods.

[0021] The technical solutions provided by the embodiments of the present application have at least the following beneficial effects:

[0022] The embodiments of the present application first acquire travel record data; fit the probability distribution of electric vehicle travel and energy use according to the travel record data; construct a probability model of electric vehicle usage mode according to the probability distribution; perform Markov chain simulation on the uncertain travel of the electric vehicle according to the probability model to obtain chain data of multi-period travel under different space types; secondly, screen and integrate the chain data to obtain multi-parking period uncertain demand data of the electric vehicle under the space type where the device is located; calculate the electric vehicle dispatchable flexibility constraint coupled in the multi-parking period on the travel chain according to the multi-parking period uncertain demand data of the electric vehicle under the space type where the device is located; finally, construct a Markov decision process according to the dynamic dispatchable flexibility constraint of the electric vehicle coupled in the multi-parking period on the travel chain, and use a deep reinforcement learning algorithm to optimize the charging and discharging strategy of the electric vehicle, and output the real-time optimal charging and discharging strategy of the electric vehicle.

[0023] In view of the single time period, static scheduling and local optimal limitation problems existing in the traditional electric vehicle charging and discharging optimization method, the charging and discharging problem of the electric vehicle is extended to multiple time periods in the embodiment of the application, and effective load transfer is carried out in a wide time range. Unlike the existing optimization technology which stays in the single parking time period scheduling, the embodiment of the application constructs a scheduling optimization framework based on multi-period dynamic charging and discharging optimization, and fully considers the time sequence coupling characteristics in the charging and discharging process of the electric vehicle, the dynamic change rule of the battery state and the randomness of the user travel demand. Through the deep reinforcement learning optimization method, the embodiment of the application can not only realize the demand response effect of the traditional charging and discharging optimization in the single parking time period under the premise of ensuring the user travel demand, but also realize the cross-period load transfer in the intraday scale according to the multi-dimensional information such as the electricity price signal, the future parking time period and the energy demand prediction, conforms to the random use mode of the electric vehicle in reality, and enhances the flexibility potential of the electric vehicle charging and discharging scheduling. The real-time scheduling of the cross-period load transfer decision can be realized by adopting the embodiment of the application, and the cross-period load transfer is realized in multiple discontinuous electric vehicle parking time periods by combining the electricity price signal and based on the future parking time period and the energy demand prediction. The decision mechanism of the embodiment of the application can guarantee the real-time response capability while taking into account the long-term optimization effect, and can meet the real-time scheduling timeliness requirement. BRIEF DESCRIPTION OF DRAWINGS

[0024] In order to more clearly illustrate the technical solutions in the embodiments of the application, the drawings needed in the embodiment description will be briefly introduced. Obviously, the drawings in the following description are only some embodiments of the application, and other drawings can be obtained by those skilled in the art without creative labor.

[0025] Figure 1 is a multi-period demand electric vehicle charging and discharging real-time optimization scheduling method flowchart provided by the embodiment of the application;

[0026] Figure 2 is a time-space path schematic diagram of an electric vehicle provided by the embodiment of the application;

[0027] Figure 3 is a time distribution simulation result diagram of the parking time period of an electric vehicle in a residential area provided by the embodiment of the application;

[0028] Figure 4 is a deep reinforcement learning-based electric vehicle charging and discharging optimization framework diagram provided by the embodiment of the application;

[0029] Figure 5 is an electric vehicle charging and discharging strategy comparison diagram provided by the embodiment of the application;

[0030] Figure 6 is a multi-period demand electric vehicle charging and discharging real-time optimization scheduling device block diagram provided by an embodiment of the application.

[0031] Figure 7 is a multi-period demand electric vehicle charging and discharging real-time optimization scheduling device structure diagram provided by an embodiment of the application. DETAILED DESCRIPTION

[0032] The technical solutions in the application will be described below with reference to the drawings.

[0033] In the embodiments of the application, the words such as "example", "for example" and the like are used to represent an example, illustration or description. Any embodiment or design scheme described as "example" in the application should not be interpreted as more preferred or more advantageous than other embodiments or design schemes. Rather, the word "example" is intended to present the concept in a specific manner. In addition, in the embodiments of the application, the meaning expressed by "and / or" can be both, or can be one of the two.

[0034] In the embodiments of the application, "image" and "picture" can be used interchangeably at times, and it should be pointed out that the meanings expressed are consistent when the distinction is not emphasized. "Of", "corresponding" and "corresponding" can be used interchangeably at times, and it should be pointed out that the meanings expressed are consistent when the distinction is not emphasized.

[0035] In the embodiments of the application, sometimes the subscript such as W1 can be written in the form of non-subscript such as W1, and the meanings expressed are consistent when the distinction is not emphasized.

[0036] To make the technical problems, technical solutions and advantages to be solved by the application more clear, specific embodiments will be described in detail below with reference to the drawings.

[0037] The embodiments of the application provide a multi-period demand electric vehicle charging and discharging real-time optimization scheduling method, which can be implemented by a multi-period demand electric vehicle charging and discharging real-time optimization scheduling device. The multi-period demand electric vehicle charging and discharging real-time optimization scheduling device can be a terminal or a server. Figure 1 As shown in the multi-period demand electric vehicle charging and discharging real-time optimization scheduling method flow chart, the processing flow of the method can include the following steps:

[0038] S1, obtaining travel record data; fitting the probability distribution of electric vehicle travel and energy use according to the travel record data; and constructing a probability model of electric vehicle use mode according to the probability distribution.

[0039] In the embodiment of the present application, the travel record data of the National Household Travel Survey (NHTS) database is selected to construct a probability model of uncertain demand of electric vehicles. The travel data generally refers to electric vehicle travel record data or GPS data of private electric vehicles in urban traffic scenarios. The urban travel scenarios can include residential areas, office areas, commercial public areas, and other urban area types.

[0040] Under the premise of assuming that the driving habits of electric vehicle users are consistent with those of ordinary fuel vehicle users, various traffic travel survey data can also be used. Such data generally have a relatively mature mechanism and high accuracy.

[0041] The uncertain demand refers to the randomness and volatility of the load generated by electric vehicles on the power system at different time periods and in different space types due to significant individual differences and volatility of electric vehicle users in terms of travel time, travel mileage, parking time, and charging behavior. The uncertain demand considered in the embodiment of the present application includes the uncertain access state of electric vehicles in the charging and discharging infrastructure space, the uncertain time window that can participate in charging and discharging, and the uncertain demand for charging energy.

[0042] The travel record data generally includes information such as departure location, start and end time, and travel purpose, and can truly reflect the behavior patterns and vehicle use rules of drivers, and can provide key features for modeling the uncertainty of electric vehicle charging and discharging, including: travel start and end time distribution, travel starting location and travel purpose type, travel distance distribution, travel duration, and range anxiety.

[0043] The travel start and end time distribution represents the parking and dispatchable period of the electric vehicle, reflecting the demand uncertainty in the time dimension; the travel starting location and travel purpose type can be associated with the dispatchability of electric vehicles in a specific space type, reflecting the demand uncertainty in space; the travel distance distribution can directly determine the battery energy consumption and charging energy demand, reflecting the demand uncertainty in energy; the travel duration can affect the battery state change and charging energy demand, reflecting the demand uncertainty in energy; and the range anxiety can be associated with the charging and discharging behavior patterns of electric vehicle users, reflecting the demand uncertainty in energy. The input data of the probability model of uncertain demand of electric vehicles includes: travel ID, travel purpose or travel purpose type, start and end time, duration, travel distance, and range anxiety.

[0044] The uncertainty in the demand for electric vehicles exhibits the Markov property, meaning that the next travel destination depends only on the current departure point and time, not on previous trips. Furthermore, travel destinations are associated with specific spatial types, and the same spatial type can be associated with multiple travel destinations. In one feasible implementation, during a trip, the electric vehicle determines its next travel destination based on the current time and departure point.

[0045] In one feasible implementation, the probability distributions of travel under different types of spaces can be extracted using the conditional probabilities of the following formula (1):

[0046] (1)

[0047] in, Indicates the starting point of an electric vehicle trip; Indicates the purpose of a trip in an electric vehicle; Indicates starting from a specific location For a specific travel purpose The conditional probability of departure; Indicates starting from a specific location The probability of departure; Indicates starting from a specific location Departure and purpose of departure The probability of both occurring simultaneously.

[0048] One feasible implementation method is, for example Figure 2 The diagram illustrates the spatiotemporal path of an electric vehicle according to an embodiment of the present invention. This embodiment uses a Markov chain model to simulate the travel sequence of the electric vehicle moving between different spatial types, recording a temporally coherent spatial transfer path. The spatial types include residential areas, work areas, and commercial areas. To describe a standard sequence within the chain of uncertain demands for electric vehicles, this embodiment uses the following feature parameters: departure time, departure location, travel destination, travel time, travel distance, and parking duration at the travel destination. A standard sequence within the chain of uncertain demands for electric vehicles can be represented as a feature parameter vector. ,in, Indicates the departure point, Indicates departure time, Indicates travel time, Indicates travel distance; Indicate the purpose of the trip; This indicates the duration of parking at the destination.

[0049] S2, according to the probability model, simulating the Markov chain of the uncertain travel of the electric vehicle, obtaining the chain data of the travel in different space types and in different time periods;

[0050] Optionally, the implementation process of S2 includes S21-S24:

[0051] S21, initializing the travel chain of the electric vehicle in the simulation day, setting the departure place, using the probability model of the use mode of the electric vehicle, generating the departure time of the first travel of the electric vehicle;

[0052] S22, according to the probability model of the use mode of the electric vehicle, randomly generating the travel purpose, travel time, travel distance and parking time of the travel sequence of the electric vehicle in this travel; saving the characteristic parameters of the travel sequence; wherein the characteristic parameters include: the departure time, the departure place, the travel purpose, the travel time, the travel distance and the parking time;

[0053] S23, iteratively updating the initial parameters of the next travel, including the departure time and the departure place, saving the characteristic parameters of the travel sequence obtained;

[0054] S24, repeating steps S22-S23 until the electric vehicle reaches the end state of the travel chain, ending the Markov chain simulation of the electric vehicle, and saving the total number of travels of the electric vehicle in different urban space types.

[0055] Wherein, as shown in Table 1, the electric vehicle Markov chain simulation algorithm.

[0056] Table 1

[0057]

[0058] In a feasible implementation, the execution steps of the above electric vehicle Markov chain simulation algorithm include: first initializing an empty set for storing the travel chain of the electric vehicle , and the number of travel sequences contained in the travel chain ; according to the daily travel probability of the electric vehicle, determining whether the electric vehicle travels in the simulation day, if the electric vehicle travels in the simulation day, setting the departure place of the first travel, and the departure place of each subsequent travel is determined by the travel purpose of the previous travel. According to the time probability distribution of the first travel of the electric vehicle every day, the departure time of the first travel of the electric vehicle is generated , and the departure time of each subsequent travel is calculated from the departure time, travel time and parking time of the previous travel sequence.

[0059] Secondly, start the electric vehicle trip simulation cycle, according to the probability model of the electric vehicle uncertain demand, randomly generate the trip purpose, travel time, trip distance and parking time of the electric vehicle trip sequence, save the characteristic parameters of the trip sequence.

[0060] Finally, if the trip purpose of the trip sequence is the same as the set end state, end the cycle, otherwise, the electric vehicle continues the next trip. Update the trip parameters, add one to the number of electric vehicle trips, and update the departure location and departure time of the next trip according to the foregoing method. If the cycle ends, save the total number of electric vehicle trips.

[0061] Among them, the existing technical method is generally to use a homogeneous electric vehicle demand model, which cannot simulate the complex actual use mode of electric vehicles, and the embodiment of the present application is to construct a probability model of electric vehicle use mode in different space types and different time periods, further simulate the Markov chain of electric vehicle uncertain trip, obtain the chain data of multi-time period trip in different space types, realize high-precision characterization of electric vehicle behavior mode, and accurately capture the spatio-temporal heterogeneity characteristics of the load, overcome the limitation problem of the homogeneous model existing in the prior art.

[0062] S3, screening and integrating the chain data to obtain the multi-parking time period uncertain demand data of the electric vehicle in the space type where the device is located;

[0063] In a feasible implementation manner, wherein the different space types of the city include: residential area, working area and commercial area; the embodiment of the present application is aimed at the charging and discharging optimization strategy of the electric vehicle in a space type, including the multiple discontinuous parking time periods in the residential area, working area and commercial area. The embodiment of the present application takes the residential area as an example for description, and the residential area can be replaced by one of the working area or the commercial area.

[0064] Among them, the embodiment of the present application simulates the multi-parking time period and uncertain energy demand of the electric vehicle in the residential area space type based on the chain data of the electric vehicle uncertain trip obtained by simulation. The parking sequence occurring in the residential area is recorded, and the trip and parking sequence occurring in other space types outside the residential area are merged and fused to integrate the trip distance and duration characteristic parameters of the time period into the next associated parking sequence in the residential area, so as to ensure the consistency of the multi-time period uncertain demand of the electric vehicle in the residential area space and the simulation result of the electric vehicle uncertain demand in step two.

[0065] Optionally, the specific implementation process of S3 includes S31-S33:

[0066] S31, initialize to create an empty set, set the space type of the device as a residential area; input the travel times of the electric vehicle in multiple space types and the corresponding chain data of uncertain travel;

[0067] S32, sequentially extract each group of travel sequences in the chain data of the uncertain demand of the electric vehicle according to the time sequence, obtain a feature parameter vector; according to the travel purpose in the feature parameter vector, determine whether the travel sequence belongs to the parking period of the residential area by screening; wherein the travel purposes associated with the parking period of the residential area include'short stay' and 'overnight parking';

[0068] S33, for the travel sequence associated with the parking period of the residential area, calculate the start time of the parking period, calculate the travel energy consumption according to the travel distance, save the departure time, travel energy consumption, parking duration and parking start time in the feature parameter vector to the set; for the travel sequence not associated with the parking period of the residential area, use the integration method, temporarily store the value of the travel distance in the feature parameter vector, and add it to the travel distance in the feature parameter vector of the next travel sequence;

[0069] S34, repeat steps S32-S33 until all chain data is traversed, save the total number of travel sequences associated with the parking period of the residential area.

[0070] As shown in Figure 3 Fig. 1 is a time distribution simulation result diagram of the parking period of the electric vehicle in the residential area provided by the embodiment of the present application; Figure 3 The curve shown in (a) shows the average parking rate of 1, 10, 30 and 50 electric vehicles in the residential area; Figure 3 The heat map shown in (b) shows the average parking rate change of the number of electric vehicles in the residential area from 1 to 50 in a day; Figure 3 The curve shown in (c) shows the average parking rate of 50 electric vehicles in the residential area at 5:00, 10:00, 15:00 and 20:00; the simulation method used in the embodiment of the present application can simulate the uncertainty of the demand of a single electric vehicle in time and space, and the individual uncertainty demand conforms to the group random behavior mode, from Figure 3 It can be seen from the simulation results that as the number of vehicles increases, the time distribution of the parking rate tends to be stable.

[0071] Table 2 shows the simulation algorithm of the multi-period uncertain demand of the electric vehicle.

[0072] Table 2

[0073]

[0074] In a feasible implementation manner, according to each group of associated residents in the short stay and overnight parking travel sequence in the resident area, obtaining a feature parameter vector , calculating the parking sequence feature parameters of the electric vehicle in the short stay and overnight parking of the resident area, including the departure time , arrival time , parking time at home , and travel distance ; for non-associated travel sequences, the corresponding travel and parking period will be merged into the absence duration ; wherein the binary variable represents whether the electric vehicle departs from the resident area, and m is the number of intermittent parking periods; finally, feature parameter vectors are stored in , and each feature parameter vector can be represented as , wherein represents the travel energy consumption, represents the energy consumption per kilometer of the electric vehicle, and the vector of all travel sequences in is simulated by a cycle mechanism until all travel sequences in are traversed.

[0075] Wherein the short stay parking period charging and discharging scheduling flexibility is ignored in the current prior art research, and the embodiment of the present application considers the overnight parking and short stay two parking modes, distinguishes the overnight parking and short stay of the electric vehicle, and can capture the uncertainty of the electric vehicle under the differentiated use mode and the energy demand.

[0076] S4, according to the multiple parking period uncertainty demand data of the electric vehicle in the space type under the device, calculate the electric vehicle scheduling flexibility constraint coupled in the travel chain of multiple parking periods;

[0077] In a feasible implementation, the embodiment of the present application further judges the scheduling flexibility of the electric vehicle in the non-parking period and the parking period of the electric vehicle by calculating the flexibility of all time steps in the current parking period, so that the coupling relationship of charging and discharging decision between the short stay period and the front and rear parking periods is more closely, and the benefit of electric vehicle charging and discharging optimization can be improved. The embodiment of the present application can efficiently utilize the flexibility potential for load transfer in the relatively short time window of short stay, so as to balance the resource allocation between short stay and overnight parking periods.

[0078] Optionally, the scheduling flexibility includes: time scheduling flexibility, space scheduling flexibility and energy scheduling flexibility;

[0079] The schedulable flexibility in space is determined according to the obtained parking state of the electric vehicle in the residential area, and the electric vehicle can access the charging and discharging infrastructure only when the electric vehicle is in the residential area.

[0080] The schedulable flexibility in time refers to a time window in which the electric vehicle participates in scheduling, and is determined according to the obtained start time and end time of parking of the electric vehicle in the residential area.

[0081] The schedulable flexibility in energy is calculated according to the battery capacity limit and the target charging amount; and the schedulable flexibility in energy is used to set upper and lower limit constraints of the state of charge of the battery of the electric vehicle in all time steps of the current parking period.

[0082] When the electric vehicle is in a non-parking period, that is, , the electric vehicle does not have schedulable flexibility, wherein, denotes the end time of the previous parking period; t denotes the current time step; denotes the start time of the current parking period m; denotes the number of parking periods. When the electric vehicle is in a parking period, that is, , the electric vehicle has schedulable flexibility in time and space, wherein, denotes the start time of the current parking period m; denotes the end time of the current parking period m, and the electric vehicle has schedulable flexibility in time and space.

[0083] Optionally, the calculation process of the schedulable flexibility in energy includes:

[0084] The required power for the uncertain travel demand is calculated according to the driving mileage between two parking periods and the maximum driving mileage of the electric vehicle design;

[0085] The calculation process of the required power for the uncertain travel demand considers the required power for the current travel and all subsequent travels on the travel chain, and is represented by the following formula (2):

[0086] (2)

[0087] Wherein, denotes the required power for the uncertain travel demand; denotes the parking period in which the current time step is located; denotes the number of parking periods; denotes the energy consumption of the m+1th travel; denotes the discharge depth limit; denotes the maximum capacity limit of the battery;

[0088] Wherein, in The calculation considers the energy demand of subsequent multiple time periods. The charging and discharging scheduling strategy of the electric vehicle during the current parking period can strictly meet the energy demand of subsequent travel. The embodiment of the present application considers the influence of the preference factor of the user's charging demand. In actual use, when the user proposes a demand higher than the energy demand of the subsequent travel due to range anxiety, the embodiment of the present application can still ensure that the charging and discharging scheduling strategy of the electric vehicle during the current parking period can strictly meet the energy demand of subsequent travel. By calculating the electric quantity required by uncertain travel demand, the coupling relationship between the energy dispatching flexibility of the current parking period and the energy demand of the subsequent multiple time periods of the travel chain can be constructed.

[0089] According to the energy demand of the user's range anxiety and the electric quantity required by uncertain travel demand, the target charging quantity that the electric vehicle should reach before the end of the parking period is calculated.

[0090] The target charging quantity that the electric vehicle should reach before the end of the parking period is represented by the following formula (3):

[0091] (3)

[0092] Among them, represents the target charging quantity that the electric vehicle should reach before the end of the parking period; represents the energy demand of the user's range anxiety, which represents a psychological expected minimum electric quantity threshold that the user expects to remain in the electric vehicle battery, and reflects the user's individualized charging preference.

[0093] The upper and lower limits of the battery state of charge are used to construct the relationship between the change of the battery state of charge and the charging and discharging optimization scheduling when the electric vehicle has time and space dispatching flexibility.

[0094] When the electric vehicle is in the parking period, the electric vehicle has time and space dispatching flexibility, and the change of the battery state of charge of the electric vehicle is obtained by travel energy consumption calculation.

[0095] When the electric vehicle is in the non-parking period, the electric vehicle does not have dispatching flexibility, and the change of the battery state of charge of the electric vehicle is obtained by calculating the battery state of charge at the end of the previous parking period and the travel power consumption between two parking periods.

[0096] The battery capacity limit includes the maximum capacity limit and the discharge depth limit. In order to protect the battery, the embodiment of the present application sets the discharge depth limit to 20%. The battery state of charge of the electric vehicle follows the law of conservation of energy and the rule of continuous change.

[0097] ​​​wherein the target charging amount is calculated by the minimum acceptable amount of electricity to alleviate range anxiety and the amount of electricity required for uncertain travel demand.

[0098] Optionally, the upper and lower bound constraints of the battery state of charge at all time steps during the current parking period of the electric vehicle are represented by equation (4) as follows:

[0099] (4)

[0100] wherein represents the lower bound constraint of the battery state of charge at the current time step; represents the maximum capacity limit of the battery; represents the battery state of charge at the current time step; represents the discharge depth limit of the battery; represents the rated charging power and discharge efficiency of the electric vehicle; represents the charging efficiency; t represents the current time step; represents the parking duration; represents the parking start time; C represents the battery capacity.

[0101] Optionally, when the electric vehicle is in the parking period, the electric vehicle has time and space schedulable flexibility, and the change of the battery state of charge of the electric vehicle is calculated by travel energy consumption, which is represented by equation (5) as follows:

[0102] (5)

[0103] wherein represents the battery state of charge at the next time step; represents the electric vehicle discharge power at the current time step; represents the length of a time step; represents the charging power at the current time step; represents the discharge efficiency; represents the parking start time of the current parking period m; represents the end time of the current parking period m; represents the number of parking periods;

[0104] wherein when the electric vehicle is in the non-parking period, energy is consumed due to travel, so the battery state of charge of the electric vehicle at the beginning of a parking period is determined by the battery state of charge at the end of the previous parking period and the travel energy consumption between the two parking periods.

[0105] Wherein, when the electric vehicle is in a non-parking period, the electric vehicle does not have a dispatchable flexibility, and the battery state of charge of the electric vehicle is calculated by the battery state of charge at the end of the last parking period and the driving power consumption between the two parking periods, and is represented by the following formula (6):

[0106] (6)

[0107] Wherein, represents the battery state of charge at the beginning of the current parking period; represents the battery state of charge at the end of the last parking period; represents the driving power consumption of the electric vehicle between the last parking period and the current parking period; represents the end time of the last parking period.

[0108] S5, according to the dynamic dispatchable flexibility of the electric vehicle coupled in multiple parking periods on the travel chain, an electric vehicle charging and discharging optimization problem is constructed; according to the electric vehicle charging and discharging optimization problem, a deep reinforcement learning algorithm is used to optimize the charging and discharging strategy of the electric vehicle, and the optimal charging and discharging strategy of the electric vehicle is output.

[0109] Wherein, the embodiment of the application constructs a deep reinforcement learning framework, in terms of timeliness, the framework enables the agent to directly interact with the accurate modeling environment of the multiple-period uncertainty demand of the electric vehicle, thereby mastering the flexibility characteristics of the electric vehicle in different periods. At the same time, the agent can prospectively evaluate the dispatching benefits of multiple periods in the future, efficiently generate charging and discharging control actions, and meet the timeliness requirements of real-time scheduling. In terms of time sequence decision, the application fully utilizes the time sequence reasoning ability of the deep reinforcement learning agent, effectively excavates the charging and discharging flexibility potential contained in multiple parking periods, and accurately estimates the coupling influence between adjacent parking stages charging and discharging strategies, thereby significantly improving the overall optimality of the real-time scheduling strategy in continuous multiple periods.

[0110] In a feasible implementation manner, the embodiment of the application models the electric vehicle charging and discharging optimization problem as a Larkov decision process, which can be represented by ; wherein, represents the state of the current time step environment; represents the state of the next time step environment; represents the action performed by the agent in the environment; represents the reward. Wherein, the state vector is represented as , represents whether the electric vehicle is in a parking period, represents the real-time electricity price, and the state is a continuous state; the action vector is represented as , represents the charging and discharging power of an electric vehicle, represents that the charging and discharging of the electric vehicle is a discrete action, including charging, discharging and non-charging and non-discharging action; wherein the reward function is represented as a weighted sum of user comfort and energy cost, and is represented by the following formula (7):

[0111] (7)

[0112] wherein, represents the first weight factor; represents the second weight factor; represents the user comfort loss, which is a penalty term given by the deep learning agent for violating the multi-period flexibility constraint; by introducing the penalty term, the deep learning agent is inhibited from violating the multi-period flexibility constraint, so as to avoid damaging the comfort of the user; represents the power at the current time step; represents the electricity price at the current time step; represents the length of one time step.

[0113] Among them, the embodiment of the present application designs a reward function by introducing a penalty term, which can guide the agent to make decisions to meet the flexibility constraint through the reward signal; when the action of the agent exceeds the physical constraint range of the electric vehicle battery, the agent is punished, which can accelerate the convergence speed of the agent strategy in a high uncertainty environment.

[0114] Among them, the embodiment of the present application optimizes the discharging strategy of the electric vehicle by using a deep learning algorithm, and the deep reinforcement learning algorithm can select one of a deep Q-learning algorithm and a DDPG algorithm. For example, Figure 4 is an electric vehicle charging and discharging optimization framework based on deep reinforcement learning provided by the embodiment of the present application. Among them, the embodiment of the present application adopts a linear annealing greedy strategy selects an action at each time step; wherein the agent can select an action according to the probability randomly selects an action to explore the action space, or selects an action with a probability of 1- selects an action with the maximum action value based on the current state. After each iteration, gradually decreases at a linear decay rate until a minimum exploration rate is reached. The above method allows the agent to explore a lot in the initial stage, and focuses on utilizing the experience obtained in the later training. In the training process, the initial exploration rate is set to 0.5, which gradually decays to 0.05 after about 500 rounds; the learning rate is set to 0.001, and the discount factor is set to 0.99.

[0115] Among them, for example, Figure 5A comparison chart of the charging and discharging strategy of an electric vehicle is shown, and the comparison chart is provided by an embodiment of the application; Figure 5 Fig. 7 shows the comparison results of the battery state of charge and the charging and discharging power curves of the electric vehicle using the plug-and-charge strategy and the optimization strategy based on the deep reinforcement learning under the real-time electricity price, respectively. Figure 5 It can be seen that by using the deep reinforcement learning optimization framework, the agent can consider the energy demand in the future parking period, take a proactive charging and discharging strategy, increase the discharging potential in the peak period and the charging potential in the valley period, and thus optimize the charging and discharging cost of the user.

[0116] The embodiment of the application first acquires travel record data, fits the probability distribution of the electric vehicle travel and energy use according to the travel record data, constructs a probability model of the electric vehicle use mode according to the probability distribution, and performs Markov chain simulation on the uncertain travel of the electric vehicle according to the probability model to obtain chain data of the multi-period travel under different space types. Secondly, the chain data is screened and integrated to obtain the multi-parking period uncertain demand data of the electric vehicle under the space type where the device is located. According to the multi-parking period uncertain demand data of the electric vehicle under the space type where the device is located, the electric vehicle dispatchable flexibility constraint coupled in the multi-parking period on the travel chain is calculated. Finally, according to the dynamic dispatchable flexibility constraint of the electric vehicle coupled in the multi-parking period on the travel chain, a Markov decision process is constructed, a deep reinforcement learning algorithm is used to optimize the charging and discharging strategy of the electric vehicle, and the real-time optimal charging and discharging strategy of the electric vehicle is output.

[0117] In view of the single time period, static scheduling and local optimal limitation problems existing in the traditional electric vehicle charging and discharging optimization method, the charging and discharging problem of the electric vehicle is extended to multiple time periods in the embodiment of the application, and effective load transfer is carried out in a wide time range. Unlike the existing optimization technology which stays in single parking time period scheduling, the embodiment of the application constructs a scheduling optimization framework based on multi-time period dynamic charging and discharging optimization, fully considers the time sequence coupling characteristics in the electric vehicle charging and discharging process, the dynamic change law of the battery state and the randomness of the user travel demand. Through the deep reinforcement learning optimization method, the embodiment of the application not only realizes the demand response effect of the traditional charging and discharging optimization in the single parking time period under the premise of ensuring the user travel demand, but also realizes the cross-time period load transfer in the intraday scale according to the multi-dimensional information such as the electricity price signal, the future parking time period and the energy demand prediction, conforms to the random use mode of the electric vehicle in reality, and enhances the flexibility potential of the electric vehicle charging and discharging scheduling. The real-time scheduling of the cross-time period load transfer decision can be realized by using the embodiment of the application, and the cross-time period load transfer is realized in multiple discontinuous electric vehicle parking time periods by combining the electricity price signal and based on the future parking time period and the energy demand prediction. The decision mechanism of the embodiment of the application can guarantee the real-time response ability while taking into account the long-term optimization effect, and can meet the real-time scheduling timeliness requirement.

[0118] Figure 6 It is a kind of multi-time period demand electric vehicle charging and discharging real-time optimization scheduling device block diagram shown according to an exemplary embodiment, and the device is used for multi-time period demand electric vehicle charging and discharging real-time optimization scheduling method. Referring to Figure 6 The device includes construction unit 610, first acquisition unit 620, second acquisition unit 630, calculation unit 640 and optimization unit 650. Wherein:

[0119] Construction unit 610 is used to obtain travel record data;According to the travel record data, the probability distribution of electric vehicle travel and energy use is fitted;According to the probability distribution, the probability model of electric vehicle use mode is constructed;

[0120] The first acquisition unit 620 is used to simulate the Markov chain of the electric vehicle uncertainty travel according to the probability model, and obtain the chain data of multi-time period travel under different space types;

[0121] The second acquisition unit 630 is used to screen and integrate the chain data, and obtain the multi-parking time period uncertainty demand data of the electric vehicle in the space type where the device is located;

[0122] The calculation unit 640 is used to calculate the electric vehicle schedulable flexibility constraint coupled in the travel chain in multiple parking time periods according to the multi-parking time period uncertainty demand data of the electric vehicle in the space type where the device is located.

[0123] The optimization unit 650 is configured to construct a Markov decision process according to the dynamic schedulable flexibility constraint of the electric vehicle coupled in the multi-parking period of the trip chain, to optimize the charging and discharging strategy of the electric vehicle by using a deep reinforcement learning algorithm, and to output the real-time optimal charging and discharging strategy of the electric vehicle.

[0124] Optionally, the first acquisition unit 620 is configured to:

[0125] (1) initialize the electric vehicle trip chain of the simulation day, set a departure place, generate a departure time of the first electric vehicle trip by using a probability model of the electric vehicle usage mode;

[0126] (2) randomly generate a trip purpose, a travel time, a trip distance and a parking duration of the trip sequence of the electric vehicle in this trip according to the probability model of the electric vehicle usage mode; save the characteristic parameters of the trip sequence; wherein the characteristic parameters include the departure time, the departure place, the trip purpose, the travel time, the trip distance and the parking duration;

[0127] (3) iteratively update the initial parameters of the next trip, including the departure time and the departure place, and save the characteristic parameters of the obtained trip sequence;

[0128] (4) repeat steps (2)-(3) until the electric vehicle reaches the end state of the trip chain, end the Markov chain simulation of the electric vehicle, and save the total number of trips of the electric vehicle between multiple urban space types.

[0129] Optionally, the second acquisition unit 630 is configured to:

[0130] (1) initialize an empty set, set the space type where the device is located as a residential area, and input the trip number of the electric vehicle in multiple space types and the corresponding chain data of uncertain trips;

[0131] (2) sequentially extract each group of trip sequences in the chain data of the uncertain demand of the electric vehicle according to time sequence, obtain a characteristic parameter vector, and determine whether the trip sequence belongs to the parking period of the residential area according to the trip purpose in the characteristic parameter vector by screening; wherein the trip purposes associated with the parking period of the residential area include “short stay” and “overnight parking”;

[0132] (3) For the travel sequence associated with the parking period of the residential area, the start time of the parking period is calculated, the travel energy consumption is calculated according to the travel distance, and the departure time, travel energy consumption, parking duration and parking start time in the feature parameter vector are saved to the set; for the travel sequence not associated with the parking period of the residential area, the integrated mode is adopted, the value of the travel distance in the feature parameter vector is temporarily stored, and is added to the travel distance in the feature parameter vector of the next travel sequence;

[0133] (4) Circulating steps (2)-(3) until all the chain data is traversed, and the total number of travel sequences associated with the parking period of the residential area is saved.

[0134] Optionally, the schedulable flexibility includes: schedulable flexibility in time, schedulable flexibility in space, and schedulable flexibility in energy;

[0135] The schedulable flexibility in space is determined according to the obtained parking state of the electric vehicle in the residential area, and the electric vehicle can access the charging and discharging infrastructure only when the electric vehicle is in the residential area;

[0136] The schedulable flexibility in time refers to the time window of the electric vehicle participating in scheduling, which is determined according to the obtained start time and end time of the electric vehicle parking in the residential area;

[0137] The schedulable flexibility in energy is calculated according to the battery capacity limit and the target charging amount; and the schedulable flexibility in energy is used to set the upper and lower limit constraints of the battery state of charge of the electric vehicle in all time steps of the current parking period;

[0138] The upper and lower limit constraints of the battery state of charge are used to construct the relationship between the battery state of charge change and the charging and discharging optimization scheduling when the electric vehicle has time and space schedulable flexibility;

[0139] When the electric vehicle is in the parking period, the electric vehicle has time and space schedulable flexibility, and the battery state of charge change of the electric vehicle is obtained by calculating the travel energy consumption;

[0140] When the electric vehicle is in the non-parking period, the electric vehicle does not have schedulable flexibility, and the battery state of charge change of the electric vehicle is obtained by calculating the battery state of charge at the end of the last parking period and the driving power consumption between two parking periods;

[0141] The battery capacity limit includes the maximum capacity limit and the discharge depth limit;

[0142] The target charging amount is obtained by calculating the minimum acceptable amount of electricity to alleviate range anxiety and the amount of electricity required for uncertain travel demand.

[0143] Optionally, the calculation process of the schedulable flexibility on the energy comprises:

[0144] According to the driving mileage between two parking time periods and the maximum driving mileage of the electric vehicle, the required electricity of the uncertain travel demand is calculated;

[0145] Wherein, the calculation process of the required electricity of the uncertain travel demand considers the required electricity of the current travel and all subsequent travels on the travel chain, and is represented by the following formula (1):

[0146] (1)

[0147] Wherein, represents the required electricity of the uncertain travel demand; represents the parking period in which the current time step is located; represents the number of parking periods; represents the energy consumption of the m+1th travel; represents the discharge depth limit; represents the maximum capacity limit of the battery;

[0148] According to the minimum acceptable electricity for relieving range anxiety and the required electricity of the uncertain travel demand, the target charging amount that the electric vehicle should reach before the end of the parking phase is calculated;

[0149] Wherein, the target charging amount that the electric vehicle should reach before the end of the parking phase is represented by the following formula (2):

[0150] (2)

[0151] Wherein, represents the target charging amount that the electric vehicle should reach before the end of the parking phase; represents the minimum acceptable electricity for relieving range anxiety, representing a psychological expected minimum electricity threshold that the user expects to reserve in the battery of the electric vehicle, reflecting the user's personalized charging preference.

[0152] Optionally, the upper and lower limit constraints of the battery state of charge of the electric vehicle at all time steps in the current parking period are represented by the following formula (3):

[0153] (3)

[0154] Wherein, represents the lower limit constraint of the battery state of charge at the current time step; represents the maximum capacity limit of the battery; represents the battery state of charge at the current time step; represents the discharge depth limit of the battery; denotes the rated charging power and discharging efficiency of the electric vehicle; denotes the charging efficiency; t denotes the current time step; denotes the parking duration; denotes the parking start time; C denotes the battery capacity.

[0155] Optionally, when the electric vehicle is in the parking period, the electric vehicle has time and space schedulable flexibility, the battery state of charge change of the electric vehicle is obtained by travel energy consumption calculation, and is represented by the following formula (4):

[0156] (4)

[0157] wherein, denotes the battery state of charge at the next time step; denotes the electric vehicle discharging power at the current time step; denotes the length of one time step; denotes the charging power at the current time step; denotes the discharging efficiency; denotes the parking start time of the current parking period m; denotes the end time of the current parking period m; denotes the number of parking periods;

[0158] wherein, when the electric vehicle is in the non-parking period, the electric vehicle does not have schedulable flexibility, the battery state of charge change of the electric vehicle is obtained by the battery state of charge at the end of the last parking period and the travel energy consumption between two parking periods, and is represented by the following formula (5):

[0159] (5)

[0160] wherein, denotes the battery state of charge at the start of the current parking period; denotes the battery state of charge at the end of the last parking period; denotes the travel energy consumption of the electric vehicle between the last parking period and the current parking period; denotes the end time of the last parking period.

[0161] The embodiment of the application firstly acquires travel record data; according to the travel record data, fitting the probability distribution of electric vehicle travel and energy use; according to the probability distribution, constructing the probability model of electric vehicle use mode; according to the probability model, Markov chain simulation is carried out on the uncertain travel of the electric vehicle, and chain data of multi-period travel under different space types is obtained; secondly, the chain data is screened and integrated, and the multi-parking period uncertain demand data of the electric vehicle under the space type where the device is located is obtained; according to the multi-parking period uncertain demand data of the electric vehicle under the space type where the device is located, the electric vehicle schedulable flexibility constraint coupled in the travel chain of the multi-parking period is calculated; finally, according to the dynamic schedulable flexibility constraint of the electric vehicle coupled in the travel chain of the multi-parking period, a Markov decision process is constructed, a deep reinforcement learning algorithm is used to optimize the charging and discharging strategy of the electric vehicle, and the real-time charging and discharging optimal strategy of the electric vehicle is output.

[0162] In view of the single period, static scheduling and local optimal limitation problems existing in the traditional electric vehicle charging and discharging optimization method, the charging and discharging problem of the electric vehicle is expanded to multiple periods in the embodiment of the application, and effective load transfer is carried out in a wide time range. Unlike the existing optimization technology which stays in single parking period scheduling, the embodiment of the application constructs a scheduling optimization framework based on multi-period dynamic charging and discharging optimization, fully considers the time sequence coupling characteristics in the electric vehicle charging and discharging process, the dynamic change law of the battery state and the randomness of the user travel demand. Through the deep reinforcement learning optimization method, the embodiment of the application not only can realize the demand response effect of the traditional charging and discharging optimization in a single parking period under the premise of ensuring the user travel demand, but also can realize cross-period load transfer on the intraday scale according to the electricity price signal, future parking period and energy demand prediction and other multi-dimensional information, conforms to the uncertain random use mode of the electric vehicle in reality, and enhances the flexibility potential of the electric vehicle charging and discharging scheduling. The real-time scheduling of the cross-period load transfer decision can be realized by using the embodiment of the application, and the cross-period load transfer can be realized in multiple discontinuous electric vehicle parking periods by combining the electricity price signal and based on the future parking period and energy demand prediction. The decision mechanism can guarantee the real-time response ability while taking into account the long-term optimization effect by using the embodiment of the application, and can meet the real-time scheduling timeliness requirement.

[0163] Figure 7 is a structural schematic diagram of a multi-period demand electric vehicle charging and discharging real-time optimization scheduling device provided by the embodiment of the application, as Figure 7 shown, the multi-period demand electric vehicle charging and discharging real-time optimization scheduling device can include the multi-period demand electric vehicle charging and discharging real-time optimization scheduling apparatus shown in the above Figure 6 . Optionally, the multi-period demand electric vehicle charging and discharging real-time optimization scheduling device 710 can include a first processor 2001.

[0164] Optionally, the real-time optimization scheduling device 710 for electric vehicle charging and discharging with multi-period demand may also include a memory 2002 and a transceiver 2003.

[0165] The first processor 2001, memory 2002, and transceiver 2003 can be connected via a communication bus.

[0166] The following is combined with Figure 7 The following is a detailed introduction to the various components of the 710 electric vehicle charging and discharging real-time optimization scheduling equipment with multi-period demand:

[0167] The first processor 2001 is the control center of the real-time optimized scheduling device 710 for electric vehicle charging and discharging with multi-period demand. It can be a single processor or a collective term for multiple processing elements. For example, the first processor 2001 can be one or more central processing units (CPUs), application-specific integrated circuits (ASICs), or one or more integrated circuits configured to implement embodiments of the present invention, such as one or more digital signal processors (DSPs), or one or more field-programmable gate arrays (FPGAs).

[0168] Optionally, the first processor 2001 can execute various functions of the real-time optimization scheduling device 710 for electric vehicle charging and discharging based on multi-period requirements by running or executing software programs stored in the memory 2002 and calling data stored in the memory 2002.

[0169] In a specific implementation, as one example, the first processor 2001 may include one or more CPUs, for example... Figure 7 CPU0 and CPU1 are shown in the diagram.

[0170] In a specific implementation, as one example, the real-time optimization scheduling device 710 for electric vehicle charging and discharging with multi-period demand may also include multiple processors, for example... Figure 7 The first processor 2001 and the second processor 2004 are shown in the diagram. Each of these processors can be a single-core processor or a multi-core processor. Here, a processor can refer to one or more devices, circuits, and / or processing cores used to process data (such as computer program instructions).

[0171] The memory 2002 is configured to store a software program for implementing the scheme of the present application, and the first processor 2001 is configured to control the execution of the software program. The specific implementation can refer to the method embodiments described above, and will not be described here.

[0172] Alternatively, the memory 2002 can be a read-only memory (ROM) or other type of static storage device that can store static information and instructions, a random access memory (RAM) or other type of dynamic storage device that can store information and instructions, an electrically erasable programmable read-only memory (EEPROM), a compact disc read-only memory (CD-ROM) or other optical disk storage, a magneto-optical disk storage (including a compact disk, a laser disc, an optical disc, a digital versatile disc, a Blu-ray disc, etc.), a magnetic disk storage medium or other magnetic storage device, or any other medium that can be used to carry or store desired program code in the form of instructions or data structures and that can be accessed by a computer, but is not limited to this. The memory 2002 can be integrated with the first processor 2001, or can exist independently, and is coupled to the first processor 2001 through an interface circuit (not shown in the figure) of the multi-period demand electric vehicle charging and discharging real-time optimization scheduling device 710. The embodiments of the present application do not make specific limitations here. Figure 7

[0173] The transceiver 2003 is configured to communicate with a network device or a terminal device.

[0174] Alternatively, the transceiver 2003 can include a receiver and a transmitter (not shown separately in the figure). The receiver is configured to implement the receiving function, and the transmitter is configured to implement the transmitting function. Figure 7

[0175] Alternatively, the transceiver 2003 can be integrated with the first processor 2001, or can exist independently, and is coupled to the first processor 2001 through an interface circuit (not shown in the figure) of the multi-period demand electric vehicle charging and discharging real-time optimization scheduling device 710. The embodiments of the present application do not make specific limitations here. Figure 7

[0176] It should be noted that the structure of the multi-period demand electric vehicle charging and discharging real-time optimization scheduling device 710 shown in the figure does not constitute a limitation on the router. The actual knowledge structure recognition device can include more or fewer components than those shown in the figure, or combine certain components, or different component arrangements. Figure 7 ​​​​

[0177] In addition, the technical effects of the electric vehicle charging and discharging real-time optimization scheduling device 710 according to the multi-period demand can refer to the technical effects of the multi-period demand electric vehicle charging and discharging real-time optimization scheduling method according to the method embodiments described above, and will not be repeated here.

[0178] It should be understood that the first processor 2001 in the embodiments of the present application can be a central processing unit (CPU), and the processor can also be other general-purpose processors, digital signal processors (DSP), application specific integrated circuits (ASIC), ready programmable gate arrays (FPGA) or other programmable logic devices, discrete gates or transistor logic devices, discrete hardware components, etc. The general-purpose processor can be a microprocessor or the processor can also be any conventional processor.

[0179] It should also be understood that the memory in the embodiments of the present application can be a volatile memory or a non-volatile memory, or can include both volatile and non-volatile memories. Among them, the non-volatile memory can be a read-only memory (ROM), a programmable read-only memory (PROM), an erasable programmable read-only memory (EPROM), an electrically EPROM (EEPROM) or a flash memory. The volatile memory can be a random access memory (RAM) used as an external cache. By way of example but not limitation, many forms of random access memory (RAM) are available, such as static random access memory (SRAM), dynamic random access memory (DRAM), synchronous dynamic random access memory (SDRAM), double data rate synchronous dynamic random access memory (DDR SDRAM), enhanced synchronous dynamic random access memory (ESDRAM), synchronous link dynamic random access memory (SLDRAM) and direct memory bus random access memory (direct rambus RAM, DR RAM).

[0180] The above-described embodiments can be implemented in whole or in part by software, hardware (such as a circuit), firmware, or any combination thereof. When implemented in software, the above-described embodiments can be implemented in the form of a computer program product. The computer program product includes one or more computer instructions or computer programs. When the computer instructions or computer programs are loaded or executed on a computer, the processes or functions described in the embodiments of the present application are wholly or partially generated. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable devices. The computer instructions can be stored in a computer-readable storage medium or transferred from one computer-readable storage medium to another computer-readable storage medium, for example, the computer instructions can be transferred from one website, computer, server, or data center to another website, computer, server, or data center through a wired (such as infrared, wireless, microwave, etc.) manner. The computer-readable storage medium can be any available medium accessible by a computer or a data storage device such as a server, data center, etc. containing one or more available medium collections. The available medium can be a magnetic medium (such as a floppy disk, a hard disk, a magnetic tape), an optical medium (such as a DVD), or a semiconductor medium. The semiconductor medium can be a solid-state disk.

[0181] It should be understood that the term "and / or" herein merely describes an association relationship of associated objects, which means that there can be three relationships, for example, A and / or B can represent three cases of A alone, A and B together, and B alone, where A and B can be singular or plural. In addition, the character " / " herein generally represents an "or" relationship between the front and rear associated objects, but can also represent an "and / or" relationship, which can be understood in the context before and after.

[0182] In the present application, "at least one" means one or more, and "multiple" means two or more. "At least one of the following" or the like means any combination of the items, including any combination of single or multiple items. For example, at least one of a, b, or c can represent a, b, c, a-b, a-c, b-c, or a-b-c, where a, b, and c can be single or multiple.

[0183] It should be understood that in various embodiments of the present application, the size of the sequence number of the above-described processes does not mean the order of execution, and the execution order of the processes should be determined by their functions and internal logic, and should not constitute any limitation on the implementation process of the embodiments of the present application.

[0184] Those skilled in the art can clearly understand that the units and algorithm steps of each example described in combination with the embodiments disclosed herein can be realized by electronic hardware or a combination of computer software and electronic hardware. Whether the functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of the present application.

[0185] Those skilled in the art can clearly understand that, for the convenience and brevity of the description, the specific working processes of the devices, apparatuses and units described above can refer to the corresponding processes in the foregoing method embodiments, which will not be repeated here.

[0186] In several embodiments provided by the present application, it should be understood that the disclosed devices, apparatuses and methods can be implemented in other ways. For example, the apparatus embodiments described above are merely schematic, for example, the division of the units is only a logical function division, and actual implementation can have another division manner, for example, multiple units or components can be combined or integrated into another device, or some features can be ignored or not executed. In addition, the coupling or direct coupling or communication connection between the units shown or discussed can be indirect coupling or communication connection through some interfaces, devices or units, which can be electrical, mechanical or other forms.

[0187] The units described as separate components can or can not be physically separated, and the components shown as units can or can not be physical units, that is, they can be located in one place, or can be distributed on multiple network units. Part or all of the units can be selected according to actual needs to achieve the purpose of the embodiment scheme.

[0188] In addition, each functional unit in each embodiment of the present application can be integrated into a processing unit, or each unit can exist physically independently, or two or more units can be integrated into one unit.

[0189] If the functions are realized in the form of software function units and sold or used as independent products, they can be stored in a computer readable storage medium. Based on this understanding, the technical solutions of the present application or the parts of the present application that essentially contribute to the prior art or the parts of the technical solutions can be embodied in the form of software products. The computer software product is stored in a storage medium and includes a plurality of instructions for causing a computer device (which can be a personal computer, a server, or a network device, etc.) to execute all or part of the steps of the method described in the various embodiments of the present application. The aforementioned storage medium includes a U disk, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk, and various media that can store program codes.

[0190] The above is only a specific implementation of the present application, but the protection scope of the present application is not limited thereto. Any person skilled in the art can easily think of changes or replacements within the technical range disclosed by the present application, which should be covered within the protection scope of the present application. Therefore, the protection scope of the present application should be subject to the protection scope of the claims.

Claims

1. A method for real-time optimization scheduling of multi-period demand for electric vehicle charging and discharging, characterized in that, The method comprises: S1, obtaining travel record data; fitting the probability distribution of electric vehicle travel and energy use according to the travel record data; and constructing a probability model of electric vehicle use mode according to the probability distribution; S2, simulating the Markov chain of the electric vehicle uncertain travel according to the probability model to obtain chain data of multi-period travel under different space types; S3, screening and integrating the chain data to obtain multi-parking period uncertain demand data of the electric vehicle under the space type where the device is located; S4, calculating the electric vehicle dispatchable flexibility constraint coupled with the multi-parking period on the travel chain according to the multi-parking period uncertain demand data of the electric vehicle under the space type where the device is located; The dispatchable flexibility includes time, space and energy dispatchable flexibility; The space dispatchable flexibility is determined according to the obtained parking state of the electric vehicle in the residential area, and the electric vehicle can access the charging and discharging infrastructure only when it is in the residential area; The time dispatchable flexibility refers to the time window of the electric vehicle participating in the dispatch, which is determined according to the obtained start time and end time of the electric vehicle parking in the residential area; The energy dispatchable flexibility is calculated according to the battery capacity limit and the target charging amount, and is used to set the upper and lower limit constraints of the battery state of charge of the electric vehicle at all time steps in the current parking period; The upper and lower limit constraints of the battery state of charge are used to build the relationship between the battery state of charge change and the charging and discharging optimization dispatch when the electric vehicle has time and space dispatchable flexibility; When the electric vehicle is in the parking period, the electric vehicle has time and space dispatchable flexibility, and the battery state of charge change of the electric vehicle is obtained by calculating the travel energy consumption; When the electric vehicle is in the non-parking period, the electric vehicle has no dispatchable flexibility, and the battery state of charge change of the electric vehicle is obtained by calculating the battery state of charge at the end of the last parking period and the travel power consumption between two parking periods; The battery capacity limit includes the maximum capacity limit and the discharge depth limit; The target charging amount is calculated by the minimum acceptable amount of electricity to alleviate the range anxiety and the amount of electricity required by the uncertain travel demand; S5, constructing a Markov decision process according to the dynamic dispatchable flexibility constraint of the electric vehicle coupled with the multi-parking period on the travel chain, and using a deep reinforcement learning algorithm to optimize the charging and discharging strategy of the electric vehicle, and outputting the real-time optimal charging and discharging strategy of the electric vehicle. 2.The multi-period demand real-time optimization scheduling method for EV charging and discharging according to claim 1, wherein, S2, simulating the Markov chain of the electric vehicle uncertain travel according to the probability model to obtain chain data of multi-period travel under different space types, comprises: S21, initializing the electric vehicle travel chain of the simulation day, setting the departure place, generating the departure time of the first electric vehicle travel by using the probability model of electric vehicle use mode; S22, according to the probability model of the use mode of the electric vehicle, randomly generating the trip purpose, travel time, trip distance and parking duration of the current trip sequence of the electric vehicle; saving the characteristic parameters of the current trip sequence; wherein the characteristic parameters include: departure time, departure location, trip purpose, travel time, trip distance and parking duration; S23, iteratively updating the initial parameters of the next trip, including the departure time and the departure location, and saving the characteristic parameters of the obtained trip sequence; S24, repeating steps S22-S23 until the electric vehicle reaches the end state of the trip chain, ending the Markov chain simulation of the electric vehicle, and saving the total number of trips of the electric vehicle between multiple urban space types. 3.The multi-period demand real-time optimization scheduling method for EV charging and discharging according to claim 1, wherein, The S3 filters and integrates the chain data to obtain the uncertainty demand data of the electric vehicle in the space type under the device, including: S31, initializing an empty set and setting the space type of the device as a residential area; inputting the number of trips of the electric vehicle in multiple space types and the corresponding chain data of uncertain trips; S32, extracting each group of trip sequences in the chain data of the electric vehicle's uncertain demand in time sequence, obtaining a characteristic parameter vector; according to the trip purpose in the characteristic parameter vector, determining whether the trip sequence belongs to the residential parking period by filtering; wherein the trip purposes associated with the residential parking period include "short stay" and "overnight parking"; S33, for the trip sequence associated with the residential parking period, calculating the start time of the parking period, calculating the travel energy consumption according to the trip distance, and saving the departure time, travel energy consumption, parking duration and parking start time in the characteristic parameter vector to the set; for the trip sequence not associated with the residential parking period, using the integration method, temporarily storing the value of the trip distance in the characteristic parameter vector, and adding it to the trip distance in the characteristic parameter vector of the next trip sequence; S34, repeating steps S32-S33 until all chain data is traversed, saving the total number of trip sequences associated with the residential parking period. 4.The multi-period demand real-time optimization scheduling method for electric vehicle charging and discharging according to claim 1, wherein, The calculation process of the schedulable flexibility in energy includes: According to the travel mileage between the two parking time periods and the maximum travel mileage of the electric vehicle, the required power of the uncertain trip demand is calculated; Wherein, the calculation process of the required power of the uncertain trip demand considers the required power of the current trip and all subsequent trips on the trip chain, which is represented by the following formula (1): where SOC a represents the required power for uncertain travel demand; m represents the parking period in which the current time step is located; N sp represents the number of parking periods; represents the energy consumption of the m+1th travel; SOC dod represents the discharge depth limit; SOC max represents the battery maximum capacity limit; C represents the battery capacity; According to the minimum acceptable power for relieving range anxiety and the required power of the uncertain trip demand, the target charging amount that the electric vehicle should reach before the end of the parking phase is calculated; Wherein, the target charging amount that the electric vehicle should reach before the end of the parking phase is represented by the following formula (2): SOC exp = max(SOC ma , SOC a ) (2) wherein SOC exp represents the target state of charge that the electric vehicle should reach at the end of the parking phase; SOC ma represents the minimum acceptable state of charge to alleviate range anxiety, representing a psychological expected minimum charge threshold that the user desires to be left in the electric vehicle's battery, reflecting the user's individualized charging preferences. 5.The multi-period demand real-time optimization scheduling method for electric vehicle charging and discharging according to claim 1, wherein, The upper and lower limit constraints of the state of charge of the battery of the electric vehicle at all time steps in the current parking period are represented by the following formula (3): SOC min,t ≤ SOC t ≤ SOC max where SOC min,t represents the lower bound constraint on the state of charge of the battery at the current time step; SOC max represents the maximum capacity constraint on the battery; SOC t represents the state of charge of the battery at the current time step; SOC dod represents the depth of discharge constraint on the battery; P r represents the rated charging power and discharging efficiency of the electric vehicle; η ch represents the charging efficiency; t represents the current time step; represents the parking duration; represents the start time of parking; C represents the battery capacity; SOC exp represents the target charge amount that the electric vehicle should reach before the end of the parking phase. 6.The multi-period demand real-time optimization scheduling method for electric vehicle charging and discharging according to claim 1, wherein, When the electric vehicle is in the parking period, the electric vehicle has schedulable flexibility in time and space, and the change of the state of charge of the battery of the electric vehicle is obtained by calculating the travel energy consumption, which is represented by the following formula (4): where SOC t+1 represents the battery state of charge at the next time step; P dch,t represents the electric vehicle discharge power at the current time step; Δt represents the length of one time step; P ch，t represents the charging power at the current time step; η dch represents the discharge efficiency; represents the start time of the current parking period m; represents the end time of the current parking period m; N sp represents the number of parking periods; η ch represents the charging efficiency; C represents the battery capacity; SOC t represents the battery state of charge at the current time step; Wherein, when the electric vehicle is in a non-parking period, the electric vehicle does not have schedulable flexibility, the battery state of charge of the electric vehicle is calculated by the battery state of charge at the end of the last parking period and the driving power consumption between two parking periods, and is represented by the following formula (5): wherein, represents the state of charge of the battery at the start of the current parking period; represents the state of charge of the battery at the end of the previous parking period; represents the amount of electricity consumed by the electric vehicle for travel between the previous parking period and the current parking period; represents the end time of the previous parking period.

7. A multi-period demand electric vehicle charging and discharging real-time optimization scheduling device for implementing the multi-period demand electric vehicle charging and discharging real-time optimization scheduling method according to any one of claims 1-6. The device comprises: The construction unit is configured to acquire trip record data, fit a probability distribution of electric vehicle trips and energy use according to the trip record data, and construct a probability model of electric vehicle usage patterns according to the probability distribution; The first acquisition unit is configured to perform Markov chain simulation on uncertain electric vehicle trips according to the probability model, and obtain chain data of multiple time periods of trips under different spatial types; The second acquisition unit is configured to filter and integrate the chain data, and obtain multiple parking period uncertain demand data of the electric vehicle under the spatial type where the device is located; The calculation unit is configured to calculate the schedulable flexibility constraint of the electric vehicle coupled with multiple parking periods on the trip chain according to the multiple parking period uncertain demand data of the electric vehicle under the spatial type where the device is located; The optimization unit is configured to construct a Markov decision process according to the schedulable flexibility constraint of the electric vehicle coupled with multiple parking periods on the trip chain, optimize the charging and discharging strategy of the electric vehicle by using a deep reinforcement learning algorithm, and output the real-time optimal charging and discharging strategy of the electric vehicle.

8. A multi-period demand electric vehicle charging and discharging real-time optimization scheduling device, characterized in that, The multi-period demand electric vehicle charging and discharging real-time optimization scheduling device comprises: A processor; A memory, the memory having computer readable instructions stored thereon, the computer readable instructions being executed by the processor to implement the method of any one of claims 1 to 6.

9. A computer readable storage medium, characterized in that, The computer readable storage medium stores program code, which can be called and executed by the processor to implement the method of any one of claims 1 to 6.

Images