Large-scale charging station space-time pricing method and system considering user behaviors

Abstract

The invention relates to a large-scale charging station space-time pricing method and system considering user behaviors. The method comprises the following steps: firstly, dividing a charging station into a plurality of areas, and collecting charging demand data, power grid end electricity price and charging station area distribution; constructing an NSGAII algorithm based on reinforcement learning, and constructing a space-time pricing optimization model based on the algorithm; and finally, inputting the collected charging demand data, the power grid end electricity price and the charging station location distribution into a space-time pricing optimization model, simultaneously considering time and space factors in the optimization process, taking the charging station operation efficiency and charging load balance as optimization targets, and finally outputting the optimal pricing of each region at each moment. Compared with the prior art, the method has the advantages that long-term high load or idling of the transformer in the local area is avoided, healthy and stable operation of charging equipment is guaranteed, and the overall resource utilization efficiency of the charging station is improved.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of electric vehicles, in particular to a large charging station space-time pricing method and system considering user behavior. BACKGROUND

[0002] With the popularity of electric vehicles, the number of electric vehicles is gradually increasing, and the demand for public charging facilities is also growing.

[0003] Large charging stations in commercial and office areas have the characteristics of large area and wide area. The charging convenience of different areas in the charging station for drivers is different. According to the on-the-spot investigation, the charging fee of the public charging station is composed of two parts of electricity fee and service fee, wherein the electricity fee is formulated by the power supply network, and the service fee is formulated by the charging station operator. The service fee pricing mode of most public charging stations is uniform pricing, that is, the service fee of all time and all charging spaces is consistent.

[0004] In the scenario of large charging stations, this uniform pricing method will cause the transformer in the charging area with high charging convenience to be in a high load state for a long time, and the transformer in the charging area with low charging convenience to be in an idle state for a long time, which is not conducive to the long-term healthy and stable operation of the charging equipment.

[0005] On the one hand, uniform pricing cannot guide users to reasonable distribution, resulting in unbalanced allocation of charging resources. Since there is no difference in the use cost of convenient areas and other areas, users generally tend to choose areas with high accessibility for charging, so that the charging piles and supporting transformers in these popular areas are in a high load operation state for a long time, which not only easily causes problems such as overheating of equipment, frequent failures, and shortens the service life of the charging equipment, but also may cause charging efficiency to decline and waiting time to be prolonged due to overload, aggravating the user's "charging anxiety"; while the areas with lower convenience are in an idle state for a long time, the utilization rate of the charging piles is insufficient, causing waste of resources in charging facility construction investment On the other hand, the uniform pricing mode lacks the response ability to market supply and demand changes, which is not conducive to the improvement of user experience and restricts the healthy development of the industry. In the peak period of electricity consumption, the congestion problem in popular areas is particularly prominent, and uniform pricing cannot adjust demand through price leverage; in the valley period, it is also difficult to attract users to choose idle areas, resulting in uneven resource utilization efficiency throughout the day. At the same time, this pricing method makes the user experience worse, and the charging chaos further aggravates the difficulty of user decision-making, making the supply and demand contradiction that can be alleviated through scientific pricing more prominent.

[0006] In summary, the uniform pricing mode of the existing large charging station cannot fully consider the spatial convenience difference, time supply and demand fluctuation and user selection behavior, leading to unbalanced charging resource allocation, increased equipment operation risk and low operation efficiency, and has been difficult to adapt to the energy supplement service demand under the background of popularization of electric vehicles. Therefore, it is urgent to build a differentiated pricing mechanism considering the spatial and temporal differences and user behavior to solve the core pain points currently faced by the industry and promote the development of public charging services in an efficient, balanced and sustainable direction. SUMMARY

[0007] The purpose of the present application is to overcome the defects of the prior art and provide a large charging station space-time pricing method and system considering user behavior.

[0008] The purpose of the present application can be achieved by the following technical solutions: According to one aspect of the present application, a large charging station space-time pricing method considering user behavior is provided, and the method steps include: S1, the charging station is divided into multiple areas, and charging demand data, grid end price and charging station location distribution are collected; S2, an NSGAII algorithm based on reinforcement learning is constructed, and a space-time pricing optimization model is constructed based on the algorithm; S3, the collected charging demand data, grid end price and charging station location distribution are input into the space-time pricing optimization model, and the factors of time and space are considered in the optimization process, taking the charging station operation efficiency and charging load balance as the optimization target, and finally outputting the optimal pricing of each area at each time.

[0009] As a preferred technical solution, the execution process of the NSGAII algorithm based on reinforcement learning includes: S31, encode the required decision variable, i.e. the charging service fee of the jth area at time t, into a matrix of T rows and J columns to form a chromosome matrix; each element in the chromosome is the pricing of the charging service fee of each sub-area at time t; S32, take the chromosome matrix as an individual, randomly generate an initial population of size N, and initialize the Q value table to all 0 table; S33, calculate the target value of each individual in the population, define and calculate the population State, and the target value includes the first target value and the second target value; wherein the first target value is the charging station operation efficiency, and the second target value is the charging load balance index; S34, select Action using the ε-greedy strategy, and the action is defined as the selection probability of the combination of crossover and mutation operators; S35, execute the selected Action, take the current latest generation population as the parent population, and perform crossover and mutation on the parent population individuals to generate a child population with the same size as the parent population; S36, calculate the target value of the sub-population and the population State, compare the population State of the parent and the child, and update the Q-value table based on the comparison result; S37, merge the parent and child populations, perform fast non-dominated sorting and crowded distance calculation on the merged population, select the optimal N individuals according to the elite strategy to form a new generation population; S38, calculate the hyper-volume and distance index of the current latest generation population, if the hyper-volume and distance index converge or reach the preset maximum iteration number, output the Pareto optimal solution set in the current latest generation population as the optimal pricing of each region at each time; otherwise, return to execute step S33.

[0010] As a preferred technical solution, the pricing of the charging service fee of each sub-region at time t in S31 needs to meet the price limit [0.5, 1.3].

[0011] Limiting the charging service fee in the range of [0.5, 1.3] ensures that the pricing meets the price regulation requirements, avoids abnormal price fluctuations, and balances the user payment rationality and the charging station operation compliance.

[0012] As a preferred technical solution, the population State is a four-tuple, including: the normalized average value of the first target value of the population individual, the normalized average value of the second target value of the population individual, the normalized average value of the first target value of the population Pareto frontier individual, and the normalized average value of the second target value of the population Pareto frontier individual.

[0013] As a preferred technical solution, the specific process of comparing the population State of the parent and the child and updating the Q-value table in S36 is: comparing the population State of the parent and the child, comparing each dimension in the four-tuple respectively; if there is improvement in a certain dimension, the Reward result of the dimension is assigned as 1; if there is no change in a certain dimension, the Reward result of the dimension is assigned as 0; otherwise, the Reward result of the dimension is assigned as -1; the Reward value is composed of the Reward results of the four dimensions, and the interval range of the Reward value is [-4, 4]; update the Q-value value based on the Reward value, and the update of the Q-value value follows the time difference implementation of the Bellman optimal equation, and the specific formula is: wherein, is the current Q-value of the state and the action performed at time , which represents the expected long-term reward of the state-action pair; is the updated Q-value at time The Q-value is the new expectation after the algorithm learns; The learning rate controls the magnitude of Q-value updates; The reward value represents the state. Next action Then, feedback on changes in the population state; s is the discount factor; s is the state; For action.

[0014] As a preferred technical solution, the learning rate is adaptively adjusted based on the access frequency and temporal difference error of the state-action pair, that is, the learning rate satisfies the following formula: in, For a moment State-action pairs The adaptive learning rate is the learning rate mentioned above. Dynamic adjustment; State-action pairs Number of visits; This refers to timing difference error; This represents the average temporal difference error over the most recent 100 generations. is the frequency attenuation coefficient, and is a preset hyperparameter.

[0015] As a preferred technical solution, the charging station operating efficiency in S33 is the total charging operating efficiency of all charging areas within the decision time period, and its specific formula is as follows: in, To improve the operating efficiency of charging stations; Indicates time area The actual number of vehicles receiving charging services; Average charging power; Unit duration; for Constant charging area The charging efficiency; The total number of moments within the decision-making time period; This represents the total number of charging areas.

[0016] As a preferred technical solution, the charging load balancing index in S33 is the sum of the load deviations of each charging area from the average load at each time point within the decision-making period, and its specific formula is as follows: wherein, is a charging load balancing index; is the time average load of each charging spot in the charging area , is the time average load of each charging spot in the charging station, and are respectively the positive deviation value and the negative deviation value of the average load of each charging spot in the charging area and the average load of each charging spot in the charging station.

[0017] As a preferred technical solution, the value of the -greedy strategy in S34 is set to 0.9, that is, there is a 90% probability of selecting the action with the maximum Q-value under the current population State, and a 10% probability of randomly selecting the action. The setting of =0.9 in the -greedy strategy reserves a small amount of exploration space while utilizing high-quality operator combinations with a high probability, which can efficiently utilize existing evolutionary experience and avoid falling into local optimum, and guarantee the diversity and quality of the solution.

[0018] The setting of =0.9 in the -greedy strategy reserves a small amount of exploration space while utilizing high-quality operator combinations with a high probability, which can efficiently utilize existing evolutionary experience and avoid falling into local optimum, and guarantee the diversity and quality of the solution.

[0019] According to another aspect of the present application, a large charging station space-time pricing system considering user behavior is provided, which comprises a data acquisition module, an algorithm and model construction module, and a pricing optimization module; The data acquisition module divides the charging station into multiple areas, and acquires charging demand data, grid end electricity price and charging station location distribution; The algorithm and model construction module constructs an NSGAII algorithm based on reinforcement learning, and constructs a space-time pricing optimization model based on the algorithm; The pricing optimization module inputs the acquired charging demand data, grid end electricity price and charging station location distribution into the space-time pricing optimization model, and considers the time and space factors in the optimization process, so as to take the charging station operation efficiency and charging load balancing as the optimization target, and finally outputs the optimal pricing of each area at each time.

[0020] Compared with the prior art, the present application has the following beneficial effects: ​​​1、In the present application, by comprehensively considering time, space factors and user behavior, the charging station operation efficiency and charging load balance are taken as the optimization target, the pricing strategy of each time and each region considering the benefits and load distribution can be output, the service fees of different regions and different time periods with different charging conveniences are differentiated, the users are guided to shunt to idle areas and off-peak periods, the local area transformer is prevented from long-term high load or idling, the healthy and stable operation of the charging equipment is ensured, the overall resource utilization efficiency of the charging station is improved, and the user experience is improved. In the present application, the NSGAII algorithm based on reinforcement learning is constructed, and the space-time pricing optimization model is constructed based on the algorithm, so that the model can adaptively adjust the combination of crossover and mutation operators, efficiently solve the multi-objective optimization problem, and the output Pareto optimal solution set converges faster and better, providing a scientific and accurate space-time pricing scheme for large charging stations, further strengthening the guiding effect on user behavior and the balancing ability of equipment load.

[0021] 2、In the present application, reinforcement learning and NSGAII algorithm are combined, state quantization, action adaptive selection and population iterative optimization are used, so that the algorithm can dynamically adjust the direction according to the evolution state, balance exploration and utilization, speed up the convergence speed and avoid local optimum, and improve the optimization quality of the strategy.

[0022] 3、In the present application, the population State is defined as a four-tuple containing the population and the statistical characteristics of the Pareto front target value, which can comprehensively quantify the population evolution state, provide accurate input, and improve the guidance accuracy of the selection of crossover and mutation operators. By defining Reward through state comparison and updating Q value based on Bellman equation, the algorithm can effectively learn the long-term value of state-action, accurately feedback the influence of action on population optimization, and enhance the guiding effectiveness of reinforcement learning on the evolution process.

[0023] 4、In the present application, based on the adaptive learning rate of state-action pair access frequency and timing difference error, the update amplitude can be dynamically balanced, the learning rate can be reduced to reduce fluctuations for high-frequency access, and the learning rate can be increased to speed up convergence for high-error samples, thereby improving the stability and convergence efficiency of the algorithm.

[0024] 5、In the present application, the charging station operation efficiency is quantified by parameters such as the number of vehicles, power and time length, which can accurately reflect the overall operating income in the decision period and provide a scientific and quantifiable evaluation basis for the optimization target. In the present application, the load balance index is defined based on the deviation of regional load and average load, which can directly quantify the load distribution difference of each region, guide the pricing strategy to optimize in the direction of reducing load imbalance, and improve the stability of the utilization of charging station equipment. BRIEF DESCRIPTION OF DRAWINGS

[0025] Figure 1 It is a step schematic diagram of a large charging station space-time pricing method considering user behavior in the present application. Figure 2 A charging area division schematic diagram in the embodiment; Figure 3 A model input and output schematic diagram in the embodiment; Figure 4 A flowchart of an algorithm in the embodiment; Figure 5 An encoding structure diagram in the embodiment; Figure 6 A Q value representation intention in the embodiment; Figure 7 An HV calculation schematic diagram in the embodiment; Figure 8 A whole flowchart of an example test in the embodiment; Figure 9 A Pareto frontier dominance example diagram of an NSGAII algorithm based on reinforcement learning in the embodiment; Figure 10 A Pareto frontier equivalence example diagram of two algorithms in the embodiment. DETAILED DESCRIPTION

[0026] The technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the drawings in the embodiments of the present application. Obviously, the described embodiments are part of the embodiments of the present application, rather than all the embodiments of the present application. Based on the embodiments in the present application, all other embodiments obtained by a person of ordinary skill in the art without creative labor should fall within the protection scope of the present application.

[0027] In the scenario of a large charging station, the unified pricing method can cause the transformer of a charging area with high charging convenience to be in a high-load state for a long time, and the transformer of a charging area with low charging convenience to be in an idle state for a long time, which is not conducive to the long-term healthy and stable operation of the charging equipment.

[0028] To solve the above problems, in the present solution, a large charging station space-time pricing problem considering user behavior is researched, a large charging station space-time pricing optimization model considering user behavior is established by taking the charging service fee of different areas in different time periods as a decision variable, an NSGAII solving algorithm based on reinforcement learning is proposed, and the effectiveness of the model and the algorithm is proved through numerical experiments.

[0029] Embodiment 1 In the present embodiment, a large charging station space-time pricing method considering user behavior is adopted, and the method steps are as shown in Figure 1 The specific steps include: S1, divide the charging station into multiple areas, collect charging demand data, power grid end price and charging station location distribution; S2, construct an NSGAII algorithm based on reinforcement learning, and construct a spatiotemporal pricing optimization model based on the algorithm; S3, input the collected charging demand data, power grid end price and charging station location distribution into the spatiotemporal pricing optimization model, consider the factors of time and space in the optimization process, take the charging station operation efficiency and charging load balance as the optimization objectives, and finally output the optimal pricing of each region at each time.

[0030] The execution process of the NSGAII algorithm (Non-dominated Sorting Genetic Algorithm II) based on reinforcement learning includes: S31, encode the required decision variable, i.e. the charging service fee of the jth region at time t, into a matrix of T rows and J columns to form a chromosome matrix; each element in the chromosome is the pricing of the charging service fee of each sub-region at time t; S32, take the chromosome matrix as an individual, randomly generate an initial population of size N, and initialize the Q-value table to all 0s; S33, calculate the objective function value of each individual in the population, define and calculate the population State, and the objective function value includes the charging station operation efficiency and the charging load balance; S34, select Action using the ε-greedy strategy, and the action is defined as the selection probability of the combination of crossover and mutation operators; S35, execute the selected Action, take the current latest generation population as the parent population, and perform crossover and mutation on the parent population individuals to generate a child population with the same size as the parent population; S36, calculate the objective function value and population State of the child population, compare the population States of the parent and child populations, and update the Q-table based on the comparison result; S37, merge the parent and child populations, perform fast non-dominated sorting and crowding distance calculation on the merged population, select the optimal N individuals according to the elite strategy, and form a new generation population; S38, calculate the hyper-volume and distance indicators of the current latest generation population, if both the hyper-volume and distance indicators converge or reach the preset maximum number of iterations, output the Pareto optimal solution set in the current latest generation population as the optimal pricing of each region at each time; otherwise, return to step S33.

[0031] The present scheme aims to set different charging service fees for different regions of large charging stations at different times to improve the operation income of the charging station and balance the charging load of different regions at different times. The problem is defined as follows: Charging area division: The charging bays in a large charging station are divided into areas according to the distance from the elevator lobby or entrance, and the set is used to represent it, where the smaller the number, the higher the convenience, and the higher the corresponding charging service fee. Figure 2 The charging area division of a large super-charging station is shown in the figure. The red area can be regarded as a convenient area, and the blue area can be regarded as an economic area.

[0032] Time set: The decision cycle is one day, and the unit time is divided into T time points, .

[0033] Driver type: Different types of drivers have different sensitivities to charging prices, charging time periods, and charging area convenience, which can be represented by the set . Class driver selection to charge at time point The probability of charging can be represented as The above formula is a multinomial Logit model (MNL model), which is commonly used to analyze individual selection behavior among multiple mutually exclusive options. Among them, represents the distance from charging area j to the nearest elevator lobby, is the charging service fee of charging area at time point is the electricity price at time point and are the preference parameters of class charging users for the distance from charging area j to the nearest elevator lobby and the charging price.

[0034] Charging station operation efficiency calculation: The charging operation efficiency of all charging areas within the decision time cycle, the calculation formula is as follows: Among them, represents the actual number of vehicles that accept charging services in area at time point is the average charging power, is the unit time, is the charging operation efficiency of charging area at time point

[0035] Charging load balancing measure: The sum of the deviation of each charging area load from the average load within the decision cycle, the calculation formula is as follows: wherein, is the average load of each charging spot in the charging area at time is the average load of each charging spot in the charging station at time and are respectively the positive deviation value and the negative deviation value of the average load of each charging spot in the charging area and the average load of each charging spot in the charging station.

[0036] In this scheme, aiming at the above problem background, a NSGAII algorithm based on reinforcement learning is proposed, which can output the charging service fee pricing strategy of each charging station in each region at each time, the corresponding charging station operation efficiency and the corresponding charging load balance index under the conditions of given charging demand data, power grid end price, charging station location distribution data, price limit, and price and convenience preference coefficient of various drivers, with charging service fee in different regions at different times as decision variables, and charging station operation efficiency and charging load balance as optimization objectives, as shown in the input and output of Figure 3 .

[0037] In this scheme, the model-free Q-learning algorithm in reinforcement learning is used in the NSGAII algorithm to guide the selection of the crossover and mutation operators in the iteration of the NSGA-II algorithm, so that the NSGA-II algorithm can adaptively adjust the evolution direction according to the evolution state of the current population, better balance exploration and utilization, speed up the convergence speed, and avoid falling into local optimal solution, and the algorithm flow is as shown in Figure 4 .

[0038] The algorithm execution steps are as follows: (1) Chromosome coding: the decision variable in the space-time pricing optimization model is the charging service fee of j region at t time Therefore, the chromosome is coded as a T-row J-column matrix, wherein the element of the tth row and jth column represents the charging service fee of the jth region at the tth time; the chromosome is the service fee pricing scheme of the total charging region with T rows and J columns, and each element in the chromosome is the pricing of the charging service fee of each sub-region. The coding structure diagram is as shown in Figure 5 At the same time, it is necessary to ensure that the charging service fee meets the government price limit [0.5, 1.3] and the charging region with high convenience degree is not lower than the charging region with low convenience degree. ​​

[0039] (2) Generating initial population and initializing Q-table: an initial population with 100 chromosomes is randomly generated, and the elements in the Q-table are initialized to 0 or a small number. The Q-table is one of the important elements in the Q-learning algorithm; the Q-table is specifically as shown in Figure 6 , which records the corresponding (State, Action) corresponding Q-value value, the greater the Q-value value, the greater the Reward value that can be obtained by selecting the Action under the State value, which is more conducive to the update of the population.

[0040] (3) Calculate individual objective function value and current population State value: calculate the charging station operation efficiency and load balancing of each individual in the population, and define the population State value as a four-tuple of (the normalized average of the first objective value of the population individual, the normalized average of the second objective value of the population individual, the normalized average of the first objective value of the population Pareto individual, and the normalized average of the second objective value of the population Pareto individual). The State value is used to identify the evolution state of the current population, and the search process of the solution. After normalization, each element in the state tuple is in the interval of 0-1. Since the Q-table table needs to be updated, the state must be finite. Therefore, the elements in each position of the state are equally divided into buckets, and the -interval is divided into several intervals, such as 0-0.2 represented by index 1, 0.2-0.4 represented by 2, and so on. Mapping facilitates the Q-table to more accurately guide the crossover and mutation process of NSGA-II.

[0041] (4) Select Action (action) using -greedy strategy: Action is the probability of selecting the crossover and mutation operator. The crossover and mutation operator process is used to improve the solution. Since there are multiple crossover and mutation operators, the action here is the combination of the selection probability of these operators. In this embodiment, -greedy strategy value is set to 0.9, indicating that there is a 90% probability of selecting the Action with the maximum Q-value value under the current population State, and a 10% probability of randomly selecting the Action. Among them, Action is defined as the selection probability of the combination of the population crossover and mutation operator. According to the difference between the exploration ability and the utilization ability of the operator, the corresponding combination is designed, so that when the evolution is in the state of local optimal solution, the operator combination with stronger exploration ability is selected, and when the evolution is in the initial state, the operator combination with stronger utilization ability is selected.

[0042] wherein the exploration ability refers to the ability of the operator combination to make a wide range of strong disturbance changes to the existing solution (chromosome, i.e. the pricing strategy matrix), so as to jump out of the small range of solution space of the current local optimal solution, explore a wider unknown solution space, and find a potentially better solution. The exploitation ability refers to the ability of the operator combination to make small range and fine changes to the existing solution, so as to fine-tune around the current discovered better solution, improve the quality of the fine-tuned solution, and fully tap the potential of the known high-quality solution space.

[0043] (5) Perform Action (crossover mutation operation) to generate a new generation of sub-population: perform The crossover mutation operator of the greedy strategy selection generates a new generation of sub-population with a size of pop_size. The sub-population is a set of new solutions obtained after performing the crossover mutation operation on the current population.

[0044] (6) Calculate the objective function value of the new generation of sub-population individuals and the State value of the sub-population, calculate the Reward (feedback) and update the Q-table: the definition of Reward should be able to reflect the changes after the population performs the corresponding Action. In the Q-learning algorithm, Reward is defined as 1 if each element in the State four-tuple is improved, 0 if there is no change, and -1 otherwise. The update of Q-value value follows the time sequence difference implementation of Bellman's optimal equation, as shown in the following formula.

[0045] To ensure the efficiency and stability of reinforcement learning, an adaptive learning rate adjustment strategy based on state-action pair access frequency and time sequence difference error is designed. This method considers the trade-off between exploration and exploitation and error sensitivity. The calculation formula of the learning rate is as shown in the following formula. This method has good dynamic adaptability. It automatically reduces the learning rate for high-frequency access (state, action) to reduce variance, and increases the learning rate for high-error samples to speed up convergence.

[0046] wherein is the time sequence difference error, take the average of the time sequence difference error of the last 100 generations. is the frequency decay coefficient, which is a hyperparameter and needs to be set by oneself. At the same time, in order to prevent the over-decay or oscillation of the learning rate, the learning rate is set between

[0047] ​(7) Select the new generation population according to the elitist strategy: combine the parent population and the offspring population, and perform fast non-dominated sorting and crowded distance calculation to select the better pop_size individuals to form the new generation population.

[0048] Fast non-dominated sorting is an algorithm for hierarchical sorting of the combined population, i.e. all candidate solutions. The core is to divide the levels according to the dominance relationship of the solutions, and to ensure that better solutions are retained first. Its function is to quickly distinguish the quality level of the solution, provide priority basis for selection, and preferentially retain solutions with high levels.

[0049] Dominance relationship: for two pricing strategies (solutions) A and B, if the running efficiency of A ≥ B and the load balancing ≥ B (at least one target is strictly better), then A dominates B.

[0050] Sorting process: Layer 1 (best layer): all solutions that are not dominated by any other solution, i.e. Pareto optimal solutions; Layer 2: all solutions that are only dominated by layer 1 solutions and are not dominated by other layer solutions; In this way, the higher the layer, the worse the quality of the solution.

[0051] Crowded distance is an index that measures the sparsity of the distribution of solutions within the same layer. It is used for two-dimensional target space to ensure the diversity of the Pareto optimal solution set and avoid high-quality solutions in a certain area.

[0052] Calculation logic: for a solution X in the same layer, the corresponding target value coordinates are (x1, x2): Find the nearest solution with a target value less than X and the nearest solution with a target value greater than X in the layer; Crowded distance = Euclidean distance of the two nearest solutions in two-dimensional target space, or simply the square root of the difference in target value; The greater the distance, the sparser the solutions around X, and retaining it can improve the diversity of the solution set.

[0053] (8) Calculate the evaluation index HV and SP of the new generation population: Hypervolume (HV) to evaluate the quality of the Pareto solution set, which refers to the volume enclosed by the individuals in the solution set and the reference point in space, where the reference point must be dominated by all points on the Pareto frontier. When comparing HV between algorithms, it is necessary to select the same reference point to calculate HV, which makes sense. The greater the value of hypervolume, the better the distribution and convergence of the Pareto solution set. The calculation of hypervolume is the volume of the shaded part in Figure 7 The calculation formula is shown in the following formula.

[0054] The spacing (SP) index is often used to measure the uniformity of the distribution of the Pareto front solution set. The smaller the value, the more uniform the Pareto front. The formula for calculating SP is as follows.

[0055] wherein The calculation is as follows: In this embodiment, the charging order data of a certain large public charging station is used as the input of the algorithm, and two groups of operators with small and large scales are designed according to the size of the charging station and the size of the charging demand to test and verify the NSGAII algorithm based on reinforcement learning proposed in this project. The overall flowchart of the algorithm test is shown in Figure 8 , and the main steps include data preparation, algorithm running, and algorithm comparison.

[0056] Table 1 Comparison results of algorithms The results of the basic NSGA-II algorithm and the NSGA-II algorithm based on reinforcement learning (NSGAII-RL) in this scheme in small-scale and large-scale examples are shown in Table 1. The results with an asterisk are the better ones under the same example.

[0057] From the table, it can be seen that in the commonly used evaluation indexes HV and SP of multi-objective optimization algorithms, the NSGA-II algorithm based on reinforcement learning in this scheme is better than the basic NSGA-II algorithm whether in small-scale examples or in large-scale examples, which verifies the effectiveness of the NSGA-II algorithm based on reinforcement learning.

[0058] In the comparison of the Pareto front, the small-scale and large-scale examples were independently run for 10 times, and 20 groups of Pareto front comparison graphs were obtained. Since the heuristic algorithm itself has a certain randomness, the obtained Pareto front comparison graphs are divided into three types, namely, the NSGAII algorithm based on reinforcement learning is superior, the two algorithms are equivalent, and the standard NSGAII algorithm is superior. Different types of Pareto front comparison examples are shown in Figure 9 and Figure 10 (due to the absence of obvious standard NSGAII algorithm superior situation in the example, therefore, the example is not shown), Figure 9 and Figure 10In the middle, the blue dot is the load balancing of the standard NSGAII algorithm; the red dot is the load balancing of the NSGA-II algorithm based on reinforcement learning. The results show that in the small-scale example, the proportion of the NSGAII algorithm based on reinforcement learning is 70%, and the proportion of the two algorithms is 30%. In the large-scale example, the proportion of the NSGAII algorithm based on reinforcement learning is 60%, and the proportion of the two algorithms is 40%. Both groups of examples do not appear to be significantly superior to the standard NSGAII, indicating that the NSGAII algorithm based on reinforcement learning proposed in this chapter is better than the standard NSGAII.

[0059] The scheme has the following characteristics: (1) Spatiotemporal pricing: In this model, not only is time-based pricing performed, but also the factors of time and space are considered, which is a differentiated pricing strategy for large charging stations in terms of time and region; (2) Consider user behavior: The multi-logit selection model is used to describe the selection probability of the user's charging time and charging area; (3) Good effect of solving algorithm: The NSGAII solving algorithm based on reinforcement learning is proposed, and the solving result is better than the basic NSGAII algorithm.

[0060] Example 2 In this embodiment, a large charging station spatiotemporal pricing system considering user behavior is adopted, which includes a data acquisition module, an algorithm and model construction module, and a pricing optimization module; The data acquisition module divides the charging station into multiple regions, and acquires charging demand data, grid-side electricity price and charging station location distribution; The algorithm and model construction module constructs the NSGAII algorithm based on reinforcement learning, and constructs the spatiotemporal pricing optimization model based on the algorithm; The pricing optimization module inputs the acquired charging demand data, grid-side electricity price and charging station location distribution into the spatiotemporal pricing optimization model, considers the factors of time and space in the optimization process, takes the charging station operation efficiency and charging load balancing as the optimization objectives, and finally outputs the optimal pricing of each region at each time; The specific implementation process of the system is the same as that of Example 1.

[0061] In the system, the data acquisition module divides the charging station area reasonably and accurately collects core data such as charging demand, power grid end price and district distribution, provides comprehensive and reliable basic input for subsequent pricing optimization, and ensures that the model can be calculated according to the actual scene characteristics. The algorithm and model construction module combines reinforcement learning and NSGAII algorithm to construct a time and space pricing optimization model, which not only takes advantage of NSGAII in multi-objective optimization, but also realizes the adaptive adjustment of operator selection with the help of reinforcement learning, and improves the processing capacity and optimization efficiency of the model for complex time and space factors. The pricing optimization module integrates the collected data and considers time and space factors in optimization, and outputs the optimal pricing with the goals of running efficiency and load balancing, which can balance the charging station revenue and resource utilization, and improve the overall operation scientificity.

[0062] The above merely illustrates the specific embodiments of the present application, but the protection scope of the present application is not limited thereto, and any person skilled in the art can easily think of various equivalent modifications or replacements within the technical range disclosed by the present application, which should be covered in the protection scope of the present application. Therefore, the protection scope of the present application should be subject to the protection scope of the claims.

Claims

1. A large charging station spatiotemporal pricing method considering user behavior, characterized in that, The method steps comprise: S1, dividing the charging station into multiple areas, collecting charging demand data, grid end price and charging station location distribution; S2, constructing an NSGAII algorithm based on reinforcement learning, and constructing a space-time pricing optimization model based on the algorithm; S3, inputting the collected charging demand data, grid end price and charging station location distribution into the space-time pricing optimization model, considering the factors of time and space in the optimization process, taking the charging station operation efficiency and charging load balance as the optimization objectives, and finally outputting the optimal pricing of each area at each time. 2.The large charging station spatiotemporal pricing method considering user behavior of claim 1, wherein, The execution process of the NSGAII algorithm based on reinforcement learning comprises: S31, encoding the required decision variable, i.e. the charging service fee of the jth area at the tth time, into a matrix of T rows and J columns to form a chromosome matrix; each element in the chromosome is the pricing of the charging service fee of each sub-area at the tth time; S32, taking the chromosome matrix as an individual, randomly generating an initial population with a size of N, and initializing the Q value table as an all-0 table; S33, calculating the objective value of each individual in the population, defining and calculating the population State, and the objective value comprises a first objective value and a second objective value; wherein the first objective value is the charging station operation efficiency, and the second objective value is the charging load balance index; S34, selecting Action by adopting an ε-greedy strategy, and the action is defined as the selection probability of the combination of the crossover and mutation operators; S35, executing the selected Action, taking the current latest generation population as the parent population, and performing crossover and mutation on the parent population individuals to generate a child population with the same size as the parent population; S36, calculating the objective value of the child population and the population State, comparing the population States of the parent and child populations, and updating the Q value table based on the comparison result; S37, merging the parent and child populations, performing fast non-dominated sorting and crowdedness distance calculation on the merged population, selecting the optimal N individuals according to the elite strategy to form a new generation population; S38, calculating the hyper-volume and spacing index of the current latest generation population, and if the hyper-volume and spacing index are converged or reach the preset maximum iteration number, outputting the Pareto optimal solution set in the current latest generation population as the optimal pricing of each area at each time; otherwise, returning to step S33.

3. The large charging station spatiotemporal pricing method considering user behavior according to claim 2, characterized in that, In S31, the pricing of the charging service fee of each sub-area at the tth time needs to meet the price limit [0.5, 1.3].

4. The large charging station spatiotemporal pricing method considering user behavior according to claim 2, characterized in that, The population State is a four-tuple, comprising: the normalized average value of the first objective value of the population individuals, the normalized average value of the second objective value of the population individuals, the normalized average value of the first objective value of the Pareto frontier individuals of the population, and the normalized average value of the second objective value of the Pareto frontier individuals of the population.

5. The large charging station spatiotemporal pricing method considering user behavior according to claim 3, characterized in that, In S36, the specific process of comparing the population States of the parent and child populations and updating the Q value table based on the comparison result is: comparing the population States of the parent and child populations, and comparing each dimension in the four-tuple; if there is improvement in a certain dimension, the Reward result of the dimension is assigned as 1; if there is no change in a certain dimension, the Reward result of the dimension is assigned as 0. Otherwise, the Reward result of the dimension is assigned a value of -1; the Reward value is composed of the Reward results of the four dimensions, and the interval range of the Reward value is [-4, 4]; The Q-value value is updated based on the Reward value, and the update of the Q-value value follows the time difference implementation of the Bellman optimal equation, and the specific formula is: where, is the time instant , the state , the action , the current Q-value, representing the expected long-term reward for the state-action pair; is the updated Q-value at time instant , the new expectation after learning; is the learning rate, controlling the magnitude of the Q-value update; is the Reward value, representing the feedback of the change in population state after performing action in state ; is the discount factor; s is the state; is the action.

6. The large charging station spatiotemporal pricing method considering user behavior according to claim 5, characterized in that, The learning rate is adaptively adjusted based on the state-action pair access frequency and the time difference error, that is, the learning rate satisfies the following formula: wherein, is the time step, , the state-action pair, is the adaptive learning rate, which is a dynamic adjustment of the learning rate . is the visit count of the state-action pair . is the time difference error, is the average of the time difference error of the last 100 generations, is the frequency decay coefficient, and is a preset hyperparameter.

7. The large charging station spatiotemporal pricing method considering user behavior according to claim 2, characterized in that, The charging station operation efficiency in the S33 is the total charging operation efficiency of all charging areas in the decision time period, and the specific formula is: wherein, is the operation efficiency of the charging station; denotes the time region the number of vehicles actually receiving charging service; is the average charging power; is the unit time length; is the charging operation efficiency of the charging region at the time; is the total number of times within the decision time period; is the total number of charging regions. 8.The large charging station spatiotemporal pricing method considering user behavior of claim 2, wherein, The charging load balancing index in the S33 is the sum of the load deviation of each charging area from the average load at each time in the decision period, and the specific formula is: wherein, is a charging load balancing index; is a time instant is a charging area is an average load per charging spot of the charging area is a time instant is an average per charging spot load of charging stations within the charging area and are respectively a positive deviation value and a negative deviation value of the average load per charging spot of the charging area is a charging area and the average per charging spot load of charging stations. 9.The large charging station spatiotemporal pricing method considering user behavior of claim 2, wherein, in the S34 of the -greedy policy The value is set to 0.9, i.e. there is a 90% probability of selecting the Action with the largest Q-value under the current population State, and a 10% probability of selecting an Action at random.

10. A large charging station spatiotemporal pricing system considering user behavior, characterized in that, The system applies the large charging station space-time pricing method considering user behavior according to any one of claims 1-9, and the system comprises a data acquisition module, an algorithm and model construction module, and a pricing optimization module; The data acquisition module divides the charging station into multiple areas, and acquires charging demand data, grid end electricity price and charging station location distribution; The algorithm and model construction module constructs an NSGAII algorithm based on reinforcement learning, and constructs a space-time pricing optimization model based on the algorithm; The pricing optimization module inputs the acquired charging demand data, grid end electricity price and charging station location distribution into the space-time pricing optimization model, considers the time and space factors in the optimization process, takes the charging station operation efficiency and the charging load balancing as the optimization objectives, and finally outputs the optimal pricing of each area at each time.

Images