Optical storage charging station dynamic pricing method and system based on deep reinforcement learning
Through the method based on deep reinforcement learning, the traffic network and optical storage charging station models are constructed, the non-complete information game state is analyzed, and the optimal dynamic pricing scheme for optical storage charging stations is obtained, which solves the problem that the impact of traffic flow in the existing technology is not considered. , achieving optimization of resource allocation and improvement of urban transportation.
Patent Information
- Application Number
- CN202411021920.1
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-07-29
- Publication Date
- 2024-11-22
AI Technical Summary
The existing pricing methods for photoelectric vehicle charging stations ignore the dynamic impact of traffic flow, resulting in the inability to effectively coordinate the power and transportation networks and the inability to optimize the charging market. The existing game methods are mostly suitable for linear market environments, ignoring traffic flow. Effect on charging load.
Using a method based on deep reinforcement learning, the pricing game environment of the charging station is defined by building a traffic network model and an optical storage charging station model, and using a multi-agent deep Q network and an average field multi-agent deep reinforcement learning method to analyze non-complete information game Status, obtain the optimal dynamic pricing plan for optical storage charging stations under the influence of traffic flow.
It has achieved reasonable allocation of resources, reduced charging demand during peak periods, balanced grid load, improved the operating efficiency and market competitiveness of charging stations, optimized urban transportation, and improved the accuracy and reliability of decision-making.
Smart Images

Figure CN119006024A_ABST
Abstract
Description
Technical Field
[0001] The present invention belongs to the field of information processing technology, and in particular relates to a method and system for dynamic pricing of a photovoltaic charging station based on deep reinforcement learning. Background Art
[0002] The statements in this section merely provide background information related to the present invention and do not necessarily constitute prior art.
[0003] In recent years, with the popularization of fast charging technology, the electric vehicle industry has developed rapidly, promoting the widespread deployment of electric vehicle charging stations in smart cities. In the context of vigorously developing new energy, electric vehicles (EVs) have made significant contributions to energy conservation and emission reduction. The large-scale popularization of photovoltaic and storage electric vehicle charging stations (CS) requires further coordination of power and transportation networks, and solving many challenges such as the rationality of geographical layout, the randomness of electric vehicle charging behavior, and the coordinated economy of green electricity. It is of great significance to optimize the charging market and improve the traffic environment of smart cities through the pricing mechanism of photovoltaic and storage charging station services.
[0004] Existing CS pricing methods ignore its dynamic pricing function and market benefits, and do not consider the impact of traffic flow on EV's choice of CS. Existing game methods mostly use game methods that are only applicable to linear market environments, and the constraints of traffic flow and its impact on charging load are usually simplified or ignored. These solution methods require global information or optimal actions, which are not applicable in practice. Summary of the invention
[0005] In order to solve the above problems, the present invention proposes a dynamic pricing method and system for photovoltaic charging stations based on deep reinforcement learning. The present invention uses a model with elastic transportation network demand, defines the electric vehicle charging cost optimization problem and the charging market revenue optimization problem, and obtains the optimal dynamic pricing scheme for photovoltaic charging stations under the influence of traffic flow by effectively analyzing and approximating the Nash equilibrium under the state of incomplete information game based on deep reinforcement learning method.
[0006] According to some embodiments, the present invention adopts the following technical solutions:
[0007] A method for dynamic pricing of a photovoltaic charging station based on deep reinforcement learning, comprising the following steps:
[0008] Obtain historical traffic flow data, build a traffic network model based on the traffic flow data, determine the pricing method when the traffic is balanced, and form a traffic flow model;
[0009] Obtain historical usage data of the photovoltaic energy storage charging station and build a photovoltaic energy storage charging station model;
[0010] Based on the traffic flow model and the solar-storage charging station model, determine the pricing game environment of the solar-storage charging station;
[0011] The pricing game environment of the photovoltaic charging station is used as the interactive environment adopted by deep reinforcement learning. Each photovoltaic charging station in the environment adjusts the pricing based on the deep reinforcement learning method, observes the environmental status and rewards, and solves the optimal dynamic pricing of the photovoltaic charging station in the current environment.
[0012] As an optional implementation, the process of constructing a traffic network model based on traffic data includes: constructing an urban traffic network model through a directed graph, and considering the impact of travel time, waiting time, charging time and charging cost on traffic flow and charging price during the construction.
[0013] As an optional implementation, the process of determining the pricing method when traffic is balanced includes: calculating the charging load of each traffic flow at the corresponding charging station, and then calculating the total charging load of the charging station, combining the time cost and the electricity cost, and determining the travel route of the charging car by minimizing the travel cost to determine the optimal pricing.
[0014] As an optional implementation, the photovoltaic storage charging station model includes determining the influencing factors affecting the charging of the photovoltaic storage charging station and determining the photovoltaic storage charging station revenue objective function;
[0015] The influencing factors include the power consumed by the photovoltaic power generation system, the total power actually obtained by electric vehicles, the power required for charging of new electric vehicles, and the interactive power with the distribution network;
[0016] The revenue objective function of the photovoltaic charging station is determined by combining the influencing factors and the electric vehicle travel route in the pricing method when determining traffic balance, and the pricing is determined by solving the maximum revenue.
[0017] As an optional implementation method, based on the traffic flow model and the photovoltaic charging station model, the process of determining the pricing game environment of the photovoltaic charging station includes: expressing the pricing game between charging stations as:
[0018] The players are the set of all solar-storage electric vehicle charging stations, the action space is the available electricity selling price, the strategy is that each player selects an electricity selling price from the action space, and the reward is the profit obtained by each player through the selected price strategy; the players do not have complete information about the strategies of other players, but only know the possible action space of other players and the overall game mode;
[0019] Each charging station is trying to maximize its own profits, and different pricing will affect the travel cost of electric vehicles, which in turn affects the choice of electric vehicles for solar-storage electric vehicle charging stations;
[0020] For a game, the optimal strategy tuple determines that the player cannot obtain more rewards by deviating from the current strategy; and the equilibrium solution of the game is measured by the no-regret index.
[0021] Furthermore, the game is an incomplete information game between photovoltaic storage charging stations based on the influence of distribution network and traffic flow.
[0022] As an optional implementation method, the process of solving the optimal dynamic pricing of the photovoltaic storage charging station under the current environment includes: solving the best response value based on a multi-agent deep Q network;
[0023] Optimize the solution process based on neighborhood decomposition and mean field approximation;
[0024] Solving optimal pricing based on mean field multi-agent deep reinforcement learning method.
[0025] As a further step, the process of solving the optimal response value based on the multi-agent deep Q-network is included in the single-agent deep Q-network, and the optimal strategy is expressed by the greedy strategy.
[0026] As a further step, the process of solving the optimal pricing based on the mean field multi-agent deep reinforcement learning method includes: the value-state function of each player is equivalent to the utility function of the game, the neighborhood of the value-state function is decomposed, and based on each neighborhood, the average action and field action are iteratively calculated. The action with the highest action value is selected as the charging price. After all charging prices are determined, the reward is calculated, and each agent updates the single-agent deep Q network to maximize the benefit.
[0027] As a further step, the game equilibrium solution measured by the no-regret index is used in the process of solving the optimal pricing based on the mean field multi-agent deep reinforcement learning method.
[0028] A dynamic pricing system for photovoltaic charging stations based on deep reinforcement learning, comprising:
[0029] A data acquisition module is configured to acquire historical traffic flow data and historical usage data of the photovoltaic charging station;
[0030] The model building module is configured to build a traffic network model based on traffic data and determine a pricing method when traffic is balanced to form a traffic flow model; and to build a solar-storage charging station model based on historical usage data of the solar-storage charging station;
[0031] An environment building module is configured to determine a pricing game environment for a photovoltaic charging station based on a traffic flow model and a photovoltaic charging station model;
[0032] The pricing solution module is configured to use the photovoltaic charging station pricing game environment as an interactive environment adopted by deep reinforcement learning. Each photovoltaic charging station in the environment adjusts the pricing based on the deep reinforcement learning method, observes the environmental status and rewards, and solves the optimal dynamic pricing of the photovoltaic charging station in the current environment.
[0033] Compared with the prior art, the present invention has the following beneficial effects:
[0034] (1) The present invention can optimize resource allocation: through a dynamic pricing strategy, power resources can be allocated more reasonably, charging demand during peak hours can be reduced, and grid load can be balanced.
[0035] (2) The present invention can improve economic benefits: the optimized pricing strategy can improve the operating efficiency of charging stations, increase revenue, and attract more electric vehicle users to use charging services.
[0036] (3) The present invention can optimize urban traffic: Charging station operators can use the pricing system of the present invention to more flexibly respond to changes in traffic flow, thereby improving market competitiveness and optimizing the traffic load problem caused by new energy vehicles.
[0037] (4) The present invention can improve the accuracy of decision-making: The present invention conducts gaming based on a large amount of historical data, thereby improving the accuracy and reliability of decision-making.
[0038] In order to make the above-mentioned objects, features and advantages of the present invention more obvious and easy to understand, preferred embodiments are given below and described in detail with reference to the accompanying drawings. BRIEF DESCRIPTION OF THE DRAWINGS
[0039] The accompanying drawings in the specification, which constitute a part of the present invention, are used to provide a further understanding of the present invention. The exemplary embodiments of the present invention and their descriptions are used to explain the present invention and do not constitute improper limitations on the present invention.
[0040] Figure 1 It is a schematic diagram of an interactive environment diagram of a photovoltaic charging station pricing method based on deep reinforcement learning in an embodiment of the present invention;
[0041] Figure 2 It is a schematic diagram of the deep reinforcement learning process of the photovoltaic charging station pricing method based on deep reinforcement learning in an embodiment of the present invention. DETAILED DESCRIPTION
[0042] The present invention will be further described below in conjunction with the accompanying drawings and embodiments.
[0043] It should be noted that the following detailed descriptions are all illustrative and intended to provide further explanation of the present invention. Unless otherwise specified, all technical and scientific terms used herein have the same meanings as those commonly understood by those skilled in the art to which the present invention belongs.
[0044] It should be noted that the terms used herein are only for describing specific embodiments and are not intended to limit exemplary embodiments according to the present invention. As used herein, unless the context clearly indicates otherwise, the singular form is also intended to include the plural form. In addition, it should be understood that when the terms "comprising" and / or "including" are used in this specification, it indicates the presence of features, steps, operations, devices, components and / or combinations thereof.
[0045] In the absence of conflict, the embodiments in this application and the features in the embodiments may be combined with each other.
[0046] Embodiment 1
[0047] In one embodiment of the present disclosure, a pricing method for a photovoltaic charging station based on deep reinforcement learning is provided. Figure 2 As shown, the following steps are included:
[0048] Step S1: Obtain historical traffic flow data and build a traffic flow model;
[0049] Step S2: Obtain historical usage data of the photovoltaic storage charging station and build a photovoltaic storage charging station model;
[0050] Step S3: Determine the pricing game environment of the photovoltaic charging station based on the traffic flow model and the photovoltaic charging station model;
[0051] Step S4: Solve the optimal dynamic pricing strategy for each solar-storage charging station based on the deep reinforcement learning method.
[0052] The overall implementation method is:
[0053] Step 1: Obtain historical traffic flow data and build a traffic flow model. The specific steps include:
[0054] Step 101: Model the urban transportation network based on historical data:
[0055] The urban transportation network is modeled through a directed graph G = [N, A], such as Figure 1 As shown in the figure, N and A represent traffic nodes and paths respectively. Each EV starting from the origin O and arriving at the destination D in the graph G is called the EV's travel OD pair, and the OD pair is determined by the historical traffic flow. The feasible path set from O to D is recorded as Assume that each EV will choose a route with the minimum travel time, waiting time, charging time and charging cost. The travel cost of any route in is the same, and the traffic flow reaches equilibrium. The mathematical expression of the above model is as follows:
[0056]
[0057] Among them, f p,ω is the traffic flow on path p of a certain OD pair ω, π is the set of CS charging prices, and π is composed of the charging cost of each path and charging time Determine, ω is the number of EVs per unit time, x a is the traffic flow on arc a∈A, x k is the flow rate at charging station k∈K, d ω For traffic demand, t a is the travel time, t k is the waiting time, g w is the traffic demand function.
[0058] Traffic flow x a and charging station flow x k The calculation method is as follows:
[0059]
[0060]
[0061] in, The value is {0, 1}, judging f p,ω Whether to pass route a and charging station k.
[0062] Traffic demand ω The calculation method is as follows:
[0063]
[0064] Among them, c ω is the minimum travel cost on OD pair ω.
[0065] Travel time t a and waiting time t k The calculation method is as follows:
[0066]
[0067]
[0068] in, and is the initial travel time and waiting time, used to indicate the congestion level, affected by the traffic flow x a and charging station flow x k The impact of a and c k For traffic capacity and charging station capacity.
[0069] Step 102: Define the pricing solution for traffic flow equilibrium
[0070] Total charging load at charging station k The calculation is as follows:
[0071]
[0072] in, is the traffic flow f p,ω Charging load at charging station k. It can be calculated by iteration, specifically:
[0073] Set the initial traffic environment For each OD path to ω, if a route p with a lower cost can be found, then add that route to and update and Until no lower-cost route can be found. For each ω, The solution is as follows:
[0074]
[0075] The objective function is used to minimize the total travel cost on ω, where and The value is {0, 1}, which is used to determine whether arc a and charging station k are traversed. After that, time cost and electricity costs The calculation method is as follows:
[0076]
[0077]
[0078] The pricing H(π) when the flow is balanced can be solved by the above method.
[0079] Step 2: Obtain the historical usage data of the solar-storage charging station and build a solar-storage charging station model. The specific steps are as follows:
[0080] Step 201: Determine factors that affect the charging power of the photovoltaic charging station:
[0081] For the photovoltaic power generation system, the actual power consumption of photovoltaic power generation in the t period The calculation method is as follows:
[0082]
[0083] in, Indicates the maximum input power allowed by the power grid, represents the energy storage charging and discharging power in the tth period, represents the photovoltaic power generation in the tth period, It represents the total power demand of electric vehicles in the tth period.
[0084] The charging station system takes into account the power supply capacity limitations of photovoltaic, distribution network and energy storage. The total power actually obtained by electric vehicles in time period t is calculated as follows:
[0085]
[0086]
[0087]
[0088] in, represents the additional charging power demand of electric vehicles in period t, is the electric vehicle demand power that has not been met at the end of period t-1.
[0089] The photovoltaic charging station is connected to the distribution network as a node. The interaction power between the photovoltaic charging station and the distribution network in period t is The calculation is as follows:
[0090]
[0091] Step 202: Determine the objective function of the photovoltaic charging station revenue:
[0092] The profit U(t) of a single solar-storage charging station in period t is the operating income minus the electricity cost and the attenuation loss of the energy storage equipment, and is calculated as follows:
[0093]
[0094] in, represents the unit price of charging in the tth period, represents the unit price of electricity purchased from the power grid in period t, m ESS It represents the decay loss cost generated by the energy storage unit when exchanging one unit of electricity. This cost is amortized over its entire discharge cycle based on the purchase cost of the energy storage device. The calculation method is as follows:
[0095]
[0096] q ESS =2n cyc d dod v c
[0097]
[0098]
[0099] Among them, C ESS is the energy storage purchase cost, q ESS is the total discharge amount of energy storage during the entire use cycle n cyc is the number of discharge cycles of energy storage in the entire use period d dod is the discharge depth of the energy storage, α1 and α2 represent the energy transmission efficiency during the input and output of the distribution network, and β1 and β2 represent the energy conversion efficiency during charging and discharging of the energy storage, respectively.
[0100] Based on the above traffic flow model and solar-storage charging station model, the maximum profit calculation method of charging station k in period t can be determined as follows:
[0101]
[0102]
[0103] Among them, π k , π -k represents the pricing of charging station k and charging stations other than charging station k, for The solution, U k It can be extended to any time period U k,t The solution.
[0104] Step 3: Determine the pricing game environment of the solar-storage charging station based on the traffic flow model and the solar-storage charging station model. The specific steps are as follows:
[0105] The pricing game between charging stations can be expressed as:
[0106] The players are the set of all CSs, the action space is the electricity selling price that each CS can choose (these prices are discrete and within a given range), the strategy is that each player chooses an electricity selling price from the action space, and the reward is the profit obtained by each player through the selected price strategy. The players do not have complete information about the strategies of other players, but only know the possible action space of other players (that is, the possible price range) and the overall game mode. Each charging station is trying to maximize its own profit, and different pricing will affect the travel cost of EV, which in turn affects the choice of CS by EV.
[0107] For Gaming Best strategy tuple It can be determined that the player cannot obtain more rewards by deviating from the current strategy, that is,
[0108] The equilibrium solution (NE) of the above game can be measured by the no-regret index (NI), which is calculated as follows:
[0109]
[0110] When NE is found, NI is 1, and we can check how close the result is to NE by calculating NI.
[0111] Step 4: Solve the optimal dynamic pricing strategy for each solar-storage charging station based on the deep reinforcement learning method. The specific steps are as follows:
[0112] Step 401: Solve the optimal response value based on the multi-agent deep Q network:
[0113] In a single-agent deep Q network (DQN), the optimal policy is described by the greedy policy, which is calculated as follows:
[0114]
[0115] Where s represents the state, a represents the selected action, and Q(s, a) is the profit generated by selecting action a in state s. The calculation method for the above game Q(s, a) is as follows:
[0116]
[0117] Step 402: Optimization solution process based on neighborhood decomposition and mean field approximation:
[0118] The value state function Q(s, a) of player k is equivalent to the game The utility function, Q k (s, a) can be decomposed into the neighborhood:
[0119]
[0120] Among them, k is the neighborhood set of k, N k For the set k The size of π n is the strategy of member n of the domain set. Based on the neighborhood of agent k, the average action and domain action π n The calculation method is as follows:
[0121]
[0122]
[0123] Among them, δπ n,k represents the difference between the average action and the action of agent k. Through Taylor expansion, Q k (π -k , π k ) is simplified to:
[0124]
[0125] Mean Field Action It is approximated by the initialization process and input into each agent’s DQN. The initialization process is as follows:
[0126]
[0127] in, represents the initial mean field action and is updated in each iteration as Has the highest action value Action of π k is selected as the charging price. Among all charging prices π=[π1,...,π K ] is determined, calculate the reward r = [r1, ..., r K ], and have each agent update the DQN to maximize the reward.
[0128] Step 403: Solve the optimal pricing based on the mean field multi-agent deep reinforcement learning method:
[0129] Randomly initialize the parameters θ of agent k k , calculate the action of agent k in each training round Determine π and calculate r, and store the tuple in the buffer At the same time, take out the sample in the buffer Calculate the loss function, the loss function L(θ k ) is calculated as follows:
[0130]
[0131] According to L(θ k ) Update θ k , the action space is explored through the Boltzmann strategy during the update process, which is calculated as follows:
[0132]
[0133] in, represents agent k choosing action a i The parameter β indicates the degree of exploration during training. The maximum action. Through the above method, the optimal dynamic pricing when NI is maximum can be obtained.
[0134] Embodiment 2
[0135] In one embodiment of the present disclosure, a pricing system for a photovoltaic charging station based on deep reinforcement learning is provided, including a data acquisition module, an environment construction module, and a pricing solution module:
[0136] A data acquisition module is configured to: acquire historical traffic flow data and historical usage data of photovoltaic charging stations required for building a model;
[0137] The environment building module is configured to: build a pricing game environment for photovoltaic and energy storage charging stations required for deep reinforcement learning based on historical data;
[0138] The pricing solution module is configured to solve the optimal dynamic pricing strategy for each photovoltaic charging station based on a deep reinforcement learning method; wherein the interactive environment adopted by the deep reinforcement learning is the photovoltaic charging station pricing game environment mentioned above, and each photovoltaic charging station in the environment observes the environmental status and rewards by adjusting the pricing, and the output is the optimal dynamic pricing of the photovoltaic charging station in the environment.
[0139] Those skilled in the art will appreciate that embodiments of the present invention may be provided as methods, systems, or computer program products. Therefore, the present invention may take the form of a complete hardware embodiment, a complete software embodiment, or an embodiment combining software and hardware. Moreover, the present invention may take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.
[0140] The present invention is described with reference to flowcharts and / or block diagrams of methods, devices (systems), and computer program products according to embodiments of the present invention. It should be understood that each process and / or block in the flowchart and / or block diagram, as well as the combination of processes and / or blocks in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to produce a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.
[0141] These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.
[0142] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operating steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions for implementing the process. Figure 1 A process or multiple processes and / or boxes Figure 1 The steps for the functions specified in one or more boxes.
[0143] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. For those skilled in the art, the present invention may have various modifications and variations. Any modification, equivalent replacement, improvement, etc. made by those skilled in the art within the spirit and principle of the present invention without creative labor shall be included in the protection scope of the present invention.
Claims
1. A dynamic pricing method for photovoltaic charging stations based on deep reinforcement learning, characterized in that: The following steps are involved: Obtain historical traffic flow data, build a traffic network model based on the traffic flow data, determine the pricing method when the traffic is balanced, and form a traffic flow model; Obtain historical usage data of the photovoltaic energy storage charging station and build a photovoltaic energy storage charging station model; Based on the traffic flow model and the solar-storage charging station model, determine the pricing game environment of the solar-storage charging station; The pricing game environment of the photovoltaic charging station is used as the interactive environment adopted by deep reinforcement learning. Each photovoltaic charging station in the environment adjusts the pricing based on the deep reinforcement learning method, observes the environmental status and rewards, and solves the optimal dynamic pricing of the photovoltaic charging station in the current environment.
2. A method for dynamic pricing of a photovoltaic charging station based on deep reinforcement learning as claimed in claim 1, characterized in that: Based on the traffic data, the process of constructing a traffic network model includes: constructing an urban traffic network model through a directed graph, and considering the impact of travel time, waiting time, charging time and charging cost on traffic flow and charging price during the construction.
3. A method for dynamic pricing of a photovoltaic charging station based on deep reinforcement learning as claimed in claim 1, characterized in that: The process of determining the pricing method when traffic is balanced includes: calculating the charging load of each traffic flow at the corresponding charging station, and then calculating the total charging load of the charging station, combining time cost and electricity cost, and determining the travel route of the charging car by minimizing the travel cost to determine the optimal pricing.
4. A method for dynamic pricing of a photovoltaic charging station based on deep reinforcement learning as claimed in claim 1, characterized in that: The photovoltaic charging station model includes determining the factors affecting the charging of the photovoltaic charging station and determining the photovoltaic charging station revenue objective function; The influencing factors include the power consumed by the photovoltaic power generation system, the total power actually obtained by electric vehicles, the power required for charging of new electric vehicles, and the interactive power with the distribution network; The revenue objective function of the photovoltaic charging station is determined by combining the influencing factors and the electric vehicle travel route in the pricing method when determining traffic balance, and the pricing is determined by solving the maximum revenue.
5. A method for dynamic pricing of a photovoltaic charging station based on deep reinforcement learning as claimed in claim 1, characterized in that: Based on the traffic flow model and the solar-storage charging station model, the process of determining the pricing game environment of the solar-storage charging station includes: expressing the pricing game between charging stations as: The players are the set of all solar-storage electric vehicle charging stations, the action space is the available electricity selling price, the strategy is that each player selects an electricity selling price from the action space, and the reward is the profit obtained by each player through the selected price strategy; the players do not have complete information about the strategies of other players, but only know the possible action space of other players and the overall game mode; Each charging station is trying to maximize its own profits, and different pricing will affect the travel cost of electric vehicles, which in turn affects the choice of electric vehicles for solar-storage electric vehicle charging stations; For a game, the optimal strategy tuple determines that the player cannot obtain more rewards by deviating from the current strategy; and the equilibrium solution of the game is measured by the no-regret index.
6. A method for dynamic pricing of a photovoltaic charging station based on deep reinforcement learning as claimed in claim 5, characterized in that: The game described is an incomplete information game between photovoltaic storage charging stations based on the influence of distribution network and traffic flow.
7. A method for dynamic pricing of a photovoltaic charging station based on deep reinforcement learning as claimed in claim 1, characterized in that: The process of solving the optimal dynamic pricing of the photovoltaic charging station under the current environment includes: solving the optimal response value based on the multi-agent deep Q network; Optimize the solution process based on neighborhood decomposition and mean field approximation; Solving optimal pricing based on mean field multi-agent deep reinforcement learning method.
8. A method for dynamic pricing of a photovoltaic charging station based on deep reinforcement learning as claimed in claim 7, characterized in that: The process of solving the best response value based on the multi-agent deep Q network is included in the single-agent deep Q network, and the optimal strategy is expressed by the greedy strategy.
9. A method for dynamic pricing of a photovoltaic charging station based on deep reinforcement learning as claimed in claim 7, characterized in that: The process of solving the optimal pricing based on the mean field multi-agent deep reinforcement learning method includes: the value state function of each player is equivalent to the utility function of the game, the neighborhood of the value state function is decomposed, and the average action and field action are iteratively calculated based on each neighborhood. The action with the highest action value is selected as the charging price. After all charging prices are determined, the reward is calculated, and each agent is asked to update the single-agent deep Q network to maximize the benefit; Or, the game equilibrium solution measured by the no-regret index in the process of solving the optimal pricing based on the mean field multi-agent deep reinforcement learning method.
10. A dynamic pricing system for photovoltaic charging stations based on deep reinforcement learning, characterized in that: include: A data acquisition module is configured to acquire historical traffic flow data and historical usage data of the photovoltaic charging station; The model building module is configured to build a traffic network model based on traffic data and determine a pricing method when traffic is balanced to form a traffic flow model; and to build a solar-storage charging station model based on historical usage data of the solar-storage charging station; An environment building module is configured to determine a pricing game environment for a photovoltaic charging station based on a traffic flow model and a photovoltaic charging station model; The pricing solution module is configured to use the photovoltaic charging station pricing game environment as an interactive environment adopted by deep reinforcement learning. Each photovoltaic charging station in the environment adjusts the pricing based on the deep reinforcement learning method, observes the environmental status and rewards, and solves the optimal dynamic pricing of the photovoltaic charging station in the current environment.
Citation Information
Cited By
Multi-type micro-grid system scheduling optimization method based on mean field hypergraph game, medium and equipment
CN120410777A
Urban parking-oriented Internet of Things mobile charging system and dynamic pricing method
CN121366451A
