Intelligent charging pile scheduling method and system based on dynamic adjustment of energy storage battery pack
By introducing dynamic adjustment and multi-objective optimization methods of energy storage battery packs in the charging pile system, combined with reinforcement learning of layered multi-agents, the problems of low utilization rate of energy storage battery packs and difficult to alleviate the power grid pressure in the existing technology are solved, and the problem of low energy storage battery pack utilization rate and difficult to alleviate the power grid pressure in the existing technology is realized. Grid load balancing and user waiting time optimization.
Patent Information
- Application Number
- CN202411448921.4
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-10-17
- Publication Date
- 2024-11-15
- Estimated Expiration
- Not applicable · inactive patent
AI Technical Summary
The existing charging pile scheduling method ignores the potential role of energy storage battery packs in power scheduling, lacks real-time monitoring and dynamic adjustment capabilities, resulting in low utilization rate of energy storage battery packs, which cannot effectively alleviate the pressure of the power grid, affecting the operating efficiency and stability of the power grid. sex.
Using the intelligent charging pile scheduling method based on dynamic adjustment of energy storage battery packs, prediction is carried out through graph attention and adaptive spatiotemporal pooling mechanism, combining multi-objective particle swarm algorithm and simulated annealing algorithm to build a multi-objective optimization model, and using stratification Multi-agent reinforcement learning and soft actor-critician algorithm perform distributed collaborative scheduling.
It improves the utilization rate of energy storage battery packs, alleviates grid pressure, improves the overall operating efficiency and stability of the electric vehicle charging system, ensures grid load balance and the life of energy storage batteries, and optimizes user waiting time.
Smart Images

Figure CN118966580A_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the technical field of power management, and in particular to an intelligent charging pile scheduling method and system based on dynamic adjustment of energy storage battery packs. Background Art
[0002] With the development of electric vehicles, charging piles, as charging infrastructure, have gradually become an important part of the smart grid. With the rapid growth of the number of electric vehicles, the sharp increase in charging demand has brought huge load pressure to the power grid. The concentrated use of charging piles may lead to instability in the power system and affect the safe operation of the entire power grid. In order to optimize the scheduling of charging piles, reasonably allocate charging resources, and reduce the burden on the power grid, energy storage battery packs are gradually introduced into the charging pile system. They can store electrical energy when the power grid load is low, and provide additional power support for charging piles when the charging demand is peak, thereby alleviating the pressure on the power grid and balancing supply and demand.
[0003] Most existing charging pile scheduling methods rely on a single power grid, ignoring the potential role of energy storage battery packs in power scheduling, and lack the ability to monitor and dynamically adjust the status of energy storage battery packs in real time. Even if energy storage battery packs are introduced into some systems, the scheduling strategy is often relatively static and cannot be dynamically adjusted according to the grid load, the remaining power of the energy storage battery pack, and the real-time demand of the charging pile. Not only may it lead to low utilization of the energy storage battery pack, but it may also fail to effectively relieve the pressure on the grid during peak charging periods, thereby reducing the operating efficiency and stability of the overall system.
[0004] Therefore, in response to the above problems, there is an urgent need for an intelligent charging pile scheduling method combined with dynamic adjustment of energy storage battery groups. By real-time monitoring of the status of the energy storage battery groups, combined with the charging demand of electric vehicles and the load of the power grid, the scheduling strategy of the charging piles is dynamically adjusted to achieve coordinated optimization scheduling of the energy storage battery groups and the charging piles, maximize the utilization rate of the energy storage battery groups, alleviate the pressure on the power grid, and improve the overall operating efficiency and stability of the electric vehicle charging system. The present invention can solve the problems in the prior art. Summary of the invention
[0005] The embodiments of the present invention provide a smart charging pile scheduling method and system based on dynamic adjustment of energy storage battery groups, which can solve the problems in the prior art.
[0006] According to a first aspect of the embodiments of the present invention, Provided is a smart charging pile scheduling method based on dynamic adjustment of energy storage battery packs, comprising: Collect multi-dimensional real-time data of the charging pile group, and obtain the status information of the energy storage battery group at the same time to obtain basic data. Combined with historical load curves and weather forecast information, the future is predicted through graph attention and adaptive spatiotemporal pooling mechanism to obtain prediction results. Based on basic data and prediction results, determine the objective function including maximum charging efficiency, most balanced grid load, longest cycle life of energy storage battery, and minimum user waiting time, build a multi-objective optimization model, adopt a hybrid optimization method based on multi-objective particle swarm algorithm and simulated annealing algorithm, introduce adaptive inertia weight and dynamic constraint processing mechanism, and generate the optimal charging pile scheduling plan through iterative optimization; Based on the optimal charging pile scheduling scheme, the charging pile group is divided into multiple sub-networks through a distributed collaborative scheduling method based on graph attention network and hierarchical multi-agent reinforcement learning. The charging piles and energy storage battery packs in each sub-network are used as the underlying agents, and the controller of the sub-network is used as the high-level agent. The improved soft actor-critic algorithm is used to train the agents, and the scheduling tasks are dynamically executed through the trained agents.
[0007] In an optional embodiment, Collect multi-dimensional real-time data of the charging pile group, obtain the status information of the energy storage battery group, obtain basic data, combine historical load curves and weather forecast information, and use graph attention and adaptive spatiotemporal pooling mechanisms to predict the future. The prediction results include: Each charging pile node is represented as a capsule. Based on the activation degree of the charging pile node, the vector length of the vector inside the capsule is determined. Based on the feature change trend of the charging pile node, the vector direction of the vector inside the capsule is determined. Based on the vector length and vector direction, the correlation between capsules is calculated through the attention mechanism, and the neighborhood weight of the capsule is dynamically updated. The dynamic routing algorithm is used to realize information transmission and aggregation between capsules. Capsules with neighborhood weights greater than the preset threshold are formed into high-level capsules, and the hierarchical feature representation of the charging pile group is extracted. The graph attention capsule layer is determined, and multiple graph attention capsule layers are stacked to construct a graph attention capsule network to gradually extract high-order spatiotemporal features. Based on the high-order spatiotemporal features, combined with the attention gating unit, by learning the attention weights on different time scales, the pooling window size of the time dimension is adaptively adjusted to determine the adaptive spatiotemporal pooling mechanism. When the data fluctuation is within the preset short-term time span and is greater than the preset fluctuation threshold, the small-scale pooling window is activated to capture the fine-grained short-term features; when the data fluctuation is outside the preset long-term time span and is less than the preset stable threshold, the large-scale pooling window is activated to extract the long-term trend features; based on the fine-grained short-term features and the long-term trend features, the multi-scale features are determined; A residual self-attention module is introduced between the graph attention capsule network and the adaptive spatiotemporal pooling to model the long-range dependency between corresponding multi-scale features at different spatiotemporal positions through a multi-head self-attention mechanism; Based on long-range dependencies, the external environment characteristics determined based on weather forecast information are integrated to characterize the spatiotemporal patterns and correlation structures of the charging pile group and generate prediction results.
[0008] In an optional embodiment, The method further comprises: Build a multi-task learning framework, determine a subtask based on the prediction task of each charging pile, and all subtasks share the same model structure, including graph attention capsule network, adaptive spatiotemporal pooling mechanism and residual self-attention module, and output the prediction results for multiple time steps in the future; A feature sharing mechanism is adopted to share the hidden layer features extracted by the graph attention capsule network, the adaptive spatiotemporal pooling mechanism, and the residual self-attention module among all subtasks. The input of each subtask passes through the shared feature extraction module to obtain the common hidden layer features, which are mapped through the fully connected layer corresponding to the subtask to generate the prediction output; During model training, based on the prediction output of each subtask obtained in the previous step, the prediction sequence is determined, and a consistency regularization term is introduced. By calculating the normalized Euclidean distance between any two subtask prediction sequences, the prediction sequences generated by different subtasks are made to have the same trend. The consistency regularization term is weighted and summed with the prediction loss of all subtasks to obtain the total loss function. Through the joint training strategy, the prediction loss and consistency regularization term of all subtasks are optimized at the same time. In each training batch, data containing all charging pile input samples is randomly sampled. The prediction output, corresponding prediction loss and consistency regularization term of each subtask are calculated, and the model parameters are updated using the optimization algorithm. The multi-task learning framework constructed by the application shares hidden layer features and introduces a consistency regularization term based on the multi-task prediction output calculation to construct the total loss function. Through joint training, the guidance of consistency constraints is combined when sharing information between different sub-tasks.
[0009] In an optional embodiment, Based on the basic data and prediction results, the objective function including maximum charging efficiency, most balanced grid load, longest cycle life of energy storage battery and minimum user waiting time is determined, and a multi-objective optimization model is constructed. A hybrid optimization method based on multi-objective particle swarm algorithm and simulated annealing algorithm is adopted, and adaptive inertia weight and dynamic constraint processing mechanism are introduced. Through iterative optimization, the optimal charging pile scheduling plan is generated, including: Based on the prediction results, maximizing charging efficiency, minimizing grid load fluctuations, maximizing the cycle life of energy storage batteries, and minimizing user waiting time are taken as optimization goals, and constraints including charging power constraints, energy storage battery charging and discharging constraints, and user demand constraints are established to construct a multi-objective optimization model; The multi-objective optimization model is solved by a hybrid optimization method, including: Initialize the particle swarm, randomly generate the initial position and velocity of each particle, use the particle position vector as the decision variable of the multi-objective optimization model, use the particle velocity vector as the change of the decision variable, initialize the external archive set for storing non-dominated solutions, and initialize the adaptive inertia weight for balancing global exploration and local exploitation; For each particle, the corresponding position vector is substituted into the multi-objective optimization model, and the values of each objective function are calculated. The personal optimal position of the particle is compared and updated through the Pareto dominance relationship between particles. The global optimal position is compared and updated through the Pareto dominance relationship of the solutions in the external archive set, and the current non-dominated solution is added to the external archive set. When the external archive set reaches the maximum capacity, the crowding distance operator is used to calculate the crowding distance, sort the non-dominated solutions, and remove the solutions with crowding distance less than the preset threshold. Based on the individual optimal position, the global optimal position and the adaptive inertia weight of the particle, the velocity vector and the position vector of the particle are updated through the velocity and position update equations, and the adaptive inertia weight decreases as the number of iterations increases, thereby transferring the global exploration to the local development; Randomly select some non-dominated solutions from the external archive set as the initial solutions of simulated annealing, generate new solutions by perturbing the decision variables, calculate the acceptance probability according to the difference between the objective function value of the new solution and the current solution by the Metropolis criterion, and if the acceptance probability is greater than the preset probability threshold, accept and jump out of the local optimum, and update the external archive set, wherein the temperature parameter of simulated annealing is based on an exponential decay method, and the probability threshold is gradually reduced as the number of iterations decreases; For particles that violate the constraints during the iteration process, the violation degree index is determined, and the objective function value is penalized based on the violation degree index. The penalty term is determined in proportion to the violation degree index, and the penalty intensity corresponding to the penalty term is gradually increased as the number of iterations increases. Until the preset upper limit of iterations is reached, the iteration process is completed; An optimal charging pile scheduling solution is selected and determined from the Pareto optimal solution set of the external archive set.
[0010] In an optional embodiment, Randomly select some non-dominated solutions from the external archive set as the initial solutions of simulated annealing, generate new solutions by perturbing the decision variables, calculate the acceptance probability according to the difference between the objective function value of the new solution and the current solution by the Metropolis criterion, and if the acceptance probability is greater than a preset probability threshold, accept and jump out of the local optimum, and update the external archive set including: Calculate the difference in the objective function value between the new solution and the current solution, use the difference in the objective function value as the energy difference of the Metropolis criterion, use the current temperature parameter of simulated annealing as the temperature parameter of the Metropolis criterion, and calculate the acceptance probability of the new solution through the Boltzmann factor; Compare the acceptance probability with the random probability generated by uniform distribution, if the acceptance probability is greater than the random probability, accept the new solution as the current solution and jump out of the local optimum; otherwise, keep the current solution unchanged; Regardless of whether the new solution is accepted or not, the new solution is compared with the solutions in the external archive set in terms of Pareto dominance. If the new solution is not dominated by any solution in the external archive set, the new solution is added to the external archive set, and the solutions in the external archive set dominated by the new solution are removed, and the external archive set is updated; The temperature parameters of simulated annealing are updated by exponential decay. The initial temperature, decay coefficient and termination temperature are set. After each new solution is accepted, the temperature parameters are calculated according to the current number of iterations. The temperature parameters decrease exponentially with the number of iterations. The process is repeated until the last round of simulated annealing is reached, and the external archive set is used as the output of simulated annealing to determine the Pareto optimal solution set.
[0011] In an optional embodiment, Based on the optimal charging pile scheduling scheme, the charging pile group is divided into multiple sub-networks through a distributed collaborative scheduling method based on graph attention network and hierarchical multi-agent reinforcement learning. The charging piles and energy storage battery packs in each sub-network are used as the underlying agents, and the controller of the sub-network is used as the high-level agent. The improved soft actor-critic algorithm is used to train the agents. Through the trained agents, the scheduling tasks are dynamically executed, including: According to the geographical distribution and communication topology of charging piles, the charging pile group is divided into multiple sub-networks. Each sub-network contains multiple charging pile nodes, energy storage battery groups and a controller node. The nodes within the sub-network are connected through communication links, and the controller nodes of different sub-networks are connected through communication links, forming a two-layer network topology. Based on the two-layer network topology of the charging pile group hierarchical network topology, a hierarchical multi-agent framework is constructed. The charging pile nodes and energy storage battery groups in the sub-network are modeled as bottom-level agents, which are responsible for performing charging and discharging scheduling tasks. The controller nodes of the sub-network are modeled as high-level agents, which are responsible for coordinating and managing the collaborative operations of the bottom-level agents. The graph attention network is embedded into the hierarchical multi-agent framework. For the bottom-level agent, the graph attention layer is used to process its own state and neighbor state, learn the importance weight of the neighbor state to its own decision, aggregate the neighbor information to generate a node embedding vector, and map it to the action space to generate a scheduling strategy; for the high-level agent, the graph attention layer is used to process the state of all bottom-level agents, learn the importance weight of different bottom-level agents to the scheduling decision of the sub-network, aggregate the bottom-level information to generate a controller embedding vector, and map it to the action space to generate a scheduling instruction; An improved soft actor-critic algorithm is used to centrally train each agent in the hierarchical multi-agent framework. A parallel actor network and critic network are introduced, corresponding to agents at different levels. When the critic network estimates the state-action value function, the observations and actions of agents at all levels are concatenated. When the actor network calculates the policy gradient, the policy gradients of agents at all levels are weighted summed. Based on the trained actor network, the hierarchical multi-agent framework is executed online in a distributed manner. The bottom-level agent generates a scheduling strategy based on the local observation status through the actor network embedded in the graph attention network, executes the assigned scheduling tasks, and uploads the status information to the high-level agent; the high-level agent generates a scheduling instruction based on the status of all bottom-level agents through the actor network embedded in the graph attention network, and sends it to the corresponding bottom-level agent to complete the dynamic execution of the scheduling task.
[0012] In an optional embodiment, The improved soft actor-critic algorithm is used to centrally train each agent in the hierarchical multi-agent framework. Parallel actor networks and critic networks are introduced, corresponding to agents at different levels. When the critic network estimates the state-action value function, the observations and actions of agents at all levels are concatenated. When the actor network calculates the policy gradient, the weighted sum of the policy gradients of agents at all levels includes: Introduce multiple parallel actor networks and critic networks, where the number of actor networks and critic networks corresponds to the number of low-level agents and high-level agents, respectively; When the critic network estimates the state-action value function, the observations and actions of the bottom-level agent are concatenated with those of the high-level agent to form a state-action pair containing complete information in the entire hierarchical multi-agent framework as the critic network input. Based on the critic network input, the state-action value function of the entire hierarchical multi-agent framework is evaluated. When the actor network calculates the policy gradient, according to the importance of the bottom-level agents and the high-level agents in the entire hierarchical multi-agent framework, the corresponding weight coefficients are set, and the policy gradients of all bottom-level agents and the policy gradients of all high-level agents are weighted summed as the policy gradient of the entire hierarchical multi-agent framework to update the actor network parameters; Hierarchical exploration noise is introduced. For the actor network of the bottom-level intelligent agent, Gaussian noise with a variance less than the noise variance threshold is added when the action is output. For the actor network of the high-level intelligent agent, Gaussian noise with a variance greater than the noise variance threshold is added when the action is output.
[0013] According to a second aspect of the embodiments of the present invention, Provided is an intelligent charging pile scheduling system based on dynamic adjustment of energy storage battery packs, comprising: The first unit is used to collect multi-dimensional real-time data of the charging pile group, and obtain the status information of the energy storage battery group to obtain basic data. Combined with the historical load curve and weather forecast information, it predicts the future through graph attention and adaptive spatiotemporal pooling mechanism to obtain the prediction results; The second unit is used to determine the objective function including maximum charging efficiency, most balanced grid load, longest cycle life of energy storage battery, and minimum user waiting time based on basic data and prediction results, build a multi-objective optimization model, adopt a hybrid optimization method based on multi-objective particle swarm algorithm and simulated annealing algorithm, introduce adaptive inertia weight and dynamic constraint processing mechanism, and generate the optimal charging pile scheduling plan through iterative optimization; The third unit is used to divide the charging pile group into multiple sub-networks based on the optimal charging pile scheduling plan through a distributed collaborative scheduling method based on graph attention network and hierarchical multi-agent reinforcement learning. The charging piles and energy storage battery groups in each sub-network are used as the underlying agents, and the controller of the sub-network is used as the high-level agent. The improved soft actor-critic algorithm is used to train the agents, and the scheduling tasks are dynamically executed through the trained agents.
[0014] According to a third aspect of the embodiments of the present invention, An electronic device is provided, comprising: processor; a memory for storing processor-executable instructions; The processor is configured to call the instructions stored in the memory to execute the aforementioned method.
[0015] A fourth aspect of the embodiments of the present invention is: A computer-readable storage medium is provided, on which computer program instructions are stored. When the computer program instructions are executed by a processor, the aforementioned method is implemented.
[0016] In an embodiment of the present invention, based on the prediction results, the charging and discharging plan of the energy storage battery group can be optimized, and the charging pile resources can be reasonably allocated, especially the scheduling during peak hours; the adaptive spatiotemporal pooling mechanism can be used to filter irrelevant data, reduce the interference of redundant information, and improve the stability of the prediction model; equipment maintenance can be carried out during the load valley period to extend the equipment service life; the prediction results are compared and fed back with the actual situation, and the model can be continuously optimized to gradually improve the prediction accuracy and adapt to different usage scenarios and environmental changes; the multi-objective optimization model considers the goal of the most balanced grid load, avoids grid overload or uneven load, and improves the stability and safety of the grid; the optimization plan includes the goal of the longest cycle life of the energy storage battery, ensures that the battery is charged and discharged within a reasonable working range, and extends its service life; the objective function includes the constraint of minimizing user waiting time, optimizes the charging pile scheduling, enables users to complete charging faster, and improves user experience ; Through the hybrid optimization of multi-objective particle swarm algorithm and simulated annealing algorithm, combined with adaptive inertia weight and dynamic constraint processing mechanism, the optimal charging pile scheduling plan can be generated in the iterative process to achieve a comprehensive balance of various indicators; through hierarchical multi-agent reinforcement learning, sub-networks are divided and distributed collaborative scheduling is carried out, so that the charging piles and energy storage battery groups of each sub-network can independently and efficiently complete local scheduling, thereby improving the overall scheduling efficiency; the agents can make independent decisions and collaborate according to the dynamic environment of different sub-networks, achieve rapid response and adapt to complex charging demand changes, and enhance the flexibility and robustness of the system; the high-level agents are responsible for the global coordination between sub-networks, while the low-level agents focus on the optimization of local charging piles and energy storage, and ensure the balance between global optimization and local optimization through hierarchical management; the improved soft actor-critic algorithm is used to train agents to improve their decision-making efficiency and quality in complex environments and optimize scheduling strategies. BRIEF DESCRIPTION OF THE DRAWINGS
[0017] Figure 1 It is a flow chart of a smart charging pile scheduling method based on dynamic adjustment of energy storage battery packs according to an embodiment of the present invention; Figure 2 It is a structural schematic diagram of an intelligent charging pile scheduling system based on dynamic adjustment of energy storage battery packs according to an embodiment of the present invention. DETAILED DESCRIPTION
[0018] In order to make the purpose, technical solution and advantages of the embodiments of the present invention clearer, the technical solution in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.
[0019] The technical solution of the present invention is described in detail with specific embodiments below. The following specific embodiments can be combined with each other, and the same or similar concepts or processes may not be described in detail in some embodiments.
[0020] Figure 1 FIG. 1 is a flow chart of a smart charging pile scheduling method based on dynamic adjustment of energy storage battery packs according to an embodiment of the present invention. Figure 1 As shown, the method includes: S101. Collect multi-dimensional real-time data of the charging pile group, and obtain the status information of the energy storage battery group at the same time, obtain basic data, combine historical load curves and weather forecast information, and use graph attention and adaptive spatiotemporal pooling mechanisms to predict the future and obtain prediction results; In the management of charging pile groups, it is first necessary to establish a real-time data collection system that can capture various dimensional data of the charging pile group, such as charging power (real-time current, voltage), parking space status (idle, occupied), user behavior data (charging time, user identity), charging pile equipment status (fault information, maintenance status), and environmental data (temperature, humidity, air quality).
[0021] As an auxiliary device of the power grid, the status data of the energy storage battery pack also needs to be collected and monitored, mainly including: battery power percentage, battery health status (SOH), number of charge and discharge cycles, current charge and discharge power, and battery temperature.
[0022] The real-time data of the charging pile group and the status information of the energy storage battery group are aggregated to form a basic data set. The data needs to be preprocessed, such as: filtering of abnormal data (such as equipment failure data, missing values), standardization of data (such as unified processing of data from different units), and time sequence arrangement of data (ensuring that data is arranged in chronological order) In addition to real-time data and energy storage battery data, in order to make more accurate future predictions, it is also necessary to introduce historical load curves and weather forecast information: historical load curves, which record past load changes, can usually obtain the changing trends of electricity demand in different time periods; weather forecast information, the weather has a significant impact on charging demand and energy storage battery performance, especially meteorological factors such as temperature and humidity.
[0023] The graph attention mechanism is a deep learning model for processing graph structured data. It captures the mutual influence between different charging piles in a charging pile group by introducing the relationship between graph structure nodes. For example, the usage of adjacent charging piles may be correlated, and the idleness of a charging pile may affect the use of other charging piles. Through the graph attention mechanism, a graph structure can be established based on the spatial position and device dependency between charging piles. The model can learn the weights of different nodes (charging piles) to more accurately predict the load situation.
[0024] The adaptive spatiotemporal pooling mechanism can effectively handle the complexity of spatiotemporal data. The mechanism can dynamically select information in time and space for pooling, reduce the interference of redundant data, and thus improve the prediction accuracy of the model. Through the adaptive spatiotemporal pooling mechanism, the model can focus on the time period and spatial location that contributes most to the prediction, ignore irrelevant or noisy data, and thus improve the stability of the prediction.
[0025] Based on the graph attention mechanism and the adaptive spatiotemporal pooling mechanism, a prediction model is constructed by combining the real-time data of the charging pile group, the status information of the energy storage battery group, the historical load curve and the weather forecast. This model can predict the charging demand and load conditions in a specific time period in the future.
[0026] Finally, the prediction results are obtained and used in actual scheduling and management, such as preparing charging piles and energy storage equipment for peak hours in advance, adjusting the charging and discharging plan of energy storage battery packs, and performing equipment maintenance during off-peak hours. At the same time, the prediction results are compared with the actual situation and fed back to the model for optimization to continuously improve the prediction accuracy.
[0027] In this embodiment, through multi-dimensional data collection, combining historical load curves and weather forecasts, using graph attention mechanism and adaptive spatiotemporal pooling mechanism, the future load demand of the charging pile group can be accurately predicted, and management efficiency can be improved; based on the prediction results, the charging and discharging plan of the energy storage battery group can be optimized, and the charging pile resources can be reasonably allocated, especially the scheduling during peak hours; using the adaptive spatiotemporal pooling mechanism, irrelevant data can be filtered, the interference of redundant information can be reduced, and the stability of the prediction model can be improved; equipment maintenance can be carried out during the load valley period to extend the service life of the equipment; by comparing and feeding back the prediction results with the actual situation, the model can be continuously optimized, the prediction accuracy can be gradually improved, and it can adapt to different usage scenarios and environmental changes.
[0028] In an optional embodiment, multi-dimensional real-time data of the charging pile group is collected, and the status information of the energy storage battery group is obtained at the same time to obtain basic data. Combined with the historical load curve and weather forecast information, the future is predicted through the graph attention and adaptive spatiotemporal pooling mechanism, and the prediction results include: Each charging pile node is represented as a capsule. Based on the activation degree of the charging pile node, the vector length of the vector inside the capsule is determined. Based on the feature change trend of the charging pile node, the vector direction of the vector inside the capsule is determined. Based on the vector length and vector direction, the correlation between capsules is calculated through the attention mechanism, and the neighborhood weight of the capsule is dynamically updated. The dynamic routing algorithm is used to realize information transmission and aggregation between capsules. Capsules with neighborhood weights greater than the preset threshold are formed into high-level capsules, and the hierarchical feature representation of the charging pile group is extracted. The graph attention capsule layer is determined, and multiple graph attention capsule layers are stacked to construct a graph attention capsule network to gradually extract high-order spatiotemporal features. Based on the high-order spatiotemporal features, combined with the attention gating unit, by learning the attention weights on different time scales, the pooling window size of the time dimension is adaptively adjusted to determine the adaptive spatiotemporal pooling mechanism. When the data fluctuation is within the preset short-term time span and is greater than the preset fluctuation threshold, the small-scale pooling window is activated to capture the fine-grained short-term features; when the data fluctuation is outside the preset long-term time span and is less than the preset stable threshold, the large-scale pooling window is activated to extract the long-term trend features; based on the fine-grained short-term features and the long-term trend features, the multi-scale features are determined; A residual self-attention module is introduced between the graph attention capsule network and the adaptive spatiotemporal pooling to model the long-range dependency between corresponding multi-scale features at different spatiotemporal positions through a multi-head self-attention mechanism; Based on long-range dependencies, the external environment characteristics determined based on weather forecast information are integrated to characterize the spatiotemporal patterns and correlation structures of the charging pile group and generate prediction results.
[0029] Each charging pile node is abstracted into a capsule, which is a high-dimensional vector representation that can capture the complex features of the node. The capsule contains multiple vectors, each of which represents a certain state or feature of the charging pile. The status of the charging pile node may include the current charging power, equipment health status, parking space occupancy, etc. For example, a charging pile A is used as a capsule, and its internal vectors may include: charging power is 30 kilowatts, parking space is occupied, and equipment status is healthy.
[0030] The length of the vector inside the capsule indicates the activation level of the charging pile, that is, the significance of the current state of the charging pile. By analyzing the real-time data and historical trends of the charging pile, the importance of the node in the network is determined. The higher the activation level, the longer the vector length, indicating that the charging pile has a greater impact on the overall network at this time. For example, charging pile A is operating at high power and the device is in good condition, indicating that its activation level is high and the length of the internal vector is long.
[0031] The direction of the vector indicates the characteristic change trend of the charging pile node. If the power of the charging pile is on the rise, the vector may point to a specific direction, while if the power tends to be stable, the direction of the vector may remain unchanged. By analyzing the characteristic changes of the current node (such as power, state, etc.), the direction of the internal vector is determined. For example, the power of charging pile A has gradually increased over the past period of time, so the direction of the vector will point to an increasing trend.
[0032] By comparing the vector lengths and directions of different capsules, the attention mechanism is used to calculate the correlation between them. Capsules with high correlation will form a closer connection, indicating a greater mutual influence. The attention mechanism can dynamically adjust the weights between charging pile nodes to capture the dependencies between different nodes. For example, charging pile A and charging pile B are located in the same area and have similar power change trends. The attention mechanism will calculate that the correlation between them is high and assign a larger weight.
[0033] According to the correlation calculated by the attention mechanism, the neighborhood weight of each capsule is dynamically updated. The larger the weight, the stronger the influence of the node on the neighborhood. The spatial position and state of the charging piles will affect the distribution of neighborhood weights. For example, the correlation between charging piles A and B is high, so their neighborhood weights are increased, while the correlation with charging pile C is low and the weight is small.
[0034] The dynamic routing algorithm is used to transfer information between capsules and aggregate them according to neighborhood weights. Capsules with higher weights will preferentially transfer information, forming an effective flow of information. Through multiple iterations, efficient information transfer and feature extraction are finally achieved. For example, through the dynamic routing algorithm, the high-weight connection between charging piles A and B will promote information sharing between them and aggregate their features.
[0035] When the neighborhood weight is greater than the preset threshold, multiple capsules are aggregated into a high-level capsule. The high-level capsule can extract the hierarchical features of the charging pile group and represent more abstract features. The high-level capsule represents the overall status of the charging pile group, not just the status of a single charging pile. Exemplarily, the neighborhood weights of charging piles A, B, and C are higher than the threshold, and they are aggregated into a high-level capsule, representing the overall load of the charging piles in the area.
[0036] Multiple graph attention capsule layers are stacked to construct a graph attention capsule network. Through the multi-level capsule network, the high-order spatiotemporal features of the charging pile group are gradually extracted. The graph attention capsule network structure can capture the complex spatiotemporal dependencies between charging pile nodes and extract useful features. For example, through the multi-layer graph attention capsule network, the load changes and spatial distribution characteristics of the charging pile group in different time periods are gradually extracted.
[0037] Through the attention gating unit, the model can learn the attention weights on different time scales and automatically adjust the pooling window size of the time dimension. When the data fluctuates greatly, a small-scale pooling window is selected to capture short-term changes; when the data is stable, a large-scale pooling window is selected to capture long-term trends. For example, during peak hours when charging demand fluctuates violently, the model selects a small-scale pooling window to capture short-term load changes; and during the stable period at night, the model selects a large-scale pooling window to capture long-term trends.
[0038] Depending on the size of the pooling window, the model is able to extract fine-grained short-term features and long-term trend features, and then determine multi-scale features. Short-term features mainly reflect the rapid changes in the charging pile group, while long-term features reflect its overall trend. For example, the fine-grained short-term features during peak hours show that the load of the charging pile group rises rapidly, while the long-term trend features show that the load gradually increases within a week.
[0039] By introducing the residual self-attention mechanism, the model can model the long-range dependencies between multi-scale features at different spatiotemporal locations. This helps capture the complex interactions between different nodes in the charging pile group. The residual mechanism can also prevent the model from overfitting and accelerate training. For example, through the residual self-attention module, the model can capture the long-range dependency between charging pile A and charging pile B. Although they are located in different areas, their load changes have a certain correlation.
[0040] By introducing weather forecast information (such as temperature, humidity, etc.), the model can more accurately characterize the spatiotemporal patterns and correlation structures of the charging pile cluster. Weather conditions directly affect charging demand and must be taken into account. By integrating environmental characteristics, the model can better predict future loads. For example, the weather forecast shows that there will be high temperatures tomorrow. In combination with this information, the model predicts that the load of the charging pile cluster will increase significantly during peak hours.
[0041] Based on the extracted high-order spatiotemporal features, long-range dependencies, and weather forecast information, the model ultimately generates future load forecasts for the charging pile group. The load forecast results can be used for scheduling and management to prevent overload or waste of resources. For example, the model predicts that the load of the charging pile group will peak between 4 and 7 p.m. tomorrow, with a total load of approximately 150 kilowatts. Managers can arrange the scheduling of energy storage equipment and charging piles in advance.
[0042] In this embodiment, by representing the charging pile nodes as capsules and combining the graph attention capsule network with the adaptive spatiotemporal pooling mechanism, the multi-level and high-order spatiotemporal characteristics of the charging pile group can be accurately captured, thereby improving the accuracy of load prediction; based on the fluctuation of the data, the pooling window size is adaptively adjusted, so that fine-grained features can be captured in short-term high fluctuations and trend features can be extracted in long-term stability, thereby achieving accurate analysis of different time scales; through the residual self-attention module and the multi-head self-attention mechanism, the long-range dependencies at different time and space positions are captured, and the spatiotemporal patterns and correlation structures of the charging pile group are comprehensively characterized; the introduction of external environmental features such as weather makes the prediction model more realistic, which can effectively reflect the impact of environmental factors on charging demand and further improve the reliability of the prediction results; the dynamic routing algorithm is used to transmit and aggregate information between capsules, and the neighborhood weights of capsules are dynamically updated, effectively improving the efficiency of hierarchical feature extraction of the charging pile group.
[0043] In an optional embodiment, the method further includes: Build a multi-task learning framework, determine a subtask based on the prediction task of each charging pile, and all subtasks share the same model structure, including graph attention capsule network, adaptive spatiotemporal pooling mechanism and residual self-attention module, and output the prediction results for multiple time steps in the future; A feature sharing mechanism is adopted to share the hidden layer features extracted by the graph attention capsule network, the adaptive spatiotemporal pooling mechanism, and the residual self-attention module among all subtasks. The input of each subtask passes through the shared feature extraction module to obtain the common hidden layer features, which are mapped through the fully connected layer corresponding to the subtask to generate the prediction output; During model training, based on the prediction output of each subtask obtained in the previous step, the prediction sequence is determined, and a consistency regularization term is introduced. By calculating the normalized Euclidean distance between any two subtask prediction sequences, the prediction sequences generated by different subtasks are made to have the same trend. The consistency regularization term is weighted and summed with the prediction loss of all subtasks to obtain the total loss function. Through the joint training strategy, the prediction loss and consistency regularization term of all subtasks are optimized at the same time. In each training batch, data containing all charging pile input samples is randomly sampled. The prediction output, corresponding prediction loss and consistency regularization term of each subtask are calculated, and the model parameters are updated using the optimization algorithm. The multi-task learning framework constructed by the application shares hidden layer features and introduces a consistency regularization term based on the multi-task prediction output calculation to construct the total loss function. Through joint training, the guidance of consistency constraints is combined when sharing information between different sub-tasks.
[0044] In the multi-task learning framework, the prediction task of each charging pile is defined as a subtask. All subtasks share the same model structure, including the graph attention capsule network, the adaptive spatiotemporal pooling mechanism, and the residual self-attention module. These modules are used together to extract the features of the charging pile and finally output the prediction results for multiple time steps in the future. The goal of each subtask is to predict the future load of the charging pile. All subtasks remain consistent in structure, but the input data is different. Exemplarily, for charging piles A, B, and C, the load prediction of each charging pile is regarded as a subtask. The subtask of charging pile A predicts the load changes in the next 30 minutes, and charging piles B and C also have their own prediction tasks. All subtasks share the same model structure in the multi-task learning framework.
[0045] In the multi-task learning framework, the hidden layer features extracted by the graph attention capsule network, the adaptive spatiotemporal pooling mechanism, and the residual self-attention module are shared among all subtasks. Through this mechanism, each subtask of the charging pile can learn useful features from other subtasks. The input of each subtask (i.e., the real-time data of the charging pile) first passes through a shared feature extraction module to obtain common hidden layer features. Then, it is mapped through a subtask-specific fully connected layer to generate a predicted output for each subtask. Exemplarily, the subtask of charging pile A inputs its current power, parking space status, user behavior and other data, which are extracted from the hidden layer features through a shared graph attention capsule network and an adaptive spatiotemporal pooling mechanism. These features not only serve the subtask of charging pile A, but are also shared by the subtasks of charging piles B and C, making it easier to learn the overall features of the charging pile group.
[0046] During the model training process, the prediction output of each subtask is obtained based on the shared feature extraction module, thereby generating the corresponding prediction sequence. The prediction sequence of each subtask corresponds to the load prediction of the charging pile for multiple time steps in the future. These prediction sequences are an important basis for subsequent consistency regularization constraints. For example, the subtask of charging pile A generates a load prediction sequence for the next 60 minutes, and charging piles B and C also generate their own load prediction sequences. These sequences will be used for subsequent consistency regularization term calculations.
[0047] In order to make the prediction sequences of different subtasks consistent in trend, we achieve this goal by calculating the normalized Euclidean distance of the prediction sequences of any two subtasks. The consistency regularization term is used to constrain the prediction results of each subtask so that they are more uniform in trend. This mechanism can capture the overall load change trend of the charging pile group and avoid large deviations in the prediction results of a single charging pile. For example, the load change trends of charging pile A and charging pile B are similar. We calculate the normalized Euclidean distance of the prediction sequences of the two to ensure that their prediction results are consistent in trend. If charging pile A predicts an increase in load and charging pile B predicts a decrease in load, the model will constrain their trends to be more consistent through the regularization term.
[0048] The consistency regularization term and the prediction loss of each subtask are weighted and summed to form a total loss function. The prediction loss measures the error between the predicted output of each subtask and the true value, while the consistency regularization term constrains the prediction trends between subtasks to remain consistent. Through this total loss function, the model can simultaneously optimize the prediction accuracy of each subtask and ensure that the prediction trends between different subtasks are coordinated and consistent as a whole. For example, the prediction losses of charging piles A, B, and C are 0.1, 0.2, and 0.15, respectively. By introducing a consistency regularization term (for example, 0.05), the final total loss function is the weighted sum of each prediction loss plus the regularization term.
[0049] During the model training process, we adopt a joint training strategy to optimize the prediction loss and consistency regularization term of all subtasks. In each training batch, data containing all charging pile input samples are randomly sampled, and the predicted output, corresponding prediction loss, and consistency regularization term of each subtask are calculated. The parameters of the model are updated using an optimization algorithm (such as gradient descent) so that the model can simultaneously improve the prediction accuracy of each subtask and maintain the trend consistency between subtasks through the consistency regularization term. For example, in a training batch, input samples of charging piles A, B, and C are randomly sampled, their predicted outputs (such as the load in the next 30 minutes) are calculated, and the prediction loss of each subtask is calculated. Then, the prediction sequence consistency regularization term of charging piles A and B, B and C is calculated, and the parameters of the model are updated after combining the total loss function.
[0050] By sharing hidden layer features, all subtasks can learn useful information from other subtasks. This is particularly beneficial for the overall load prediction of a group of charging piles, because the loads of each charging pile often have certain spatiotemporal correlations. Shared features can also reduce the number of model parameters, making the model more efficient and faster to train. For example, the load change trend of charging pile A may affect the load prediction of charging pile B. By sharing hidden layer features, the subtask of charging pile B can learn from the features of A and improve its own prediction accuracy.
[0051] By introducing the consistency regularization term, the model can be guided by trend consistency when sharing information between different subtasks. In this way, even if the input data of a subtask fluctuates greatly, the model can maintain the stability of the prediction results through the sharing mechanism and the consistency regularization term. The consistency constraint can also avoid large prediction deviations between different subtasks, especially in a group of charging piles, where the load changes of multiple charging piles are often synchronized. For example, although the load changes of charging pile A are more drastic, through the guidance of the consistency regularization term, the prediction trend of charging pile A remains consistent with charging piles B and C, ensuring the stability of the overall load prediction.
[0052] Through the joint training strategy, the model can share hidden layer features while combining consistency regularization constraints to keep the prediction results of each subtask consistent in trend. The combined effect of this information sharing and consistency constraints helps to improve the overall prediction accuracy. In each training batch, the model updates the model parameters by calculating the prediction loss and consistency regularization term of each subtask, so that each subtask can benefit from other subtasks. For example, the subtask of charging pile A learns a more stable load trend from the features of charging piles B and C through shared features and consistency constraints, ultimately improving its prediction accuracy.
[0053] In this embodiment, by constructing a multi-task learning framework, the prediction tasks of all charging piles share the model structure and hidden layer features, which is conducive to information complementarity and sharing between tasks and improves the overall prediction accuracy; a feature sharing mechanism is adopted to avoid building a separate model for each subtask, reduce model complexity, and improve computational efficiency and resource utilization; a consistency regularization term is introduced to force the prediction results of different subtasks to maintain trend consistency, avoid large deviations in predictions between subtasks, and improve prediction stability; through a joint training strategy, the prediction loss of all subtasks is optimized, and consistency constraints are maintained, so that the model has better generalization ability on different subtasks and adapts to more complex charging pile load scenarios; through multi-task joint training, the model can optimize multiple tasks at the same time in each batch, speed up the training process, and improve the learning efficiency of each subtask by sharing information.
[0054] S102. Based on the basic data and prediction results, determine the objective function including maximum charging efficiency, most balanced grid load, longest cycle life of energy storage battery, and minimum user waiting time, build a multi-objective optimization model, adopt a hybrid optimization method based on multi-objective particle swarm algorithm combined with simulated annealing algorithm, introduce adaptive inertia weight and dynamic constraint processing mechanism, and generate the optimal charging pile scheduling plan through iterative optimization; In this embodiment, the goal of maximizing charging efficiency is determined by the optimization model, so that charging pile resources can be reasonably scheduled to improve the overall charging efficiency; the multi-objective optimization model considers the goal of the most balanced load on the power grid, avoids overload or uneven load on the power grid, and improves the stability and safety of the power grid; the optimization plan includes the goal of longest cycle life of the energy storage battery, ensuring that the battery is charged and discharged within a reasonable working range and extending its service life; the objective function includes the constraint of minimizing user waiting time, optimizing charging pile scheduling, enabling users to complete charging faster, and improving user experience; through the hybrid optimization of the multi-objective particle swarm algorithm and the simulated annealing algorithm, combined with the adaptive inertia weight and dynamic constraint processing mechanism, the optimal charging pile scheduling plan can be generated in the iterative process to achieve a comprehensive balance of various indicators.
[0055] In an optional embodiment, based on basic data and prediction results, an objective function including maximum charging efficiency, most balanced grid load, longest cycle life of energy storage battery, and minimum user waiting time is determined, a multi-objective optimization model is constructed, a hybrid optimization method based on a combination of a multi-objective particle swarm algorithm and a simulated annealing algorithm is adopted, an adaptive inertia weight and a dynamic constraint processing mechanism are introduced, and an optimal charging pile scheduling scheme is generated through iterative optimization, including: Based on the prediction results, maximizing charging efficiency, minimizing grid load fluctuations, maximizing the cycle life of energy storage batteries, and minimizing user waiting time are taken as optimization goals, and constraints including charging power constraints, energy storage battery charging and discharging constraints, and user demand constraints are established to construct a multi-objective optimization model; The multi-objective optimization model is solved by a hybrid optimization method, including: Initialize the particle swarm, randomly generate the initial position and velocity of each particle, use the particle position vector as the decision variable of the multi-objective optimization model, use the particle velocity vector as the change of the decision variable, initialize the external archive set for storing non-dominated solutions, and initialize the adaptive inertia weight for balancing global exploration and local exploitation; For each particle, the corresponding position vector is substituted into the multi-objective optimization model, and the values of each objective function are calculated. The personal optimal position of the particle is compared and updated through the Pareto dominance relationship between particles. The global optimal position is compared and updated through the Pareto dominance relationship of the solutions in the external archive set, and the current non-dominated solution is added to the external archive set. When the external archive set reaches the maximum capacity, the crowding distance operator is used to calculate the crowding distance, sort the non-dominated solutions, and remove the solutions with crowding distance less than the preset threshold. Based on the individual optimal position, the global optimal position and the adaptive inertia weight of the particle, the velocity vector and the position vector of the particle are updated through the velocity and position update equations, and the adaptive inertia weight decreases as the number of iterations increases, thereby transferring the global exploration to the local development; Randomly select some non-dominated solutions from the external archive set as the initial solutions of simulated annealing, generate new solutions by perturbing the decision variables, calculate the acceptance probability according to the difference between the objective function value of the new solution and the current solution by the Metropolis criterion, and if the acceptance probability is greater than the preset probability threshold, accept and jump out of the local optimum, and update the external archive set, wherein the temperature parameter of simulated annealing is based on an exponential decay method, and the probability threshold is gradually reduced as the number of iterations decreases; For particles that violate the constraints during the iteration process, the violation degree index is determined, and the objective function value is penalized based on the violation degree index. The penalty term is determined in proportion to the violation degree index, and the penalty intensity corresponding to the penalty term is gradually increased as the number of iterations increases. Until the preset upper limit of iterations is reached, the iteration process is completed; An optimal charging pile scheduling solution is selected and determined from the Pareto optimal solution set of the external archive set.
[0056] In the multi-objective optimization model, the following four optimization goals are set: maximizing charging efficiency, the goal is to improve the overall efficiency of charging piles so that vehicles can be fully charged in the shortest time; minimizing grid load fluctuations, by reasonably scheduling charging power, reducing the impact on the grid load during charging, and avoiding drastic fluctuations in the grid load; maximizing the cycle life of energy storage batteries, optimizing the charging and discharging strategies of energy storage batteries, extending their lifespan, and avoiding excessive losses caused by frequent charging and discharging; minimizing user waiting time, reducing user waiting time, optimizing the scheduling and power distribution of charging piles, and enabling users to complete charging as soon as possible.
[0057] At the same time, constraints are set: charging power constraint, the charging power of each charging pile must be within the rated power range of the equipment; energy storage battery charging and discharging constraint, the charging and discharging rate and capacity of the energy storage battery must comply with its physical limitations to avoid overcharging or over-discharging; user demand constraint, meet the user's charging needs and ensure that the user completes charging within a reasonable time.
[0058] For example, a charging pile group consists of 10 charging piles. The goal is to maximize the charging efficiency, minimize grid load fluctuations and user waiting time, and extend the service life of the energy storage battery while satisfying equipment and user constraints through the above multi-objective optimization model.
[0059] In particle swarm optimization (PSO), each particle represents a charging pile scheduling plan, including the charging power and scheduling time of each charging pile. The initial position and velocity of each particle are randomly generated. The position vector represents the current charging pile scheduling decision (such as charging power allocation, energy storage battery usage strategy, etc.); the velocity vector represents the change in the decision variable, which determines how the particle moves in each iteration.
[0060] Initialize an external archive set for storing non-dominated solutions (i.e., Pareto optimal solutions) to record the current optimal solution; initialize the adaptive inertia weight to balance global exploration and local development. The inertia weight gradually decreases with the increase in the number of iterations, allowing the algorithm to gradually shift from global exploration to local development. For example, in a system containing 10 charging piles, each particle represents a scheduling scheme, including the power allocation of each charging pile (such as 5 kW, 7 kW, etc.) and the strategy for using energy storage batteries (such as charging or discharging). The initial particle swarm consists of 20 particles, and the initial position and velocity of each particle are randomly generated.
[0061] For each particle, its position vector is substituted into the multi-objective optimization model to calculate the corresponding objective function values of charging efficiency, grid load fluctuation, energy storage battery life and user waiting time. By comparing the objective function values of different particles, it is determined whether there is a Pareto dominance relationship between particles. If a particle is better than another particle in all objectives, it is said to dominate the other particle. Each particle records its historical optimal position (i.e., personal optimality). If the current position is better than the personal optimality, the personal optimal position is updated. The global optimal position is compared and updated through the Pareto dominance relationship of the solutions in the external archive set. The current non-dominated solution is added to the external archive set. When the set reaches the maximum capacity, the congestion distance operator is used to calculate the congestion distance of each solution, and the solutions with a congestion distance less than the preset threshold are removed to ensure that the archive set maintains diversity. Exemplarily, particle 1 and particle 2 represent two scheduling schemes, respectively. Particle 1 has a higher charging efficiency but a larger grid load fluctuation, while particle 2 has a smaller grid load fluctuation but a lower charging efficiency. Judging by the Pareto dominance relationship, neither particle is a solution that completely dominates the other, so they can both be saved in the external archive set.
[0062] Based on the particle's personal optimal position, global optimal position, and adaptive inertia weight, the velocity and position update equations are used to update the particle's velocity vector and position vector. The adaptive inertia weight gradually decreases with the increase in the number of iterations, prompting the particle's search to shift from global to local. For example, in the 10th iteration, the velocity vector of particle 1 is updated to (0.3, 0.4), and the position vector is updated to (6 kW, 8 kW), indicating that the power distribution of the charging pile has changed.
[0063] Randomly select some non-dominated solutions from the external archive set as the initial solutions for simulated annealing. Generate new solutions by perturbing the decision variables, and calculate the difference in the objective function value between the new solution and the current solution. According to the Metropolis criterion, calculate the probability of accepting the new solution. If the acceptance probability is greater than the preset threshold, the new solution is accepted, the local optimum is jumped out, and the external archive set is updated. The temperature parameter of simulated annealing is based on an exponential decay method. As the number of iterations increases, the temperature and probability thresholds gradually decrease, reducing the probability of accepting a worse solution and prompting the algorithm to converge. Exemplarily, in the 50th iteration, the simulated annealing algorithm makes a small adjustment to the power distribution of the charging pile (such as adjusting the charging power from 6.8 kilowatts to 6.5 kilowatts), and calculates the difference between the new solution and the current solution based on the change in the objective function value. If the difference is small and the acceptance probability is greater than the threshold, the new solution is accepted.
[0064] For particles that violate the constraints during the iteration process, the violation degree index is determined. According to the violation degree index, the objective function value is penalized. As the number of iterations increases, the penalty corresponding to the violation degree index gradually increases, prompting the particles to meet the constraints. For example, particle 3 violates the charge and discharge constraints of the energy storage battery and exceeds the maximum allowable discharge rate. The model penalizes its objective function value according to the degree of violation, so that it tends to meet the constraints in subsequent iterations.
[0065] According to the above steps, the position and velocity of the particles are continuously updated iteratively, and simulated annealing is used to adjust the solution and process particles that violate the constraints. As the number of iterations increases, the particles gradually converge to the optimal solution. When the number of iterations reaches a preset upper limit (for example, 100 iterations), the iteration process is stopped. Exemplarily, at the 100th iteration, the particle swarm has gradually converged, the solution in the external archive set has stabilized, and is no longer significantly updated, and the iteration process ends.
[0066] The optimal charging pile scheduling scheme is selected from the Pareto optimal solutions in the external archive set. This scheme achieves a good balance between multiple optimization objectives (such as charging efficiency, grid load fluctuation, energy storage battery life, and user waiting time). For example, after 100 iterations, 5 Pareto optimal solutions are retained in the external archive set. Finally, a scheme is selected that achieves a good balance between charging efficiency, grid load fluctuation, energy storage battery life, and user waiting time as the optimal scheduling scheme for the charging pile group.
[0067] In this embodiment, through particle swarm initialization and adaptive inertia weight, the method achieves a balance between global exploration and local development, which can effectively avoid falling into the local optimal solution and improve the probability of finding the global optimal solution; for multiple objectives such as charging efficiency, grid load fluctuation, energy storage battery life and user waiting time, a multi-objective optimization model is established to ensure that each optimization objective is comprehensively balanced and the overall performance of the system is optimized; combined with the hybrid optimization method, the particle swarm optimization is combined with the simulated annealing algorithm, and the local optimality is jumped out through the Metropolis criterion, the exploration ability of the solution is enhanced, and the optimization efficiency is improved; a penalty mechanism based on the degree of constraint violation is introduced, and the penalty intensity is dynamically adjusted to ensure that the constraints such as charging power and energy storage battery charging and discharging are met during the solution process, so that the optimization result is more practical and feasible; non-dominated solutions are managed through an external archive set, and the distribution of the solution set is optimized using the congestion distance operator to ensure the diversity of the solution set, thereby providing multiple optional Pareto optimal solutions and improving decision-making flexibility; the temperature parameters and penalty items of the simulated annealing are gradually adjusted with the number of iterations, and the optimization process is more stable, so that the final charging pile scheduling plan is closer to the actual optimal configuration.
[0068] In an optional embodiment, some non-dominated solutions are randomly selected from the external archive set as initial solutions for simulated annealing, new solutions are generated by perturbing decision variables, and the acceptance probability is calculated using the Metropolis criterion according to the difference in objective function values between the new solution and the current solution. If the acceptance probability is greater than a preset probability threshold, the local optimum is accepted and jumped out, and updating the external archive set includes: Calculate the difference in the objective function value between the new solution and the current solution, use the difference in the objective function value as the energy difference of the Metropolis criterion, use the current temperature parameter of simulated annealing as the temperature parameter of the Metropolis criterion, and calculate the acceptance probability of the new solution through the Boltzmann factor; Compare the acceptance probability with the random probability generated by uniform distribution, if the acceptance probability is greater than the random probability, accept the new solution as the current solution and jump out of the local optimum; otherwise, keep the current solution unchanged; Regardless of whether the new solution is accepted or not, the new solution is compared with the solutions in the external archive set in terms of Pareto dominance. If the new solution is not dominated by any solution in the external archive set, the new solution is added to the external archive set, and the solutions in the external archive set dominated by the new solution are removed, and the external archive set is updated; The temperature parameters of simulated annealing are updated by exponential decay. The initial temperature, decay coefficient and termination temperature are set. After each new solution is accepted, the temperature parameters are calculated according to the current number of iterations. The temperature parameters decrease exponentially with the number of iterations. The process is repeated until the last round of simulated annealing is reached, and the external archive set is used as the output of simulated annealing to determine the Pareto optimal solution set.
[0069] During the simulated annealing process, each new solution needs to be compared with the current solution. First, the difference between the new solution and the current solution in the objective function is calculated. This difference can be understood as the "distance" between the two solutions in performance, which is called the energy difference. This energy difference is used to measure whether the new solution is better or worse than the current solution. The simulated annealing process requires a temperature parameter to control the probability of accepting the new solution. A higher temperature at the beginning indicates a higher probability of accepting a worse solution; as the temperature decreases, the model gradually tends to accept a better solution. For example, the objective function values of the current solution are charging efficiency 80%, grid load fluctuation 10%, energy storage battery life 85%, and user waiting time 5 minutes, while the objective function values of the new solution are charging efficiency 82%, grid load fluctuation 12%, energy storage battery life 83%, and user waiting time 4 minutes. By comparing the objective function values of the two solutions, their difference (i.e., energy difference) is calculated.
[0070] Based on the energy difference and the current temperature parameter, the Boltzmann factor is used to calculate the probability of accepting the new solution. This probability represents the possibility of accepting a worse solution at the current temperature. The higher the temperature, the greater the probability of accepting a worse solution; the lower the temperature, the more the model tends to accept only better solutions. For example, assuming that the current temperature is 100 and the energy difference is -5 (the new solution is better), the probability of accepting the new solution calculated by the Boltzmann factor is 0.95. If the energy difference is +3 (the new solution is worse), the calculated acceptance probability may be 0.35.
[0071] Generate a random probability (a random number between 0 and 1) from a uniform distribution. This random number is used to compare with the acceptance probability of the new solution. If the acceptance probability of the new solution is greater than the generated random probability, the new solution is accepted as the current solution; if the acceptance probability is less than the random probability, the current solution is retained unchanged. This mechanism allows poorer solutions to be accepted with a certain probability in the initial stage, helping to escape the local optimal solution. Exemplarily, the generated random probability is 0.6. If the calculated acceptance probability of the new solution is 0.95 (the new solution is better), then accept the new solution. If the acceptance probability of the new solution is 0.35 (the new solution is worse), reject the new solution and retain the current solution.
[0072] Whether or not to accept the new solution, the new solution needs to be compared with the solutions in the external archive set. The Pareto dominance relationship is used to determine whether the new solution dominates any solution in the external archive set. If the new solution is better than a certain solution in all objectives, the new solution is said to dominate the solution. If the new solution is not dominated by any solution in the external archive, the new solution is added to the external archive set. At the same time, the solutions dominated by the new solution in the external archive set are removed to ensure that the archive set only retains the current Pareto optimal solution. Exemplarily, the existing solutions in the external archive set are: charging efficiency 81%, grid load fluctuation 11%, energy storage battery life 82%, and user waiting time 6 minutes. The new solution has a charging efficiency of 82%, a grid load fluctuation of 12%, an energy storage battery life of 83%, and a user waiting time of 4 minutes. Since the new solution is better than the existing solution in multiple objectives, the new solution dominates the old solution, the new solution is added to the external archive set, and the dominated old solution is removed.
[0073] The temperature parameter of simulated annealing is updated in an exponential decay manner. After each new solution is accepted, the temperature parameter is updated according to the current number of iterations. As the number of iterations increases, the temperature gradually decreases, indicating that the model gradually shifts from a global random search to a local refined search. The initial temperature is usually high, the decay coefficient controls the rate of temperature decrease, and the final temperature is the termination condition. Exemplarily, at the 5th iteration, the current temperature is 100 and the decay coefficient is 0.95. After the update, the temperature becomes 95. As the number of iterations increases, the temperature will gradually decrease.
[0074] The above process (calculating energy difference, accepting new solution, updating external archive set, updating temperature parameter) will be repeated until the maximum number of iterations of simulated annealing is reached. Each round of iteration will further optimize the solution and gradually converge to the Pareto optimal solution set. For example, in the 10th iteration, the temperature has dropped to 50, and simulated annealing is still optimizing the charging pile scheduling plan to find a better solution.
[0075] When the simulated annealing reaches the maximum number of iterations, the iteration stops. The solutions in the external archive set are the Pareto optimal solution set, which represent the best balance between different objectives in the multi-objective optimization problem. Finally, according to the needs, an appropriate scheduling scheme can be selected from the Pareto optimal solution set. For example, after 50 iterations, 5 solutions are retained in the external archive set, which achieve different balances between charging efficiency, grid load fluctuations, energy storage battery life, and user waiting time. Managers can select an optimal scheduling scheme from the Pareto optimal solution set according to actual needs.
[0076] In this embodiment, the Metropolis criterion is used to compare the difference in the objective function value between the new solution and the current solution. Combined with random probability, it is possible to effectively jump out of the local optimum and enhance the global search capability. The introduction of the Pareto dominance relationship ensures that the solutions in the external archive set are always non-dominated solutions, removes inferior solutions, and improves the overall quality and diversity of the solution set. An exponentially decaying temperature update method is adopted to gradually reduce the temperature parameter to ensure that the search process gradually converges from extensive exploration to fine-grained optimization, balance exploration and development, and improve the accuracy of the final solution. Through simulated annealing iterations and continuous updating of the external archive set, the Pareto optimal solution set can be finally determined, providing high-quality candidate solutions for multi-objective optimization. In each iteration, the new solution is compared with the solution of the external archive set and updated to avoid repeated searches for invalid solutions and accelerate the optimization convergence speed.
[0077] S103. Based on the optimal charging pile scheduling scheme, the charging pile group is divided into multiple sub-networks through a distributed collaborative scheduling method based on graph attention network and hierarchical multi-agent reinforcement learning. The charging piles and energy storage battery packs in each sub-network are used as the underlying agents, and the controller of the sub-network is used as the high-level agent. The improved soft actor-critic algorithm is used to train the agents, and the scheduling tasks are dynamically executed through the trained agents.
[0078] In this embodiment, through hierarchical multi-agent reinforcement learning, sub-networks are divided and distributed collaborative scheduling is performed, so that the charging piles and energy storage battery groups in each sub-network can independently and efficiently complete local scheduling, thereby improving the overall scheduling efficiency; the agents can make independent decisions and collaborate according to the dynamic environment of different sub-networks, achieve rapid response and adapt to complex charging demand changes, and enhance the flexibility and robustness of the system; the high-level agents are responsible for the global coordination between sub-networks, while the low-level agents focus on the optimization of local charging piles and energy storage, and ensure the balance between global optimization and local optimization through hierarchical management; the improved soft actor-critic algorithm is used to train the agents to improve their decision-making efficiency and quality in complex environments and optimize the scheduling strategy; the trained agents can dynamically perform scheduling tasks according to real-time data, thereby achieving efficient and real-time scheduling management of charging pile groups.
[0079] In an optional embodiment, based on the optimal charging pile scheduling scheme, the charging pile group is divided into multiple sub-networks through a distributed collaborative scheduling method based on graph attention network and hierarchical multi-agent reinforcement learning. The charging piles and energy storage battery packs in each sub-network are used as the bottom-level agents, and the controller of the sub-network is used as the high-level agent. The improved soft actor-critic algorithm is used to train the agent. Through the trained agent, the scheduling task is dynamically executed, including: According to the geographical distribution and communication topology of charging piles, the charging pile group is divided into multiple sub-networks. Each sub-network contains multiple charging pile nodes, energy storage battery groups and a controller node. The nodes within the sub-network are connected through communication links, and the controller nodes of different sub-networks are connected through communication links, forming a two-layer network topology. Based on the two-layer network topology of the charging pile group hierarchical network topology, a hierarchical multi-agent framework is constructed. The charging pile nodes and energy storage battery groups in the sub-network are modeled as bottom-level agents, which are responsible for performing charging and discharging scheduling tasks. The controller nodes of the sub-network are modeled as high-level agents, which are responsible for coordinating and managing the collaborative operations of the bottom-level agents. The graph attention network is embedded into the hierarchical multi-agent framework. For the bottom-level agent, the graph attention layer is used to process its own state and neighbor state, learn the importance weight of the neighbor state to its own decision, aggregate the neighbor information to generate a node embedding vector, and map it to the action space to generate a scheduling strategy; for the high-level agent, the graph attention layer is used to process the state of all bottom-level agents, learn the importance weight of different bottom-level agents to the scheduling decision of the sub-network, aggregate the bottom-level information to generate a controller embedding vector, and map it to the action space to generate a scheduling instruction; An improved soft actor-critic algorithm is used to centrally train each agent in the hierarchical multi-agent framework. A parallel actor network and critic network are introduced, corresponding to agents at different levels. When the critic network estimates the state-action value function, the observations and actions of agents at all levels are concatenated. When the actor network calculates the policy gradient, the policy gradients of agents at all levels are weighted summed. Based on the trained actor network, the hierarchical multi-agent framework is executed online in a distributed manner. The bottom-level agent generates a scheduling strategy based on the local observation status through the actor network embedded in the graph attention network, executes the assigned scheduling tasks, and uploads the status information to the high-level agent; the high-level agent generates a scheduling instruction based on the status of all bottom-level agents through the actor network embedded in the graph attention network, and sends it to the corresponding bottom-level agent to complete the dynamic execution of the scheduling task.
[0080] First, according to the geographical distribution and communication topology of the charging piles, the entire charging pile group is divided into multiple sub-networks. Each sub-network contains multiple charging pile nodes, energy storage battery packs and a controller node. The charging pile nodes within the sub-network are interconnected through communication links to form a local network, and the controller nodes of different sub-networks are connected through communication links to form a two-layer network topology. The first layer is the connection between the charging pile nodes and the energy storage battery pack within the sub-network, which is responsible for local charging and discharging scheduling. The second layer is the connection between the controller nodes of each sub-network, which is responsible for coordination and management across sub-networks. For example, assume that there are 20 charging piles distributed in different areas of the city. According to the geographical location and communication capabilities, these charging piles are divided into 4 sub-networks, each of which contains 5 charging piles and 1 energy storage battery pack. Each sub-network has a controller node responsible for communicating with the controller nodes of other sub-networks.
[0081] In the hierarchical multi-agent framework, the charging pile nodes and energy storage battery groups within each sub-network are modeled as bottom-level agents. The bottom-level agent is responsible for performing local charging and discharging scheduling tasks to ensure the balance of local power demand and supply. The controller node of the sub-network is modeled as a high-level agent. The responsibility of the high-level agent is to coordinate the collaborative operations between the bottom-level agents within the sub-network and to exchange information with the controller nodes of other sub-networks to ensure global coordination. For example, in the previous division, the five charging pile nodes and energy storage battery groups in each sub-network are respectively used as bottom-level agents, and the controller node of the sub-network is used as a high-level agent. The bottom-level agent is responsible for local power scheduling, such as charging task scheduling of charging piles and charging and discharging operations of energy storage batteries; the high-level agent is responsible for managing the scheduling of the entire sub-network to ensure task coordination of each bottom-level node.
[0082] Each underlying agent uses the graph attention layer to process its own state and the state of neighboring nodes. By learning the importance weights of neighbor states to their own decisions, the underlying agent can aggregate neighbor information, generate node embedding vectors, map them to the action space, and generate corresponding local scheduling strategies. High-level agents also use the graph attention layer to process the state information of the underlying agents. By learning the importance weights of different underlying agents to the decisions of the entire sub-network, the high-level agent aggregates the information of each underlying agent, generates a controller embedding vector, and maps it to the action space to generate the scheduling instructions of the sub-network. For example, in a sub-network, the state of charging pile A has a great influence on the scheduling decision of charging pile B. The graph attention network learns this weight, and charging pile B adjusts its scheduling strategy according to the state of charging pile A. Similarly, the high-level controller of the sub-network generates the scheduling instructions for the entire sub-network by aggregating the state information of all charging piles.
[0083] In order to centrally train each agent in the hierarchical multi-agent framework, an improved soft actor-critic algorithm is used. Each agent has a parallel actor network and critic network, where the actor network generates the agent's scheduling strategy, that is, generates scheduling actions from the embedding vector; the critic network estimates the state-action value function to measure the quality of the current state and action. In the critic network, the observations and actions of all levels of agents (bottom and top) are spliced together to uniformly estimate the overall state-action value function. This ensures that the states and actions of all agents are taken into account during training. In the actor network, the policy gradient of each agent is calculated and weighted summed according to the policy gradients of all levels of agents to ensure that the agents of the entire system collaboratively optimize the scheduling strategy. Exemplarily, for a subnetwork containing 5 charging piles, the high-level agent controls the scheduling of the entire subnetwork, and the bottom-level agent is responsible for the scheduling of their own charging piles and energy storage batteries. Through the soft actor-critic algorithm, each agent (both top and bottom) has its own actor network and critic network. During training, the actions of the underlying agents (such as charging power allocation) and the instructions of the higher-level agents (such as sub-network scheduling strategies) work together to optimize the overall scheduling strategy.
[0084] After training is completed, the bottom-level agent generates a scheduling strategy based on the local observation state through the actor network of the graph attention network and performs distributed scheduling tasks. For example, the charging pile node will decide when to start charging or discharging based on the current power demand and battery status. After executing the scheduling task, the bottom-level agent will upload the status information to the high-level agent. The high-level agent generates the scheduling instructions of the sub-network through the actor network of the graph attention network based on the status information uploaded by all the bottom-level agents. These instructions will be issued to the corresponding bottom-level agents to ensure that the scheduling tasks within the sub-network are carried out in coordination. For example, in actual execution, the bottom-level agent (charging pile node) generates corresponding charging strategies (such as increasing or reducing charging power) based on the current grid load and vehicle demand. These strategies will be uploaded to the high-level agent, and the high-level agent will generate global scheduling instructions (such as adjusting the charging and discharging strategy of the energy storage battery) based on the overall load situation and the energy storage battery status to ensure that the entire sub-network operates efficiently under load balancing.
[0085] During the online execution of the entire hierarchical multi-agent system, the scheduling strategy is continuously updated through the interaction between the bottom-level agent and the high-level agent. The task status executed by the bottom level will be fed back to the high-level agent in real time, and the high-level agent will make corresponding adjustments based on the global status to ensure the dynamic execution and global optimization of the scheduling task. For example, assuming that the energy storage battery in the sub-network is charged to the maximum capacity at a certain moment, the bottom-level agent will feed back this status to the high-level agent. The high-level agent will adjust the scheduling instructions, stop charging the energy storage battery, and redistribute power to other charging piles to optimize the power scheduling of the entire sub-network.
[0086] In this embodiment, the charging pile group is divided into multiple sub-networks to form a two-layer network topology. Through the division of labor and cooperation between the bottom and high-level agents, distributed efficient scheduling is achieved, and the overall collaborative work efficiency of the system is improved. By embedding the graph attention network, each agent can dynamically adjust the weight according to the state of the neighbors, aggregate information, optimize the scheduling decision, and improve the system's adaptability to complex environments. The high-level agent is responsible for the global coordination between the sub-networks, and the bottom-level agent focuses on local scheduling. Through the hierarchical multi-agent architecture, local and global collaborative optimization is achieved to ensure the efficient completion of the scheduling task. The improved soft actor-critic algorithm is introduced, combined with parallel actor and critic networks, and agents at different levels are centrally trained, which improves the strategy optimization and decision-making accuracy of the agent. After the training is completed, the agent can generate scheduling strategies and instructions online in real time, dynamically adapt to charging needs and energy storage battery status, and realize intelligent and flexible scheduling execution. Through the hierarchical graph attention network and multi-agent framework, the scheduling performance of the charging pile is improved, the grid load is balanced, and the life of the energy storage battery is extended. At the same time, the user waiting time is reduced and the overall resource utilization is optimized.
[0087] In an optional embodiment, an improved soft actor-critic algorithm is used to centrally train each agent in the hierarchical multi-agent framework, and a parallel actor network and critic network are introduced, corresponding to agents at different levels, respectively. When the critic network estimates the state-action value function, the observations and actions of agents at all levels are concatenated; when the actor network calculates the policy gradient, the weighted summation of the policy gradients of agents at all levels includes: Introduce multiple parallel actor networks and critic networks, where the number of actor networks and critic networks corresponds to the number of low-level agents and high-level agents, respectively; When the critic network estimates the state-action value function, the observations and actions of the bottom-level agent are concatenated with those of the high-level agent to form a state-action pair containing complete information in the entire hierarchical multi-agent framework as the critic network input. Based on the critic network input, the state-action value function of the entire hierarchical multi-agent framework is evaluated. When the actor network calculates the policy gradient, according to the importance of the bottom-level agents and the high-level agents in the entire hierarchical multi-agent framework, the corresponding weight coefficients are set, and the policy gradients of all bottom-level agents and the policy gradients of all high-level agents are weighted summed as the policy gradient of the entire hierarchical multi-agent framework to update the actor network parameters; Hierarchical exploration noise is introduced. For the actor network of the bottom-level intelligent agent, Gaussian noise with a variance less than the noise variance threshold is added when the action is output. For the actor network of the high-level intelligent agent, Gaussian noise with a variance greater than the noise variance threshold is added when the action is output.
[0088] In the hierarchical multi-agent framework, each bottom-level agent and high-level agent corresponds to an independent actor network and critic network, respectively. That is, the number of bottom-level agents determines the number of bottom-level actor networks and critic networks, and the number of high-level agents determines the number of high-level actor networks and critic networks. The actor network of each agent is responsible for generating the scheduling strategy of the agent, while the critic network is responsible for evaluating the state-action value of the agent. Through this parallel mechanism, each agent can independently generate actions and evaluate state-action pairs, but during training, their strategies and value estimates are interrelated. For example, in a charging pile scheduling system, assume that there are 3 subnetworks, each of which contains 5 charging piles (bottom-level agents) and 1 controller (top-level agent). Therefore, there will be 3 high-level agents corresponding to 3 high-level actor networks and critic networks in the system, and 15 bottom-level agents corresponding to 15 bottom-level actor networks and critic networks.
[0089] When the critic network estimates the state-action value function, the observations and actions of the bottom-level agent and the high-level agent need to be spliced. Specifically, the observed state (such as the power of the charging pile, load, etc.) and action (such as charging power allocation) of the bottom-level agent are combined with the observed state (such as the overall power demand of the sub-network, the state of the energy storage battery, etc.) and action (such as the scheduling instruction of the sub-network) of the high-level agent to form a complete state-action pair. The input of the critic network is the spliced state-action pair, which contains the complete information in the entire hierarchical multi-agent framework. Through this input, the critic network evaluates the state-action value function of the entire system and measures the quality of the current strategy. For example, assume that the observed state of the bottom-level agent is the current power of the charging pile (for example, 80%), and the action of the bottom-level is the charging power allocation (such as 5 kilowatts). The observed state of the high-level agent is the power demand of the entire sub-network (such as the sub-network needs more power), and the action of the high-level is the discharge instruction of the energy storage battery (such as 30% discharge capacity). After splicing, this information is input into the critic network to evaluate the quality of the scheduling strategy of the entire sub-network.
[0090] In the actor network, each agent (whether it is the bottom or the top) will calculate its own policy gradient, which indicates the direction of improvement of the current policy. In order to ensure the coordination of the entire system, it is necessary to weight the policy gradients of all agents and assign weights according to their importance to the entire system. Different weight coefficients are set according to the importance of the bottom agent and the top agent in the entire framework. Since the top agent is usually responsible for global scheduling and the bottom agent performs specific tasks, the weight of the top agent is usually larger than that of the bottom agent. By weighted summing up the policy gradients of all agents, the overall policy gradient of the entire system is obtained, and the parameters of the actor network are updated accordingly. For example, in a subnetwork, the policy gradient of the charging pile (bottom agent) is 0.3, and the policy gradient of the subnetwork controller (top agent) is 0.7. Since the high-level agent has a greater impact on global scheduling, the weight is 0.6, and the weight of the bottom agent is 0.4. After weighted summation, the policy gradient of the entire system is 0.3×0.4 + 0.7×0.6 = 0.54. The parameters of the actor network are updated accordingly.
[0091] In order to enhance the agent's exploration ability and avoid falling into the local optimal solution, exploration noise is added to the action output of each agent during the training process. The role of noise is to randomly perturb the agent's actions within a certain range, thereby encouraging the agent to explore different scheduling strategies.
[0092] Hierarchical noise design: Add Gaussian noise with small variance to the actor network of the bottom-level agent. This is because the bottom-level agent is responsible for specific scheduling tasks, and the disturbance of the action should not be too large to avoid affecting the local scheduling accuracy; add Gaussian noise with large variance to the actor network of the high-level agent. The action of the high-level agent affects the scheduling of the entire sub-network, so a larger exploration space is needed to enable it to find the global optimal scheduling strategy.
[0093] For example, suppose that in the charging pile scheduling of the bottom-level agent, the current action is to set the charging power to 5 kW. To increase exploration, adding a Gaussian noise with a variance of 0.1 may result in a new charging power, such as 5.05 kW. For the scheduling instructions of the high-level agent, suppose that the current strategy is to set the discharge capacity of the energy storage battery to 30%. Since the high-level agent needs a larger exploration space, adding a Gaussian noise with a variance of 0.5 may cause the discharge capacity to be adjusted to 32%.
[0094] During the entire training process, the observed states and actions of the bottom-level and high-level agents are spliced and input into their respective critic networks for state-action value evaluation. Then, agents at each level generate scheduling strategies through the actor network and calculate policy gradients. Weights are assigned according to the importance of the agents, and the policy gradients of all agents are weighted summed to update the parameters of the actor network. At the same time, the hierarchical exploration noise ensures the exploration ability of the agent in the scheduling strategy and helps the system to jump out of the local optimum. For example, in a charging pile scheduling system with 3 subnetworks, each subnetwork contains 5 charging piles and 1 energy storage battery pack. Each charging pile and energy storage battery pack is used as a bottom-level agent, and the subnetwork controller is used as a high-level agent. During the training process, the state and action information of the bottom-level and high-level agents are spliced and input into the corresponding critic network for evaluation. For example, the charging power scheduling information of the bottom-level agent and the energy storage battery discharge instruction of the high-level agent will be evaluated by the critic network together to determine the quality of the current scheduling strategy. Based on the weighted summation of the policy gradient, the actor network generates a new scheduling strategy. In addition, exploration noise helps the agent to conduct appropriate exploration in local adjustments, encouraging the search for better global scheduling strategies.
[0095] In this embodiment, by splicing the observations and actions of the bottom-level agents and the high-level agents, the critic network can obtain complete information of the entire hierarchical multi-agent framework, thereby more accurately evaluating the state-action value function and improving the quality of decision-making; the weighted sum policy gradient method is adopted to optimize the overall policy update according to the importance of different agents in the framework, ensure that the contribution of agents at all levels is reasonably reflected, and improve the scheduling effect; Gaussian noise with different variances is introduced, and the bottom-level agent uses small variance noise to ensure the stability of the action, while the high-level agent uses large variance noise to promote exploration, which enhances the flexibility and adaptability of the agent in the decision-making process; through parallel actor and critic networks, multiple agents can be trained at the same time, accelerating the learning process and improving the training efficiency and responsiveness of the system; the hierarchical exploration noise design enhances the robustness of the system in the face of uncertainty and complex environments, allowing the agent to better adapt to changes and randomness, and ensure the stability and reliability of scheduling.
[0096] Figure 2 FIG. 1 is a schematic diagram of the structure of an intelligent charging pile scheduling system based on dynamic adjustment of energy storage battery packs according to an embodiment of the present invention. Figure 2 As shown, the system comprises: Provided is an intelligent charging pile scheduling system based on dynamic adjustment of energy storage battery packs, comprising: The first unit is used to collect multi-dimensional real-time data of the charging pile group, and obtain the status information of the energy storage battery group to obtain basic data. Combined with the historical load curve and weather forecast information, it predicts the future through graph attention and adaptive spatiotemporal pooling mechanism to obtain the prediction results; The second unit is used to determine the objective function including maximum charging efficiency, most balanced grid load, longest cycle life of energy storage battery, and minimum user waiting time based on basic data and prediction results, build a multi-objective optimization model, adopt a hybrid optimization method based on multi-objective particle swarm algorithm and simulated annealing algorithm, introduce adaptive inertia weight and dynamic constraint processing mechanism, and generate the optimal charging pile scheduling plan through iterative optimization; The third unit is used to divide the charging pile group into multiple sub-networks based on the optimal charging pile scheduling plan through a distributed collaborative scheduling method based on graph attention network and hierarchical multi-agent reinforcement learning. The charging piles and energy storage battery groups in each sub-network are used as the underlying agents, and the controller of the sub-network is used as the high-level agent. The improved soft actor-critic algorithm is used to train the agents, and the scheduling tasks are dynamically executed through the trained agents.
[0097] According to a third aspect of the embodiments of the present invention, An electronic device is provided, comprising: processor; a memory for storing processor-executable instructions; The processor is configured to call the instructions stored in the memory to execute the aforementioned method.
[0098] A fourth aspect of the embodiments of the present invention is: A computer-readable storage medium is provided, on which computer program instructions are stored. When the computer program instructions are executed by a processor, the aforementioned method is implemented.
[0099] The present invention may be a method, an apparatus, a system and / or a computer program product. The computer program product may include a computer-readable storage medium carrying computer-readable program instructions for executing various aspects of the present invention.
[0100] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit it. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or replace some or all of the technical features therein with equivalents. However, these modifications or replacements do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments of the present invention.
Claims
1. An intelligent charging pile scheduling method based on dynamic adjustment of energy storage battery packs, characterized in that: include: Collect multi-dimensional real-time data of the charging pile group, and obtain the status information of the energy storage battery group at the same time to obtain basic data. Combined with historical load curves and weather forecast information, the future is predicted through graph attention and adaptive spatiotemporal pooling mechanism to obtain prediction results. Based on basic data and prediction results, determine the objective function including maximum charging efficiency, most balanced grid load, longest cycle life of energy storage battery, and minimum user waiting time, build a multi-objective optimization model, adopt a hybrid optimization method based on multi-objective particle swarm algorithm and simulated annealing algorithm, introduce adaptive inertia weight and dynamic constraint processing mechanism, and generate the optimal charging pile scheduling plan through iterative optimization; Based on the optimal charging pile scheduling scheme, the charging pile group is divided into multiple sub-networks through a distributed collaborative scheduling method based on graph attention network and hierarchical multi-agent reinforcement learning. The charging piles and energy storage battery packs in each sub-network are used as the underlying agents, and the controller of the sub-network is used as the high-level agent. The improved soft actor-critic algorithm is used to train the agents, and the scheduling tasks are dynamically executed through the trained agents.
2. The method according to claim 1, characterized in that Collect multi-dimensional real-time data of the charging pile group, obtain the status information of the energy storage battery group, obtain basic data, combine historical load curves and weather forecast information, and use graph attention and adaptive spatiotemporal pooling mechanisms to predict the future. The prediction results include: Each charging pile node is represented as a capsule. Based on the activation degree of the charging pile node, the vector length of the vector inside the capsule is determined. Based on the feature change trend of the charging pile node, the vector direction of the vector inside the capsule is determined. Based on the vector length and vector direction, the correlation between capsules is calculated through the attention mechanism, and the neighborhood weight of the capsule is dynamically updated. The dynamic routing algorithm is used to realize information transmission and aggregation between capsules. Capsules with neighborhood weights greater than the preset threshold are formed into high-level capsules, and the hierarchical feature representation of the charging pile group is extracted. The graph attention capsule layer is determined, and multiple graph attention capsule layers are stacked to construct a graph attention capsule network to gradually extract high-order spatiotemporal features. Based on the high-order spatiotemporal features, combined with the attention gating unit, by learning the attention weights on different time scales, the pooling window size of the time dimension is adaptively adjusted to determine the adaptive spatiotemporal pooling mechanism. When the data fluctuation is within the preset short-term time span and is greater than the preset fluctuation threshold, the small-scale pooling window is activated to capture the fine-grained short-term features; when the data fluctuation is outside the preset long-term time span and is less than the preset stable threshold, the large-scale pooling window is activated to extract the long-term trend features; based on the fine-grained short-term features and the long-term trend features, the multi-scale features are determined; A residual self-attention module is introduced between the graph attention capsule network and the adaptive spatiotemporal pooling to model the long-range dependency between corresponding multi-scale features at different spatiotemporal positions through a multi-head self-attention mechanism; Based on long-range dependencies, the external environment characteristics determined based on weather forecast information are integrated to characterize the spatiotemporal patterns and correlation structures of the charging pile group and generate prediction results.
3. The method according to claim 2, characterized in that The method further comprises: Build a multi-task learning framework, determine a subtask based on the prediction task of each charging pile, and all subtasks share the same model structure, including graph attention capsule network, adaptive spatiotemporal pooling mechanism and residual self-attention module, and output the prediction results for multiple time steps in the future; A feature sharing mechanism is adopted to share the hidden layer features extracted by the graph attention capsule network, the adaptive spatiotemporal pooling mechanism, and the residual self-attention module among all subtasks. The input of each subtask passes through the shared feature extraction module to obtain the common hidden layer features, which are mapped through the fully connected layer corresponding to the subtask to generate the prediction output; During model training, based on the prediction output of each subtask obtained in the previous step, the prediction sequence is determined, and a consistency regularization term is introduced. By calculating the normalized Euclidean distance between any two subtask prediction sequences, the prediction sequences generated by different subtasks are made to have the same trend. The consistency regularization term is weighted and summed with the prediction loss of all subtasks to obtain the total loss function. Through the joint training strategy, the prediction loss and consistency regularization term of all subtasks are optimized at the same time. In each training batch, data containing all charging pile input samples is randomly sampled. The prediction output, corresponding prediction loss and consistency regularization term of each subtask are calculated, and the model parameters are updated using the optimization algorithm. The multi-task learning framework constructed by the application shares hidden layer features and introduces a consistency regularization term based on the multi-task prediction output calculation to construct the total loss function. Through joint training, the guidance of consistency constraints is combined when sharing information between different sub-tasks.
4. The method according to claim 1, characterized in that: Based on the basic data and prediction results, the objective function including maximum charging efficiency, most balanced grid load, longest cycle life of energy storage battery and minimum user waiting time is determined, and a multi-objective optimization model is constructed. A hybrid optimization method based on multi-objective particle swarm algorithm and simulated annealing algorithm is adopted, and adaptive inertia weight and dynamic constraint processing mechanism are introduced. Through iterative optimization, the optimal charging pile scheduling plan is generated, including: Based on the prediction results, maximizing charging efficiency, minimizing grid load fluctuations, maximizing the cycle life of energy storage batteries, and minimizing user waiting time are taken as optimization goals, and constraints including charging power constraints, energy storage battery charging and discharging constraints, and user demand constraints are established to construct a multi-objective optimization model; The multi-objective optimization model is solved by a hybrid optimization method, including: Initialize the particle swarm, randomly generate the initial position and velocity of each particle, use the particle position vector as the decision variable of the multi-objective optimization model, use the particle velocity vector as the change of the decision variable, initialize the external archive set for storing non-dominated solutions, and initialize the adaptive inertia weight for balancing global exploration and local exploitation; For each particle, the corresponding position vector is substituted into the multi-objective optimization model, and the values of each objective function are calculated. The personal optimal position of the particle is compared and updated through the Pareto dominance relationship between particles. The global optimal position is compared and updated through the Pareto dominance relationship of the solutions in the external archive set, and the current non-dominated solution is added to the external archive set. When the external archive set reaches the maximum capacity, the crowding distance operator is used to calculate the crowding distance, sort the non-dominated solutions, and remove the solutions with crowding distance less than the preset threshold. Based on the individual optimal position, the global optimal position and the adaptive inertia weight of the particle, the velocity vector and the position vector of the particle are updated through the velocity and position update equations, and the adaptive inertia weight decreases as the number of iterations increases, thereby transferring the global exploration to the local development; Randomly select some non-dominated solutions from the external archive set as the initial solutions of simulated annealing, generate new solutions by perturbing the decision variables, calculate the acceptance probability according to the difference between the objective function value of the new solution and the current solution by the Metropolis criterion, and if the acceptance probability is greater than the preset probability threshold, accept and jump out of the local optimum, and update the external archive set, wherein the temperature parameter of simulated annealing is based on an exponential decay method, and the probability threshold is gradually reduced as the number of iterations decreases; For particles that violate the constraints during the iteration process, the violation degree index is determined, and the objective function value is penalized based on the violation degree index. The penalty term is determined in proportion to the violation degree index, and the penalty intensity corresponding to the penalty term is gradually increased as the number of iterations increases. Until the preset upper limit of iterations is reached, the iteration process is completed; An optimal charging pile scheduling solution is selected and determined from the Pareto optimal solution set of the external archive set.
5. The method according to claim 4, characterized in that Randomly select some non-dominated solutions from the external archive set as the initial solutions of simulated annealing, generate new solutions by perturbing the decision variables, calculate the acceptance probability according to the difference between the objective function value of the new solution and the current solution by the Metropolis criterion, and if the acceptance probability is greater than a preset probability threshold, accept and jump out of the local optimum, and update the external archive set including: Calculate the difference in the objective function value between the new solution and the current solution, use the difference in the objective function value as the energy difference of the Metropolis criterion, use the current temperature parameter of simulated annealing as the temperature parameter of the Metropolis criterion, and calculate the acceptance probability of the new solution through the Boltzmann factor; Compare the acceptance probability with the random probability generated by uniform distribution, if the acceptance probability is greater than the random probability, accept the new solution as the current solution and jump out of the local optimum; otherwise, keep the current solution unchanged; Regardless of whether the new solution is accepted or not, the new solution is compared with the solutions in the external archive set in terms of Pareto dominance. If the new solution is not dominated by any solution in the external archive set, the new solution is added to the external archive set, and the solutions in the external archive set dominated by the new solution are removed, and the external archive set is updated; The temperature parameters of simulated annealing are updated by exponential decay. The initial temperature, decay coefficient and termination temperature are set. After each new solution is accepted, the temperature parameters are calculated according to the current number of iterations. The temperature parameters decrease exponentially with the number of iterations. The process is repeated until the last round of simulated annealing is reached, and the external archive set is used as the output of simulated annealing to determine the Pareto optimal solution set.
6. The method according to claim 1, characterized in that Based on the optimal charging pile scheduling scheme, the charging pile group is divided into multiple sub-networks through a distributed collaborative scheduling method based on graph attention network and hierarchical multi-agent reinforcement learning. The charging piles and energy storage battery packs in each sub-network are used as the underlying agents, and the controller of the sub-network is used as the high-level agent. The improved soft actor-critic algorithm is used to train the agents. Through the trained agents, the scheduling tasks are dynamically executed, including: According to the geographical distribution and communication topology of charging piles, the charging pile group is divided into multiple sub-networks. Each sub-network contains multiple charging pile nodes, energy storage battery groups and a controller node. The nodes within the sub-network are connected through communication links, and the controller nodes of different sub-networks are connected through communication links, forming a two-layer network topology. Based on the two-layer network topology of the charging pile group hierarchical network topology, a hierarchical multi-agent framework is constructed. The charging pile nodes and energy storage battery groups in the sub-network are modeled as bottom-level agents, which are responsible for performing charging and discharging scheduling tasks. The controller nodes of the sub-network are modeled as high-level agents, which are responsible for coordinating and managing the collaborative operations of the bottom-level agents. The graph attention network is embedded into the hierarchical multi-agent framework. For the bottom-level agent, the graph attention layer is used to process its own state and neighbor state, learn the importance weight of the neighbor state to its own decision, aggregate the neighbor information to generate a node embedding vector, and map it to the action space to generate a scheduling strategy; for the high-level agent, the graph attention layer is used to process the state of all bottom-level agents, learn the importance weight of different bottom-level agents to the scheduling decision of the sub-network, aggregate the bottom-level information to generate a controller embedding vector, and map it to the action space to generate a scheduling instruction; An improved soft actor-critic algorithm is used to centrally train each agent in the hierarchical multi-agent framework. A parallel actor network and critic network are introduced, corresponding to agents at different levels. When the critic network estimates the state-action value function, the observations and actions of agents at all levels are concatenated. When the actor network calculates the policy gradient, the policy gradients of agents at all levels are weighted summed. Based on the trained actor network, the hierarchical multi-agent framework is executed online in a distributed manner. The bottom-level agent generates a scheduling strategy based on the local observation status through the actor network embedded in the graph attention network, executes the assigned scheduling tasks, and uploads the status information to the high-level agent; the high-level agent generates a scheduling instruction based on the status of all bottom-level agents through the actor network embedded in the graph attention network, and sends it to the corresponding bottom-level agent to complete the dynamic execution of the scheduling task.
7. The method according to claim 6, characterized in that An improved soft actor-critic algorithm is used to centrally train each agent in the hierarchical multi-agent framework, and a parallel actor network and critic network are introduced, corresponding to agents at different levels respectively. When the critic network estimates the state-action value function, the observations and actions of agents at all levels are concatenated. When calculating the policy gradient in the actor network, the weighted sum of the policy gradients of all layers of the agent includes: Introduce multiple parallel actor networks and critic networks, where the number of actor networks and critic networks corresponds to the number of low-level agents and high-level agents, respectively; When the critic network estimates the state-action value function, the observations and actions of the bottom-level agent are concatenated with those of the high-level agent to form a state-action pair containing complete information in the entire hierarchical multi-agent framework as the critic network input. Based on the critic network input, the state-action value function of the entire hierarchical multi-agent framework is evaluated. When the actor network calculates the policy gradient, according to the importance of the bottom-level agents and the high-level agents in the entire hierarchical multi-agent framework, the corresponding weight coefficients are set, and the policy gradients of all bottom-level agents and the policy gradients of all high-level agents are weighted summed as the policy gradient of the entire hierarchical multi-agent framework to update the actor network parameters; Hierarchical exploration noise is introduced. For the actor network of the bottom-level intelligent agent, Gaussian noise with a variance less than the noise variance threshold is added when the action is output. For the actor network of the high-level intelligent agent, Gaussian noise with a variance greater than the noise variance threshold is added when the action is output.
8. An intelligent charging pile scheduling system based on dynamic adjustment of energy storage battery packs, used to implement the method described in any one of claims 1 to 7, characterized in that: include: The first unit is used to collect multi-dimensional real-time data of the charging pile group, and obtain the status information of the energy storage battery group to obtain basic data. Combined with the historical load curve and weather forecast information, it predicts the future through graph attention and adaptive spatiotemporal pooling mechanism to obtain the prediction results; The second unit is used to determine the objective function including maximum charging efficiency, most balanced grid load, longest cycle life of energy storage battery, and minimum user waiting time based on basic data and prediction results, build a multi-objective optimization model, adopt a hybrid optimization method based on multi-objective particle swarm algorithm and simulated annealing algorithm, introduce adaptive inertia weight and dynamic constraint processing mechanism, and generate the optimal charging pile scheduling plan through iterative optimization; The third unit is used to divide the charging pile group into multiple sub-networks based on the optimal charging pile scheduling plan through a distributed collaborative scheduling method based on graph attention network and hierarchical multi-agent reinforcement learning. The charging piles and energy storage battery groups in each sub-network are used as the underlying agents, and the controller of the sub-network is used as the high-level agent. The improved soft actor-critic algorithm is used to train the agents, and the scheduling tasks are dynamically executed through the trained agents.
9. An electronic device, characterized in that: include: processor; a memory for storing processor-executable instructions; The processor is configured to call the instructions stored in the memory to execute the method described in any one of claims 1 to 7.
10. A computer-readable storage medium having computer program instructions stored thereon, characterized in that: When the computer program instructions are executed by a processor, the method according to any one of claims 1 to 7 is implemented.
Citation Information
Patent Citations
Multi-charging-mode charging pile control method considering matching of electric vehicles
CN112277711A
Multi-agent reinforcement learning method and system based on dynamic hierarchical communication network
CN113919485A
Power grid line loss reduction optimization method and system for new energy access power distribution network
CN116757877A
Electric vehicle charging station group load prediction method and device based on multi-graph fusion
CN116885698A
Regional air quality prediction model based on multi-scale dynamic synchronous graph mechanism
CN117371571A
Cited By
Method and system for intelligently regulating and controlling actions of photovoltaic energy storage battery pack
CN119382299A
Energy storage networking control method, apparatus and device, and storage medium
CN119448358A
New energy vehicle charging pile safety performance life prediction method
CN119538604A
Battery charging and discharging method, system and equipment based on MCU (Microprogrammed Control Unit) and storage medium
CN119813479A
Charging pile charging control method and system, electronic equipment and storage medium
CN119911149A
