A collaborative scheduling method for new energy vehicle charging infrastructure and edge computing gateway energy storage

Abstract

The present invention discloses a method for collaborative scheduling of new energy vehicle charging infrastructure and edge computing gateway energy storage. The method constructs a state space and action space containing various parameters of the charging infrastructure and edge computing gateway energy storage system, uses a multi-head attention mechanism model to extract parameter features and analyze correlations, and then uses the policy network in the deep reinforcement learning algorithm to output the probability distribution of scheduling actions to select collaborative scheduling actions. After executing the action, an immediate reward including operating cost, charging and discharging benefits, and change in charging waiting time is obtained, and the state is updated and experience is stored for iterative training of the policy network and value network. The method also optimizes the training process through priority experience replay, gating mechanism, etc., comprehensively considers equipment loss costs, etc. to improve reward calculation, and finally obtains a stable collaborative scheduling strategy to achieve efficient collaboration between new energy vehicle charging infrastructure and edge computing gateway energy storage system, thereby improving overall resource scheduling efficiency and system operation performance.

Description

Technical Field

[0001] The present invention relates to the field of charging infrastructure and edge computing gateway energy storage supervision, and in particular to a method for collaborative scheduling of new energy vehicle charging infrastructure and edge computing gateway energy storage. Background Art

[0002] In recent years, the global new energy vehicle industry has experienced rapid growth, with the market share continuing to climb. The construction of supporting charging infrastructure has been accelerating. However, due to the random and volatile nature of new energy vehicle charging demand, traditional charging infrastructure often suffers from inefficiencies in the allocation of power resources. At the same time, the integration of edge computing gateway technology with energy storage systems has opened up new possibilities for the efficient management of power resources. The decentralized and tamper-proof nature of edge computing gateways enables secure and reliable interaction and collaboration between nodes in energy storage systems. Therefore, research on collaborative scheduling methods for charging infrastructure and edge computing gateway energy storage has important practical significance and application value.

[0003] Currently, most new energy vehicle charging infrastructure and energy storage system scheduling methods are relatively traditional, relying on manual experience or simple rule-based settings, making them difficult to adapt to the complex and changing charging demands and energy storage status. Existing technologies do not adequately explore the complex relationships between various parameters when coordinating the scheduling of charging infrastructure and edge computing gateway energy storage. Parameters such as the real-time power and demand quantity of the charging infrastructure, and the storage capacity and charge / discharge power of the edge computing gateway energy storage system all influence each other. Traditional methods often view these parameters in isolation, failing to fully utilize the inherent connections between them. This results in scheduling strategies that cannot maximize the overall benefits of the system, and presents significant deficiencies in operating cost control and revenue improvement. Summary of the Invention

[0004] In order to overcome the shortcomings and deficiencies of the existing technology, the present invention provides a method for collaborative scheduling of new energy vehicle charging infrastructure and edge computing gateway energy storage.

[0005] The technical solution adopted by the present invention is a method for collaboratively scheduling new energy vehicle charging infrastructure and edge computing gateway energy storage, comprising the following steps:

[0006] Step S1: Construct a state space including the real-time charging power of each charging pile in the new energy vehicle charging infrastructure, the number of charging demands, the storage capacity of each energy storage node in the edge computing gateway energy storage system, and the charging and discharging power. Based on the deep reinforcement learning algorithm, define an action space including the probability of scheduling action selection. The scheduling action in the action space includes the charging power allocation of each charging pile in the charging infrastructure and the charging and discharging decision of each energy storage node in the edge computing gateway energy storage system.

[0007] Step S2: Utilize a multi-head attention mechanism model to extract features and perform correlation analysis on various parameters of the coordinated dispatch of charging infrastructure and edge computing gateway energy storage in the state space. Through multi-head parallel computing, the dependencies between different parameters are captured to form a feature vector set containing multi-dimensional feature information.

[0008] Step S3: Based on the policy network in the deep reinforcement learning algorithm, the feature vector set is used as input, and the probability distribution of the scheduling action in the action space is output. The coordinated scheduling action for the charging infrastructure and the edge computing gateway energy storage system is selected based on the probability distribution;

[0009] Step S4: Execute the selected collaborative scheduling action to obtain instant rewards including the operating cost of charging infrastructure, the charging and discharging income of the edge computing gateway energy storage system, and the change in the charging waiting time of new energy vehicle users, and update the parameters in the state space;

[0010] Step S5: Store the state space, the selected scheduling action, the immediate reward, and the updated state space into the experience replay pool. Randomly extract sample data from the experience replay pool based on the deep reinforcement learning algorithm to construct a training dataset.

[0011] Step S6: Use the training data set to iteratively train and optimize the policy network and the value network in the deep reinforcement learning algorithm, return to step S3 and repeat the execution until the preset training termination condition is met, and obtain a stable new energy vehicle charging infrastructure and edge computing gateway energy storage coordinated scheduling strategy.

[0012] Furthermore, the calculation process of the multi-head attention mechanism model used in step S2 satisfies the following formula:

[0013]

[0014] in, A query matrix representing the coordinated scheduling parameters of charging infrastructure and edge computing gateway energy storage in the state space, including the real-time charging power matrix of charging piles and the storage power matrix of energy storage nodes; Represents the key matrix, and the query matrix correspond; represents the value matrix; are the weight matrices of query, key, and value for different headers respectively; is the weight matrix of multi-head output; is the dimension of the key matrix; is the number of heads in the multi-head attention mechanism; Concat is the concatenation operation.

[0015] Furthermore, the strategy network based on the deep reinforcement learning algorithm in step S3 calculates the probability distribution of its output scheduling action to satisfy the following formula:

[0016]

[0017] in, Indicates that the parameter is Under the policy network, the state Take action probability; The state in the state space constructed in step S1, including various parameters of the charging infrastructure and edge computing gateway energy storage system; The scheduling action in the action space defined for step S1; is the action space; is the output score function of the policy network, and its parameters Optimize through subsequent training.

[0018] Furthermore, the instant reward obtained in step S4 is calculated to satisfy the following formula:

[0019]

[0020] in, For immediate rewards; The operating costs of charging infrastructure, including the energy consumption costs of charging piles; Profits from charging and discharging the energy storage system for the edge computing gateway; Changes in charging waiting time for new energy vehicle users; They are the weight coefficients of charging infrastructure operation costs, charging and discharging income of edge computing gateway energy storage systems, and changes in charging waiting time for new energy vehicle users.

[0021] Furthermore, in step S5, when the deep reinforcement learning algorithm extracts sample data from the experience replay pool to construct a training data set, a priority experience replay mechanism is adopted, and the sample extraction probability satisfies the following formula:

[0022]

[0023] in, Indicates the number of The probability of drawing a sample; For the The time difference error of each sample reflects the importance of the sample to learning; is the total number of samples in the experience replay pool; is a parameter that controls the sampling bias.

[0024] Furthermore, when the value network is trained and optimized in step S6, the following loss function formula is adopted:

[0025]

[0026]

[0027] in, is the loss function of the value network, is the value network parameter; express expectations; are the state, action, immediate reward, and updated state of the sample extracted in step S5 respectively; For the value network in state Valuation under For the target value network in state Valuation under is a discount factor used to balance immediate rewards with future rewards.

[0028] Furthermore, in step S2, after extracting the features of the parameters using the multi-head attention mechanism model, the feature fusion formula is further adopted:

[0029]

[0030] in, is the fused feature vector; The feature vector extracted by the multi-head attention mechanism for charging infrastructure parameters; The feature vector of the edge computing gateway energy storage system parameters extracted by the multi-head attention mechanism; is the fusion weight coefficient.

[0031] Furthermore, in step S3, the policy network introduces a gating mechanism before outputting the probability distribution of the scheduling action, and its calculation satisfies the following formula:

[0032]

[0033] in, is the gating signal; is the activation function; is the weight matrix and bias vector of the gating mechanism; The policy network state Feature extraction results; The probability distribution of scheduling actions output by the original policy network; is the prior probability distribution; is the probability distribution of scheduling actions after adjustment by the gating mechanism.

[0034] Furthermore, in step S4, the instant reward is calculated When considering the equipment loss costs of the charging infrastructure and the edge computing gateway energy storage system, the instant reward calculation formula is adjusted to:

[0035]

[0036] in, Equipment loss costs for charging infrastructure and edge computing gateway energy storage systems; is the weight coefficient of equipment loss cost.

[0037] Beneficial effects: The present invention proposes a method for coordinated scheduling of new energy vehicle charging infrastructure and edge computing gateway energy storage. This method effectively overcomes the shortcomings of the existing technology through the organic combination of deep reinforcement learning algorithm and multi-head attention mechanism model. In terms of intelligence, the traditional scheduling method that relies on manual experience or simple rules is abandoned. The deep reinforcement learning algorithm can automatically learn and optimize the scheduling strategy according to the real-time status of the charging infrastructure and the edge computing gateway energy storage system. In the face of the new energy vehicle charging peak, it can intelligently coordinate the power distribution of charging piles and energy storage nodes to avoid overload of charging piles and idle energy storage nodes, reduce user waiting time, and improve the efficiency of power resource utilization. In parameter correlation analysis, the multi-head attention mechanism model can deeply explore the complex dependencies between the various parameters of the charging infrastructure and the edge computing gateway energy storage system, and no longer view parameters such as real-time power, demand quantity, and stored power in isolation. By extracting features and performing correlation analysis on these parameters, multi-dimensional feature information is formed, providing a comprehensive basis for scheduling decisions, thereby maximizing the overall benefits of the system, effectively controlling the operating costs of charging infrastructure, and improving the charging and discharging benefits of the edge computing gateway energy storage system. At the same time, by considering equipment loss costs, the scheduling strategy is made more comprehensive and scientific, giving full play to the advantages of coordinated scheduling of charging infrastructure and edge computing gateway energy storage. BRIEF DESCRIPTION OF THE DRAWINGS

[0038] Figure 1 A flow chart of the steps of the method of the present invention;

[0039] Figure 2 This is a diagram illustrating the method operation unit implementation of the present invention. DETAILED DESCRIPTION

[0040] It should be noted that, unless there is a conflict, the embodiments in this application and the features described in the embodiments can be combined with each other. The application is further described in detail below with reference to the accompanying drawings and specific embodiments.

[0041] like Figure 1 As shown, a method for coordinating the energy storage of new energy vehicle charging infrastructure and edge computing gateways includes the following steps:

[0042] Step S1: Construct a state space including the real-time charging power of each charging pile in the new energy vehicle charging infrastructure, the number of charging demands, the storage capacity of each energy storage node in the edge computing gateway energy storage system, and the charging and discharging power. Based on the deep reinforcement learning algorithm, define an action space including the probability of scheduling action selection. The scheduling action in the action space includes the charging power allocation of each charging pile in the charging infrastructure and the charging and discharging decision of each energy storage node in the edge computing gateway energy storage system.

[0043] Specifically, in the coordinated scheduling of new energy vehicle charging infrastructure and edge computing gateway energy storage, step S1 forms the foundational building block of the entire scheduling method. First, a state space is constructed, encompassing key operational information for the new energy vehicle charging infrastructure and edge computing gateway energy storage system. The new energy vehicle charging infrastructure component includes the real-time charging power of each charging station. This parameter directly reflects the current workload of the charging station, with different power levels corresponding to different charging speeds and energy consumption. The number of charging requests reflects the number of vehicles waiting to charge at a given moment and is a key indicator of the urgency of charging demand. Regarding the edge computing gateway energy storage system, the storage capacity of each energy storage node determines its dispatchable energy reserves. When the storage capacity is sufficient, it can be used to support charging peaks; when it is insufficient, charging must be rationally scheduled. The charge and discharge power parameters influence the efficiency of energy exchange between the energy storage node and the external power grid or charging infrastructure. An action space is defined based on a deep reinforcement learning algorithm, and scheduling actions within this action space focus on direct control of the charging infrastructure and edge computing gateway energy storage system. For charging infrastructure, it involves the allocation of charging power to each charging pile. For example, during peak charging demand, power is reasonably allocated based on the real-time status of the charging piles and the status of the energy storage nodes to avoid overloading some charging piles and leaving some idle. In the edge computing gateway energy storage system, scheduling actions are manifested as charging and discharging decisions for each energy storage node, such as determining when to use low-priced electricity to charge the energy storage node and when to discharge during peak electricity consumption to obtain profits. Each action corresponds to a certain selection probability, providing a basis for subsequent scheduling decisions.

[0044] Step S2: Utilize a multi-head attention mechanism model to extract features and perform correlation analysis on various parameters of the coordinated dispatch of charging infrastructure and edge computing gateway energy storage in the state space. This model captures the complex dependencies between different parameters through multi-head parallel computing, forming a feature vector set containing multi-dimensional feature information.

[0045] Specifically, the core of step S2 is to utilize a multi-head attention model to deeply process the parameters in the state space. In the scenario of coordinated scheduling of new energy vehicle charging infrastructure and edge computing gateway energy storage, the state space contains numerous and interrelated parameters, making it difficult for traditional methods to fully explore their inherent connections. The multi-head attention model, through its unique multi-head parallel computational approach, captures the complex dependencies between different parameters. For example, the real-time charging power of charging piles and the stored power of energy storage nodes are analyzed from different perspectives by the model's multiple "heads." One "head" might focus on the impact of high-power charging at charging piles on overall costs when energy storage nodes are fully charged; another might examine the feasibility and benefits of discharging energy storage nodes to supplement charging when charging demand surges and charging pile power is insufficient. Through this multi-head parallel computation, the model comprehensively extracts features from various parameters involved in the coordinated scheduling of charging infrastructure and edge computing gateway energy storage. The model converts the information contained in each parameter into a feature vector, further analyzes the relationships between these feature vectors, and ultimately forms a set of feature vectors containing multi-dimensional feature information. These feature vector sets are no longer isolated parameter information, but rather integrate deep information about the complex relationships between parameters, providing a more valuable basis for subsequent scheduling decisions and enabling the system to select scheduling actions based on a more comprehensive and in-depth understanding.

[0046] Step S3: Based on the policy network in the deep reinforcement learning algorithm, the feature vector set is used as input, and the probability distribution of the scheduling action in the action space is output. The coordinated scheduling action for the charging infrastructure and the edge computing gateway energy storage system is selected based on the probability distribution;

[0047] Specifically, step S3 relies on the policy network within the deep reinforcement learning algorithm, taking the feature vector set generated in step S2 as input to make scheduling decisions. The policy network acts as the "decision-making brain" throughout the coordinated scheduling process. Based on the multi-dimensional feature information input, it analyzes the current combined state of the new energy vehicle charging infrastructure and the edge computing gateway energy storage system, and then outputs a probability distribution for each scheduling action in the action space. For example, when the feature vector set indicates that charging demand is at a peak, some charging piles are nearing full capacity, and some nodes in the edge computing gateway energy storage system have sufficient charge, the policy network calculates the probabilities of scheduling actions such as increasing the discharge of energy storage nodes or adjusting the power of some charging piles. Based on this probability distribution, the system selects the corresponding coordinated scheduling action. This selection process does not simply select the action with the highest probability, but rather randomly samples according to the probability distribution. This approach considers the optimal scheduling direction under the current state while introducing a degree of exploration to avoid falling into local optimal solutions. By continuously outputting probability distributions and selecting actions based on different states, the policy network gradually learns the optimal coordinated scheduling strategy for charging infrastructure and edge computing gateway energy storage systems under various complex situations. As training progresses, the selected actions will increasingly tend to maximize the overall benefits of the system.

[0048] Step S4: Execute the selected collaborative scheduling action to obtain instant rewards including the operating cost of charging infrastructure, the charging and discharging income of the edge computing gateway energy storage system, and the change in the charging waiting time of new energy vehicle users, and update the parameters in the state space;

[0049] Specifically, step S4 implements the coordinated scheduling action selected in step S3, evaluates its effectiveness, and updates its status. Once the system determines and executes a scheduling action, it receives immediate rewards from multiple dimensions. For charging infrastructure, the change in operating costs after the action is executed is considered, including charging pile energy consumption costs and equipment wear and tear costs. If reasonable power allocation reduces energy consumption, this will be reflected in the rewards. For edge computing gateway energy storage systems, the focus is on charging and discharging benefits. Successfully discharging during high-price periods or charging during low-price periods increases benefits and is reflected in the rewards. Furthermore, the change in charging wait time for new energy vehicle users is a key evaluation metric. Shortening wait time earns positive rewards, while negative rewards result. Taken together, these rewards comprehensively reflect the impact of the scheduling action on the entire system. While receiving immediate rewards, the system updates various parameters in the state space based on the actual situation after the action is executed. For example, after executing an action in which a storage node discharges to support charging at a charging pile, the energy storage node's stored power will decrease accordingly. The charging power and number of charging piles required may also change as some vehicles complete charging. The updated state space will serve as the basis for the next scheduling decision, allowing the system to continuously optimize the scheduling strategy based on the latest system status.

[0050] Step S5: Store the state space, the selected scheduling action, the immediate reward, and the updated state space into the experience replay pool. Randomly extract sample data from the experience replay pool based on the deep reinforcement learning algorithm to construct a training dataset.

[0051] Specifically, step S5 stores key information from each scheduling process and constructs a training dataset to continuously optimize the scheduling strategy. After each scheduling action is executed and the state is updated, the system stores information including the current state space, the selected scheduling action, the immediate reward obtained, and the updated state space in the experience replay pool. The experience replay pool acts as an "experience warehouse," accumulating the system's scheduling experience at different times. Based on a deep reinforcement learning algorithm, the system randomly extracts sample data from the experience replay pool to construct a training dataset. This random sampling method breaks the temporal correlation of the data and avoids the overfitting problem caused by continuously learning similar data. The extracted sample data contains a variety of system states, scheduling actions, and their corresponding reward feedback. This diverse data provides rich material for subsequent model training. By continuously storing new scheduling experience in the experience replay pool and extracting samples from it to construct a training dataset, the system can fully leverage historical experience and learn more effective scheduling strategies from a large amount of practical data, gradually improving the accuracy and efficiency of the coordinated scheduling of new energy vehicle charging infrastructure and edge computing gateway energy storage systems.

[0052] Step S6: Use the training data set to iteratively train and optimize the policy network and the value network in the deep reinforcement learning algorithm, return to step S3 and repeat the execution until the preset training termination condition is met, and obtain a stable new energy vehicle charging infrastructure and edge computing gateway energy storage coordinated scheduling strategy.

[0053] Specifically, step S6 is the core optimization of the entire collaborative scheduling method. Using the training dataset constructed in step S5, the policy network and value network in the deep reinforcement learning algorithm are iteratively trained and optimized. The value network is used to estimate the expected future system returns from taking different actions in a given state, providing a reference for optimizing the policy network. The policy network, based on the feedback from the value network, adjusts its parameters to output a more optimal probability distribution for scheduling actions. During training, the system sequentially inputs samples from the training dataset into the network, calculates the error between the predicted and actual results, and propagates this error back through each layer of the network, updating the network parameters and continuously reducing the error. After multiple iterations of training, the performance of the policy network and value network gradually improves, enhancing the scheduling decision-making capabilities for the new energy vehicle charging infrastructure and edge computing gateway energy storage system. When preset training termination conditions are met, such as error convergence to a certain range or the number of training cycles reaching a set value, the system stops training. The resulting scheduling strategy is a stable collaborative scheduling strategy for the new energy vehicle charging infrastructure and edge computing gateway energy storage system. This strategy can continuously and efficiently coordinate the operation of charging infrastructure and edge computing gateway energy storage systems under different system states, achieving optimal resource allocation and maximizing the overall system benefits.

[0054] Preferably, the calculation process of the multi-head attention mechanism model used in step S2 satisfies the following formula:

[0055]

[0056] in, A query matrix representing the coordinated scheduling parameters of charging infrastructure and edge computing gateway energy storage in the state space, including the real-time charging power matrix of charging piles and the storage power matrix of energy storage nodes; Represents the key matrix, and the query matrix correspond; represents the value matrix; are the weight matrices of query, key, and value for different headers respectively; is the weight matrix of multi-head output; is the dimension of the key matrix; is the number of heads in the multi-head attention mechanism; Concat is the concatenation operation; the above formula is used to perform multi-head parallel feature extraction and fusion of the charging infrastructure and edge computing gateway energy storage system parameters to generate a set of feature vectors.

[0057] Specifically, the multi-head attention mechanism model uses multi-head parallel computing to analyze the parameters of the charging infrastructure and edge computing gateway energy storage system from different dimensions. For example, one "head" focuses on analyzing the dynamic relationship between the real-time charging power of the charging pile and the charging and discharging power of the energy storage node. During peak charging, it can identify the effect of the energy storage node discharge power on the charging pile load. Another "head" focuses on the matching degree between the number of charging demands and the stored power of the energy storage node, determining whether the existing energy storage can meet the expected charging demand. Through this multi-dimensional analysis, the model can capture the complex dependencies between parameters and generate a more comprehensive set of feature vectors, providing a richer information basis for subsequent scheduling decisions and improving the system's adaptability to complex scenarios.

[0058] Preferably, the strategy network based on the deep reinforcement learning algorithm in step S3 calculates the probability distribution of its output scheduling action to satisfy the following formula:

[0059]

[0060] in, Indicates that the parameter is Under the policy network, the state Take action probability; The state in the state space constructed in step S1, including various parameters of the charging infrastructure and edge computing gateway energy storage system; The scheduling action in the action space defined for step S1; is the action space; is the output score function of the policy network, and its parameters Through subsequent training optimization, this formula realizes the probability distribution of the coordinated scheduling actions of charging infrastructure and edge computing gateway energy storage based on the state space information output.

[0061] Specifically, the policy network in step S3 outputs a probability distribution for scheduling actions based on the state space information. In practice, when the state space indicates a surge in charging demand for charging piles in a certain area and sufficient energy storage nodes, the policy network will increase the probability of "increasing energy storage node discharge to support charging piles." Furthermore, by adjusting the probabilities of each action under different states, the policy network can learn the optimal scheduling strategy. For example, it can prioritize charging energy storage nodes during low electricity prices and generate revenue by discharging them during peak periods, thus achieving a balanced optimization of charging infrastructure operating costs and the revenue generated by the edge computing gateway energy storage system.

[0062] Preferably, the instant reward obtained in step S4 is calculated to satisfy the following formula:

[0063]

[0064] in, For immediate rewards; The operating costs of charging infrastructure, including the energy consumption costs of charging piles; Profits from charging and discharging the energy storage system for the edge computing gateway; Changes in charging waiting time for new energy vehicle users; The weight coefficients are respectively the operating cost of charging infrastructure, the charging and discharging income of the edge computing gateway energy storage system, and the change in the charging waiting time of new energy vehicle users. This formula is used to comprehensively calculate the immediate reward after executing the scheduling action to reflect the coordinated scheduling effect of charging infrastructure and edge computing gateway energy storage.

[0065] Specifically, the instant reward calculation takes into account multiple factors. Charging pile energy consumption accounts for a significant portion of the operating costs of charging infrastructure, and this consumption can be reduced through rational power allocation. The charging and discharging revenue of edge computing gateway energy storage systems is closely related to peak and valley electricity prices on the grid, with charging during valley hours and discharging during peak hours generating differential revenue. The change in waiting time for new energy vehicle users reflects service quality, and reducing waiting time can improve user satisfaction. The weighting coefficients α, β, and γ can be adjusted based on actual needs. For example, when the grid is under load, increasing α prioritizes controlling charging infrastructure operating costs, enabling a comprehensive assessment of the effectiveness of coordinated scheduling.

[0066] Preferably, in step S5, when the deep reinforcement learning algorithm extracts sample data from the experience replay pool to construct a training data set, a priority experience replay mechanism is adopted, and the sample extraction probability satisfies the following formula:

[0067]

[0068] in, Indicates the number of The probability of drawing a sample; For the The time difference error of each sample reflects the importance of the sample to learning; is the total number of samples in the experience replay pool; In order to control the parameters of sampling deviation, this formula realizes the priority extraction of important experience samples in the coordinated scheduling of charging infrastructure and edge computing gateway energy storage, thereby improving training efficiency.

[0069] Specifically, the prioritized experience replay mechanism improves training efficiency. In the coordinated scheduling of charging infrastructure and edge computing gateway energy storage, certain key experience samples are crucial for learning optimal strategies, such as scheduling when charging piles fail in extreme weather conditions. The time difference error reflects the importance of the sample. Samples with large errors indicate that the current strategy performs poorly in this state and require focused learning. By prioritizing these samples, the model can improve its strategy more quickly and reduce training time. This is especially important when dealing with complex electricity market price fluctuations and sudden changes in charging demand, allowing it to adapt and optimize scheduling strategies more quickly.

[0070] Preferably, when training and optimizing the value network in step S6, the following loss function formula is adopted:

[0071]

[0072]

[0073] in, is the loss function of the value network, is the value network parameter; express expectations; are the state, action, immediate reward, and updated state of the sample extracted in step S5 respectively; For the value network in state Valuation under For the target value network in state Valuation under is a discount factor used to balance immediate rewards and future rewards. The value network is optimized by minimizing this loss function to accurately evaluate the value of the coordinated scheduling status of charging infrastructure and edge computing gateway energy storage.

[0074] Specifically, value network training optimization ensures accurate assessment of state value. During periods of peak charging demand, the value network needs to accurately assess the impact of various scheduling actions under the current state on future benefits. For example, although choosing to increase the discharge of energy storage nodes can alleviate the current charging pressure, it may affect the discharge benefits during subsequent high-price periods. The discount factor γ balances immediate rewards with future rewards. A larger γ value makes the model focus more on long-term benefits, which is suitable for the long-term stable operation of the power grid; a smaller γ value focuses on immediate benefits and is suitable for short-term emergency scheduling. By minimizing the loss function, the value network can accurately predict the long-term effects of different scheduling strategies, providing a reliable basis for policy network optimization.

[0075] Preferably, in step S2, after extracting the features of the parameters using the multi-head attention mechanism model, a feature fusion formula is further adopted:

[0076]

[0077] in, is the fused feature vector; The feature vector extracted by the multi-head attention mechanism for charging infrastructure parameters; The feature vector of the edge computing gateway energy storage system parameters extracted by the multi-head attention mechanism; In order to fuse the weight coefficient, the feature vectors of the charging infrastructure and the edge computing gateway energy storage system are weightedly fused through this formula to enhance the feature representation capability.

[0078] Specifically, feature fusion enhances representation capabilities. Charging infrastructure parameters and edge computing gateway energy storage system parameters have different characteristics. The number of charging demands reflects user-side dynamics, while the amount of energy stored in energy storage nodes reflects the resource status on the system side. By weighted fusion of these two types of feature vectors, their respective advantages can be fully utilized. During peak charging hours, the charging infrastructure feature vector may indicate that some charging piles are overloaded, while the energy storage system feature vector indicates that nearby energy storage nodes have sufficient power. After fusion, more accurate scheduling recommendations can be generated, such as transferring part of the demand for overloaded charging piles to energy storage support, thereby improving the overall performance of the system.

[0079] Preferably, in step S3, the policy network introduces a gating mechanism before outputting the scheduling action probability distribution, and its calculation satisfies the following formula:

[0080]

[0081] in, is the gating signal; is the activation function; is the weight matrix and bias vector of the gating mechanism; The policy network state Feature extraction results; The probability distribution of scheduling actions output by the original policy network; is the prior probability distribution; The probability distribution of scheduling actions after adjustment by the gating mechanism is used to adjust the policy network output through the gating mechanism combined with prior knowledge to optimize the selection of coordinated scheduling actions for charging infrastructure and edge computing gateway energy storage.

[0082] Specifically, the gating mechanism optimizes action selection. Prior knowledge plays a crucial role in the coordinated scheduling of charging infrastructure and edge computing gateway energy storage. For example, historical data shows that weekend afternoons typically see peak charging demand. The gating mechanism combines the prior probability distribution with the original policy network output to automatically increase the probability of "preemptively starting energy storage node charging" during peak hours, reducing manual intervention. The gating signal dynamically adjusts the fusion ratio based on the current state. In extreme situations, such as grid failures, prior knowledge can be fully relied upon for rapid response, improving the system's robustness and ability to respond to emergencies.

[0083] Preferably, in step S4, the instant reward is calculated When considering the equipment loss costs of the charging infrastructure and the edge computing gateway energy storage system, the instant reward calculation formula is adjusted to:

[0084]

[0085] in, Equipment loss costs for charging infrastructure and edge computing gateway energy storage systems; is the weight coefficient of the equipment loss cost. This formula is used to more comprehensively calculate the immediate reward after executing the scheduling action to optimize the coordinated scheduling strategy of charging infrastructure and edge computing gateway energy storage.

[0086] Specifically, equipment loss costs are incorporated into reward calculations. Charging infrastructure and edge computing gateway energy storage systems incur losses over long periods of operation, and frequent charging and discharging shortens equipment lifespan. Incorporating equipment loss costs into immediate reward calculations encourages scheduling strategies to utilize equipment more rationally. For example, when peak and valley electricity prices are relatively close, frequent charging and discharging of energy storage nodes can be avoided, and operation can be optimized within power ranges with minimal equipment loss. Weighting coefficients can be adjusted based on factors such as equipment depreciation rates to balance short-term benefits with long-term equipment maintenance costs, achieving sustainable system operation.

[0087] like Figure 2 As shown in the figure, a method for coordinating the energy storage of new energy vehicle charging infrastructure and edge computing gateway is implemented through the following six units:

[0088] Parameter acquisition and state construction unit, used to execute the operations of constructing the state space and defining the action space in step S1, and to transmit state information and action space information to subsequent units;

[0089] A multi-head feature processing unit, connected to the parameter acquisition and state construction unit, is used to perform the operations of feature extraction and correlation analysis of the parameters using the multi-head attention mechanism model in step S2, and transmit the processed feature vector set to the strategy decision unit;

[0090] The policy decision unit is connected to the multi-head feature processing unit and the action execution unit respectively, and is used to perform the operation of selecting the scheduling action based on the probability distribution of the scheduling action output by the policy network in step S3;

[0091] An action execution and reward acquisition unit, connected to the strategy decision unit, is used to execute the operations of executing the scheduling action, obtaining the immediate reward, and updating the state space parameters in step S4, and transmit the immediate reward and the updated state space parameters to the experience storage and training unit;

[0092] The experience storage and training unit is connected to the action execution and reward acquisition unit and the policy decision unit respectively, and is used to perform the operations of storing experience in the experience replay pool and extracting samples to construct a training data set in step S5, and training and optimizing the policy network and value network in step S6, and feeding back the optimized network parameters to the policy decision unit;

[0093] The loop control unit is connected to the strategy decision unit and the experience storage and training unit, and is used to determine whether the preset training termination conditions are met, and control each unit to execute the loop operation of steps S3-S6 until a stable collaborative scheduling strategy is obtained.

[0094] The present invention proposes a method for collaborative scheduling of new energy vehicle charging infrastructure and edge computing gateway energy storage. This method addresses the problems of lack of intelligence and insufficient parameter correlation analysis in traditional technologies, and uses deep reinforcement learning algorithms and multi-head attention mechanism models to achieve optimized and upgraded resource scheduling.

[0095] Improve scheduling intelligence to cope with complex scenarios: Traditional scheduling relies on manual experience and simple rules, making it difficult to cope with the complex changes in charging demand for new energy vehicles. This collaborative scheduling method uses a deep reinforcement learning algorithm to give the system autonomous learning and decision-making capabilities. It can perceive the dynamic status of the charging infrastructure and edge computing gateway energy storage system in real time, such as the real-time charging power of the charging pile, the storage capacity of the energy storage node, and other information. When new energy vehicles have a charging peak, the system no longer causes the charging pile to be overloaded or the energy storage to be idle as in traditional methods. Instead, it automatically learns and optimizes the scheduling strategy, intelligently allocates the power of the charging pile, and reasonably arranges the charging and discharging of the energy storage node, effectively reducing user waiting time, significantly improving the efficiency of power resource utilization, and making the operation of the charging infrastructure and energy storage system more efficient and flexible.

[0096] Deeply explore parameter associations and optimize overall benefits: Previous technologies viewed the parameters of charging infrastructure and edge computing gateway energy storage systems in isolation, which failed to maximize overall benefits. This method uses a multi-head attention mechanism model to deeply explore the complex dependencies between various parameters. Whether it is the charging demand quantity of the charging infrastructure or the charging and discharging power of the edge computing gateway energy storage system, the model can perform feature extraction and correlation analysis on these parameters through multi-head parallel computing to form comprehensive multi-dimensional feature information. The scheduling strategy formulated based on this information fully considers the mutual influence between various parameters, can effectively control the operating costs of charging infrastructure, improve the charging and discharging benefits of the edge computing gateway energy storage system, and optimize the overall benefits of the system.

[0097] Multi-step collaborative optimization to ensure long-term stability: In addition to the two core breakthroughs mentioned above, this collaborative scheduling method has also been optimized in multiple aspects. In the reward calculation stage, factors such as charging infrastructure operating costs, energy storage revenue, user waiting time, and equipment loss are fully considered to comprehensively evaluate the effectiveness of scheduling actions. Through the experience replay pool and priority experience replay mechanism, historical experience is effectively utilized to improve training efficiency. Utilizing cyclic training to optimize the strategy network and value network, the scheduling strategy is continuously evolved, ultimately obtaining a stable and reliable collaborative scheduling solution to ensure the long-term stable and efficient operation of the charging infrastructure and edge computing gateway energy storage system.

[0098] In the description of the present invention, it should be noted that, unless otherwise expressly specified or limited, the terms "disposed," "installed," "connected," "connected," and "fixed" should be understood in a broad sense. For example, they may refer to fixed connections, detachable connections, or integral connections; they may refer to mechanical connections or electrical connections; they may refer to direct connections or indirect connections through an intermediate medium; and they may refer to internal communication between two components. Those skilled in the art will understand the specific meanings of the above terms in the present invention based on specific circumstances.

[0099] While embodiments of the present invention have been shown and described, it will be understood by those skilled in the art that various equivalent changes, modifications, substitutions and variations may be made to the embodiments without departing from the principles and spirit of the invention, and that the scope of the invention is defined by the appended claims and their equivalents.

Claims

1. A method for collaborative scheduling of new energy vehicle charging infrastructure and edge computing gateway energy storage, characterized in that: The following steps are involved: Step S1: Construct a state space including the real-time charging power of each charging pile of the new energy vehicle charging infrastructure, the number of charging demands, the storage capacity of each energy storage node of the edge computing gateway energy storage system, and the charging and discharging power, and define the action space of the scheduling action selection probability; Step S2: Utilize a multi-head attention mechanism model to extract features and perform correlation analysis on various parameters of the coordinated dispatch of charging infrastructure and edge computing gateway energy storage in the state space. Through multi-head parallel computing, the dependencies between different parameters are captured to form a feature vector set containing multi-dimensional feature information. Step S3: Based on the policy network, the feature vector set is used as input, and the probability distribution of the scheduling action in the action space is output. The coordinated scheduling action for the charging infrastructure and the edge computing gateway energy storage system is selected according to the probability distribution; Step S4: Execute the selected collaborative scheduling action to obtain instant rewards including the operating cost of charging infrastructure, the charging and discharging income of the edge computing gateway energy storage system, and the change in the charging waiting time of new energy vehicle users, and update the parameters in the state space; Step S5: Store the state space, the selected scheduling action, the immediate reward, and the updated state space into the experience replay pool, randomly extract sample data from the experience replay pool, and construct a training data set; Step S6: Iteratively train and optimize the policy network and the value network in the deep reinforcement learning algorithm, and return to step S3 and repeat until the preset training termination condition is met.

2. A method for collaborative scheduling of new energy vehicle charging infrastructure and edge computing gateway energy storage according to claim 1, characterized in that: In step S2, the multi-head attention mechanism model is used, and the calculation process satisfies the following formula: . in, A query matrix representing the coordinated scheduling parameters of charging infrastructure and edge computing gateway energy storage in the state space, including the real-time charging power matrix of charging piles and the storage power matrix of energy storage nodes; Represents the key matrix, and the query matrix correspond; represents the value matrix; are the weight matrices of query, key, and value for different headers respectively; is the weight matrix of multi-head output; is the dimension of the key matrix; is the number of heads in the multi-head attention mechanism; Concat is the concatenation operation.

3. A method for collaborative scheduling of new energy vehicle charging infrastructure and edge computing gateway energy storage according to claim 1, characterized in that: In step S3, the calculation of the probability distribution of the scheduling action output by the policy network satisfies the following formula: . in, Indicates that the parameter is Under the policy network, the state Take action probability; The state in the state space constructed in step S1, including various parameters of the charging infrastructure and edge computing gateway energy storage system; The scheduling action in the action space defined for step S1; is the action space; is the output score function of the policy network, parameter Optimize through subsequent training.

4. A method for collaborative scheduling of new energy vehicle charging infrastructure and edge computing gateway energy storage according to claim 1, characterized in that: In step S4, the instant reward obtained is calculated to satisfy the following formula: . in, For immediate rewards; The operating costs of charging infrastructure, including the energy consumption costs of charging piles; Profits from charging and discharging the energy storage system for the edge computing gateway; Changes in charging waiting time for new energy vehicle users; They are the weight coefficients of charging infrastructure operation costs, charging and discharging income of edge computing gateway energy storage systems, and changes in charging waiting time for new energy vehicle users.

5. The method for collaborative scheduling of new energy vehicle charging infrastructure and edge computing gateway energy storage according to claim 1 is characterized in that: In step S5, when extracting sample data from the experience replay pool to construct a training data set, a priority experience replay mechanism is adopted, and the sample extraction probability satisfies the following formula: . in, Indicates the number of The probability of drawing a sample; For the The time difference error of each sample reflects the importance of the sample to learning; is the total number of samples in the experience replay pool; is a parameter that controls the sampling bias.

6. A method for collaborative scheduling of new energy vehicle charging infrastructure and edge computing gateway energy storage according to claim 1, characterized in that: In step S6, when training and optimizing the value network, the following loss function formula is used: . . in, is the loss function of the value network, is the value network parameter; express expectations; are the state, action, immediate reward, and updated state of the sample extracted in step S5 respectively; For the value network in state Valuation under For the target value network in state Valuation under is a discount factor used to balance immediate rewards with future rewards.

7. A method for collaboratively scheduling new energy vehicle charging infrastructure and edge computing gateway energy storage according to claim 1, characterized in that: In step S3, the policy network introduces a gating mechanism before outputting the probability distribution of the scheduling action, and the calculation satisfies the following formula: . in, is the gating signal; is the activation function; is the weight matrix and bias vector of the gating mechanism; The policy network state Feature extraction results; The probability distribution of scheduling actions output by the original policy network; is the prior probability distribution; is the probability distribution of scheduling actions after adjustment by the gating mechanism.

8. A method for collaboratively scheduling new energy vehicle charging infrastructure and edge computing gateway energy storage according to claim 4, characterized in that: In step S4, the instant reward is calculated When considering the equipment loss costs of the charging infrastructure and the edge computing gateway energy storage system, the instant reward calculation formula is adjusted to: . in, Equipment loss costs for charging infrastructure and edge computing gateway energy storage systems; is the weight coefficient of equipment loss cost.

Images