A multi-agent-based residential area charging load prediction and orderly charging method and system

Abstract

The present application relates to a kind of based on multi-agent residential area charging load prediction and orderly charging method, specifically as follows: first, obtain charging pile load data, carry out missing value and abnormal value correction;Second, the one-dimensional load data after correction is converted to two-dimensional space, extracts multi-scale time sequence feature, obtains periodic characteristic sequence;Second, construct charging load prediction network, using periodic characteristic sequence training;Then, based on load prediction network, construct electric vehicle orderly charging reinforcement learning environment;Then, using reinforcement learning environment, train the multi-agent orderly charging strategy network of shared parameter;Finally, using real-time load prediction and optimal strategy network, generate residential area electric vehicle orderly charging scheme.The scheme improves the accuracy of residential area charging load prediction and the performance of orderly charging strategy.

Description

Technical Field

[0001] This invention belongs to the field of orderly charging technology for electric vehicles, and mainly relates to a method and system for predicting and orderly charging charging load in residential areas based on multi-agent systems. Background Technology

[0002] With the rapid growth in the number of electric vehicles, the impact of residential charging load on the safe and economical operation of the power distribution network is becoming increasingly significant. Influenced by users' travel patterns and daily routines, residential charging behavior exhibits a clear temporal concentration. Therefore, implementing accurate load forecasting and orderly charging scheduling is of great importance for peak shaving and valley filling, and reducing user costs.

[0003] Currently, deep learning technology has been gradually incorporated into charging load prediction methods, but these methods still have the following shortcomings in small-sample scenarios in residential areas: existing models have complex structures and are prone to overfitting when the number of charging record samples is limited, resulting in a significant decrease in prediction performance; most methods rely only on unidirectional time-series modeling, making it difficult to fully capture the forward and backward dependencies of charging load, leading to high prediction errors. Regarding ordered charging, existing research mostly employs deterministic optimization or Monte Carlo simulation methods, which are offline static strategies and cannot adapt to the real-time dynamic changes of multiple vehicles charging concurrently. Some methods introduce deep reinforcement learning, but they mostly use a single-agent architecture, which cannot effectively coordinate the charging power allocation of multiple electric vehicles, easily leading to transformer overload or insufficient power when some vehicles leave the site.

[0004] Load modeling methods based on time-series forecasting and orderly charging methods based on multi-agent reinforcement learning have shown potential in their respective fields. However, how to organically combine the two to achieve accurate prediction of charging load and dynamic orderly charging of multiple vehicles in small-sample scenarios in residential areas remains an unsolved problem. For example, although time-series forecasting models can improve prediction accuracy through periodic feature extraction and bidirectional time-series modeling, they are prone to overfitting under small-sample conditions. Furthermore, there is a lack of effective transition methods for embedding the prediction results as forward-looking constraints into the orderly charging environment and guiding agents to adjust charging power in advance. Although multi-agent deep deterministic policy gradient algorithms can achieve multi-vehicle power coordination through centralized training, the optimization objective of scheduling actions and the dynamic coupling relationship with the predicted load are complex, making it difficult to balance short-term charging demand with long-term distribution network stability.

[0005] To address the aforementioned issues, there is an urgent need for a method and system for predicting and orderly charging in residential areas based on multi-agent systems. Summary of the Invention

[0006] To address the shortcomings of existing technologies, this invention provides a method and system for predicting and orderly charging charging loads in residential areas based on multi-agent systems. First, charging pile load data is acquired, and missing and outlier values ​​are corrected. Next, the corrected one-dimensional load data is transformed into a two-dimensional space, and multi-scale temporal features are extracted to obtain a periodic feature sequence. Then, a charging load prediction network is constructed and trained using the periodic feature sequence. Next, based on the load prediction network, an orderly charging reinforcement learning environment for electric vehicles is constructed. Then, using the reinforcement learning environment, a multi-agent orderly charging strategy network with shared parameters is trained. Finally, using real-time load prediction and the optimal strategy network, an orderly charging scheme for electric vehicles in residential areas is generated.

[0007] To achieve the above objectives, the present invention provides the following solution:

[0008] In a first aspect, this invention relates to a method for predicting and orderly charging in residential areas based on multi-agent systems, the method comprising the following steps:

[0009] S1. Obtain charging pile load data and correct missing and outlier values;

[0010] S2. Convert the corrected one-dimensional load data to a two-dimensional space, extract multi-scale time series features, and obtain a periodic feature sequence.

[0011] S3. Construct a charging load prediction network and train it using periodic feature sequences;

[0012] S4. Construct a reinforcement learning environment for orderly charging of electric vehicles based on a load prediction network;

[0013] S5. Utilize a reinforcement learning environment to train a multi-agent ordered charging policy network with shared parameters.

[0014] S6. Utilize real-time load forecasting and optimal strategy networks to generate an orderly charging scheme for electric vehicles in residential areas.

[0015] As a preferred solution, in S1, the charging pile load data is acquired, and missing and outlier values ​​are corrected, including:

[0016] Step 1: Collect historical charging load data of residential charging piles from the metering equipment and management system of electric vehicle charging stations, and represent it as follows: ,in Indicates a point in time Load data collected in real time This indicates the length of the time series. The value of each sample point represents the overall load of the charging station at the sampling time, in kilowatts (kW).

[0017] Step 2: Perform missing value repair and outlier correction based on the lightweight Moment pre-trained model.

[0018] First, the original historical charging load data is reversibly instantiated, denoted as:

[0019] (1)

[0020] in, This represents the normalized sequence values. and The mean and standard deviation of the sequence are used to obtain the normalized sequence. Then, a masking operation is performed to eliminate the differences in amplitude and offset distributions between different sequences. Patching is used to segment the normalized sequence into non-overlapping subsequences of different lengths. These are then mapped to embedding vectors by the encoding module and superimposed with learnable absolute position codes to preserve temporal position information. Finally, the sequence is reconstructed using the reconstruction module, denoted as:

[0021] (2)

[0022] in, Indicates a point in time A one-dimensional time series, This represents the overall mapping function of the Moment pre-trained model, which includes operations such as embedding, encoding, and decoding, thus yielding a one-dimensional time series output after preprocessing. ,

[0023] The above steps introduce a lightweight Moment pre-trained model. Leveraging the general temporal prior knowledge obtained through pre-training, this model, compared to existing interpolation or statistical repair methods, achieves context-based anomaly repair on small sample data through reversible instance normalization and mask reconstruction. This avoids the amplification of biases in traditional local linear methods when data is sparse, providing highly reliable input for subsequent feature extraction.

[0024] As a preferred embodiment, in S2, the corrected one-dimensional load data is transformed into a two-dimensional space, and multi-scale time-series features are extracted to obtain a periodic feature sequence, including:

[0025] Step 1: Convert the corrected one-dimensional load data to two-dimensional space and capture the two-dimensional time variation.

[0026] The main periods of the input one-dimensional time series data are identified by Fast Fourier Transform. Based on this, the time series data is reconstructed into a two-dimensional tensor by stacking the periods, thus capturing the two-dimensional time variation features in the time series data, denoted as:

[0027] (3)

[0028] in, express The average amplitude intensity of each frequency component Indicates averaging. Indicates amplitude calculation, Indicates the Fast Fourier Transform. Represents a two-dimensional tensor. Indicates according to the first The group periodic frequency reconstructs the original sequence. This indicates a zero-padding operation. The number of periods identified by the Fast Fourier Transform. , For the first The length of each cycle For the first The number of rows in each cycle, This represents the feature tensor output after extracting two-dimensional temporal variation features, whose dimension is the same as the input. This indicates that features are captured using dual-scale convolution kernels.

[0029] Step 2: After capturing the changing features, the dimensionality of the two-dimensional data is reduced. Tensors at each scale are weighted based on amplitude to aggregate and capture multi-scale temporal features, outputting a periodic feature sequence, denoted as:

[0030] (4)

[0031] in, This represents the one-dimensional data obtained after dimensionality reduction. This represents the truncation and zeroing operation. This represents converting a two-dimensional tensor back to one dimension. This represents the periodic feature sequence of the output. Indicates the amplitude according to each frequency of the signal. For one-dimensional data Weighted normalization, the final output time series is .

[0032] The above steps reconstruct one-dimensional data into two-dimensional tensors according to the period using fast Fourier transform, so that the temporal changes within and between periods can be captured simultaneously by dual-scale convolution kernels, and multi-scale features are aggregated based on amplitude weighting, which breaks through the limitations of existing models in period perception and provides a more structured periodic input for prediction networks.

[0033] As a preferred embodiment, in S3, a charging load prediction network is constructed and trained using periodic feature sequences, including:

[0034] Step 1: During training, the periodic feature sequence is fed into the constructed charging load prediction network, and the output hidden state sequence is denoted as:

[0035] (5)

[0036] in, , For forward prediction networks in , The hidden state at all times , For backward prediction networks in , The hidden state at all times To predict the unit computation function of the network, , To predict the weight matrix of the network, This is the bias matrix of the output layer. For charging load prediction networks in The final output at any given moment,

[0037] Step 2: Extract three pooling features—last position, mean, and maximum value—from the output hidden state sequence to construct a global sequence vector. This vector is then mapped through a fully connected layer to output the load prediction value, denoted as:

[0038] (6)

[0039] in, This is the global sequence vector after concatenating pooled features. , and These represent bottom-end pooling, mean pooling, and maximum pooling, respectively. For charging load prediction networks in Output at any moment This indicates taking the maximum value element by element. This indicates the predicted total charging load at the next moment. This represents the weight matrix of the fully connected layer. This is a bias term.

[0040] Example: The load prediction model proposed in this invention is compared with TimesNet, Bi-LSTM, Moment, CNN-GRU, and Transformer models, with each model undergoing 100 rounds of training. The results are shown in the attached figure. Figure 1As shown, the other comparative models exhibit varying degrees of prediction bias at peaks, troughs, or inflection points of rapid load changes, resulting in limited overall fitting accuracy. In contrast, the model proposed in this invention demonstrates the highest fitting accuracy across all time points. This is because this invention upscales the one-dimensional time series to a two-dimensional space to capture multi-scale periodic dependencies, and then performs bidirectional time series trimming, significantly improving context awareness and fluctuation robustness. This verifies the superiority of the proposed load prediction model in small-sample scenarios.

[0041] As a preferred embodiment, in S4, a reinforcement learning environment for orderly charging of electric vehicles is constructed based on a load prediction network, including:

[0042] Step 1: Model the orderly charging problem of electric vehicles as a Markov decision process, and construct the state space and action space of reinforcement learning for orderly charging of electric vehicles in residential areas based on the predicted load.

[0043] The state space and action space of the electric vehicle charging process are defined as follows:

[0044] (7)

[0045] in, This indicates the current grid electricity price. Indicates vehicle exist Normalized state of charge at time, , Indicates vehicle The goal is to normalize the state of charge. , Indicates vehicle The remaining time of stay, This indicates the current electric vehicle charging load under the power distribution network. This is a globally shared feature, representing the electric vehicle charging load predicted for the next moment based on historical data, allowing each agent to adjust its current charging power in advance. Indicates vehicle exist Normalized charging load request at time, Indicates vehicle exist The actual charging load at that time Indicates vehicle The rated maximum charging power,

[0046] Step 2: Construct the reward function and constraints for the orderly charging reinforcement learning environment of electric vehicles in residential areas based on the predicted load.

[0047] The reward function for the electric vehicle charging process is defined as:

[0048] (8)

[0049] in, For the total reward function, Reward users for charging costs. Rewards for peak shaving and valley filling projects in power distribution networks. , For each corresponding item, the weight coefficient is... This represents the number of electric vehicles in the current power distribution network. Maximum charging power for electric vehicles For time intervals, The average value of the load curve.

[0050] The constraints of the electric vehicle charging process are defined as follows:

[0051] (9)

[0052] in, This refers to the rated power of the transformer. As a penalty factor for predicted load, the current maximum allowable total power is reduced when the predicted charging load is high at the next moment.

[0053] Existing reinforcement learning environment designs rely solely on the current load and lack foresight. The steps described above embed the predicted load as a global state into the observation space of each agent and introduce the predicted value into the constraints to dynamically adjust the power upper limit. This allows agents to perceive the load evolution trend in advance during training, achieving an environment upgrade from passive response to proactive pre-adjustment, thus improving the safety and predictability of scheduling.

[0054] As a preferred approach, in S5, a multi-agent ordered charging policy network with shared parameters is trained using a reinforcement learning environment, including:

[0055] Step 1: Using a reinforcement learning environment, train and update the Actor and Critic networks with shared parameters based on the multi-agent deep deterministic policy gradient algorithm.

[0056] When the number of samples in the experience replay pool reaches the batch requirement, a small batch of samples is sampled from it. Each evaluation network is updated sequentially by minimizing the temporal difference loss, the shared policy network is updated by the policy gradient, and the target network is updated using a soft update method to solve for the optimal policy network.

[0057] The formula for calculating the policy gradient of an Actor network with shared parameters is:

[0058] (10)

[0059] in, Actor network parameters The gradient is used to derive the agent's gradient. The Actor network is being updated. and For each intelligent agent Observation and actions For the current Critic network, For the current network of the Actor, Represents status information. To draw samples from the experience replay pool Expected value This is an experience replay pool, storing experience tuples.

[0060] The formula for updating the loss function of the Critic network is as follows:

[0061] (11)

[0062] in, For intelligent agents The Critic network loss function value, The time-difference objective is used to guide the update of the current Q value. For intelligent agents Instant rewards As a discount factor, For the Critic target network, This represents the global state at the next moment. For the next moment's action,

[0063] Step 2: Train and update the target Actor network and Critic network based on the multi-agent deep deterministic policy gradient algorithm.

[0064] The soft update calculation formulas for the target Actor network and Critic network are as follows:

[0065] (12)

[0066] in, The update rate for soft target parameters is typically a relatively small value that satisfies... , and The parameters for the Actor training network and the target network are respectively. and These are the parameters for the Critic training network and the target network, respectively.

[0067] Example: The training process for the above-mentioned agent is shown in the attached diagram. Figure 2As shown in the figure, the proposed ordered charging strategy based on the shared parameter multi-agent deep deterministic policy gradient algorithm is compared with the unordered charging, time-series prediction curve, DDQN, and DDPG algorithms. Figure 3 As shown. Compared to disordered charging and time-series prediction curves, all scheduling algorithms can achieve a certain degree of peak shaving and valley filling. Among them, the strategy proposed in this invention reduces the load from a disordered 230.54kW to 180.48kW during peak hours and increases the load from a disordered 21.10kW to 51.04kW during off-peak hours. The peak reduction and valley filling effects are better than the DDQN and DDPG algorithms, thus achieving the peak shaving and valley filling goal more evenly.

[0068] As a preferred solution, in S6, a residential area electric vehicle orderly charging scheme is generated using real-time load forecasting and an optimal policy network. The specific process is as follows: Based on real-time load forecasting, the scheme is embedded in reinforcement learning environment modeling. The above steps of environment interaction, priority sampling, network parameter updates, and target network soft updates are repeatedly executed until the preset number of training rounds is completed. The decision-making behavior of each agent is gradually optimized, the Q-value evaluation accuracy of the Critic network is continuously improved, and the action sequence output by the Actor network gradually approaches the globally optimal orderly charging strategy. After training, the optimal parameters of the Actor network are saved. Without real-time communication and data sharing, each charging pile calls the network to output a charging power request, generating the optimal residential area orderly charging scheme, denoted as:

[0069] (13)

[0070] in, Indicates vehicle exist The optimal charging power at any given time. These are the optimal parameters for the Actor network. For vehicles exist Local observation at a given moment.

[0071] As a preferred embodiment, the method adjusts the charging power of each charging pile in the residential area based on the charging load prediction results, thereby achieving orderly and optimized management of electric vehicle charging.

[0072] A second aspect of this invention relates to a multi-agent-based residential area charging load prediction and orderly charging system. The system includes a data preprocessing module, a load prediction module, a charging environment modeling module, and an orderly charging module. The data preprocessing module acquires historical charging load data of residential charging piles from the metering equipment and management system of electric vehicle charging stations, performs reversible instance normalization based on a lightweight Moment pre-trained model, and repairs missing values ​​and outliers in the corrected data. The load prediction module converts the corrected one-dimensional load data to a two-dimensional space, extracts multi-scale time-series features, obtains a periodic feature sequence, and constructs a charging load prediction system. The load prediction network is trained using periodic feature sequences and outputs predicted load. The charging environment modeling module is used to construct the state space, action space, reward function, and dynamic power constraints of a Markov decision process based on the output predicted load, real-time electricity price, state of charge of each electric vehicle, and transformer rated power, thus forming an orderly charging environment for electric vehicles in residential areas. The orderly charging module uses a reinforcement learning environment and a multi-agent deep deterministic policy gradient algorithm with shared parameters to train the model, continuously update the network parameters, save the optimal policy network after training, output the optimal charging strategy, and generate an orderly charging scheme for each vehicle at each time period.

[0073] The beneficial effects of this invention are as follows: Compared with the prior art, the residential area charging load prediction and orderly charging method and system based on multi-agent technology in this invention uses a lightweight Moment pre-trained model for data preprocessing, transforms the corrected one-dimensional load data into a two-dimensional space, constructs a charging load prediction network, uses periodic feature sequences for training and outputs predicted loads, constructs an orderly charging environment, and uses a multi-agent deep deterministic strategy gradient algorithm with shared parameters to solve the problem, generating orderly charging schemes for each vehicle at each time period. Based on the time-series prediction results, this invention regulates the charging power of each charging pile in the residential area, improving the prediction accuracy and multi-vehicle collaborative charging optimization capabilities in small sample scenarios.

[0074] The beneficial effects of the present invention also include:

[0075] 1. In this invention, a lightweight Moment pre-trained model is used to perform reversible instance normalization, masking, and reconstruction repair on the original electricity meter charging load data. This can effectively handle data missing and abnormal fluctuation problems in small sample scenarios in residential areas, improve the reliability of input data, and avoid the repair bias of traditional interpolation or statistical methods when data is sparse.

[0076] 2. In this invention, the corrected one-dimensional load data is transformed into a two-dimensional space, multi-scale time-series features are extracted, periodic feature sequences are obtained, a charging load prediction network is constructed, and the periodic feature sequences are used for training. This fully captures the time-series features of the charging load, improves the problem of overfitting of a single model under small sample conditions, and significantly improves the prediction accuracy.

[0077] 3. This invention employs a multi-agent deep deterministic strategy gradient algorithm with shared parameters. Through centralized training and distributed execution, it achieves dynamic collaborative optimization of charging power for multiple vehicles, avoiding insufficient power when transformers are overloaded or some vehicles leave the site. This reduces user charging costs while minimizing the peak-valley difference in grid load. Furthermore, each agent does not require real-time communication or data sharing during the execution phase, effectively protecting user privacy. Attached Figure Description

[0078] Figure 1 This is a comparison chart of the load prediction model proposed in this invention with other prediction models;

[0079] Figure 2 This is a flowchart illustrating the training process of the multi-agent deep deterministic policy gradient algorithm based on shared parameters proposed in this invention.

[0080] Figure 3 This is a comparison diagram of the control effects of the ordered charging method proposed in this invention with other optimized methods; Detailed Implementation

[0081] To make the objectives, technical solutions, and advantages of this invention clearer, the technical solutions of this invention will be clearly and completely described below with reference to the accompanying drawings of the embodiments of this invention. The embodiments described in this invention are merely some embodiments of this invention, and not all embodiments. Based on the spirit of this invention, all other embodiments not described in this invention obtained by those skilled in the art based on the embodiments described in this invention without creative effort should fall within the protection scope of this invention.

[0082] The first aspect of this invention relates to a method for predicting and orderly charging in residential areas based on multi-agent systems, the method comprising the following steps:

[0083] S1. Obtain charging pile load data and correct missing and outlier values;

[0084] S2. Convert the corrected one-dimensional load data to a two-dimensional space, extract multi-scale time series features, and obtain a periodic feature sequence.

[0085] S3. Construct a charging load prediction network and train it using periodic feature sequences;

[0086] S4. Construct a reinforcement learning environment for orderly charging of electric vehicles based on a load prediction network;

[0087] S5. Utilize a reinforcement learning environment to train a multi-agent ordered charging policy network with shared parameters.

[0088] S6. Utilize real-time load forecasting and optimal strategy networks to generate an orderly charging scheme for electric vehicles in residential areas.

[0089] In S1, the original electricity meter charging load data and charging pile load data are obtained, and missing and outlier value corrections are performed, including:

[0090] Step 1: Collect historical charging load data of residential charging piles from the metering equipment and management system of electric vehicle charging stations, and represent it as follows: ,in Indicates at a point in time Load data collected in real time This indicates the length of the time series. The value of each sample point represents the overall load of the charging station at the sampling time, in kilowatts (kW).

[0091] Step 2: Perform missing value repair and outlier correction based on the lightweight Moment pre-trained model.

[0092] First, the original historical charging load data is reversibly instantiated, denoted as:

[0093] (1)

[0094] in, This represents the normalized sequence values. and The mean and standard deviation of the sequence are used to obtain the normalized sequence. Then, a masking operation is performed to eliminate the differences in amplitude and offset distributions between different sequences. Patching is used to segment the normalized sequence into non-overlapping subsequences of different lengths. These are then mapped to embedding vectors by the encoding module and superimposed with learnable absolute position codes to preserve temporal position information. Finally, the sequence is reconstructed using the reconstruction module, denoted as:

[0095] (2)

[0096] in, Indicates at a point in time A one-dimensional time series, This represents the overall mapping function of the Moment pre-trained model, which includes operations such as embedding, encoding, and decoding, thus yielding a one-dimensional time series output after preprocessing. .

[0097] In S2, the corrected one-dimensional load data is transformed into a two-dimensional space, and multi-scale time-series features are extracted to obtain a periodic feature sequence, including:

[0098] Step 1: Convert the corrected one-dimensional load data to two-dimensional space and capture the two-dimensional time variation.

[0099] The main periods of the input one-dimensional time series data are identified by Fast Fourier Transform. Based on this, the time series data is reconstructed into a two-dimensional tensor by stacking the periods, thus capturing the two-dimensional time variation features in the time series data, denoted as:

[0100] (3)

[0101] in, express The average amplitude intensity of each frequency component Indicates averaging. Indicates amplitude calculation, Indicates the Fast Fourier Transform. Represents a two-dimensional tensor. Indicates according to the first The group periodic frequency reconstructs the original sequence. This indicates a zero-padding operation. The number of periods identified by the Fast Fourier Transform. , For the first The length of each cycle For the first The number of rows in each cycle, This represents the feature tensor output after extracting two-dimensional temporal variation features, whose dimension is the same as the input. This indicates that features are captured using dual-scale convolution kernels.

[0102] Step 2: After capturing the changing features, the dimensionality of the two-dimensional data is reduced. Tensors at each scale are weighted based on amplitude to aggregate and capture multi-scale temporal features, outputting a periodic feature sequence, denoted as:

[0103] (4)

[0104] in, This represents the one-dimensional data obtained after dimensionality reduction. This represents the truncation and zeroing operation. This represents converting a two-dimensional tensor back to one dimension. This represents the periodic feature sequence of the output. Indicates the amplitude according to each frequency of the signal. For one-dimensional data Weighted normalization, the final output time series is .

[0105] In S3, a charging load prediction network is constructed and trained using periodic feature sequences, including:

[0106] Step 1: During training, the periodic feature sequence is fed into the constructed charging load prediction network, and the output hidden state sequence is denoted as:

[0107] (5)

[0108] in, , For forward prediction networks in , The hidden state at all times , For backward prediction networks in , The hidden state at all times To predict the unit computation function of the network, , To predict the weight matrix of the network, This is the bias matrix of the output layer. For charging load prediction networks in The final output at any given moment,

[0109] Step 2: Extract three pooling features—last position, mean, and maximum value—from the output hidden state sequence to construct a global sequence vector. This vector is then mapped through a fully connected layer to output the load prediction value, denoted as:

[0110] (6)

[0111] in, This is the global sequence vector after concatenating pooled features. , and These represent bottom-end pooling, mean pooling, and maximum pooling, respectively. For charging load prediction networks in Output at any moment This indicates taking the maximum value element by element. This indicates the predicted total charging load at the next moment. This represents the weight matrix of the fully connected layer. This is a bias term.

[0112] Example: The load prediction model proposed in this invention is compared with TimesNet, Bi-LSTM, Moment, CNN-GRU, and Transformer models, with each model undergoing 100 rounds of training. The results are shown in the attached figure. Figure 1As shown, the other comparative models all exhibit varying degrees of prediction bias at peaks, troughs, or inflection points of rapid load changes, resulting in limited overall fitting accuracy. In contrast, the model proposed in this invention demonstrates the highest fitting accuracy across all time points. This is because this invention upscales the one-dimensional time series to a two-dimensional space to capture multi-scale periodic dependencies, and then performs bidirectional time series trimming, significantly improving context awareness and fluctuation robustness. This verifies the superiority of the proposed load prediction model in small-sample scenarios.

[0113] In S4, a reinforcement learning environment for orderly charging of electric vehicles is constructed based on a load prediction network, including:

[0114] Step 1: Model the orderly charging problem of electric vehicles as a Markov decision process, and construct the state space and action space of reinforcement learning for orderly charging of electric vehicles in residential areas based on the predicted load.

[0115] The state space and action space of the electric vehicle charging process are defined as follows:

[0116] (7)

[0117] in, This indicates the current grid electricity price. Indicates vehicle exist Normalized state of charge at time, , Indicates vehicle The goal is to normalize the state of charge. , Indicates vehicle The remaining time of stay, This indicates the current electric vehicle charging load under the power distribution network. This is a globally shared feature, representing the electric vehicle charging load predicted for the next moment based on historical data, allowing each agent to adjust its current charging power in advance. Indicates vehicle exist Normalized charging load request at time, Indicates vehicle exist The actual charging load at that time Indicates vehicle The rated maximum charging power,

[0118] Step 2: Construct the reward function and constraints for the orderly charging reinforcement learning environment of electric vehicles in residential areas based on the predicted load.

[0119] The reward function for the electric vehicle charging process is defined as:

[0120] (8)

[0121] in, For the total reward function, Reward users for charging costs. Rewards for peak shaving and valley filling projects in power distribution networks. , For each corresponding item, the weight coefficient is... This represents the number of electric vehicles in the current power distribution network. Maximum charging power for electric vehicles For time intervals, The average value of the load curve.

[0122] The constraints of the electric vehicle charging process are defined as follows:

[0123] (9)

[0124] in, This refers to the rated power of the transformer. As a penalty factor for predicted load, the current maximum allowable total power is reduced when the predicted charging load is high at the next moment.

[0125] In S5, a multi-agent ordered charging policy network with shared parameters is trained using a reinforcement learning environment, including:

[0126] Step 1: Using a reinforcement learning environment, train and update the Actor and Critic networks with shared parameters based on the multi-agent deep deterministic policy gradient algorithm.

[0127] When the number of samples in the experience replay pool reaches the batch requirement, a small batch of samples is sampled from it. Each evaluation network is updated sequentially by minimizing the temporal difference loss, the shared policy network is updated by the policy gradient, and the target network is updated using a soft update method to solve for the optimal policy network.

[0128] The formula for calculating the policy gradient of an Actor network with shared parameters is:

[0129] (10)

[0130] in, Actor network parameters The gradient is used to derive the agent's gradient. The Actor network is being updated. and For each intelligent agent Observation and actions For the current Critic network, For the current network of the Actor, Represents status information. To draw samples from the experience replay pool Expected value This is an experience replay pool, storing experience tuples.

[0131] The formula for updating the loss function of the Critic network is as follows:

[0132] (11)

[0133] in, For intelligent agents The Critic network loss function value, The time-difference objective is used to guide the update of the current Q value. For intelligent agents Instant rewards As a discount factor, For the Critic target network, This represents the global state at the next moment. For the next moment's action,

[0134] Step 2: Train and update the target Actor network and Critic network based on the multi-agent deep deterministic policy gradient algorithm.

[0135] The soft update calculation formulas for the target Actor network and Critic network are as follows:

[0136] (12)

[0137] in, The update rate for soft target parameters is typically a relatively small value that satisfies... , and The parameters for the Actor training network and the target network are respectively. and These are the parameters for the Critic training network and the target network, respectively.

[0138] Example: The training process for the above-mentioned agent is shown in the attached diagram. Figure 2 As shown in the figure, the proposed ordered charging strategy based on the shared parameter multi-agent deep deterministic policy gradient algorithm is compared with the unordered charging, time-series prediction curve, DDQN, and DDPG algorithms. Figure 3As shown. Compared to disordered charging and time-series prediction curves, all scheduling algorithms can achieve a certain degree of peak shaving and valley filling. Among them, the strategy proposed in this invention reduces the load from a disordered 230.54kW to 180.48kW during peak hours and increases the load from a disordered 21.10kW to 51.04kW during off-peak hours. The peak reduction and valley filling effects are better than the DDQN and DDPG algorithms, thus achieving the peak shaving and valley filling goal more evenly.

[0139] In S6, a residential area electric vehicle orderly charging scheme is generated using real-time load forecasting and an optimal policy network. The specific process is as follows: Based on real-time load forecasting, the scheme is embedded in reinforcement learning environment modeling. The above steps of environment interaction, priority sampling, network parameter updates, and target network soft updates are repeatedly executed until the preset number of training rounds is completed. The decision-making behavior of each agent is gradually optimized, the Q-value evaluation accuracy of the Critic network continuously improves, and the action sequence output by the Actor network gradually approaches the globally optimal orderly charging strategy. After training, the optimal parameters of the Actor network are saved. Without real-time communication and data sharing, each charging pile calls the network to output a charging power request, generating the optimal residential area orderly charging scheme, denoted as:

[0140] (13)

[0141] in, Indicates vehicle exist The optimal charging power at any given time. These are the optimal parameters for the Actor network. For vehicles exist Local observation at a given moment.

[0142] The method, based on the charging load prediction results, regulates the charging power of each charging pile in the residential area to achieve orderly and optimized management of electric vehicle charging.

[0143] A second aspect of this invention relates to a multi-agent-based residential area charging load prediction and orderly charging system. The system includes a data preprocessing module, a load prediction module, a charging environment modeling module, and an orderly charging module. The data preprocessing module acquires historical charging load data of residential charging piles from the metering equipment and management system of electric vehicle charging stations, performs reversible instance normalization based on a lightweight Moment pre-trained model, and repairs missing values ​​and outliers in the corrected data. The load prediction module converts the corrected one-dimensional load data to a two-dimensional space, extracts multi-scale time-series features, obtains a periodic feature sequence, and constructs a charging load prediction system. The load prediction network is trained using periodic feature sequences and outputs predicted load. The charging environment modeling module is used to construct the state space, action space, reward function, and dynamic power constraints of a Markov decision process based on the output predicted load, real-time electricity price, state of charge of each electric vehicle, and transformer rated power, thus forming an orderly charging environment for electric vehicles in residential areas. The orderly charging module uses a reinforcement learning environment and a multi-agent deep deterministic policy gradient algorithm with shared parameters to train the model, continuously update the network parameters, save the optimal policy network after training, output the optimal charging strategy, and generate an orderly charging scheme for each vehicle at each time period.

[0144] Those skilled in the art will understand that embodiments of the present invention can be provided as methods, systems, or computer program products. Therefore, the present invention can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, the present invention can take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0145] This invention is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of the invention. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, generate instructions for implementing the flowchart illustrations and / or block diagrams. Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.

[0146] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and / or boxes Figure 1 The function specified in one or more boxes.

[0147] These computer program instructions may also be loaded onto a computer or other programmable data processing equipment to cause a series of operational steps to be performed on the computer or other programmable equipment to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable equipment for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.

[0148] The above description is only a preferred embodiment of the present invention. It should be noted that for those skilled in the art, several improvements and modifications can be made without departing from the principle of the present invention, and these improvements and modifications should also be considered within the scope of protection of the present invention.

Claims

1. A method for predicting residential charging load and orderly charging based on multi-agent systems, characterized in that, The method includes the following steps: S1. Obtain charging pile load data and correct missing and outlier values; S2. Convert the corrected one-dimensional load data to a two-dimensional space, extract multi-scale time series features, and obtain a periodic feature sequence. S3. Construct a charging load prediction network and train it using periodic feature sequences; S4. Construct a reinforcement learning environment for orderly charging of electric vehicles based on a load prediction network; S5. Utilize a reinforcement learning environment to train a multi-agent ordered charging policy network with shared parameters; S6. Utilize real-time load forecasting and optimal strategy networks to generate an orderly charging scheme for electric vehicles in residential areas.

2. The method for predicting and orderly charging in residential areas based on multi-agent systems as described in claim 1, characterized in that, In step S1, the charging pile load data is obtained, and missing and outlier values ​​are corrected, including: Step 1: Collect historical charging load data of residential charging piles from the metering equipment and management system of electric vehicle charging stations, and represent it as follows: ,in Indicates at a point in time Load data collected in real time This indicates the length of the time series. The value of each sample point represents the overall load of the charging station at the sampling time, and the unit is kilowatts (kW). Step 2: Perform missing value repair and outlier correction based on the lightweight Moment pre-trained model. First, the original historical charging load data is reversibly instantiated, denoted as: (1) in, This represents the normalized sequence values. and The mean and standard deviation of the sequence are used to obtain the normalized sequence. Then, a Masking operation is performed to eliminate the distribution differences in amplitude and offset between different sequences. Patching is then used to segment the normalized sequence into non-overlapping subsequences of different lengths. These subsequences are mapped to embedding vectors by the Encoding module and superimposed with learnable absolute position codes to preserve temporal position information. Finally, the Reconstruction module completes the sequence reconstruction, denoted as: (2) in, Indicates at a point in time A one-dimensional time series, This represents the overall mapping function of the Moment pre-trained model, which includes embedding, encoding, and decoding operations, thus yielding a one-dimensional time series output after preprocessing. .

3. The method for predicting residential charging load and orderly charging based on multi-agent systems as described in claim 1, characterized in that, In step S2, the corrected one-dimensional load data is transformed into a two-dimensional space, and multi-scale time-series features are extracted to obtain a periodic feature sequence, including: Step 1: Convert the corrected one-dimensional load data to two-dimensional space and capture the two-dimensional time variation. The main periods of the input one-dimensional time series data are identified by Fast Fourier Transform. Based on this, the time series data is reconstructed into a two-dimensional tensor by stacking the periods, thus capturing the two-dimensional time variation features in the time series data, denoted as: (3) in, express The average amplitude intensity of each frequency component Indicates averaging. Indicates amplitude calculation, Indicates the Fast Fourier Transform. Represents a two-dimensional tensor. Indicates according to the first The group periodic frequency reconstructs the original sequence. This indicates a zero-padding operation. The number of periods identified by the Fast Fourier Transform. , For the first The length of each cycle, For the first The number of rows in each cycle, This represents the feature tensor output after extracting two-dimensional temporal variation features, whose dimension is the same as the input. This indicates that features are captured using dual-scale convolution kernels. Step 2: After capturing the changing features, the dimensionality of the two-dimensional data is reduced. Tensors at each scale are weighted based on amplitude to aggregate and capture multi-scale temporal features, outputting a periodic feature sequence, denoted as: (4) in, This represents the one-dimensional data obtained after dimensionality reduction. This represents the truncation and zeroing operation. This represents converting a two-dimensional tensor back to one dimension. This represents the periodic feature sequence of the output. Indicates the amplitude according to each frequency of the signal. For one-dimensional data Weighted normalization, the final output time series is .

4. The method for predicting residential charging load and orderly charging based on multi-agent systems as described in claim 1, characterized in that, In step S3, a charging load prediction network is constructed and trained using periodic feature sequences, including: Step 1: During training, the periodic feature sequence is fed into the constructed charging load prediction network, and the output hidden state sequence is denoted as: (5) in, , For forward prediction networks in , The hidden state at all times , For backward prediction networks in , The hidden state at all times To predict the unit computation function of the network, , To predict the weight matrix of the network, This is the bias matrix of the output layer. For charging load prediction networks in The final output at any given moment, Step 2: For the output hidden state sequence, extract the last, mean, and maximum pooling features respectively to construct a global sequence vector. This vector is then mapped through a fully connected layer to output the load prediction value, denoted as: (6) in, This is the global sequence vector after concatenating pooled features. , and These represent bottom-end pooling, mean pooling, and maximum pooling, respectively. For charging load prediction networks in Output at any moment This indicates taking the maximum value element by element. This indicates the predicted total charging load at the next moment. This represents the weight matrix of the fully connected layer. This is a bias term.

5. The method for predicting and orderly charging in residential areas based on multi-agent systems as described in claim 1, characterized in that, In step S4, a reinforcement learning environment for orderly charging of electric vehicles is constructed based on the load prediction network, including: Step 1: Model the orderly charging problem of electric vehicles as a Markov decision process, and construct the state space and action space of reinforcement learning for orderly charging of electric vehicles in residential areas based on the predicted load. The state space and action space of the electric vehicle charging process are defined as follows: (7) in, This indicates the current grid electricity price. Indicates vehicle exist Normalized state of charge at time, , Indicates vehicle The target is to normalize the state of charge. , Indicates vehicle The remaining time of stay, This indicates the current electric vehicle charging load under the power distribution network. This is a globally shared feature, representing the electric vehicle charging load predicted for the next moment based on historical data, allowing each agent to adjust its current charging power in advance. Indicates vehicle exist Normalized charging load request at time, Indicates vehicle exist The actual charging load at that time Indicates vehicle The rated maximum charging power, Step 2: Construct the reward function and constraints for the orderly charging reinforcement learning environment of electric vehicles in residential areas based on the predicted load. The reward function for the electric vehicle charging process is defined as: (8) in, For the total reward function, Reward users for charging costs. Rewards for peak shaving and valley filling projects in power distribution networks. , For each corresponding item, the weight coefficient is... This represents the number of electric vehicles in the current power distribution network. Maximum charging power for electric vehicles For time intervals, The average value of the load curve. The constraints of the electric vehicle charging process are defined as follows: (9) in, This refers to the rated power of the transformer. As a penalty factor for predicted load, the current maximum allowable total power is reduced when the predicted charging load is high at the next moment.

6. The method for predicting residential charging load and orderly charging based on multi-agent systems as described in claim 1, characterized in that, In step S5, a multi-agent ordered charging policy network with shared parameters is trained using a reinforcement learning environment, including: Step 1: Using a reinforcement learning environment, train and update the Actor and Critic networks with shared parameters based on the multi-agent deep deterministic policy gradient algorithm. When the number of samples in the experience replay pool reaches the batch requirement, a small batch of samples is sampled from it. Each evaluation network is updated sequentially by minimizing the temporal difference loss, the shared policy network is updated by the policy gradient, and the target network is updated using a soft update method to solve for the optimal policy network. The formula for calculating the policy gradient of an Actor network with shared parameters is: (10) in, Actor network parameters The gradient is used to derive the agent's gradient. The Actor network is being updated. and For each intelligent agent Observation and actions For the current Critic network, For the current network of the Actor, Represents status information. To draw samples from the experience replay pool Expected value This is an experience replay pool, storing experience tuples. The formula for updating the loss function of the Critic network is as follows: (11) in, For intelligent agents The Critic network loss function value, The time-difference objective is used to guide the update of the current Q value. For intelligent agents Instant rewards As a discount factor, For the Critic target network, This represents the global state at the next moment. For the next moment's action, Step 2: Train and update the target Actor network and Critic network based on the multi-agent deep deterministic policy gradient algorithm. The soft update calculation formulas for the target Actor network and Critic network are as follows: (12) in, The update rate for soft target parameters is typically a relatively small value that satisfies... , and The parameters for the Actor training network and the target network are respectively. and These are the parameters for the Critic training network and the target network, respectively.

7. The method for predicting residential charging load and orderly charging based on multi-agent systems as described in claim 1, characterized in that, In step S6, a residential area electric vehicle orderly charging scheme is generated using real-time load forecasting and an optimal policy network. The specific process is as follows: Based on real-time load forecasting, the scheme is embedded in reinforcement learning environment modeling. The environmental interaction, priority sampling, network parameter updates, and target network soft updates in the above steps are repeatedly executed until the preset number of training rounds are completed. The decision-making behavior of each agent is gradually optimized, the Q-value evaluation accuracy of the Critic network is continuously improved, and the action sequence output by the Actor network gradually approaches the globally optimal orderly charging strategy. After training, the optimal parameters of the Actor network are saved. Without real-time communication and data sharing, each charging pile calls the charging power request output by the network to generate the optimal residential area orderly charging scheme, denoted as: (13) in, Indicates vehicle exist The optimal charging power at any given time. These are the optimal parameters for the Actor network. For vehicles exist Local observation at a given moment.

8. A residential area charging load prediction and orderly charging system based on multi-agent technology, characterized in that, This method is used to implement the multi-agent-based residential area charging load prediction and orderly charging method according to any one of claims 1-7. The system includes a data preprocessing module, a load forecasting module, a charging environment modeling module, and an orderly charging module, wherein... The data preprocessing module is used to obtain historical charging load data of residential charging piles from the metering equipment and management system of electric vehicle charging stations, perform reversible instance normalization based on the lightweight Moment pre-trained model, and perform missing value repair and outlier correction on the corrected data. The load prediction module is used to convert the corrected one-dimensional load data into a two-dimensional space, extract multi-scale time-series features, obtain periodic feature sequences, construct a charging load prediction network, train it using the periodic feature sequences, and output the predicted load. The charging environment modeling module is used to construct the state space, action space, reward function, and dynamic power constraints of the Markov decision process based on the output predicted load, real-time electricity price, state of charge of each electric vehicle, rated power of the transformer, and other information, so as to form an orderly charging environment for electric vehicles in residential areas. The ordered charging module utilizes a reinforcement learning environment to train the model based on a multi-agent deep deterministic policy gradient algorithm with shared parameters, continuously updates the network parameters, saves the optimal policy network after training, outputs the optimal charging policy, and generates an ordered charging scheme for each vehicle at each time period.

9. A storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by the processor, it implements the multi-agent-based residential area charging load prediction and orderly charging method as described in any one of claims 1 to 7.

Images