Charging pile load prediction method and system based on multi-head attention mechanism

Abstract

The invention relates to a charging pile load prediction method and system based on a multi-head attention mechanism, and the method comprises the following steps: collecting load related data of a charging station, and carrying out the preprocessing of the load related data; extracting load features, spatial features and time features of the load-related data, and fusing the load features, the spatial features and the time features to obtain load-related features; constructing an attention extraction model according to an attention mechanism, and inputting the load-related features into the attention extraction model to extract spatial context cross attention, temporal context cross attention, temporal self-attention and spatial self-attention; spatial self-attention is input into a full-connection layer for mapping to obtain a charging pile load prediction result, and a multi-head cross attention mechanism is introduced, so that the problem of complexity of space-time dependent modeling is solved.

Description

Technical Field

[0001] The present invention relates to a charging pile load prediction method and system based on a multi-head attention mechanism, belonging to the technical field of power load prediction. Background Art

[0002] In recent years, with the rapid development of the electric vehicle (EV) industry, EV charging load forecasting technology has gradually become an important tool for optimizing power system management and improving energy distribution efficiency. The large-scale popularization of electric vehicles has not only had a profound impact on the transportation industry, but also posed new challenges to the stability of the power grid and power load management. Existing EV charging load forecasting technologies mainly rely on a variety of data-driven methods and simulation technologies to achieve accurate prediction of complex power loads. In early research in this field, methods based on Monte Carlo simulation are more common. Monte Carlo simulation simulates the load fluctuation of electric vehicles connected to the power grid by setting multiple input variables (such as electric vehicle type, battery capacity, charging rate, etc.) and randomly sampling multiple possible scenarios. This method can comprehensively consider the changes in charging demand under different circumstances and provide a technical reference for the impact of electric vehicles on the power grid. However, the prediction accuracy of this type of method is more dependent on the setting of the simulation scenario, and it is difficult to cope with the complex and dynamic charging environment in actual operation, so it has certain limitations in practical applications.

[0003] The patent document with the patent number "CN118487264A" has developed a method and system for electric vehicle charging station load forecasting based on multi-source feature fusion. The problem with this system is that a complex spatial relationship diagram of charging stations, time series signals and load metadata are established through the data acquisition module, which requires a lot of computing resources and storage space. The GAT-LSTM module is adopted, which is a complex model that combines the graph attention network (GAT) and the long short-term memory network (LSTM) to extract spatiotemporal dynamic features. However, the complexity of this model may lead to its poor interpretability and difficulty in intuitively understanding the source of the prediction results. The spatiotemporal dynamic features and the features of the load metadata are spliced ​​in the fusion layer. This fusion method may ignore the potential correlation and mutual influence between different features. Summary of the invention

[0004] In order to solve the above problems existing in the prior art, the present invention proposes a charging pile load prediction method and system based on a multi-head attention mechanism.

[0005] The technical solution of the present invention is as follows:

[0006] On the one hand, the present invention provides a charging pile load prediction method based on a multi-head attention mechanism, comprising the following steps:

[0007] Collecting load-related data of the charging station, the load-related data includes charging load historical data, charging load geographic space information and charging load time series information, and preprocessing the load-related data, the preprocessing includes regional classification, filtering outliers, missing value filling and normalization processing;

[0008] Extract the load characteristics, spatial characteristics and time characteristics of the load-related data, and fuse the load characteristics, spatial characteristics and time characteristics to obtain the load-related characteristics;

[0009] An attention extraction model is constructed based on the attention mechanism, and the load-related features are input into the attention extraction model to extract spatial context cross-attention, temporal context cross-attention, temporal self-attention, and spatial self-attention;

[0010] The spatial self-attention is input into the fully connected layer and mapped to obtain the charging pile load prediction result.

[0011] As a preferred implementation, the missing value filling adopts linear interpolation:

[0012]

[0013] Among them, Q x represents the supplementary load-related data, Q1 represents the load-related data at time t1, Q2 represents the load-related data at time t2, and t x Indicates the time point of the missing value, t1 indicates t x t points before t, t2 represents t x The last t time points of ;

[0014] The normalization method is:

[0015]

[0016] Among them, X ′ represents the load-related data after normalization, X represents the load-related data before normalization, and X max Indicates the maximum value of load-related data, X min Indicates the minimum value of load-related data.

[0017] As a preferred implementation, the method for extracting the load characteristics is:

[0018] Let charging load history data X elc The first dimension is T, the second dimension is N, and the third dimension is C;

[0019] Use Multi-layer Perceptron MLP to extract charging load history data X elc Load characteristics E elc ;

[0020] The spatial feature extraction method is:

[0021] Let charging load geospatial information X s The first dimension is T, the second dimension is N, and the third dimension is 2;

[0022] Extracting charging load geospatial information using multi-layer perceptron (MLP) s The spatial characteristics of E s ;

[0023] Time feature E t The extraction method is:

[0024] Let the charging load time series information X dow , X tod The first dimension is T, the second dimension is N, and the third dimension is 6;

[0025] According to the preset timestamp X ts Use multi-layer perceptron MLP to extract charging load time series information X dow , X tod The time characteristics of E t ;

[0026] E elc 、E s and E t Fusion is performed to obtain load-related features:

[0027] H = concat(E elc ,E s ,E t );

[0028] Among them, H represents the load-related features, and concat() represents the feature fusion function.

[0029] As a preferred implementation, the method for extracting spatial context cross attention is:

[0030] Slice the H to obtain the query vector Key Vector and the value vector

[0031]

[0032] Among them, H[…,slice(s)] means slicing H to extract spatial features, and H[…,slice(elc)] means slicing H to extract load features. represents the preset coefficient matrix, (s) represents the identifier of the spatial feature, (elc) represents the identifier of the load feature, h represents the hth attention head, (Qs), (Kelc), (Velc) represent identifier symbols;

[0033] According to the obtained as well as Calculate the attention score CrossAttention(s,elc)(h) of the query load key-value pair:

[0034]

[0035] Among them, softmax() means converting the score into a probability distribution function, and D means Dimensions or The dimension of , (h) represents h, express The transpose of

[0036] CrossAttention(s,elc)(h) is multi-headed concatenated to obtain the first multi-head attention MHCA(a,b)(X):

[0037]

[0038] Among them, ∥ represents the concatenation operation of several attention heads. It means that the outputs of several attention heads are concatenated in the order of the attention heads. O represents the preset projection matrix, (X) represents H, and a and b are used to represent features;

[0039] The first multi-head attention MHCA(a,b)(X) is then fed through a feedforward network and normalized to obtain spatial load attention With load spatial attention Then and And H is connected with residual to get spatial context cross attention

[0040] As a preferred implementation, the method for extracting temporal context cross attention is:

[0041] The Perform a transposition operation to obtain the transposed spatial context cross attention

[0042] Let the even time step pos(t,2d) be expressed as:

[0043]

[0044] Let the odd time step pos(t,2d+1) be expressed as:

[0045]

[0046] Among them, 2d represents an even-numbered dimension, 2d+1 represents an odd-numbered dimension, t represents the position in the time series, and D represents the total number of dimensions. Represents a coded value;

[0047] Concatenate the even-numbered time steps with the odd-numbered time steps to obtain the final position code;

[0048] The final position encoding and Add together to get spatial context cross-temporal position attention

[0049] Will Input the second multi-head attention MHCA(a,b)(X):

[0050]

[0051] Among them, X represents

[0052] The second multi-head attention MHCA (a, b) (X) is processed by the feedforward network and normalization, and the parameter list (a, b) (X) replaces a with the time feature (t) and b with the load feature (elc) to obtain the time load attention Replace a in the parameter list (a, b)(X) with the load feature (elc) and b with the time feature (t) to obtain the load-time attention

[0053] Will and Perform residual connection and concatenate back to H to get temporal context cross attention

[0054] As a preferred implementation, the method for extracting temporal self-attention is:

[0055] Will Input the third multi-head attention MHA(X):

[0056]

[0057] Among them, X represents W O Represents the preset projection matrix, Q h_t represents the temporal self-attention query vector, K h_t represents the temporal self-attention key vector, V h_tRepresents the temporal self-attention value vector, Attention(Q h_t ,K h_t ,V h_t ) represents the score of temporal self-attention weight;

[0058] The third multi-head attention MHA(X) is processed by the feedforward network and normalized Perform residual connection to obtain temporal self-attention

[0059] As a preferred implementation, the method for extracting spatial self-attention is:

[0060] Will Transpose to get the transposed temporal self-attention

[0061] Enter the fourth multi-head attention MHA(X):

[0062]

[0063] Among them, Attention(Q h_s ,K h_s ,V h_s ) The score of spatial self-attention weight, Q h_s represents the spatial self-attention query vector, K h_s represents the spatial self-attention key vector, V h_s Represents the spatial self-attention value vector;

[0064] The fourth multi-head attention MHA(X) is processed by the feedforward network and normalized with Perform residual connection to obtain spatial self-attention

[0065] On the other hand, the present invention provides a charging pile load prediction system based on a multi-head attention mechanism, comprising:

[0066] Data collection module: collects load-related data of charging stations, including charging load historical data, charging load geographic space information, and charging load time series information, and preprocesses the load-related data, including regional classification, outlier filtering, missing value filling, and normalization processing;

[0067] Feature fusion module: extracts load features, spatial features and time features of load-related data, and fuses the load features, spatial features and time features to obtain load-related features;

[0068] Attention extraction module: construct an attention extraction model based on the attention mechanism, input the load-related features into the attention extraction model to extract spatial context cross-attention, temporal context cross-attention, temporal self-attention and spatial self-attention;

[0069] Prediction module: Map the spatial self-attention input into the fully connected layer to obtain the charging pile load prediction result.

[0070] The present invention has the following beneficial effects:

[0071] The present invention performs comprehensive preprocessing on the load-related data of the charging station, including regional classification, filtering outliers, filling missing values ​​and normalization. These steps can ensure the quality and consistency of the data, and provide a reliable basis for subsequent feature extraction and model training. Not only the load features are extracted, but also the spatial features and temporal features are considered, and these features are fused to obtain load-related features. This refined feature extraction method can more comprehensively reflect the complexity and dynamics of the charging load. An attention extraction model is constructed based on the attention mechanism, which can realize the extraction of spatial context cross-attention, temporal context cross-attention, temporal self-attention and spatial self-attention. This model structure can capture the correlation and mutual influence between different features, thereby improving the accuracy and stability of the prediction. BRIEF DESCRIPTION OF THE DRAWINGS

[0072] Figure 1 The present invention is a flowchart for implementing the method. DETAILED DESCRIPTION

[0073] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0074] It should be understood that the step numbers used in this document are only for convenience of description and are not intended to limit the order in which the steps are executed.

[0075] It should be understood that the terms used in the present specification are only for the purpose of describing specific embodiments and are not intended to limit the present invention. As used in the present specification and the appended claims, unless the context clearly indicates otherwise, the singular forms "a", "an" and "the" are intended to include plural forms.

[0076] The terms “include” and “comprising” indicate the presence of described features, integers, steps, operations, elements and / or components, but do not exclude the presence or addition of one or more other features, integers, steps, operations, elements, components and / or combinations thereof.

[0077] The term "and / or" means and includes any and all possible combinations of one or more of the associated listed items.

[0078] Embodiment 1:

[0079] See also Figure 1 The present invention provides a charging pile load prediction method based on a multi-head attention mechanism, comprising the following steps:

[0080] Collecting load-related data of the charging station, the load-related data includes charging load historical data, charging load geographic space information and charging load time series information, and preprocessing the load-related data, the preprocessing includes regional classification, filtering outliers, missing value filling and normalization processing;

[0081] Extract the load characteristics, spatial characteristics and time characteristics of the load-related data, and fuse the load characteristics, spatial characteristics and time characteristics to obtain the load-related characteristics;

[0082] An attention extraction model is constructed based on the attention mechanism, and the load-related features are input into the attention extraction model to extract spatial context cross-attention, temporal context cross-attention, temporal self-attention, and spatial self-attention;

[0083] The spatial self-attention is input into the fully connected layer and mapped to obtain the charging pile load prediction result.

[0084] As a preferred implementation, the missing value filling adopts linear interpolation:

[0085]

[0086] Among them, Q x represents the supplementary load-related data, Q1 represents the load-related data at time t1, Q2 represents the load-related data at time t2, and t x Indicates the time point of the missing value, t1 indicates t x t points before t, t2 represents t x The last t time points of ;

[0087] The normalization method is:

[0088]

[0089] Among them, X ′represents the load-related data after normalization, X represents the load-related data before normalization, and X max Indicates the maximum value of load-related data, X min Indicates the minimum value of load-related data.

[0090] As a preferred implementation, the method for extracting the load characteristics is:

[0091] Let charging load historical data X elc The first dimension of is T, which represents the number of discrete time intervals, the second dimension is N, which represents the number of spatial nodes corresponding to the location of the charging station, and the third dimension is C, which represents the number of attributes of the charging pile, that is, a tensor of the number of channels of the charging load data;

[0092] Use Multi-layer Perceptron MLP to extract charging load history data X elc Load characteristics E elc ;

[0093] The spatial feature extraction method is:

[0094] Let charging load geospatial information X s The first dimension of is T, which represents the number of discrete time intervals, the second dimension is N, which represents the number of spatial nodes corresponding to the location of the charging station, and the third dimension is 2, which represents a tensor of the longitude and latitude of the charging station;

[0095] Extracting charging load geospatial information using multi-layer perceptron (MLP) s The spatial characteristics of E s ;

[0096] Time feature E t The extraction method is:

[0097] Let the charging load time series information X dow , X tod The first dimension is T, which represents the number of discrete time intervals, the second dimension is N, which represents the number of spatial nodes corresponding to the location of the charging station, and the third dimension is 6, which represents the feature dimension of the timestamp, that is, a tensor of year, month, day, hour, minute, and second;

[0098] X dow Indicates a day of the week, X tod Indicates a time of day;

[0099] According to the preset timestamp X ts Use multi-layer perceptron MLP to extract charging load time series information X dow , X tod The time characteristics of E t ;

[0100] Eelc 、E s and E t Fusion is performed to obtain load-related features:

[0101] H = concat(E elc ,E s ,E t );

[0102] Among them, H represents the load-related features, and concat() represents the feature fusion function.

[0103] The core idea of ​​the self-attention mechanism is to learn different spatiotemporal dependency patterns through multiple attention heads, and each head independently focuses on the correlation between different charging stations and charging loads. Taking the space-charging load cross extraction as an example, the spatial feature representation E s As a query, the charging load characteristic represents E elc As a key-value pair, it is input into the cross attention mechanism. Then the charging load feature representation E elc is the query, the spatial feature representation E s As a key-value pair, it is input into the cross attention mechanism. By calculating the dot product similarity between the query and the key, the attention weight is generated to capture the correlation between the spatial feature and the charging load. Here we only explain the spatial feature representation E s As a query, the charging load characteristic represents E elc As a key-value calculation process, the charging load characteristic represents E elc is the query, the spatial feature representation E s The calculation process of the key-value pair is the same, where the corresponding variables can be replaced.

[0104] As a preferred implementation, the method for extracting spatial context cross attention is:

[0105] Slice the H to obtain the query vector Key Vector and the value vector

[0106]

[0107] Among them, H[…,slice(s)] means slicing H to extract spatial features, and H[…,slice(elc)] means slicing H to extract load features. represents the preset coefficient matrix, (s) represents the identifier of the spatial feature, (elc) represents the identifier of the load feature, h represents the hth attention head, (Qs), (Kelc), (Velc) represent identifier symbols, ... represent omission symbols, and since h includes t, s, and elc, it plays the role of ignoring t and elc, so that only s is taken;

[0108] According to the obtained as well as Calculate the attention score CrossAttention(s,elc)(h) of the query load key-value pair:

[0109]

[0110] Among them, softmax() means converting the score into a probability distribution function, and D means Dimensions or The dimension of , (h) represents h, express The transpose of

[0111] CrossAttention(s,elc)(h) is multi-headed concatenated to obtain the first multi-head attention MHCA(a,b)(X):

[0112]

[0113] Among them, ∥ represents the concatenation operation of several attention heads. It means that the outputs of several attention heads are concatenated in the order of the attention heads. O represents the preset projection matrix, (X) represents H, and a and b are used to represent features;

[0114] The first multi-head attention MHCA(a,b)(X) is then fed through a feedforward network and normalized to obtain spatial load attention With load spatial attention Then and And H is connected with residual to get spatial context cross attention

[0115] The variables a and b represent different features respectively. For example, in the formula CrossAttention(s,elc)(h), the spatial feature (s) is used as a and the load feature (elc) is used as b, that is, a=s, b=elc.

[0116] Let a represent s and b represent elc to get Similarly, using a to represent elc and b to represent s, we can get

[0117] In order to retain the information of the original input features, the output of the multi-head cross attention is added to the part of the original input features corresponding to the target modality, and a residual connection operation is completed to form the initial feature fusion result. Subsequently, the fusion result is subjected to a layer normalization operation, and the normalized feature representation is obtained by adjusting the distribution of the features. Next, the normalized features are input into the feedforward neural network. The feedforward neural network consists of two layers of fully connected networks, which can further extract high-order features from the input features and enhance the expression of complex feature relationships. The output result of the feedforward neural network is added again to the normalized feature representation of the previous stage for residual connection, thereby generating a fusion representation containing high-order feature information. In order to further optimize the distribution of features, the result of the residual connection is subjected to another layer normalization operation. The normalization process further improves the stability and training efficiency of the features, and outputs the final normalized feature representation.

[0118] As a preferred implementation, the method for extracting temporal context cross attention is:

[0119] The Perform a transposition operation to obtain the transposed spatial context cross attention

[0120] Let the even time step pos(t,2d) be expressed as:

[0121]

[0122] Let the odd time step pos(t,2d+1) be expressed as:

[0123]

[0124] Among them, 2d represents an even-numbered dimension, 2d+1 represents an odd-numbered dimension, t represents the position in the time series, and D represents the total number of dimensions. Represents a coded value;

[0125] The position encoding is calculated using sine and cosine functions, applied to even and odd dimensions respectively. For even dimension 2d, the position encoding is calculated using the first formula; for odd dimension 2d+1, it is calculated using the second formula. Here, t represents the position in the sequence, d represents the dimension index, and D is the total number of dimensions. By using different sine and cosine functions, this method can generate unique vectors for each position in the sequence, and these vectors can well capture the relative position information in the sequence.

[0126] Concatenate the even-numbered time steps with the odd-numbered time steps to obtain the final position code;

[0127] The final position encoding and Add together to get spatial context cross-temporal position attention

[0128] Will Input the second multi-head attention MHCA(a,b)(X):

[0129]

[0130] Among them, X represents

[0131] The second multi-head attention MHCA (a, b) (X) is processed by the feedforward network and normalization, and the parameter list (a, b) (X) replaces a with the time feature (t) and b with the load feature (elc) to obtain the time load attention Replace a in the parameter list (a, b)(X) with the load feature (elc) and b with the time feature (t) to obtain the load-time attention

[0132] Will and Perform residual connection and concatenate back to H to get temporal context cross attention

[0133] As a preferred implementation, the method for extracting temporal self-attention is:

[0134] Will Input the third multi-head attention MHA(X):

[0135]

[0136] Among them, X represents W O Represents the preset projection matrix, Q h_t represents the temporal self-attention query vector, K h_t represents the temporal self-attention key vector, V h_t Represents the temporal self-attention value vector, Attention(Q h_t ,K h_t ,V h_t ) represents the score of temporal self-attention weight;

[0137] The third multi-head attention MHA(X) is processed by the feedforward network and normalized Perform residual connection to obtain temporal self-attention

[0138] As a preferred implementation, the method for extracting spatial self-attention is:

[0139] Will Transpose to get the transposed temporal self-attention

[0140] Enter the fourth multi-head attention MHA(X):

[0141]

[0142] Among them, Attention(Q h_s ,K h_s ,V h_s ) The score of spatial self-attention weight, Q h_s represents the spatial self-attention query vector, K h_s represents the spatial self-attention key vector, V h_s Represents the spatial self-attention value vector;

[0143] The fourth multi-head attention MHA(X) is processed by the feedforward network and normalized with Perform residual connection to obtain spatial self-attention

[0144] By inputting spatial features as queries and charging load features as key-value pairs into the cross-attention mechanism, the dot product similarity between the query and the key is calculated to generate attention weights. This method can capture the correlation between spatial features and charging load, so as to more accurately understand the impact of spatial distribution on charging load. The extraction of spatial context cross-attention takes into account the interaction between spatial features and load features. This refined feature fusion method helps to improve the accuracy of charging pile load prediction.

[0145] The temporal context cross attention captures the dynamics of temporal features by considering the encoding values ​​at different positions in the time series (e.g., even-numbered time steps and odd-numbered time steps). This dynamics is crucial for understanding the variation of charging load over time. By extracting temporal features and load features with cross attention, the expressiveness of temporal features can be further enhanced, making them better reflect the temporal distribution characteristics of charging load.

[0146] Temporal self-attention captures the dependencies between temporal features by calculating the dot product similarity between the query vector, key vector, and value vector within the temporal feature. This dependency helps understand the mutual influence of charging load at different time points. By considering the intrinsic relationship between temporal features, the extraction of temporal self-attention helps improve the stability of prediction results and reduce prediction errors caused by time series fluctuations.

[0147] Spatial self-attention emphasizes the importance of spatial features in charging pile load prediction by calculating the dot product similarity between the query vector, key vector, and value vector within the spatial feature. This importance is reflected in the direct impact of spatial features on the charging load distribution. Through the extraction of spatial self-attention, spatial feature information can be more effectively utilized and its contribution in the prediction model can be improved, thereby further improving the accuracy and reliability of the prediction.

[0148] Embodiment 2:

[0149] The present invention provides a charging pile load prediction system based on a multi-head attention mechanism, comprising:

[0150] Data collection module: collects load-related data of charging stations, including charging load historical data, charging load geographic space information, and charging load time series information, and preprocesses the load-related data, including regional classification, outlier filtering, missing value filling, and normalization processing;

[0151] Feature fusion module: extracts load features, spatial features and time features of load-related data, and fuses the load features, spatial features and time features to obtain load-related features;

[0152] Attention extraction module: construct an attention extraction model based on the attention mechanism, input the load-related features into the attention extraction model to extract spatial context cross-attention, temporal context cross-attention, temporal self-attention and spatial self-attention;

[0153] Prediction module: Map the spatial self-attention input into the fully connected layer to obtain the charging pile load prediction result.

[0154] In the embodiments of the present application, "at least one" refers to one or more, and "more than one" refers to two or more. "And / or" describes the association relationship of associated objects, indicating that three relationships may exist. For example, A and / or B can represent the existence of A alone, the existence of A and B at the same time, and the existence of B alone. Among them, A and B can be singular or plural. The character " / " generally indicates that the previous and next associated objects are in an "or" relationship. "At least one of the following" and similar expressions refer to any combination of these items, including any combination of single or plural items. For example, at least one of a, b and c can be represented by: a, b, c, a and b, a and c, b and c, or a and b and c, where a, b, c can be single or multiple.

[0155] Those of ordinary skill in the art will appreciate that the various units and algorithm steps described in the embodiments disclosed herein can be implemented in a combination of electronic hardware, computer software, and electronic hardware. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of this application.

[0156] Those skilled in the art can clearly understand that, for the convenience and brevity of description, the specific working processes of the systems, devices and units described above can refer to the corresponding processes in the aforementioned method embodiments and will not be repeated here.

[0157] In several embodiments provided in the present application, if any function is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present application, or the part that contributes to the prior art or the part of the technical solution, can be embodied in the form of a software product, which is stored in a storage medium and includes several instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to perform all or part of the steps of the method described in each embodiment of the present application. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (Read-Only Memory; hereinafter referred to as: ROM), random access memory (Random Access Memory; hereinafter referred to as: RAM), disk or optical disk, and other media that can store program codes.

[0158] The above descriptions are merely embodiments of the present invention and are not intended to limit the patent scope of the present invention. Any equivalent structure or equivalent process transformation made using the contents of the present invention specification and drawings, or directly or indirectly applied in other related technical fields, are also included in the patent protection scope of the present invention.

Claims

1. A charging pile load prediction method based on a multi-head attention mechanism, characterized in that: The following steps are involved: Collecting load-related data of the charging station, the load-related data includes charging load historical data, charging load geographic space information and charging load time series information, and preprocessing the load-related data, the preprocessing includes regional classification, filtering outliers, missing value filling and normalization processing; Extract the load characteristics, spatial characteristics and time characteristics of the load-related data, and fuse the load characteristics, spatial characteristics and time characteristics to obtain the load-related characteristics; An attention extraction model is constructed based on the attention mechanism, and the load-related features are input into the attention extraction model to extract spatial context cross-attention, temporal context cross-attention, temporal self-attention, and spatial self-attention; The spatial self-attention is input into the fully connected layer and mapped to obtain the charging pile load prediction result.

2. The charging pile load prediction method based on the multi-head attention mechanism according to claim 1 is characterized in that: The missing values ​​are filled using linear interpolation: Among them, Q x represents the supplementary load-related data, Q1 represents the load-related data at time t1, Q2 represents the load-related data at time t2, t x Indicates the time point of the missing value, t1 indicates t x t points before t, t2 represents t x The last t time points of ; The normalization method is: Among them, X ′ represents the load-related data after normalization, X represents the load-related data before normalization, and X max Indicates the maximum value of load-related data, X min Indicates the minimum value of load-related data.

3. The charging pile load prediction method based on multi-head attention mechanism according to claim 1 is characterized in that: The method for extracting the load characteristics is: Let charging load history data X elc The first dimension is T, the second dimension is N, and the third dimension is C; Use Multi-layer Perceptron MLP to extract charging load history data X elc Load characteristics E elc ; The spatial feature extraction method is: Let charging load geospatial information X s The first dimension is T, the second dimension is N, and the third dimension is 2; Extracting charging load geospatial information using multi-layer perceptron (MLP) s The spatial characteristics of E s ; Time feature E t The extraction method is: Let the charging load time series information X dow , X tod The first dimension is T, the second dimension is N, and the third dimension is 6; According to the preset timestamp X ts Use multi-layer perceptron MLP to extract charging load time series information X dow , X tod The time characteristics of E t ; E elc 、E s and E t Fusion is performed to obtain load-related features: H =concat(E elc ,AND s ,AND t ); Among them, H represents the load-related features, and concat() represents the feature fusion function.

4. The charging pile load prediction method based on multi-head attention mechanism according to claim 3 is characterized in that: The method for extracting the spatial context cross attention is: Slice the H to obtain the query vector Key Vector and the value vector Among them, H[..., slice(s)] means slicing H to extract spatial features, H[..., slice(elc)] means slicing H to extract load features, represents the preset coefficient matrix, (s) represents the identifier of the spatial feature, (elc) represents the identifier of the load feature, h represents the hth attention head, (Qs), (Kelc), (Velc) represent identifier symbols; According to the obtained as well as Calculate the attention score CrossAttention(s, elc)(h) of the query load key-value pair: Among them, softmax() means converting the score into a probability distribution function, and D means Dimensions or The dimension of , (h) represents h, express The transpose of CrossAttention(s,elc)(h) is multi-headed concatenated to obtain the first multi-head attention MHCA(a,b)(X): Among them, || represents the concatenation operation of several attention heads, It means that the outputs of several attention heads are concatenated in the order of the attention heads. O represents the preset projection matrix, (X) represents H, and a and b are used to represent features; The first multi-head attention MHCA(a, b)(X) is then fed through a feedforward network and normalized to obtain spatial load attention With load spatial attention Then and And H is connected with residual to get spatial context cross attention 5. The charging pile load prediction method based on multi-head attention mechanism according to claim 4 is characterized in that: The extraction method of temporal context cross attention is: The Perform a transposition operation to obtain the transposed spatial context cross attention Let the even time step pos(t, 2d) be expressed as: Let the odd time step pos(t, 2d+1) be expressed as: Among them, 2d represents an even-numbered dimension, 2d+1 represents an odd-numbered dimension, t represents the position in the time series, and D represents the total number of dimensions. Represents a coded value; Concatenate the even-numbered time steps with the odd-numbered time steps to obtain the final position code; The final position encoding and Add together to get spatial context cross-temporal position attention Will Input the second multi-head attention MHCA(a, b)(X): Among them, X represents The second multi-head attention MHCA (a, b) (X) is processed by the feedforward network and normalization, and the parameter list (a, b) (X) replaces a with the time feature (t) and b with the load feature (elc) to obtain the time load attention Replace a in the parameter list (a, b) (X) with the load feature (elc) and b with the time feature (t) to obtain the load-time attention Will and Perform residual connection and concatenate back to H to get temporal context cross attention 6. The charging pile load prediction method based on multi-head attention mechanism according to claim 5 is characterized in that: The method for extracting the temporal self-attention is: Will Input the third multi-head attention MHA(X): Among them, X represents W O Represents the preset projection matrix, Q h_t represents the temporal self-attention query vector, K h_t represents the temporal self-attention key vector, V h_t Represents the temporal self-attention value vector, Attention(Q h_t , K h_t , V h_t ) represents the score of temporal self-attention weight; The third multi-head attention MHA(X) is processed by the feedforward network and normalized Perform residual connection to obtain temporal self-attention 7. The charging pile load prediction method based on multi-head attention mechanism according to claim 6 is characterized in that: The extraction method of the spatial self-attention is: Will Transpose to get the transposed temporal self-attention Enter the fourth multi-head attention MHA(X): Among them, Attention(Q h_s , K h_s , V h_s ) The score of spatial self-attention weight, Q h_s represents the spatial self-attention query vector, K h_s represents the spatial self-attention key vector, V h_s Represents the spatial self-attention value vector; The fourth multi-head attention MHA(X) is processed by the feedforward network and normalized with Perform residual connection to obtain spatial self-attention 8. A charging pile load prediction system based on a multi-head attention mechanism, characterized in that: include: Data collection module: collects load-related data of charging stations, including charging load historical data, charging load geographic space information, and charging load time series information, and preprocesses the load-related data, including regional classification, outlier filtering, missing value filling, and normalization processing; Feature fusion module: extracts load features, spatial features and time features of load-related data, and fuses the load features, spatial features and time features to obtain load-related features; Attention extraction module: construct an attention extraction model based on the attention mechanism, input the load-related features into the attention extraction model to extract spatial context cross-attention, temporal context cross-attention, temporal self-attention and spatial self-attention; Prediction module: Map the spatial self-attention input into the fully connected layer to obtain the charging pile load prediction result.

9. The charging pile load prediction system based on multi-head attention mechanism according to claim 8 is characterized in that: In the data collection module, missing values ​​are filled using linear interpolation: Among them, Q x represents the supplementary load-related data, Q1 represents the load-related data at time t1, Q2 represents the load-related data at time t2, and t x Indicates the time point of the missing value, t1 indicates t x t points before t, t2 represents t x The last t time points of ; The normalization method is: Where X′ represents the load-related data after normalization, X represents the load-related data before normalization, and X max Indicates the maximum value of load-related data, X min Indicates the minimum value of load-related data.

10. The charging pile load prediction system based on multi-head attention mechanism according to claim 8, characterized in that: In the feature fusion module, the method for extracting load features is as follows: Let charging load history data X elc The first dimension is T, the second dimension is N, and the third dimension is C; Use Multi-layer Perceptron MLP to extract charging load history data X elc Load characteristics E elc ; The spatial feature extraction method is: Let charging load geospatial information X s The first dimension is T, the second dimension is N, and the third dimension is 2; Extracting charging load geospatial information using multi-layer perceptron (MLP) s The spatial characteristics of E s ; Time feature E t The extraction method is: Let the charging load time series information X dow , X tod The first dimension is T, the second dimension is N, and the third dimension is 6; According to the preset timestamp X ts Use multi-layer perceptron MLP to extract charging load time series information X dow , X tod The time characteristics of E t ; E elc 、E s and E t Fusion is performed to obtain load-related features: H =concat(E elc ,AND s ,AND t ); Among them, H represents the load-related features, and concat() represents the feature fusion function.

Images