Photovoltaic time series dynamic prediction method and system based on attention mechanism

Abstract

The application discloses the photovoltaic time sequence dynamic prediction method and system based on attention mechanism in the photovoltaic power generation prediction technical field. The method comprises the following steps: data preprocessing is carried out on the original photovoltaic time sequence data of a target region in a preset historical time period, and the preprocessed photovoltaic time sequence data is input into a trained photovoltaic time sequence dynamic prediction model; the preprocessed photovoltaic time sequence data is subjected to time sequence feature embedding to obtain an embedded feature vector; the embedded feature vector is input into an improved multi-head self-attention layer to perform dynamic attention calculation and competitive balance, and an optimized context feature vector is obtained; a feedforward neural network layer is used to perform nonlinear transformation and feature dimension integration on the optimized context feature vector, and a reinforced high-level feature vector is obtained; and an output layer is used to obtain a photovoltaic load prediction value sequence of a future specified time span. The application can improve the accuracy, stability and robustness of photovoltaic load prediction in a high-proportion photovoltaic access scenario.

Description

Technical Field

[0001] This invention relates to the field of photovoltaic power generation prediction technology, and in particular to a photovoltaic time-series dynamic prediction method and system based on an attention mechanism. Background Technology

[0002] As the global energy structure transitions towards cleaner and lower-carbon energy, photovoltaic (PV) power generation, as an important renewable energy source, is experiencing rapid and continuous growth in installed capacity and penetration. However, PV output exhibits significant intermittency, randomness, and volatility, posing serious challenges to power grid power balance, safe and stable operation, and economic dispatch. Against this backdrop, high-precision PV time-series forecasting—that is, predicting the net load of a specific region in conjunction with the characteristics of PV power generation—has become a key technology for ensuring reliable power system operation, optimizing energy storage configuration, and enhancing the absorption capacity of new energy sources.

[0003] To improve prediction accuracy, the industry has shifted from traditional statistical methods and physical models to data-driven approaches, such as deep learning. In recent years, attention mechanisms, especially multi-head self-attention mechanisms capable of capturing complex dependencies in long sequences, have been introduced into the field of time series forecasting and have demonstrated superior performance. Numerous patented technologies have attempted to apply these advanced models to photovoltaic forecasting.

[0004] However, existing technologies still have inherent limitations when dealing with the special but critical scenario of extreme weather changes. Specifically, when rapid cloud movement, short-term strong convection, or other weather events cause drastic and non-stationary changes in features such as illumination and temperature, models based on standard multi-head self-attention mechanisms may exhibit competitive over-focusing. The root cause lies in the fact that the multiple attention heads in the standard multi-head attention mechanism, designed in parallel, are intended to collaboratively capture information from different aspects of the sequence. However, in extreme scenarios with drastic feature changes, multiple attention heads may be attracted to strong features at the same point of change, unanimously allocating extremely high weights to that moment or adjacent moments, resulting in highly similar or even redundant attention distributions. This defeats the original design intent of the multi-head mechanism's division of labor and cooperation; essentially, multiple heads are competing to focus on the same most salient feature point, rather than comprehensively and evenly analyzing the entire context sequence.

[0005] This technical problem will have direct and serious consequences in practical engineering applications:

[0006] (1) Because the model focuses too much on the mutation point and relatively ignores the normal evolution trend and periodicity before and after the mutation, the model output is overly sensitive to input noise and the mutation point itself. Its prediction results may produce violent fluctuations or peaks near the mutation point that do not conform to the physical characteristics of photovoltaic power output, which seriously affects the smoothness and rationality of the prediction curve;

[0007] (2) This overfitting to extreme features makes the model unstable on similar mutation patterns not seen in the training set, increasing the prediction variance. The model becomes fragile, and its predictive performance is highly dependent on whether the training data contains the exact same mutation patterns;

[0008] (3) The load forecast curve is a core input for key applications such as automatic generation control and intraday rolling dispatch. Abnormal fluctuations in the forecast caused by the above-mentioned problems can directly mislead dispatch decisions, potentially leading to unnecessary spinning reserve calls, increased system operating costs, and even, in severe cases, erroneous control commands, threatening the transient stability of the power grid.

[0009] Therefore, there is an urgent need for a new method that can suppress abnormal competition of multi-head attention in extreme scenarios and achieve robust feature extraction and prediction. Summary of the Invention

[0010] The purpose of this invention is to overcome the shortcomings of the prior art and provide a photovoltaic time-series dynamic prediction method and system based on an attention mechanism to solve the problem of multi-head competitive over-focusing under extreme weather change scenarios, thereby improving the accuracy, stability and robustness of photovoltaic load prediction and meeting the stringent requirements of the power grid for prediction accuracy under high proportion of photovoltaic access.

[0011] To solve the above-mentioned technical problems, the present invention is implemented using the following technical solution:

[0012] In a first aspect, the present invention provides a photovoltaic time-series dynamic prediction method based on an attention mechanism, comprising:

[0013] The original photovoltaic time-series data of the target area within a preset historical time period are preprocessed to obtain preprocessed photovoltaic time-series data.

[0014] The preprocessed photovoltaic time-series data is input into the trained photovoltaic time-series dynamic prediction model:

[0015] The preprocessed photovoltaic time-series data is embedded with time-series features to obtain an embedded feature vector;

[0016] The embedded feature vector is input into the improved multi-head self-attention layer for dynamic attention calculation and competition balancing to obtain the optimized context feature vector.

[0017] The optimized context feature vector is nonlinearly transformed and integrated with the feature dimension using a feedforward neural network layer to obtain an enhanced high-level feature vector.

[0018] The high-level feature vectors are input into the output layer to obtain a sequence of photovoltaic load prediction values ​​for a specified future time span.

[0019] Optionally, the original photovoltaic time-series data includes photovoltaic power generation sequence, meteorological factor sequence, and historical load sequence;

[0020] The data preprocessing includes missing value handling, normalization, and smoothing and denoising.

[0021] Optionally, the missing value processing is implemented using a linear interpolation algorithm;

[0022] The normalization process is implemented using the maximum-minimum value normalization algorithm;

[0023] The smoothing and denoising process is implemented using a Savitzky-Golay filter.

[0024] Optionally, the preprocessed photovoltaic time-series data is subjected to time-series feature embedding to obtain an embedded feature vector, including:

[0025] The preprocessed photovoltaic time series data is input into multiple independent fully connected layers, and the output feature vectors of the multiple independent fully connected layers are fused by adding them element by element to obtain the fused feature vector.

[0026] A unique positional encoding vector is generated for each time step of the fused feature vector through a positional encoding operation, and the fused feature vector at each time step is added element by element to the corresponding positional encoding vector to obtain the embedded feature vector.

[0027] Optionally, the embedded feature vector is input into an improved multi-head self-attention layer for dynamic attention calculation and competitive balancing to obtain an optimized context feature vector, including:

[0028] The embedded feature vectors are mapped to query matrices through three independent linear transformation layers. Key matrix and value matrix ;

[0029] For each attention head, according to the query matrix Bond matrix Calculate the attention score matrix of the attention head, and then combine the attention score matrix with the value matrix. Multiplying these together yields the initial context feature vector of the attention head;

[0030] Calculate the Jensen-Shannon divergence between the attention score matrices of any two attention heads to obtain the divergence matrix. ;

[0031] Traversing the scatter matrix This will satisfy the condition that the Jensen-Shannon divergence value is less than the dynamic similarity threshold. The two attention heads are marked as competing;

[0032] Construct an undirected graph based on the labeled competition relationships; where nodes in the undirected graph represent attention heads and edges represent competition relationships.

[0033] The undirected graph is traversed using the connected component algorithm, and the attention heads corresponding to all nodes in the largest connected subgraph of the undirected graph are identified as a competitive attention head group. ;

[0034] From the competing attention groups In the process, an initial context feature vector of a randomly selected attention head is used as a reference vector for competing attention head groups. For any remaining attention head, calculate the projection component of its initial context feature vector in the direction of the reference vector, and apply orthogonal damping to its initial context feature vector according to the projection component to obtain the damped context feature vector.

[0035] All damped context feature vectors, reference vectors, and attention head groups not belonging to the competing attention head group The initial context feature vectors of all attention heads are concatenated along the feature dimension, and the concatenation result is reduced in dimension using a linear projection layer to obtain the damped and equalized context feature vector. ;

[0036] Based on the timestamp information of the preprocessed photovoltaic time series data, the continuous time step range where meteorological factor fluctuations exceed a set threshold is located, and the context feature vector after damping equalization is extracted. The feature segments corresponding to the continuous time step range ;

[0037] The feature fragment was analyzed using the Daubechies4 wavelet basis function. Each feature channel is independently subjected to three-level wavelet packet decomposition to obtain a set of coefficients;

[0038] Multiply the high-frequency detail coefficient subbands in the coefficient set by the attenuation factor Multiply the low-frequency approximation coefficient subband by the enhancement factor This yields the processed set of coefficients.

[0039] The processed coefficient set is reconstructed using the Daubechies4 wavelet basis functions to obtain the corrected feature fragments. ;

[0040] Using the corrected feature fragments Replaced context feature vector after damping equalization Feature fragments in The optimized context feature vector is obtained. .

[0041] Optionally, the query matrix Key matrix and value matrix It can be obtained through the following formula:

[0042] ,

[0043] ,

[0044] ,

[0045] in, This represents the learnable query weight matrix. This represents the learnable key weight matrix. This represents the learnable weight matrix. Represents the embedded feature vector;

[0046] The attention score matrix is ​​obtained using the following formula:

[0047] ,

[0048] in, Indicates the first Attention score matrix of each attention head. For activation function, Representing the query matrix Used in the first A query vector with attention heads. Indicates transpose. Key matrix Used in the first The key vector of each attention head. Represents the key vector The dimension;

[0049] The initial context feature vector is obtained by the following formula:

[0050] ,

[0051] in, Indicates the first The initial context feature vector of each attention head, Represents the value matrix Used in the first The value vector of each attention head;

[0052] The Jensen-Shannon divergence value is obtained by the following formula:

[0053] ,

[0054] in, Indicates the first Attention score matrix of each attention head. Represents the Jensen-Shannon divergence function. express and The Jensen-Shannon divergence values ​​between them;

[0055] The projection component of the initial context feature vector onto the reference vector direction is obtained by the following formula:

[0056] ,

[0057] in, Indicating competitive attention head groups The Middle The initial context feature vector of each attention head, Represents the reference vector. express exist Projection components in the direction, Represents the reference vector The square of the L2 norm;

[0058] The context feature vector after applying damping is obtained by the following formula:

[0059] ,

[0060] in, Indicating competitive attention head groups The Middle The context feature vector after applying damping to each attention head. is a learnable damping coefficient vector.

[0061] Optionally, the method of the present invention further includes correcting the optimized context feature vector through attention guided by physical constraints. :

[0062] Based on the optimized context feature vector A preliminary load forecast sequence is generated using a single-layer linear transformation network. ;

[0063] Calculate the preliminary load forecast sequence The power ramp rate sequence is obtained by differentiating between adjacent time steps. ;

[0064] Identify the power ramp rate sequence All of the physical system's maximum permissible ramp rate thresholds exceeded the preset threshold. The time points at which the constraints were violated are denoted as the set of time points at which the constraints were violated. ;

[0065] like Not empty, calculate the optimized context feature vector. Mid-time step The gradient of the feature's contribution to the constraint violation ;in, ;

[0066] For any time step The optimized context feature vector is reduced according to the normalization ratio of the contribution gradient. Mid-time step The feature magnitude is increased, and the optimized context feature vector is enlarged. Mid-time step The characteristic amplitude; among which, Indicates time step The set of adjacent normal time steps, ;

[0067] The contribution gradient It can be obtained through the following formula:

[0068] ,

[0069] ,

[0070] in, Represents the power ramp rate sequence At time step Power ramp rate at the location. This represents the total loss that measures the degree of constraint violation. This represents the function that takes the maximum value. Represents the power ramp rate sequence At time step Power ramp rate at the location;

[0071] The optimized context feature vector is reduced according to the normalization ratio of the contribution gradient. Mid-time step The characteristic amplitude is achieved by the following formula:

[0072]

[0073] in, Represents the context feature vector before and after scaling. Mid-time step Features Represents the scaled-down and optimized context feature vector Mid-time step Features This represents element-wise multiplication. To reduce hyperparameters, , To and All one vectors of the same dimension Represents the optimized context feature vector Mid-time step The gradient of the contribution of the feature to the violation of the constraint;

[0074] The increased and optimized context feature vector Mid-time step The characteristic amplitude is achieved by the following formula:

[0075]

[0076] in, Represents the context feature vector before and after optimization. Mid-time step Features This represents the optimized context feature vector after enlargement. Mid-time step Features To increase hyperparameters, , Indicates time step The set of adjacent time steps that violate the constraint. , Represents the optimized context feature vector Mid-time step The feature contributes a gradient to the violation of constraints. Represents a set The number of elements in the middle.

[0077] Optionally, the feedforward neural network layer includes a first fully connected layer, a GELU activation layer, and a second fully connected layer connected in sequence, as expressed below:

[0078] ,

[0079] in, This represents the enhanced high-level feature vector. This represents the optimized context feature vector. This represents the weights of the first fully connected layer. This indicates the bias of the first fully connected layer. Indicates the weights of the second fully connected layer. This indicates the bias of the second fully connected layer. This represents the GELU activation function;

[0080] The output layer comprises a sequentially connected one-dimensional causal convolutional layer and a linear layer, as expressed below:

[0081] ,

[0082] in, Represents the sequence of photovoltaic load forecast values. This represents a one-dimensional causal convolution operation. This represents a linear transformation operation.

[0083] Optionally, the photovoltaic time-series dynamic prediction model is trained using a course learning strategy;

[0084] The loss function of the photovoltaic time-series dynamic prediction model is as follows:

[0085] ,

[0086] in, This represents the total loss of the photovoltaic time-series dynamic prediction model. This represents the loss due to the mean squared error in prediction. , This represents the total number of training samples. Indicates the first A sequence of actual photovoltaic load values ​​for each training sample. Indicates the first A sequence of photovoltaic load prediction values ​​for training samples. This indicates a penalty for loss of attention to diversity. , and For hyperparameters, This represents the natural exponential function. express and The Jensen-Shannon divergence values ​​between them Indicates the first Attention score matrix of each attention head. Indicates the first Attention score matrix for each attention head.

[0087] Secondly, the present invention provides a photovoltaic time-series dynamic prediction system based on an attention mechanism, comprising:

[0088] The data preprocessing module is used to: preprocess the raw photovoltaic time-series data of the target area within a preset historical time period to obtain preprocessed photovoltaic time-series data;

[0089] The photovoltaic time-series dynamic prediction module is used to: input the preprocessed photovoltaic time-series data into the trained photovoltaic time-series dynamic prediction model.

[0090] The preprocessed photovoltaic time-series data is embedded with time-series features to obtain an embedded feature vector;

[0091] The embedded feature vector is input into the improved multi-head self-attention layer for dynamic attention calculation and competition balancing to obtain the optimized context feature vector.

[0092] The optimized context feature vector is nonlinearly transformed and integrated with the feature dimension using a feedforward neural network layer to obtain an enhanced high-level feature vector.

[0093] The high-level feature vectors are input into the output layer to obtain a sequence of photovoltaic load prediction values ​​for a specified future time span.

[0094] Compared with existing technologies, the beneficial effects achieved by this invention are as follows:

[0095] 1. By calculating the Jensen-Shannon divergence of the multi-head attention weight distribution, we can automatically identify attention head groups that are over-focusing in extreme mutation scenarios, providing a precise target for subsequent damping equalization. Applying damping adjustment based on orthogonality constraints to the outputs of competing attention heads effectively reduces the similarity between competing heads, maintains the diversity and division of labor capabilities of the multi-head mechanism, and avoids over-focusing of multiple heads at the same mutation point. Wavelet packet decomposition is performed on the feature components at the mutation moment. By applying differentiated weight corrections to high-frequency and low-frequency components, the excessive influence of high-frequency noise in the mutation is weakened, while trend information is preserved, further improving the stability of prediction.

[0096] 2. A physical constraint-guided attention correction mechanism is introduced. After time-frequency domain rebalancing, a verification and gradient-guided correction based on power ramp rate constraint is added to ensure that the prediction results conform to the physical variation law of photovoltaic output and effectively eliminate non-physical prediction peaks.

[0097] 3. It significantly suppresses non-physical oscillations in the prediction curve under extreme weather change scenarios. Compared with the ordinary Transformer model, it can significantly reduce MAPE, effectively improve the stability and physical rationality of the prediction, provide a more reliable basis for the rapid power balance scheduling of the power grid under severe weather, help reduce system operating costs, improve the transient stability of the power grid, and has good engineering application prospects. Attached Figure Description

[0098] Figure 1 This is a schematic diagram of the photovoltaic time-series dynamic prediction method based on the attention mechanism provided in an embodiment of the present invention;

[0099] Figure 2 This is a schematic diagram of the dynamic attention calculation and competitive equilibrium process provided in an embodiment of the present invention;

[0100] Figure 3 This is a schematic diagram of the training process of the photovoltaic time-series dynamic prediction model provided according to an embodiment of the present invention. Detailed Implementation

[0101] The technical solution of the present invention will be described in detail below with reference to the accompanying drawings and specific embodiments. It should be understood that the embodiments and specific features in the embodiments are detailed descriptions of the technical solution of the present application, rather than limitations thereof. In the absence of conflict, the embodiments and technical features in the embodiments can be combined with each other.

[0102] It should be noted that the term "and / or" in this article is merely a description of the relationship between related objects, indicating that three relationships can exist. For example, A and / or B can represent: A existing alone, A and B existing simultaneously, or B existing alone. Additionally, the character " / " in this article generally indicates that the preceding and following related objects have an "or" relationship.

[0103] Example 1

[0104] This invention discloses a photovoltaic time-series dynamic prediction method based on an attention mechanism, with reference to... Figure 1 As shown, the specific steps include the following:

[0105] S1, perform data preprocessing on the original photovoltaic time series data of the target area within a preset historical time period to obtain preprocessed photovoltaic time series data;

[0106] S2, input the preprocessed photovoltaic time-series data into the trained photovoltaic time-series dynamic prediction model:

[0107] S2.1, embed time-series features into the preprocessed photovoltaic time-series data to obtain an embedded feature vector;

[0108] S2.2, The embedded feature vector is input into the improved multi-head self-attention layer for dynamic attention calculation and competition balancing to obtain the optimized context feature vector;

[0109] S2.3, The optimized context feature vector is subjected to nonlinear transformation and feature dimension integration using a feedforward neural network layer to obtain an enhanced high-level feature vector;

[0110] S2.4, Input the high-level feature vector into the output layer to obtain a sequence of photovoltaic load prediction values ​​for a specified future time span.

[0111] Specifically, in step S1, the data acquisition and monitoring system deployed at the target photovoltaic power station and the associated power grid synchronously collects the original photovoltaic time-series data within the historical time window at a first preset sampling frequency. This includes photovoltaic power generation sequence, meteorological factor sequence, and historical load sequence. The photovoltaic power generation sequence originates from the output power measurement unit of the power station inverter, the meteorological factor sequence originates from the meteorological sensor group deployed at the power station site, and the historical load sequence originates from the load record database of the power grid dispatch center. The data acquisition and monitoring system packages the three types of sequence data and adds a unified timestamp through an industrial communication protocol to form the original photovoltaic time-series data stream.

[0112] The data preprocessing includes missing value processing, normalization processing, and smoothing and denoising processing. The missing value processing uses a linear interpolation algorithm to fill in missing values ​​in the data stream. The normalization processing uses a maximum-minimum normalization algorithm to map the data of each sequence to a closed interval between zero and one. The smoothing and denoising processing uses a Savitzky-Golay filter to smooth and denoise the normalized sequence. The filter window length and polynomial order are preset according to the sampling frequency of the sequence. After the above three sequential processing steps, the standardized and denoised photovoltaic time series data are output.

[0113] In step S2.1, the step of embedding time-series features into the preprocessed photovoltaic time-series data to obtain an embedded feature vector includes:

[0114] S2.1.1 The preprocessed photovoltaic time series data, namely the photovoltaic power generation sequence, meteorological factor sequence and historical load sequence, are respectively input into three independent fully connected layers. The three fully connected layers have the same output dimension. The output feature vectors of the three fully connected layers are added element by element and fused to obtain the fused feature vector.

[0115] S2.1.2, a unique position encoding vector is generated for each time step of the fused feature vector through position encoding operation. The dimension of the position encoding vector is the same as the dimension of the fused feature vector. The fused feature vector of each time step is added element by element to the corresponding position encoding vector to obtain the embedded feature vector. In this embodiment, the position encoding operation adopts a deterministic encoding method combining sine and cosine functions.

[0116] In step S2.2, refer to Figure 2 As shown, the step of inputting the embedded feature vector into the improved multi-head self-attention layer for dynamic attention calculation and competitive balancing to obtain the optimized context feature vector includes:

[0117] S2.2.1, the embedded feature vector is mapped to a query matrix through three independent linear transformation layers. Key matrix and value matrix :

[0118] ,

[0119] ,

[0120] ,

[0121] in, This represents the learnable query weight matrix. This represents the learnable key weight matrix. This represents the learnable weight matrix. Represents the embedded feature vector;

[0122] S2.2.2, for each attention head, according to the query matrix Bond matrix Calculate the attention score matrix of the attention head, and then combine the attention score matrix with the value matrix. Multiplying these together yields the initial context feature vector of the attention head:

[0123] ,

[0124] ,

[0125] in, Indicates the first Attention score matrix of each attention head. For activation function, Representing the query matrix Used in the first A query vector with attention heads. Indicates transpose. Key matrix Used in the first The key vector of each attention head. Represents the key vector Dimensions Indicates the first The initial context feature vector of each attention head, Represents the value matrix Used in the first The value vector of each attention head;

[0126] S2.2.3, calculate the Jensen-Shannon divergence between the attention score matrices of any two attention heads to obtain the divergence matrix. :

[0127] ,

[0128] in, Indicates the first Attention score matrix of each attention head. Represents the Jensen-Shannon divergence function. express and The Jensen-Shannon divergence values ​​between them;

[0129] S2.2.4, Traversing the divination matrix This will satisfy the condition that the Jensen-Shannon divergence value is less than the dynamic similarity threshold. Two attention heads are marked as having a competing relationship, wherein the dynamic similarity threshold The value of each element in the scatter matrix D in the current training batch is adaptively determined based on its statistical distribution. An undirected graph is constructed based on the labeled competition relationships. Nodes in the undirected graph represent attention heads, and edges represent competition relationships. The connected component algorithm is used to traverse the undirected graph, identifying all attention heads corresponding to nodes within the largest connected subgraph as a group of competing attention heads. ;

[0130] S2.2.5, from the competing attention head group In the process, an initial context feature vector of a randomly selected attention head is used as a reference vector for competing attention head groups. For any remaining attention head, calculate the projection component of its initial context feature vector onto the reference vector direction, and apply orthogonal damping to its initial context feature vector based on the projection component to obtain the damped context feature vector:

[0131] ,

[0132] ,

[0133] in, Indicating competitive attention head groups The Middle The initial context feature vector of each attention head, Represents the reference vector. express exist Projection components in the direction, Indicating competitive attention head groups The Middle The context feature vector after applying damping to each attention head. This is a learnable damping coefficient vector; it is updated during model training using gradient descent; this operation enables... towards and Orthogonal orientation adjustments reduce redundancy;

[0134] S2.2.6, all damped context feature vectors, reference vectors, and those not belonging to the competing attention head group are included. The initial context feature vectors of all attention heads are concatenated along the feature dimension, and the concatenation result is reduced in dimension using a linear projection layer to obtain the damped and equalized context feature vector. ;

[0135] S2.2.7, Based on the timestamp information of the preprocessed photovoltaic time series data, locate the continuous time step range where meteorological factor fluctuations exceed a set threshold, and extract the context feature vector after damping equalization. The feature segments corresponding to the continuous time step range ;

[0136] S2.2.8, using the Daubechies4 wavelet basis functions to analyze the feature segments. Each feature channel is independently subjected to three-level wavelet packet decomposition to obtain a set of coefficients;

[0137] S2.2.9, multiply the high-frequency detail coefficient subbands in the coefficient set by an attenuation factor. Multiply the low-frequency approximation coefficient subband by the enhancement factor This yields the processed coefficient set; where, ;

[0138] S2.2.10, the processed coefficient set is reconstructed using the Daubechies4 wavelet basis functions to obtain the corrected feature fragments. ;

[0139] S2.2.11, using the corrected feature fragments Replaced context feature vector after damping equalization Feature fragments in The optimized context feature vector is obtained. .

[0140] In this embodiment, the method further includes attention-guided correction of the optimized context feature vector through physical constraints. :

[0141] S2.2.12, based on the optimized context feature vector A preliminary load forecast sequence is generated using a single-layer linear transformation network. ;

[0142] S2.2.13, Calculate the preliminary load forecast sequence The power ramp rate sequence is obtained by differentiating between adjacent time steps. ;

[0143] S2.2.14, Identify the power ramp-up rate sequence All of the physical system's maximum permissible ramp rate thresholds exceeded the preset threshold. The time points at which the constraints were violated are denoted as the set of time points at which the constraints were violated. ;

[0144] S2.2.15, if Not empty, calculate the optimized context feature vector. Mid-time step The gradient of the feature's contribution to the constraint violation ;

[0145] ,

[0146] ,

[0147] in, , Represents the power ramp rate sequence At time step Power ramp rate at the location. This represents the total loss that measures the degree of constraint violation. This represents the function that takes the maximum value. Represents the power ramp rate sequence At time step Power ramp rate at the location;

[0148] S2.2.16, for any time step The optimized context feature vector is reduced according to the normalization ratio of the contribution gradient. Mid-time step Characteristic amplitude:

[0149]

[0150] in, Represents the context feature vector before and after scaling. Mid-time step Features Represents the scaled-down and optimized context feature vector Mid-time step Features This represents element-wise multiplication. To reduce hyperparameters, , To and All one vectors of the same dimension Represents the optimized context feature vector Mid-time step The gradient of the contribution of the feature to the violation of the constraint;

[0151] Increase the optimized context feature vector Mid-time step The characteristic amplitude; among which, Indicates time step The set of adjacent normal time steps, :

[0152]

[0153] in, Represents the context feature vector before and after optimization. Mid-time step Features This represents the optimized context feature vector after enlargement. Mid-time step Features To increase hyperparameters, , Indicates time step The set of adjacent time steps that violate the constraint. , Represents the optimized context feature vector Mid-time step The feature contributes a gradient to the violation of constraints. Represents a set The number of elements in the middle.

[0154] In step S2.3, the feedforward neural network layer includes a first fully connected layer, a GELU activation layer, and a second fully connected layer connected in sequence, as expressed below:

[0155] ,

[0156] in, This represents the enhanced high-level feature vector. This represents the optimized context feature vector. This represents the weights of the first fully connected layer. This indicates the bias of the first fully connected layer. Indicates the weights of the second fully connected layer. This indicates the bias of the second fully connected layer. This represents the GELU activation function.

[0157] In step S2.4, the output layer includes a sequentially connected one-dimensional causal convolutional layer and a linear layer. The causal convolutional layer is used to capture the local temporal dependencies of the output sequence, and the linear layer maps the convolutional output to the final prediction dimension; its expression is as follows:

[0158]

[0159] in, Represents the sequence of photovoltaic load forecast values. This represents a one-dimensional causal convolution operation. This represents a linear transformation operation.

[0160] In this embodiment, the photovoltaic time-series dynamic prediction model employs a course learning strategy for model training. Specifically, the course learning strategy involves: in the initial training phase... Set a large initial value to force attention head differentiation, and gradually decrease it as the training rounds increase. This value is used to optimize overall prediction accuracy.

[0161] refer to Figure 3 As shown, the training process of the photovoltaic time-series dynamic prediction model includes:

[0162] Obtain a historical dataset with labels for real photovoltaic load sequences containing extreme weather scenarios;

[0163] Based on the historical dataset, the forward propagation process of the photovoltaic time-series dynamic prediction model is carried out in batches to obtain a sequence of photovoltaic load prediction values.

[0164] The total loss is calculated based on the actual photovoltaic load sequence labels and the photovoltaic load forecast sequence.

[0165] Backpropagation and gradient updates are performed based on the total loss to update all parameters of the photovoltaic time-series dynamic prediction model.

[0166] Adjust the course learning strategy, that is, gradually reduce the number of training rounds. This value is used to optimize overall prediction accuracy.

[0167] The loss function of the photovoltaic time-series dynamic prediction model is as follows:

[0168] ,

[0169] in, This represents the total loss of the photovoltaic time-series dynamic prediction model. This represents the loss due to the mean squared error in prediction. , This represents the total number of training samples. Indicates the first A sequence of actual photovoltaic load values ​​for each training sample. Indicates the first A sequence of photovoltaic load prediction values ​​for training samples. This indicates a penalty for loss of attention to diversity. , and For hyperparameters, This represents the natural exponential function. express and The Jensen-Shannon divergence values ​​between them Indicates the first Attention score matrix of each attention head. Indicates the first An attention score matrix for each attention head; training employs a course-based learning strategy, with a relatively large initial training set. The value is used to force attention to differentiate, and is gradually reduced in subsequent training phases. This value is used to optimize overall prediction accuracy.

[0170] Example 2

[0171] This invention discloses a photovoltaic time-series dynamic prediction method based on an attention mechanism, with reference to... Figure 1 As shown, the specific steps include the following:

[0172] S1, perform data preprocessing on the original photovoltaic time series data of the target area within a preset historical time period to obtain preprocessed photovoltaic time series data;

[0173] S2, input the preprocessed photovoltaic time-series data into the trained photovoltaic time-series dynamic prediction model:

[0174] S2.1, embed time-series features into the preprocessed photovoltaic time-series data to obtain an embedded feature vector;

[0175] S2.2, The embedded feature vector is input into the improved multi-head self-attention layer for dynamic attention calculation and competition balancing to obtain the optimized context feature vector;

[0176] S2.3, The optimized context feature vector is subjected to nonlinear transformation and feature dimension integration using a feedforward neural network layer to obtain an enhanced high-level feature vector;

[0177] S2.4, Input the high-level feature vector into the output layer to obtain a sequence of photovoltaic load prediction values ​​for a specified future time span.

[0178] Specifically, this embodiment takes the actual operation scenario of a 50MW photovoltaic power station and its feedin local distribution network before and after a sudden thunderstorm on a summer day as an example. In step S1, the raw photovoltaic time-series data for the past 6 hours is obtained through the data acquisition and monitoring SCADA system of the photovoltaic power station, with a sampling frequency of 1 minute. The data time period is from 10:00 to 16:00 on July 15, 2023. During this period, the weather changed from sunny to thunderstorm.

[0179] The photovoltaic power generation sequence is derived from the aggregated active power data of all inverters in the power station, in kW. Examples of data points include: at 10:00, the power was 42350kW; at 14:05—the start of the sudden change—the power was 42800kW; and at 14:20—during heavy rainfall—the power plummeted to 1520kW. The meteorological factor sequence is derived from measurements taken at the power station's meteorological station, including irradiance (W / m²), ambient temperature (°C), and humidity (%). Examples of data points include: at 10:00, irradiance was 850W / m²; at 14:05, it was 830W / m²; and at 14:20, it plummeted to 65W / m². The historical load sequence is derived from the distribution network dispatch center and represents the net load data for the power supply area of ​​the photovoltaic power station, i.e., the total load minus the photovoltaic output, in kW.

[0180] All data is transmitted via the IEC60870-5-104 protocol and stamped with a unified timestamp, forming a raw data stream containing 360 time points over 6 hours x 60 minutes, four core data channels for power, irradiance, temperature and humidity, and corresponding load labels.

[0181] In step S2, data preprocessing includes the following missing value handling, normalization, and smoothing / denoising:

[0182] Missing value handling: Inspection revealed that the humidity data at 14:07 was lost due to a momentary communication interruption with the sensor. Linear interpolation was used, employing 65% of the data from 14:06 and 68% from 14:08, to calculate the value for 14:07 as 66.5%, which was then used to fill the gap.

[0183] Normalization: Maximum-minimum normalization is used. For the photovoltaic power series, the maximum value in the historical 6-hour period is... kW, minimum value kW. Photovoltaic power at any given time normalization results Calculate using the following formula:

[0184] ,

[0185] in, This represents the data before normalization. Data before normalization The minimum value in, Data before normalization The maximum value in, This indicates that the normalized data, specifically the original power of 42350kW at 10:00, has been normalized to: ;

[0186] Smoothing and Denoising: A Savitzky-Golay filter with a window length of 5 minutes and a polynomial order of 2 was used to smooth the four normalized sequences. After processing, the output is a standardized and denoised photovoltaic time series data matrix with dimensions [360,4].

[0187] In step S2.1, the data from the four channels—normalized power, irradiance, temperature, and humidity—are input into a fully connected layer, projecting the feature dimension from 4 to the model's hidden dimension. This operation can learn to fuse information from different physical quantities, outputting a feature matrix of [360, 128]; it employs sinusoidal position encoding for each time step. and feature dimension index Location code value Calculate using the following formula:

[0188]

[0189] The position encoding matrix and the fused feature matrix are added element-wise to obtain the final embedded feature vector. Its shape is [360, 128].

[0190] In step S2.2, the number of attention heads Improved multi-head self-attention layer receives embedded feature vectors And perform the following operations:

[0191] Embedded feature vectors The linear transformations are respectively mapped to query matrices. ,key matrix and value matrix Each matrix has a shape of [360, 128]; subsequently, the... , By attention head Divide into, and get the first The query matrix corresponding to each attention head Key matrix and value matrix All of them have a shape of [360, 16];

[0192] For the Each attention head has an attention weight matrix. Calculate using the following formula:

[0193] ,

[0194] in, The shape is [360, 360], used to represent the correlation weights between time steps in the sequence. This is the activation function. At the mutation time (corresponding sequence index) ), observed multiple attention heads exist The weights in this column are unusually high;

[0195] Calculate 8 attention score matrices Jensen-Shannon divergence values ​​between pairs Discover the attention weight matrices of attention heads 1, 3, and 5. , , The distributions of mutation times are highly similar, and their pairwise similarities are significant. The values ​​were 0.08, 0.05, and 0.07, respectively, all less than the dynamic threshold. Therefore, attention heads 1, 3, and 5 are identified as a competing attention head group. .

[0196] With attention head 1's initial context feature vector Based on this, for attention head 3, its initial context feature vector is calculated. exist The projection onto the surface is applied, and damping is applied. Let the learnable damping coefficient be... Then, the initial context feature vector after applying damping A similar operation is performed on attention head 5. Then, the outputs of all eight attention heads (attention heads 1, 3, and 5 are the initial context feature vectors after applying damping, and the rest are the initial context feature vectors) are concatenated and projected back to 128 dimensions through a linear layer to obtain the damped and balanced context feature vector. .

[0197] Cut Feature fragments of 21 time steps corresponding to the mutation period in the middle ,index to .right Each of the 128 features is independently decomposed into 3-level Daubechies-4 wavelet packet decomposition. The high-frequency detail coefficient subbands obtained from the decomposition are multiplied by an attenuation factor. Low-frequency approximation coefficient subband multiplied by The reconstructed fragment is the corrected segment. Replace them back in their original positions to form the optimized context feature vector. This operation reduces the excessive influence of high-frequency noise from mutations, preserving trend information.

[0198] In step S2.3, the feedforward neural network layer includes a first fully connected layer, a GELU activation layer, and a second fully connected layer connected in sequence, as expressed below:

[0199] ,

[0200] in, This represents the enhanced high-level feature vector. This represents the optimized context feature vector. This represents the weights of the first fully connected layer. This indicates the bias of the first fully connected layer. Indicates the weights of the second fully connected layer. This indicates the bias of the second fully connected layer. This represents the GELU activation function; Expand the dimension from 128 to 512. Map back from 512 to 128. Output the enhanced high-level feature vector, which still has the shape [360, 128].

[0201] In step S2.4, the output layer includes a sequentially connected one-dimensional causal convolutional layer and a linear layer. The causal convolutional layer has a kernel size of 3, padding of 2, and 128 input and output channels, and is used to aggregate local temporal information. The linear layer maps the 128-dimensional features to the load prediction values ​​for the next 30 minutes, i.e., 30 time points. Its expression is as follows:

[0202] ,

[0203] in, Represents the sequence of photovoltaic load forecast values. This represents a one-dimensional causal convolution operation. This represents a linear transformation operation, outputting the final photovoltaic load forecast sequence. The shape is

[30] , corresponding to the net load forecast for each minute from 16:01 to 16:30.

[0204] Example 3

[0205] Based on the same inventive concept as Embodiments 1 and 2, this embodiment of the invention discloses a photovoltaic time-series dynamic prediction system based on an attention mechanism, comprising:

[0206] The data preprocessing module is used to: preprocess the raw photovoltaic time-series data of the target area within a preset historical time period to obtain preprocessed photovoltaic time-series data;

[0207] The photovoltaic time-series dynamic prediction module is used to: input the preprocessed photovoltaic time-series data into the trained photovoltaic time-series dynamic prediction model.

[0208] The preprocessed photovoltaic time-series data is embedded with time-series features to obtain an embedded feature vector;

[0209] The embedded feature vector is input into the improved multi-head self-attention layer for dynamic attention calculation and competition balancing to obtain the optimized context feature vector.

[0210] The optimized context feature vector is nonlinearly transformed and integrated with the feature dimension using a feedforward neural network layer to obtain an enhanced high-level feature vector.

[0211] The high-level feature vectors are input into the output layer to obtain a sequence of photovoltaic load prediction values ​​for a specified future time span.

[0212] The specific functions of each module described above are explained in the relevant content of the method in Embodiment 1, and will not be repeated here.

[0213] Those skilled in the art will understand that embodiments of the present invention can be provided as methods, systems, or computer program products. Therefore, the present invention can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, the present invention can take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0214] This invention is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of the invention. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, generate instructions for implementing the flowchart illustrations and / or block diagrams. Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.

[0215] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and / or boxes Figure 1 The function specified in one or more boxes.

[0216] These computer program instructions may also be loaded onto a computer or other programmable data processing equipment to cause a series of operational steps to be performed on the computer or other programmable equipment to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable equipment for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.

[0217] The embodiments of the present invention have been described above with reference to the accompanying drawings. However, the present invention is not limited to the specific embodiments described above. The specific embodiments described above are merely illustrative and not restrictive. Those skilled in the art can make many other forms under the guidance of the present invention without departing from the spirit and scope of the claims. All of these forms are within the protection scope of the present invention.

Claims

1. A photovoltaic time-series dynamic prediction method based on an attention mechanism, characterized in that, include: The original photovoltaic time-series data of the target area within a preset historical time period are preprocessed to obtain preprocessed photovoltaic time-series data. The preprocessed photovoltaic time-series data is input into the trained photovoltaic time-series dynamic prediction model: time-series feature embedding is performed on the preprocessed photovoltaic time-series data to obtain the embedded feature vector; The embedded feature vector is input into an improved multi-head self-attention layer for dynamic attention calculation and competitive balancing to obtain an optimized context feature vector; the optimized context feature vector is then subjected to nonlinear transformation and feature dimension integration using a feedforward neural network layer to obtain an enhanced high-level feature vector; the high-level feature vector is then input into the output layer to obtain a sequence of photovoltaic load prediction values ​​for a specified future time span. The embedded feature vector is input into the improved multi-head self-attention layer for dynamic attention calculation and competition balancing to obtain the optimized context feature vector, including: The embedded feature vectors are mapped to query matrices through three independent linear transformation layers. Key matrix and value matrix ; For each attention head, according to the query matrix Bond matrix Calculate the attention score matrix of the attention head, and then combine the attention score matrix with the value matrix. Multiplying these together yields the initial context feature vector of the attention head; Calculate the Jensen-Shannon divergence between the attention score matrices of any two attention heads to obtain the divergence matrix. ; Traversing the scatter matrix This will satisfy the condition that the Jensen-Shannon divergence value is less than the dynamic similarity threshold. The two attention heads are marked as competing; Construct an undirected graph based on the labeled competition relationships; where nodes in the undirected graph represent attention heads and edges represent competition relationships. The undirected graph is traversed using the connected component algorithm, and the attention heads corresponding to all nodes in the largest connected subgraph of the undirected graph are identified as a competitive attention head group. ; From the competing attention groups In the process, an initial context feature vector of a randomly selected attention head is used as a reference vector for competing attention head groups. For any remaining attention head, calculate the projection component of its initial context feature vector in the direction of the reference vector, and apply orthogonal damping to its initial context feature vector according to the projection component to obtain the damped context feature vector. All damped context feature vectors, reference vectors, and attention head groups not belonging to the competing attention head group The initial context feature vectors of all attention heads are concatenated along the feature dimension, and the concatenation result is reduced in dimension using a linear projection layer to obtain the damped and equalized context feature vector. ; Based on the timestamp information of the preprocessed photovoltaic time series data, the continuous time step range where meteorological factor fluctuations exceed a set threshold is located, and the context feature vector after damping equalization is extracted. The feature segments corresponding to the continuous time step range ; The feature fragment was analyzed using the Daubechies4 wavelet basis function. Each feature channel is independently subjected to three-level wavelet packet decomposition to obtain a set of coefficients; Multiply the high-frequency detail coefficient subbands in the coefficient set by the attenuation factor Multiply the low-frequency approximation coefficient subband by the enhancement factor This yields the processed set of coefficients. The processed coefficient set is reconstructed using the Daubechies4 wavelet basis functions to obtain the corrected feature fragments. ; Using the corrected feature fragments Replaced context feature vector after damping equalization Feature fragments in The optimized context feature vector is obtained. .

2. The photovoltaic time-series dynamic prediction method based on attention mechanism according to claim 1, characterized in that, The original photovoltaic time-series data includes photovoltaic power generation sequence, meteorological factor sequence, and historical load sequence; The data preprocessing includes missing value handling, normalization, and smoothing and denoising.

3. The photovoltaic time-series dynamic prediction method based on attention mechanism according to claim 2, characterized in that, The missing value processing is implemented using a linear interpolation algorithm; The normalization process is implemented using the maximum-minimum value normalization algorithm; The smoothing and denoising process is implemented using a Savitzky-Golay filter.

4. The photovoltaic time-series dynamic prediction method based on attention mechanism according to claim 1, characterized in that, The preprocessed photovoltaic time-series data is embedded with time-series features to obtain an embedded feature vector, including: The preprocessed photovoltaic time series data is input into multiple independent fully connected layers, and the output feature vectors of the multiple independent fully connected layers are fused by adding them element by element to obtain the fused feature vector. A unique positional encoding vector is generated for each time step of the fused feature vector through a positional encoding operation, and the fused feature vector at each time step is added element by element to the corresponding positional encoding vector to obtain the embedded feature vector.

5. The photovoltaic time-series dynamic prediction method based on attention mechanism according to claim 1, characterized in that, The query matrix Key matrix and value matrix It can be obtained through the following formula: ， ， ， in, This represents the learnable query weight matrix. This represents the learnable key weight matrix. This represents the learnable weight matrix. Represents the embedded feature vector; The attention score matrix is ​​obtained using the following formula: ， in, Indicates the first Attention score matrix of each attention head. For activation function, Representing the query matrix Used in the first A query vector with attention heads. Indicates transpose. Key matrix Used in the first The key vector of each attention head. Represents the key vector The dimension; The initial context feature vector is obtained by the following formula: ， in, Indicates the first The initial context feature vector of each attention head, Represents the value matrix Used in the first The value vector of each attention head; The Jensen-Shannon divergence value is obtained by the following formula: ， in, Indicates the first Attention score matrix of each attention head. Represents the Jensen-Shannon divergence function. express and The Jensen-Shannon divergence values ​​between them; The projection component of the initial context feature vector onto the reference vector direction is obtained by the following formula: ， in, Indicating competitive attention head groups The Middle The initial context feature vector of each attention head, Represents the reference vector. express exist Projection components in the direction, Represents the reference vector The square of the L2 norm; The context feature vector after applying damping is obtained by the following formula: ， in, Indicating competitive attention head groups The Middle The context feature vector after applying damping to each attention head. is a learnable damping coefficient vector.

6. The photovoltaic time-series dynamic prediction method based on attention mechanism according to claim 1, characterized in that, It also includes attention-guided correction of the optimized context feature vector through physical constraints. : Based on the optimized context feature vector A preliminary load forecast sequence is generated using a single-layer linear transformation network. ; Calculate the preliminary load forecast sequence The power ramp rate sequence is obtained by differentiating between adjacent time steps. ; Identify the power ramp rate sequence All of the physical system's maximum permissible ramp rate thresholds exceeded the preset threshold. The time points at which the constraints were violated are denoted as the set of time points at which the constraints were violated. ; like Not empty, calculate the optimized context feature vector. Mid-time step The contribution gradient of the feature to the constraint violation ;in, ; For any time step The optimized context feature vector is reduced according to the normalization ratio of the contribution gradient. Mid-time step The feature magnitude is increased, and the optimized context feature vector is enlarged. Mid-time step The characteristic amplitude; among which, Indicates time step Adjacent normal time step set ; The contribution gradient It can be obtained through the following formula: ， ， in, Represents the power ramp rate sequence At time step Power ramp rate at the location. This represents the total loss that measures the degree of constraint violation. This represents the function that takes the maximum value. Represents the power ramp rate sequence At time step Power ramp rate at the location; The optimized context feature vector is reduced according to the normalization ratio of the contribution gradient. Mid-time step The characteristic amplitude is achieved by the following formula: ,， in, Represents the context feature vector before and after scaling. Mid-time step Features This represents the optimized context feature vector after scaling. Mid-time step Features This represents element-wise multiplication. To reduce hyperparameters, , To and All one vectors of the same dimension Represents the optimized context feature vector Mid-time step The gradient of the contribution of the feature to the violation of the constraint; The increased and optimized context feature vector Mid-time step The characteristic amplitude is achieved by the following formula: ， in, Represents the context feature vector before and after optimization. Mid-time step Features This represents the optimized context feature vector after enlargement. Mid-time step Features To increase hyperparameters, , Indicates time step The set of adjacent time steps that violate the constraint. , Represents the optimized context feature vector Mid-time step The feature contributes a gradient to the violation of constraints. Represents a set The number of elements in the middle.

7. The photovoltaic time-series dynamic prediction method based on attention mechanism according to claim 1, characterized in that, The feedforward neural network layer comprises a first fully connected layer, a GELU activation layer, and a second fully connected layer connected in sequence, as expressed below: ， in, This represents the enhanced high-level feature vector. This represents the optimized context feature vector. This represents the weights of the first fully connected layer. This indicates the bias of the first fully connected layer. Indicates the weights of the second fully connected layer. This indicates the bias of the second fully connected layer. This represents the GELU activation function; The output layer comprises a sequentially connected one-dimensional causal convolutional layer and a linear layer, as expressed below: ， in, Represents the sequence of photovoltaic load forecast values. This represents a one-dimensional causal convolution operation. This represents a linear transformation operation.

8. The photovoltaic time-series dynamic prediction method based on attention mechanism according to claim 1, characterized in that, The photovoltaic time-series dynamic prediction model is trained using a course learning strategy. The loss function of the photovoltaic time-series dynamic prediction model is as follows: ， in, This represents the total loss of the photovoltaic time-series dynamic prediction model. This represents the loss due to the mean squared error in prediction. , This represents the total number of training samples. Indicates the first A sequence of actual photovoltaic load values ​​for each training sample. Indicates the first A sequence of photovoltaic load prediction values ​​for training samples. This indicates a penalty for loss of attention to diversity. , and For hyperparameters, This represents the natural exponential function. express and The Jensen-Shannon divergence values ​​between them Indicates the first Attention score matrix of each attention head. Indicates the first Attention score matrix for each attention head.

9. A photovoltaic time-series dynamic prediction system based on an attention mechanism, characterized in that, include: The data preprocessing module is used to: preprocess the raw photovoltaic time-series data of the target area within a preset historical time period to obtain preprocessed photovoltaic time-series data; The photovoltaic time-series dynamic prediction module is used to: input the preprocessed photovoltaic time-series data into the trained photovoltaic time-series dynamic prediction model; and embed time-series features into the preprocessed photovoltaic time-series data to obtain an embedded feature vector. The embedded feature vector is input into an improved multi-head self-attention layer for dynamic attention calculation and competitive balancing to obtain an optimized context feature vector; the optimized context feature vector is then subjected to nonlinear transformation and feature dimension integration using a feedforward neural network layer to obtain an enhanced high-level feature vector; the high-level feature vector is then input into the output layer to obtain a sequence of photovoltaic load prediction values ​​for a specified future time span. The embedded feature vector is input into the improved multi-head self-attention layer for dynamic attention calculation and competition balancing to obtain the optimized context feature vector, including: The embedded feature vectors are mapped to query matrices through three independent linear transformation layers. Key matrix and value matrix ; For each attention head, according to the query matrix Bond matrix Calculate the attention score matrix of the attention head, and then combine the attention score matrix with the value matrix. Multiplying these together yields the initial context feature vector of the attention head; Calculate the Jensen-Shannon divergence between the attention score matrices of any two attention heads to obtain the divergence matrix. ; Traversing the scatter matrix This will satisfy the condition that the Jensen-Shannon divergence value is less than the dynamic similarity threshold. The two attention heads are marked as competing; Construct an undirected graph based on the labeled competition relationships; where nodes in the undirected graph represent attention heads and edges represent competition relationships. The undirected graph is traversed using the connected component algorithm, and the attention heads corresponding to all nodes in the largest connected subgraph of the undirected graph are identified as a competitive attention head group. ; From the competing attention groups In the process, an initial context feature vector of a randomly selected attention head is used as a reference vector for competing attention head groups. For any remaining attention head, calculate the projection component of its initial context feature vector in the direction of the reference vector, and apply orthogonal damping to its initial context feature vector according to the projection component to obtain the damped context feature vector. All damped context feature vectors, reference vectors, and attention head groups not belonging to the competing attention head group The initial context feature vectors of all attention heads are concatenated along the feature dimension, and the concatenation result is reduced in dimension using a linear projection layer to obtain the damped and equalized context feature vector. ; Based on the timestamp information of the preprocessed photovoltaic time series data, the continuous time step range where meteorological factor fluctuations exceed a set threshold is located, and the context feature vector after damping equalization is extracted. The feature segments corresponding to the continuous time step range ; The feature fragment was analyzed using the Daubechies4 wavelet basis function. Each feature channel is independently subjected to three-level wavelet packet decomposition to obtain a set of coefficients; Multiply the high-frequency detail coefficient subbands in the coefficient set by the attenuation factor Multiply the low-frequency approximation coefficient subband by the enhancement factor This yields the processed set of coefficients. The processed coefficient set is reconstructed using the Daubechies4 wavelet basis functions to obtain the corrected feature fragments. ; Using the corrected feature fragments Replaced context feature vector after damping equalization Feature fragments in The optimized context feature vector is obtained. .

Images