Multi-modal time sequence fusion Transform power load prediction method based on quantum particle swarm optimization

Abstract

The invention relates to a multi-modal time sequence fusion Transform power load prediction method based on quantum particle swarm optimization, and belongs to the field of power load prediction. According to the method, a density clustering algorithm and a time sequence weighted interpolation method are utilized to preprocess power load and associated influence factor data, a random forest feature screening model is constructed, model parameters are optimized through quantum particle swarm optimization, the model is trained, and a power load key influence feature set is determined according to a feature importance threshold. The power load prediction method comprises the following steps: performing modal classification and standardization on key features, then establishing a multi-modal time sequence feature matrix adaptive to Transform, constructing a multi-modal time sequence fusion Transform model containing an attention modal fusion layer, and predicting the power load in the future seven days by using the optimized model. According to the method, efficient fusion of the multi-modal data and intelligent optimization of the model parameters are realized, the prediction precision is remarkably improved, and the problems of low pre-processing precision of the multi-modal data, redundancy of feature screening, insufficient optimization of the model parameters and large long-time prediction errors of a traditional prediction method are solved.

Description

TECHNICAL FIELD

[0001] The application belongs to the field of power load prediction, and particularly relates to a multimodal time sequence fusion Transformer power load prediction method based on quantum particle swarm optimization. BACKGROUND

[0002] Power load prediction is a core supporting technology for power system planning and design, dispatching operation and energy optimization configuration, and its prediction accuracy directly affects the safety, economy and reliability of the power grid. With the advancement of new power system construction, high proportion of new energy grid connection, growth of user side flexible load and popularization of multi-energy complementary mode, the coupling influence of time, weather, economy, policy and other multidimensional factors on power load is increasingly significant. The traditional load prediction method has been difficult to meet the prediction demand of long period and high precision, and has become a key bottleneck restricting the intelligent upgrading of the power grid. The limitations of the current traditional method are more focused on three core pain points: firstly, multi-modal influencing factors are mostly used for shallow splicing or single-modal independent modeling, which cannot capture dynamic coupling correlation, resulting in insufficient feature representation; secondly, long period time sequence modeling is prone to problems such as rapid increase in attention calculation complexity and insufficient long distance dependence capture, causing error accumulation; thirdly, model parameters depend on manual debugging, which is difficult to adapt to complex scenarios of different regions and different load types, and has poor generalization.

[0003] Therefore, it is an urgent need to develop a method capable of realizing accurate long period data prediction to solve the bottleneck problem of low power load prediction accuracy and support intelligent operation of new power systems. The multimodal time sequence fusion Transformer method based on quantum particle swarm optimization proposed in the application realizes deep interaction of multi-modal data through the design of cross-modal time sequence attention module, uses the global optimization ability of quantum particle swarm optimization to adaptively optimize model parameters, balances long period prediction accuracy and computational efficiency by combining sparse time sequence attention, and can effectively break through the above technical bottlenecks, not only significantly improve the long period prediction accuracy and reduce the error, but also enhance the model generalization and engineering landing efficiency, providing more reliable technical support for new power system planning and dispatching. SUMMARY

[0004] The application aims to provide a multimodal time sequence fusion Transformer power load prediction method based on quantum particle swarm optimization, which can solve the problems of low multi-modal data preprocessing accuracy, redundant feature selection, insufficient model parameter optimization and large long period prediction error of traditional methods.

[0005] To achieve the above-mentioned purpose, the technical scheme of the application is as follows: a multimodal time sequence fusion Transformer power load prediction method based on quantum particle swarm optimization, comprising the following steps:

[0006] Step S1, the power load and the associated influencing factor data are preprocessed by using a density clustering algorithm and a time series weighted interpolation method;

[0007] Step S2, a random forest feature screening model is constructed for the key influence characteristics of the power load, and a quantum particle swarm optimization algorithm is used to optimize the parameters;

[0008] Step S3, the feature screening model is trained, and the importance threshold is used to determine the key influence feature set of the power load;

[0009] Step S4, the modal classification is performed, and the standardization processing is performed on each modal data, and a multi-modal time series feature matrix suitable for the input of the Transformer is established;

[0010] Step S5, a multi-modal time series fusion Transformer power load prediction model containing an attention modal fusion layer is constructed;

[0011] Step S6, the model constructed in step S5 is used to predict the power load in the next seven days.

[0012] Further, in step S1, the density clustering algorithm uses the DBSCAN algorithm based on spatial clustering to identify and remove outliers in the data. The specific implementation is as follows:

[0013] For any data sample , its neighborhood is:

[0014]

[0015] wherein, is the original data set, containing power load data and associated influencing factor data , is the number of samples, is the influencing factor modal dimension, which will be referred to as modal dimension hereinafter, is the Euclidean distance of the sample , is the neighborhood radius, represents all sample sets with a distance of from ;

[0016] The core judgment point is judged. If , then is the core point, and the formula is:

[0017]

[0018] wherein, The minimum number of neighborhood samples required for the core point. For the core set;

[0019] Noise points are removed; samples that are not "reachable" by any core point are defined as noise points. And there is no core point. make Transfer such samples from After removing noise points, the dataset is obtained. .

[0020] Furthermore, in step S1, the time-weighted interpolation method is implemented as follows:

[0021] Time-weighted imputation is used to fill missing values. There are missing parts Based on the temporal correlation of time series data, a weighted interpolation method is used for calculation, and the formula is as follows:

[0022]

[0023]

[0024] in, For the first The load missing value at any given time. To interpolate the window length, The first missing value The effective load value at each moment. For the first The weights of each valid sample, This is the time decay coefficient;

[0025] Finally, the complete preprocessed dataset is obtained. .

[0026] Furthermore, in step S2, the random forest feature selection model is constructed as follows:

[0027] by Medium load data As the dependent variable, related influencing factors Using as the independent variable, construct a random forest model, the formula of which is:

[0028]

[0029] in, For the set of decision tree node samples, For load classification tree, For nodes The middle belongs to the first The percentage of samples belonging to each class;

[0030] Importance of a feature is the average of the reduction of Gini coefficient after splitting on this feature in all decision trees, formula is:

[0031]

[0032] Where, is the number of decision trees in random forest, is the parent node using split in the th tree, is the child node set after splitting, , are the sample size of child node and parent node respectively. The position of particle in quantum particle swarm optimization algorithm is determined by global optimal position and individual optimal position, formula is:

[0033]

[0034]

[0035]

[0036] Where, is the position of the th particle in the th iteration in the th dimension, is the global optimal position of the th dimension in the th iteration, is the number of particles, is the individual optimal position of the th particle in the th dimension, is the contraction expansion coefficient, is the uniform random number in the interval .

[0037] The optimization objective function is:

[0038]

[0039] Where, is the number of samples in the th validation set, is the th true load value in the th validation set, is the model prediction value, the optimal parameters are obtained after optimization, and the optimized feature screening model is constructed. ​​​

[0040] Further, step S3 is implemented as follows:

[0041] Step S31, divide the data set into training set and test set according to 7:3 and test set Train the model, update the decision tree splitting rule iteratively until the loss of the model on the training set converges;

[0042] Step S32, calculate the feature importance threshold, use the "mean + 1 times standard deviation" method to determine the threshold , the formula is:

[0043]

[0044]

[0045]

[0046] wherein, is the mean of all feature importance, is the standard deviation of feature importance;

[0047] Step S33, select the features with importance greater than the threshold to form the power load key influence feature set , the formula is:

[0048]

[0049] Let dimension is , then the power load key influence feature set is represented as .

[0050] Further, step S4 is implemented as follows:

[0051] Step S41, according to the physical properties of the key influence features in the power load key influence feature set , divide into time modal , meteorological modal and historical load modal ;

[0052] Step S42, respectively use Z-score standardization on each modal data to eliminate dimension difference, the formula is:

[0053]

[0054]

[0055]

[0056] ​wherein, is the total number of time points, is the first class modal first feature value at the time point, is the corresponding original value, is the first class modal first feature mean value, is the corresponding standard deviation;

[0057] The normalized modal data is respectively denoted as , and ;

[0058] Step S43, taking the time step as the window, the normalized multi-modal data is spliced in the time dimension to construct a Transformer input feature matrix , the dimension is , is dimension, the formula is:

[0059]

[0060]

[0061] wherein, is the number of samples, is the time sequence feature matrix of the sample, is the time modal normalized feature vector of the sample at the time step.

[0062] Further, step S5 is implemented as follows:

[0063] Step S51, injecting time sequence information into the Transformer, using sine-cosine encoding, the formula is:

[0064]

[0065]

[0066] wherein, is the position encoding value of the time step dimension feature, the position encoding is superimposed with the feature matrix to obtain ;

[0067] Step S52, design inter-modal attention mechanism, dynamically calculate the weight of each modality and fuse, the formula is:

[0068]

[0069]

[0070] wherein, is the attention weight of the m-th modality, is the global feature vector of the modality, is the dimension of the m-th modality, is the similarity scoring function; Step S53, construct the Transformer encoder-decoder structure, the encoder contains 3 layers of encoding units, each layer is composed of multi-head self-attention and feedforward network, the multi-head self-attention divides the feature tensor obtained after multi-modal fusion into 8 heads, parallel computing attention and splicing, the formula is:

[0071]

[0072]

[0073] wherein,

[0074] is the output of multi-head self-attention, is the output of the m-th attention head, is the attention function, is the feature to be queried, is the feature to be matched, is the feature to be output, , , , , and are the projection matrices of the m-th head, is the output projection matrix; The feedforward network adopts two layers of full connection network, the formula is:

[0075]

[0076]

[0077] wherein, and are weight matrices, and are biases;

[0078] ​​The decoder comprises three layers of decoding units, each layer being composed of a masked multi-head self-attention, an encoder-decoder attention and an FFN, and outputting a load prediction value of a future time step;

[0079] In step S54, a mean square error is used as a loss function, and the formula is as follows:

[0080]

[0081] wherein, is a prediction time step, is a real load value of a sample at a time, is a prediction value of the model. The loss is minimized by an Adam optimizer to complete model training, and an ultimate power load prediction model is obtained .

[0082] Further, in step S6, the following is specifically implemented:

[0083] In step S61, key feature data of the previous hours at a prediction time is taken, normalized according to the method in step S4, and a feature matrix is constructed

[0084] . In step S62, the is input into the ultimate power load prediction model obtained in step S5, and the decoder of the ultimate power load prediction model outputs normalized prediction load of a future

[0085] time step . In step S63, the normalized prediction value is restored to an original load scale, and an ultimate power load prediction sequence of the future 7 days is output ,

[0086] is normalized prediction load of a time step. The application further provides a multi-modal time sequence fusion Transformer power load prediction system based on quantum particle swarm optimization, comprising a memory, a processor and computer program instructions stored in the memory and capable of being executed by the processor, when the processor executes the computer program instructions, steps of the method described above can be realized.

[0087] The application further provides a computer readable storage medium, which stores computer program instructions capable of being executed by a processor, when the processor executes the computer program instructions, steps of the method described above can be realized.

[0088] The application further provides a computer readable storage medium, which stores computer program instructions capable of being executed by a processor, when the processor executes the computer program instructions, steps of the method described above can be realized.

[0089] ​​​Compared with the prior art, the present application has the following beneficial effects: the method of the present application aims at the core technical bottlenecks of difficult multi-modal data modeling, low-efficiency parameter optimization and insufficient long-period prediction accuracy in the field of power load prediction, and takes multi-modal efficient fusion and intelligent optimization of model parameters as the core innovation direction: on the one hand, through modal classification, separate standardization and attention dynamic weighting mechanism, the dimensional barriers and semantic fragmentation of multi-modal data such as time, weather, historical load, etc. are broken, and the collaborative modeling of multi-dimensional influencing factors is realized; on the other hand, relying on quantum particle swarm optimization algorithm, the global optimization of key parameters of random forest feature screening model and Transformer prediction model is carried out, and the problem of parameter optimization falling into local optimum is effectively avoided. BRIEF DESCRIPTION OF DRAWINGS

[0090] Figure 1 The power load prediction flowchart of the present application.

[0091] Figure 2 The prediction model structure diagram of the present application. DETAILED DESCRIPTION

[0092] The technical solutions of the present application will be specifically described below in combination with the drawings.

[0093] As shown in Figure 1 , 2 , the present application embodiment provides a multi-modal time series fusion Transformer power load prediction method based on quantum particle swarm optimization, comprising the following steps:

[0094] Step S1, the power load and associated influencing factor data are preprocessed by using density clustering algorithm and time series weighted interpolation method; specifically, the density clustering algorithm uses density-based spatial clustering application with noise algorithm to identify and eliminate outliers in the data, and the specific implementation manner is as follows:

[0095] For any data sample , its neighborhood is:

[0096]

[0097] Among them, is the original data set, containing power load data and associated influencing factor data , is the number of samples, is the influencing factor modal dimension, which will be referred to as modal dimension hereinafter, is the Euclidean distance of the sample , is the neighborhood radius, indicates the distance between the sample and Distance not exceeding The set of all samples;

[0098] Judge the core judgment points, if ,but With this as the core point, the formula is:

[0099]

[0100] in, The minimum number of neighborhood samples required for the core point. For the core set;

[0101] Noise points are removed; samples that are not "reachable" by any core point are defined as noise points. And there is no core point. make Transfer such samples from After removing noise points, the dataset is obtained. .

[0102] Time-weighted imputation is used to fill missing values. There are missing parts Based on the temporal correlation of time series data, a weighted interpolation method is used for calculation, and the formula is as follows:

[0103]

[0104]

[0105] in, For the first The load missing value at any given time. To interpolate the window length, The first missing value The effective load value at each moment. For the first The weights of each valid sample, This is the time decay coefficient;

[0106] Finally, the complete preprocessed dataset is obtained. .

[0107] Step S2: Construct a random forest feature selection model for key impact characteristics of electricity load, and optimize the parameters using the quantum particle swarm optimization algorithm; specifically,

[0108] by Medium load data As the dependent variable, related influencing factors Using as the independent variable, construct a random forest model, the formula of which is:

[0109]

[0110] wherein, is the sample set of the decision tree node, is the load classification tree, is the node the proportion of samples belonging to the first class in the node;

[0111] The importance of a certain feature is the average of the reduction of Gini coefficient after the feature splits in all decision trees, and the formula is:

[0112]

[0113] wherein, is the number of decision trees in the random forest, is the parent node using split in the th tree, is the child node set after split, , are the sample numbers of the child node and the parent node, respectively;

[0114] In the quantum particle swarm optimization algorithm, the position of a particle is updated by the global optimal position and the individual optimal position, and the formula is:

[0115]

[0116]

[0117] wherein, is the position of the th particle in the th iteration in the th dimension parameter, is the global optimal position of the th dimension parameter in the th iteration, is the number of particles, is the individual optimal position of the th particle in the th dimension, is the contraction-expansion coefficient, is a uniform random number in the interval

[0118] The optimization objective function is:

[0119]

[0120] wherein, is the number of decision trees in the random forest, ​The number of fold validation set samples, For the The first real load value of fold validation set , The model predicted value, the optimal parameter after optimization , build an optimized feature screening model .

[0121] Step S3, train the feature screening model and determine the key influence feature set of power load combined with the importance threshold; specifically,

[0122] Step S31, divide the According to 7:3, it is divided into training set And test set Training, iteratively updating the decision tree splitting rule until the model loss on the training set converges;

[0123] Step S32, calculate the feature importance threshold, use the "mean + 1 times standard deviation" method to determine the threshold , the formula is:

[0124]

[0125]

[0126]

[0127] Among them, The mean of all feature importance, The standard deviation of feature importance;

[0128] Step S33, select the features with importance greater than the threshold to form the key influence feature set of power load , the formula is:

[0129]

[0130] Let The dimension is , then the key influence feature set of power load is represented as .

[0131] Step S4, based on modal classification and standardization processing of each modal data, establish a multi-modal time series feature matrix suitable for Transformer input; specifically,

[0132] Step S41, according to the physical properties of the key influence features in the key influence feature set , it is divided into time modality , meteorological modality And historical load modality ;

[0133] Step S42, Z-score standardization is used for each modality data respectively to eliminate dimensional differences, and the formula is:

[0134]

[0135]

[0136]

[0137] wherein, is the total number of time points, is the th feature of the th modality, is the standardized value of the th feature of the th modality at the th time point, is the corresponding original value, is the mean value of the th feature of the

[0138] th modality, is the corresponding standard deviation; After standardization, each modality data is respectively denoted as ,

[0139] and ; Step S43, taking the time step as the window, the standardized multi-modal data is spliced according to the time dimension to construct the Transformer input feature matrix , the dimension is ,

[0140] is dimension, and the formula is:

[0141]

[0142] wherein, is the sample number, is the time sequence feature matrix of the th sample, is the time modality standardized feature vector of the th time step of the th sample.

[0143] Step S5, a multi-modal time sequence fusion Transformer power load prediction model containing an attention modality fusion layer is constructed; specifically,

[0144] Step S51, the Transformer is injected with time sequence information, and the sine-cosine encoding is used, and the formula is:

[0145]

[0146]

[0147] wherein, is the position encoding value of the -th feature in the -th dimension, the position encoding is superimposed with the feature matrix to obtain ;

[0148] Step S52, design an inter-modal attention mechanism to dynamically calculate the weight of each modality and fuse, the formula is:

[0149]

[0150]

[0151] wherein, is the attention weight of the -th modality, is the global feature vector of the modality, is the dimension of the -th modality, is a similarity scoring function;

[0152] Step S53, construct a Transformer encoder-decoder structure, the encoder contains 3 layers of encoding units, each layer is composed of multi-head self-attention and feedforward network, the multi-head self-attention divides the feature tensor obtained after multi-modal fusion into 8 heads, parallel computing attention and splicing, the formula is:

[0153]

[0154]

[0155] wherein, is the output of the multi-head self-attention, is the output of the multi-head self-attention, is the output of a certain attention head, is an attention function, is the feature to be queried, is the feature to be matched, is the feature to be output, , , and are the projection matrices of the -th head, is the output projection matrix;

[0156] The feedforward network uses a two-layer fully connected network, and the formula is:

[0157]

[0158] in, and This is the weight matrix. and For bias;

[0159] The decoder contains three layers of decoding units, each consisting of a masked multi-head self-attention, an encoder-decoder attention, and an FFN, which outputs the load prediction value for future time steps;

[0160] Step S54: Use the mean squared error as the loss function, the formula is:

[0161]

[0162] in, To predict the time step, For the first The first sample The actual load value at any given time. These are the model's predicted values;

[0163] The model is trained by minimizing the loss using the Adam optimizer, resulting in the final power load prediction model. .

[0164] Step S6: Predict the power load for the next seven days using the model constructed in step S5; specifically,

[0165] Step S61: Take the time before the prediction time The key feature data for each hour are standardized according to the method in step S4 to construct a feature matrix. ;

[0166] Step S62, will Input the final power load prediction model obtained in step S5 Final power load forecasting model Decoder output future Standardized forecast load at each time step ;

[0167] Step S63: Restore the standardized forecast values ​​to the original load scale, and finally output the power load forecast sequence for the next 7 days. , Standardized forecast load for a given time step.

[0168] The application further provides a multi-modal time sequence fusion Transformer power load prediction system based on quantum particle swarm optimization, comprising a memory, a processor and computer program instructions stored on the memory and capable of being executed by the processor, when the processor executes the computer program instructions, the steps of the method described above can be realized.

[0169] The application further provides a computer readable storage medium, which stores computer program instructions capable of being executed by a processor, when the processor executes the computer program instructions, the steps of the method described above can be realized.

[0170] The above is the preferred embodiment of the application, any change made according to the technical solution of the application, as long as the function generated does not exceed the scope of the technical solution of the application, belongs to the protection scope of the application.

Claims

1. A multi-modal time series fusion Transformer power load forecasting method based on quantum particle swarm optimization, characterized in that, Comprising the following steps: Step S1, using density clustering algorithm and time series weighted interpolation method to preprocess power load and related influence factor data; Step S2, constructing a random forest feature selection model for power load key influence characteristics, and optimizing parameters through quantum particle swarm optimization algorithm; Step S3, training the feature selection model and determining the power load key influence feature set in combination with the importance threshold value; Step S4, based on modal classification and standardization processing of each modal data, establishing a multi-modal time series feature matrix suitable for the input of the Transformer; Step S5, constructing a multi-modal time series fusion Transformer power load prediction model containing an attention modal fusion layer; Step S6, predicting the future seven-day power load through the model constructed in step S5.

2. The multi-modal temporal fusion Transformer power load forecasting method based on quantum particle swarm optimization according to claim 1, characterized in that, In step S1, the density clustering algorithm adopts the DBSCAN algorithm to identify and eliminate outliers in the data based on spatial clustering, and the specific implementation is as follows: For any data sample , which neighborhood is: wherein, is the original dataset containing power load data and associated influencing factor data , is the number of samples, is the influencing factor modal dimension, i.e. the number of modal classes, is the Euclidean distance of a sample to a sample, is the neighborhood radius, denotes the set of all samples with a distance not exceeding to the sample, . If the core point is determined, if then the core point is determined, and the formula is: wherein, is the minimum number of neighboring samples required for a core point, is the set of core points; The noise points are removed, and the samples not "reachable" by any core point are defined as noise points, i.e. and there is no core point Make , remove such samples from , and obtain the data set after removing the noise points .

3. The multi-modal temporal fusion Transformer power load forecasting method based on quantum particle swarm optimization according to claim 2, characterized in that, In step S1, the time series weighted interpolation method, the specific implementation is as follows: The missing values are filled by time series weighted interpolation There are missing values in the middle Based on the time correlation of time series data, the weighted interpolation method is used to calculate, and the formula is: wherein, is the first is the missing load value at the time instant, is the interpolation window length, is the valid load value at the time instant preceding the missing value, is the weight of the first is the weight of the first is the weight of the first is the time decay coefficient; Final complete pre-processed dataset .

4. The multi-modal temporal fusion Transformer power load forecasting method based on quantum particle swarm optimization according to claim 3, characterized in that, In step S2, the random forest feature selection model is constructed as follows: With medium load data dependent variable, the correlation of influencing factors independent variable, the formula of the random forest model is: wherein, is a set of samples of a decision tree node, is a load classification tree, is a node is a proportion of samples belonging to the class in the node. For a feature The importance of a feature is the average reduction in Gini coefficient across all decision trees after splitting on that feature, calculated as: wherein, is the number of decision trees in the random forest, is the number of trees in the random forest, is the parent node of the split used in the split in the is the number of samples in the child node, and , is the number of samples in the parent node, respectively. In the quantum particle swarm optimization algorithm, the particle position update is determined by the global optimal position and the individual optimal position, and the formula is: wherein, is the number of particles, is the number of iterations, is the position of the i-th particle in the j-th dimension in the i-th iteration, is the global optimal position of the i-th dimension in the i-th iteration, is the number of particles, is the number of iterations, is the individual optimal position of the i-th particle in the j-th dimension, is the number of particles, is the number of iterations, is the individual optimal position of the i-th particle in the j-th dimension, is the number of particles, is the number of iterations, is the number of particles, is the number of iterations, The optimization objective function is: Wherein, is the first fold validation set sample number, is the first fold validation set the first true load value, is the model predicted value, the optimal parameters are obtained after optimization , and an optimized feature screening model is constructed .

5. The multi-modal temporal fusion Transformer power load forecasting method based on quantum particle swarm optimization according to claim 4, characterized in that, Step S3, the specific implementation is as follows: Step S31, divide into a training set and a test set Train, iteratively update the decision tree split rule until the loss of the model on the training set converges; Step S32, the feature importance threshold value is calculated, and the "mean + 1 times standard deviation" method is used to determine the threshold value The formula is: wherein, is the mean of the importance of all features, is the standard deviation of the importance of features; Step S33, screening the feature set with importance greater than a threshold value to form a power load key influence feature set The formula is: Let The dimension of The set of critical influence features of the power load is represented as .

6. The multi-modal temporal fusion Transformer power load forecasting method based on quantum particle swarm optimization according to claim 1, characterized in that, Step S4, the specific implementation is as follows: Step S41, according to the power load key influence feature set The physical attributes of the key influence features are divided into time modalities , weather modalities , and historical load modalities ; Step S42, Z-score standardization is used for each modal data to eliminate dimension differences, and the formula is: wherein, is the total number of time points, is the number of the class modal number of the feature at the time point, is the corresponding raw value, is the number of the class modal number of the feature mean value, is the corresponding standard deviation; The standardized data of each modality are denoted as , and , respectively. Step S43, concatenating the normalized multi-modal data in the time dimension as a window , and constructing a Transformer input feature matrix , with a dimension of , ​​ wherein, is the number of samples, is the time series feature matrix of the th sample, is the time modality standardized feature vector of the th time step of the th sample.

7. The multi-modal temporal fusion Transformer power load forecasting method based on quantum particle swarm optimization according to claim 6, characterized in that, Step S5, the specific implementation is as follows: Step S51, inject time series information into the Transformer, use sine-cosine encoding, and the formula is: wherein, is the first time step, dimensional position encoding value for the feature, the position encoding is superimposed with the feature matrix to obtain ; Step S52, design an inter-modal attention mechanism to dynamically calculate the weight of each modal and fuse them, and the formula is: wherein, is the attention weight of the modal global feature vector, is the dimension of the similarity score function; Step S53, a Transformer encoder-decoder structure is constructed, the encoder contains 3 layers of encoding units, each layer is composed of multi-head self-attention and a feedforward network, and the multi-head self-attention is used to process the feature tensor obtained after multi-modal fusion is divided into 8 heads, and the attention is calculated in parallel and then spliced, and the formula is: in, For multi-head self-attention output, To concatenate the outputs of multiple attention heads along their feature dimensions, This represents the output of a certain attention head. Let be the attention function. For the features to be queried, For the features used for matching, For the features used in the output, , , and For the first Projection matrix of each head, To output the projection matrix; The feedforward network adopts a two-layer fully connected network, and the formula is: wherein, and is a weight matrix, and is a bias; The decoder contains 3 decoding units, each of which consists of a masked multi-head self-attention, an encoder-decoder attention, and an FFN, and outputs the future time step load prediction value; Step S54, use mean square error as the loss function, and the formula is: wherein, is the predicted time step, is the true load value for the th sample at the th time instant, is the model predicted value; The model training is completed by minimizing the loss through an Adam optimizer to obtain a final power load prediction model .

8. The multi-modal temporal fusion Transformer power load forecasting method based on quantum particle swarm optimization according to claim 1, characterized in that, Step S6, the specific implementation is as follows: Step S61, take the predicted time before hourly key feature data, normalized by the method of step S4, construct a feature matrix ; Step S62, obtaining the final power load prediction model from the input step S5 , the final power load prediction model outputs the normalized predicted load for future time steps from the decoder ; Step S63, restore the standardized predicted value to the original load scale, finally output the power load prediction sequence of the next 7 days , is the standardized predicted load for a certain time step.

9. A multi-modal time series fusion Transformer power load forecasting system based on quantum particle swarm optimization, characterized in that, A computer readable storage medium having stored thereon computer program instructions capable of being executed by a processor, when the processor executes the computer program instructions, the steps of the method as claimed in any one of claims 1-8 can be implemented.

10. A computer readable storage medium having stored thereon computer program instructions capable of being executed by a processor, when the processor executes the computer program instructions, the steps of the method as claimed in any one of claims 1-8 can be implemented.

Images