Power load prediction method based on cloud model improved Transform

Abstract

The invention discloses a power load prediction method based on a cloud model improved Transform, and belongs to the technical field of power load prediction. A dual-path CMIT model quantifies the uncertainty of load data into fuzzy membership, entropy, hyper entropy and other characteristics by introducing a cloud model, the anti-interference capability of input data to noise is enhanced, and the power load prediction accuracy is improved. By combining an improved Transform model structure of a self-attention mechanism and dynamically capturing a complex non-linear relationship among different variables, efficient modeling of multi-dimensional load data is realized, fuzzy description characteristics are generated through a cloud model, input with randomness and fuzziness is provided, and when data distribution changes or external interference exists, the model can be used for modeling the multi-dimensional load data in real time. Compared with the prior art, the method has the advantages that high prediction precision can still be kept, and through combination of cloud model preprocessing and sequence modeling of a Transform model structure, the method can adapt to load data characteristics in different scenes and can flexibly capture periodicity, randomness and trend changes of load data.

Description

Technical Field

[0001] The present invention belongs to the technical field of power load forecasting, and in particular relates to a power load forecasting method based on cloud model-based improved Transformer. Background Art

[0002] Power load forecasting is an important part of power system operation and planning, and its accuracy directly affects the economy and safety of power grid operation. However, due to the complex characteristics of power load, such as significant randomness, nonlinearity, and periodicity, traditional forecasting methods show obvious limitations when dealing with complex scenarios (such as large-scale network cluster load forecasting). Especially in multidimensional load data, noise, uncertainty, and complex correlations between multidimensional variables are bottlenecks that traditional methods are difficult to overcome.

[0003] To solve these problems, prediction methods based on deep learning have received widespread attention in recent years.

[0004] As a model with self-attention mechanism, Transformer has gradually been introduced into the field of power load forecasting due to its excellent performance in natural language processing and sequence modeling.

[0005] However, the traditional load forecasting model based on the standard Transformer model lacks the ability to model uncertainty, is sensitive to noise, lacks integration of multi-dimensional spatiotemporal characteristics, and is difficult to optimize parameters. Its performance is still limited when faced with multi-dimensional power load data. Summary of the invention

[0006] In order to solve the problems raised in the above background technology, the present invention provides an electric power load forecasting method based on cloud model improved Transformer to solve the problems of modeling uncertainty, insufficient integration of multi-dimensional spatiotemporal characteristics, and high difficulty in parameter optimization existing in traditional load forecasting models.

[0007] To achieve the above object, the present invention provides the following technical solutions: The power load forecasting method based on the cloud model to improve Transformer includes the following steps: S1: Collect and pre-process the load data sequence of electricity; S2: Embed the cloud model into the layer normalization module of the Transformer model structure to obtain an improved Transformer model structure; S3: Design a dual-path CMIT model. The dual-path CMIT model includes two paths. One path is the traditional Transformer model structure, and the other path is the improved Transformer model structure in S2. The traditional Transformer model structure is used to capture the explicit features in the time series of the input preprocessed data, and the improved Transformer model structure is used to capture the implicit fuzzy features and random features of the input preprocessed data. Both paths are connected to the fusion module of the dual-path CMIT model. The fusion module is used to integrate the outputs of the two paths by weighted fusion and dynamically adjust the weights of the dual-path CMIT model. S4: Using the preprocessed data, the dual-path CMIT model is trained and optimized using the back propagation algorithm. When the number of training rounds reaches the upper limit or the performance of the dual-path CMIT model reaches expectations, the training of the dual-path CMIT model is terminated; S5: preprocess the data to be predicted and send it to the trained dual-path CMIT model, and the trained dual-path CMIT model outputs the prediction result.

[0008] Preferably, the pretreatment comprises the following steps: S1.1: Data cleaning operation; remove outliers and missing values ​​from the load data sequence, and use local interpolation and wavelet transform noise filtering methods for the load data sequence; S1.2: Normalization processing operation: normalize the values ​​of the load data sequence to the range of [0,1]; S1.3: Time series feature construction operation; extract the time lag term of the load data sequence, extract the periodic characteristics of the load data sequence, and then integrate the external features to complete the preprocessing.

[0009] Preferably, in S1.3, the periodic characteristics include characteristics of year, month, day, and season, and the external characteristics include characteristics of temperature, humidity, and holiday markings.

[0010] Preferably, in S2, embedding the cloud model into the layer normalization module of the Transformer model structure specifically includes the following steps: S2.1: Calculate the expectation, entropy and super entropy of the data through the forward generator of the cloud model to generate cloud droplets; S2.2: Replace the traditional layer normalization in the Transformer model structure with a cloud model-based normalization method, replacing the mean and variance with the membership information of cloud droplets; S2.3: Capture and adapt to the fuzziness and randomness characteristics of power load data by dynamically adjusting the expectation, entropy and super entropy of the cloud model.

[0011] Preferably, in the fusion module, a particle swarm optimization algorithm is used to dynamically adjust the weights of the dual-path CMIT model, including the following steps: S3.1: Initialization; the fusion module randomly generates a swarm of particles, each particle represents the weight of a set of paths; S3.2: Fitness calculation: Calculate the fitness value of the particle based on the prediction error, which includes mean square error and MSE; S3.3: Speed ​​and position update; dynamically adjust weights based on the particle's historical optimal position and global optimal position; S3.4: Iterate until convergence; repeat S3.2-S3.3, and stop updating when the fitness value reaches the preset threshold or the number of updates exceeds the maximum limit.

[0012] Preferably, in S4, a back propagation algorithm based on an Adam optimizer is used in combination with a learning rate scheduler to optimize the parameters of the dual-path CMIT model.

[0013] Preferably, S4 also includes a validation phase, and after the training phase is completed, a cross-validation method is used to evaluate the generalization ability of the dual-path CMIT model.

[0014] Compared with the prior art, the present invention has the following beneficial effects: This application proposes an improved Transformer power load forecasting method based on a cloud model. The dual-path CMIT model introduces a cloud model to quantify the uncertainty of load data into characteristics such as fuzzy membership, entropy, and super entropy. The anti-interference ability of the input data to noise is enhanced, and the improved Transformer model structure combined with the self-attention mechanism is used to dynamically capture the complex nonlinear relationship between different variables, and realize efficient modeling of multidimensional load data. Fuzzy description features are generated through the cloud model, which provides a random and fuzzy input for the Transformer model structure, so that the model can still maintain a high prediction accuracy when the data distribution changes or there is external interference (such as noise, outliers). By combining cloud model preprocessing with sequence modeling of the Transformer model structure, it can not only adapt to the characteristics of load data in different scenarios, but also flexibly capture the periodicity, randomness and trend changes of load data. This scenario adaptability makes the dual-path CMIT model an effective tool for dealing with complex load forecasting problems. BRIEF DESCRIPTION OF THE DRAWINGS

[0015] Figure 1 It is a schematic diagram of the workflow of the dual-path CMIT model; Figure 2 This is the data preprocessing flow chart; Figure 3 This is a flow chart of the normalization layer of the cloud model; Figure 4 Optimize the flow chart for particle swarm; Figure 5 This is a comparison chart of the prediction results. DETAILED DESCRIPTION

[0016] In order to facilitate those skilled in the art to understand the technical content of the present invention, the present invention is further described in detail below in conjunction with the accompanying drawings and specific examples. It should be understood that the specific examples described herein are only used to explain the present invention and are not used to limit the present invention.

[0017] Example 1 The power load forecasting method based on the cloud model to improve Transformer includes the following steps: S1: Extract historical load data and weather data from the power monitoring system, and combine them with public holiday information to form a load data sequence, perform preprocessing, and then use 80% of the preprocessed data as the training set, 20% of the data as the test set, and the validation set is randomly divided into 10% from the training set; Figure 2 As shown, the preprocessing includes the following steps: S1.1: Data cleaning operation; remove outliers and missing values ​​from the load data sequence by using the triple standard deviation method and linear interpolation method, and use the wavelet transform noise filtering method for the load data sequence; S1.2: Normalization processing operation: normalize the values ​​of the load data sequence to the range of [0,1]; S1.3: Time series feature construction operation; extract the time lag term of the load data sequence, extract the periodic characteristics of the load data sequence, and then integrate the external characteristics to complete the preprocessing. Among them, the periodic characteristics include the characteristics of year, month, day, and season, and the external characteristics include the characteristics of temperature, humidity, and holiday marks.

[0018] S2: The traditional Transformer model structure includes an input module, an encoder module, a decoder module, and an output module. The cloud model is embedded in the layer normalization module of the encoder module of the Transformer model structure to obtain an improved Transformer model structure, where the Transformer model structure has 6 layers, 8 heads, a hidden layer dimension of 256, and an activation function of ReLU; Figure 3 As shown in the figure, embedding the cloud model into the layer normalization module of the Transformer model structure specifically includes the following steps: S2.1: Calculate the expectation, entropy and super entropy of the data through the forward generator of the cloud model to generate cloud droplets; S2.2: Replace the traditional layer normalization in the Transformer model structure with a cloud model-based normalization method, replacing the mean and variance with the membership information of cloud droplets; S2.3: Capture and adapt to the fuzziness and randomness characteristics of power load data by dynamically adjusting the expectation, entropy and super entropy of the cloud model.

[0019] S3: Figure 1 As shown in FIG. 1 , a dual-path CMIT model is designed. The dual-path CMIT model includes two paths. One path is the traditional Transformer model structure, and the other path is the improved Transformer model structure in S2. The traditional Transformer model structure is used to capture the explicit features in the time series of the input preprocessed data, and the improved Transformer model structure is used to capture the implicit fuzzy features and random features of the input preprocessed data. Both paths are connected to the fusion module of the dual-path CMIT model. The fusion module is used to integrate the outputs of the two paths by weighted fusion and dynamically adjust the weights of the dual-path CMIT model. In the fusion module, the particle swarm optimization algorithm is used to dynamically adjust the weights of the dual-path CMIT model, as shown in FIG. Figure 4 As shown, the following steps are included: S3.1: Initialization; The fusion module randomly generates a particle swarm, each particle represents the weight of a group of paths, initializes the velocity vector and the individual optimal position, and the two paths of the dual-path CMIT model are a group of paths. The weight of a group of paths refers to the combination of the weight of the first path ω1 and the weight of the second path ω2, which can be expressed as [ω1, ω2]; S3.2: Fitness calculation: Calculate the fitness value of the particle based on the prediction error, which includes mean square error and MSE; S3.3: Speed ​​and position update; dynamically adjust weights based on the particle's historical optimal position and global optimal position; S3.4: Iterate until convergence; repeat S3.2-S3.3, and stop updating when the fitness value reaches the preset threshold or the number of updates exceeds the maximum limit.

[0020] S4: The parameters of the dual-path CMIT model are trained and optimized by using the back-propagation algorithm based on the Adam optimizer and the learning rate scheduler through the preprocessed data. The initial value of the learning rate is 0.001. When the number of training rounds reaches the upper limit or the performance of the dual-path CMIT model reaches the expectation, the training of the dual-path CMIT model is terminated. After the training phase is completed, the cross-validation method is used to evaluate the generalization ability of the dual-path CMIT model. S5: preprocess the data to be predicted and send it to the trained dual-path CMIT model, and the trained dual-path CMIT model outputs the prediction result.

[0021] In this embodiment, the present application proposes a power load forecasting method based on a cloud model to improve the Transformer. The dual-path CMIT model introduces a cloud model to quantify the uncertainty of the load data into characteristics such as fuzzy membership, entropy, and super entropy. The anti-interference ability of the input data to noise is enhanced, and the present application dynamically captures the complex nonlinear relationship between different variables by combining the improved Transformer model structure with the self-attention mechanism, and realizes efficient modeling of multidimensional load data. The fuzzy description features are generated by the cloud model, and the input with randomness and fuzziness is provided for the Transformer model structure, so that the model can still maintain a high prediction accuracy when the data distribution changes or there is external interference (such as noise, outliers). By combining cloud model preprocessing and sequence modeling of the Transformer model structure, it can not only adapt to the characteristics of load data in different scenarios, but also flexibly capture the periodicity, randomness and trend changes of load data. This scenario adaptability makes the dual-path CMIT model an effective tool for dealing with complex load forecasting problems.

[0022] Experimental process: Hardware environment: NVIDIA A100 GPU, 64-core CPU, 256GB memory; Software environment: Python 3.9, PyTorch 2.0, NumPy 1.24; The weather forecast data for the next seven days is input into the dual-path CMIT model to generate load forecast values, which are then compared and analyzed with the actual values; The experimental results are expressed as mean square error (MSE), mean absolute error (MAE), and coefficient of determination (R 2 ) are evaluated by three indicators, as shown in Table 1: Table 1

[0023] Comparison of the predicted curve and actual curve of the traditional Transformer model and the dual-path CMIT model Figure 5 As shown in Figure 2, the MSE and MAE of the dual-path CMIT model are 41.4% and 38.9% lower than those of the traditional Transformer model, respectively, and the prediction error is significantly reduced. 2 It increased from 0.85 to 0.93, indicating that the fitting ability of the dual-path CMIT model on complex power load data is significantly enhanced.

[0024] Based on sample testing, the average prediction error of the dual-path CMIT model is 1.85%, which is better than the industry average of 5%. Compared with traditional models, the inference speed of the dual-path CMIT model is increased by about 30% under the same hardware configuration. In the presence of noisy data (10% random outliers), the prediction performance of the dual-path CMIT model only decreases by 3.5%.

Claims

1. The power load forecasting method based on the cloud model to improve Transformer is characterized by: The following steps are involved: S1: Collect and pre-process the load data sequence of electricity; S2: Embed the cloud model into the layer normalization module of the Transformer model structure to obtain an improved Transformer model structure; S3: Design a dual-path CMIT model. The dual-path CMIT model includes two paths. One path is the traditional Transformer model structure, and the other path is the improved Transformer model structure in S2. The traditional Transformer model structure is used to capture the explicit features in the time series of the input preprocessed data, and the improved Transformer model structure is used to capture the implicit fuzzy features and random features of the input preprocessed data. Both paths are connected to the fusion module of the dual-path CMIT model. The fusion module is used to integrate the outputs of the two paths by weighted fusion and dynamically adjust the weights of the dual-path CMIT model. S4: Using the preprocessed data, the dual-path CMIT model is trained and optimized using the back propagation algorithm. When the number of training rounds reaches the upper limit or the performance of the dual-path CMIT model reaches expectations, the training of the dual-path CMIT model is terminated; S5: preprocess the data to be predicted and send it to the trained dual-path CMIT model, and the trained dual-path CMIT model outputs the prediction result.

2. The power load forecasting method based on cloud model improved Transformer according to claim 1 is characterized in that: The preprocessing includes the following steps: S1.1: Data cleaning operations; Remove outliers and missing values ​​from the load data sequence, and use local interpolation and wavelet transform noise filtering methods on the load data sequence; S1.2: Normalization processing operation: normalize the values ​​of the load data sequence to the range of [0,1]; S1.3: Time series feature construction operation; extract the time lag term of the load data sequence, extract the periodic characteristics of the load data sequence, and then integrate the external features to complete the preprocessing.

3. The power load forecasting method based on cloud model improved Transformer according to claim 2 is characterized in that: In S1.3, the periodic features include the features of year, month, day, and season, and the external features include the features of temperature, humidity, and holiday markings.

4. The power load forecasting method based on cloud model improved Transformer according to claim 1 is characterized in that: In S2, embedding the cloud model into the layer normalization module of the Transformer model structure specifically includes the following steps: S2.1: Calculate the expectation, entropy and super entropy of the data through the forward generator of the cloud model to generate cloud droplets; S2.2: Replace the traditional layer normalization in the Transformer model structure with a cloud model-based normalization method, replacing the mean and variance with the membership information of cloud droplets; S2.3: Capture and adapt to the fuzziness and randomness characteristics of power load data by dynamically adjusting the expectation, entropy and super entropy of the cloud model.

5. The power load forecasting method based on cloud model improved Transformer according to claim 1 is characterized in that: In the fusion module, the particle swarm optimization algorithm is used to dynamically adjust the weights of the dual-path CMIT model, including the following steps: S3.1: Initialization; the fusion module randomly generates a swarm of particles, each particle represents the weight of a set of paths; S3.2: Fitness calculation: Calculate the fitness value of the particle based on the prediction error, which includes mean square error and MSE; S3.3: Speed ​​and position update; dynamically adjust weights based on the particle's historical optimal position and global optimal position; S3.4: Iterate until convergence; repeat S3.2-S3.3, and stop updating when the fitness value reaches the preset threshold or the number of updates exceeds the maximum limit.

6. The power load forecasting method based on cloud model improved Transformer according to claim 1 is characterized in that: In S4, the back propagation algorithm based on the Adam optimizer is used in combination with the learning rate scheduler to optimize the parameters of the dual-path CMIT model.

7. The power load forecasting method based on cloud model improved Transformer according to claim 6 is characterized in that: S4 also includes a validation phase. After the training phase is completed, the generalization ability of the dual-path CMIT model is evaluated using a cross-validation method.

Images