New energy power prediction method and system based on cross-architecture
By using a cross-architecture knowledge distillation framework and the adaptive distillation network SeqAdaptNet, combined with a multi-head attention mechanism, the problems of high model complexity, large resource requirements, and poor generalization ability with small sample data in new energy power prediction are solved, achieving efficient and accurate power prediction on resource-constrained equipment.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2025-10-14
- Publication Date
- 2026-03-13
AI Technical Summary
Existing time series forecasting models for new energy power forecasting suffer from problems such as high model complexity, large computational resource requirements, insufficient long-term data capture, poor generalization ability with small sample data, and unbalanced forecasting needs across multiple time scales, making it difficult to achieve real-time deployment and high-precision forecasting on resource-constrained edge devices.
We employ a cross-architecture knowledge distillation framework, utilizing the adaptive distillation network SeqAdaptNet, multiple teacher models, and a sequence adaptation module to dynamically generate weights. Combined with a multi-head attention mechanism, this guides a lightweight student model to predict power data, achieving cross-architecture knowledge transfer and efficient prediction.
It significantly improves the accuracy and efficiency of lightweight student models in long-term, short-term and ultra-short-term power forecasting tasks, reduces computational resource consumption, and is suitable for real-time applications in resource-constrained scenarios. In particular, it outperforms independent training or traditional distillation methods on small sample datasets.
Smart Images
Figure CN121663451A_ABST
Abstract
Description
Technical Field
[0001] This invention belongs to the field of artificial intelligence and power forecasting technology, and relates to a new energy power forecasting method and system based on cross-architecture. Specifically, it relates to a knowledge distillation framework based on a multi-head attention mechanism, which is used to guide a lightweight student model through a teacher model to improve the performance and efficiency of time series forecasting tasks. Background Technology
[0002] Time series forecasting technology has crucial application value in power systems, especially in the field of renewable energy power forecasting. With the large-scale integration of renewable energy sources such as wind and solar power into the grid, the volatility and intermittency of their output power pose severe challenges to the stable operation of the grid, dispatch optimization, and energy management. Accurately forecasting renewable energy power output (such as wind farm output and solar power generation) has become a key technical means to ensure grid security and improve energy utilization efficiency. Traditionally, time series forecasting models rely on statistical methods or shallow machine learning models, but when processing renewable energy power data, they often struggle to effectively capture its complex nonlinear time-series dependencies and high-frequency fluctuation characteristics.
[0003] In recent years, deep learning models have made significant progress in time series prediction tasks. For example, Transformer-based models (such as PatchTST and iTransformer) can capture long-term dependencies through self-attention mechanisms, making them suitable for multivariate time series prediction; Convolutional Neural Network (CNN)-based models (such as ModernTCN) efficiently extract short-term features through local convolution operations, offering advantages in computational efficiency. These models have demonstrated high prediction accuracy on publicly available benchmark datasets (such as the ETTh and ETTm series).
[0004] However, in practical applications of new energy power forecasting, the above model still faces many challenges:
[0005] High model complexity and deployment cost: Large models such as Transformer have a large number of parameters and high computational resource requirements, making it difficult to achieve real-time deployment on resource-constrained edge devices (such as power plant local controllers and embedded systems), which limits their application in latency-sensitive scenarios.
[0006] Insufficient long-term dependency capture: Although lightweight models such as CNNs are computationally efficient, they are weak in modeling long-term time series patterns and have difficulty adapting to the long-term dependency characteristics such as periodicity and trend in new energy output data.
[0007] Small sample data has poor generalization ability: New energy power generation data (such as wind power and photovoltaic power output) usually have a limited sample size and are affected by factors such as weather and seasons, resulting in high noise and non-stationary characteristics. This makes it easy for complex models to overfit, while simple models have difficulty learning stable patterns from them.
[0008] Multi-timescale forecasting requirements: Power dispatching requires models to maintain high accuracy across multiple timescales, including long-term (e.g., day-ahead forecasting), short-term (e.g., hourly), and ultra-short-term (e.g., minute-ahead) forecasting. Existing models often exhibit uneven performance across different scales.
[0009] To balance model accuracy and efficiency, knowledge distillation has been introduced into time series forecasting tasks. This involves transferring knowledge from a large teacher model to a lightweight student model to improve the latter's performance. However, traditional knowledge distillation methods have significant limitations in new energy power forecasting.
[0010] Insufficient modeling of temporal dependencies makes it difficult to effectively convey the teacher model's ability to represent complex temporal patterns.
[0011] Knowledge transfer between cross-architecture models is inefficient, especially when the teacher model and student model have significant structural differences.
[0012] In scenarios with small sample sizes, the teacher model may experience a decline in distillation performance due to overfitting, and may even be worse than the student model trained independently.
[0013] Therefore, there is an urgent need for a cross-architecture knowledge distillation framework that can adapt to the characteristics of new energy power data and balance prediction accuracy and computational efficiency, so as to improve the performance of lightweight models in complex time-series tasks and meet the strict requirements of real-time performance and resource constraints of actual power systems. Summary of the Invention
[0014] The purpose of this invention is to address the shortcomings of existing technologies by providing a cross-architecture-based method and system for predicting new energy power. In long-term, short-term, and ultra-short-term prediction tasks on large benchmark datasets and small new energy power generation datasets, a lightweight student model is guided by a teacher model. This significantly improves the performance and efficiency of the student model in time series prediction tasks while maintaining low resource consumption, making it suitable for deployment in resource-constrained scenarios.
[0015] To achieve the above objectives, the technical solution adopted by the present invention is as follows:
[0016] In a first aspect, the present invention provides a new energy power forecasting method based on cross-architecture architecture, comprising:
[0017] Acquire historical, timestamped new energy power data and manually annotate it;
[0018] The power data is preprocessed and then divided into training, testing, and validation sets according to a certain ratio.
[0019] An adaptive distillation network, SeqAdaptNet, is constructed and trained using a training set. The SeqAdaptNet includes multiple teacher models, one student model, and a teacher output fusion module. Each teacher model's output is connected in series with a sequence adaptation module, which generates corresponding weights for the current teacher model. The teacher output fusion module is used to perform weighted fusion of the outputs of multiple teacher models with their corresponding weights.
[0020] The trained student model is tested using the test set, and the tested student model is validated using the validation set.
[0021] Use tested and validated student models to predict renewable energy power generation.
[0022] Preferably, the new energy power data includes at least one of the following: wind speed, load, oil temperature, and electric field output of the power equipment.
[0023] Preferably, the power equipment includes at least one of the following types: wind farm, photovoltaic power station, and power transformer.
[0024] Preferably, the preprocessing includes: normalizing the power data.
[0025] Preferably, the construction and training process of the adaptive distillation network SeqAdaptNet includes:
[0026] 1) Teacher model loading:
[0027] Load multiple pre-trained teacher models, initialize the model parameters of the teacher models through pre-training checkpoints, set the models to evaluation mode, and freeze their parameters to ensure the stability of knowledge transfer.
[0028] 2) Student model initialization:
[0029] Initialize the student model, but the number of model parameters and computational complexity of the student model are lower than those of the teacher model;
[0030] 3) Sequence adaptation module processing:
[0031] The preprocessed power data x in the training set is mapped to the hidden dimension through the input projection layer to obtain the features. ;
[0032] Will Multiple attention heads are used to compute the dependencies between time steps in the sequence in parallel, generating context-sensitive features. ;
[0033] By global average pooling The sequence dimension is used for mean pooling to obtain the pooled features. ;
[0034] Generate weight distribution through the output layer :
[0035] i = 1,2,...,N
[0036] in for weight For bias, N is the number of teacher models. Let the weights of the i-th teacher model satisfy the following condition: ;
[0037] 4) Teacher output combination: based on the weights output by the sequence adaptive module. The outputs of multiple teacher models are weighted and fused:
[0038] = ;
[0039] in This is the output of the i-th teacher model; Output the weights for the sequence adaptive module corresponding to the i-th teacher model; N represents the total number of teacher models.
[0040] 5) Total loss calculation:
[0041]
[0042] ;
[0043] softmax( / T)) ;
[0044] in These are the weighting coefficients, and T is the temperature parameter; This represents the output of the student model. With real labels The mean squared error loss; express and KL divergence loss, Denotes the KL divergence function; log-softmax( ) is the logarithmic softmax function;
[0045] 6) Parameter optimization: The Adam optimizer is used to optimize the parameters of the student model and the sequence adaptation module, while the parameters of the teacher model are kept frozen.
[0046] Preferably, in the SeqAdaptNet adaptive distillation network, multiple teacher models independently select models with larger parameters and the same or different architectures. The input of the teacher model includes normalized power data, and the output is predicted power data. The student model selects a lightweight model, and the input of the student model includes normalized power data, and the output is predicted power data. The architectures of the model with larger parameters and the lightweight model can be the same or different.
[0047] Preferably, only the student model is used for prediction during the inference phase, while the sequence adaptation module and the teacher model do not participate in the inference process.
[0048] Secondly, the present invention provides a new energy power forecasting system, comprising:
[0049] The data acquisition module is used to acquire historical new energy power data with timestamps and preprocess it;
[0050] The data processing module is used to input the preprocessed new energy power data into the tested and validated student model to obtain the predicted future new energy power data.
[0051] Thirdly, the present invention provides an electronic device including a processor and a memory, the memory storing machine-executable instructions executable by the processor, the processor executing the machine-executable instructions to implement the method.
[0052] Fourthly, the present invention provides a machine-readable storage medium storing machine-executable instructions that, when invoked and executed by a processor, cause the processor to implement the method.
[0053] The beneficial effects of the present invention include at least the following:
[0054] 1) In power forecasting, power data exhibits both strong periodicity and random noise. By employing multiple pre-trained large teacher models and dynamically generating weights using a sequence adaptation module based on a multi-head attention mechanism, noise bias can be effectively suppressed and key features reconstructed. This allows the lightweight student model to learn more robust time-series representations. Its output is aligned with the teacher model's internal representation at the response level, significantly improving the accuracy of the student model in long-term, short-term, and ultra-short-term power load and output forecasting tasks.
[0055] 2) The sequence adaptive module employed in this invention introduces a multi-head self-attention (MHA) mechanism, enabling parallel computation of complex dependencies between time steps in the sequence. This characteristic allows it to effectively capture long-term dependency patterns such as periodicity and trends spanning hours or even days in power data. Therefore, even linear student models with few parameters show significantly improved performance in long-term prediction tasks after knowledge distillation, overcoming the memory decay defect commonly found in traditional simple models for long-sequence prediction and alleviating the memory forgetting problem of traditional models.
[0056] 3) Power training samples are often private datasets with small sample sizes. To address this, this invention uses knowledge distillation to transfer the latent features and patterns learned by large teacher models on general benchmark datasets to student models. Even if individual teacher models perform poorly on small sample data, the weighted fusion mechanism of the sequence adaptation module can balance the contributions of each model, enabling student models to obtain more stable and generalized representation capabilities. As a result, student models outperform their independent training or traditional distillation results in real power prediction tasks with limited data.
[0057] 4) High-efficiency inference: The inference stage relies only on the lightweight student model, with computational resource consumption comparable to the original student model, while maintaining prediction accuracy close to that of the teacher model. The inference speed is fast and suitable for real-time applications.
[0058] 5) Cross-architecture compatibility: Through the general design of the sequence adaptation module, it supports learning knowledge from multiple teacher models and is suitable for model combinations of different architectures. Attached Figure Description
[0059] Figure 1 This is a schematic diagram of the architecture of the adaptive distillation network SeqAdaptNet provided in an embodiment of the present invention.
[0060] Figure 2 This is a schematic diagram of the architecture of the sequence adaptive module of the present invention. Detailed Implementation
[0061] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.
[0062] The terms "comprising" and "having," and any variations thereof, used in the embodiments of this invention are intended to cover non-exclusive inclusion. For example, a process, method, system, product, or device that includes a series of steps or units is not limited to the steps or units listed, but may optionally include other steps or units not listed, or may optionally include other steps or units inherent to such processes, methods, products, or devices.
[0063] Based on this, this embodiment provides a cross-architecture-based new energy power forecasting method, which is a time series forecasting knowledge distillation method based on a multi-head self-attention mechanism. It learns time series forecasting knowledge from multiple large teacher models through a lightweight student model, achieving high accuracy and efficient deployment in long-term, short-term, and ultra-short-term forecasting tasks, while significantly reducing computational resource consumption and inference time. This addresses the problems of existing time series forecasting methods struggling to balance accuracy and efficiency in complex scenarios, and their insufficient generalization ability on small sample datasets. This invention introduces a sequence adaptation module to dynamically integrate knowledge from multiple teacher models, enhancing the student model's adaptability to different time scales and data sizes, making it particularly suitable for resource-constrained edge devices and real-time forecasting scenarios.
[0064] Specifically, the method includes:
[0065] Step S1: Obtain historical new energy power data with timestamps and manually annotate it.
[0066] The new energy power data includes at least one of the following: power output data of the power equipment, wind speed, load, and oil temperature.
[0067] Power equipment includes at least one of the following types: wind farms, photovoltaic power stations, and power transformers.
[0068] In practical implementation, the input time series data first uses the ETTh1, ETTh2 (hourly data) and ETTm1, ETTm2 (minute-level data) datasets, as well as data from photovoltaic and wind power output prediction tasks, including:
[0069] Load the dataset (such as ETTh2, ETTm2, or solar / wind power output data file) from the specified path. The data format includes a timestamp column and a feature column, where the target feature (such as OT or power value) is the object of prediction.
[0070] Step S2: Preprocess the power data, then divide the dataset into training set, test set, and validation set according to the proportion.
[0071] The preprocessing includes data normalization of the power data.
[0072] For example, data normalization uses methods such as StandardScaler, MinMaxScaler, or RobustScaler to eliminate dimensional differences and enhance the numerical stability of the model. The normalization parameters are fitted based on the training set and remain consistent during validation and testing.
[0073] In some embodiments, time features are generated based on the frequency of the time series (hourly or minute-level), including month, date, weekday, hour, minute (for minute-level data, minute values are grouped in 15-minute intervals), or frequency domain features are generated through Fourier transform to capture the periodicity and trend of the time series.
[0074] Step S3: Construct the adaptive distillation network SeqAdaptNet and train it using the training set; see Appendix for details. Figure 1 The adaptive distillation network SeqAdaptNet includes multiple teacher models, one student model, and a teacher output fusion module. Each teacher model's output is connected in series with a sequence adaptation module, which generates corresponding weights for the current teacher model. The teacher output fusion module is used to perform weighted fusion of the outputs of multiple teacher models with their corresponding weights.
[0075] Multiple teacher models independently select models with larger parameters, either with the same or different architectures, such as Transformer-Based or CNN-Based. The input of the teacher model includes normalized power data, and the output is predicted power data.
[0076] The student model is a lightweight model, such as a linear model (DLinear, Linear), a shallow CNN, etc. The input of the student model includes normalized power data, and the output is predicted power data.
[0077] The architecture of the model with larger parameters and the lightweight model can be the same or different.
[0078] For example, if the teacher model is the Autoformer in Transformer-Based, the input includes normalized power data and frequency domain features, and the output is predicted power data. The frequency domain features can be generated by Fourier transforming the normalized power data in the previous step S2 preprocessing.
[0079] If the student model is DLinear, then the input to the student model includes normalized power data and frequency domain features, and the output is predicted power data. The frequency domain features can be generated by Fourier transforming the normalized power data in the previous step S2 preprocessing.
[0080] If the student model is Linear or NLinear, then the input to the student model includes normalized power data and time features, and the output is predicted power data. Time features The data, including month, date, day of the week, and hour, can be generated from the normalized power data in the previous step S2 preprocessing.
[0081] 3-1 Teacher Model Loading: Load multiple pre-trained large teacher models, initialize model parameters through pre-trained checkpoints, set the model to evaluation mode, and freeze its parameters to ensure the stability of knowledge transfer.
[0082] 3-2 Student Model Initialization: Initialize the student model by loading parameters through random initialization or pre-trained weights to ensure that the number of model parameters and computational complexity are much lower than those of the teacher model, in order to adapt to resource-constrained deployment environments.
[0083] 3-3 Sequence Adaptation Module Initialization: Initialize the sequence adaptation module, see Appendix Figure 2 It includes an input projection layer, a multi-head self-attention layer, a global average pooling layer, and an output layer. The module parameters are initialized randomly, and the hidden dimension H (e.g., 512) and the number of attention heads (e.g., 8) are set to balance performance and computational efficiency.
[0084] The training phase aims to optimize the predictive performance of the student model through sequence adaptation modules and knowledge distillation. The specific steps are as follows:
[0085] 3-3-1 Preprocess the power data in the training set ( ) The features are obtained by mapping the input dimension D to the hidden dimension H through the input projection layer. ;
[0086] ;
[0087] in B is the batch size, and S is the sequence length (e.g., S = 96).
[0088] During training, based on the prediction task requirements, the input sequence length S, label length, and prediction length P are set (e.g., S = 96, P = 96), and the input sequence x∈ is constructed. and output sequence y∈ Where B is the batch size, D is the feature dimension, and L is the label length.
[0089] 3-3-2 will By using multiple attention heads to compute the dependencies between time steps in the sequence in parallel, context-sensitive features are generated. ;
[0090] = MHA( ) ;
[0091] Among them, the multi-head self-attention mechanism (MHA) calculates the dependencies between time steps in the sequence in parallel by multiple attention heads, which can capture local and global patterns in the time series and improve the robustness of feature representation.
[0092] 3-3-3 Using global average pooling The sequence dimension is used for mean pooling to obtain the pooled features. .
[0093] 3-3-4 Generate weight distribution through the output layer :
[0094] i = 1,2,...,N
[0095] in ∈ , ∈ N represents the number of teacher models. ∈ Let the weights of the i-th teacher model satisfy the following condition: Weight Used to dynamically combine the outputs of multiple teacher models.
[0096] The sequence adaptation module proposed in this invention is based on a multi-head self-attention mechanism, which is used to generate a soft mask and perform element-wise multiplication with the input features to reconstruct the features.
[0097] 3-4 Teacher Output Combination: Weights based on the output of the sequence adaptive module The outputs of multiple teacher models are weighted and fused:
[0098] = ;
[0099] in This is the output of the i-th teacher model; Output the weights for the sequence adaptive module corresponding to the i-th teacher model; N represents the total number of teacher models.
[0100] 3-5 Total Loss Calculation:
[0101]
[0102] ;
[0103] softmax( / T)) ;
[0104] in , All are weighting coefficients, and T is the temperature parameter; This represents the output of the student model. With real labels The mean squared error loss; express and KL divergence loss, Denotes the KL divergence function; log-softmax( ) is the logarithmic softmax function.
[0105] The log-softmax function is used to calculate log probabilities, given a j-dimensional logit vector z = [ , ,…, The i-th output of log-softmax is defined as:
[0106] = = - ;
[0107] softmax( ) is the normalization function, which transforms the input vector into a probability distribution, and is defined as:
[0108] = , = 1;
[0109] KLDiv(·,·) is the KL divergence function, which measures the difference between two probability distributions and is defined as:
[0110] KLDiv(p,q) = ( - );
[0111] 3-6 Parameter Optimization: The Adam optimizer is used to optimize the parameters of the student model and the sequence adaptation module, while the teacher model parameters are kept frozen; combined with the OneCycleLR learning rate scheduling strategy, the training efficiency and convergence stability are improved by dynamically adjusting the learning rate; in addition, automatic mixed precision (AMP) technology is used to accelerate training and reduce memory usage.
[0112] The student model of this invention achieves cross-architecture knowledge distillation by freezing all parameters except the sequence adaptation module, thereby aligning its output distribution with the teacher model's output distribution at the response level.
[0113] Step S4: Test the trained student model using the test set, and validate the tested student model using the validation set.
[0114] Step S5: Use the tested and validated student model to predict new energy power.
[0115] In some embodiments, step S5, the inference stage, involves using only the student model for prediction, while the sequence adaptation module and the teacher model do not participate in the inference process; the specific steps are as follows:
[0116] 1. Data input: Load test data or real-time data, and perform the same preprocessing process as the training phase (normalization and temporal feature encoding).
[0117] 2. Student Model Prediction: Input the preprocessed data into the student model to generate prediction output. ;
[0118] 3. Result output: Save the prediction results as a file or use them directly in real-time application scenarios, such as scheduling optimization of photovoltaic or wind power output prediction.
[0119] 4. This invention has been extensively validated on the ETTh1, ETTh2, ETTm1, and ETTm2 datasets, as well as on photovoltaic and wind power output prediction tasks:
[0120] 5. Dataset Description: ETTh1 and ETTh2 are hourly multivariate time series datasets containing 7 features and spanning 16 months; ETTm1 and ETTm2 are minute-level datasets with higher time resolution; the photovoltaic and wind power output datasets are private datasets containing power output and related environmental features (such as irradiance and wind speed).
[0121] 6. Performance Indicators: Mean Square Error (MSE) = Mean Absolute Error Root mean square error (RMSE) = Evaluate model performance.
[0122] 7. Results Analysis: On ETTh1 and ETTh2, the distilled student model reduced MSE by approximately 13% and MAE by approximately 11% in long-term prediction tasks (prediction length P=720) and short-term prediction tasks (P=96); on ETTm1 and ETTm2, minute-level prediction accuracy was significantly improved, with MSE reduced by approximately 4% and MAE reduced by approximately 6%; in photovoltaic and wind power output prediction tasks, the student model outperformed the original model in small sample scenarios, with MAE reduced by approximately 6%.
[0123] 8. Improved efficiency: The inference phase uses only the student model, reducing GPU memory usage by about 60% and inference time by about 40%, making it suitable for edge device deployment.
[0124] Table 1. Accuracy comparison of the SeqAdapt model of this invention with student and teacher models using the same architecture.
[0125]
[0126] Experimental results demonstrate that the proposed method performs well in various time series prediction tasks:
[0127] 1) On large benchmark datasets (such as new energy time series datasets), the distilled student model significantly outperforms the original student model in terms of mean squared error (MSE), mean absolute error (MAE), and root mean square error (RMSE), especially in long-term prediction tasks, where it alleviates the memory forgetting problem.
[0128] 2) On private datasets with small sample sizes, the student model learns the latent features of the teacher model and demonstrates superior performance to the original student model in short-term and ultra-short-term prediction tasks.
[0129] 3) In terms of computational efficiency, the resource consumption during the inference stage is only related to the student model. Compared with directly using a large teacher model, GPU memory usage is reduced by about 50%, and inference time is shortened by about 30%, making it suitable for edge devices and real-time prediction scenarios.
[0130] In summary, this invention, through the innovative design of a sequence adaptive module, combined with a multi-head self-attention mechanism and knowledge distillation technology, provides an efficient, universal, and high-performance method for power forecasting involving time series data. It not only approaches the accuracy of large teacher models but also has significant advantages in computational efficiency and deployment flexibility.
[0131] This embodiment also provides a new energy power forecasting system, including:
[0132] The data acquisition module is used to acquire historical new energy power data with timestamps and preprocess it;
[0133] The data processing module is used to input the preprocessed new energy power data into the tested and validated student model to obtain the predicted future new energy power data.
[0134] Those skilled in the art will understand that embodiments of the present invention can be provided as methods, systems, or computer program products. Therefore, the present invention can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, the present invention can take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.
[0135] This invention is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of the invention. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, generate instructions for implementing the flowchart illustrations and / or block diagrams. Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.
[0136] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and / or boxes Figure 1 The function specified in one or more boxes.
[0137] These computer program instructions may also be loaded onto a computer or other programmable data processing equipment to cause a series of operational steps to be performed on the computer or other programmable equipment to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable equipment for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.
[0138] The above-described embodiments are only used to illustrate the technical solutions of the present invention, and are not intended to limit it. Although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present invention.
Claims
1. A new energy power forecasting method based on cross-architecture architecture, characterized in that, The method includes: Acquire historical, timestamped new energy power data and manually annotate it; The power data is preprocessed and then divided into training, testing, and validation sets according to a certain ratio. An adaptive distillation network, SeqAdaptNet, is constructed and trained using a training set. The SeqAdaptNet includes multiple teacher models, one student model, and a teacher output fusion module. Each teacher model's output is connected in series with a sequence adaptation module, which generates corresponding weights for the current teacher model. The teacher output fusion module is used to perform weighted fusion of the outputs of multiple teacher models with their corresponding weights. The trained student model is tested using the test set, and the tested student model is validated using the validation set. Use tested and validated student models to predict renewable energy power generation.
2. The method according to claim 1, characterized in that, The new energy power data includes at least one of the following: wind speed, load, oil temperature, and electric field output of the power equipment.
3. The method according to claim 2, characterized in that, The power equipment includes at least one of the following types: wind farm, photovoltaic power station, and power transformer.
4. The method according to claim 1, characterized in that, The preprocessing includes: normalizing the power data.
5. The method according to claim 1, characterized in that, The construction and training process of the adaptive distillation network SeqAdaptNet includes: 1) Teacher model loading: Load multiple pre-trained teacher models, initialize the model parameters of the teacher models through pre-training checkpoints, set the models to evaluation mode, and freeze their parameters to ensure the stability of knowledge transfer. 2) Student model initialization: Initialize the student model, but the number of model parameters and computational complexity of the student model are lower than those of the teacher model; 3) Sequence adaptation module processing: The preprocessed power data x in the training set is mapped to the hidden dimension through the input projection layer to obtain the features. ; Will Multiple attention heads are used to compute the dependencies between time steps in the sequence in parallel, generating context-sensitive features. ; By global average pooling The sequence dimension is used for mean pooling to obtain the pooled features. ; Generate weight distribution through the output layer : , i = 1, 2, ..., N Equation (1) in for weights, For bias, N is the number of teacher models. Represents the weights of the i-th teacher model, satisfying ; 4) Teacher output combination: based on the weights output by the sequence adaptive module. The outputs of multiple teacher models are weighted and fused: = Equation (2) in This is the output of the i-th teacher model; Output the weights for the sequence adaptive module corresponding to the i-th teacher model; N represents the total number of teacher models. 5) Total loss calculation: 。 6. Equation (4) softmax( / T)) Equation (5) in These are the weighting coefficients, and T is the temperature parameter; This represents the output of the student model. With real labels The mean squared error loss; express and KL divergence loss, Denotes the KL divergence function; log-softmax( ) is the logarithmic softmax function; 6) Parameter optimization: The Adam optimizer is used to optimize the parameters of the student model and the sequence adaptation module, while the parameters of the teacher model are kept frozen.
7. The method according to claim 1, characterized in that, In the SeqAdaptNet adaptive distillation network, multiple teacher models independently choose the model with larger parameters, either with the same or different architectures. The input of the teacher model includes normalized power data, and the output is predicted power data. The student model chooses a lightweight model, and the input of the student model includes normalized power data, and the output is predicted power data. The architectures of the model with larger parameters and the lightweight model can be the same or different.
8. The method according to claim 1, characterized in that, During the inference phase, only the student model is used for prediction; the sequence adaptation module and the teacher model do not participate in the inference process.
9. A new energy power forecasting system implementing the method of any one of claims 1-7, characterized in that, include: The data acquisition module is used to acquire historical new energy power data with timestamps and preprocess it; The data processing module is used to input the preprocessed new energy power data into the tested and validated student model to obtain the predicted future new energy power data.
10. An electronic device, characterized in that, The method includes a processor and a memory, the memory storing machine-executable instructions that can be executed by the processor, the processor executing the machine-executable instructions to implement the method of any one of claims 1-7.
11. A machine-readable storage medium, characterized in that, The machine-readable storage medium stores machine-executable instructions that, when invoked and executed by a processor, cause the processor to implement the method of any one of claims 1-7.