Self-adaptive tensor decomposition attention mechanism for virtual power plant and prediction optimization method of self-adaptive tensor decomposition attention mechanism

Abstract

The invention belongs to the technical field of power system automation, and provides a self-adaptive tensor decomposition attention mechanism for a virtual power plant and a prediction optimization method thereof.The method comprises the steps that firstly, multi-modal data such as historical loads and weather of the virtual power plant are collected, and preprocessing such as normalization and feature extraction is conducted; then, an adaptive tensor decomposition attention network is built based on the processed data, an improved Xavier method is adopted to initialize a core tensor, features are fused after multi-modal data are coded, and a load prediction model is built through adaptive tensor decomposition attention calculation; and finally, adaptive core tensor updating is executed, through optimization strategies such as Hessian matrix low-rank approximation, adaptive sparse mask generation and Nesterov acceleration momentum calculation, batch processing and parallel calculation, adaptive resource allocation and core tensor updating are combined, data calculation is input, and a load prediction result is obtained. The objective of the invention is to solve the problems of high attention mechanism calculation complexity, parameter redundancy, low multi-modal data fusion efficiency, insufficient load sudden change period prediction precision and lack of an adaptive resource allocation mechanism in the existing virtual power plant load prediction technology.

Description

Technical Field

[0001] The present invention belongs to the technical field of power system automation and relates to an adaptive tensor decomposition attention mechanism for virtual power plants and a prediction optimization method thereof. Background Art

[0002] With the rapid development of the Energy Internet and smart grids, virtual power plants (VPPs) have become a key approach to addressing renewable energy consumption and improving the efficiency of distributed resource utilization. Leveraging information and communication technologies, they integrate distributed generation resources, controllable loads, and energy storage systems to form a dispatchable virtual resource pool, enabling participation in the electricity market and providing grid ancillary services. Load forecasting is crucial in the operational system of VPPs, as its accuracy directly impacts the effectiveness and economic efficiency of dispatch decisions. Deep learning technology has achieved significant results in the field of time series forecasting. The application of attention mechanisms has significantly enhanced models' ability to capture long-term dependencies. The self-attention mechanism of the Transformer model has been widely used in various time series forecasting tasks.

[0003] In the practical application of virtual power plant load forecasting, numerous issues hinder both forecast accuracy and the efficient operation of the system. First, virtual power plant load characteristics are influenced by a combination of factors, exhibiting high volatility, uncertainty, and complex spatiotemporal dependencies, making accurate forecasting extremely challenging. Second, traditional attention mechanisms exhibit numerous flaws when processing virtual power plant-related data. Their computational complexity scales quadratically with the sequence length (O(n²)), leading to significant performance bottlenecks when processing massive amounts of data in real time, resulting in slow system response. Fully parameterized attention matrices result in a large number of parameters, increasing the computational burden and easily leading to overfitting. This is particularly problematic when processing multimodal data, where parameter redundancy is a major concern. Traditional attention mechanisms employ simple fusion methods for multimodal data, making it difficult to exploit the complex interactions between modalities and fully exploit the value of heterogeneous multi-source data. Furthermore, traditional methods suffer from severe forecasting inaccuracies during periods of sudden load changes and lack adaptive resource allocation mechanisms, resulting in low computing resource utilization and insufficient support for edge computing and real-time response requirements.

[0004] Other studies have attempted to reduce complexity by improving the structure or computational methods of the attention mechanism. For example, Longformer uses a sliding window attention mechanism (proposed by Beltagy et al. in 2020) to reduce complexity to O(n), but the fixed window size limits the model's ability to capture long-distance dependencies. Katharopoulos et al. proposed linearizing the attention mechanism in 2020, which reduced complexity but reduced expressiveness. The Performer model proposed by Choromanski et al. in 2021 uses orthogonal random features to approximate the attention matrix, reducing computational complexity while maintaining expressiveness, but suffers from approximation errors. In terms of multimodal data fusion, the tensorfusion network proposed by Zadeh et al. in 2019 achieves multimodal fusion through outer product operations, but the computational complexity and number of parameters increase exponentially with the number of modalities.

[0005] Overall, existing methods have failed to fully address the challenges of virtual power plant load forecasting, including high computational complexity, parameter redundancy, inefficient multimodal data fusion, insufficient prediction accuracy during periods of sudden load changes, and a lack of adaptive resource allocation mechanisms. Therefore, a new approach is needed that can effectively integrate multi-source heterogeneous data. While reducing computational complexity and parameter redundancy, it can significantly improve load forecasting accuracy, especially during periods of sudden load changes, and enable adaptive resource allocation to meet the practical needs of intelligent virtual power plant operations. Summary of the Invention

[0006] The present invention provides an adaptive tensor decomposition attention mechanism for virtual power plants and a prediction optimization method thereof, which solves the problems existing in existing virtual power plant load forecasting technologies, such as high computational complexity of the attention mechanism, parameter redundancy, low efficiency of multimodal data fusion, insufficient prediction accuracy during load mutation periods, and lack of an adaptive resource allocation mechanism.

[0007] In order to solve the above problems, the technical solution adopted by the invention is: An adaptive tensor decomposition attention mechanism for virtual power plants and a prediction optimization method thereof are characterized by comprising: S01 Data Collection and Preprocessing: Collect multimodal data of virtual power plants, perform normalization, extract time-frequency domain features, clean outliers, align and resample time series data, and perform feature engineering and encoding processing; S02 model construction: Build an adaptive tensor decomposition attention network based on the processed data. The specific method is as follows: Based on the preprocessed data, an adaptive tensor decomposition attention network is built. The core tensor is initialized using the improved Xavier initialization method. The formula is:

[0008] in, Indicates the The size of the dimensions, Indicates the number of dimensions of the tensor; Perform feature encoding on multimodal data and convert it into a tensor representation suitable for model processing. The formula is , Indicates the The feature representation of each modality, Indicates the The original data of each modality, Indicates the The encoder of the modality, Represents encoder parameters; By using the multimodal feature fusion mechanism, the formula

[0009] The core tensor is then fused with each modal feature to form a multimodal fusion feature representation, where : The fused feature tensor, represents the core tensor, Indicates the The feature representation of each modality, Indicates the The weight matrix of each mode, Indicates that along The tensor product of the modes; Then perform adaptive tensor decomposition attention calculation and weighted feature aggregation, and calculate the attention score based on the fusion feature to evaluate the feature importance. The formula is

[0010] And perform weighted aggregation on the features according to the attention weight, the formula is ,in represents the weighted feature representation, represents the attention weight matrix, represents the fused feature tensor, represents the feature dimension, express The transpose of represents the normalized function, and the virtual power plant load forecasting model is obtained; S03 model optimization: perform adaptive core tensor updates, calculate the Hessian matrix through low-rank approximation, and then dynamically generate adaptive sparse masks based on gradient sensitivity analysis. At the same time, introduce Nesterov acceleration momentum and update the core tensor based on the above calculation results. Use sparse conjugate projection technology to ensure that the parameters meet physical constraints. Use a gradient-guided dynamic core tensor adjustment strategy to calculate the gradient importance distribution and adjust the core tensor. Combined with batch processing and parallel computing strategies, organize the core tensors of multiple instances into batch form, and use parallel computing technology to update multiple core tensors at the same time. Introduce an adaptive computing resource allocation strategy to dynamically allocate computing resources according to module importance. The updated core tensor and the trained model are input into the collected data for calculation to obtain the virtual power plant load forecast result.

[0011] The principles and advantages of this solution are: The solution revolves around virtual power plant load forecasting, and its principles cover four key links: data processing, model building, optimization, and forecasting. In the data processing stage, multimodal data is collected and normalized, and time-frequency domain feature extraction and other operations are performed. Normalization gives the data a unified dimension and eliminates scale differences between data. For example, meteorological data and load data in different ranges are converted into standardized data with a mean of 0 and a variance of 1 to facilitate model processing; time-frequency domain feature extraction mines data features from the time and frequency dimensions to capture the dynamic changes of the data. Cleaning outliers can remove erroneous or abnormal data points to ensure data quality; time series data alignment and resampling solves the problem of inconsistent sampling frequencies of different modal data and ensures data consistency in the time dimension; feature engineering and encoding provide richer information for the model by constructing multiple features and encoding category features.

[0012] An improved Xavier initialization method is used to initialize the core tensor, providing a stable initial value for model training. After encoding the multimodal data separately, a multimodal feature fusion mechanism is used to fuse the core tensor with the features of each modality to form a comprehensive feature representation. Adaptive tensor decomposition, attention calculation, and weighted feature aggregation are then performed to assess feature importance and enhance key features, thereby establishing a virtual power plant load forecasting model.

[0013] The algorithm reduces computational complexity through low-rank approximation of the Hessian matrix while capturing second-order interactions between parameters. Based on gradient sensitivity analysis, it generates an adaptive sparse mask, dynamically adjusts parameter importance, automatically identifies and retains key parameters, and prunes redundant parameters. Nesterov momentum acceleration is introduced to improve the convergence performance of the optimization process. The core tensor is updated based on these results, and sparse conjugate projection technology is used to ensure that the parameters meet physical constraints. A gradient-guided dynamic core tensor adjustment strategy is used to further optimize the core tensor based on gradient information. Furthermore, batch processing and parallel computing strategies are combined to improve computational efficiency, and an adaptive computing resource allocation strategy is introduced to rationally allocate resources based on module importance.

[0014] Finally, the optimized model is input into the collected and processed data to calculate the load forecast results of the virtual power plant.

[0015] Compared with existing technologies, this solution significantly reduces computational complexity through low-rank approximation of the Hessian matrix, batch processing, and parallel computing strategies. For example, when processing large-scale virtual power plant data, traditional methods have high computational complexity and low processing efficiency. However, this solution improves computational efficiency by 7.2 times, can simultaneously process multimodal sequences of over 68,000 time points, achieve nanosecond response, and rapidly process massive amounts of data to meet the real-time scheduling requirements of virtual power plants.

[0016] In terms of prediction accuracy, this solution's adaptive core tensor update strategy and multimodal feature fusion mechanism effectively improve prediction accuracy. For example, when the load suddenly changes, the adaptive core tensor update strategy can promptly adjust the attention weight, accurately predict the load change, and provide a reliable basis for virtual power plant scheduling decisions.

[0017] In terms of parameter efficiency, this solution's adaptive sparse mask generation mechanism and dynamic core tensor adjustment strategy reduce the number of parameters by 91.8%, requiring only 7.6K parameters to achieve optimal performance. In contrast, existing technologies use a large number of parameters, which can easily lead to overfitting and hinder deployment on edge devices. While reducing parameters, this solution achieves higher prediction accuracy and can operate efficiently on resource-constrained edge computing devices, supporting the distributed deployment of virtual power plants.

[0018] Existing technologies for multimodal data fusion have low fusion efficiency and difficulty in mining complex interactions between modalities. This solution's multimodal feature fusion mechanism fuses features from each modality through tensor products, effectively integrating heterogeneous data from multiple sources, fully leveraging the value of data and improving the model's ability to process complex data.

[0019] In addition, this solution introduces an adaptive computing resource allocation strategy to dynamically allocate computing resources according to the importance of modules, improve resource utilization and avoid resource waste.

[0020] Furthermore, the multimodal data in S01 includes historical load data, real-time load data, meteorological data, equipment operation status data, electricity price data and user behavior data.

[0021] Furthermore, the formula for the normalization process in S01 is:

[0022] in, Represents the original data, represents the data mean, represents the standard deviation of the data, represents a small constant that prevents division by zero; Furthermore, in S01, an adaptive Hampel filter is used to detect and correct abnormal values ​​in the original data, and the processing process can be expressed as follows:

[0023] in represents the data points after cleaning, represents the original data points, Indicates A sliding window centered on Display window The median of the data, Display window The median absolute deviation of the data, The threshold coefficient γ is adaptively adjusted according to the data distribution. The value range of the threshold coefficient γ is [3,5]. It is usually adaptively adjusted according to the data noise level. The default value is 3.5.

[0024] Furthermore, the frequency domain feature extraction in S01 is expressed as:

[0025] in, Represents time series data at time The value of Indicates the number of items, represents the time-frequency analysis parameters, represents the time-frequency analysis index parameter, Represents the center point parameter of time-frequency analysis, the time-frequency analysis parameter 、 、 It is automatically learned by stochastic gradient descent, an optimization algorithm that minimizes prediction loss, where m is the preset number of time-frequency components, usually 8≤m≤16.

[0026] Furthermore, the adaptive core tensor update strategy in S03 is:

[0027] in ， Indicates the The core tensor of the step, represents a low-rank approximation to the Hessian matrix, represents the gradient of the loss with respect to the core tensor, represents the adaptive sparse mask, represents the learning rate, represents the Nesterov acceleration momentum term, represents the Hadamard product, represents the constraint space projection, represents the soft threshold parameter, 、 、 Indicates control parameters.

[0028] Furthermore, the S03 adaptive sparse mask generation mechanism is dynamically generated through gradient sensitivity analysis:

[0029] in, Represents the time step Adaptive sparse mask of represents the Sigmoid function, Represents the gradient sensitivity parameter, with a value range of , : Parameter penalty coefficient, value range , : Baseline threshold, value range , represents the absolute value of the gradient of the loss with respect to the core tensor, represents the absolute value of the core tensor parameter, represents the Frobenius norm, Represents a small constant, set to .

[0030] Furthermore, the low-rank approximation calculation of the Hessian matrix in S03 is calculated by an efficient iterative algorithm:

[0031] in, represents the front of the Hessian matrix eigenvalues, represents the corresponding eigenvector, represents the low-rank approximation dimension, which is usually much smaller than the parameter dimension.

[0032] Furthermore, the gradient-guided dynamic core tensor adjustment strategy in S03 includes calculating the gradient importance distribution: and adjust the core tensor , in, represents the gradient importance distribution, Indicates the The gradient norm in the dimension, represents the temperature parameter, which controls the smoothness of the distribution, Indicates the adjustment strength parameter. BRIEF DESCRIPTION OF THE DRAWINGS

[0033] Figure 1 Flowchart of the system structure of the present invention. DETAILED DESCRIPTION

[0034] Example 1, as Figure 1 As shown, an adaptive tensor decomposition attention mechanism and prediction optimization method for virtual power plants include: S01 Data Collection and Preprocessing: Collect multimodal data of virtual power plants, perform normalization, extract time-frequency domain features, clean outliers, align and resample time series data, and perform feature engineering and encoding processing; S02 model construction: Build an adaptive tensor decomposition attention network based on the processed data. The specific method is as follows: Based on the preprocessed data, an adaptive tensor decomposition attention network is built. The core tensor is initialized using the improved Xavier initialization method. The formula is:

[0035] in, Indicates the The size of the dimensions, Indicates the number of dimensions of the tensor; Perform feature encoding on multimodal data and convert it into a tensor representation suitable for model processing. The formula is , Indicates the The feature representation of each modality, Indicates the The original data of each modality, Indicates the The encoder of the modality, Represents encoder parameters; By using the multimodal feature fusion mechanism, the formula

[0036] The core tensor is then fused with each modal feature to form a multimodal fusion feature representation, where : The fused feature tensor, represents the core tensor, Indicates the The feature representation of each modality, Indicates the The weight matrix of each mode, Indicates that along The tensor product of the modes; Then perform adaptive tensor decomposition attention calculation and weighted feature aggregation, and calculate the attention score based on the fusion feature to evaluate the feature importance. The formula is

[0037] And perform weighted aggregation on the features according to the attention weight, the formula is ,in represents the weighted feature representation, represents the attention weight matrix, represents the fused feature tensor, represents the feature dimension, express The transpose of Represents the normalized function, and the virtual power plant load forecasting model is obtained.

[0038] S03 model optimization: perform adaptive core tensor updates, calculate the Hessian matrix through low-rank approximation, and then dynamically generate adaptive sparse masks based on gradient sensitivity analysis. At the same time, introduce Nesterov acceleration momentum and update the core tensor based on the above calculation results. Use sparse conjugate projection technology to ensure that the parameters meet physical constraints. Use a gradient-guided dynamic core tensor adjustment strategy to calculate the gradient importance distribution and adjust the core tensor. Combined with batch processing and parallel computing strategies, organize the core tensors of multiple instances into batch form, and use parallel computing technology to update multiple core tensors at the same time. Introduce an adaptive computing resource allocation strategy to dynamically allocate computing resources according to module importance. The updated core tensor and the trained model are input into the collected data for calculation to obtain the virtual power plant load forecast result.

[0039] This solution focuses on load forecasting for virtual power plants. During the data processing phase, multimodal data is collected and normalized, and time-frequency domain feature extraction is performed. Normalization gives the data a unified dimension and eliminates scale differences between data. For example, meteorological data and load data from different ranges are converted into standardized data with a mean of 0 and a variance of 1 to facilitate model processing. Time-frequency domain feature extraction mines data features from the time and frequency dimensions to capture the dynamic changes in the data. Outlier cleaning removes erroneous or abnormal data points to ensure data quality. Time series data alignment and resampling solves the problem of inconsistent sampling frequencies of different modal data and ensures data consistency in the time dimension. Feature engineering and encoding provide richer information for the model by constructing multiple features and encoding categorical features.

[0040] An improved Xavier initialization method is used to initialize the core tensor, providing a stable initial value for model training. After encoding the multimodal data separately, a multimodal feature fusion mechanism is used to fuse the core tensor with the features of each modality to form a comprehensive feature representation. Adaptive tensor decomposition, attention calculation, and weighted feature aggregation are then performed to assess feature importance and enhance key features, thereby establishing a virtual power plant load forecasting model.

[0041] The algorithm reduces computational complexity through low-rank approximation of the Hessian matrix while capturing second-order interactions between parameters. Based on gradient sensitivity analysis, it generates an adaptive sparse mask, dynamically adjusts parameter importance, automatically identifies and retains key parameters, and prunes redundant parameters. Nesterov momentum acceleration is introduced to improve the convergence performance of the optimization process. The core tensor is updated based on these results, and sparse conjugate projection technology is used to ensure that the parameters meet physical constraints. A gradient-guided dynamic core tensor adjustment strategy is used to further optimize the core tensor based on gradient information. Furthermore, batch processing and parallel computing strategies are combined to improve computational efficiency, and an adaptive computing resource allocation strategy is introduced to rationally allocate resources based on module importance.

[0042] Finally, the optimized model is input into the collected and processed data to calculate the load forecast results of the virtual power plant.

[0043] In terms of computational efficiency, this solution significantly reduces computational complexity through low-rank approximation of the Hessian matrix, batch processing, and parallel computing strategies. For example, when processing large-scale virtual power plant data, traditional methods have high computational complexity and low processing efficiency. However, this solution improves computational efficiency by 7.2 times, can simultaneously process multimodal sequences of over 68,000 time points, achieve nanosecond response, and rapidly process massive amounts of data to meet the real-time scheduling requirements of virtual power plants.

[0044] In terms of prediction accuracy, this solution's adaptive core tensor update strategy and multimodal feature fusion mechanism effectively improve prediction accuracy. For example, when the load suddenly changes, the adaptive core tensor update strategy can promptly adjust the attention weight, accurately predict the load change, and provide a reliable basis for virtual power plant scheduling decisions.

[0045] In terms of parameter efficiency, this solution's adaptive sparse mask generation mechanism and dynamic core tensor adjustment strategy reduce the number of parameters by 91.8%, requiring only 7.6K parameters to achieve optimal performance. In contrast, existing technologies use a large number of parameters, which can easily lead to overfitting and hinder deployment on edge devices. While reducing parameters, this solution achieves higher prediction accuracy and can operate efficiently on resource-constrained edge computing devices, supporting the distributed deployment of virtual power plants.

[0046] Existing technologies for multimodal data fusion have low fusion efficiency and difficulty in mining complex interactions between modalities. This solution's multimodal feature fusion mechanism fuses features from each modality through tensor products, effectively integrating heterogeneous data from multiple sources, fully leveraging the value of data and improving the model's ability to process complex data.

[0047] In addition, this solution introduces an adaptive computing resource allocation strategy to dynamically allocate computing resources according to the importance of modules, improve resource utilization and avoid resource waste.

[0048] The multimodal data in S01 includes historical load data, real-time load data, meteorological data, equipment operating status data, electricity price data, and user behavior data. Historical load data records electricity usage over different time periods and can reflect load trends and cyclical patterns. Combined with real-time load data, the model can track current electricity usage in real time and promptly capture real-time load fluctuations. Meteorological data such as temperature, humidity, wind speed, and sunlight are closely related to power load. For example, in hot weather, increased air conditioning use significantly increases load. When sunlight is abundant, increased photovoltaic power generation also affects overall load. Equipment operating status data reflects the operating conditions of power generation equipment and energy storage devices, directly affecting power supply and consumption. Electricity price data influences user electricity usage behavior. Users' electricity usage habits vary during peak and off-peak periods, thus affecting load. User behavior data reflects the electricity usage preferences and patterns of different user groups. By integrating this multimodal data, the model can learn the factors that influence load from multiple dimensions, more comprehensively capturing the underlying patterns of load fluctuations, and significantly improving the accuracy of virtual power plant load forecasting.

[0049] The formula for the normalization process in S01 is:

[0050] in, Represents the original data, represents the data mean, represents the standard deviation of the data, Represents a small constant to prevent division by zero. Gradient descent is a commonly used optimization method in neural network training. When the input data distribution varies greatly, the contour lines of the objective function may be oblong ellipsoids, resulting in a tortuous gradient descent path and slow convergence. After normalization, the data distribution is closer to a standard normal distribution, the contour lines of the objective function are closer to circles, and the gradient descent direction is more direct, significantly accelerating model convergence. For example, in the Adam optimizer, normalization allows for more stable adjustment of the learning rate, reduces oscillations, and enables the model to reach the optimal solution in fewer iterations. In S01, an adaptive Hampel filter is used to detect and correct abnormal values ​​in the original data. The processing process can be expressed as follows:

[0051] in represents the data points after cleaning, represents the original data points, Indicates A sliding window centered on Display window The median of the data, Display window The median absolute deviation of the data, γ represents the threshold coefficient, which is adaptively adjusted based on the data distribution. The threshold coefficient γ ranges from [3, 5] and is typically adaptively adjusted based on the data noise level. The default value is 3.5. Virtual power plant raw data is complex, diverse, and subject to fluctuations, making traditional outlier detection methods difficult to adapt to these changes. The adaptive Hampel filter effectively captures local data characteristics by calculating the median and median absolute deviation through a sliding window centered on the raw data point. The threshold coefficient is adaptively adjusted based on the data distribution, allowing the filter to flexibly adapt to varying data conditions. When data fluctuates significantly, the threshold is adaptively increased to avoid misclassifying normal fluctuations as anomalies. When data is stable, the threshold is reduced to accurately identify minor anomalies. This method not only accurately detects and corrects outliers, but also preserves the true characteristics of the data to the greatest extent possible, minimizing the interference of outliers on subsequent data analysis and model training. This improves data quality, lays a solid foundation for building an accurate virtual power plant load forecasting model, and enhances the reliability and stability of the entire forecasting system.

[0052] The frequency domain feature extraction in S01 is expressed as:

[0053] in, Represents time series data at time The value of Indicates the number of items, represents the time-frequency analysis parameters, represents the time-frequency analysis index parameter, Represents the center point parameter of time-frequency analysis, the time-frequency analysis parameter 、 、 This feature extraction method is automatically learned using stochastic gradient descent, an optimization algorithm designed to minimize prediction loss. m is the preset number of time-frequency components, typically 8 ≤ m ≤ 16. Multimodal data from virtual power plants contains rich information in both the time and frequency dimensions, and this time-frequency domain feature extraction formula can exploit this information from multiple perspectives. By performing a weighted summation of the values ​​of time series data at different moments, involving the synergistic effects of time-frequency analysis parameters, exponential parameters, and center point parameters, it accurately captures the fluctuation characteristics of the data at different frequencies. For example, for load data, it can detect periodic variations at different time scales, such as daily and weekly cycles, and can also identify short-term high-frequency fluctuations, which are often associated with transient equipment startups and shutdowns or sudden power demand. The features extracted in this way greatly enrich the data representation, enabling subsequent models to learn more comprehensive and detailed data features, enhancing their understanding and grasp of complex data patterns, and significantly improving the accuracy and adaptability of virtual power plant load forecasting models to complex operating conditions.

[0054] The adaptive core tensor update strategy in S03 is:

[0055] in, Indicates the The core tensor of the step, represents a low-rank approximation to the Hessian matrix, represents the gradient of the loss with respect to the core tensor, represents the adaptive sparse mask, represents the learning rate, represents the Nesterov acceleration momentum term, represents the Hadamard product, represents the constraint space projection, represents the soft threshold parameter, 、 、 To represent control parameters, the virtual power plant load forecasting model needs to accurately and efficiently update core tensors to adapt to complex and changing data and improve forecasting performance. In this strategy, the low-rank approximation of the Hessian matrix effectively reduces computational complexity, allowing the model to run efficiently even when processing large-scale data. It also captures second-order interactions between parameters. Compared with methods that rely solely on first-order information, it can more comprehensively characterize parameter relationships and optimize update directions. Adaptive sparse masks dynamically adjust parameter importance based on gradients and parameter sizes, automatically identify and retain key parameters, and trim redundant parameters, reducing computational burden while improving the model's generalization ability and avoiding overfitting. The Nesterov accelerated momentum term accelerates the convergence of the optimization process, enabling the model to find the optimal solution more quickly and saving training time. The constraint space projection operation ensures that the parameters meet physical constraints, ensuring the rationality and interpretability of the model. The setting of multiple control parameters makes the strategy more flexible and can be fine-tuned according to the characteristics of different virtual power plant data and application scenarios. Combining these advantages, the adaptive core tensor update strategy significantly improves the model's expressiveness and training efficiency, thereby improving the accuracy of virtual power plant load forecasting and providing strong guarantees for the optimized scheduling and efficient operation of virtual power plants.

[0056] The S03 adaptive sparse mask generation mechanism is dynamically generated through gradient sensitivity analysis:

[0057] in, Represents the time step Adaptive sparse mask of represents the Sigmoid function, Represents the gradient sensitivity parameter, with a value range of , : Parameter penalty coefficient, value range , Indicates the baseline threshold, the value range is , represents the absolute value of the gradient of the loss with respect to the core tensor, represents the absolute value of the core tensor parameter, express Frobenius norm, Represents a small constant, set to Virtual power plant data is complex and highly variable, requiring precise adjustment of model parameters to improve predictive performance. This mechanism dynamically generates a sparse mask through gradient sensitivity analysis and uses a sigmoid function to map the result to the (0, 1) interval, ensuring that the mask values ​​are reasonable and can be used to control parameter updates. The gradient sensitivity parameter, parameter penalty coefficient, and baseline threshold are flexibly adjustable within the ranges of [1.0, 2.0], [0.5, 1.0], and [-0.5, 0.5], respectively. This allows for precise balancing of the impact of gradient sensitivity and parameter size on mask generation based on data characteristics and model training status. By combining the absolute value of the loss with respect to the core tensor gradient, the absolute value of the core tensor parameter, and the Frobenius norm, this mechanism intelligently identifies important and redundant parameters. During training, the mask allows updates to critical parameters, strengthening their role. Updates to redundant or unimportant parameters are reduced or even stopped, reducing computational complexity and preventing overfitting. Small constants are set to ensure computational stability and avoid division by zero errors. Overall, this mechanism enables the model to maintain prediction accuracy while optimizing the parameter update process, improving training efficiency and enhancing the model's adaptability to complex data of virtual power plants.

[0058] The low-rank approximation calculation of the Hessian matrix in S03 is calculated by an efficient iterative algorithm:

[0059] in, represents the front of the Hessian matrix eigenvalues, represents the corresponding eigenvector, Represents the low-rank approximation dimension, which is usually much smaller than the parameter dimension. The model training of the virtual power plant involves a large number of parameters, and the traditional calculation method is highly complex. This low-rank approximation calculation is done through an efficient iterative algorithm. k The Hessian matrix is ​​approximated by eigenvalues ​​and corresponding eigenvectors, where k The dimension is much smaller than the parameter dimension. This significantly reduces computational complexity, enabling the model to quickly calculate complex data and improve training efficiency. At the same time, it preserves critical second-order information, allowing the model to more accurately capture the interactions between parameters during optimization, optimize the update direction of the core tensor, and thus improve the model's prediction accuracy and enhance its adaptability to complex changes in virtual power plant loads.

[0060] The gradient-guided dynamic core tensor adjustment strategy in S03 includes calculating the gradient importance distribution:

[0061] and adjust the core tensor , in, represents the gradient importance distribution, Indicates the The gradient norm in the dimension, represents the temperature parameter, which controls the smoothness of the distribution, Indicates the adjustment intensity parameter. In the complex data environment of the virtual power plant, the load forecasting model needs to keenly capture the key information in the data to improve the forecast accuracy. This strategy calculates the gradient importance distribution according to the i The gradient norm and temperature parameter of each dimension determine the importance of each dimension. The temperature parameter can adjust the smoothness of the distribution to prevent the importance distribution from being too concentrated or dispersed, allowing the model to flexibly adjust its focus based on the characteristics of the data. Based on this distribution, the core tensor is adjusted, and the adjustment intensity parameter is used to control the adjustment amplitude. This allows the model to dynamically focus on feature dimensions that have a greater impact on the prediction results during training, strengthen the role of these dimensions, and weaken the influence of unimportant dimensions. In this way, the model can not only learn data features more efficiently, but also enhance its adaptability to complex situations such as sudden load changes, significantly improving the performance and prediction accuracy of the virtual power plant load forecasting model.

[0062] In actual use, we divide it into three stages: data collection stage, preprocessing stage, model building stage, model optimization stage, and model training and operation stage. Data collection and preprocessing stage Utilize a multi-source heterogeneous data acquisition network to collect historical load data, real-time load data, meteorological data related to virtual power plant operation, including temperature, humidity, wind speed, light intensity, equipment operation status data, electricity price data, and user behavior data. For example, load data is obtained from smart meters, meteorological data is collected through meteorological monitoring stations, and electricity price data is obtained from power trading platforms. Adaptive Hampel filter is used to process abnormal values ​​in the original data. Taking the load data at a certain moment as an example, assuming that the current data point is , set the sliding window with it as the center , calculate the median of the data in the window and median absolute deviation , adaptively adjust the threshold coefficient according to data distribution ,like , then the cleaned data points ,otherwise .

[0063] The data of different modes are standardized so that their mean is 0 and their variance is 1. Taking the historical load data X as an example, according to the formula Normalize, where μ(X) is the mean of historical load data, σ(X) is the standard deviation, Set to Prevent division by zero errors. After normalization, data of different dimensions are scaled uniformly, making it easier to process subsequent models.

[0064] To solve the problem of inconsistent sampling frequencies of different modal data, an adaptive resampling method based on B-spline interpolation is adopted. For example, the sampling frequency of load data is once every 15 minutes, and the sampling frequency of meteorological data is once every 1 hour.

[0065] Resample the data to ensure that all modal data are aligned in the time dimension, is the data value of the control point, for d Order B-spline basis function, n is the number of control points.

[0066] Construct various features based on the characteristics of the virtual power plant. For example, from the time dimension, construct time features such as hours, days, weeks, months, seasons, and holidays; extract meteorological features such as temperature, humidity, wind speed, and light from meteorological data; generate load features such as historical load and load change rate from load data; and obtain equipment features such as operating status and efficiency coefficient from equipment operating status data. For category features, a combination of optimized one-hot encoding and embedded encoding is used, such as feature i Embedding vector of , E is the embedding matrix, Features The one-hot encoding vector provides richer information for the model.

[0067] Model building phase: Based on the preprocessed multimodal data, the core tensor is initialized using the improved Xavier method. Initialize. Assume that the core tensor dimension is n , No. The size of the dimension is , according to the formula

[0068] Initialization is performed to provide stable initial values ​​for model training.

[0069] Each modal data is feature coded separately. Taking meteorological data modality as an example, its original data is , through the corresponding encoder and parameters , converting it into a tensor representation suitable for model processing

[0070] The features of each modality are fused with the core tensor through tensor product. For example, suppose there are three modalities: meteorological data, load data, and equipment operation status data. Their feature representations are 、 、 , the core tensor is G, and the corresponding weight matrix is 、 、 , through the formula , forming a multimodal fusion feature representation .

[0071] Based on the fusion feature F, the adaptive tensor decomposition attention score is calculated according to the formula

[0072] Evaluate feature importance, is the attention weight matrix, is the feature dimension. Then the features are weighted and aggregated according to the attention weight, and the formula is , is the weighted feature representation, and the virtual power plant load forecasting model is obtained.

[0073] Model optimization phase The improved Lanczos algorithm is used to calculate the low-rank approximation of the Hessian matrix. Assume that at a certain training step t, the first k eigenvalues ​​of the Hessian matrix are calculated. and the corresponding eigenvector , through the formula

[0074] Perform low-rank approximation, and k is usually set to 5%-10% of the total parameter dimension. This reduces computational complexity while retaining key second-order information and optimizing the core tensor update direction. Based on the gradient information and parameter size, the sparse mask is generated dynamically. During training, the absolute value of the gradient of the core tensor is calculated. and the absolute values ​​of the core tensor parameters , combined with the Frobenius norm , through the formula

[0075] Generate adaptive sparse masks .in, is the Sigmoid function, α The value range is [1.0, 2.0], the value range of β is [0.5, 1.0], c The value range is [−0.5, 0.5], Set to 10−6. This mask can intelligently identify important parameters and redundant parameters and control parameter updates.

[0076] Nesterov accelerated momentum calculation: Nesterov accelerated momentum is introduced to improve the convergence performance of the optimization process. At time step t, according to the formula

[0077] Calculating the momentum term , Typically set to 0.9.

[0078] Combined with the above calculation results, the core tensor is accurately updated. At time step t, according to the formula

[0079] Update the core tensor, where or is the learning rate, the initial value is set to 0.001, and the cosine annealing strategy is adopted; I is the identity matrix; r Usually set to 0.01; is the gradient of the loss with respect to the core tensor; ⊙ is the Hadamard product; d Usually set to 0.1; x Usually set to 0.001; Usually set to 0.0001; sign(⋅) is the sign function; To constrain the spatial projection operation and ensure that the parameters meet the physical constraints, Represents the operation of projecting parameters into the physical constraint space Ω, ensuring that the parameters meet the physical constraints of device operation, such as non-negativity and power limits.

[0080] Gradient-guided dynamic core tensor adjustment, calculate the gradient importance distribution, according to the formula

[0081] For the The gradient norm in the dimension, t is the temperature parameter that controls the smoothness of the distribution and is usually set to 0.1-0.5. Based on this distribution, according to the formula

[0082] Adjust the core tensor, λ is the adjustment strength parameter, usually set to 0.05-0.2, so that the model can dynamically focus on important feature dimensions during training.

[0083] Organize the core tensors of multiple instances into batches, such as

[0084] is the batch size, usually set to 32 or 64. Using parallel computing technology to update multiple core tensors at the same time, the formula is Improve computing efficiency.

[0085] Based on the gradient and computational complexity, the importance of each module is evaluated. Calculate the importance score of the i-th module

[0086] For the The L2 norm of the gradient of the module parameters, , for the The floating point operations of each module are calculated, and n is the total number of modules. Computational resources are allocated according to the importance of the modules. The formula is:

[0087] m is the resource allocation efficiency factor, usually set to 0.95-0.99, oh To assign a sensitivity parameter, usually set to 1.0-3.0, is the total available computing resources.

[0088] Model training phase: A multi-objective loss function that comprehensively considers prediction accuracy and model complexity is used is the mean absolute error, is the root mean square error, is the smoothing loss, For the core tensor L1 norm, The values ​​were set to 0.5, 0.2, and 0.1, respectively. The model was trained using the modified Adam optimizer with a cosine annealing learning rate scheduling strategy. During training, the model's performance on the validation set was recorded. Training was terminated after 10 consecutive rounds of validation without improvement.

[0089] After training is completed, the prediction results are output and the model performance is evaluated by calculating indicators such as mean absolute percentage error (MAPE) and root mean square error (RMSE). Special attention is paid to the prediction accuracy during the load mutation period, and the mean absolute percentage error during the load mutation period is calculated.

[0090] , C is the sample set during the load mutation period, is the number of samples during the load mutation period.

[0091] Deploy the trained model to an actual virtual power plant environment. Perform lightweight processing on the model by using a pruning function to remove elements with absolute values ​​less than a threshold. Optimize the model's computational graph and merge adjacent computational operations. Implement distributed deployment of the model based on the system architecture. Develop a real-time prediction interface to support interaction with other systems, enabling real-time scheduling and optimized control of the virtual power plant.

[0092] Example 2 1. Experimental environment: The experimental environment for this example uses a Huawei Atlas 900 AI cluster, which is configured with eight Tesla V100 GPUs, each with 32GB of video memory, for a total system memory of 384GB of RAM. The processor is an Intel Xeon Platinum 8260 CPU. The experimental data is based on actual operational data from a virtual power plant of a provincial power grid company from 2020 to 2024, encompassing 15 distributed energy facilities (including photovoltaic, wind power, and small energy storage) and 38 controllable loads (including industrial, commercial, and residential loads). Meteorological data for the corresponding time period was also collected, including 25 meteorological characteristics such as temperature, humidity, wind speed, and light intensity, as well as device operating status data covering 382 device status parameters. The experiment was implemented using the PyTorch 2.0.1 framework, running in the CUDA 11.7 environment, using Python 3.9 as the programming language, and Pandas 2.0.0 and NumPy 1.23.5 for data processing.

[0093] Comparison plan 1 description: Comparison solution 1 uses the standard Transformer attention mechanism (Standard-Attention) to implement the classic scaled dot product attention structure. In this solution, the attention calculation expression is

[0094] , its computational complexity is O(n²) ,in n The model uses a 6-layer Transformer encoder, each with 8 attention heads. The hidden layer dimension is 512, and the feedforward network dimension is 2048. The number of parameters is approximately 92.5K, and the model size is 3.72MB. Training uses the Adam optimizer with a learning rate of 1e-4, weight decay of 1e-5, and a batch size of 128. Comparison solution 1 uses a cosine learning rate scheduler with 5 warmup epochs and a maximum of 100 epochs. The model structure of this solution fully adheres to the original Transformer architecture, without any computational complexity optimization.

[0095] Comparison plan 2 description: Comparative solution 2 uses the linear attention mechanism (Linear-Attention), which reduces the attention calculation complexity from O(n²) to O(n) through the kernel technique. This solution uses random projection to approximate the attention matrix, which is expressed as ,in It is a random feature mapping function that maps raw vectors to a high-dimensional feature space. The model uses an 8-layer structure with 4 attention heads per layer, a hidden layer dimension of 384, and a feedforward network dimension of 1536. It has approximately 38.4K parameters and a model size of 1.58MB. Training uses the AdamW optimizer with a learning rate of 5e-4, a weight decay of 1e-2, and a batch size of 256. A linear learning rate warmup with a multi-step decay strategy is used, with 10 warmup epochs. The learning rate is reduced to 0.1x the original value at the 40th and 80th epochs. This solution reduces computational complexity through approximate computing and matrix factorization techniques, but at the expense of certain prediction accuracy.

[0096] 2. Experimental steps: First, data preprocessing was performed, applying Z-score normalization to the raw data to eliminate differences in feature dimensions. A sliding window technique (with a window size of 168 hours and a sliding step of 1 hour) was used to generate training samples. The input window length was 168 hours (one week), and the load curve for the next 24 hours was predicted. The dataset was divided chronologically into a training set (January 2020 to July 2023), a validation set (August 2023 to November 2023), and a test set (December 2023 to March 2024) in an 8:1:1 ratio. This ensured that the test data was later than the training data, consistent with practical application scenarios.

[0097] Then the model is initialized. The three schemes initialize parameters according to their algorithm characteristics. For the scheme of the present invention, the core tensor adopts the improved Xavier initialization method, and the initial range is

[0098] ,in Indicates the The core tensor dimensions are set to [32, 48, 64]. This configuration was optimized through grid search. The control parameters for the adaptive sparse mask generation mechanism were set to α = 0.75, β = 0.25, and γ = 0.1, as determined by performance tuning on the validation set.

[0099] The model is then trained. All schemes are trained for a maximum of 100 rounds. The MAPE indicator on the validation set is used for early stopping control, and training is stopped when there is no improvement in performance after 10 consecutive rounds of verification. The training of the scheme of the present invention adopts an improved second-order optimization strategy, and the dimension k of the low-rank approximation of the Hessian matrix is ​​set to 16. This configuration balances the computational complexity and the number of parameters. During the training process, the performance indicators of the model on the validation set are calculated and recorded after each round of iteration. The initial value of the learning rate of the scheme of the present invention is set to 3e-4, and a cosine annealing scheduling strategy is adopted, with a minimum learning rate of 1% of the initial value. The training uses a mixed precision strategy, with FP16 for forward propagation and backpropagation, and FP32 for updating model parameters, to balance computational efficiency and numerical stability.

[0100] Finally, performance testing was conducted, using a test set to evaluate prediction accuracy while also measuring metrics such as computational efficiency, parameter efficiency, and memory usage. During testing, a batch size of 1024 was used to simulate the high throughput requirements of a real-world production environment. A specific assessment was conducted during periods of sudden load fluctuations (defined as periods where the intraday peak-to-valley rate of change exceeds 30%) to verify the model's adaptability to rapid load changes. Hardware adaptability testing deployed the trained model on a server GPU and a Jetson Xavier NX edge computing device (8GB of RAM, a 6-core ARM processor, and 48 CUDA cores) to test its real-time responsiveness under varying computing power. For edge computing scenarios, TensorRT optimization and INT8 quantization were further employed to reduce inference latency.

[0101] 3. Test methods and standards: The prediction accuracy is comprehensively evaluated using multiple indicators: the mean absolute percentage error (MAPE) is used to evaluate the relative error between the predicted value and the actual value, and its calculation formula is:

[0102] ,in is the actual value, is the predicted value; the root mean square error (RMSE) is used to evaluate the absolute error of the predicted value, and the calculation formula is

[0103] The coefficient of determination (R²) is used to evaluate the ability of the model to explain variation and is calculated as

[0104] in AThe average of the actual values ​​is used in the computational efficiency test. The number of samples processed per second (samples / s) reflects the model's throughput. The test used the maximum batch size and averaged 1000 runs. The average inference time per sample (ms) reflects the model's latency. The test used a batch size of 1 and averaged 5000 runs. The P95 and P99 latencies were also recorded to ensure the reliability of the results. Each test was repeated 10 times, and the average was calculated with the standard deviation.

[0105] Parameter efficiency analysis evaluates the number of model parameters and model size (MB). The parameter efficiency ratio (PER) is used to determine the prediction accuracy for a given number of parameters. The calculation formula is PER = 1 / (MAPE × PER). The load-sudden-change adaptability test evaluates prediction errors during periods of significant load fluctuations (rates exceeding 30% / hour). The PER includes the MAPE and RMSE during the fluctuation period, as well as the accuracy of early warnings for sudden events. The accuracy of early warnings is defined as the proportion of correctly predicted sudden-change times (with an error of no more than 30 minutes).

[0106] The resource usage test records GPU memory peak usage, average CPU utilization, and power consumption. Power consumption measurements are monitored in real time using the NVIDIA-SMI tool and averaged. Edge deployment performance is tested on the Jetson Xavier NX platform, measuring inference speed, latency stability, and energy efficiency (samples processed per watt) to evaluate the model's performance in resource-constrained environments. Furthermore, the model's robustness is tested by adding Gaussian noise (σ = 0.1) and random missing values ​​(missing rate 5%) to the input data to assess the model's robustness to data quality issues.

[0107] 4. Comparison of experimental results:

[0108] From the perspective of prediction accuracy, the overall MAPE of this solution is only 2.60%, a 46.5% decrease compared to 4.86% for Comparison Scheme 1 and a 16.7% decrease compared to 3.12% for Comparison Scheme 2. This significant improvement in accuracy stems from the efficient capture of the intrinsic structure of the data by the Multimodal Fusion Adaptive Tensor Decomposition Attention Mechanism (MF-ATDAM). Especially during periods of sudden load changes, the MAPE of this solution is only 0.93%, a 93.2% decrease compared to 13.75% for Comparison Scheme 1. This outstanding performance is due to the adaptive core tensor update strategy that accurately captures the sudden changes in load and dynamically adjusts the attention weights. In terms of RMSE, the proposed solution achieves 85.2kW, a 65.6% decrease compared to Scheme 1 and a 46.2% decrease compared to Scheme 2, indicating a significant reduction in the absolute error of the prediction. The coefficient of determination (R²) reaches 0.9754, higher than that of the comparison schemes, indicating that the model has a stronger ability to explain data variation.

[0109] In terms of computational efficiency, this solution achieves an inference speed of 68,324 samples per second, 7.2 times that of Comparative Solution 1 and 2.9 times that of Comparative Solution 2. Meanwhile, the average inference time per sample is only 1.20ms, which is only 13.9% of that of Comparative Solution 1 and 35.2% of that of Comparative Solution 2. This significant efficiency improvement is attributed to the reduced computational complexity brought about by tensor decomposition, which is 92.7% lower than traditional methods. The improvement in computational throughput is even more significant, reaching 622.8% of that of Comparative Solution 1, enabling the model to handle larger-scale data streams. In a time series length expansion test, the solution of the present invention was able to handle sequence lengths exceeding 68,000 points, while Comparative Solution 1 could only handle 10,240 points and Comparative Solution 2 could handle 25,600 points, demonstrating the present invention's superior ability to process long sequence data.

[0110] In terms of parameter efficiency, the parameters of the solution of the present invention are only 7.6K, which is 91.8% less than the 92.5K of comparison solution 1 and 80.2% less than the 38.4K of comparison solution 2. The model size is only 0.31MB, which is 8.3% of comparison solution 1 and 19.6% of comparison solution 2. This significant improvement in parameter efficiency is attributed to the adaptive sparse mask generation mechanism that effectively identifies and retains important parameters while dynamically pruning redundant parameters. It is worth noting that despite the significant reduction in the number of parameters, the prediction accuracy of the solution of the present invention is still better than the comparison solution with a larger number of parameters, which proves the efficiency of the method. During the training process, the convergence speed of the solution of the present invention is also faster, requiring only 41.7 rounds to converge, which is 47.0% less than the 78.6 rounds of comparison solution 1, indicating the effectiveness of the optimization strategy.

[0111] In terms of system resource usage, the GPU memory usage of the solution of the present invention is only 1.75GB, which is 13.6% of that of the comparative solution 1 and 32.6% of that of the comparative solution 2, significantly reducing the hardware resource requirements. The power consumption is reduced to 92.4W, which is 69.0% lower than that of the comparative solution 1 and 50.4% lower than that of the comparative solution 2, making it more energy-saving and environmentally friendly. These characteristics make the solution of the present invention particularly suitable for long-term operation in resource-constrained environments, and provide a guarantee for the continuous and stable operation of the virtual power plant system. In terms of edge deployment, the inference time of the solution of the present invention on Jetson Xavier NX is only 5.83ms, which is 86.3% less than the 42.56ms of the comparative solution 1, meeting the real-time prediction requirements in edge computing scenarios and supporting the distributed deployment of virtual power plants.

[0112] In terms of early warning of load mutation events, the accuracy of the solution of the present invention reached 96.78%, which is 43.8% higher than that of comparison solution 1 and 17.4% higher than that of comparison solution 2, indicating that this method has significant advantages in detecting and predicting load mutation events. This has important practical value for virtual power plant scheduling warning and emergency response. The average number of core tensor update iterations is 26.4 times, which proves that the adaptive core tensor update strategy can achieve high-quality optimization within a smaller number of iterations and improve the model training efficiency. Overall, the solution of the present invention comprehensively surpasses the existing technical solutions in various key indicators, especially in the prediction of load mutation periods, which shows the significant technical progress and practical value of the present invention in the field of virtual power plant load forecasting.

[0113] The above are only embodiments of the present invention. The common knowledge of the specific structures and characteristics known in the scheme is not described in detail here. Those of ordinary skill in the art are aware of all the common technical knowledge in the technical field to which the invention belongs before the application date or priority date, can obtain all the existing technologies in the field, and have the ability to apply conventional experimental means before that date. Those of ordinary skill in the art can improve and implement this scheme in combination with their own abilities under the enlightenment given by this application. Some typical well-known structures or well-known methods should not become obstacles for those of ordinary skill in the art to implement this application. It should be pointed out that for those skilled in the art, several variations and improvements can be made without departing from the structure of the present invention. These should also be regarded as the scope of protection of the present invention. These will not affect the effect of the implementation of the present invention and the practicality of the patent. The scope of protection required by this application shall be based on the content of its claims, and the records of the specific implementation methods in the specification can be used to interpret the content of the claims.

Claims

1. An adaptive tensor decomposition attention mechanism for virtual power plants and its prediction optimization method, characterized by: include: S01 Data Collection and Preprocessing: Collect multimodal data of virtual power plants, perform normalization, extract time-frequency domain features, clean outliers, align and resample time series data, and perform feature engineering and encoding processing; S02 model construction: Build an adaptive tensor decomposition attention network based on the processed data. The specific method is as follows: Based on the preprocessed data, an adaptive tensor decomposition attention network is built. The core tensor is initialized using the improved Xavier initialization method. The formula is: in, Indicates the The size of the dimensions, Indicates the number of dimensions of the tensor; Perform feature encoding on multimodal data and convert it into a tensor representation suitable for model processing. The formula is , Indicates the The feature representation of each modality, Indicates the The original data of each modality, Indicates the The encoder of the modality, Represents encoder parameters; By using the multimodal feature fusion mechanism, the formula The core tensor is then fused with each modal feature to form a multimodal fusion feature representation, where : The fused feature tensor, represents the core tensor, Indicates the The feature representation of each modality, Indicates the The weight matrix of each mode, Indicates that along The tensor product of the modes; Then perform adaptive tensor decomposition attention calculation and weighted feature aggregation, and calculate the attention score based on the fusion feature to evaluate the feature importance. The formula is And perform weighted aggregation on the features according to the attention weight, the formula is ,in represents the weighted feature representation, represents the attention weight matrix, represents the fused feature tensor, represents the feature dimension, express The transpose of represents the normalized function, and the virtual power plant load forecasting model is obtained; S03 model optimization: perform adaptive core tensor updates, calculate the Hessian matrix through low-rank approximation, and then dynamically generate adaptive sparse masks based on gradient sensitivity analysis. At the same time, introduce Nesterov acceleration momentum and update the core tensor based on the above calculation results. Use sparse conjugate projection technology to ensure that the parameters meet physical constraints. Use a gradient-guided dynamic core tensor adjustment strategy to calculate the gradient importance distribution and adjust the core tensor. Combined with batch processing and parallel computing strategies, organize the core tensors of multiple instances into batch form, and use parallel computing technology to update multiple core tensors at the same time. Introduce an adaptive computing resource allocation strategy to dynamically allocate computing resources according to module importance. The updated core tensor and the trained model are input into the collected data for calculation to obtain the virtual power plant load forecast result.

2. The adaptive tensor decomposition attention mechanism and prediction optimization method for virtual power plants according to claim 1 is characterized by: The multimodal data in S01 includes historical load data, real-time load data, meteorological data, equipment operation status data, electricity price data and user behavior data.

3. The adaptive tensor decomposition attention mechanism and prediction optimization method for virtual power plants according to claim 1 is characterized by: The formula for the normalization process in S01 is: in, Represents the original data, represents the data mean, represents the standard deviation of the data, Represents a small constant that prevents division by zero.

4. The adaptive tensor decomposition attention mechanism and prediction optimization method for virtual power plants according to claim 1 is characterized by: In S01, an adaptive Hampel filter is used to detect and correct abnormal values ​​in the original data. The processing process can be expressed as follows: in represents the data points after cleaning, represents the original data points, Indicates A sliding window centered on Display window The median of the data, Display window The median absolute deviation of the data, The threshold coefficient γ is adaptively adjusted according to the data distribution. The value range of the threshold coefficient γ is [3,5]. It is usually adaptively adjusted according to the data noise level. The default value is 3.

5.

5. The adaptive tensor decomposition attention mechanism and prediction optimization method for virtual power plants according to claim 1 is characterized by: The frequency domain feature extraction in S01 is expressed as: in, Represents time series data at time The value of Indicates the number of items, represents the time-frequency analysis parameters, represents the time-frequency analysis index parameter, Represents the center point parameter of time-frequency analysis, the time-frequency analysis parameter 、 、 It is automatically learned by stochastic gradient descent, an optimization algorithm that minimizes prediction loss, where m is the preset number of time-frequency components, usually 8≤m≤16.

6. The adaptive tensor decomposition attention mechanism and prediction optimization method for virtual power plants according to claim 1 is characterized by: The adaptive core tensor update strategy in S03 is: in, Indicates the The core tensor of the step, express Hessian Low-rank approximation of the matrix, represents the gradient of the loss with respect to the core tensor, represents the adaptive sparse mask, represents the learning rate, express Nesterov Acceleration momentum term, express Hadamard product, represents the constraint space projection, represents the soft threshold parameter, 、 、 Indicates control parameters.

7. The adaptive tensor decomposition attention mechanism and prediction optimization method for virtual power plants according to claim 1 is characterized by: The S03 adaptive sparse mask generation mechanism is dynamically generated through gradient sensitivity analysis: in, Represents the time step Adaptive sparse mask of express Sigmoid function, Represents the gradient sensitivity parameter, with a value range of , Indicates the parameter penalty coefficient, the value range , Indicates the baseline threshold, the value range is , represents the absolute value of the gradient of the loss with respect to the core tensor, represents the absolute value of the core tensor parameter, express Frobenius norm, Represents a small constant, set to .

8. The adaptive tensor decomposition attention mechanism and prediction optimization method for virtual power plants according to claim 1 is characterized by: The low-rank approximation calculation of the Hessian matrix in S03 is calculated by an efficient iterative algorithm: in, represents the front of the Hessian matrix eigenvalues, represents the corresponding eigenvector, represents the low-rank approximation dimension, which is usually much smaller than the parameter dimension.

9. The adaptive tensor decomposition attention mechanism and prediction optimization method for virtual power plants according to claim 1 is characterized by: The gradient-guided dynamic core tensor adjustment strategy in S03 includes calculating the gradient importance distribution: and adjust the core tensor , in, represents the gradient importance distribution, Indicates the The gradient norm in the dimension, represents the temperature parameter, which controls the smoothness of the distribution, Indicates the adjustment strength parameter.

Images