Power large model training optimization method based on stochastic gradient descent variants

Abstract

This invention discloses a power system large-scale model training optimization method based on a variant of stochastic gradient descent, belonging to the field of power system large-scale model training optimization technology. It addresses the conflict between high-frequency dynamic fluctuations and long-term stability in power system data, enhancing the convergence and stability of the model. The method includes the following steps: preprocessing historical power system data to remove random noise and dividing the data into short-term fluctuation data and long-term trend data. This invention accurately captures high-frequency fluctuations in power load through weighted mean square error and a sliding window mechanism, improving short-term forecast accuracy and optimizing grid dispatch. Simultaneously, it employs a trend fitting loss function to enhance long-term load forecasting capabilities, facilitating advance planning during peak electricity consumption periods and reducing operating costs. Combining time decay regularization and Adam adaptive learning rate enhances the model's stability and generalization ability, improving the reliability of power forecasting and dispatch.

Description

Technical Field

[0001] This invention relates to the field of power large model training and optimization technology, and specifically to a power large model training and optimization method based on a variant of stochastic gradient descent. Background Technology

[0002] Large-scale power models based on stochastic gradient descent variants refer to the application of large-scale machine learning models in power systems. These models are trained using variants of optimization algorithms (such as stochastic gradient descent and its derivatives) to effectively handle the complex prediction, optimization, and control problems involved in power systems. Stochastic gradient descent (SGD) is a commonly used optimization algorithm for training deep learning models, and its variants (such as momentum descent, AdaGrad, and Adam) perform better in handling different data characteristics or convergence speeds. These models are called "large-scale power models" because they often need to process large amounts of power system data, including multiple stages such as generation, transmission, distribution, and load forecasting, and have a wide range of application scenarios and data requirements.

[0003] In this application scenario, stochastic gradient descent variants can quickly adjust model parameters, enabling the model to better fit historical and real-time data from the power system. For example, in load forecasting or grid dispatching problems, large models can be optimized and adjusted in real time by learning time-series patterns of electricity demand and external influencing factors. Variants employing optimization algorithms can improve the efficiency of model training, reduce overfitting or underfitting, and thus enable the model to exhibit higher accuracy and stability during power system operation.

[0004] The existing technology has the following shortcomings:

[0005] A common problem encountered when training large-scale power system models using stochastic gradient descent variants is the conflict between high-frequency dynamic fluctuations and long-term trends in power system data. Power system data exhibits complex temporal characteristics, such as instantaneous load fluctuations, seasonal variations, and the impact of unforeseen events. This makes it easy for optimization algorithms to overemphasize short-term dynamic changes during model training, neglecting the long-term stable trends in the data. Although stochastic gradient descent variants can quickly adjust model parameters, overly frequent adjustments may prevent the algorithm from converging to the global optimum or even be affected by high-frequency fluctuations. To address this issue, it is necessary to balance short-term fluctuations and long-term trends during optimization. This can be achieved by introducing techniques such as regularization or smoothing to enhance model stability, enabling it to capture both short-term changes and identify long-term trends.

[0006] The information disclosed in the background section is only intended to enhance the understanding of the background of this disclosure, and therefore may include information that does not constitute prior art known to those skilled in the art. Summary of the Invention

[0007] The purpose of this invention is to provide a training and optimization method for large-scale power models based on a variant of stochastic gradient descent. By employing a weighted mean squared error loss function, combined with a sliding window and dynamic weighting mechanism, it effectively captures high-frequency fluctuations in power load, improves the accuracy of short-term forecasts, helps power companies cope with sudden changes, and enhances grid dispatch efficiency. Simultaneously, through a trend-fitting loss function, the model can accurately predict long-term power load changes, capture seasonal and climatic impacts, and help power companies plan dispatch in advance during peak electricity demand periods, optimize generation resources, and reduce operating costs. Furthermore, by combining time decay regularization and Adam adaptive learning rate, the model exhibits better stability and generalization ability during dynamic adjustments, ensuring accurate predictions and improving the overall reliability of grid dispatch, thereby addressing the problems mentioned in the background art.

[0008] To achieve the above objectives, this invention provides the following technical solution: a power system large-scale model training and optimization method based on a stochastic gradient descent variant, used to resolve the conflict between high-frequency dynamic fluctuations and long-term stability in power system data, and to enhance the convergence and stability of the model, comprising the following steps:

[0009] Historical data of the power system are preprocessed to remove random noise and divide the data into short-term fluctuation data and long-term trend data;

[0010] An optimized variant algorithm based on stochastic gradient descent is adopted. Through an adaptive learning rate adjustment mechanism, the learning rate is dynamically adjusted according to the local gradient changes of the power system data to avoid overfitting or underfitting caused by high-frequency fluctuations.

[0011] Establish a loss function that includes short-term and long-term objectives. The short-term objective is used to capture high-frequency fluctuations in the data, while the long-term objective is used to identify long-term trends. By adding a regularization term to the loss function, the short-term and long-term objectives are weighted to ensure that both short-term and long-term data features can be considered during the training process.

[0012] During training, a multi-scale sliding window technique is used to extract features from data at different time periods and dynamically select training samples within a specific time range based on the window size to ensure that the model can capture features at different time scales.

[0013] During model training, regularization techniques are introduced to address the time-series characteristics of power system data, thereby suppressing the model's sensitivity to short-term noise while maintaining effective capture of long-term stable trends.

[0014] After training, the model is evaluated using a validation set. Based on the evaluation results, the learning rate and regularization parameters are further adjusted, and the hyperparameters during the training process are optimized to ensure that the model can adapt to the dynamic changes and long-term stability of the power system.

[0015] Preferably, the data preprocessing further includes the following sub-steps:

[0016] Instantaneous load fluctuation data is processed based on empirical mode decomposition (EMD) to decompose the load data into multiple intrinsic mode functions (IMF) to separate different frequency components and eliminate local data noise;

[0017] A sliding window is used to smooth seasonal data, generating trend data for different periods. Periodic feature normalization is introduced to address seasonal changes, ensuring that the data is uniformly input into the model.

[0018] Preferably, the adaptive learning rate adjustment mechanism specifically uses the Adam optimization algorithm, whose sub-steps include:

[0019] During training, the learning rate is adaptively adjusted by calculating the first moment estimate (momentum) and the second moment estimate (exponential moving average of the squared gradient).

[0020] For high-frequency fluctuations, the learning rate is automatically reduced to avoid short-term fluctuations interfering with model convergence;

[0021] For the long-term trend part, maintain an appropriate learning rate to ensure that the model can fully capture long-term stability.

[0022] Preferably, the loss function is further subdivided into two parts:

[0023] The short-term volatility loss function uses mean squared error (MSE) as the optimization objective to measure the volatility of the model's prediction accuracy over a short period of time.

[0024] The long-term trend loss function uses a trend-fit-based loss (such as error calculation based on linear regression or multinomial fitting) to ensure that the model accurately tracks the long-term trend, and uses weight parameters to adjust the two parts of the loss.

[0025] A loss function is established that includes short-term and long-term objectives. By adding a regularization term to the loss function, weights are assigned to the short-term and long-term objectives. The specific steps are as follows:

[0026] To address short-term data fluctuations in power systems, a short-term objective loss function L is defined. short This loss function captures high-frequency fluctuations in the data and is calculated using a weighted mean square error. The specific calculation formula is as follows: In the formula, N is the number of samples in the short-term data, and y i Indicates the actual value. w represents the model's predicted value. i The weights for each sample are used to highlight the impact of high-frequency fluctuations, and are specifically defined as follows: In the formula, t i It's a timestamp, T short It is a short-term cycle (e.g., 24 hours is the load fluctuation cycle). α is a hyperparameter that controls the intensity of fluctuations. It usually takes a value between [0, 1]. This weighting function ensures that the part of short-term fluctuations is given a greater influence.

[0027] To capture long-term trends in electricity data, a long-term objective loss function L is defined. long Unlike short-term fluctuations, long-term trends require more smoothness. Therefore, the long-term loss is calculated using the mean squared error based on the trend fitting. The specific calculation formula is as follows: In the formula, M is the number of samples in the long-term data, and T(·) represents the trend fitting function for the long-term data, which can be a smoothing algorithm such as linear regression, polynomial fitting, or local regression. j )and These represent the actual trend of long-term data and the trend predicted by the model, respectively.

[0028] This loss function focuses on the model's accuracy in fitting long-term trends, ensuring good predictive ability for long-term changes in the power system.

[0029] To prevent the model from overfitting short-term fluctuations or long-term trends, and to improve the model's robustness, a regularization term R(θ) is introduced to constrain the model parameter θ. This regularization term considers not only conventional L2 regularization but also time-weighted regularization. The specific calculation formula is as follows: In the formula, λ is the regularization coefficient, P is the number of model parameters, and t k For timestamps, T long It is a long-term period, and β is a hyperparameter that controls the decay rate of the time weight;

[0030] This regularization term introduces the concept of time decay, imposing stronger constraints on data from more distant times, ensuring that the model pays appropriate attention to historical data without over-relying on it.

[0031] Short-term loss L short and long-term loss L long Combined to form the total loss function L total By introducing the weight parameter λ short and λ long The balance between short-term and long-term goals is dynamically adjusted, and the expression for dynamic adjustment is: L total =λ short ·L short +λ long ·L long +R(θ), where λ short and λ longR(θ) is a weight parameter that controls the relative importance of short-term and long-term losses and can be adjusted using a validation set. R(θ) is a defined regularization term.

[0032] Preferably, multi-scale sliding window processing is performed in the following manner:

[0033] Use small time windows (such as minute or hourly) for short-term volatile data and large time windows (such as day or week) for long-term data.

[0034] During each training session, an appropriate window is automatically selected based on different time scales, and the data within the window is weighted to ensure that the impact of the time span can be flexibly adjusted at different training stages of the model.

[0035] Preferably, time series regularization further includes the introduction of time decay weighting, specifically:

[0036] Apply smaller weights to earlier data in the time series and larger weights to more recent data to ensure that the model is more sensitive to learning from recent data;

[0037] The weights are adjusted using an exponential decay function to smooth the impact of historical data on the model, thereby preventing historical data from causing excessive interference with future predictions.

[0038] Preferably, the model evaluation further includes:

[0039] Cross-validation is used to divide the training data of the model into multiple folds, and the mean error and variance are calculated for each fold to evaluate the stability of the model over different time periods.

[0040] Based on the evaluation results, the weight parameters, learning rate decay parameters, and regularization coefficients in the loss function are dynamically adjusted to continuously optimize the model's adaptability and prediction accuracy in the power system.

[0041] The technical effects and advantages provided by the present invention in the above technical solution are as follows:

[0042] This invention employs a short-term fluctuation capture method, particularly a weighted mean square error loss function, enabling the model to more accurately capture high-frequency fluctuations in power load. Traditional forecasting models often struggle to effectively handle drastic load changes within a short period. This invention, through a sliding window and dynamic weighting mechanism, ensures that the model can adjust forecasts more quickly and accurately during periods of significant load fluctuation (such as morning and evening peak hours), thereby improving grid dispatch efficiency. This efficient short-term forecasting method not only helps power companies respond promptly to sudden load changes, preventing grid overload or power shortages, but also provides more accurate data support for electricity market price regulation and power generation planning within a short timeframe.

[0043] To address seasonal or long-term trend changes in power systems, this invention introduces a trend-fitting loss function, effectively improving the model's predictive ability for long-term power load variations. Through trend-fitting techniques such as multinomial regression or locally weighted regression on historical data, the model can capture the impact of long-term factors like seasonality and climate change on power load. This long-term predictive capability enables power companies to make advance scheduling and planning during peak electricity demand periods, especially accurately predicting power demand before the winter and summer peak seasons, thus avoiding power supply shortages caused by surging demand. This method can also optimize the scheduling of power generation resources, effectively reducing operating costs caused by over-scheduling or under-suppliing.

[0044] This invention significantly improves the stability and robustness of the model by combining time decay regularization and Adam adaptive learning rate optimization. The time decay regularization method ensures that the model can flexibly adjust weights according to the temporal attributes of data when processing data from different time periods, reducing the excessive influence of historical data on the model and thus avoiding overfitting to historical fluctuations. Simultaneously, the Adam adaptive learning rate adjusts the learning rate according to the dynamic changes in the gradient, accelerating parameter adjustment during periods of severe power load fluctuations to ensure rapid model response, while slowing down the learning speed during stable phases to improve model convergence. Through this optimization strategy, the model exhibits better adaptability and generalization ability in complex power systems, effectively balancing model accuracy and robustness in dealing with both short-term fluctuations and long-term trends, thereby improving the overall reliability of power forecasting and grid dispatching. Attached Figure Description

[0045] To more clearly illustrate the technical solutions in the embodiments of this application or the prior art, the drawings used in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments recorded in this invention. For those skilled in the art, other drawings can be obtained based on these drawings.

[0046] Figure 1This is a flowchart of the method for training and optimizing a large power model based on a variant of stochastic gradient descent, as described in this invention. Detailed Implementation

[0047] Exemplary embodiments will now be described more fully with reference to the accompanying drawings. However, these exemplary embodiments can be implemented in many forms and should not be construed as limited to the examples set forth herein; rather, they are provided so that the description of this disclosure will be more complete and fully convey the concept of the exemplary embodiments to those skilled in the art.

[0048] This invention provides, for example Figure 1 The power system large model training optimization method based on a stochastic gradient descent variant, shown below, is used to resolve the conflict between high-frequency dynamic fluctuations and long-term stability in power system data, and to enhance the convergence and stability of the model. It includes the following steps:

[0049] The historical data of the power system is preprocessed, including time series data such as instantaneous load fluctuations, seasonal changes and sudden events. The steps include denoising, normalization and segmentation, removing random noise and dividing the data into short-term fluctuation data and long-term trend data.

[0050] An optimized variant algorithm based on stochastic gradient descent (SGD) is adopted. Through an adaptive learning rate adjustment mechanism, such as Adam, RMSProp or AdaGrad, the learning rate is dynamically adjusted according to the local gradient changes of the power system data to avoid overfitting or underfitting caused by high-frequency fluctuations.

[0051] Establish a loss function that includes short-term and long-term objectives. The short-term objective is used to capture high-frequency fluctuations in the data, while the long-term objective is used to identify long-term trends. By adding a regularization term to the loss function, the short-term and long-term objectives are weighted to ensure that both short-term and long-term data features can be considered during the training process.

[0052] During training, a multi-scale sliding window technique is used to extract features from data at different time periods (short-term and long-term), and to dynamically select training samples within a specific time range based on the window size, ensuring that the model can capture features at different time scales.

[0053] During model training, regularization techniques, such as L2 regularization or time-weighted regularization, are introduced to address the time-series characteristics of power system data. This suppresses the model's sensitivity to short-term noise while maintaining effective capture of long-term stable trends.

[0054] After training, the model is evaluated using a validation set, and metrics such as convergence and accuracy are calculated. Based on the evaluation results, the learning rate and regularization parameters are further adjusted, and the hyperparameters during the training process are optimized to ensure that the model can adapt to the dynamic changes and long-term stability of the power system.

[0055] The data preprocessing further includes the following sub-steps:

[0056] Instantaneous load fluctuation data is processed based on empirical mode decomposition (EMD) to decompose the load data into multiple intrinsic mode functions (IMF) to separate different frequency components and eliminate local data noise;

[0057] A sliding window is used to smooth seasonal data, generating trend data for different periods. Periodic feature normalization is introduced to address seasonal changes, ensuring that the data is uniformly input into the model.

[0058] The adaptive learning rate adjustment mechanism specifically uses the Adam optimization algorithm, and its sub-steps include:

[0059] During training, the learning rate is adaptively adjusted by calculating the first moment estimate (momentum) and the second moment estimate (exponential moving average of the squared gradient).

[0060] For high-frequency fluctuations, the learning rate is automatically reduced to avoid short-term fluctuations interfering with model convergence;

[0061] For the long-term trend part, maintain an appropriate learning rate to ensure that the model can fully capture long-term stability.

[0062] The loss function is further subdivided into two parts:

[0063] The short-term volatility loss function uses mean squared error (MSE) as the optimization objective to measure the volatility of the model's prediction accuracy over a short period of time.

[0064] The long-term trend loss function uses a trend-fit-based loss (such as error calculation based on linear regression or multinomial fitting) to ensure that the model accurately tracks the long-term trend, and uses weight parameters to adjust the two parts of the loss.

[0065] A loss function is established that includes short-term and long-term objectives. By adding a regularization term to the loss function, weights are assigned to the short-term and long-term objectives. The specific steps are as follows:

[0066] To address short-term data fluctuations in power systems, a short-term objective loss function L is defined. short This loss function captures high-frequency fluctuations in the data and is calculated using a weighted mean square error. The specific calculation formula is as follows: In the formula, N is the number of samples in the short-term data, and yi Indicates the actual value. w represents the model's predicted value. i The weights for each sample are used to highlight the impact of high-frequency fluctuations, and are specifically defined as follows: In the formula, t i It's a timestamp, T short It is a short-term cycle (e.g., 24 hours is the load fluctuation cycle). α is a hyperparameter that controls the intensity of fluctuations. It usually takes a value between [0, 1]. This weighting function ensures that the part of short-term fluctuations is given a greater influence.

[0067] To capture long-term trends in electricity data, a long-term objective loss function L is defined. long Unlike short-term fluctuations, long-term trends require more smoothness. Therefore, the long-term loss is calculated using the mean squared error based on the trend fitting. The specific calculation formula is as follows: In the formula, M is the number of samples in the long-term data, and T(·) represents the trend fitting function for the long-term data, which can be a smoothing algorithm such as linear regression, polynomial fitting, or local regression. j )and These represent the actual trend of long-term data and the trend predicted by the model, respectively.

[0068] This loss function focuses on the model's accuracy in fitting long-term trends, ensuring good predictive ability for long-term changes in the power system.

[0069] To prevent the model from overfitting short-term fluctuations or long-term trends, and to improve the model's robustness, a regularization term R(θ) is introduced to constrain the model parameter θ. This regularization term considers not only conventional L2 regularization but also time-weighted regularization. The specific calculation formula is as follows: In the formula, λ is the regularization coefficient, P is the number of model parameters, and t k For timestamps, T long It is a long-term period, and β is a hyperparameter that controls the decay rate of the time weight;

[0070] This regularization term introduces the concept of time decay, imposing stronger constraints on data from more distant times, ensuring that the model pays appropriate attention to historical data without over-relying on it.

[0071] Short-term loss L short and long-term loss L long Combined to form the total loss function L total By introducing the weight parameter λ short and λ long The balance between short-term and long-term goals is dynamically adjusted, and the expression for dynamic adjustment is: L total =λ short ·Lshort +λ long ·L long +R(θ), where λ short and λ long R(θ) is a weight parameter that controls the relative importance of short-term and long-term losses and can be adjusted using a validation set. R(θ) is a defined regularization term.

[0072] Multi-scale sliding window processing is performed in the following way:

[0073] Use small time windows (such as minute or hourly) for short-term volatile data and large time windows (such as day or week) for long-term data.

[0074] During each training session, an appropriate window is automatically selected based on different time scales, and the data within the window is weighted to ensure that the impact of the time span can be flexibly adjusted at different training stages of the model.

[0075] Time series regularization further includes the introduction of time decay weighting, specifically:

[0076] Apply smaller weights to earlier data in the time series and larger weights to more recent data to ensure that the model is more sensitive to learning from recent data;

[0077] The weights are adjusted using an exponential decay function to smooth the impact of historical data on the model, thereby preventing historical data from causing excessive interference with future predictions.

[0078] The model evaluation further includes:

[0079] Cross-validation is used to divide the training data of the model into multiple folds, and the mean error and variance are calculated for each fold to evaluate the stability of the model over different time periods.

[0080] Based on the evaluation results, the weight parameters, learning rate decay parameters, and regularization coefficients in the loss function are dynamically adjusted to continuously optimize the model's adaptability and prediction accuracy in the power system.

[0081] Implementation method 1: Short-term fluctuation capture based on power load data;

[0082] This implementation proposes an optimization method based on a short-term objective loss function to accurately capture high-frequency fluctuations in power load, addressing the short-term volatility characteristics of power load. Short-term power load fluctuations typically manifest as frequent intraday load fluctuations, such as sharp increases or decreases in power demand during morning and evening peak hours. These short-term load fluctuations have a significant impact on power grid dispatching; therefore, capturing these fluctuation characteristics is crucial during model training. First, the input power load data undergoes preprocessing, employing a sliding window mechanism for segmented processing. This sliding window sets multiple windows based on the time scale: short-term windows capture high-frequency fluctuations at the minute or hour level, while long-term windows capture subsequent long-term trends. Through sliding window processing, the model can effectively decompose the load data into high-frequency fluctuations and low-frequency trend components, processing them separately. In the data preprocessing stage, noise filtering technology is also introduced, utilizing the Empirical Mode Decomposition (EMD) method to separate intrinsic mode functions (IMFs) of different frequencies, further eliminating noise interference and ensuring that the model focuses only on the true fluctuation characteristics.

[0083] In constructing the short-term objective loss function, **Weighted Mean Squared Error (WMSE)** was used for optimization. Unlike traditional mean squared error (MSE), WMSE introduces dynamic weights, allowing the model to assign greater weight to high-frequency fluctuations. This weighting function enables the model to pay more attention to load peaks and troughs when capturing short-term fluctuations. Since the amplitude of power load fluctuations varies across different time periods, dynamically adjusted weights can effectively enhance the model's sensitivity to high-frequency fluctuations. For example, during morning and evening peak hours, power demand may suddenly surge or drop. The weighted loss function allows the model to quickly respond to load changes, improving prediction accuracy. During periods of smaller load fluctuations, the weights are reduced accordingly to prevent the model from overfitting to data with small fluctuations, thus ensuring the overall stability of the prediction.

[0084] Furthermore, to enhance the model's ability to capture short-term fluctuations, this implementation also employs an adaptive learning rate optimization strategy. Specifically, the Adam optimization algorithm is used to control the model's update speed by dynamically adjusting the learning rate. For short-term, high-frequency fluctuations, the algorithm can accelerate the model's convergence speed with a higher learning rate, enabling it to respond promptly to rapid data changes and avoid prediction lag. For relatively stable time periods, the Adam algorithm automatically reduces the learning rate through its adaptive mechanism to prevent the model from over-adjusting parameters and causing unnecessary fluctuations. This learning rate adjustment strategy, combined with the weighting mechanism of the short-term loss function, not only improves the model's responsiveness to short-term fluctuations but also effectively suppresses overfitting, enhancing the model's applicability in actual power grid dispatching. In practical applications of load forecasting or power grid dispatching, this optimization method can significantly improve the power system's response speed and stability, helping power companies better cope with instantaneous load fluctuations and achieve efficient power grid management.

[0085] Implementation Method 2: Long-term goal optimization based on seasonal trends;

[0086] This implementation addresses the seasonal load variations in power systems by designing a long-term objective-based optimization method focused on capturing long-term trends in power load. Electricity demand typically exhibits significant cyclical fluctuations with seasonal changes, such as increased load in winter due to heating demand or peak electricity consumption in summer due to increased air conditioning usage. To effectively capture these long-term trends, the model introduces a trend fitting loss function (MSE-TF), specifically designed to optimize the fitting effect on long-term data. In practical applications, long-term load data exhibits strong stability and predictability; therefore, the model's primary task is to improve the accuracy of future load predictions by fitting these long-term trends. This long-term objective loss function generates a trend curve by performing multinomial regression, locally weighted regression, or other trend fitting methods on historical data, and optimizes the loss value using the difference between the actual load value and the fitted trend.

[0087] To ensure the model accurately captures these seasonal trends, a multi-window sliding window mechanism is employed in the data preprocessing stage, especially for long-term data, which is processed using **larger time windows (e.g., weekly or monthly units).** After processing by a trend-fitting algorithm, the data within the long-term window reveals the dominant patterns of electricity demand variation. These patterns are typically influenced by climate conditions, holidays, and economic cycles. By assigning greater weight to long-term data in the loss function, the model focuses more on these long-term trends rather than short-term fluctuations during optimization. In this way, the model significantly improves the accuracy of predicting seasonal changes in electricity demand, particularly during the winter and summer seasons when demand fluctuates significantly, helping grid operators plan and schedule power supply in advance.

[0088] Another key element of long-term objective optimization is the dynamic adjustment of model parameters. Since seasonal fluctuations in electricity load are not uniformly distributed, the model needs to flexibly adjust its learning rate and regularization term according to the characteristics of different seasons and time periods. During peak electricity demand periods in winter and summer, the model needs to maintain high sensitivity to sharp fluctuations in load data, appropriately increasing the learning rate to cope with these rapidly changing trends. Conversely, during the relatively stable electricity demand periods of spring and autumn, the model needs to reduce the learning rate to ensure stable convergence and avoid overfitting. In addition to the long-term loss function, a time-weighted regularization term is added to further constrain the model parameters, preventing the model from overemphasizing noise or abnormal fluctuations in historical data. Through this time-weighted mechanism, the model maintains an accurate grasp of the overall trend over a longer period, without being disturbed by short-term outliers.

[0089] Through long-term objective optimization in this implementation method, the model can effectively improve its ability to capture long-term trends in the power system and demonstrates significant advantages in seasonal load forecasting. This optimization method is particularly suitable for scenarios where electricity demand fluctuates cyclically, helping power companies and grid operators plan power supply in advance and avoid power shortages or overloads during peak demand periods. By accurately predicting long-term load trends, grid dispatching can more precisely allocate and reserve power resources, further improving the operational efficiency and reliability of the grid.

[0090] Implementation Method 3: Optimization by combining regularization and adaptive learning rate;

[0091] This implementation combines time decay regularization and adaptive learning rate optimization strategies to address the conflict between short-term fluctuations and long-term trends in power system data. The core of this combined strategy is the introduction of time-weighted regularization, which allows the model to dynamically adjust its focus on data from different time periods during training. Specifically, the regularization term not only imposes conventional constraints on model parameters but also introduces a time decay factor, assigning different weights based on the distance of the data from the current moment. This regularization method ensures that the model's influence on more distant historical data gradually diminishes, while imposing greater weight on more recent data, thus enabling the model to flexibly respond to the different characteristics of historical and real-time data. This method effectively solves the balance problem between historical data and current fluctuations in power systems, preventing the model from over-relying on historical data and ignoring current changes, while also preventing noise in recent data from causing excessive interference to the model.

[0092] For learning rate optimization, the Adam optimization algorithm was employed. This adaptive learning rate optimization method automatically adjusts the model's learning rate based on gradient changes. Power system data is complex and volatile, especially during periods of frequent load fluctuations. The model needs to respond quickly to these data changes. Therefore, during these periods of high volatility, the Adam algorithm automatically increases the learning rate to facilitate rapid parameter updates and ensure the model can quickly capture these dynamic characteristics. Conversely, during periods of stable data change, such as long-term load trends or seasonal low-fluctuation phases, the Adam algorithm automatically decreases the learning rate to prevent unnecessary parameter adjustments and avoid overfitting. This adaptive learning rate mechanism allows the model to maintain its optimal learning state across different time periods, effectively balancing the model's adjustment speed and accuracy when dealing with both short-term fluctuations and long-term trends.

[0093] This optimization strategy, combining regularization and adaptive learning rate, improves model prediction accuracy while enhancing its stability and robustness. Through time-decaying regularization, the model avoids over-reliance on noise or outliers in historical data, thereby improving its generalization ability. Furthermore, the adaptive adjustment of the Adam optimization algorithm allows the model to autonomously adjust its learning rate in response to load changes at different time scales, ensuring its flexibility and adaptability. This combined approach is particularly suitable for handling large-scale, dynamically changing power system data, such as in power load forecasting and grid dispatch optimization. The model can quickly adapt to the data characteristics of different time periods, improving the overall operating efficiency and prediction accuracy of the power system. In addition, the flexibility of this optimization method allows for effective application in power systems of varying sizes and complexities, providing a more accurate and reliable solution for smart grid dispatch and management.

[0094] Example 1: Used for short-term fluctuation forecasting of power load;

[0095] The first embodiment of this invention is applied to the prediction of short-term fluctuations in power load. To cope with the high-frequency dynamic changes in power load demand, especially when instantaneous load fluctuations are frequent in the power system, the accuracy of prediction is particularly important. Short-term fluctuations in power load may be affected by sudden events, short-term climate changes, or user electricity consumption habits. Existing prediction models often struggle to achieve stable convergence when faced with such complex time-series data. Therefore, this embodiment aims to improve the prediction performance of short-term fluctuations through the method of this invention.

[0096] In this embodiment, the historical data of the power system is first preprocessed, dividing the data into short-term fluctuation data and long-term trend data. The core of this step is to remove random noise to reduce interference factors in the data. For example, instantaneous load data is processed hierarchically using Empirical Mode Decomposition (EMD) to extract the high-frequency fluctuation component while filtering out noise. Next, the high-frequency fluctuation data is further processed using a sliding window technique. The sliding window divides the data into different time periods, such as 1-hour or 2-hour windows, to capture the fluctuation characteristics within these time periods.

[0097] In the modeling process, a variant of the stochastic gradient descent optimization algorithm was employed, combined with an adaptive learning rate mechanism, to dynamically adjust the learning rate based on local gradient changes in load fluctuations. Specifically, for short-term fluctuations, the model automatically reduces the learning rate to avoid overfitting to local fluctuations. The Adam optimization algorithm dynamically adjusts the learning rate of each parameter by calculating the first and second moments, making the model more flexible when dealing with short-term data. Furthermore, to improve the model's short-term prediction accuracy, this invention designs weighted mean squared error as the optimization objective of the loss function.

[0098] After several rounds of training, cross-validation was used to evaluate the model's predictive performance. Experimental results show that the model significantly improves prediction accuracy under conditions of significant short-term fluctuations. Through the sliding window technique, the model can automatically select the optimal time window to capture high-frequency fluctuations in the load, reducing prediction errors by 15%-20% compared to traditional models. This advantage is particularly evident in power grid dispatching, especially during sudden load increases or decreases, where the model's response speed and prediction accuracy show significant improvements. This not only optimizes the power company's dispatching decisions but also reduces grid instability caused by short-term fluctuations, thereby improving system operating efficiency.

[0099] Example 2: Seasonal load forecasting in power systems;

[0100] The second embodiment of this invention focuses on seasonal load forecasting in power systems. Seasonal variations are a significant factor in power load changes; for example, peak electricity demand in winter and summer and off-peak demand in spring and autumn pose challenges to power grid dispatching. Traditional forecasting models often struggle to balance the conflict between short-term fluctuations and long-term trends when capturing long-term seasonal trends, especially when dealing with complex time series in power systems, easily leading to overfitting or underfitting problems. This invention significantly improves the model's performance in seasonal load forecasting by introducing a trend fitting loss function and time decay regularization techniques.

[0101] First, historical power system data is preprocessed to remove random noise and seasonally adjust load data. For example, annual data is divided into four time periods based on seasonal variations: spring, summer, autumn, and winter. For each time period, a long-term trend model is constructed, and multi-scale sliding window techniques are used to extract features within different time periods. During the summer peak electricity consumption period, the model focuses on high-frequency fluctuations, while during the stable electricity consumption periods of spring and autumn, it focuses on long-term trend data.

[0102] During training, the model is optimized using an adaptive learning rate mechanism, and dynamically adjusted for seasonal load data using the Adam algorithm. To improve the model's long-term predictive ability, a long-term objective loss function based on trend fitting is designed. This loss function is combined with a short-term volatility loss function, and weight parameters are set to dynamically adjust the importance of both, ensuring that the model can simultaneously focus on short-term fluctuations and long-term trends.

[0103] Validated with data from multiple seasons, the model demonstrates high stability in predicting seasonal loads, particularly during peak summer and winter electricity consumption periods. It effectively anticipates load increases, helping power companies optimize generation plans and reduce operating costs. For example, during the summer peak season, the model can predict peak load a week in advance, reducing prediction error by approximately 10% compared to traditional models and effectively mitigating the risk of power shortages during peak periods. Furthermore, the introduction of time decay regularization enhances the model's adaptability to new data, especially under the influence of seasonal unforeseen events (such as extreme weather), allowing the model to adjust rapidly and ensuring prediction accuracy and stability.

[0104] Example 3: Long-term trend prediction in power grid dispatching;

[0105] The third embodiment of this invention is applied to long-term trend prediction in power grid dispatching. In the actual operation of the power grid, in addition to dealing with short-term load fluctuations, long-term load growth trends are also an important basis for power companies to formulate future development plans. Long-term forecasting can help power companies plan the construction and dispatching of power generation facilities in advance, optimize resource allocation, and improve overall operational efficiency. Addressing the long-term stability problem in the power system, this invention enhances the model's ability to capture long-term load trends through regularization processing and the design of a loss function with long-term objectives.

[0106] In this embodiment, historical power system data is further divided into short-term fluctuation data and long-term trend data. During data preprocessing, a sliding window technique is used for multi-scale analysis, dividing the power load data into weekly, monthly, and yearly time periods. For the long-term trend portion, a trend-fitting loss function is employed. The core purpose of this design is to ensure that the model is not affected by short-term noise during the training process on long-term data, thereby enabling accurate prediction of long-term trends.

[0107] The model employs an adaptive learning rate adjustment mechanism, combined with the Adam optimization algorithm, to dynamically adjust the learning rate. To prevent the model from overfitting short-term fluctuations or long-term trends, a regularization term is designed to constrain the loss function for the long-term load trend component. By introducing a time decay mechanism, the model can assign smaller weights to earlier data and enhance its focus on recent data, thereby improving the accuracy and stability of predictions.

[0108] Model validation has shown that the method of this invention significantly improves the accuracy of long-term load forecasting. For example, when predicting load growth for the next year, the model reduces the forecast error by 15% compared to traditional methods. This accuracy in long-term trend forecasting provides power companies with a reliable basis for rationally allocating power generation resources and making advance scheduling arrangements as electricity demand gradually increases. Particularly in grid expansion and the planning of new power generation facilities, long-term forecasting can effectively reduce the risk of equipment overload or idleness, optimizing the overall efficiency of power supply.

[0109] This invention employs a short-term fluctuation capture method, particularly a weighted mean square error loss function, enabling the model to more accurately capture high-frequency fluctuations in power load. Traditional forecasting models often struggle to effectively handle drastic load changes within a short period. This invention, through a sliding window and dynamic weighting mechanism, ensures that the model can adjust forecasts more quickly and accurately during periods of significant load fluctuation (such as morning and evening peak hours), thereby improving grid dispatch efficiency. This efficient short-term forecasting method not only helps power companies respond promptly to sudden load changes, preventing grid overload or power shortages, but also provides more accurate data support for electricity market price regulation and power generation planning within a short timeframe.

[0110] To address seasonal or long-term trend changes in power systems, this invention introduces a trend-fitting loss function, effectively improving the model's predictive ability for long-term power load variations. Through trend-fitting techniques such as multinomial regression or locally weighted regression on historical data, the model can capture the impact of long-term factors like seasonality and climate change on power load. This long-term predictive capability enables power companies to make advance scheduling and planning during peak electricity demand periods, especially accurately predicting power demand before the winter and summer peak seasons, thus avoiding power supply shortages caused by surging demand. This method can also optimize the scheduling of power generation resources, effectively reducing operating costs caused by over-scheduling or under-suppliing.

[0111] This invention significantly improves the stability and robustness of the model by combining time decay regularization and Adam adaptive learning rate optimization. The time decay regularization method ensures that the model can flexibly adjust weights according to the temporal attributes of data when processing data from different time periods, reducing the excessive influence of historical data on the model and thus avoiding overfitting to historical fluctuations. Simultaneously, the Adam adaptive learning rate adjusts the learning rate according to the dynamic changes in the gradient, accelerating parameter adjustment during periods of severe power load fluctuations to ensure rapid model response, while slowing down the learning speed during stable phases to improve model convergence. Through this optimization strategy, the model exhibits better adaptability and generalization ability in complex power systems, effectively balancing model accuracy and robustness in dealing with both short-term fluctuations and long-term trends, thereby improving the overall reliability of power forecasting and grid dispatching.

[0112] The foregoing has only described certain exemplary embodiments of the present invention by way of illustration. Undoubtedly, those skilled in the art can modify the described embodiments in various ways without departing from the spirit and scope of the present invention. Therefore, the foregoing drawings and descriptions are illustrative in nature and should not be construed as limiting the scope of protection of the claims of the present invention.

Claims

1. A training and optimization method for a large power model based on a variant of stochastic gradient descent, characterized in that, To resolve the conflict between high-frequency dynamic fluctuations and long-term stability in power system data, and to enhance the convergence and stability of the model, the following steps are included: Historical data of the power system are preprocessed to remove random noise and divide the data into short-term fluctuation data and long-term trend data; An optimized variant algorithm based on stochastic gradient descent is adopted. Through an adaptive learning rate adjustment mechanism, the learning rate is dynamically adjusted according to the local gradient changes of the power system data to avoid overfitting or underfitting caused by high-frequency fluctuations. Establish a loss function that includes short-term and long-term objectives. The short-term objective is used to capture high-frequency fluctuations in the data, while the long-term objective is used to identify long-term trends. By adding a regularization term to the loss function, the short-term and long-term objectives are weighted to ensure that both short-term and long-term data features can be considered during the training process. During training, a multi-scale sliding window technique is used to extract features from data at different time periods and dynamically select training samples within a specific time range based on the window size to ensure that the model can capture features at different time scales. During model training, regularization techniques are introduced to address the time-series characteristics of power system data, thereby suppressing the model's sensitivity to short-term noise while maintaining effective capture of long-term stable trends. After training, the model is evaluated using a validation set. Based on the evaluation results, the learning rate and regularization parameters are further adjusted, and the hyperparameters during training are optimized to ensure that the model can adapt to the dynamic changes and long-term stability of the power system. Data preprocessing further includes the following sub-steps: Instantaneous load fluctuation data is processed based on empirical mode decomposition, which decomposes the load data into multiple layers of intrinsic mode functions to separate different frequency components and eliminate local data noise; A sliding window is used to smooth seasonal data, generating trend data for different periods. Periodic feature normalization is introduced to account for seasonal variations, ensuring that the data input to the model is homogenized. The adaptive learning rate adjustment mechanism specifically uses the Adam optimization algorithm, and its sub-steps include: During training, the learning rate is adaptively adjusted by calculating the first-order moment estimate and the second-order moment estimate; For high-frequency fluctuations, the learning rate is automatically reduced to avoid short-term fluctuations interfering with model convergence; For the long-term trend part, maintain an appropriate learning rate to ensure that the model can fully capture long-term stability; The model is used for seasonal load forecasting in power systems and exhibits high stability in predicting seasonal loads, especially during peak electricity consumption periods in summer and winter. It can capture the upward trend of load in advance, helping power companies optimize power generation plans and reduce operating costs.

2. The method for training and optimizing a large power model based on a variant of stochastic gradient descent according to claim 1, characterized in that, The loss function is further subdivided into two parts: The short-term volatility loss function uses mean squared error as the optimization objective to measure the volatility of the model's prediction accuracy over a short period of time. The long-term trend loss function uses a trend-fit-based loss to ensure that the model accurately tracks the long-term trend, and weight parameters are used to adjust the two parts of the loss. A loss function is established that includes short-term and long-term objectives. By adding a regularization term to the loss function, weights are assigned to the short-term and long-term objectives. The specific steps are as follows: To address short-term data fluctuations in power systems, a short-term objective loss function is defined. This loss function captures high-frequency fluctuations in the data and is calculated using a weighted mean square error. The specific calculation formula is as follows: In the formula, N It is the sample size of short-term data. Indicates the actual value. This represents the model's predicted value. The weights for each sample are used to highlight the impact of high-frequency fluctuations, and are specifically defined as follows: In the formula, It's a timestamp. It is a short-term cycle. It is a hyperparameter that controls the intensity of fluctuations; To capture long-term trends in electricity data, a long-term objective loss function is defined. The long-term loss is calculated using the mean squared error based on trend fitting. The specific calculation formula is as follows: In the formula, M For long-term data, the sample size This represents a function that fits the trend of long-term data. and These represent the actual trend of long-term data and the trend predicted by the model, respectively. To prevent the model from overfitting short-term fluctuations or long-term trends, and to improve the model's robustness, a regularization term is introduced. For model parameters The constraints are applied, and the specific calculation formula is as follows: In the formula, The regularization coefficient is . P The number of model parameters, For timestamps, It is a long-term cycle. The hyperparameter used to control the decay rate of time weights; short-term losses and long-term losses Combined to form the total loss function By introducing weight parameters and The balance between short-term and long-term goals is dynamically adjusted, and the expression for this dynamic adjustment is: In the formula, and The weighting parameter controls the relative importance of short-term and long-term losses. The regularization term is defined.

3. The method for training and optimizing a large power model based on a variant of stochastic gradient descent according to claim 1, characterized in that, Multi-scale sliding window processing is performed in the following way: Use a small time window for short-term fluctuation data and a large time window for long-term data; During each training session, an appropriate window is automatically selected based on different time scales, and the data within the window is weighted to ensure that the impact of the time span can be flexibly adjusted at different training stages of the model.

4. The method for training and optimizing a large power model based on a variant of stochastic gradient descent according to claim 1, characterized in that, Time series regularization further includes the introduction of time decay weights, specifically: Give smaller weights to earlier data in the time series and larger weights to more recent data to ensure that the model is more sensitive to learning new data. By using an exponential decay function to adjust the weights, the influence of historical data is smoothed out, avoiding excessive interference from historical data in future predictions.

5. The method for training and optimizing a large power model based on a variant of stochastic gradient descent according to claim 1, characterized in that, The model evaluation further includes: Cross-validation is used to divide the training data of the model into multiple folds, and the mean error and variance are calculated for each fold to evaluate the stability of the model over different time periods. Based on the evaluation results, the weight parameters, learning rate decay parameters, and regularization coefficients in the loss function are dynamically adjusted to continuously optimize the model's adaptability and prediction accuracy in the power system.

Images