Time series load data enhancement generation method and device based on hidden markov model

Abstract

The application provides a time series load data enhancement generation method and device based on a hidden Markov model, relates to the technical field of power system data processing, divides and time-aligns historical and real-time original time series load data according to a key time period, extracts fluctuation characteristics, obtains historical and real-time fluctuation characteristics, models different fluctuation operation states in the historical fluctuation characteristics by using a hidden Markov model, establishes an online identification model of the fluctuation operation states, takes the real-time fluctuation characteristics as input, outputs the identified fluctuation operation states in real time, adaptively generates corresponding noise to inject into time-aligned real-time load data, and generates enhanced load data. The application adopts a training-free load data synthesis framework based on a hidden Markov model, can automatically and accurately identify periodic characteristics and different fluctuation operation states in load data without depending on a large amount of historical data, and completes data enhancement through adaptive noise injection.

Description

Technical Field

[0001] This invention relates to the field of power system data processing technology, and in particular to a method and apparatus for enhancing and generating time-series load data based on a hidden Markov model. Background Technology

[0002] As the penetration rate of distributed renewable energy sources (such as photovoltaics and wind power) in the power system continues to increase, the net load of the power grid exhibits highly nonlinear and uncertain characteristics. Against this backdrop, the long-term operation optimization, dispatch planning, and risk assessment of the power system increasingly rely on massive amounts of high-quality load data. However, due to factors such as data privacy, trade secrets, and cybersecurity, power companies face strict restrictions on sharing real load data, resulting in a severe "data shortage" problem for traditional planning methods and artificial intelligence models based on historical data.

[0003] To address the challenge of data scarcity, synthetic load data generation technology has become an important research direction. Existing technologies mainly fall into the following categories: First, methods based on generative adversarial networks (GANs), such as using ensemble GANs or combining conditional GANs with kernel density estimation to generate residential load patterns; second, optimization methods based on time-series data generation, such as using a bidirectional LSTM and style transfer reconstruction-based GAN framework to improve the diversity of photovoltaic power generation time-series data; and third, methods based on probabilistic analysis, such as probabilistic decomposition analysis of net load curves for high-penetration distributed photovoltaic scenarios.

[0004] Nevertheless, existing technologies still face significant technical bottlenecks when applied to long-term power grid planning and operation scheduling:

[0005] 1. Data scarcity constraint: It is difficult to obtain high-dimensional long-term load datasets, coupled with strict privacy restrictions, resulting in insufficient training data for generative models, weak generalization ability, and a significant decrease in accuracy when dealing with new load scenarios or extended operating conditions.

[0006] 2. Insufficient ability to identify periodic features: Existing methods are unable to automatically and effectively identify and retain key time periodic features in load data (such as significant differences between weekdays and weekends), resulting in the generated synthetic data failing to accurately reflect the true long-term load evolution pattern.

[0007] 3. Limited applicability of the assessment framework: Current assessment methods rely heavily on statistical distribution comparisons, while ignoring key time dynamic characteristics such as multi-year cycle changes and short-term operational transients, which greatly reduces the applicability of synthetic data in actual power grid decision-making.

[0008] Therefore, there is an urgent need to develop a new method that can accurately capture the temporal characteristics of the load, does not rely on a large amount of training data, and can adaptively generate high-quality synthetic load data. Summary of the Invention

[0009] To address the technical problems of existing synthetic load data generation methods, which heavily rely on large amounts of historical data and are difficult to train, while failing to accurately capture and retain key periodic and fluctuating characteristics in load data, resulting in insufficient applicability of the generated data in long-term power grid planning and operation analysis, this invention provides a time-series load data enhancement generation method and apparatus based on a hidden Markov model. This method can accurately capture load time-series characteristics, does not rely on large amounts of training data, and can adaptively generate high-quality synthetic load data.

[0010] To achieve the above objectives, the present invention provides the following technical solution:

[0011] In a first aspect, this invention provides a method for augmenting and generating time-series load data based on a hidden Markov model, comprising:

[0012] S1. Divide and time-align the acquired historical and real-time raw time-series load data according to key time periods to obtain time-aligned historical load data and real-time load data; S2. Extract fluctuation features from the time-aligned historical and real-time load data to obtain historical fluctuation features and real-time fluctuation features. Use a hidden Markov model to model different fluctuation states in the historical fluctuation features, establish an online identification model for fluctuation states, and output the identified fluctuation states in real time with real-time fluctuation features as input; S3. Adaptively generate corresponding noise based on the identified fluctuation states and inject it into the time-aligned real-time load data to generate enhanced load data.

[0013] The present invention provides a preferred embodiment in its first aspect, wherein S1 involves dividing and aligning the load data according to key time periods, specifically dividing it according to weekdays and weekends, aligning the load data of the historical source year to the calendar of the target year to correct the misalignment of weekdays and weekends caused by the different starting days of each year. Specifically, this includes the following steps: S11. Calculating the weekday offset between the source year and the target year; S12. Converting the weekday offset to an hourly offset, and using modulo operations to cyclically shift the original time-series source load data sequence based on the hourly offset, calculating the load value at time t after time alignment, and obtaining the time-aligned load data sequence as the predicted baseline load data; S13. Establishing a hierarchical autoregressive model for the residuals between the actual observed load data and the predicted baseline load data to eliminate systematic bias, obtaining the bias-eliminated baseline load data. This preferred embodiment effectively eliminates the periodic misalignment problem caused by differences in the distribution of weekdays between years through weekday alignment.

[0014] The present invention provides a preferred embodiment in its first aspect, comprising the following steps: S21. Aggregating the time-aligned load data at different time scales according to their respective time scales to obtain aggregated load data at each time scale, and calculating the logarithmic return of the aggregated load data at each time scale; S22. Extracting volatility index features of the aggregated load data at each time scale based on the corresponding logarithmic return to characterize random behavior; S23. Interpolating the volatility index features of all time scales to a unified time resolution to form a comprehensive volatility feature vector, which presents a clustering pattern corresponding to different volatility states; S24. Modeling the temporal evolution of different volatility states in the comprehensive volatility feature vector using a Gaussian Hidden Markov Model to establish an online identification model for volatility states; S25. After parameter estimation, the online identification model takes real-time volatility features as input and outputs the identified volatility states in real time. This preferred embodiment achieves fine-grained division and temporal dependency capture of load volatility states through multi-scale volatility feature fusion and Hidden Markov Dynamic Modeling.

[0015] In its first aspect, the present invention provides a preferred embodiment, in S22, where the volatility index features include empirical volatility features for quantifying the short-term volatility intensity of load data, and GARCH-type volatility clustering features for simulating the temporal clustering phenomenon of load data volatility. This preferred embodiment employs a method combining empirical volatility and a GARCH model, which not only captures the dramatic volatility characteristics of load sequences in a short period of time, but also characterizes the persistence and clustering effect of volatility over time, effectively improving the ability to describe complex load behavior.

[0016] In a first aspect, the present invention provides a preferred embodiment, in S2, where the use of a Hidden Markov Model to model different wave states in historical wave characteristics specifically utilizes a model with... A Gaussian Hidden Markov Model with multiple latent states, each representing a unique fluctuating state with stochastic characteristics, is used. The temporal evolution of different fluctuating states is modeled using a first-order Markov chain. Each Gaussian Hidden Markov Model consists of a state transition model and an observation model. This preferred scheme employs multiple Gaussian Hidden Markov Models with latent states, enabling refined characterization of various load fluctuation patterns and accurate tracking of dynamic evolution paths.

[0017] In a first aspect, this invention provides a preferred embodiment where, in step S25, the model parameters are estimated using an expectation-maximization algorithm with diagonal covariance constraints; and a forward algorithm is used to predict and identify the fluctuating operating state. This preferred embodiment employs an expectation-maximization algorithm with diagonal covariance constraints for parameter estimation, which effectively avoids the covariance matrix singularity problem and improves the stability and convergence speed of model training. Simultaneously, using the forward algorithm to predict and identify the fluctuating operating state enables efficient decoding of the implicit fluctuating states in historical load data and accurate tracking of their temporal evolution path.

[0018] The present invention provides a preferred embodiment in its first aspect, wherein S3 specifically includes: S31. Calculating the mean fluctuation level and adaptive scaling factor for each identified fluctuation operating state; S32. Generating adaptive noise proportional to the fluctuation characteristics of the current fluctuation operating state based on the mean fluctuation level and adaptive scaling factor; S33. Injecting the adaptive noise into the baseline load data, synthesizing it with the baseline load data, and generating enhanced load data through upper and lower limit pruning. This preferred embodiment ensures that the synthesized data is within the physical meaning and operating boundary range through upper and lower limit pruning, avoiding the generation of abnormal or infeasible scenarios; at the same time, it preserves the statistical characteristics and dynamic structure of the original fluctuation pattern, improving data diversity and representativeness.

[0019] In a second aspect, this invention provides a method for enhancing and generating time-series load data based on a hidden Markov model, comprising: S10. Dividing and aligning the acquired historical and real-time raw time-series load data according to key time periods to obtain time-aligned historical load data and real-time load data; synchronizing the acquired historical and real-time external condition data with the time-aligned historical load data and real-time load data respectively to obtain time-synchronized historical external condition data and real-time external condition data; S20. Extracting fluctuation features from the time-aligned historical load data and real-time load data to obtain historical fluctuation features and real-time fluctuation features, and performing time synchronization on the time-synchronized historical load data and real-time load data. Historical and real-time external condition data are feature-encoded to obtain historical and real-time external condition features. A conditional hidden Markov model is used to model different fluctuation states and historical external condition features within the historical fluctuation characteristics, establishing an improved online identification model for fluctuation states. This model takes real-time fluctuation features and real-time external condition features as input and outputs the identified fluctuation states in real time. Specifically, the conditional hidden Markov model is obtained by introducing external condition vectors and weights into the observation model of the hidden Markov model. S30. Based on the identified fluctuation states, corresponding noise is adaptively generated and injected into the time-aligned real-time load data to generate enhanced load data.

[0020] This solution introduces a multi-source information fusion and conditional generation framework, embedding external environmental factors as constraint variables into the observation model of the Hidden Markov Model. This enables the generated synthetic data to proactively respond to changes in external driving factors, significantly enhancing its realism, causality, and scenario adaptability in extreme scenario simulation, policy intervention assessment, and long-term planning analysis. It effectively improves the practical value of synthetic data in power system risk simulation and decision support.

[0021] In a third aspect, this invention provides a time-series load data enhancement and generation device based on a Hidden Markov Model (HMM), used to execute the method proposed in the first aspect, comprising: a time alignment module, used to divide and time-align the acquired historical and real-time raw time-series load data according to key time periods to obtain time-aligned historical load data and real-time load data; a real-time identification module, used to extract fluctuation features from the time-aligned historical load data and real-time load data to obtain historical fluctuation features and real-time fluctuation features, and to model different fluctuation states in the historical fluctuation features using an HMM to establish an online identification model of the fluctuation states, taking real-time fluctuation features as input and outputting the identified fluctuation states in real time; and a data enhancement module, used to adaptively generate corresponding noise based on the identified fluctuation states and inject it into the time-aligned real-time load data to generate enhanced load data.

[0022] Compared with the prior art, the present invention has the following beneficial technical effects:

[0023] This invention employs a training-free load data synthesis framework based on Hidden Markov Models (HMMs). It can automatically and accurately identify periodic characteristics (such as weekday and weekend patterns) and different fluctuating operating states in load data without relying on large amounts of historical data. Furthermore, it generates high-quality, highly applicable synthetic load data through adaptive noise injection, achieving efficient data augmentation. This method not only significantly reduces the data volume requirements, but more importantly, the generated high-quality synthetic data accurately preserves the temporal characteristics and random behavior of the load, thus demonstrating excellent applicability and reliability in practical applications such as long-term power grid planning and operation analysis. Attached Figure Description

[0024] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on the provided drawings without creative effort.

[0025] Figure 1This is a flowchart of the time-series load data augmentation and generation method based on hidden Markov models provided in Embodiment 1 of the present invention;

[0026] Figure 2 This is a block diagram of the time-series load data enhancement and generation device based on a hidden Markov model provided in Embodiment 1 of the present invention;

[0027] Figure 3 The diagram shows the effect of using a hierarchical autoregressive model for weekday-weekend cycle alignment in the time-series load data augmentation generation method based on a hidden Markov model provided in Embodiment 2 of the present invention.

[0028] Figure 4 The diagram shows the effect of using the Hidden Markov Model (HMM) to detect load volatility in the time-series load data augmentation generation method based on the Hidden Markov Model provided in Embodiment 2 of the present invention.

[0029] Figure 5 This is a radar chart showing the feature comparison of different volatility ranges in the time-series load data enhancement generation method based on hidden Markov models provided in Embodiment 2 of the present invention. Detailed Implementation

[0030] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0031] Example 1: Please refer to Figure 1 In one optional implementation, a method for augmenting and generating time-series load data based on a hidden Markov model is provided, which is mainly implemented through the following steps:

[0032] S1. Divide and align the acquired historical and real-time raw time-series load data according to key time periods to obtain time-aligned historical load data and real-time load data;

[0033] S2. Extract fluctuation features from time-aligned historical load data and real-time load data to obtain historical fluctuation features and real-time fluctuation features. Use a hidden Markov model to model different fluctuation states in the historical fluctuation features, establish an online identification model for fluctuation states, and output the identified fluctuation states in real time with real-time fluctuation features as input.

[0034] S3. Based on the identified fluctuating operating state, the corresponding noise is adaptively generated and then injected into the time-aligned real-time load data to generate enhanced load data.

[0035] Please refer to Figure 2 Corresponding to this embodiment, a time-series load data augmentation and generation device based on a hidden Markov model is provided for executing the data augmentation and generation method provided in this embodiment. It mainly consists of the following parts:

[0036] Time alignment module 1 is used to divide and align the acquired historical and real-time raw time-series load data according to key time periods to obtain time-aligned historical load data and real-time load data.

[0037] The real-time identification module 2 is used to extract fluctuation features from time-aligned historical load data and real-time load data, obtain historical fluctuation features and real-time fluctuation features, use a hidden Markov model to model different fluctuation operating states in the historical fluctuation features, establish an online identification model of fluctuation operating states, and output the identified fluctuation operating states in real time with real-time fluctuation features as input.

[0038] Data augmentation module 3 is used to generate augmented load data by adaptively generating corresponding noise based on the identified fluctuating operating state and then injecting it into the time-aligned real-time load data.

[0039] Example 1 employs a training-free load data synthesis framework based on Hidden Markov Models (HMMs). This framework can automatically and accurately identify periodic characteristics (such as weekday and weekend patterns) and different fluctuating operating states in load data without relying on large amounts of historical data. Furthermore, it generates high-quality, highly applicable synthetic load data through adaptive noise injection, achieving efficient data augmentation. This addresses the technical problems of existing synthetic load data generation methods, which heavily rely on large amounts of historical data and face training difficulties. These methods also struggle to accurately capture and retain key periodic characteristics and fluctuating features such as weekday and weekend patterns in load data, resulting in insufficient applicability of the generated data in long-term power grid planning and operation analysis. This implementation not only significantly reduces the data volume requirements but, more importantly, generates high-quality synthetic data that accurately preserves the temporal characteristics and random behavior of the load, thus demonstrating excellent applicability and reliability in practical applications such as long-term power grid planning and operation analysis.

[0040] Example 2: Based on the above implementation method, a more preferred and detailed implementation method is proposed, as follows:

[0041] This implementation provides a method for augmenting and generating time-series load data based on a hidden Markov model, which is mainly achieved through the following steps:

[0042] S1. Divide and align the acquired historical and real-time raw time-series load data according to key time periods to obtain time-aligned historical and real-time load data. Step S1 involves dividing and aligning the data according to key time periods, specifically by weekdays and weekends. The load data from the historical source year is aligned to the calendar of the target year to correct the misalignment between weekdays and weekends caused by the different starting days of each year. This is further achieved through the following steps:

[0043] S11. Calculate the weekday offset between the source year and the target year;

[0044] S12. Convert the weekday offset to the hourly offset. Based on the hourly offset, perform a cyclic shift on the original time-series source load data sequence through modulo operation to calculate the load value at time t after time alignment. Obtain the time-aligned load data sequence as the reference load data for prediction.

[0045] S13. Establish a hierarchical autoregressive model for the residuals between the actual observed load data and the predicted baseline load data to eliminate systematic bias and obtain the baseline load data after bias elimination.

[0046] More specifically, the above process is a hierarchical autoregressive weekday alignment process. This process eliminates the time misalignment between the source and target years by calculating systematic weekday shifts, thereby maintaining accurate weekday and weekend load periodicity. Specifically, let... Represent the weekday index (0–6, where 0 represents Monday and 6 represents Sunday) for January 1st of the target year. This represents the weekday index for January 1st of the historical source year. As shown in formulas (1), (2), and (3):

[0047] (1);

[0048] (2);

[0049] (3);

[0050] The mod function is used for loop boundary handling, which reverses out-of-range values ​​according to a predetermined period (7 days or the length of the source data sequence) to ensure time continuity. This indicates the workday offset calculated on a daily basis. This indicates the time offset calculated in hours; It is the aligned load value at time t, which is also the predicted baseline load value; It is the original source load data sequence; This represents the total duration (in hours) of the source sequence (original load data sequence); t represents the time scale in hours, with a value range of [0, ..., ... -1]. Through the above alignment process, it is ensured that each working day hour of the target year accurately matches the corresponding working day hours of the historical source year. After clarifying the mathematical expression of the alignment problem, the hierarchical autoregressive model introduces a multi-stage lag structure to systematically preserve the temporal dependencies at different time scales, which can achieve robust alignment while maintaining the stochastic characteristics of the load, as shown in formulas (4) and (5):

[0051] (4);

[0052] (5);

[0053] The meanings of the parameters in the formula are as follows: is the predicted residual value at time t; l is the hierarchical index, representing the time step of different periods; The length of the backtracking window represents the number of historical moments involved in the prediction calculation; , which is the weighting coefficient of the l-th layer, used to balance the contribution of different historical periods; The characteristic coefficients of the j-th historical period in the l-th layer; The historical residual value at time tj; This is the intercept term for the j-th historical period; Let be the random error term at time t, which follows a normal distribution N(0,σ²). In the model, through residuals Its function is indirectly reflected. Specifically, residuals Defined as actual observed load value Compared with the baseline forecast load value The deviation between them, i.e. By establishing a hierarchical regression model on the residual sequence, the systematic bias that still exists after workday alignment can be effectively eliminated. This embodiment, through workday alignment, effectively eliminates the periodic misalignment problem caused by differences in workday distribution across years, significantly improving the temporal consistency and realism of the synthetic data in terms of long-term load characteristics. This method not only preserves the core fluctuation patterns of the original load data but also achieves high-fidelity data augmentation with a small number of historical samples, providing reliable data support for power grid planning.

[0054] S2. Extract fluctuation features from time-aligned historical load data and real-time load data to obtain historical fluctuation features and real-time fluctuation features. Utilize a Hidden Markov Model to model different fluctuation states within the historical fluctuation features, establishing an online identification model for fluctuation states. Using real-time fluctuation features as input, the model outputs the identified fluctuation states in real-time. Step S2 is further implemented through the following steps:

[0055] S21. Aggregate the time-aligned load data at each time scale according to the corresponding time scale to obtain aggregated load data at each time scale, and calculate the logarithmic rate of return of the aggregated load data at each time scale.

[0056] S22. Combine the corresponding logarithmic returns to extract volatility index features of aggregated load data for each time scale to characterize stochastic behavior. In a more preferred embodiment, for step S22, the volatility index features include empirical volatility features for quantifying the short-term volatility intensity of load data, and GARCH-type volatility clustering features for simulating the temporal clustering phenomenon of load data volatility. This embodiment adopts a method combining empirical volatility and GARCH models, which not only captures the drastic volatility characteristics of load sequences in a short period of time, but also characterizes the persistence and clustering effect of volatility over time, effectively improving the ability to describe complex load behavior. By introducing GARCH-type volatility clustering features, the model can reflect the time-varying variance characteristics and memory effect of load volatility, further enhancing the ability to characterize extreme load events. Empirical volatility extracts instantaneous volatility information from high-frequency data, compensating for the loss of detail caused by low-frequency aggregation. The combination of the two makes the multi-scale volatility features have stronger representational consistency and dynamic adaptability, and can stably identify typical volatility patterns under different load operating environments, providing a solid foundation for subsequent state transition analysis and scenario generation.

[0057] S23. Interpolate the volatility index features of all time scales to a unified time resolution to form a comprehensive volatility feature vector. The comprehensive volatility feature vector presents a clustering pattern corresponding to different volatility states.

[0058] S24. The temporal evolution of different wave states in the comprehensive wave characteristic vector is modeled using a Gaussian Hidden Markov Model, and an online identification model for wave states is established.

[0059] S25. After parameter estimation, the online identification model takes real-time fluctuation characteristics as input and outputs the identified fluctuation operating state in real time. In a more preferred embodiment, for step S25, the model parameters are estimated using an expectation-maximization algorithm with diagonal covariance constraints; the fluctuation operating state is predicted and identified using a forward algorithm. This preferred embodiment uses an expectation-maximization algorithm with diagonal covariance constraints for parameter estimation, which can effectively avoid the covariance matrix singularity problem and improve the stability and convergence speed of model training; at the same time, it retains the independence assumption of each dimension of features, reduces parameter redundancy, and enhances the accuracy of characterizing the spatial structure of fluctuation features. This method improves the accuracy of the hidden Markov model in identifying load operating states while ensuring computational efficiency, especially showing stronger robustness in the face of noise interference and data sparsity scenarios. At the same time, using the forward algorithm to predict and identify the fluctuation operating state can achieve efficient decoding of the hidden fluctuation state in historical load data and accurately track its temporal evolution path.

[0060] This embodiment achieves fine-grained segmentation and temporal dependency capture of load fluctuation states through multi-scale fluctuation feature fusion and hidden Markov dynamic modeling, significantly improving the accuracy and robustness of operational state identification. By aggregating time-aligned load data at different time scales, i.e., statistically summarizing the same load sequence under different time window lengths, this multi-scale aggregation process can extract fluctuation features at different time granularities from the same basic data. Combined with logarithmic return rate, the volatility index features of the aggregated load data at each time scale are extracted, effectively characterizing the random fluctuation characteristics of the load sequence and avoiding information loss caused by single-scale analysis. At the same time, the volatility index features of all time scales are interpolated to a unified time resolution to form a comprehensive fluctuation feature vector, realizing the collaborative expression of multi-source heterogeneous data and enhancing the integrity and representativeness of the feature space.

[0061] In a more preferred embodiment, for step S2, a hidden Markov model is used to model different fluctuation states in historical fluctuation characteristics, specifically utilizing a model with... A Gaussian Hidden Markov Model with 12 potential states, each representing a unique fluctuating state with random characteristics. The time evolution of different fluctuating states is modeled by a first-order Markov chain. The Gaussian Hidden Markov Model consists of a state transition model and an observation model.

[0062] More specifically, load data generation needs to simultaneously capture the inherent systemic consumption patterns and random fluctuations in real electricity demand. To address this challenge, this preferred embodiment proposes a hierarchical hidden Markov model framework that can detect fluctuation states online and generate adaptive noise based on different states. This method can achieve real-time load data synthesis without requiring a large amount of historical training data. The method first extracts fluctuation features from load data at multiple time scales to capture unique fluctuation patterns within different time ranges. For each scale s∈{1, 24, 168}, the logarithmic return on aggregated load data is calculated as shown in Equation (6):

[0063] (6);

[0064] Wherein, s=1 represents the hourly scale (hourly analysis), s=24 represents the daily scale (aggregation every 24 hours), and s=168 represents the weekly scale (aggregation every 7 days, i.e., 168 hours). Through multi-scale analysis, the short-term fluctuations (hourly), intraday periodicity (daily), and weekly periodicity (weekly) characteristics of the load can be captured simultaneously.

[0065] In the formula, This represents the load data aggregated by scale s; Represents the time index (integer), with a value range of 1000. ∈[1, N], where N is the total number of data points at this scale. Subsequently, two main volatility indicators are extracted to characterize randomness: empirical volatility and GARCH-type volatility clustering (i.e., GARCH-type volatility), and their calculation formulas are shown in equations (7) and (8):

[0066] (7);

[0067] (8);

[0068] in, Representing scale Next Realized volatility over a given period. Representing scale Next GARCH-type volatility over a given period; w represents the adaptively selected sliding window length for each scale; and the parameters in the GARCH model. and Used to characterize the persistence of fluctuations, among which , ; t represents the current time index (integer); Indicates the first s under scale s The square of the logarithmic returns for each time period is calculated using formula (6). The RV component quantifies the intensity of short-term volatility, while the GARCH component simulates the temporal clustering of volatility commonly found in load data. Subsequently, these features are interpolated to a uniform time resolution to form a comprehensive volatility feature vector. :

[0069] (9);

[0070] Where T is the transpose sign. This is achieved by analyzing the fluctuation characteristic vector of historical load data. Cluster analysis reveals distinct clustering patterns among these features, with different clusters corresponding to different fluctuating operating states of the power load. To systematically identify these fluctuating operating states, this embodiment employs a Hidden Markov Model (HMM) with K potential states, each representing a fluctuating operating state with unique stochastic characteristics. In practical applications, the value of K is typically set to 3-5 to balance model complexity and descriptive power. Through the HMM's state identification mechanism, each moment of historical load data can be automatically assigned to the most probable fluctuating operating state category. This most probable fluctuating operating state category is the fluctuating operating state predicted and identified using the forward algorithm. In the Hidden Markov Model (HMM) framework described in this invention, the "most probable fluctuating operating state" refers to the state that maximizes the posterior probability of the hidden state at the current moment, given current and historical observation data (i.e., the fluctuating feature vector sequence). This is a decision criterion based on Maximum aPosteriori (MAP). Specifically, in this embodiment, the "most probable fluctuating operating state" is determined through forward probability... It is quantified by maximizing, that is, choosing the option that maximizes the forward probability. The maximum state k.

[0071] Hidden State Sequence According to the transition probability matrix Its definition is shown in formula (10):

[0072] (10);

[0073] in, This represents the probability of transitioning to state j at time t, given that the state is i at time t-1. The observed fluctuation characteristics are generated by the Gaussian distribution corresponding to each running state. This HMM framework is formally defined by two key components: the state transition model (Equation (11)) and the observation model (Equation (12)).

[0074] (11);

[0075] (12);

[0076] in, and Let represent the mean vector and covariance matrix of the k-th running state, respectively. The model parameters are estimated using the Expectation-Maximization (EM) algorithm with diagonal covariance constraints to ensure computational stability and prevent overfitting. The hidden state (potential operating state) represents time t, and the value of k is a discrete set {1, 2, ..., K}, indicating that the current load is in any one of K predefined fluctuating operating states; The hidden state represents time t-1, and the potential running state represents the previous time. This represents the transition probability vector indexed by S[t-1]. Forward probability The recursive updates are shown in equations (13) and (14):

[0077] (13);

[0078] (14);

[0079] Indicates that in a given observation sequence Under the condition that time t is in running state k and has been observed arrive The joint probability of all data; The forward probability of being in state j at the previous time (t-1) is then used to determine the predicted fluctuating state at time t using the following formula:

[0080] (15);

[0081] The above mechanism enables the classification of online fluctuating operating states without requiring a large amount of historical load data or prior knowledge.

[0082] S3. Based on the identified fluctuating operating state, adaptively generate corresponding noise and inject it into the time-aligned real-time load data to generate enhanced load data. Step S3 is specifically implemented through the following steps:

[0083] S31. Calculate the mean volatility level and adaptive scaling factor for each identified volatility state;

[0084] S32. Generate adaptive noise that is proportional to the fluctuation characteristics of the current fluctuation state based on the mean fluctuation level and adaptive scaling factor;

[0085] S33. Adaptive noise is injected into the baseline load data and synthesized with it. Enhanced load data is then generated through upper and lower limit pruning. Upper and lower limit pruning ensures that the synthesized data is within the physical meaning and operational boundaries, avoiding the generation of abnormal or infeasible scenarios; at the same time, it preserves the statistical characteristics and dynamic structure of the original fluctuation pattern, improving data diversity and representativeness.

[0086] More specifically, once a fluctuating operating state is identified, adaptive noise is generated based on its detected random characteristics. This adaptive noise is injected into the baseline load data, added to it, and then cropped through upper and lower limits to generate enhanced load data. Each operating state k is composed of the mean fluctuation level calculated from the data. Characterized by its unique noise properties:

[0087] (16);

[0088] in, Let be the noise intensity at time t. For numerical stability parameters, Used to prevent numerical instability; It is a scaling factor related to the fluctuation state, used to adaptively adjust the noise intensity according to the detected fluctuation state. This is the baseline load value (scalar) at time t, which is the load value obtained in step S1 after weekday alignment and hierarchical regression processing. For: the baseline load value at time t The enhanced load data generated after injecting adaptive noise synthesis and then clipping through upper and lower limits is shown in Equation (17):

[0089] (17);

[0090] in, It provides random perturbations under specific operating conditions, while Clip represents the pruning operation: limiting the synthetic data to a reasonable operating range. Coefficients a and b define the lower and upper limits relative to the original load, respectively, controlling the allowable deviation.

[0091] This adaptive operating mechanism automatically adjusts the noise intensity based on detected fluctuations, ensuring that disturbances are amplified during periods of high volatility while minimizing disturbances during stable operation. This completes the synthesis of real load data that preserves the potential stochastic mechanisms and operating characteristics of actual electricity consumption patterns.

[0092] Example 2 preprocesses the source data through a hierarchical autoregressive workday alignment step. The aim is to correct the misalignment of workdays and weekends across different years, establishing a load sequence with a unified time base and accurate periodic characteristics for subsequent analysis. Next, an online state extraction step based on a Hidden Markov Model (HMM) is used to analyze the time-aligned data. By extracting fluctuation characteristics across multiple time scales, different fluctuating operating states of the load (such as high fluctuation or stability) are automatically identified, thus patterning and state-basedizing the dynamic behavior of the load. Finally, based on the identified states, an adaptive data augmentation step is performed. The key is to adaptively adjust the intensity and characteristics of noise according to the current specific fluctuation state and inject it into the data, ultimately generating high-quality synthetic data that retains the core patterns of the true load while possessing necessary randomness.

[0093] To verify the effectiveness of the preferred embodiment (Example 2) of the present invention, an experiment was conducted. The experiment used a real substation load dataset, all of which was annual data at hourly resolution. The data came from a provincial power grid in China and contained 325 load curves, each with 8,784 hours of observations (366 days × 24 hours, leap year). The following are the example test results:

[0094] To quantitatively evaluate the effectiveness of the hierarchical autoregressive model proposed in this embodiment, autocorrelation analysis was subsequently performed on the data. The autoregressive factor was evaluated using the derived autoregressive coefficients, and a 24-hour filler was applied to the load data of domestic provincial power grids. The ACF (Autocorrelation Function) value was 0.8745, representing the autocorrelation coefficient of the synthesized data with a 24-hour lag. This value closely approximates the autocorrelation characteristics of the original data, verifying that the synthesized data maintains the time-series dependence of the original load. Detailed ACF analysis results can be found in [link to ACF analysis]. Figure 3 . Figure 3 The blue curve represents historical load data, and the orange curve represents the 24-hour forecast results from the hierarchical autoregressive model. The horizontal axis (Index) represents the time index (hours), and the vertical axis (Loard, MW) represents the load value (megawatts). As can be seen from the graph, the forecast curve closely matches the historical data, successfully capturing the diurnal cyclical fluctuations of the load and validating the model's effectiveness. The autocorrelation function (ACF) value of the forecast data is 0.8745, indicating that the synthetic data retains the time-series dependence and periodicity of the original load. For the data that has already been aligned to workdays, the HMM fluctuation operating mode was automatically identified based on volatility feature extraction. This method extracts several statistical indicators that help distinguish operating modes and facilitate interpretation and analysis.

[0095] Figure 4The figure shows the load data distribution for seven fluctuating operating states (Regime 1 to Regime 7, i.e., operating states 1 to 7) identified by the Hidden Markov Model (HMM). The horizontal axis represents Time (hours), and the vertical axis represents Load (MW). As can be seen from the figure, different colors represent different fluctuating operating states, and each state exhibits significant differences in load level and fluctuation characteristics. Figure 5 The differences between the seven operating states (Regime 1 to Regime 7, i.e., operating states 1 to 7) across six statistical characteristic dimensions were compared using radar charts. In the chart, the six statistical dimensions Mean Load (MW), Variance (MW²), Coeff. of Variation, Peak Ratio, Load Factor, and Ramping Rate (MW / h) are the average load (MW), variance (MW²), coefficient of variation, peak load factor, and ramping rate (MW / h), respectively. It can be seen from the chart that: (1) Operating state 4 (red) has the largest values ​​in the dimensions of average load, variance, and coefficient of variation, showing high load and high volatility characteristics; (2) Operating states 1 and 7 (dark purple) have smaller values ​​in each dimension, corresponding to low volatility and stable operating states; (3) Operating state 3 (green) shows outstanding performance in the load factor dimension, reflecting the characteristic of high load utilization; (4) The radar charts of different operating states have significantly different shapes, verifying that the HMM model effectively identifies operating states with different statistical characteristics, providing a reliable state basis for adaptive noise generation.

[0096] Figure 5 The interpretability of these statistical features demonstrates the effectiveness of the runtime identification method in this embodiment. Table 1 lists the scaled empirical volatility RV (×10²) and GARCH volatility (×10²) values ​​used in the identification process.

[0097] Table 1: Volatility Results for Different Volatility States

[0098] ;

[0099] As can be seen from Table 1: (1) there are significant differences in RV values ​​for different operating states, verifying the effective classification ability of the HMM model; (2) the RV values ​​for operating states 1-3 (low-wave dynamics) are relatively small (7.64-8.04), while the RV values ​​for operating states 4-7 (high-wave dynamics) are relatively large (8.09-9.07), which conforms to the expected stratification of fluctuation characteristics; (3) these statistical differences indicate that the synthetic data successfully reproduced the fluctuation characteristics of different operating scenarios. The parameters controlling the noise injection intensity during data augmentation can be adjusted according to the required level of data diversity. In this embodiment, these parameters, namely the parameter pairs in formulas (16) and (17), are respectively set as follows: a=0.9, b=1.1. Here, a and b define the lower and upper limits relative to the original load, respectively, controlling the allowable deviation range. To verify the effectiveness of the time-series load data augmentation generation method based on the Hidden Markov Model, a comprehensive comparison was conducted with four benchmark schemes: two deep generation models: GAN and VAE, and two model-free methods: Bootstrap resampling (BR) and Fourier synthesis (FS). As can be seen from the comparison results in Table 2, among the five evaluation indicators, the proposed method significantly outperforms the comparative methods in three key indicators: weekday-weekend load difference, annual seasonal dynamics, and multi-year cycle pattern (0.889, 0.982, and 0.989, respectively), while deep generation models such as GAN and VAE perform poorly in these indicators (GAN: 0.431, 0.095, 0.434; VAE: 0.331, 0.096, 0.516). The analysis results show that, limited by the amount of data, deep generative models are significantly insufficient in overall long-term load generation, struggling to capture diverse load characteristics and prone to pattern collapse. Specifically: (1) GAN and VAE scored only 0.431 and 0.331 on the weekday-weekend load difference index, respectively, far lower than the 0.889 of the proposed method, indicating their inability to effectively preserve weekday pattern characteristics; (2) On the annual seasonal dynamic index, GAN and VAE scored only 0.095 and 0.096, respectively, indicating their severely insufficient ability to capture seasonal characteristics; (3) Deep generative models are prone to pattern collapse during training, resulting in insufficient diversity of generated data. These comparative results validate the necessity and effectiveness of developing a model-free architecture (i.e., the HMM-based method proposed in this paper).

[0100] Table 2: Similarity Comparison Experiment:

[0101] ;

[0102] Comparative analysis shows that although Bootstrap resampling performs slightly better in preserving weekday-weekend patterns and daily load curve characteristics due to directly partitioning and reorganizing the original data, the method proposed in this invention consistently demonstrates superior performance in all other evaluation metrics.

[0103] Example 3: Based on Example 2, a more preferred implementation method is given below, which will be described in detail below:

[0104] Given that the current model presented in Example 2 primarily relies on the internal statistical characteristics of historical load data, such as volatility, it only focuses on the historical load data itself and cannot predict or respond to external driving factors that cause load changes. Therefore, it cannot perceive and model abnormal load changes caused by external shocks such as extreme weather, sudden events (such as large-scale events), or sudden changes in electricity pricing policies. In these scenarios, the generated synthetic data may not reflect the true extreme operating conditions, thus affecting the accuracy of risk assessment. To overcome this deficiency, this implementation proposes an iterative upgrade scheme, introducing a multi-source information fusion and conditional generation framework to upgrade the model from simple time-series generation to conditional time-series generation. This provides the generation process with "context" or "conditions," allowing the model to learn what behavior the load is most likely to exhibit under a given external environment. This can fundamentally improve the causality and scenario specificity of the generated data. Specifically, as follows:

[0105] S10. Divide and time-align the acquired historical and real-time raw time-series load data according to key time periods to obtain time-aligned historical load data and real-time load data; synchronize the acquired historical and real-time external condition data with the time-aligned historical load data and real-time load data respectively to obtain time-synchronized historical external condition data and real-time external condition data.

[0106] S20. Extract fluctuation features from time-aligned historical load data and real-time load data to obtain historical fluctuation features and real-time fluctuation features. Perform feature encoding on time-synchronized historical external condition data and real-time external condition data to obtain historical external condition features and real-time external condition features. Use a conditional hidden Markov model to model different fluctuation states and historical external condition features in the historical fluctuation features, and establish an improved online identification model for fluctuation states. Take real-time fluctuation features and real-time external condition features as input, and output the identified fluctuation states in real time. Specifically, the conditional hidden Markov model is obtained by introducing external condition vectors and weights into the observation model of the hidden Markov model.

[0107] S30. Based on the identified fluctuating operating state, the corresponding noise is adaptively generated and injected into the time-aligned real-time load data to generate enhanced load data.

[0108] More specifically:

[0109] First, acquire and input external variables: construct an external condition vector that is synchronized with the load data time. Regarding the selection of data sources, meteorological data (temperature, humidity, wind speed, and solar radiation intensity, which are key factors affecting heating and cooling loads and distributed photovoltaic output), calendar information (holiday markers, seasonal markers, whether it is a workday with adjusted holidays, etc.), economic indicators, and even real-time electricity price data (such as time-of-use pricing signals, demand response event trigger signals, etc.) can be used as additional observation variables. Based on these external observation variables, feature construction is performed, holidays are coded, and statutory holidays, traditional festivals, long holidays, and short holidays are distinguished; all features are normalized to form a unified conditional feature vector. It is input into the model along with the comprehensive fluctuation feature vector.

[0110] Secondly, a conditional hidden Markov model is constructed: the observation model of the HMM is modified so that it depends not only on the hidden state but also on these external conditions. As an example, equation (12) can be improved to (18):

[0111] (18);

[0112] in W is the external condition vector, and W is the weight matrix, which needs to be updated synchronously during parameter estimation.

[0113] This embodiment introduces a multi-source information fusion and conditional generation framework, embedding external environmental factors (such as meteorological conditions, holiday information, and policy signals) as constraint variables into the observation model of the Hidden Markov Model. This transforms the load data generation process from isolated time-series inference to context-aware modeling. This framework enables the generated synthetic data to proactively respond to changes in external driving factors, significantly enhancing its realism, causality, and scenario adaptability in extreme scenario simulations, policy intervention assessments, and long-term planning analyses. It effectively improves the practical value of synthetic data in power system risk simulation and decision support.

[0114] Those skilled in the art will understand that all or part of the processes in the above embodiments can be implemented by a computer program instructing related hardware. The program can be stored in a computer-readable storage medium, and when executed, it can include the processes of the embodiments of the above methods. The storage medium can be a magnetic disk, optical disk, read-only memory (ROM), or random access memory (RAM), etc.

[0115] The technical features of the above embodiments can be combined arbitrarily. For the sake of brevity, not all possible combinations of the technical features in the above embodiments are described. However, as long as the combination of these technical features does not contradict each other, it should be considered within the scope of this specification. Furthermore, the above embodiments only illustrate several implementation methods of the present invention, and their descriptions are relatively specific and detailed, but they should not be construed as limiting the scope of the present invention. For those skilled in the art, several modifications and improvements can be made without departing from the concept of the present invention, and these all fall within the protection scope of the present invention.

Claims

1. A method for augmenting and generating time-series load data based on hidden Markov models, characterized in that, include: S1. Divide and align the acquired historical and real-time raw time-series load data according to key time periods to obtain time-aligned historical load data and real-time load data; S2. Extract fluctuation features from time-aligned historical load data and real-time load data to obtain historical fluctuation features and real-time fluctuation features. Use a hidden Markov model to model different fluctuation states in the historical fluctuation features, establish an online identification model for fluctuation states, and output the identified fluctuation states in real time with real-time fluctuation features as input. S3. Based on the identified fluctuating operating state, the corresponding noise is adaptively generated and injected into the time-aligned real-time load data to generate enhanced load data; S2 includes: S21. Aggregate the time-aligned load data at each time scale according to the corresponding time scale to obtain aggregated load data at each time scale, and calculate the logarithmic rate of return of the aggregated load data at each time scale. S22. Combine the corresponding logarithmic returns to extract the volatility index features of the aggregated load data for each time scale, in order to characterize stochastic behavior; S23. Interpolate the volatility index features of all time scales to a uniform time resolution to form a comprehensive volatility feature vector. The comprehensive volatility feature vector presents a clustering pattern corresponding to different volatility states. S24. Use a Gaussian Hidden Markov Model to model the time evolution of different wave states in the comprehensive wave feature vector, and establish an online identification model for wave states; S25. After parameter estimation, the online identification model takes real-time fluctuation characteristics as input and outputs the identified fluctuation operation state in real time. S3 includes: S31. Calculate the mean volatility level and adaptive scaling factor for each identified volatility state; S32. Generate adaptive noise that is proportional to the fluctuation characteristics of the current fluctuation state based on the mean fluctuation level and adaptive scaling factor; S33. Inject adaptive noise into the baseline load data, synthesize it with the baseline load data, and generate enhanced load data by upper and lower limit clipping.

2. The method for augmenting and generating time-series load data based on a hidden Markov model according to claim 1, characterized in that, S1 divides and aligns data according to key time periods, specifically weekdays and weekends. It aligns the load data from the historical source year to the calendar of the target year to correct the misalignment of weekdays and weekends caused by the different starting days of each year. The specific steps include the following: S11. Calculate the weekday offset between the source year and the target year; S12. Convert the weekday offset to the hourly offset. Based on the hourly offset, perform a cyclic shift on the original time-series source load data sequence through modulo operation to calculate the load value at time t after time alignment. Obtain the time-aligned load data sequence as the reference load data for prediction. S13. Establish a hierarchical autoregressive model for the residuals between the actual observed load data and the predicted baseline load data to eliminate systematic bias and obtain the baseline load data after bias elimination.

3. The method for augmenting and generating time-series load data based on a hidden Markov model according to claim 1, characterized in that, In S22, volatility index features include empirical volatility features used to quantify the short-term volatility intensity of load data, and GARCH-type volatility clustering features used to simulate the temporal clustering of load data.

4. The method for augmenting and generating time-series load data based on a hidden Markov model according to claim 1, characterized in that, In S2, the use of a Hidden Markov Model to model different wave states in historical wave characteristics specifically utilizes a model with... K A Gaussian Hidden Markov Model with 12 potential states, each representing a unique fluctuating state with random characteristics. The time evolution of different fluctuating states is modeled by a first-order Markov chain. The Gaussian Hidden Markov Model consists of a state transition model and an observation model.

5. The method for augmenting and generating time-series load data based on a hidden Markov model according to claim 4, characterized in that, In S25, the model parameters are estimated using the expectation-maximization algorithm with diagonal covariance constraints; the forward algorithm is used to predict and identify the fluctuating operating state.

6. A method for augmenting and generating time-series load data based on hidden Markov models, characterized in that, include: S10. Divide and time-align the acquired historical and real-time raw time-series load data according to key time periods to obtain time-aligned historical load data and real-time load data; synchronize the acquired historical and real-time external condition data with the time-aligned historical load data and real-time load data respectively to obtain time-synchronized historical external condition data and real-time external condition data. S20. Extract fluctuation features from time-aligned historical load data and real-time load data to obtain historical fluctuation features and real-time fluctuation features. Perform feature encoding on time-synchronized historical external condition data and real-time external condition data to obtain historical external condition features and real-time external condition features. Use a conditional hidden Markov model to model different fluctuation states and historical external condition features in the historical fluctuation features, and establish an improved online identification model for fluctuation states. Take real-time fluctuation features and real-time external condition features as input, and output the identified fluctuation states in real time. Specifically, the conditional hidden Markov model is obtained by introducing external condition vectors and weights into the observation model of the hidden Markov model. This includes: S21. Processing the time-aligned load data at different time scales according to corresponding... S21. Aggregate load data across time scales to obtain aggregated load data for each time scale, and calculate the logarithmic return of the aggregated load data for each time scale; S22. Extract volatility index features of the aggregated load data for each time scale based on the corresponding logarithmic return to characterize stochastic behavior; S23. Interpolate the volatility index features of all time scales to a unified time resolution to form a comprehensive volatility feature vector, which presents clustering patterns corresponding to different volatility states; S24. Use a Gaussian Hidden Markov Model to model the temporal evolution of different volatility states in the comprehensive volatility feature vector, and establish an online identification model for volatility states; S25. After parameter estimation, the online identification model takes real-time volatility features as input and outputs the identified volatility states in real time. S30. Based on the identified fluctuating operating state, adaptively generate corresponding noise and inject it into the time-aligned real-time load data to generate enhanced load data; including: S31. Calculate the mean fluctuation level and adaptive scaling factor for each identified fluctuating operating state; S32. Generate adaptive noise proportional to the fluctuation characteristics of the current fluctuating operating state based on the mean fluctuation level and adaptive scaling factor; S33. Inject the adaptive noise into the baseline load data, synthesize it with the baseline load data, and generate enhanced load data by upper and lower limit clipping.

7. The method for augmenting and generating time-series load data based on a hidden Markov model according to claim 6, characterized in that, The external condition data includes meteorological data, calendar information, and economic indicators.

8. A time-series load data augmentation and generation apparatus based on a hidden Markov model, used to perform the method described in any one of claims 1 to 5, characterized in that, include: The time alignment module is used to divide and align the acquired historical and real-time raw time-series load data according to key time periods to obtain time-aligned historical load data and real-time load data. The real-time identification module extracts fluctuation features from time-aligned historical load data and real-time load data, obtaining historical and real-time fluctuation features. It uses a Hidden Markov Model to model different fluctuation states within the historical fluctuation features, establishing an online identification model for these fluctuation states. Taking real-time fluctuation features as input, it outputs the identified fluctuation states in real time. This includes: S21. Aggregating the time-aligned load data at different time scales according to their respective time scales to obtain aggregated load data for each time scale, and calculating the logarithmic rate of return for each aggregated load data at each time scale; S22. Combining... The corresponding logarithmic return rate extracts the volatility index features of the aggregated load data for each time scale to characterize stochastic behavior; S23. The volatility index features of all time scales are interpolated to a uniform time resolution to form a comprehensive volatility feature vector, which presents a clustering pattern corresponding to different volatility states; S24. A Gaussian Hidden Markov Model is used to model the temporal evolution of different volatility states in the comprehensive volatility feature vector to establish an online identification model for volatility states; S25. After parameter estimation, the online identification model takes real-time volatility features as input and outputs the identified volatility states in real time. The data augmentation module is used to adaptively generate corresponding noise based on the identified fluctuating operating state and inject it into the time-aligned real-time load data to generate augmented load data. This includes: S31. Calculating the mean fluctuation level and adaptive scaling factor for each identified fluctuating operating state; S32. Generating adaptive noise proportional to the fluctuation characteristics of the current fluctuating operating state based on the mean fluctuation level and adaptive scaling factor; S33. Injecting the adaptive noise into the baseline load data, synthesizing it with the baseline load data, and generating augmented load data through upper and lower limit clipping.

Images