A mixed data processing method, system and device based on power big data

Abstract

The application discloses a kind of based on power big data's mixed frequency data processing method, system and equipment, it is related to power data processing technical field, including: with low-frequency timestamp as benchmark to high-frequency power time series data is time slice;The sequence variance of current slice in high-frequency power time series data is calculated with time correlation index;With the variance and index as optimization constraint condition, dynamically demarcate time window search interval and weight function parameter boundary;In the boundary, with fitting error minimization as goal iteration solution, extract target dimensionality reduction parameter;Based on target dimensionality reduction parameter, high-frequency power time series data is executed sliding weighted summation, generates derivative sequence, combines low-frequency historical data and outputs low-frequency estimated result.The application breaks the limitation of traditional model parameter static solidification by dynamic limited optimization driven by physical characteristics, effectively reduces the computing power overhead while significantly improving the response speed to physical boundary mutation signal and the accuracy of data fusion estimation.

Description

Technical Field

[0001] This invention relates to the field of power data processing technology, and more specifically, to a method, system, and device for mixed-frequency data processing based on power big data. Background Technology

[0002] With the development of big data and IoT technologies, the fusion processing of multi-source heterogeneous time series data plays a crucial role in complex decision-making systems such as macro-situational awareness. In the field of cross-analysis of electricity and economics, smart meters in the power grid often generate high-frequency continuous data (such as daily and weekly data), while target economic variables (such as quarterly GDP) are usually low-frequency data. This frequency mismatch between high and low frequency data presents significant technical challenges to the fusion modeling of multi-source data.

[0003] To address the problem of mixed-frequency data fusion, existing technologies have proposed several mixed-frequency data processing schemes based on complex architectures. For example, CN122022203A discloses an economic forecasting method, system, electronic device, and medium based on weekly, monthly, and quarterly big data. First, it employs a Mixed-Frequency Data Sampling Model (MIDAS) combined with machine learning variable selection to perform high-frequency forecasts from weekly to monthly. Then, the output results are passed to a Mixed-Frequency Dynamic Factor Model (MF-DFM), and Kalman filtering and expectation-maximization algorithms are used for parameter iteration to complete the final forecast from monthly to quarterly.

[0004] Although this solution has achieved the integration of multi-frequency data to some extent, the following significant technical shortcomings have still been exposed in the actual operation of the system: First, the computational cost is enormous and the response to sudden changes is slow. The underlying state-space model and Kalman filter iterative mechanism of the existing solution are extremely computationally intensive when dealing with massive amounts of high-frequency power data; moreover, the traditional smoothing mechanism is prone to masking real anomalies, resulting in a serious lag in the model's response to sudden signals such as extreme shocks to the power grid.

[0005] Second, the parameter optimization mechanism is detached from the underlying physical characteristics. When setting dimensionality reduction weights and time windows, existing models often rely on "static fixation" based on human experience or "global blind grid search," lacking a restricted optimization mechanism that is dynamically driven by the underlying physical fluctuation characteristics. This not only results in serious waste of computing power but also easily leads to distortion in the extraction of high-frequency features.

[0006] Third, traditional alignment mechanisms are prone to data leakage risks. Existing technologies mostly use rigid timestamp truncation alignment methods, which can easily lead to models reading high-frequency data from the future when dealing with cross-time boundaries. This can cause "data leakage" vulnerabilities that violate the laws of temporal causality, causing the model to fail in actual industrial deployments. Summary of the Invention

[0007] To overcome the aforementioned deficiencies of the prior art, embodiments of the present invention provide a method, system, and device for mixed-frequency data processing based on power big data. By extracting the sequence variance and time correlation index of high-frequency power time-series data as optimization constraints, the time window search interval and the boundary of the weight function parameters are dynamically defined, and the optimal dimensionality reduction parameters are automatically iteratively solved within the defined parameter space. This solves the problem in the prior art that the core parameters of mixed-frequency data processing rely on static configuration based on human experience and cannot be dynamically and adaptively matched according to the fluctuation characteristics of the data itself.

[0008] To achieve the above objectives, the present invention provides the following technical solution: A mixed-frequency data processing method based on power big data includes the following steps: acquiring low-frequency target time-series data and corresponding high-frequency power time-series data, and performing time slicing based on low-frequency timestamps; calculating the sequence variance and time correlation index of the high-frequency power time-series data within the current slice; using the variance and index as optimization constraints, dynamically defining the time window search interval and the boundary of the weight function parameters; iteratively solving within the boundary with the goal of minimizing the fitting error, and extracting the target dimensionality reduction parameters; performing sliding weighted summation on the high-frequency power time-series data based on the target dimensionality reduction parameters to generate frequency-aligned derived sequences, and outputting low-frequency estimation results by combining low-frequency historical data.

[0009] In a preferred embodiment, the step of acquiring low-frequency target time-series data and corresponding high-frequency power time-series data, and performing time slicing based on low-frequency timestamps, includes: extracting the reference time boundary of the low-frequency target time-series data on the time axis; establishing a time tolerance interval for edge data points in the high-frequency power time-series data that cross the reference time boundary; calculating the time overlap ratio between the edge data points and adjacent time slices within the time tolerance interval; and converting the power load value of the edge data points according to the time overlap ratio, and assigning the converted load values ​​to the corresponding adjacent time slices to generate an aligned initial mixing data sequence.

[0010] In a preferred embodiment, the step of calculating the sequence variance and time correlation index of the high-frequency power time series data within the current slice includes: performing trend separation processing on the high-frequency power time series data within the current time slice to extract the high-frequency residual sequence; calculating the sample variance of the high-frequency residual sequence as the sequence variance; obtaining the benchmark lag order corresponding to the power grid physical dispatch cycle; and calculating the time correlation index of the high-frequency power time series data under the benchmark lag order.

[0011] In a preferred embodiment, the time correlation index includes the autocorrelation coefficient, partial autocorrelation coefficient, or Hearst exponent.

[0012] In a preferred embodiment, the step of using the variance and index as optimization constraints to dynamically define the time window search interval and the boundary of the weight function parameters includes: constructing a first mapping relationship between the sequence variance and the time window search interval, such that the upper limit of the search interval shrinks in the shortening direction as the sequence variance increases, and the lower limit expands in the lengthening direction as the sequence variance decreases; constructing a second mapping relationship between the time correlation index and the boundary of the weight function parameters, such that the upper bound of the weight function parameter values ​​monotonically compresses in the rapid decay direction as the time correlation index decreases; and using the search interval adjusted by the first mapping relationship and the parameter boundary adjusted by the second mapping relationship to jointly define the parameter space.

[0013] In a preferred embodiment, the iterative solution within the boundary with the objective of minimizing the fitting error, and the extraction of the target dimensionality reduction parameters at convergence, includes: calculating the basic fitting residual between historical low-frequency observations and the estimated values ​​under the current parameters; calculating the parameter difference of the weight function at adjacent high-frequency time steps as the basic smoothing penalty term; constructing an inverse mapping relationship between the sequence variance and the penalty weight, driving the penalty weight to decrease as the sequence variance increases; dynamically weighting the basic smoothing penalty term using the penalty weight, and fusing it with the basic fitting residual to construct a composite objective function; and iteratively solving within the parameter space with the minimization of the composite objective function as the convergence condition, extracting the time window length and weight function parameters at convergence as the target dimensionality reduction parameters.

[0014] In a preferred embodiment, the step of performing sliding weighted summation on high-frequency power time-series data based on target dimensionality reduction parameters to generate a frequency-aligned derived sequence includes: mapping and generating a normalized weight sequence equidistant from the time window length based on the attenuation parameter in the target dimensionality reduction parameters; using the timestamp of the low-frequency target time-series data as a sliding anchor point, extracting high-frequency segments of equal length to the time window length in reverse order from the high-frequency power time-series data; performing an inner product operation on each high-frequency data point within the high-frequency segment and the normalized weight sequence to generate a single-point dimensionality reduction feature value corresponding to the current time slice; and stepping along the time axis to slide the anchor point and splicing the single-point dimensionality reduction feature values ​​in time sequence to output the derived sequence.

[0015] In a preferred embodiment, the step of combining low-frequency historical data to output low-frequency estimation results includes: lag-aligning the derived sequence with the historical observation sequence of the low-frequency target time series data to construct an input feature set containing high-frequency derived features and low-frequency historical values; and based on the input feature set, performing joint mapping estimation on the target value of the current low-frequency time slice to output low-frequency estimation results.

[0016] A mixed-frequency data processing system based on power big data includes: a data processing module for acquiring low-frequency target time-series data and corresponding high-frequency power time-series data, and performing time slicing based on low-frequency timestamps; a feature extraction module for calculating the sequence variance and time correlation index of the high-frequency power time-series data within the current slice; a constraint optimization module for using the variance and index as optimization constraints to dynamically define the time window search interval and the boundary of the weight function parameters; iteratively solving within the boundary with the goal of minimizing the fitting error to extract target dimensionality reduction parameters; and a dimensionality reduction output module for performing sliding weighted summation on the high-frequency power time-series data based on the target dimensionality reduction parameters to generate frequency-aligned derived sequences, and outputting low-frequency estimation results in conjunction with low-frequency historical data.

[0017] An electronic device includes a memory and a processor: the memory is used to store a program; the processor is used to execute the program to implement the various steps of the method.

[0018] The technical effects and advantages of the mixed-frequency data processing method, system, and equipment based on power big data of the present invention are as follows: 1. This invention extracts the sequence variance and time correlation index of high-frequency power time series data and uses them as optimization constraints to dynamically define the time window search interval and the boundary of the weight function parameters. This realizes the automated dynamic optimization of the mixed frequency data processing parameters based on the fluctuation characteristics of the data itself. It avoids the problem of amplified fitting error caused by relying on manual experience to statically set the form of the weight function and the time window strategy in the prior art, and improves the extraction accuracy of high-frequency leading information.

[0019] 2. When generating low-frequency estimation results, this invention adopts a strategy of directly fusing derived sequences and low-frequency historical data, replacing the complex architecture of existing technologies that rely on state-space models for Kalman filtering and expectation-maximization iteration. This significantly reduces computational overhead while ensuring prediction accuracy and improves the response speed to sudden changes in real physical world signals. Attached Figure Description

[0020] Figure 1 A flowchart illustrating a method for mixed-frequency data processing based on power big data, provided in an embodiment of the present invention; Figure 2 A functional block diagram of a mixing data processing system based on power big data provided in an embodiment of the present invention; Figure 3 This invention provides an internal hardware topology diagram of an electronic device. Figure 4 This is a visual schematic diagram of trend separation and residual extraction processing provided in an embodiment of the present invention; Figure 5This is a schematic diagram illustrating the change of the dynamic constraint optimization parameter space provided in an embodiment of the present invention; Figure 6 This is a mapping curve of adaptive penalty weight as a function of sequence variance, provided for an embodiment of the present invention. Detailed Implementation

[0021] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those of ordinary skill in the art without creative effort are within the scope of protection of the present invention.

[0022] In the embodiments of the present invention, the final output "low-frequency estimation result" refers to an adaptive, high-precision estimation of macroeconomic indicators or specific industry operating status variables with long statistical periods and significant release lags, achieved through cross-frequency nonlinear mapping using high-frequency continuously monitored power time-series data. To enable those skilled in the art to more clearly understand the physical form and business value of the present invention in actual industrial and economic operation decision-making scenarios, this embodiment provides the following three parallel and mutually substitutable business application scenarios: Scenario 1: Macroeconomic Situation Awareness. In this scenario, high-frequency electricity time-series data represents the daily total electricity consumption sequence, while low-frequency target time-series data represents the quarterly GDP growth rate. When the system reaches the middle of the current quarter, it can calculate and output the estimated GDP growth rate for that quarter in real time, providing a highly timely leading macroeconomic indicator for macroeconomic control decisions.

[0023] Scenario 2: Regional Industrial Operation Monitoring. In this scenario, high-frequency power time-series data represents the hourly load sequence of energy-intensive industries, while low-frequency target time-series data represents the monthly industrial added value of enterprises above a designated size. By extracting the non-stationary abrupt changes in the operating rate of industrial enterprises, the system outputs the estimated industrial added value for the current month in advance, achieving proactive identification of changes in industrial cycle trends.

[0024] Scenario 3: Analysis of Consumption Vitality in Business Districts. In this scenario, high-frequency electricity time-series data represents the daily commercial electricity load sequence of the core business district, while low-frequency target time-series data represents the monthly total retail sales of consumer goods. The system extracts cyclical memory features driven by calendar effects (weekends and weekdays) to achieve a quantitative estimate of consumption vitality.

[0025] To more intuitively demonstrate the universality and flexibility of the present invention, the core configurations in the above scenarios are summarized in Table 1.

[0026] Table 1: Examples of Mixing Prediction Model Configurations in Different Business Scenarios

[0027] Example 1, Figure 1 This invention presents a method for mixing frequency data processing based on power big data, comprising the following steps: S1, acquire low-frequency target time series data and corresponding high-frequency power time series data, and perform time slicing based on low-frequency timestamps; In this embodiment, the acquisition of low-frequency target time-series data and corresponding high-frequency power time-series data, using low-frequency timestamps as a basis for time slicing, specifically includes the following steps: The low-frequency target time series data refers to a series of macroeconomic operation or economic indicators with a low sampling frequency, such as quarterly gross domestic product (GDP), monthly industrial added value, or monthly total retail sales of consumer goods; the high-frequency power time series data refers to a series of high-frequency continuous monitoring data from smart meters, SCADA systems, or operation and dispatching systems, such as daily total electricity consumption, hourly power load, or hourly transformer operating load.

[0028] Because low-frequency economic indicators are typically statistically analyzed strictly on a monthly or quarterly basis, while high-frequency electricity data may have non-natural period offsets in meter reading cycles, settlement cycles, or physical data collection times (for example, some industrial electricity uses the 25th of each month as the settlement boundary), high- and low-frequency data cannot be directly aligned at time boundaries. To eliminate this "edge gap" phenomenon caused by inconsistent statistical standards from heterogeneous data sources, this embodiment performs flexible time-slice alignment processing on the acquired initial mixed-frequency data, specifically including the following sub-steps: S11, first extract the reference time boundary of the low-frequency target time series data on the time axis (denoted as...). For example, 24:00:00 on the last day of a calendar month, or 24:00:00 on the last day of a quarter. Subsequently, high-frequency power time-series data was retrieved to identify periods whose duration spanned this baseline time boundary. The edge data point. For this edge data point, the system adaptively establishes a time tolerance interval:

[0029] The half span of the time tolerance interval The settings are dynamically configured based on the sampling period of the high-frequency data. For example, if the high-frequency power data represents daily electricity consumption, then... It can be set to 12 hours or 24 hours to ensure that the entire lifecycle of the cross-border edge data point is fully contained within the tolerance range.

[0030] S12 calculates the temporal overlap ratio between edge data points and adjacent time slices within the temporal tolerance interval. Specifically: Let the actual total duration of the cross-boundary edge data point on the time axis be . (e.g., a complete data collection cycle). The system uses a high-precision timestamp comparison algorithm to calculate the distance between the edge data point and its two adjacent time slices (i.e., based on the reference time boundary) within the time tolerance interval. The current time slice to the left of the dividing line. And the next time slice on the right The length of time overlap.

[0031] Suppose that the edge data point falls within the current time slice. The length of the time period within is Falling into the next time slice The length of the time period within is And satisfy The time overlap ratio between the edge data point and the two adjacent time slices before and after it is calculated using the following formula. and :

[0032] S13, Obtain the power load value of the edge data point in the original high-frequency time-series data (denoted as...). The system performs lossless recalculation of the load value based on the calculated time overlap ratio, separating its true physical contribution to the two consecutive time slices. The recalculated value is then incorporated into the current time slice. The load conversion value is :

[0033] After conversion, it will be included in the next time slice. The load conversion value is :

[0034] Furthermore, perform sequence reorganization: for sequences that fall entirely within the current time slice... Non-edge high-frequency data points within the slice are directly assigned to the slice based on their original load values; for edge data points crossing boundaries, the calculated load values ​​are used instead. Accumulate to slice At the end, Accumulate to the next slice The process begins by iterating through all reference time boundaries on the timeline and repeating the aforementioned conversion and aggregation steps, ultimately generating an initial mixing data sequence that is strictly aligned with the low-frequency timestamps on the time scale.

[0035] Existing technologies typically handle edge data points that cross time boundaries by either directly discarding them or rigidly assigning them entirely to a single adjacent slice. If the cross-boundary data point is entirely assigned to a previous slice, some load information occurring in the next cycle will be incorrectly introduced into the historical slice, causing data traversal and disrupting temporal causality. If the edge data point is directly discarded, irreversible loss of critical abrupt change signals will occur. This embodiment establishes a time tolerance interval and calculates the time overlap ratio to split and assign the power load values ​​of edge data points according to their actual time assignment. This ensures that the data assigned to each adjacent slice strictly corresponds to the actual load portion occurring within the time interval of that slice, fundamentally preventing data traversal while completely preserving all original sampling information of the edge data point, thus avoiding signal loss.

[0036] Based on time overlap ratio and The conversion mechanism ensures that edge data points satisfy the conversion requirements before and after conversion. This means that the original total load value is distributed to adjacent slices without loss, which conforms to the physical constraint of grid energy conservation. Rigid truncation will artificially create jumps or zeroing of load values ​​at the slice boundaries, forming data cliffs or pseudo-peaks, introducing false fluctuation components into the distribution feature extraction in the subsequent step S2, and thus interfering with the constraint optimization module in step S3's identification of real data features.

[0037] S2, calculate the sequence variance and time correlation index of high-frequency power time series data in the current slice; In this embodiment, the calculation of the sequence variance and time correlation index of high-frequency power time series data within the current slice specifically includes the following steps: S21, let's assume the current time slice Within, the aligned initial high-frequency power time series data constitute a set. ,in This represents the total number of high-frequency data points within the slice. For the first The power load observations at each time step. This embodiment uses the standard Hodrick-Prescott filter (HP filter) to... Perform temporal smoothing decomposition. HP filtering separates the long-term trend component by minimizing the following loss function. :

[0038] In this formula, the first term is the fitting penalty term, and the second term is the smoothing penalty term. The preset smoothing prior parameter (preferably set to [value] for daily high-frequency data) For weekly data, the preferred setting is 14400.

[0039] Solve for the trend components Then, the high-frequency residual sequence after filtering out low-frequency trends is extracted through interpolation. ,in .

[0040] Figure 4 This is a visual schematic diagram illustrating trend separation and residual extraction processing provided in an embodiment of the present invention. Figure 4 As shown in the "Before Extraction" section, the original high-frequency power time series data contains a significant long-term trend component (represented in the figure as a gray dashed line that rises diagonally along the time axis). This trend component easily masks and interferes with microscopic abrupt changes. After the system calls the Hodrick-Prescott (HP) filtering algorithm to perform the stripping operation, as shown... Figure 4 As shown in the "After Extraction" section, the long-term trend component is forcibly zeroed out (flattened to the 0 mark), thereby accurately outputting a high-frequency residual sequence that purely reflects random physical shocks, eliminating macroscopic base interference for subsequent variance calculation.

[0041] S22, based on the extracted high-frequency residual sequence Calculate its unbiased sample variance and use it as the sequence variance characterizing the degree of drastic fluctuations in the current slice of the power physics system (denoted as ). ):

[0042] in, This is the sample mean of the high-frequency residual sequence within the current slice. The magnitude of the value directly reflects the intensity of external physical shocks (such as load surges caused by extreme high temperatures) within that time window.

[0043] S23, obtain the reference lag order (denoted as) that matches the power grid physical dispatch cycle. In this embodiment, due to the significant "weekend effect" and "weekday effect" of the power grid load, if the data frequency is daily, then... The preferred setting is 7.

[0044] Based on initial high-frequency power time series data Calculate the lag order at the reference level. The autocorrelation coefficient below is used as the time correlation index (denoted as ). ):

[0045] in, Original sequence The mean of the time correlation coefficient. In addition to the autocorrelation coefficient, in other alternative embodiments, the time correlation index may also use the partial autocorrelation coefficient (PACF) or the Hurst exponent to characterize the long-term memory of time series data.

[0046] Existing technologies typically operate directly on the original sequence when calculating sequence variance, failing to distinguish the impact of slow variables such as macroeconomic growth on load fluctuations from sudden physical shocks. This can easily lead to misinterpreting trend changes as drastic fluctuations, resulting in frequent and erroneous adjustments to the time window in subsequent step S3. This embodiment extracts and removes the trend component through HP filtering, obtaining a high-frequency residual sequence that purely reflects random load fluctuations. The sequence variance is then calculated based on this residual sequence. This sequence variance, stripped of macroeconomic trend components, only increases significantly when a real physical shock occurs, providing a precise and error-free feedforward trigger signal for the dynamic delineation of the time window search interval in step S3.

[0047] Existing complex mixing architectures typically rely on cross-validation to perform a global grid search across the entire parameter space, resulting in computational overhead that increases exponentially with the parameter dimension. This embodiment obtains a baseline lag order corresponding to the power grid physical scheduling cycle and binds the calculation of the autocorrelation coefficient to this baseline lag order, quantifying the memory decay degree of load shocks after one physical scheduling cycle. This time correlation index, as a priori physical feature, applies a directional constraint based on the power grid operation mechanism to the boundary values ​​of the attenuation parameter of the weighting function before the substantive iterative solution in step S3. This compresses the global blind search into a locally oriented optimization constrained by physical laws, significantly reducing computational overhead and preventing the optimization result from falling into a mathematically well-fitted but physically causal local optimum.

[0048] S3, using the variance and index as optimization constraints, dynamically define the time window search interval and the boundary of the weight function parameters; In this embodiment, the step of using the variance and index as optimization constraints to dynamically define the time window search interval and the boundary of the weight function parameters specifically includes the following steps: S31, Let the system's preset baseline time window span value be... (For high-frequency daily electricity series, this benchmark value is preferably set to the maximum lag days of the corresponding low-frequency quarterly cycle, such as...) The time window search interval includes the lower limit of the optimization range. and upper limit .

[0049] This embodiment constructs the first mapping relationship, such that the upper limit of the search interval is... With the said sequence variance As it increases, it continuously contracts in the direction of shortening, with the lower limit... With the said sequence variance The scaling factor decreases and then continuously expands in the extension direction. To ensure smooth scaling and prevent interval overflow, the first mapping relationship is implemented using a mathematical function of the following exponential decay form:

[0050]

[0051] in, This is the absolute minimum time window span allowed by the algorithm (e.g., set to 7 days to ensure the basic statistical validity of the dimensionality reduction features). and This is a preset positive coefficient for scale adjustment.

[0052] According to the above formula, when a severe physical shock occurs to the power grid, leading to... When it increases sharply, the exponential term Approaching 0, driving upper limit Continuous contraction to approach The model is forced to capture only the most recent and sensitive shock signals; conversely, when the power grid is operating smoothly and... Extremely small, upper limit Automatically extend to the baseline value This increases the amount of historical information absorbed, thereby dynamically completing the time window search interval. Adaptive dynamic delineation.

[0053] S32, In this embodiment, a two-parameter Beta weighting function is used as an example. Its functional form is: ,in , These are parameters used to control the weight decay pattern. In engineering practice, they are usually fixed. Only for Perform single-dimensional optimization. The larger the value, the faster the weight of historical data from a longer period decays.

[0054] This embodiment constructs the second mapping relationship, such that upper bound of values Follow The rate of decrease leads to monotonically compressed towards the direction of rapid decay. The second mapping relationship is implemented through the following bounded rational mapping function:

[0055] in, Use the upper limit of the basic attenuation reference value (e.g., set to 5). This is the incremental adjustment coefficient. This represents the sensitivity constraint coefficient.

[0056] According to this formula, when the calculated time correlation index When the denominator decreases significantly (indicating extremely weak physical correlation inertia between high- and low-frequency time-series data within the current slice and that historical information is mostly useless noise), the calculated value decreases, leading to a lower denominator. Monotonically compressing in the direction of increasing values ​​guides the parameter space to a restricted region where high-frequency hysteresis information rapidly returns to zero.

[0057] S33, search the time window interval dynamically scaled by the first mapping relationship. The first parameter dimension constraint is used as the boundary of the weight parameters after dynamic compression via the second mapping relationship. As a second parameter dimension constraint, a closed polygonal confined parameter space is jointly clipped and defined on the dual-axis coordinate system, which is dynamically controlled by the physical operating state of the power grid. .

[0058] Figure 5 This is a schematic diagram illustrating the variation of the parameter space in dynamic constraint optimization, provided as an embodiment of the present invention. Figure 5 As shown, this embodiment constructs a time window length... With the horizontal axis as the base, The vertical axis represents a two-dimensional optimization coordinate system. Traditional mixing models, lacking physical constraints, must perform a fully blind search within a huge "global static search space" (the outer dashed box in the figure), resulting in extremely high computational overhead. This invention, however, utilizes the first and second mapping relationships to drive the upper limit of the search interval to vary with the sequence variance. As it increases, it contracts inward, while simultaneously driving the upper bound of the parameter value to correlate with time. As the range decreases, the space is compressed downwards, ultimately clipping out a significantly reduced, closed, constrained parameter space from the intersection. (The solid line area in the figure) implements structural computational pruning at the algorithm's underlying level.

[0059] This invention does not employ traditional fixed boundaries or piecewise functions. Instead, it introduces continuous negative exponents and rational mapping functions, enabling continuous fluctuations in the physical world to be seamlessly and smoothly mapped to scaling actions in the computing space. This ensures the continuity of boundary adjustments and completely eliminates numerical oscillations in the algorithm caused by piecewise abrupt changes.

[0060] Traditional optimization requires a blind full-grid search within a fixed large rectangular space. This invention utilizes... and As a feedforward constraint, the parameter space is dynamically "trimmed" into a feasible region containing only physically reasonable solutions. When encountering a sudden change signal, the computing space is compressed instantly, which greatly improves the convergence speed and ensures that the system can make a rapid and accurate boundary response to physical changes in the power grid with extremely low computing power overhead.

[0061] S4, within the boundary, iteratively solve the problem with the goal of minimizing the fitting error, and extract the target dimensionality reduction parameters; In this embodiment, the step of iteratively solving within the boundary with the objective of minimizing the fitting error to extract the target dimensionality reduction parameters specifically includes the following steps: S41, Obtain a series of historical low-frequency observations prior to the current low-frequency time slice, and base them on the test time window length of the current iteration step. With test weight function parameters The system performs frequency mixing and weighted dimensionality reduction on the high-frequency power time series data within the corresponding historical time slice using a preset frequency mixing estimation model, and outputs the corresponding historical estimation value sequence.

[0062] Calculate the deviations between historical low-frequency observations and their corresponding estimates, and aggregate the sum of squares of each deviation as the baseline fitting residual characterizing the data fitting accuracy under the current parameters. .

[0063] S42, To prevent drastic oscillations in the timing of high-frequency weights that violate physical laws, the system performs shape smoothness extraction on the current test weight function. Specifically, the system calculates the second-order parameter difference of the weight function in adjacent high-frequency time steps (i.e., the change in the difference between the weight in the current step and the weights in the two steps before and after), and accumulates the squares of the parameter differences in all time steps as the basic smoothing penalty term. The magnitude of this penalty objectively reflects the complexity of the current weight allocation pattern.

[0064] S43. In the existing penalized regression, the penalty weight is usually a global static constant, which cannot distinguish between real physical abrupt changes and regular random noise. When the power grid experiences a real physical shock, the weight function should respond quickly and allow for a certain degree of steep change, but the fixed penalty will forcibly smooth out this real signal.

[0065] To overcome the above-mentioned shortcomings, this embodiment constructs the sequence variance extracted in step S2 (denoted as ). ) and dynamic penalty weight (denoted as The inverse mapping relationship drives the penalty weights. With series variance The rate of increase is not linearly decreasing. To ensure the continuity of the algorithm, the reverse mapping relationship uses the following rational decay formula to implement the fallback calculation:

[0066] in, This is the preset maximum penalty coefficient (used to ensure basic smoothing strength). This is a scalar for sensitivity adjustment. According to this mechanism, when the power grid encounters a real physical shock (… During a surge, the penalty weight The weights are dynamically and rapidly reduced, thus "unleashing" the underlying algorithm and allowing for steep, abrupt weight changes; conversely, a high penalty is maintained to filter out random white noise.

[0067] S44, the calculated dynamic penalty weights are used to perform a product weighting operation on the basic smoothing penalty term, and then added to the basic fitting residuals to construct a composite objective function (denoted as...). ):

[0068] Figure 6 This is a mapping curve of adaptive penalty weights as a function of sequence variance, provided as an embodiment of the present invention. Figure 6 As shown, the horizontal axis represents the variance of the extracted sequence. The vertical axis represents the generated penalty weights. When the power grid is in a stable operating range (variance) (Lower), penalty weight Keep close to the base maximum penalty coefficient At higher levels, the constraint model generates highly smooth weights to resist white noise; however, when the system encounters physically abrupt regions (variance...) (Surge occurs), penalty weight By rapidly sliding along a nonlinear curve, the smoothing constraint is adaptively relaxed, allowing the system to generate a non-smooth weight distribution with high local sensitivity, ensuring the hardware system's extremely rapid transient response to real physical shocks.

[0069] S45, the parameter space defined in step S3 Within the framework, iterative optimization is performed using the minimization of the composite objective function as the convergence condition. Iterative convergence is determined when the decrease in the objective function value is less than a preset convergence threshold, or when the number of iterations reaches a preset upper limit. The time window length and weight function parameters at convergence are extracted and used as the output objective dimensionality reduction parameters for subsequent steps in generating derived sequences.

[0070] Existing penalized least squares methods typically employ globally static smoothing penalty coefficients, which cannot distinguish between real physical shocks and conventional random noise. This embodiment constructs an inverse mapping relationship between sequence variance and penalty weights. When a real physical shock occurs to the power grid, the penalty weights automatically decay, relaxing the smoothing constraints; during stable operation, the penalty weights automatically recover, strengthening noise suppression, thus achieving an adaptive balance between fitting accuracy and noise-resistant smoothing. This embodiment does not directly solve the composite objective function in an unconstrained global space, but strictly confines the optimization process to the feasible region tailored based on the physical characteristics of the power grid in step S3. This allows the optimization algorithm to avoid mathematical pseudo-solutions that are not permissible by physical common sense, improving the iterative convergence speed, while ensuring that the extracted dimensionality-reduced parameters have solid practical significance in power grid engineering.

[0071] S5 performs sliding weighted summation on high-frequency power time-series data based on the target dimensionality reduction parameters to generate frequency-aligned derived sequences, and outputs low-frequency estimation results by combining low-frequency historical data.

[0072] In this embodiment, the step of performing a sliding weighted summation on high-frequency power time-series data based on the target dimensionality reduction parameters to generate a frequency-aligned derived sequence, and then combining it with low-frequency historical data to output a low-frequency estimation result, specifically includes the following steps: S51, based on the attenuation parameter (denoted as) in the target dimensionality reduction parameters. Substitute it into the preset polynomial weighting function (in this embodiment, fixed parameters are used). Taking the Beta function as an example), the time window length (denoted as ) in the mapping generation and target dimensionality reduction parameters is used. Equal-interval normalized weight sequence , where the first in the sequence time steps ( Normalized weight single-point value The calculation formula is:

[0073] in, This is an index variable for high-frequency time steps, with a value range of 1 to... , This is an auxiliary index variable used in the denominator weight normalization and summation process.

[0074] S52, let the time slice corresponding to the current low-frequency target time series data to be estimated be... Timestamps of low-frequency target time series data As a sliding anchor point, reverse-order truncation of high-frequency power time-series data is performed, with the time window length as the reference point. Equal-length high-frequency segments .

[0075] S53, for high-frequency segments High-frequency data points and normalized weight sequences within Perform inner product operations to generate single-point dimensionality-reduced feature values ​​corresponding to the current low-frequency time slice. Its calculation formula is:

[0076] in, These are the high-frequency data points at the corresponding locations.

[0077] S54, Stepping Anchor Points Along the Time Axis The process of reversing the truncation and inner product operations is repeated. The generated single-point dimensionality-reduced feature values ​​are then concatenated in time sequence to output a frequency-aligned derived sequence. .

[0078] Obtain the historical observation sequence of the low-frequency target time series data. Perform hysteresis alignment between the derived sequence and the historical observation sequence of the low-frequency target time series data to construct an input feature set containing high-frequency derived features and low-frequency historical values. .

[0079] S55, based on the input feature set, perform joint mapping estimation on the target value of the current low-frequency time slice, and output the low-frequency estimation result. In this embodiment, the mathematical expression for joint mapping estimation is:

[0080] in, The output is the low-frequency estimation result. For the intercept term of the cross-frequency fusion estimation model, For corresponding high-frequency derived features High-frequency drive parameters, For the corresponding number Autoregressive parameters of low-frequency historical observations, The preset low-frequency hysteresis order, As an auxiliary index variable in the process of summing low-frequency historical data, This represents the random error term of the model.

[0081] Existing technologies for cross-frequency data dimensionality reduction, if employing conventional forward sliding or globally centralized window alignment, can easily lead to future high-frequency power loads being prematurely mixed into the current low-frequency time slices, causing data leakage. This results in pseudo-high accuracy in historical data backtesting, but serious deviations in online predictions during actual grid operation. This embodiment uses the timestamp of the low-frequency target time series data as the sliding anchor point, and enforces reverse truncation in the high-frequency power time series data. This ensures that the timestamp of every high-frequency data point participating in subsequent inner product operations is absolutely earlier than the current sliding anchor point. A causal constraint is established at the computer memory reading level, completely eliminating the possibility of the algorithm acquiring data in the future, and significantly improving the system's generalization ability and prediction reliability in real production environments.

[0082] To verify the actual performance of the method of this invention, this embodiment conducted a retrospective verification based on real provincial power grid high-frequency electricity load data and quarterly GDP indicators for the same period. The comparison objects were the traditional mixed-frequency dynamic factor model (MF-DFM) and the conventional static parameter MIDAS model. The performance evaluation comparison results are shown in Table 2.

[0083] Table 2: Performance Comparison of the Invention Method and Traditional Mixing Prediction Model

[0084] Example 2, Figure 2 A mixing data processing system based on power big data is presented, and the functional steps of this system correspond one-to-one with those of the method embodiment 1 described above. Specifically, the mixing data processing system based on power big data includes the following functional modules: The data processing module is used to acquire low-frequency target time-series data and corresponding high-frequency power time-series data, and to perform time slicing based on low-frequency timestamps. The feature extraction module is used to calculate the sequence variance and time correlation index of high-frequency power time series data within the current slice; The constraint optimization module is used to use the variance and index as optimization constraints to dynamically define the time window search interval and the boundary of the weight function parameters; within the boundary, iteratively solves the problem with the goal of minimizing the fitting error and extracts the target dimensionality reduction parameters. The dimensionality reduction output module is used to perform sliding weighted summation on high-frequency power time series data based on the target dimensionality reduction parameters, generate frequency-aligned derived sequences, and output low-frequency estimation results in combination with low-frequency historical data.

[0085] Example 3, Figure 3 An electronic device is provided, characterized in that it includes a memory and a processor: The memory is used to store programs; The processor is used to execute the program to implement the various steps of the mixing data processing method based on power big data as described in Embodiment 1.

[0086] In practical industrial deployments, the electronic device can be directly integrated into the central server of the power grid dispatch center, cloud computing nodes, or the hardware terminal of the smart grid big data analysis platform. When the processor executes the above program instructions, its internal arithmetic logic unit (ALU) and register set will directly perform high-frequency data access interactions with memory. By performing low-level hardware-level pipelined calculations on timestamps, power load matrices, and penalty weight operators, the huge computing power overhead caused by traditional Kalman filter architectures to system hardware is significantly eliminated. In the event of sudden power surges, the parameter space can be adaptively adjusted within nanoseconds, ensuring the hardware system's extremely rapid response to sudden changes in real physical world signals.

[0087] The above formulas are all dimensionless calculations. The formulas are derived from software simulations based on a large amount of collected data to obtain the most recent real-world results. The preset parameters in the formulas are set by those skilled in the art according to the actual situation.

[0088] The above embodiments can be implemented, in whole or in part, by software, hardware, firmware, or any other combination thereof. When implemented using software, the above embodiments can be implemented, in whole or in part, in the form of a computer program product.

[0089] Those skilled in the art will recognize that the modules and algorithm steps of the various examples described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this application.

[0090] In addition, the functional modules in the various embodiments of this application can be integrated into one processing module, or each module can exist physically separately, or two or more modules can be integrated into one module.

[0091] The above description is merely a specific embodiment of this application, but the scope of protection of this application is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the scope of the technology disclosed in this application should be included within the scope of protection of this application. Therefore, the scope of protection of this application should be determined by the scope of the claims.

[0092] In conclusion, the above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the protection scope of the present invention.

Claims

1. A method for mixing frequency data processing based on power big data, characterized in that, Includes the following steps: Acquire low-frequency target time-series data and corresponding high-frequency power time-series data, and perform time slicing based on low-frequency timestamps; Calculate the sequence variance and time correlation index of high-frequency power time-series data within the current slice; The variance and index are used as optimization constraints to dynamically define the time window search interval and the boundary of the weight function parameters. Within the defined boundary, the solution is iteratively obtained by minimizing the fitting error to extract the target dimensionality reduction parameters. Based on the target dimensionality reduction parameters, a sliding weighted summation is performed on the high-frequency power time series data to generate a frequency-aligned derived sequence, and the low-frequency estimation result is output by combining it with low-frequency historical data.

2. The method according to claim 1, characterized in that, The acquisition of low-frequency target time-series data and corresponding high-frequency power time-series data, using low-frequency timestamps as a basis for time slicing, includes: Extract the reference time boundary of low-frequency target time series data on the time axis, and establish a time tolerance interval for edge data points in high-frequency power time series data that cross the reference time boundary; Calculate the temporal overlap ratio between edge data points and adjacent time slices within the time tolerance interval; Based on the time overlap ratio, the power load values ​​of edge data points are converted, and the converted load values ​​are assigned to the corresponding adjacent time slices to generate an aligned initial mixing data sequence.

3. The method according to claim 2, characterized in that, The calculation of the sequence variance and time correlation index of high-frequency power time series data within the current slice includes: Perform trend separation processing on the high-frequency power time series data within the current time slice to extract the high-frequency residual sequence; The sample variance of the high-frequency residual sequence is calculated as the sequence variance; Obtain the benchmark lag order corresponding to the power grid physical dispatch cycle, and calculate the time correlation index of high-frequency power time series data under the benchmark lag order.

4. The method according to claim 3, characterized in that, The time correlation indicators include autocorrelation coefficient, partial autocorrelation coefficient, or Hearst exponent.

5. The method according to claim 4, characterized in that, The step of using the variance and index as optimization constraints to dynamically define the time window search interval and the boundary of the weight function parameters includes: A first mapping relationship is established between sequence variance and time window search interval, such that the upper limit of the search interval shrinks in the shortening direction as the sequence variance increases, and the lower limit expands in the lengthening direction as the sequence variance decreases; A second mapping relationship is constructed between the time correlation index and the boundary of the weight function parameter, so that the upper bound of the weight function parameter value is monotonically compressed in the direction of rapid decay as the time correlation index decreases; The parameter space is jointly defined by the search interval adjusted by the first mapping relationship and the parameter boundary adjusted by the second mapping relationship.

6. The method according to claim 5, characterized in that, The iterative solution within the boundary, with the objective of minimizing the fitting error, extracts the target dimensionality reduction parameters at convergence, including: Calculate the basic fit residuals between historical low-frequency observations and estimated values ​​under current parameters; Calculate the parameter difference of the weighting function at adjacent high-frequency time steps, and use it as the basic smoothing penalty term; A reverse mapping relationship between sequence variance and penalty weight is established, driving the penalty weight to decrease as the sequence variance increases; The basic smoothing penalty term is dynamically weighted using penalty weights and then fused with the basic fitting residuals to construct a composite objective function; The solution is iteratively solved in the parameter space with the minimization of the composite objective function as the convergence condition, and the time window length and weight function parameters at the convergence point are extracted as the objective dimensionality reduction parameters.

7. The method according to claim 6, characterized in that, The process of performing a sliding weighted summation on high-frequency power time-series data based on target dimensionality reduction parameters to generate frequency-aligned derived sequences includes: Based on the decay parameter in the target dimensionality reduction parameters, a normalized weight sequence with an interval equal to the time window length is generated. Using the timestamps of low-frequency target time series data as sliding anchor points, high-frequency segments of equal length to the time window are extracted in reverse order from high-frequency power time series data. Perform inner product operation on each high-frequency data point in the high-frequency segment and the normalized weight sequence to generate the single-point dimensionality reduction feature value corresponding to the current time slice; By stepping along the time axis and sliding anchor points, the dimensionality reduction feature values ​​of each single point are spliced ​​together in time sequence to output the derived sequence.

8. The method according to claim 7, characterized in that, The low-frequency estimation result output by combining low-frequency historical data includes: The derived sequence is aligned with the historical observation sequence of low-frequency target time series data by lag, and an input feature set containing high-frequency derived features and low-frequency historical values ​​is constructed. Based on the input feature set, the target value of the current low-frequency time slice is jointly mapped and estimated, and the low-frequency estimation result is output.

9. A mixing data processing system based on power big data, characterized in that, include: The data processing module is used to acquire low-frequency target time-series data and corresponding high-frequency power time-series data, and to perform time slicing based on low-frequency timestamps. The feature extraction module is used to calculate the sequence variance and time correlation index of high-frequency power time series data within the current slice; The constraint optimization module is used to use the variance and index as optimization constraints to dynamically define the time window search interval and the boundary of the weight function parameters; within the boundary, iteratively solves the problem with the goal of minimizing the fitting error and extracts the target dimensionality reduction parameters. The dimensionality reduction output module is used to perform sliding weighted summation on high-frequency power time series data based on the target dimensionality reduction parameters, generate frequency-aligned derived sequences, and output low-frequency estimation results in combination with low-frequency historical data.

10. An electronic device, characterized in that, Including memory and processor: The memory is used to store programs; The processor is used to execute the program to implement each step of the mixing data processing method based on power big data as described in any one of claims 1-8.

Images