Data restoration and measurement alignment method and system based on continuous time tensor decomposition

Abstract

The invention discloses a data restoration and measurement alignment method and system based on continuous time tensor decomposition, and the method comprises the steps: obtaining original measurement data, and carrying out the preprocessing of the original measurement data; decomposing the three-dimensional tensor structure by utilizing a CP tensor decomposition method, expanding a time factor from a fixed vector into a trajectory function which continuously changes along with time, and modeling dynamic characteristics of the time factor through a Gaussian process, so as to obtain a continuous time tensor model which can be evaluated at any time point; approximating each observation item as a local Gaussian item on a corresponding time factor by using conditional expectation propagation CEP, and obtaining a local Gaussian observation mean value and precision through iteration to convergence; performing forward filtering inference on the time factors by using a Kalman filtering forward inference algorithm to obtain a posterior mean value and a covariance of each rank time factor; carrying out RTS smoothing on each time factor chain to obtain an optimal time trajectory of the whole time period; therefore, automatic restoration and reconstruction of the measurement data are realized.

Description

Technical Field

[0001] This invention belongs to the technical field of measurement data repair methods, and particularly relates to a data repair and measurement alignment method and system based on continuous time tensor decomposition. Background Technology

[0002] With the integration of new energy equipment, electric vehicles, virtual power plants, and other modules into the power grid, the operational complexity and uncertainty of the power system have significantly increased. This necessitates increasingly robust monitoring and dynamic sensing of power system operation. Accurate operational data acquisition is fundamental to achieving overall power grid monitoring, sensing, and operational control. Measurement data such as voltage, current, active power, and reactive power are collected through devices like meters. However, due to equipment failures, environmental interference, and unstable signal transmission, measurement data often suffers from incompleteness. Furthermore, the sampling frequency and timestamps of different measurement points are inconsistent due to factors such as communication bandwidth, equipment sampling frequency, and clock synchronization accuracy, hindering subsequent data analysis and power system sensing.

[0003] Currently, there is a wealth of research on missing data restoration for power system measurements. Mainstream methods typically convert the data into a high-dimensional tensor format and use the low-rank approximation of the tensor to fill in missing values. However, these traditional tensor decompositions generally assume that the time dimension is an equally spaced and complete discrete sequence, which is often difficult to satisfy in real-world power systems. In reality, the sampling frequencies of sensor nodes may be inconsistent, and the data often contains missing measurements and irregular timestamps. In such cases, traditional discrete tensor models cannot directly handle non-uniform time sampling and lack a continuous representation of time, thus presenting certain limitations.

[0004] To address the aforementioned issues, this invention proposes a time alignment and missing data repair method that combines tensor factorization with continuous time trajectory modeling. By extending the time dimension from discrete fixed sequences to continuous modelable trajectories, it achieves unified processing of irregular sampling, asynchronous sampling, and missing points. This method can reconstruct complete, smooth, high-resolution measurement sequences at a unified time scale, providing a reliable data foundation for dynamic monitoring, operation analysis, and intelligent control of power systems. Summary of the Invention

[0005] To address the problems existing in the background technology, this invention proposes a multi-node asynchronous measurement data modeling method for power systems based on continuous time tensor trajectory learning. This method addresses the issues of missing power system measurement data, inconsistent sampling frequencies of measurement data at different nodes, and misaligned timestamps. By modeling time as a continuous variable and using a Gaussian process-state space fusion algorithm to learn the latent space trajectory of nodes, the method achieves automatic repair and reconstruction of measurement data, as well as time scale alignment of asynchronous sampling data.

[0006] The present invention adopts the following technical solution:

[0007] A data restoration and measurement alignment method based on continuous-time tensor decomposition includes the following steps:

[0008] Acquire raw measurement data and preprocess the raw measurement data to unify the raw measurement data into a standardized time batch structure and sampling interval;

[0009] Based on the preprocessed data, a three-dimensional tensor structure with nodes, measurement types, and continuous time dimensions is obtained. The CP tensor decomposition method is used to decompose the three-dimensional tensor structure, and the time factor is extended from a fixed vector to a trajectory function that changes continuously with time. Its dynamic characteristics are modeled by Gaussian process, thereby obtaining a continuous time tensor model that can be evaluated at any time point. The Gaussian process GP corresponding to the kernel function is transformed into a state-space model through linear time-invariant stochastic differential equations, thereby reducing the total time complexity from cubic to linear.

[0010] By using Conditional Expectation Propagation (CEP), each observation is approximated as a local Gaussian term on the corresponding time factor, thus decomposing the complex joint observation into several independent approximation factors that can be recursively processed. By iterating until convergence, the mean and accuracy of the local Gaussian observations are obtained.

[0011] The Kalman filter forward inference algorithm is used to perform forward filtering inference on the time factors until the entire time grid is traversed, and the posterior mean and covariance of each rank time factor are obtained.

[0012] After filtering through the entire time grid, RTS smoothing is performed on each time factor chain to obtain the optimal time trajectory for the entire time period.

[0013] By utilizing the obtained optimal time trajectory and combining it with the original observation labels, complete high-resolution data is generated.

[0014] Furthermore, the three-dimensional tensor structure is decomposed using the CP tensor decomposition method, expanding the time factor from a fixed vector to a trajectory function that varies continuously with time. Its dynamic characteristics are then modeled using a Gaussian process, thus obtaining a continuous-time tensor model that can be evaluated at any time point.

[0015] The CP tensor decomposition is used to represent the high-dimensional measurement tensor in a low-rank factor space as a combination of multiple linear subspaces:

[0016] ;

[0017] Among them, a r b represents the node mode factor; r Indicates the measurement type mode factor; c rRepresents the time pattern factor, where R is the number of ranks in the tensor CP decomposition;

[0018] By modeling with Gaussian processes, a continuous-time tensor model that can be evaluated at any time point is obtained:

[0019] ;

[0020] in, Represents the trajectory of node type factors. The measurement type factor trajectory is represented by ε(t), and the fitting noise of the model is represented by ε(t).

[0021] right , Reparameterization:

[0022] ;

[0023] Among them, a i,r b represents the static weight of node i in the r-th pattern; j,r Represents the static weight of measurement type j in the r-th mode; g r h r It is a common time basis function;

[0024] Define the common time trajectory:

[0025] ;

[0026] After eliminating the noise term, we finally obtain the simplified expression for the continuous-time tensor model:

[0027] .

[0028] Furthermore, the method to reduce the total time complexity from cubic to linear by transforming the Gaussian process GP corresponding to the kernel function into a state-space model through linear time-invariant stochastic differential equations is as follows:

[0029] The continuous-time definition of a Gaussian process GP:

[0030] ;

[0031] Among them, z r (t) represents the hidden state with time factor; A is the state transition matrix, L is the process noise influence matrix; H is the observation matrix; β r (t) represents standard white noise;

[0032] After discretization, at time step Δt:

[0033] ;

[0034] ;

[0035] Among them, z r,t Let e ​​represent the state of the r-th trajectory at discrete time t. AΔt Let Q(Δt) be the matrix exponent, and let Q(Δt) represent the covariance of the process noise accumulated up to the current time step.

[0036] By analyzing the hidden state variable z r,t The recursion enables the generation of time factor estimates at any time step:

[0037] ;

[0038] in, This represents continuous time factor estimation. Corresponding covariance estimation.

[0039] Furthermore, by utilizing Conditional Expectation Propagation (CEP) to approximate each observation as a local Gaussian term on the corresponding time factor, the complex joint observations are decomposed into several recursively processable independent approximation factors. The method for obtaining the mean and accuracy of the local Gaussian observations through iteration until convergence is as follows:

[0040] The core formula for Conditional Expectation Propagation (CEP) is:

[0041] ;

[0042] Among them, O t z represents all observations at time t. r,t Ω represents the state of the r-th trajectory at discrete time t; t N(.|μ,τ) represents the set of all observations in (i,j); -1 ) represents a Gaussian distribution, τ -1 Let y be the variance. ij (T k ) represents the observation at index (i,j) at time t, and a i,r b represents the static weight of node i in the r-th pattern; j,r This represents the static weight of measurement type j in the r-th mode. Represents the scalar value of the time factor;

[0043] We apply the Gaussian approximation so that the joint distribution can be decomposed into independent approximate distributions for each rank:

[0044] ;

[0045] in, For the site factor generated by the r-th trajectory in the current observation, each local term All are in Gaussian form;

[0046] During iteration:

[0047] Removing the old site from the r-th chain yields the cavity distribution:

[0048] ;

[0049] in, This is the posterior approximation of the trajectory of r at the current time.

[0050] Combining true likelihood:

[0051] ;

[0052] calculate Mean and covariance:

[0053] ;

[0054] ;

[0055] Update site parameters:

[0056] ;

[0057] ;

[0058] in, The mean of the site factor. This is the precision matrix of the site factor. It is the average value of the cavity. It is the cavity covariance;

[0059] Iterate until convergence to obtain the mean and accuracy of local Gaussian observations.

[0060] Furthermore, the method for obtaining the posterior mean and covariance of each rank time factor by using the Kalman filter forward inference algorithm to perform forward filtering inference on the time factors until the entire time grid is traversed is as follows:

[0061] The time-factor state transition equation is:

[0062] ;

[0063] The update equation is:

[0064] ;

[0065] ;

[0066] From the CEP output, extract the local Gaussian approximation of the time factor trajectory for the current batch and treat it as the equivalent observation information at the current moment;

[0067] Based on the predicted covariance With observation accuracy Calculate the update weight K for the current batch based on the relative size of the values. r,t Using K r,t The predicted mean is corrected, and the hidden states are updated to the posterior mean. :

[0068] ;

[0069] ;

[0070] After completing the mean update, combine K r,t With predicted covariance To obtain the posterior covariance P at the current moment. r,t :

[0071] ;

[0072] The current result ( , This is used as the prior input for the next time step until the entire time grid is traversed, obtaining the posterior mean and covariance of each rank time factor.

[0073] Furthermore, after filtering through the entire time grid, RTS smoothing is performed on each time factor chain to obtain the optimal time trajectory for the entire time period.

[0074] Iterate backwards from the last moment, constructing a smooth gain matrix for each moment. By smoothing the gain The updated smoothed mean is obtained. and the updated smoothed covariance :

[0075] ;

[0076] ;

[0077] ;

[0078] By recursively working backwards until the starting moment, the optimal time trajectory for the entire time period can be obtained. And obtain the full-time posterior of the trajectories of each time factor:

[0079] ;

[0080] .

[0081] A data inpainting and measurement alignment system based on continuous-time tensor decomposition includes:

[0082] The data preprocessing module is used to acquire raw measurement data and preprocess the raw measurement data to unify the raw measurement data into a standardized time batch structure and sampling interval.

[0083] The continuous-time tensor model acquisition and processing module is used to obtain a three-dimensional tensor structure with nodes, measurement types, and continuous time dimensions based on preprocessed data. It decomposes the three-dimensional tensor structure using the CP tensor decomposition method, expands the time factor from a fixed vector to a trajectory function that changes continuously with time, and models its dynamic characteristics through a Gaussian process, thereby obtaining a continuous-time tensor model that can be evaluated at any time point. The Gaussian process GP corresponding to the kernel function is transformed into a state-space model through a linear time-invariant stochastic differential equation, thereby reducing the total time complexity from cubic to linear.

[0084] The module for obtaining the mean and accuracy of local Gaussian observations is used to approximate each observation as a local Gaussian term on the corresponding time factor using conditional expectation propagation (CEP). This decomposes complex joint observations into several independent approximation factors that can be recursively processed. By iterating until convergence, the mean and accuracy of local Gaussian observations are obtained.

[0085] The Kalman filter forward inference module is used to perform forward filtering inference on time factors using the Kalman filter forward inference algorithm until the entire time grid is traversed, and to obtain the posterior mean and covariance of each rank time factor.

[0086] The RTS smoothing module is used to smooth each time factor chain after filtering through the entire time grid to obtain the optimal time trajectory for the entire time period.

[0087] The high-resolution data acquisition module is used to generate complete high-resolution data by utilizing the obtained optimal time trajectory and combining it with the original observation labels.

[0088] A non-transitory computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the data repair and measurement alignment system based on continuous-time tensor decomposition as described above.

[0089] An electronic device includes a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor, when executing the program, implements the data repair and measurement alignment system based on continuous-time tensor decomposition as described above.

[0090] The beneficial technical effects of this invention are as follows:

[0091] This invention proposes a high-resolution time alignment and repair method for distribution network measurement data based on tensor factor decomposition and continuous-time trajectory modeling. This method does not rely on complex physical models and achieves unified modeling of multi-source asynchronous measurement data by separating node factors, measurement type factors, and time factors. Utilizing a continuous-time state-space model, conditional expectation propagation, and Kalman filter-RTS smoothing mechanism, it can adaptively acquire time evolution patterns and reconstruct smooth, continuous, high-resolution time series even in the presence of missing points, outliers, and different sampling frequencies. Simultaneously, a fidelity fusion strategy is introduced to ensure the complete preservation of original valid observations and to accurately complete missing and low-frequency sampling points. This method can achieve complete reconstruction of the entire network's measurement data at a unified time scale, providing a high-quality data foundation for the operation monitoring, state estimation, and analysis of distribution substations. Attached Figure Description

[0092] Figure 1 This is a flowchart of the data repair and measurement alignment method based on continuous-time tensor decomposition in Embodiment 1 of the present invention;

[0093] Figure 2 This is the three-dimensional distribution of all node voltage measurements in Embodiment 2 of the present invention;

[0094] Figure 3 This is the three-dimensional distribution of all node current measurements in Embodiment 2 of the present invention;

[0095] Figure 4 This is the three-dimensional distribution of the active power measurement values ​​of all nodes in Embodiment 2 of the present invention;

[0096] Figure 5 This is a diagram showing the effect of voltage one-week data restoration in Embodiment 2 of the present invention;

[0097] Figure 6 This is a diagram showing the effect of current cycle data restoration in Embodiment 2 of the present invention;

[0098] Figure 7 This is a diagram showing the effect of restoring one week's active power data in Embodiment 2 of the present invention;

[0099] Figure 8 This is a summary of the reconstruction results of all node voltage data with uncertainties in Embodiment 2 of the present invention;

[0100] Figure 9 (a) is a line graph showing the change of mean absolute error (MSE) with missing rate in Embodiment 2 of the present invention;

[0101] Figure 9 (b) is a line graph showing the change of root mean square error (RMSE) with missing rate in Embodiment 2 of the present invention;

[0102] Figure 10(a) is a line graph showing the change of mean absolute percentage error (MAPE) with missing rate in Embodiment 2 of the present invention;

[0103] Figure 10(b) is a line graph showing the change of the coefficient of determination R2 with the missing rate in Embodiment 2 of the present invention. Detailed Implementation

[0104] The data repair and measurement alignment method based on continuous-time tensor decomposition provided by the present invention will be described more clearly and completely below with reference to the accompanying drawings:

[0105] Example 1

[0106] like Figure 1 As shown, this embodiment provides a data repair and measurement alignment method based on continuous-time tensor decomposition, which includes the following steps:

[0107] S1. Obtain the raw measurement data and preprocess the raw measurement data to unify the raw measurement data into a standardized time batch structure and sampling interval.

[0108] The raw measurement data consists of multiple nodes n i (i=1,2,...,N) and multiple measurement types m j Composed of two dimensions (j=1,2,...,M), the measurement frequencies of each node differ; the sampling type is defined as:

[0109] ;

[0110] Wherein, high represents low-resolution sampling and low represents high-resolution sampling; in this embodiment, it is assumed that the high-resolution data sampling interval is 15 minutes and the low-resolution sampling interval is 60 minutes.

[0111] Preprocessing of raw measurement data includes the following aspects:

[0112] 1) Time series standardization and coding:

[0113] Map all measurement timestamps to a continuous real time axis t∈[0,1]:

[0114] ;

[0115] in, T represents the original sampling time. min and T max These are the global minimum and maximum times, respectively; after determining the final output data resolution, define the global time set:

[0116] ;

[0117] in, In this embodiment, the time interval for the final output of high-resolution data is assumed to be 15 minutes; all original sampling times will be aligned to the nearest grid point to form a unified discrete time step.

[0118] Each time step corresponds to a unique number BN t The encoding formula is:

[0119] ;

[0120] This results in a strictly monotonically increasing time index sequence {BN}. t};

[0121] 2) Asynchronous sampling data insertion and labeling:

[0122] For node i, the observation y of measurement type j ij (T k ), mapped to triples:

[0123] ;

[0124] If the node sampling type is high, it will be recorded every 15 minutes; if it is low, it will be recorded only once per hour.

[0125] Define the observation indicator function:

[0126] ;

[0127] If a node has an observation at time t, it is marked as 1; otherwise, it is marked as 0; thus forming a sparse observation matrix and a mask matrix.

[0128] 3) Batch construction and streaming input mechanism:

[0129] The continuous time series is divided into several time batches Bn, each batch containing a set of time indices:

[0130] ;

[0131] The batch update interval can be set to an equal time interval or a sliding window. If the current batch contains any high-sampled data, the batch type is set to "high-frequency batch"; if it only contains low-sampled data, the type is "low-frequency batch". For batches containing both types of data, a time-aligned interpolation operation is performed to make high-frequency samples dominant.

[0132] The structure of each batch is defined as follows:

[0133] ;

[0134] Among them; P n This represents the number of samples in this batch.

[0135] 4) Generation of batch index and status input

[0136] To compensate for the temporal sparsity of low-frequency nodes, missing records are generated at relevant grid positions before the hour to ensure consistent temporal coverage with high-frequency nodes. In this embodiment, the hourly observation time for the low-frequency channel is t. low Then, its three adjacent 15-minute missing measurement points are defined as follows:

[0137] ;

[0138] and the δ corresponding to these times ij (t)=0 is recorded in the tensor index set for later model repair; if δ ij (t)=1, this sample is fixed as the observation anchor point in the subsequent tensor repair stage and does not participate in the re-estimation, thus ensuring data fidelity.

[0139] Combine batch number n with batch time The common encoding is used as the model input vector:

[0140] ;

[0141] This is used for updating the time transition function F(Δt) and process noise Q(Δt) for subsequent state-space Gaussian processes.

[0142] S2. Based on the preprocessed data, a three-dimensional tensor structure with nodes, measurement types, and continuous time dimensions is obtained. The three-dimensional tensor structure is decomposed using the CP tensor decomposition method. The time factor is extended from a fixed vector to a trajectory function that changes continuously with time. Its dynamic characteristics are modeled by Gaussian process, thereby obtaining a continuous time tensor model that can be evaluated at any time point. The Gaussian process GP corresponding to the kernel function is transformed into a state-space model through linear time-invariant stochastic differential equations, thereby reducing the total time complexity from cubic to linear.

[0143] After data preprocessing, the input data has been standardized into a time-batch structure and a uniform sampling interval. At this point, measurement data with multiple nodes, multiple measurement types, and continuous time dimensions can be organized into a three-dimensional tensor structure:

[0144] ;

[0145] Where I represents the number of nodes, M represents the number of measurement types, T represents the uniform time series length, and X represents the number of nodes. i,j,t This represents the value of node i at time t corresponding to measurement type j; the mask matrix output by the data preprocessing section is used to identify observed and missing elements.

[0146] To achieve spatiotemporal joint compression and missing measurement completion, this invention employs the CP tensor decomposition model, representing the high-dimensional measurement tensor in a low-rank factor space as a combination of multiple linear subspaces, then:

[0147] ;

[0148] Among them, a r b represents the node mode factor; r Indicates the measurement type mode factor; c r Represents the time pattern factor, and R is the number of ranks in the tensor CP decomposition.

[0149] Traditional CP decomposition typically treats the time dimension as a static dimension, describing time patterns in a fixed way. However, in the dynamic measurement scenario of distribution networks, the system state changes continuously over time, and there are time-varying coupling characteristics between nodes and measurements.

[0150] If the static matrix method is still used, the change of the time factor can only be described discretely in a limited dimension, which cannot characterize the smooth evolution of measurements such as node power and voltage, nor can it guarantee continuity when the sampling interval is inconsistent.

[0151] To address this problem, this invention proposes to extend the time factor from a fixed vector to a trajectory function that changes continuously with time, and to model its dynamic characteristics through a Gaussian process, thereby obtaining a continuous-time tensor model that can be evaluated at any point in time.

[0152] By modeling with Gaussian processes, a continuous-time tensor model that can be evaluated at any time point is obtained:

[0153] ;

[0154] in, Represents the trajectory of node type factors. The measurement type factor trajectory is represented by ε(t), and the fitting noise of the model is represented by ε(t).

[0155] right , Reparameterization:

[0156] ;

[0157] Among them, a i,r b represents the static weight of node i in the r-th pattern; j,r Represents the static weight of measurement type j in the r-th mode; g r h r It is a common time basis function;

[0158] Define the common time trajectory:

[0159] ;

[0160] After eliminating the noise term, we finally obtain the simplified expression for the continuous-time tensor model:

[0161] ;

[0162] This step centralizes all time dependencies into c. r (t), the node and measurement type two-mode factors remain static, effectively reducing the number of parameters and improving the stability of filtering. This reparameterization does not change the expressive power of the model.

[0163] For each c r (t), introducing the Gaussian process prior:

[0164] ;

[0165] Kernel functions using Matérn 3 / 2 form

[0166] ;

[0167] Where, σ 2 ρ is the amplitude parameter, and ρ is the time scale parameter; this kernel ensures smoothness while allowing for moderate time abrupt changes.

[0168] While directly using GP to model continuous time trajectories is accurate, its computational complexity is high. When the number of time points is large, it is necessary to invert the covariance matrix of all time points at once, which has extremely high computational and storage costs. To reduce computational complexity, this invention transforms the GP corresponding to the Matérn kernel into a state-space model through linear time-invariant stochastic differential equations, thereby reducing the total time complexity from cubic to linear.

[0169] The continuous-time definition of a Gaussian process GP:

[0170] ;

[0171] Among them, z r (t) represents the hidden state with time factor; A is the state transition matrix, L is the process noise influence matrix; H is the observation matrix; β r (t) represents standard white noise; where

[0172] ;

[0173] In the formula, λ is the system attenuation parameter.

[0174] After discretization, at time step Δt:

[0175] ;

[0176] ;

[0177] Among them, z r,t Let e ​​represent the state of the r-th trajectory at discrete time t. AΔt Let Q(Δt) be the matrix exponent, and let Q(Δt) represent the covariance of the process noise accumulated up to the current time step.

[0178] In this way, approximate inference of the general GP can be achieved by relying only on state updates of adjacent time steps, thereby significantly reducing the computational cost while maintaining dynamic smoothness.

[0179] By analyzing the hidden state variable z r,t The recursion enables the generation of time factor estimates at any time step:

[0180] ;

[0181] in, This represents continuous time factor estimation. Corresponding covariance estimation;

[0182] According to the formula, the dynamic tensor form is time-continuous. This expression can calculate the predicted value at any point in time, realize the time-continuous completion and reconstruction of the measurement data of the distribution network node, and provide the initial state and prior support for the subsequent CEP Kalman filtering and RTS smoothing.

[0183] S3. Using Conditional Expectation Propagation (CEP), each observation is approximated as a local Gaussian term on the corresponding time factor, so that the complex joint observation is decomposed into several independent approximation factors that can be recursively processed. By iterating until convergence, the mean and accuracy of the local Gaussian observations are obtained.

[0184] Continuous modeling of time factor trajectories was achieved by combining CP tensor decomposition with state-space Gaussian process (GP-SSM). However, due to the nonlinear product structure of the tensor observations, there is multiplicative coupling between different rank factors, which makes it impossible to directly decompose the posterior distribution of each time batch into an independent Gaussian form, thus making it impossible to directly apply standard Kalman filtering.

[0185] Based on this, the present invention introduces the Conditional Expectation Propagation (CEP) algorithm into the dynamic tensor decomposition framework, approximating the multimodal coupled observation probability as a local Gaussian term on the trajectory of each time factor, thereby realizing parallel and decoupled filtering and updating.

[0186] The core formula for Conditional Expectation Propagation (CEP) is:

[0187] ;

[0188] Among them, O t z represents all observations at time t. r,tΩ represents the state of the r-th trajectory at discrete time t; t N(.|μ,τ) represents the set of all observations in (i,j); -1 ) represents a Gaussian distribution, τ -1 Let y be the variance. ij (T k ) represents the observation at index (i,j) at time t, and a i,r b represents the static weight of node i in the r-th pattern; j,r This represents the static weight of measurement type j in the r-th mode. This represents the scalar value of the time factor.

[0189] We apply the Gaussian approximation so that the joint distribution can be decomposed into independent approximate distributions for each rank:

[0190] ;

[0191] Each local item All are in Gaussian form:

[0192] ;

[0193] in, Let r be the site factor generated by the current observation for the r-th trajectory. The mean of the site factor. This is the precision matrix of the site factor;

[0194] parameter It is obtained by matching the first and second moments of the approximate distribution with the true distribution.

[0195] During iteration:

[0196] Removing the old site from the r-th chain yields the cavity distribution:

[0197] ;

[0198] in, This is the posterior approximation of the trajectory of r at the current time.

[0199] Combining true likelihood:

[0200] ;

[0201] calculate Mean and covariance:

[0202] ;

[0203] ;

[0204] Update site parameters:

[0205] ;

[0206] ;

[0207] in, It is the average value of the cavity. It is the cavity covariance;

[0208] Iterate until convergence to obtain the mean and accuracy of local Gaussian observations.

[0209] S4. Use the Kalman filter forward inference algorithm to perform forward filtering inference on the time factors until the entire time grid is traversed, and obtain the posterior mean and covariance of each rank time factor.

[0210] Specifically, before entering the current time batch, the mean of the hidden states obtained by smoothing or filtering in the previous time step is used. Covariance The predicted mean at the current time is obtained through state transition. With predicted covariance This serves as a priori for this batch;

[0211] The time-factor state transition equation is:

[0212] ;

[0213] The update equation is:

[0214] ;

[0215] ;

[0216] From the CEP output, extract the local Gaussian approximation of the time factor trajectory for the current batch, i.e., the site mean. and site precision matrix This is considered as equivalent observation information at the current moment; where the Gaussian form and mean of the site are... and site precision matrix As already given above.

[0217] Compare the predicted mean and CEP site average The difference between them yields the observed residual vector:

[0218] ;

[0219] The reliability of the residuals is assessed by combining the predicted observation covariance:

[0220] ;

[0221] Among them, S r,t To observe the covariance.

[0222] Based on the predicted covariance With observation accuracy Calculate the update weight K for the current batch based on the relative size of the values. r,t Using K r,t The predicted mean is corrected, and the hidden states are updated to the posterior mean. :

[0223] ;

[0224] ;

[0225] After completing the mean update, combine K r,t With predicted covariance To obtain the posterior covariance P at the current moment. r,t :

[0226] ;

[0227] The current result ( , This is used as the prior input for the next time step until the entire time grid is traversed, obtaining the posterior mean and covariance of each rank time factor.

[0228] S5. After filtering through the entire time grid, perform RTS smoothing on each time factor chain to obtain the optimal time trajectory for the entire time period.

[0229] Iterate backwards from the last moment, constructing a smooth gain matrix for each moment. By smoothing the gain The updated smoothed mean is obtained. and the updated smoothed covariance :

[0230] ;

[0231] ;

[0232] ;

[0233] By recursively working backwards until the starting moment, the optimal time trajectory for the entire time period can be obtained. And obtain the full-time posterior of the trajectories of each time factor:

[0234] ;

[0235] ;

[0236] S6. Using the obtained optimal time trajectory and combining it with the original observation labels, generate complete high-resolution data.

[0237] Through the smoothed time factor trajectory Sampling on a time set allows for the generation of predictions at any point in time; all nodes and measurement types are reconstructed on a full-resolution time scale.

[0238] ;

[0239] In the above formula It is a reconstructed and repaired value. To reconstruct the time factor trajectory; and to ensure that the final output of the system retains both the original measurement information and continuous completion features, a fidelity fusion strategy is defined:

[0240] ;

[0241] in, This represents the final output after high-fidelity fusion. Represents the original observation value. This indicates that there was an observation at that moment; To preserve the accuracy of the weights, this strategy retains the original values ​​for the whole-point samples in the low-frequency channel, and fills in the missing measurement points using the model.

[0242] Example 2

[0243] Taking measured data from a low-voltage distribution substation in Henan Province as an example; due to varying sampling requirements at different nodes of the power grid, different sampling devices are used. The data collection interval for smart meters is 1 hour, while the data collection interval for smart fusion terminals is 15 minutes. For ease of verification, relatively complete data collection data from 10 smart fusion terminals were selected, renumbered, and used as the data basis for algorithm verification.

[0244] Figure 2 , 3 Figure 4 is a diagram of the complete node data. The x-axis represents different nodes, the y-axis represents sampling time, and the z-axis represents sampled data. The load changes at different nodes and times are shown for one week's worth of data. Figure 2 , 3 As shown in Figure 4;

[0245] The data is processed by performing low-resolution missing data processing on half of the nodes to simulate the actual situation of different sampling resolutions. Then, random missing data processing is performed on all nodes to simulate the situation where measurement data is missing due to random problems such as device failure in reality.

[0246] The missing data after processing was repaired using the method in Example 1. Based on the method of the present invention, the data repair result with a tensor decomposition rank of 3 and a random missing rate of 20% is as follows: Figure 5 , 6 As shown in Figures 7 and 8. Figure 5 , 6 Figures 7 and 8 respectively show the repair effects on the same node in three measurement dimensions: voltage, current, and active power. The figures simultaneously plot the original complete data, the low-resolution observation data after missing data processing, and the results after repair using this method, thus providing a direct comparison of the differences before and after repair and clearly demonstrating the repair effect of this invention. Figure 8 The overall results show the reconstruction results of the voltage of each node in the example and their uncertainty range, demonstrating the effectiveness and stability of the proposed method in recovering voltage trajectories at the network scale.

[0247] The effectiveness of data restoration varies with the missing data rate. Here, we use MAE (Mean Absolute Error, measuring the average absolute deviation between the restored value and the true value), RMSE (Root Mean Square Error, measuring the overall magnitude of the restoration error), MAPE (%) (Mean Absolute Percentage Error, reflecting the average relative error percentage of the restored value relative to the true value), and R² (Coefficient of Determination, the closer to 1, the better the fit) to quantify the data restoration effect. Table 1 summarizes the evaluation indicators for data restoration of transformer substation nodes under different missing data rates. Figure 9(a) , 9(b) Tables 10(a) and 10(b) show the line trends of MAE, RMSE, MAPE, and R² as a function of the missing data rate. As can be seen from Table 1 and the related figures, when the missing data rate gradually increases from 20% to 60%, MAE, RMSE, and MAPE (%) continuously rise, indicating that the higher the degree of missing data, the greater the repair error. Meanwhile, R² gradually decreases (from approximately 0.912 to 0.797), indicating that the fit between the repaired results and the actual data weakens as the missing data rate increases. Nevertheless, under the high missing data rate of 60%, R² remains close to 0.8, indicating that this method can still maintain good reconstruction accuracy and stability in high-proportion missing data scenarios, demonstrating good robustness.

[0248] Table 1. Results of Data Repair Indicators for Substation Nodes under Different Missing Rates

[0249] Missing rate MAE RMSE MAPE (%) <![CDATA[R 2 ]]> 20% 0.016424 0.028159 3.903 0.911774 30% 0.018949 0.032636 4.653 0.873823 40% 0.021767 0.036547 5.510 0.869399 50% 0.025375 0.041835 6.517 0.833429 60% 0.030029 0.048488 7.772 0.797403

[0250] Example 3

[0251] This embodiment provides a data repair and measurement alignment system based on continuous-time tensor decomposition, including:

[0252] The data preprocessing module is used to acquire raw measurement data and preprocess the raw measurement data to unify the raw measurement data into a standardized time batch structure and sampling interval.

[0253] The continuous-time tensor model acquisition and processing module is used to obtain a three-dimensional tensor structure with nodes, measurement types, and continuous time dimensions based on preprocessed data. It decomposes the three-dimensional tensor structure using the CP tensor decomposition method, expands the time factor from a fixed vector to a trajectory function that changes continuously with time, and models its dynamic characteristics through a Gaussian process, thereby obtaining a continuous-time tensor model that can be evaluated at any time point. The Gaussian process GP corresponding to the kernel function is transformed into a state-space model through a linear time-invariant stochastic differential equation, thereby reducing the total time complexity from cubic to linear.

[0254] The module for obtaining the mean and accuracy of local Gaussian observations is used to approximate each observation as a local Gaussian term on the corresponding time factor using conditional expectation propagation (CEP). This decomposes complex joint observations into several independent approximation factors that can be recursively processed. By iterating until convergence, the mean and accuracy of local Gaussian observations are obtained.

[0255] The Kalman filter forward inference module is used to perform forward filtering inference on time factors using the Kalman filter forward inference algorithm until the entire time grid is traversed, and to obtain the posterior mean and covariance of each rank time factor.

[0256] The RTS smoothing module is used to smooth each time factor chain after filtering through the entire time grid to obtain the optimal time trajectory for the entire time period.

[0257] The high-resolution data acquisition module is used to generate complete high-resolution data by utilizing the obtained optimal time trajectory and combining it with the original observation labels.

[0258] A non-transitory computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the data repair and measurement alignment system based on continuous-time tensor decomposition as described above.

[0259] Furthermore, the present invention adopts the following technical solution:

[0260] An electronic device includes a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor, when executing the program, implements the data repair and measurement alignment system based on continuous-time tensor decomposition as described above.

[0261] From the above description of the embodiments, those skilled in the art will clearly understand that the facilities of the present invention can be implemented using software plus necessary general-purpose hardware platforms. Embodiments of the present invention can be implemented using existing processors, or by dedicated processors used for this or other purposes for suitable systems, or by hardwired systems. Embodiments of the present invention also include non-transitory computer-readable storage media, comprising machine-readable media for carrying or having machine-executable instructions or data structures stored thereon; such machine-readable media can be any available medium accessible by a general-purpose or special-purpose computer or other machine with a processor. For example, such machine-readable media can include RAM, ROM, EPROM, EEPROM, CD-ROM or other optical disc storage, disk storage or other magnetic storage devices, or any other medium that can be used to carry or store the required program code in the form of machine-executable instructions or data structures and is accessible by a general-purpose or special-purpose computer or other machine with a processor. When information is transmitted or provided to a machine via a network or other communication connection (hardwired, wireless, or a combination of hardwired and wireless), that connection is also considered a machine-readable medium.

[0262] The technical solution of the present invention has been described above with reference to the preferred embodiments shown in the accompanying drawings. However, it will be readily understood by those skilled in the art that the scope of protection of the present invention is obviously not limited to these specific embodiments. Without departing from the principles of the present invention, those skilled in the art can make equivalent changes or substitutions to the relevant technical features, and the technical solutions after such changes or substitutions will all fall within the scope of protection of the present invention.

Claims

1. A method for data repair and metrology alignment based on continuous-time tensor decomposition, characterized in that, The method comprises the following steps: Obtain the original measurement data, and preprocess the original measurement data to unify the original measurement data into a standardized time batch structure and a sampling interval; Based on the preprocessed data, a three-dimensional tensor structure of nodes, measurement types and continuous time dimensions is obtained, the three-dimensional tensor structure is decomposed by using a CP tensor decomposition method, a time factor is expanded from a fixed vector to a trajectory function continuously changing with time, and a dynamic characteristic of the trajectory function is modeled by using a Gaussian process, so that a continuous time tensor model that can be evaluated at any time point is obtained, a Gaussian process GP corresponding to a kernel function is converted into a state space model by using a linear time-invariant random differential equation, so that the full time complexity is reduced from cubic to linear; Each observation item is approximated as a local Gaussian item on a corresponding time factor by using a conditional expectation propagation CEP, so that a complex joint observation is decomposed into several independent approximate factors that can be recursively processed, and local Gaussian observation mean values and precisions are obtained by iteration to convergence; A Kalman filter forward inference algorithm is used to perform forward filtering inference on the time factor until the entire time grid is traversed, so that posterior mean values and covariances of the time factors of all ranks are obtained; After filtering the entire time grid, RTS smoothing is performed on each time factor chain to obtain optimal time trajectories in the entire time period; The optimal time trajectories are used to generate complete high-resolution data in combination with original observation labels.

2. The continuous-time tensor decomposition based data repair and metrology alignment method of claim 1, wherein, The method for obtaining a continuous time tensor model that can be evaluated at any time point by using a CP tensor decomposition method to decompose the three-dimensional tensor structure, expanding a time factor from a fixed vector to a trajectory function continuously changing with time, and modeling a dynamic characteristic of the trajectory function is as follows: A high-dimensional measurement tensor is represented as a combination of multiple linear subspaces in a low-rank factor space by using CP tensor decomposition: A continuous time tensor model that can be evaluated at any time point is obtained by modeling using a Gaussian process: ； wherein a r represents a node pattern factor; b r represents a measurement type pattern factor; c r represents a time pattern factor, R is the number of tensor CP decomposition ranks; A time common trajectory is defined: ； wherein, denotes a node type factor trajectory, denotes a measurement type factor trajectory, and ε(t) denotes a model's residual noise; Parametrize the , : ； where a i,r represents the static weight of node i in the rth mode; b j,r represents the static weight of measurement type j in the rth mode; g r , h r is a common time base function; After removing noise terms, a simplified expression of the continuous time tensor model is finally obtained: ； The method for converting a Gaussian process GP corresponding to a kernel function into a state space model by using a linear time-invariant random differential equation, so that the full time complexity is reduced from cubic to linear, is as follows: 。 3. The continuous-time tensor decomposition based data repair and metrology alignment method of claim 1, wherein, A continuous time definition of a Gaussian process GP is as follows: After discretization, at a time step Δt, the following is obtained: ； where z r (t) is the hidden state of the time factor; A is the state transition matrix, L is the process noise influence matrix; H is the observation matrix; β r (t) is a standard white noise; The method for approximating each observation item as a local Gaussian item on a corresponding time factor by using a conditional expectation propagation CEP, so that a complex joint observation is decomposed into several independent approximate factors that can be recursively processed, and local Gaussian observation mean values and precisions are obtained by iteration to convergence, is as follows: ； ； where z r,t denotes the state of the rth trajectory at discrete time t, e AΔt is the matrix exponential, and Q(Δt) denotes the covariance of the process noise accumulated up to the current time. By recursion on the hidden state variable z r,t the time factor estimate is generated at any time step: ； wherein, denotes the continuous-time factor estimate, corresponds to the covariance estimate.

4. The continuous-time tensor decomposition based data repair and metrology alignment method of claim 1, wherein, A conditional expectation propagation CEP core formula is as follows: Gaussian approximation is performed, so that a joint distribution can be decomposed into an independent approximate distribution corresponding to each rank: ； where O t represents all observations of time t, z r,t represents the state of the rth trajectory at discrete time t; Ω t represents the observation set in all (i,j); N(.|μ,τ -1 ) represents a Gaussian distribution, τ -1 is a variance, y ij (T k ) represents the observation value at time t, index (i,j), a i,r represents the static weight of node i in the rth mode; b j,r represents the static weight of measurement type j in the rth mode, represents a time factor scalar value; When iteration is performed, the following is obtained: ； where, site factor for the rth trajectory at the current observation, each local term is in Gaussian form; An old site of the rth chain is removed to obtain a cavity distribution: A true likelihood is combined: ； wherein, is the posterior approximation of the trajectory at the current time r; A site parameter is updated: ； Computing the mean and covariance of: ； ； Iteration is performed to convergence, and local Gaussian observation mean values and precisions are obtained. ； ； wherein, is the mean of the site factor, is the precision matrix of the site factor, is the mean of the cavity, is the covariance of the cavity; The method for performing forward filtering inference on the time factor by using a Kalman filter forward inference algorithm until the entire time grid is traversed, so that posterior mean values and covariances of the time factors of all ranks are obtained, is as follows:

5. The continuous-time tensor decomposition based data repair and metrology alignment method of claim 4, wherein, A time factor state transition equation is as follows: ​ ； The update equation is: ； ； From the CEP output, the local Gaussian approximation of the time factor trajectory of the current batch is extracted, which is regarded as the equivalent observation information at the current time; According to the predicted covariance and the relative size of the observation accuracy , the update weight K of the current batch is calculated r,t , the predicted mean is corrected using K r,t , and the hidden state is updated to the posterior mean : ； ； After the mean update is completed, combine K r,t with the predicted covariance to get the posterior covariance P r,t at the current time instant. ； The current obtained , ) is taken as the input prior of the next time step, until the entire time grid is traversed, obtaining the posterior mean and covariance of each rank time factor.

6. The continuous-time tensor decomposition based data repair and metrology alignment method of claim 5, wherein, After filtering through the entire time grid, RTS smoothing is performed on each time factor chain to obtain the optimal time trajectory of the entire time period. iterating backwards from the last time instant onwards, constructing a smoothing gain matrix for each time instant , obtaining an updated smoothed mean and an updated smoothed covariance by smoothing gain ​ ； ； ； Thus, the reverse recursion is performed until the initial time, obtaining the optimal time trajectory for the entire time period and obtaining the full-time posterior of each time factor trajectory ； 。 7. A data repair and metrology alignment system based on continuous-time tensor decomposition, characterized in that, The method comprises: A data preprocessing module is configured to obtain original measurement data and preprocess the original measurement data, and unify the original measurement data into a standardized time batch structure and sampling interval. A continuous time tensor model acquisition and processing module is configured to obtain a three-dimensional tensor structure of nodes, measurement types and continuous time dimensions based on the preprocessed data, decompose the three-dimensional tensor structure by using a CP tensor decomposition method, expand the time factor from a fixed vector to a trajectory function that changes continuously over time, and model the dynamic characteristics of the trajectory function by using a Gaussian process, so as to obtain a continuous time tensor model that can be evaluated at any time point. A local Gaussian observation mean and precision acquisition module is configured to approximate each observation item to a local Gaussian item on a corresponding time factor by using conditional expectation propagation (CEP), so that a complex joint observation is decomposed into several independent approximate factors that can be processed recursively. A Kalman filter forward inference module is configured to perform forward filtering inference on the time factor by using a Kalman filter forward inference algorithm until the entire time grid is traversed, so as to obtain the posterior mean and covariance of each rank time factor. An RTS smoothing module is configured to perform RTS smoothing on each time factor chain after filtering through the entire time grid, so as to obtain the optimal time trajectory of the entire time period. A high-resolution data acquisition module is configured to generate complete high-resolution data by using the obtained optimal time trajectory and combining the original observation label.

8. A non-transitory computer-readable storage medium having stored thereon a computer program, characterized in that, The computer program is executed by the processor to implement the data repair and measurement alignment system based on continuous time tensor decomposition as claimed in any one of claims 1 to 6.

9. An electronic device comprising a memory, a processor, and a computer program stored on the memory and executable on the processor, characterized in that, The processor executes the program to implement the data repair and measurement alignment system based on continuous time tensor decomposition as claimed in any one of claims 1 to 6.

Images