Method and device for recovering sensing data of electric power internet of things based on improved SVT (Scalable Vector Transform)

Abstract

The invention discloses a power internet of things sensing data recovery method and device based on an improved SVT. The method comprises the following steps: performing phase-space reconstruction on a single-variable sequence containing missing value data of the power Internet of Things, and generating a reconstruction input matrix; the zero matrix is adopted to initialize the reconstructed input matrix, and a similar matrix is obtained; an improved SVT algorithm is adopted, the similar matrix is updated according to the input matrix, and an estimation matrix is obtained; and reconstructing the estimation matrix to obtain recovered one-dimensional time sequence data.

Description

Technical Field

[0001] This invention relates to the field of power IoT sensing data recovery technology, and more specifically, to a power IoT sensing data recovery method and apparatus based on an improved SVT. Background Technology

[0002] The Internet of Things (IoT) is a network connecting objects, also known as a sensor network. As a relatively new concept, the IoT lacks a unified international standard for its system architecture, leading to varying understandings of the concept across different countries and sectors. Currently, there are three internationally accepted definitions of the IoT. In my country, the generally accepted definition is: The IoT refers to a vast network that connects objects to the internet through various information sensing devices such as RFID (Radio Frequency Identification), infrared sensors, GPS, and laser scanners, following agreed-upon protocols and corresponding interfaces. This network facilitates information exchange and communication, enabling intelligent identification, location, tracking, monitoring, and management.

[0003] Clearly, the current application status and future development of the Internet of Things (IoT) vary across different industries, with varying degrees of demand and different focuses. The IoT is an inevitable product of the socialization and intelligentization of production; it is an organic combination of modern information network technology and traditional commodity markets, promoting social and economic development and changing lifestyles.

[0004] Essentially, the Internet of Things (IoT) has three characteristics: first, the characteristics of the Internet, meaning that the network must be able to interconnect and communicate with objects that need to be connected; second, the characteristics of identification and communication, meaning that the "objects" included in the IoT must have the functions of automatic identification and machine-to-machine (M2M) communication; and third, the characteristics of intelligence, meaning that the network system should have the characteristics of automation, self-feedback and intelligent control.

[0005] In recent years, the widespread application of the Internet of Things (IoT) has enabled intelligent management and control of goods, improving their utilization efficiency, saving labor costs and energy, and making it easier for people to access information about goods. From an economic perspective, the development of IoT technology not only ensures quality in production, facilitates orderly and efficient distribution, optimizes resource allocation, and significantly improves consumer safety, but also fosters emerging industries, new jobs, and new occupational categories. Furthermore, the wide range of applications, high demand, and significant room for development of IoT technology allow it to provide a series of solutions for optimizing management in areas such as intelligent traffic management, intelligent security, intelligent manufacturing, smart grid monitoring, intelligent healthcare, intelligent living services, intelligent safety production management, and intelligent environmental protection and energy conservation. It can also improve enterprises' R&D, design, manufacturing integration, and marketing service levels, significantly reduce production, operation, and social management costs, and help control energy consumption, reduce pollution emissions, and transform economic development patterns.

[0006] With the continuous development and widespread application of IoT technology, various industries are actively exploring how to utilize IoT to achieve intelligent data collection, transmission, processing, and application. In the energy sector, especially the power sector, the application of IoT technology has become one of the important means to improve energy efficiency and optimize energy supply and demand matching. As one of the important indicators of power system operation, the accurate collection and effective processing of power load data is a pressing technical problem that needs to be solved to ensure the safe and stable operation of the power system. Summary of the Invention

[0007] To address the shortcomings of existing technologies, this invention provides a method and apparatus for recovering power IoT sensing data based on an improved SVT (Static Dynamics Transmission Device).

[0008] According to one aspect of the present invention, a power IoT sensing data recovery method based on improved SVT is provided, comprising:

[0009] Phase space reconstruction is performed on univariate sequences of power IoT data containing missing values ​​to generate a reconstruction input matrix;

[0010] The reconstructed input matrix is ​​initialized with a zero matrix to obtain a similar matrix;

[0011] An improved SVT algorithm is used to update similar matrices based on the input matrix to obtain the estimated matrix;

[0012] The estimated matrix is ​​reconstructed to obtain the recovered one-dimensional time series data.

[0013] Alternatively, the expression for reconstructing the input matrix is:

[0014] M input =M known +Munknown

[0015] In the formula, M imput M is the input matrix before singular value decomposition; known and M unknown These are the matrices representing the known and unknown parts of the input matrix, respectively.

[0016] Optionally, an improved SVT algorithm is used to update similar matrices based on the input matrix to obtain an estimated matrix, including:

[0017] Step 1: Perform singular value decomposition on the similar matrix to obtain the singular values ​​of the similar matrix and the singular vectors corresponding to the singular values;

[0018] Step 2: Perform soft thresholding on the singular values ​​according to the preset singular value threshold to obtain the processed singular values;

[0019] Step 3: Reconstruct the matrix based on the processed singular values ​​and corresponding singular vectors to obtain the estimated matrix;

[0020] Step 4: Determine the update step size using a variable step size adaptive algorithm, and use the update step size to update the similar matrix based on the estimated matrix and the input matrix to obtain the updated similar matrix. Return to Step 1 for iterative calculation.

[0021] Step 5: Determine whether the algorithm has reached the maximum number of iterations or whether the matrix reconstruction error has reached the preset threshold. If so, terminate the iteration to obtain the optimal estimated matrix; otherwise, continue the iteration.

[0022] Optionally, step 2: Perform soft thresholding on the singular values ​​according to a pre-set singular value threshold to obtain processed singular values, including:

[0023] Subtract the singular value threshold from each singular value, set the singular values ​​less than zero to zero, and keep the singular values ​​greater than or equal to the singular value threshold to obtain the processed singular values.

[0024] Optionally, the expression for soft thresholding is:

[0025]

[0026] In the formula, σ′ i It is the i-th singular value after processing; σ i It is the original i-th singular value, and τ is the set singular value threshold.

[0027] Optionally, the expression for the variable step size adaptive algorithm is:

[0028]

[0029] γ(n) = qγ(n-1), 0 < q < 1

[0030] Δ(n+1)=(μ(n)+γ(n)k(|e(n)|-|e(n-1)|))*d

[0031] In the formula, μ(n) is the step size factor adjustment function; β is the vertical scale factor; α is the horizontal scale factor, and the amplitude and shape of the function can be adjusted by adjusting β and α; e(n) is the reconstruction error magnitude of the nth iteration; γ(n) is the geometric term, q is the common ratio of the geometric sequence; Δ(n+1) is the update step size; d is the ratio of the total number of matrix elements to the amount of data without missing data; k is used to control the error difference magnitude.

[0032] Alternatively, the update formula for a similar matrix is:

[0033] M init =M init +Δ(n+1)*W*(M input -M * )

[0034] In the formula, M init For example, M is a similar matrix; imput M is the input matrix; * M is the estimation matrix; Δ(n+1) is the update step size; W is the original input matrix M. imput The mask matrix.

[0035] Optionally, the expression for the reconstruction error ε is:

[0036]

[0037] In the formula, M imput M is the input matrix; * M is the estimation matrix; W is the original input matrix. imput The mask matrix.

[0038] Optionally, the reconstruction expression for one-dimensional time series data is:

[0039]

[0040] in,

[0041]

[0042] In the formula, X q The q-th data point corresponds to the reconstructed original univariate sample; N represents the total number of data points in the original univariate sample; L represents the input matrix M. imput The number of rows; K represents the input matrix M. imput The number of columns; l min and l max Xq Corresponding estimation matrix M * The minimum and maximum number of rows in the relevant data.

[0043] According to another aspect of the present invention, a power IoT sensing data recovery device based on an improved SVT is provided, comprising:

[0044] The reconstruction module is used to reconstruct the phase space of univariate sequences of IoT data containing missing values ​​and generate a reconstruction input matrix.

[0045] The initialization module is used to initialize the reconstructed input matrix with a zero matrix to obtain a similar matrix;

[0046] The update module is used to update similar matrices based on the input matrix using an improved SVT algorithm to obtain the estimated matrix;

[0047] The recovery module is used to reconstruct the estimated matrix to obtain the recovered one-dimensional time series data.

[0048] According to another aspect of the present invention, a computer-readable storage medium is provided, the storage medium storing a computer program for performing the methods described in any of the above aspects of the present invention.

[0049] According to another aspect of the present invention, an electronic device is provided, the electronic device comprising: a processor; a memory for storing executable instructions of the processor; the processor being configured to read the executable instructions from the memory and execute the instructions to implement the method described in any of the preceding aspects of the present invention.

[0050] Therefore, this invention uses the improved SVT algorithm to recover power IoT sensing data, and utilizes phase space reconstruction to obtain hidden information and prior knowledge of power load time series data, thereby deeply mining more information in the time series. The improved SVT algorithm can achieve better data recovery results, and the convergence speed, accuracy and stability are all improved, which has strong engineering applicability. Attached Figure Description

[0051] Exemplary embodiments of the present invention can be more fully understood by referring to the following figures:

[0052] Figure 1 This is a flowchart illustrating an exemplary embodiment of the present invention for a power IoT sensing data recovery method based on an improved SVT.

[0053] Figure 2 This is a schematic diagram of the phase space reconstruction embedding dimension m selection process provided by an exemplary embodiment of the present invention;

[0054] Figure 3This is another flowchart illustrating the power IoT sensing data recovery method based on improved SVT, provided by an exemplary embodiment of the present invention.

[0055] Figure 4 This is a schematic diagram of the structure of a power IoT sensing data recovery device based on an improved SVT, provided in an exemplary embodiment of the present invention;

[0056] Figure 5 This is the structure of an electronic device provided in an exemplary embodiment of the present invention. Detailed Implementation

[0057] Hereinafter, exemplary embodiments according to the present invention will be described in detail with reference to the accompanying drawings. Obviously, the described embodiments are merely some embodiments of the present invention, and not all embodiments of the present invention. It should be understood that the present invention is not limited to the exemplary embodiments described herein.

[0058] It should be noted that, unless otherwise specifically stated, the relative arrangement, numerical expressions, and values ​​of the components and steps described in these embodiments do not limit the scope of the invention.

[0059] Those skilled in the art will understand that the terms "first," "second," etc., in the embodiments of the present invention are only used to distinguish different steps, devices, or modules, and do not represent any specific technical meaning, nor do they indicate a necessary logical order between them.

[0060] It should also be understood that in the embodiments of the present invention, "multiple" can refer to two or more, and "at least one" can refer to one, two or more.

[0061] It should also be understood that any component, data or structure mentioned in the embodiments of the present invention can generally be understood as one or more unless explicitly defined or given contrary instructions in the context.

[0062] Furthermore, the term "and / or" in this invention is merely a description of the relationship between related objects, indicating that three relationships can exist. For example, A and / or B can represent: A existing alone, A and B existing simultaneously, or B existing alone. Additionally, the character " / " in this invention generally indicates that the preceding and following related objects have an "or" relationship.

[0063] It should also be understood that the description of the various embodiments in this invention emphasizes the differences between the various embodiments, and the similarities or similarities can be referred to each other. For the sake of brevity, they will not be described in detail.

[0064] At the same time, it should be understood that, for ease of description, the dimensions of the various parts shown in the accompanying drawings are not drawn according to actual scale.

[0065] The following description of at least one exemplary embodiment is merely illustrative and is in no way intended to limit the invention or its application or use.

[0066] Techniques, methods, and equipment known to those skilled in the art may not be discussed in detail, but where appropriate, they should be considered part of the specification.

[0067] It should be noted that similar labels and letters in the following figures indicate similar items; therefore, once an item is defined in one figure, it does not need to be discussed further in subsequent figures.

[0068] The embodiments of this invention can be applied to electronic devices such as terminal devices, computer systems, and servers, and can operate together with a wide range of other general-purpose or special-purpose computing system environments or configurations. Well-known examples of terminal devices, computing systems, environments, and / or configurations suitable for use with electronic devices such as terminal devices, computer systems, and servers include, but are not limited to: personal computer systems, server computer systems, thin clients, thick clients, handheld or laptop devices, microprocessor-based systems, set-top boxes, programmable consumer electronics, network PCs, minicomputer systems, mainframe computer systems, and distributed cloud computing environments including any of the above systems, etc.

[0069] Electronic devices such as terminal devices, computer systems, and servers can be described in the general context of computer system executable instructions (such as program modules) executed by a computer system. Typically, program modules can include routines, programs, object programs, components, logic, data structures, etc., which perform specific tasks or implement specific abstract data types. Computer systems / servers can be implemented in distributed cloud computing environments, where tasks are executed by remote processing devices linked through communication networks. In distributed cloud computing environments, program modules can reside on local or remote computing system storage media, including storage devices.

[0070] Exemplary methods

[0071] Figure 1 This is a schematic flowchart of a power IoT sensing data recovery method based on an improved SVT, provided by an exemplary embodiment of the present invention. This embodiment can be applied to electronic devices, such as… Figure 1 As shown, the power IoT sensing data recovery method 100 based on improved SVT includes the following steps:

[0072] Step 101: Perform phase space reconstruction on the univariate sequence of power IoT data with missing values ​​to generate a reconstruction input matrix;

[0073] Step 102: Initialize the reconstructed input matrix using a zero matrix to obtain a similar matrix;

[0074] Step 103: Using the improved SVT algorithm, similar matrices are updated based on the input matrix to obtain the estimated matrix;

[0075] Step 104: Reconstruct the estimated matrix to obtain the recovered one-dimensional time series data.

[0076] Specifically, this invention addresses the problems of missing, duplicate, and outlier values ​​in massive sample data within the power Internet of Things (IoT) data. It analyzes the requirements for data recovery, data memory usage, and computational speed in different scenarios; establishes a data recovery algorithm; and studies optimization techniques for the data recovery algorithm to achieve an optimal balance between accuracy and speed. Specifically, it includes:

[0077] (1) This study outlines the requirements for data recovery, memory usage, and computing speed of IoT sensing data in different scenarios. Through analysis of data storage technologies, it transforms and processes the large amount of data collected by power IoT sensors in different scenarios. By converting the collected data into a machine-readable format, the study analyzes and statistically analyzes the memory usage before and after data recovery using a computer. By comparing the accuracy and speed of data recovery using different data manipulation technologies on the same computer, the study identifies the storage and operation requirements of IoT sensing data in different scenarios of the power system.

[0078] (2) Research on data recovery algorithms based on singular value decomposition (SVD). By collecting heavy operational data from the power Internet of Things (IoT) system, and addressing issues such as missing, duplicate, and anomalies in the massive sample data, research on data recovery algorithms based on SVD will be conducted. A data recovery algorithm based on SVD will be designed, and a comparative model will be established to compare the accuracy and speed of different data recovery algorithms.

[0079] (3) Research the optimization techniques of data recovery algorithms to achieve a balance between accuracy and speed. For the data recovery algorithm of singular value decomposition, it is necessary to consider that the computational complexity of the algorithm will inevitably increase as the size of the original matrix increases. Optimize the data decomposition algorithm from the perspective of matrix decomposition. Based on the compressed sensing theory, propose a low-rank matrix decomposition framework to solve the problem of increased algorithm complexity caused by singular value decomposition, thereby obtaining the optimal balance between accuracy and speed of the data recovery algorithm.

[0080] I. Improved SVT Method

[0081] The Singular Value Thresholding (SVT) algorithm is widely used in matrix completion, data restoration, recommender systems, and image processing, but its computational efficiency and accuracy still have room for improvement. Research indicates that improvements to the SVT algorithm can be achieved through measures including acceleration strategies, threshold optimization, and parallelization.

[0082] (1) Accelerate SVD calculation (optimization of core steps)

[0083] The bottleneck of SVT lies in the need to calculate SVD in each iteration. Improvements to SVD can be made to optimize SVT.

[0084] Randomized SVD (rSVD): This method uses random projections to approximate the singular value decomposition of the matrix (such as the algorithm by Halko et al.), reducing the complexity from O(mnk) to O(mn log k+k2(m+n)). This method is suitable for large-scale matrices. Its improvement lies in introducing an orthogonalization step to enhance stability (such as QR decomposition iterative optimization).

[0085] Incremental SVD update: Incremental updates are performed on local changes to the matrix (such as the addition of new data) to avoid recalculating the full matrix SVD (suitable for dynamic data scenarios).

[0086] Sparse matrix optimization: If the matrix is ​​sparse, the Lanczos or Arnoldi iterative method can be used to accelerate feature calculation (e.g., the ARPACK library).

[0087] (2) Adaptive threshold selection

[0088] The original SVT used a fixed threshold τ; an improved method could be considered as follows:

[0089] Statistically based thresholds: dynamically adjusted according to noise distribution or matrix residuals (e.g., Generalized Cross-Validation (GCV), Stein Unbiased Risk Estimation (SURE)).

[0090] Non-convex thresholding functions: Replacing soft thresholding with non-convex penalties (such as SCAD, MCP) reduces singular value shrinkage bias and improves the accuracy of low-rank estimation.

[0091] (3) Parallelization and Distributed Computing

[0092] Block SVT: Divide the large matrix into blocks, compute the SVD of each submatrix in parallel, and then aggregate them (boundary consistency needs to be handled, such as in the ADMM framework).

[0093] GPU acceleration: Utilizes CUDA libraries (such as cuSOLVER) to accelerate SVD and matrix operations, suitable for ultra-large-scale data (such as user-item matrices in recommendation systems).

[0094] (4) Algorithm fusion and hybrid strategy

[0095] SVT+Alternating Direction Multiplier Method (ADMM): Embedding SVT into the ADMM framework decomposes the optimization problem into subproblems, improving convergence speed.

[0096] Preprocessing techniques: Preprocessing the input matrix (such as spectral transformation and sparsification) reduces the condition number and accelerates SVT convergence.

[0097] (5) Theoretical Improvement: Convergence and Stability

[0098] Accelerated convergence strategies include: 1) Nesterov momentum acceleration: introducing a momentum term in the iteration to reduce oscillations; 2) Adaptive step size: adjusting the step size according to gradient changes (similar to line search strategies).

[0099] Robustness enhancement: For data containing noise or outliers, a robust kernel norm (such as Huber loss instead of the Frobenius norm) is used.

[0100] (6) Customized improvements for application scenarios

[0101] Streaming Data SVT: Developing online SVT algorithms (such as incremental updates based on stochastic gradient descent) for real-time data streams.

[0102] Structured low-rank optimization: Preserving structural information during thresholding operations by incorporating specific structural constraints (such as the temporal smoothness of the spatiotemporal matrix).

[0103] To address the shortcomings of the SVT method in processing large-scale power data, this study overcomes the limitations of the SVT algorithm and establishes a new improved SVT method: First, the embedding dimension is determined based on the phase space reconstruction effect. The optimal embedding dimension is selected by comparing the model performance under different embedding dimensions. The specific process is as follows: Figure 3 As shown; further input the data to be recovered, set the delay time and the embedding dimension to be determined, and then perform model training and evaluation under different m, compare the model performance, and select the embedding dimension with the best performance.

[0104] Phase space reconstruction maps the original massive power time series data samples containing missing values ​​to a high-dimensional space, making full use of the correlation and structural features between the original data, and providing prior knowledge for the operation of subsequent data recovery algorithms.

[0105] 2. Improve the data preprocessing method of SVT

[0106] Data preprocessing is a crucial step in the data analysis process. It includes operations such as data cleaning, data transformation, data integration, and data reduction, aiming to improve the quality and accuracy of data and provide a reliable data foundation for subsequent analysis and modeling.

[0107] 1) Data Cleaning: Data cleaning refers to the identification and processing of errors, missing values, duplicates, or inconsistencies in data. Common cleaning operations include removing duplicate values, filling in missing values, handling outliers, and correcting data formatting.

[0108] 2) Data Transformation: Data transformation involves converting raw data into a form suitable for analysis and modeling. Common transformation operations include data standardization, data encoding, feature extraction, and feature selection to better represent the relationships between data.

[0109] 3) Data Integration: Data integration involves combining data from different data sources for comprehensive analysis. During data integration, issues such as data matching, data redundancy, and data conflicts between different data sources need to be addressed.

[0110] 4) Data Reduction: Data reduction is the process of reducing the amount of data by compressing its representation while preserving key information in the dataset. Common reduction operations include data aggregation, data sampling, and feature selection to reduce dataset complexity and improve analytical efficiency.

[0111] The purpose of data preprocessing is to improve the understandability, reliability, and usability of data, thereby providing more meaningful results for subsequent data analysis and modeling. Data preprocessing operations such as data cleaning, data transformation, data integration, and data reduction are performed on multi-element data from IoT sensing modules. Due to factors such as power acquisition equipment failure, communication equipment failure, and external interference, power data is prone to incompleteness, which hinders relevant power decision-making. Therefore, it is necessary to study data recovery algorithms to improve the quality of power load data and provide effective technical support for subsequent fields such as power load forecasting, grid optimization, and power equipment fault diagnosis.

[0112] This invention proposes an improved SVT algorithm for multi-element data recovery in IoT sensing modules. The specific implementation process and method are as follows:

[0113] 3-phase space reconstruction process

[0114] Phase space reconstruction is an analytical method based on time series data. Its purpose is to reconstruct the phase space structure of a system from observed data in order to obtain more hidden information and reveal the dynamic characteristics of the system. Based on dynamical system theory, it assumes that the observed time series data is generated by a high-dimensional dynamical system. By delaying and embedding the time series data, it transforms it into a high-dimensional vector sequence, which corresponds to the state points of the system in phase space.

[0115] In 1980, Packard et al. proposed two methods for reconstructing the phase space of time series: the derivative reconstruction method and the coordinate delay reconstruction method. However, in practical applications, the derivative reconstruction method suffers from problems such as excessive differences in scale between coordinates and sensitivity to noise leading to large calculation errors. Therefore, the coordinate delay reconstruction method is usually used for reconstructing the phase space of time series data. As shown in Equation (1), the phase space of any univariate time series {x(i), i = 1, 2, ..., N} with a total length of N is transformed by the coordinate delay method.

[0116]

[0117] In the formula, t represents the delay time; m represents the embedding dimension; and d represents the number of columns in the phase space reconstruction matrix. If the delay time is too small, the values ​​between the two coordinate components in the phase space vector will be very close and indistinguishable, resulting in the inability to provide two independent coordinate components. Conversely, if the delay time is too large, the two coordinate components will be completely independent, leading to a complete lack of correlation between adjacent rows in the reconstruction matrix. The determination of the embedding dimension m directly affects subsequent analysis and modeling. If m is too large, it will amplify the noise in the data and increase the computational load of the model; if m is too small, it will reduce the accuracy of time series data processing and analysis. Therefore, the key to phase space reconstruction lies in the selection of the embedding dimension m and the delay time t.

[0118] 4. Determining the delay time

[0119] In phase space reconstruction and time series data analysis, the choice of delay time is crucial. Appropriate methods are needed to determine a suitable delay time to achieve a balance between independence and correlation. The delay time refers to the number of time steps selected in the time series to construct the point set in the phase space. Choosing a suitable delay time ensures that the reconstructed point set accurately reflects the dynamic characteristics of the system.

[0120] The delay time should be greater than or equal to the system's minimum delay time, i.e., the shortest correlation time between the system's state variables. If the delay time is too small, the points in the reconstructed phase space may be too densely packed, losing some of the system's characteristics. Conversely, choosing an excessively large delay time may result in points in the reconstructed phase space being too far apart, leading to information loss. The choice of delay time should also consider the problem of false neighbors. Choosing an excessively small delay time may increase the likelihood of false neighbors, causing points in the reconstructed phase space to be too close together. Therefore, a suitable delay time should minimize the impact of false neighbors, allowing the reconstructed phase space to better reflect the true characteristics of the system.

[0121] The Autocorrelation Function (ACF) method and the Mutual Information (MI) method are currently widely used methods. Among them, the ACF method observes the effect of the delay time on the autocorrelation function and then finds a suitable delay time so that the important dynamic characteristics of the system can be captured in the phase space reconstruction. The algorithm is simple and effective, but it can only extract the linear correlation of time series data.

[0122] Compared to ACF, MI has a stronger nonlinear expressive power and can measure the correlation and dependency between two variables, thus better balancing the relationships within the phase space. Therefore, considering the strong nonlinearity of power time series data, this study chooses to use MI to determine the optimal phase space reconstruction delay time. First, mutual information is calculated to obtain a more accurate delay factor for subsequent data analysis and processing. The MI calculation formula is as follows:

[0123]

[0124] In the formula, x represents the original time series; y represents the new series obtained after delaying x by t; p(x) and p(y) represent the marginal probability distributions of variables x and y, respectively; and p(x,y) represents the joint probability distribution of variables x and y.

[0125] Next, taking different delay time parameters t, the mutual information between system variables is calculated sequentially.

[0126] I(t)=H(x)+H(x t )-H(x,x t (3)

[0127] In the formula, H(·) represents the information entropy of the system.

[0128] Finally, a graph of the mutual information function I(t) as a function of the delay time t is plotted, and the t value at which the graph first reaches its minimum is used to determine the delay time for phase space reconstruction.

[0129] 5. Determining the Embedding Dimension

[0130] In phase space reconstruction, the selection of the embedding dimension is a crucial issue, determining how the original system is transformed into a set of points in a high-dimensional space to facilitate subsequent analysis. The choice of embedding dimension typically involves several theoretical and practical considerations.

[0131] Theoretical Basis: According to Takens' theorem, for data from a system with a sufficiently long time series, the phase space of its chaotic dynamics can be mapped to a set of points in a high-dimensional space through phase space reconstruction. The embedding dimension should be at least greater than or equal to the minimum embedding dimension of the system to capture the system's main characteristics.

[0132] Minimum embedding dimension: The minimum embedding dimension is usually determined by the system's degrees of freedom, i.e., the number of the system's state variables. However, in practical applications, due to factors such as noise and sampling rate, the minimum embedding dimension may need to be slightly higher than the system's degrees of freedom.

[0133] False proximity and noise: The choice of embedding dimension should also take into account the problem of false proximity, i.e., points that appear close in phase space do not actually belong to the same orbit. Too low an embedding dimension increases the likelihood of false proximity. Furthermore, the presence of noise also affects the choice of embedding dimension; too high an embedding dimension may cause noise to be mistakenly treated as a feature of the system.

[0134] Rules of thumb: Several rules of thumb have been proposed to guide the selection of embedding dimension, such as analysis of variance and maximum embedding dimension estimation. While these methods are simple and easy to use, they are usually only applicable to specific types of systems and data.

[0135] In practical applications, choosing the embedding dimension is often a complex problem, requiring comprehensive consideration of factors such as model accuracy, noise impact, computational efficiency, and the specific analytical objectives. It necessitates combining theoretical analysis and experimental verification to determine a suitable embedding dimension. Common methods for determining the embedding dimension include those based on chaos evaluation metrics and those based on phase space reconstruction results. The former calculates chaos evaluation metrics at different m values ​​to assess whether the results converge at different m values, ultimately selecting the dimension in the steady state as the embedding dimension. The latter conducts experiments for different m values ​​and determines the final embedding dimension by observing the phase space reconstruction results.

[0136] This study determines the embedding dimension based on the phase space reconstruction effect, and selects the optimal embedding dimension by comparing the model performance under different embedding dimensions. The specific process is as follows: Figure 2 As shown in the diagram, first input the data to be recovered, then set the delay time and the embedding dimension to be determined. Next, train and evaluate the model under different values ​​of m, compare the model performance, and select the embedding dimension with the best effect.

[0137] Phase space reconstruction maps the original massive power time series data samples containing missing values ​​to a high-dimensional space, making full use of the correlation and structural features between the original data, and providing prior knowledge for the operation of subsequent data recovery algorithms.

[0138] II. A Power IoT Sensing Data Recovery Method Based on Improved SVT

[0139] Compared to traditional data recovery algorithms, singular value decomposition (SVD) exhibits significant advantages in data recovery, including linearity and unbiasedness, best approximation, adaptability, and robustness. Based on these advantages, singular value thresholding (SVT) algorithms are now widely used in data recovery tasks across various fields.

[0140] Unlike SVD, the SVT algorithm performs singular value decomposition on the matrix, then uses soft thresholding based on the singular values ​​to reduce the matrix's rank to the required level for the data, and finally reconstructs the matrix. (Reference) Figure 3 As shown, the main steps are as follows:

[0141] 1) Input data preparation

[0142] For a matrix reconstructed from the phase space of a univariate sequence containing missing values, it can be represented as the sum of a known part and an unknown part. That is:

[0143] M input =M known +M unknown (4)

[0144] In the formula, M imput M is the input matrix before singular value decomposition; known and M unknown These are the matrices representing the known and unknown parts of the input matrix, respectively.

[0145] 2) Matrix initialization

[0146] For the unknown part of the matrix M unknown This study chooses zero-filling and initializes the input matrix M with a zero matrix. imput Form a similar matrix M init .

[0147] 3) Iterative update process

[0148] The estimation of the unknown part of the matrix is ​​gradually improved through an iterative process. The main process is as follows:

[0149] Step 1: Singular Value Decomposition. For matrix M... init Performing singular value decomposition yields the singular values ​​and corresponding singular vectors of the input matrix. The decomposition formula is:

[0150]

[0151] In the formula, U and V are orthogonal matrices, and Σ is a diagonal matrix with singular values ​​on the diagonal.

[0152] Step 2: Soft thresholding of singular values. Subtract a threshold τ from each element of the diagonal matrix and set elements less than zero to zero. This removes singular values ​​less than the threshold and retains those greater than or equal to the threshold. This process can be represented as:

[0153]

[0154] In the formula, σ′ i It is the i-th singular value after processing; σ i It is the original i-th singular value, and τ is the set singular value threshold.

[0155] Step 3: Matrix Reconstruction. Using the thresholded singular values ​​and singular vectors, an estimated matrix M is reconstructed. * The reconstruction process is as follows:

[0156]

[0157] Step 4: Update. Using the given update step size, combined with the input matrix M... imput and estimation matrix M * The difference is used to update the singular value decomposition matrix. The update process is as follows:

[0158] M init =M init +Δ*W*(M input -M * (8)

[0159] In the formula, M init To initialize the all-zero matrix; Δ is the given step size; W is the mask matrix of the original input matrix.

[0160] Step 5: Termination Condition Check. The algorithm checks whether the iteration has reached the set maximum number of iterations or whether the matrix reconstruction error has reached a set threshold to determine if the termination condition has been met. If the termination condition is met, the algorithm ends; otherwise, it returns to Step 3, the matrix reconstruction process. The reconstruction error is expressed in terms of the matrix's 2-norm:

[0161]

[0162] 4) Output Results

[0163] After the algorithm converges, the final estimation matrix M is obtained. * This serves as the result of restoring the original matrix. However, considering that the original samples are univariate time series data with missing values, the restoration matrix needs to be reconstructed to become one-dimensional time series data in the final restoration of missing values. Specifically:

[0164]

[0165]

[0166] In equations (10), (11), and (12), X q The q-th number corresponding to the reconstructed original univariate sample; N represents the total number of data points in the original univariate sample; L represents the input matrix M. imput The number of rows; K represents the input matrix M. imput The number of columns; l min and l max X q Corresponding to M * X is calculated from the minimum and maximum rows where the relevant data exists in the matrix. q Numerical size.

[0167] The convergence and speed of the SVT algorithm are affected by the choice of its update step size. Too large a step size will cause the algorithm to diverge during iteration, preventing it from converging to the optimal solution. Too small a step size will slow down the convergence speed and significantly increase computational cost. To address these issues, this study proposes an improved SVT algorithm with adaptive variable step size, dynamically adjusting the step size based on the results of each iteration to achieve the optimal balance between convergence and speed.

[0168] Most variable step-size adaptive algorithms adjust the step size based on the error of the current step size. Common step-size adjustment functions include the hyperbolic tangent function and the sigmoid function. This study proposes a variable step-size method that incorporates an improved sigmoid function with a geometric term. First, the basic function for variable step-size is constructed based on logarithms and the sigmoid function. Then, a geometric term is introduced to achieve faster early convergence. Specifically:

[0169]

[0170] γ(n)=qγ(n-1),0<q<1 (14)

[0171] Δ(n+1)=(μ(n)+γ(n)k(|e(n)|-|e(n-1)|))*d (15)

[0172] In equation (13), μ(n) is the step size factor adjustment function; β is the vertical scale factor; α is the horizontal scale factor. The amplitude and shape of the function can be adjusted by adjusting β and α; e(n) is the error magnitude of the nth iteration. In equation (14), γ(n) is the geometric term. Since 0 < q < 1, this term can improve the convergence speed in the early stage without affecting the effect in the later stage. Equation (15) is the step size update method, where d is the ratio of the total number of matrix elements (including missing elements) to the amount of data without missing elements; k is used to control the error difference magnitude.

[0173] After the improvement, step four of the original SVT-based data recovery algorithm is changed to using the update step size calculated by equation (16) and combining it with the input matrix M. imput and estimation matrix M * The difference is used to update the singular value decomposition matrix. The update process is as follows:

[0174] M init =M init +Δ(n+1)*W*(M input -M * (16)

[0175] The innovative aspects of this invention are as follows:

[0176] 1. An Improved SVT Algorithm. This study proposes and constructs an improved SVT algorithm for big data recovery, which utilizes phase space reconstruction to obtain hidden information and prior knowledge of power load time series data, and further mines more information from the time series.

[0177] 2. Constructing an Adaptive Variable Step Size SVT Algorithm. Addressing the shortcomings of the SVT algorithm, such as slow convergence speed and insufficient convergence accuracy, this paper constructs a variable step size function using logarithms and the sigmoid function, introducing a geometric term to achieve faster initial convergence. Based on this, an adaptive variable step size SVT algorithm is constructed, improving the algorithm's convergence speed, convergence accuracy, and stability.

[0178] 3. The new algorithm converges faster. Comparative analysis with other commonly used data recovery algorithms on different datasets under various conditions shows that the new algorithm proposed in this study converges faster, has higher accuracy, and possesses good universality, thus completing the research task as planned.

[0179] Therefore, this invention addresses the problems of traditional Singular Value Threshold (SVT) data recovery algorithms neglecting prior information in power load data processing and low computational efficiency for large-scale data. It proposes and establishes an improved SVT data recovery algorithm based on phase space reconstruction and adaptive variable step size. To solve the problem of traditional SVT easily ignoring prior information, a phase space reconstruction method is introduced to map the original missing data to a high-dimensional space. This utilizes the correlation and structural features between data points to provide prior knowledge for data processing and recovery algorithms. A variable step size foundation function is constructed by combining logarithms and the sigmoid function, and a geometric term is used to increase the initial step size, thus constructing an adaptive variable step size SVT algorithm. This overcomes the problem of low computational efficiency of traditional SVT in large-scale data scenarios. Comparative experiments are conducted using multiple public power load datasets and various commonly used power load data processing and recovery algorithms. The results show that the improved SVT algorithm achieves better data recovery results, with improved convergence speed, accuracy, and stability, demonstrating strong engineering applicability.

[0180] Exemplary device

[0181] Figure 4 This is a schematic diagram of the structure of a power IoT sensing data recovery device based on an improved SVT, provided in an exemplary embodiment of the present invention. Figure 4 As shown, the device 400 includes:

[0182] The reconstruction module 410 is used to perform phase space reconstruction on the univariate sequence of power IoT data with missing values ​​and generate a reconstruction input matrix.

[0183] Initialization module 420 is used to initialize the reconstructed input matrix with a zero matrix to obtain a similar matrix;

[0184] The update module 430 is used to update similar matrices based on the input matrix using the improved SVT algorithm to obtain the estimated matrix;

[0185] The recovery module 440 is used to reconstruct the estimated matrix to obtain the recovered one-dimensional time series data.

[0186] Exemplary electronic devices

[0187] Figure 5 This is the structure of an electronic device provided in an exemplary embodiment of the present invention. For example... Figure 5 As shown, the electronic device 50 includes one or more processors 51 and memory 52.

[0188] The processor 51 may be a central processing unit (CPU) or other form of processing unit with data processing and / or instruction execution capabilities, and may control other components in the electronic device to perform desired functions.

[0189] The memory 52 may include one or more computer program products, which may include various forms of computer-readable storage media, such as volatile memory and / or non-volatile memory. The volatile memory may include, for example, random access memory (RAM) and / or cache memory. The non-volatile memory may include, for example, read-only memory (ROM), hard disk, flash memory, etc. One or more computer program instructions may be stored on the computer-readable storage medium, and the processor 51 may execute the program instructions to implement the methods of the software programs of the various embodiments of the present invention described above, and / or other desired functions. In one example, the electronic device may also include an input device 53 and an output device 54, these components being interconnected via a bus system and / or other forms of connection mechanisms (not shown).

[0190] In addition, the input device 53 may also include, for example, a keyboard, a mouse, etc.

[0191] The output device 54 can output various information to the outside. The output device 54 may include, for example, a display, a speaker, a printer, and a communication network and its connected remote output devices, etc.

[0192] Of course, for the sake of simplicity, Figure 5 Only some of the components of this electronic device relevant to the present invention are shown, omitting components such as buses, input / output interfaces, etc. In addition, the electronic device may include any other suitable components depending on the specific application.

[0193] Exemplary computer program products and computer-readable storage media

[0194] In addition to the methods and apparatus described above, embodiments of the present invention may also be computer program products, which include computer program instructions that, when executed by a processor, cause the processor to perform the steps in the methods according to various embodiments of the present invention described in the "Exemplary Methods" section above.

[0195] The computer program product can be written in any combination of one or more programming languages ​​to perform the operations of the embodiments of the present invention. The programming languages ​​include object-oriented programming languages ​​such as Java and C++, as well as conventional procedural programming languages ​​such as C or similar languages. The program code can be executed entirely on the user's computing device, partially on the user's computing device, as a standalone software package, partially on the user's computing device and partially on a remote computing device, or entirely on a remote computing device or server.

[0196] Furthermore, embodiments of the present invention may also be computer-readable storage media storing computer program instructions thereon, which, when executed by a processor, cause the processor to perform the steps of the methods according to various embodiments of the present invention described in the "Exemplary Methods" section above.

[0197] The computer-readable storage medium may be any combination of one or more readable media. A readable medium may be a readable signal medium or a readable storage medium. A readable storage medium may be, for example, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, device, or any combination thereof. More specific examples (a non-exhaustive list) of readable storage media include: an electrical connection having one or more wires, a portable disk, a hard disk, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage device, magnetic storage device, or any suitable combination thereof.

[0198] The basic principles of the present invention have been described above with reference to specific embodiments. However, it should be noted that the advantages, benefits, and effects mentioned in the present invention are merely examples and not limitations, and should not be considered as essential features of each embodiment of the present invention. Furthermore, the specific details disclosed above are for illustrative and facilitative purposes only, and are not limitations. These details do not limit the present invention to the necessity of employing the aforementioned specific details.

[0199] The various embodiments in this specification are described in a progressive manner, with each embodiment focusing on its differences from other embodiments. Similar or identical parts between embodiments can be referred to interchangeably. For system embodiments, since they largely correspond to method embodiments, the description is relatively simple; relevant parts can be referred to the descriptions in the method embodiments.

[0200] The block diagrams of devices, systems, devices, and systems involved in this invention are merely illustrative examples and are not intended to require or imply that they must be connected, arranged, or configured in the manner shown in the block diagrams. As those skilled in the art will recognize, these devices, systems, devices, and systems can be connected, arranged, and configured in any manner. Words such as “comprising,” “including,” “having,” etc., are open-ended terms meaning “including but not limited to,” and are used interchangeably with them. The terms “or” and “and” as used herein refer to the terms “and / or,” and are used interchangeably with them unless the context clearly indicates otherwise. The term “such as” as used herein refers to the phrase “such as but not limited to,” and is used interchangeably with it.

[0201] The methods and systems of the present invention may be implemented in many ways. For example, they may be implemented by software, hardware, firmware, or any combination of software, hardware, and firmware. The above-described order of steps for the methods is for illustrative purposes only, and the steps of the methods of the present invention are not limited to the order specifically described above unless otherwise specifically stated. Furthermore, in some embodiments, the present invention may also be implemented as a program recorded on a recording medium, the program comprising machine-readable instructions for implementing the methods according to the present invention. Thus, the present invention also covers recording media storing programs for performing the methods according to the present invention.

[0202] It should also be noted that in the systems, apparatus, and methods of the present invention, the components or steps can be disassembled and / or recombined. These disassemblies and / or recombinations should be considered equivalents of the present invention. The above description of the disclosed aspects is provided to enable any person skilled in the art to make or use the invention. Various modifications to these aspects will be readily apparent to those skilled in the art, and the general principles defined herein can be applied to other aspects without departing from the scope of the invention. Therefore, the invention is not intended to be limited to the aspects shown herein, but rather to be carried out within the widest scope consistent with the principles and novel features disclosed herein.

[0203] The above description has been given for purposes of illustration and description. Furthermore, this description is not intended to limit the embodiments of the invention to the forms disclosed herein. Although numerous exemplary aspects and embodiments have been discussed above, those skilled in the art will recognize certain variations, modifications, alterations, additions, and sub-combinations thereof.

Claims

1. A power IoT sensing data recovery method based on improved SVT, characterized in that, include: Phase space reconstruction is performed on univariate sequences of power IoT data containing missing values ​​to generate a reconstruction input matrix; The reconstructed input matrix is ​​initialized using a zero matrix to obtain a similar matrix; An improved SVT algorithm is used to update the similar matrices based on the input matrix to obtain the estimated matrix; The estimated matrix is ​​reconstructed to obtain the recovered one-dimensional time series data.

2. The method according to claim 1, characterized in that, The expression for the reconstructed input matrix is: M input =M known +M unknown In the formula, M imput M is the input matrix before singular value decomposition; known and M unknown These are the matrices representing the known and unknown parts of the input matrix, respectively.

3. The method according to claim 1, characterized in that, The improved SVT algorithm is used to update the similar matrix based on the input matrix to obtain the estimated matrix, including: Step 1: Perform singular value decomposition on the similar matrix to obtain the singular values ​​of the similar matrix and the singular vectors corresponding to the singular values; Step 2: Perform soft thresholding on the singular values ​​according to the preset singular value threshold to obtain the processed singular values; Step 3: Reconstruct the matrix based on the processed singular values ​​and corresponding singular vectors to obtain the estimated matrix; Step 4: Determine the update step size using a variable step size adaptive algorithm, and use the update step size to update the similar matrix based on the estimated matrix and the input matrix to obtain the updated similar matrix. Then return to step 1 for iterative calculation. Step 5: Determine whether the algorithm has reached the maximum number of iterations or whether the matrix reconstruction error has reached the preset threshold. If so, terminate the iteration to obtain the optimal estimated matrix; otherwise, continue the iteration.

4. The method according to claim 3, characterized in that, Step 2: Perform soft thresholding on the singular values ​​according to a pre-set singular value threshold to obtain processed singular values, including: Subtract the singular value threshold from each singular value, set the singular values ​​less than zero to zero, and retain the singular values ​​greater than or equal to the singular value threshold to obtain the processed singular values.

5. The method according to claim 3, characterized in that, The expression for the soft threshold processing is: In the formula, σ′ i It is the i-th singular value after processing; σ i It is the original i-th singular value, and τ is the set singular value threshold.

6. The method according to claim 3, characterized in that, The expression for the variable step size adaptive algorithm is: γ(n) = qγ(n-1), 0 < q < 1 Δ(n+1)=(μ(n)+γ(n)k(|e(n)|-|e(n-1)|))*d In the formula, μ(n) is the step size factor adjustment function; β is the vertical scale factor; α is the horizontal scale factor, and the amplitude and shape of the function can be adjusted by adjusting β and α; e(n) is the reconstruction error magnitude of the nth iteration; γ(n) is the geometric term, q is the common ratio of the geometric sequence; Δ(n+1) is the update step size; d is the ratio of the total number of matrix elements to the amount of data without missing data; k is used to control the error difference magnitude.

7. The method according to claim 3, characterized in that, The update formula for the similar matrix is: M init =M init +Δ(n+1)*W*(M input -M * ) In the formula, M init For example, M is a similar matrix; imput M is the input matrix; * M is the estimation matrix; Δ(n+1) is the update step size; W is the original input matrix M. imput The mask matrix.

8. The method according to claim 3, characterized in that, The expression for the reconstruction error ε is: In the formula, M imput M is the input matrix; * M is the estimation matrix; W is the original input matrix. imput The mask matrix.

9. The method according to claim 1, characterized in that, The reconstruction expression for the one-dimensional time series data is: in, In the formula, X q The q-th data point corresponds to the reconstructed original univariate sample; N represents the total number of data points in the original univariate sample; L represents the input matrix M. imput The number of rows; K represents the input matrix M. imput The number of columns; l min and l max X q Corresponding estimation matrix M * The minimum and maximum number of rows in the relevant data.

10. A power IoT sensing data recovery device based on an improved SVT, characterized in that, include: The reconstruction module is used to reconstruct the phase space of univariate sequences of IoT data containing missing values ​​and generate a reconstruction input matrix. An initialization module is used to initialize the reconstructed input matrix using a zero matrix to obtain a similar matrix; The update module is used to update the similar matrix based on the input matrix using the improved SVT algorithm to obtain the estimated matrix; The recovery module is used to reconstruct the estimated matrix to obtain the recovered one-dimensional time series data.

11. A computer-readable storage medium, characterized in that, The storage medium stores a computer program for performing the method described in any one of claims 1-9.

12. An electronic device, characterized in that, The electronic device includes: processor; Memory used to store the processor's executable instructions; The processor is configured to read the executable instructions from the memory and execute the instructions to implement the method described in any one of claims 1-9.

Images