A power system database abnormal missing data completion method, system and product

CN119179692BActive Publication Date: 2026-09-18SHANDONG UNIV
View PDF 1 Cites 0 Cited by

Patent Information

Application Number
CN202411176267.6
Authority / Receiving Office
CN · China
Patent Type
Patents(China)
Current Assignee / Owner
Filing Date
2024-08-26
Publication Date
2026-09-18
Estimated Expiration
2044-08-26

AI Technical Summary

Technical Problem

然而目前主流的数据预测方法,普遍采用了时间衰减机制,这主要是为了处理缺失数据的时间衰减效应,同时对于长时间序列数据,容易发生梯度消失或梯度爆炸问题,因此,这些模型在处理长距离依赖方面存在着很大的局限性

Benefits of technology

[0033] This invention performs integrity detection on multivariate time series data. If missing values ​​exist in the data, the specific location and time point of the missing values ​​are determined. Based on the multivariate time series data and the missing data prediction model, a prediction value for the missing data is obtained. When predicting missing data, a dual model based on history and features is used to predict the missing data from both the past and future directions. A bidirectional recurrent neural network combined with a bidirectional attention mechanism is used to capture long-distance dependencies between data, thereby improving the accuracy of the prediction.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN119179692B_ABST
    Figure CN119179692B_ABST
Patent Text Reader

Abstract

The application belongs to the technical field of big data, and provides a power system database abnormal missing data completion method, system and product, a technical scheme of which is to acquire multi-source data of a power system, process to obtain multi-element time series data; based on the multi-element time series data and a missing data prediction model, prediction is performed to obtain a predicted value of the missing data, based on historical and feature data, the missing data is predicted from two directions of before and after in combination with a bidirectional attention mechanism, the predicted value of the predicted missing data is interpolated into a missing value position, and the data after the missing value is completed is stored into the power system database, thereby solving the limitation of existing missing data processing, and the application performs excellently in long sequence data prediction and a high missing rate environment, and provides reliable data prediction support for power system operation and management.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] This invention belongs to the field of big data technology, and in particular relates to a method, system and product for completing abnormal missing data in a power system database. Background Technology

[0002] The statements in this section are merely background information related to the present invention and do not necessarily constitute prior art.

[0003] Traditional power systems face challenges such as supply and demand imbalance and energy waste. The application of smart grid technology is driving the industry towards a green transformation. To meet the data support needs of typical energy-consuming industries in areas such as energy efficiency, energy consumption, production consumption, energy conservation, and surveys, a database needs to be built to store information such as electricity consumption. However, during data acquisition, various factors such as smart meter malfunctions or data transmission channel congestion can lead to irregular gaps in power data. These gaps are uncertain not only in quantity but also in timing. Especially when the amount of missing data is large, simply deleting the missing data results in significant information loss, affecting the integrity and accuracy of the data.

[0004] Existing methods for handling missing data mainly include deletion, imputation, or no processing. Deletion is the simplest method, but its effectiveness is far from ideal when the amount of missing data is large. Imputation methods are mainly divided into two categories: statistical methods and machine learning methods. Statistical methods include regression analysis and multiple imputation, but most of these methods do not consider the category attributes of individual data objects, and the imputation results may be affected by different categories of objects in the entire dataset, resulting in relatively limited accuracy. Therefore, to better complete missing data, more refined methods are needed to capture the spatiotemporal distribution characteristics of the data, thereby improving the imputation effect.

[0005] With the significant increase in computing power, machine learning-based imputation methods have been widely adopted. In many contexts, machine learning techniques have begun to be used to handle missing values ​​in time series data. These methods include algorithms such as K-Nearest Neighbor (KNN), Recurrent Neural Networks (RNN), Random Forests, and Matrix Factorization, which are widely used in the task of imputing missing values. In existing technologies, one approach is to set a threshold to select nearest neighbors based on the Naive K-Nearest Neighbor method, then combine this with the grey relational coefficient to calculate the weight coefficients of the nearest neighbors, and finally impute the missing data sequentially. However, this method may not perform well in terms of computation time and storage requirements when dealing with large-scale datasets. Another novel deep learning model based on RNNs, called GRU-D, effectively integrates two missing value pattern representations into the deep model architecture, enabling the model to not only capture long-term temporal dependencies in time series data but also utilize missing value patterns to obtain better prediction results. However, current mainstream data prediction methods generally employ time decay mechanisms, mainly to handle the time decay effect of missing data. Furthermore, for long-term series data, gradient vanishing or gradient exploding problems are prone to occur. Therefore, these models have significant limitations in handling long-distance dependencies. Summary of the Invention

[0006] To address at least one of the technical problems mentioned above, this invention provides a method, system, and product for completing abnormally missing data in a power system database. It uses a dual model based on history and features to predict missing data from both directions, and utilizes a bidirectional recurrent neural network combined with a bidirectional attention mechanism to capture long-distance dependencies between data, thereby improving the accuracy of prediction.

[0007] To achieve the above objectives, the present invention adopts the following technical solution:

[0008] The first aspect of the present invention provides a method for completing abnormally missing data in a power system database, comprising the following steps:

[0009] Acquire multi-source data from the power system and process it to obtain multivariate time series data;

[0010] Prediction is performed based on multivariate time series data and a missing data prediction model to obtain predicted values ​​for the missing data; wherein, the construction process of the missing data prediction model includes:

[0011] For each time step, based on the extracted mask and time interval of the current time step, a time decay factor is calculated, the hidden state is updated according to the time decay factor, historical predicted missing values ​​are calculated according to the hidden state, and the missing values ​​are filled to obtain the preliminary completed time series data; based on the preliminary completed time series data, feature estimation is performed in conjunction with the attention mechanism to calculate the feature estimate; the combined weight of historical predicted missing values ​​and feature estimate is calculated, and the combined weight is combined with historical predicted missing values ​​and feature estimate to obtain the combined estimate; the hidden state and cell state of the LSTM unit are updated based on the combined estimate.

[0012] After the time step loop ends, the hidden states of all time steps are processed to obtain a context vector. A linear layer is used to process the context vector to obtain the predicted value of the most missing data. The predicted value of the missing data is then inserted into the position of the missing value, and the data after the missing value is filled is stored in the power system database.

[0013] Furthermore, the formula for calculating the time decay factor is:

[0014]

[0015] Where, γ h and γ x W represents the time decay factor. x b x W h b h Indicates model parameters, The time interval t represents the time between the current time t and the last time data was detected, s t For the current step time series data x t At the corresponding time, s t-1 For the time series data x from the previous step t-1 At the corresponding time, This represents the time interval between the current time t and the last time data was detected. The observed time series data x t The dth feature, For the corresponding The mask vector.

[0016] Furthermore, the formula for calculating the feature estimate value based on the initially completed time series data and combined with the attention mechanism is as follows:

[0017]

[0018] in, This is time series data initially supplemented based on historical data. Q, K, and V are the parameters W obtained by linear transformation based on the initially supplemented time series data. Q W K W V The vector matrix after linear transformation, d model z is the dimension of the input vector. t This is the feature estimate.

[0019] Furthermore, when processing the hidden states at all time steps, a multi-head attention mechanism is adopted. The multi-head attention mechanism first uses the parameter matrix to perform a linear transformation on the hidden states at all time steps, and divides the vector matrix after the linear transformation into H heads. Based on the query, key, and value vector of the i-th head, the attention weight of each head is calculated. The outputs of each head are concatenated and average pooling is performed. Then, the context vector obtained after average pooling is fed into a linear transformation layer to obtain the final predicted value of the missing data.

[0020] Furthermore, after obtaining the multivariate time series data, the acquired data is preprocessed, including cleaning, noise reduction, and standardization.

[0021] Furthermore, after preprocessing the acquired data, the multivariate time series data is subjected to integrity checks. If missing values ​​are found in the data, the specific location and time point of the missing values ​​are determined.

[0022] A second aspect of the present invention provides a system for completing missing abnormal data in a power system database, comprising:

[0023] The data acquisition module is used to acquire multi-source data from the power system and process it to obtain multivariate time series data.

[0024] The missing value prediction module is used to predict the missing values ​​of data based on multivariate time series data and a missing data prediction model; wherein, the construction process of the missing data prediction model includes:

[0025] For each time step, based on the extracted mask and time interval of the current time step, a time decay factor is calculated, the hidden state is updated according to the time decay factor, historical predicted missing values ​​are calculated according to the hidden state, and the missing values ​​are filled to obtain the preliminary completed time series data; based on the preliminary completed time series data, feature estimation is performed in conjunction with the attention mechanism to calculate the feature estimate; the combined weight of historical predicted missing values ​​and feature estimate is calculated, and the combined weight is combined with historical predicted missing values ​​and feature estimate to obtain the combined estimate; the hidden state and cell state of the LSTM unit are updated based on the combined estimate.

[0026] After the time step loop ends, the hidden states of all time steps are processed to obtain the context vector. The context vector is then processed using a linear layer to obtain the predicted value of the least missing data.

[0027] The missing value completion module is used to insert the predicted values ​​of the predicted missing data into the positions of the missing values, and to store the data after missing value completion into the power system database.

[0028] Furthermore, the system also includes a missing value detection module, which is used to preprocess the acquired data and perform integrity detection on the multivariate time series data. If there are missing values ​​in the data, the module determines the specific location and time point of the missing values.

[0029] Furthermore, in the missing value prediction module, when processing the hidden states at all time steps, a multi-head attention mechanism is adopted. The multi-head attention mechanism first uses a parameter matrix to perform a linear transformation on the hidden states at all time steps, dividing the vector matrix after the linear transformation into H heads. Based on the query, key, and value vectors of the i-th head, the attention weights of each head are calculated. The outputs of each head are concatenated and average pooling is performed. Then, the context vector obtained after average pooling is fed into a linear transformation layer to obtain the final predicted value of the missing data.

[0030] A third aspect of the present invention provides a program product.

[0031] A program product, which is a computer program product, includes a computer program that, when executed by a processor, implements the steps of the method for completing abnormally missing data in a power system database as described above in the first aspect.

[0032] Compared with the prior art, the beneficial effects of the present invention are:

[0033] This invention performs integrity detection on multivariate time series data. If missing values ​​exist in the data, the specific location and time point of the missing values ​​are determined. Based on the multivariate time series data and the missing data prediction model, a prediction value for the missing data is obtained. When predicting missing data, a dual model based on history and features is used to predict the missing data from both the past and future directions. A bidirectional recurrent neural network combined with a bidirectional attention mechanism is used to capture long-distance dependencies between data, thereby improving the accuracy of the prediction.

[0034] Advantages of additional aspects of the invention will be set forth in part in the description which follows, and in part will be obvious from the description, or may be learned by practice of the invention. Attached Figure Description

[0035] The accompanying drawings, which form part of this invention, are used to provide a further understanding of the invention. The illustrative embodiments of the invention and their descriptions are used to explain the invention and do not constitute an improper limitation of the invention.

[0036] Figure 1 This is a flowchart of a method for completing abnormal missing data in a power system database, provided by an embodiment of the present invention. Detailed Implementation

[0037] The present invention will be further described below with reference to the accompanying drawings and embodiments.

[0038] It should be noted that the following detailed description is illustrative and intended to provide further explanation of the invention. Unless otherwise specified, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this invention pertains.

[0039] It should be noted that the terminology used herein is for the purpose of describing particular embodiments only and is not intended to limit the scope of exemplary embodiments according to the invention. As used herein, the singular form is intended to include the plural form as well, unless the context clearly indicates otherwise. Furthermore, it should be understood that when the terms "comprising" and / or "including" are used in this specification, they indicate the presence of features, steps, operations, devices, components, and / or combinations thereof.

[0040] Terminology Explanation

[0041] Normalization: Normalization is a data preprocessing technique that aims to transform data of different scales and ranges to a common scale. It usually involves adjusting the data to a specific range (such as between 0 and 1) or giving it specific statistical properties.

[0042] Attention Mechanism: The attention mechanism is an important artificial intelligence technique that allows models to focus on key information within a sequence while ignoring less important information. The core idea is to assign different weights to each position in the input sequence based on its importance. This mechanism more effectively captures and utilizes key information, significantly improving the performance of models processing long sequences of data.

[0043] Recurrent Neural Networks (RNNs) are neural network structures used to process sequential data. Unlike traditional feedforward neural networks, RNNs have recurrent connections, enabling them to process time-series data and other sequentially related information. In RNNs, neurons in the hidden layers receive not only the current input but also the hidden state from the previous time step, thus retaining and utilizing previous information. RNNs are widely used in natural language processing, speech recognition, time series prediction, and other fields.

[0044] After the system collects, stores, and preprocesses the data, the missing data detection module detects the missing parts in the preprocessed data and marks the location of the missing values. It then uses a bidirectional attention mechanism combined with a recurrent neural network to predict the missing data. This embodiment uses large-scale power data as an example. During the acquisition of power data, various factors, such as smart meter malfunctions or data transmission channel blockages, may lead to irregular data gaps. The method of this invention first identifies the missing data within the abnormal phenomena and then calculates and supplements it.

[0045] Example 1

[0046] like Figure 1 As shown in the figure, this embodiment provides a method for completing abnormally missing data in a power system database, including the following steps:

[0047] S101. Acquire various data from the power system and store them in a centralized database;

[0048] In this embodiment, the power system data includes various forms of data such as sensor data, log files, and external APIs.

[0049] The data collection module needs to have efficient data transmission and storage capabilities to ensure data integrity and timeliness. Data is stored in a centralized database to achieve centralized data collection and management.

[0050] S102. Preprocess the acquired data, including cleaning, noise reduction and standardization, to ensure data quality and consistency.

[0051] The cleaning process includes removing duplicate data, correcting outliers, and filling in missing values; the noise reduction process aims to filter out noise in the data and retain useful information; and the standardization process converts the data into a unified format and unit for subsequent processing and analysis.

[0052] S103. Check the integrity of the data and record the location of missing data;

[0053] If missing values ​​are detected in the data, the system will record the specific location and time of these missing values ​​to ensure that the subsequent completion process can accurately locate the missing data.

[0054] S104. Based on the constructed missing data prediction model, predict and complete the missing data, specifically including the following steps:

[0055] For each time step, perform the following operations:

[0056] S401. Extract the value, mask, and time interval of the current time step;

[0057] In the real world, multivariate time series data X = {x1, x2, ..., x...} t ,…,x T As an observation sequence, T is the length of the time series, and each observed value x t It may consist of D features, such as Because data may be missing, a mask vector m is introduced. t , indicating the absence of a certain feature, is calculated using the following formula:

[0058]

[0059] in, The observed time series data x t The dth feature.

[0060] To facilitate the analysis of the relationship between data and time, δ was introduced. t :

[0061]

[0062] in, The time interval t represents the time between the current time t and the last time data was detected, in seconds. t For x t At the corresponding time, s t-1 For x t-1 At the corresponding time, This represents the time interval between the current time t and the last time data was detected. For the corresponding The mask vector.

[0063] S402. Based on the extracted mask and time interval of the current time step, calculate the time decay factor, update the hidden state according to the time decay factor, calculate the historical predicted missing values ​​according to the hidden state, fill in the missing values, and obtain the preliminarily completed time series data; for historical-based estimation, since the value x of the current time step... t There may be missing values, so x cannot be used directly.t Instead of using the value as input, the preliminary completed data x is calculated using formula (1). bt During initialization, the hidden state h is a zero tensor, and x is set to zero. t Using the mask vector m through formula (2) t After supplementation Then proceed to the next level of calculation.

[0064] To prevent irregular sampling of time, a time decay factor γ is first introduced through formulas (3) and (4). h and γ x The former is mainly used to decay the hidden state h, while the latter is mainly used to process the input data. The formula is expressed as follows:

[0065] x bt =W his h t-1 +b his (1),

[0066]

[0067] γ h =exp{-ReLU(W h ·δ t +b h )} (3),

[0068] γ x =exp{-ReLU(W x ·δ t +b x )} (4),

[0069] In the formula, x bt This indicates missing values ​​initially predicted based on historical data. W represents time series data initially supplemented based on historical data. his b his W x b x W h b h Represents the model parameters, m t Denotes the mask vector, δ t γ represents the time interval between the current time t and the last time data was detected. h and γ x This represents the time decay factor.

[0070] S403. Based on the initially completed time series data, feature estimation is performed using an attention mechanism to calculate the feature estimate value.

[0071] Equation (2) yields a prediction based on historical data, assuming that each observed attribute is independent. However, in many cases, the values ​​of different attributes can influence each other. To further improve the accuracy of the prediction, an attention mechanism is used to implement feature-based estimation. Q, K, and V are obtained by linear transformation of the sequence completed by the historical estimation, as shown in equations (5), (6), and (7). After processing by the softmax function in equation (8), the feature-based estimate z is obtained. t The formula is expressed as follows:

[0072]

[0073] z t =Attention(Q,K,V) (9),

[0074] S404. Calculate the missing value x based on historical predictions. bt and feature-based estimation z t Combined weight α t Based on the combined weights and historical and feature estimates, a combined estimate is obtained, and missing values ​​are filled in. The filled values ​​are used as input, and the hidden state and cell state of the LSTM unit are updated through a self-attention mechanism. The specific formula is as follows:

[0075] α t =σ(W c ·[γ x ,m]+b c (10),

[0076] c t =α t ⊙z t +(1-α t )⊙x bt (11),

[0077]

[0078] After the time step loop ends:

[0079] S405. Use a multi-head attention mechanism to process the hidden states at all time steps to obtain a context vector. Use a linear layer to process the context vector to obtain the final prediction value and calculate the prediction loss.

[0080] The hidden states at all the above time steps are processed through a multi-head attention mechanism to obtain context vectors. These context vectors help the model better understand and utilize the overall structure and features of time series data in the prediction task. Then, a linear layer is used to process the context vectors to obtain the final predicted value and calculate the prediction loss.

[0081] For the hidden states at all the time steps mentioned above, the multi-head attention mechanism first uses the parameter matrix W. q W k W v Perform a linear transformation on it, and the transformed vectors are represented as Q, K, and V, respectively. Q in formula (15) i K i V i All values ​​are obtained from the hidden state h, and the subscript i represents different attention mechanism heads. The formula is as follows:

[0082]

[0083] y h =σ(W out ·avg_context+b out (17),

[0084] The Q, K, and V vectors are then divided into H heads, each head having a dimension of d. model / H, where d model Let Q be the dimension of the input vector, H be the number of heads, and then the attention weights are calculated for each head, as shown in Equation (15), where Q is the number of heads. i K i V i Let Q and K represent the query, key, and value vectors of the i-th header, respectively. Specifically, first calculate the similarity between Q and K, then divide by d. model Scaling is performed, and after normalization by the softmax function, the attention weights of each head are obtained. The attention weights are multiplied by V to obtain the attention output of each head. Finally, the outputs of each head are concatenated and average pooling is performed. The context vector avg_context obtained after average pooling is then fed into a linear transformation layer to obtain the final output as shown in formulas (16) and (17), where W out b out These are the parameters of the model.

[0085] S105. The predicted values ​​of the missing data are inserted into the positions of the missing values, and the data after the missing values ​​are filled is stored in the power system database.

[0086] The imputation process for predicted values ​​needs to ensure data continuity and consistency, avoiding abrupt changes or unreasonable situations in the imputed data. After imputation is complete, the completed data will be stored again in a centralized database.

[0087] In summary, the model of this invention, by combining RNN and multi-head attention mechanisms, fully utilizes the long-term and short-term dependencies and feature correlations in time-series data to achieve efficient and accurate missing value prediction. The model's modular design and flexible parameter settings enable it to adapt to different application scenarios, providing strong technical support for data completion.

[0088] Example 2

[0089] This embodiment provides a system for completing missing abnormal data in a power system database, including...

[0090] The data acquisition module is used to acquire multi-source data from the power system and process it to obtain multivariate time series data.

[0091] The missing value prediction module is used to predict the missing values ​​of data based on multivariate time series data and a missing data prediction model; wherein, the construction process of the missing data prediction model includes:

[0092] For each time step, based on the extracted mask and time interval of the current time step, a time decay factor is calculated, the hidden state is updated according to the time decay factor, historical predicted missing values ​​are calculated according to the hidden state, and the missing values ​​are filled to obtain the preliminary completed time series data; based on the preliminary completed time series data, feature estimation is performed in conjunction with the attention mechanism to calculate the feature estimate; the combined weight of historical predicted missing values ​​and feature estimate is calculated, and the combined weight is combined with historical predicted missing values ​​and feature estimate to obtain the combined estimate; the hidden state and cell state of the LSTM unit are updated based on the combined estimate.

[0093] After the time step loop ends, the hidden states of all time steps are processed to obtain the context vector. The context vector is then processed using a linear layer to obtain the predicted value of the least missing data.

[0094] The missing value completion module is used to insert the predicted values ​​of the predicted missing data into the positions of the missing values, and to store the data after missing value completion into the power system database.

[0095] Example 3

[0096] This embodiment provides a computer-readable storage medium storing a computer program that, when executed by a processor, implements the steps of the above-described method for completing abnormally missing data in a power system database.

[0097] Example 4

[0098] This embodiment provides a computer device, including a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the program, it implements the steps in the above-described method for completing abnormal missing data in a power system database.

[0099] Example 5

[0100] This embodiment provides a program product, which is a computer program product, including a computer program. When the computer program is executed by a processor, it implements the steps in the above-described method for completing abnormal missing data in a power system database.

[0101] The above description is merely a preferred embodiment of the present invention and is not intended to limit the invention. Various modifications and variations can be made to the present invention by those skilled in the art. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the scope of protection of the present invention.

Claims

1. A method for completing abnormally missing data in a power system database, characterized in that, Includes the following steps: Acquire multi-source data from the power system and process it to obtain multivariate time series data; the multi-source data includes sensor data, log files, and external API data; Prediction is performed based on multivariate time series data and a missing data prediction model to obtain predicted values ​​for the missing data; wherein, the construction process of the missing data prediction model includes: For each time step, based on the extracted mask and time interval of the current time step, the time decay factor is calculated. The formula for calculating the time decay factor is: , , , , in, and Indicates the time decay factor. W x 、b x 、W h 、b h Indicates model parameters, Indicates the current time t The time interval since the last time data was detected. For the current step time series data x t At the corresponding time, For the time series data from the previous step x t-1 At the corresponding time, Indicates the current time t The time interval between the last time data was detected Represents the observed time series data x t The d One characteristic, For the corresponding The mask vector; The hidden state is updated based on the time decay factor. The historical predicted missing values ​​are calculated based on the hidden state, and the missing values ​​are filled to obtain the preliminary completed time series data. Based on the initially completed time-series data, feature estimation is performed using an attention mechanism to calculate the feature estimates. The combined weights of historical predicted missing values ​​and feature estimates are then calculated. Based on these combined weights, a combined estimate is obtained. The hidden states and cell states of the LSTM unit are updated based on this combined estimate, as shown below: , , , , , After each time step loop, the hidden states of all time steps are processed. A multi-head attention mechanism is used, which first performs a linear transformation on the hidden states of all time steps using a parameter matrix. The transformed vector matrix is ​​then divided into H heads, based on the first... i The query, key, and value vectors of each header are used to calculate the attention weight for each header. The outputs of each header are concatenated and average pooling is performed. The resulting context vector is then fed into a linear transformation layer to obtain the final predicted value for the missing data. The predicted value for the missing data is then inserted into the position of the missing value, and the data with the missing value filled in is stored in the power system database.

2. The method for completing abnormally missing data in a power system database as described in claim 1, characterized in that, The formula for calculating the feature estimate based on the initially completed time series data and combined with the attention mechanism is as follows: , , , , , in, This is time series data initially supplemented based on historical data. Q, K, and V are the parameters obtained by linear transformation based on the initially supplemented time series data. , , The vector matrix after linear transformation of the matrix, d model The dimension of the input vector. This is the feature estimate.

3. The method for completing abnormally missing data in a power system database as described in claim 1, characterized in that, After obtaining multivariate time series data, the acquired data is preprocessed, including cleaning, noise reduction and standardization.

4. The method for completing abnormally missing data in a power system database as described in claim 3, characterized in that, After preprocessing the acquired data, integrity checks are performed on the multivariate time series data. If missing values ​​are found in the data, the specific location and time point of the missing values ​​are determined.

5. A system for completing abnormally missing data in a power system database, employing the method for completing abnormally missing data in a power system database as described in any one of claims 1-4, characterized in that, include: The data acquisition module is used to acquire multi-source data from the power system and process it to obtain multivariate time series data; the multi-source data includes sensor data, log files and external API data. The missing value prediction module is used to predict the missing values ​​of data based on multivariate time series data and a missing data prediction model; wherein, the construction process of the missing data prediction model includes: For each time step, based on the extracted mask and time interval of the current time step, a time decay factor is calculated, the hidden state is updated according to the time decay factor, historical predicted missing values ​​are calculated according to the hidden state, and the missing values ​​are filled to obtain the preliminary completed time series data; based on the preliminary completed time series data, feature estimation is performed in conjunction with the attention mechanism to calculate the feature estimate; the combined weight of historical predicted missing values ​​and feature estimate is calculated, and the combined weight is combined with historical predicted missing values ​​and feature estimate to obtain the combined estimate; the hidden state and cell state of the LSTM unit are updated based on the combined estimate. After the time step loop ends, the hidden states of all time steps are processed to obtain the context vector. The context vector is then processed using a linear layer to obtain the predicted value of the least missing data. The missing value completion module is used to insert the predicted values ​​of the predicted missing data into the positions of the missing values, and to store the data after missing value completion into the power system database.

6. A power system database abnormal missing data completion system as described in claim 5, characterized in that, The system also includes a missing value detection module, which is used to preprocess the acquired data and perform integrity detection on the multivariate time series data. If there are missing values ​​in the data, the module determines the specific location and time point of the missing values.

7. A program product, said program product being a computer program product, comprising a computer program, characterized in that, When the computer program is executed by the processor, it implements the steps in the method for completing abnormal missing data in a power system database as described in any one of claims 1-4.

Citation Information

Patent Citations

  • Traffic missing data complementation method based on space-time attention mechanism

    CN113094357A