Time series data quality enhancement method and device based on prediction large model

Abstract

The embodiment of the invention provides a time series data quality enhancement method and device based on a prediction large model. The method is applied to the technical field of data processing, and comprises the following steps: performing multi-scale feature analysis preprocessing according to power grid source load time sequence data to obtain a feature sequence set, performing time sequence modeling through a prediction model to obtain a prediction sequence result, calculating deviation features by adopting a sliding window method to obtain a time sequence deviation feature vector, and calculating the time sequence deviation feature vector; and performing confidence evaluation based on the time sequence deviation feature vector to generate a data exception mark, performing missing value positioning on the time sequence to obtain a null value index set, and performing repair processing in combination with a data completion model to obtain a complete time sequence data set. According to the method, comprehensive quality improvement of the power grid source load data is realized through a time sequence data quality enhancement method of a prediction large model.

Description

TECHNICAL FIELD

[0001] The present disclosure relates to the field of data processing, and in particular to a time series data quality enhancement method and device based on a large prediction model. BACKGROUND

[0002] The quality of power grid source-load data is crucial for the safe and stable operation of the power system. Existing data quality processing methods mainly include anomaly detection based on statistical analysis, data cleaning based on rules, and data completion based on traditional machine learning techniques. Statistical analysis methods identify anomalies by setting fixed thresholds, rule-based data cleaning relies on pre-defined cleaning rules for data processing, and machine learning methods use historical data to train models for data repair. These methods have achieved certain results in practical applications and provided technical support for improving the quality of power grid data.

[0003] However, existing technologies still have obvious deficiencies in processing complex power grid source-load data. First, fixed thresholds and predetermined rules are difficult to adapt to the dynamic characteristics of power grid data, leading to false positives and false negatives. Second, traditional machine learning methods do not adequately consider the time series characteristics of data, making it difficult to effectively capture long-term dependencies in the data. Third, existing data completion methods often process each missing point independently, ignoring the temporal correlation between data, resulting in poor continuity and consistency of the completion results. In addition, existing methods lack a unified framework for processing multi-scale data, making it difficult to comprehensively utilize information from different time scales for data quality enhancement. SUMMARY

[0004] The present disclosure provides a time series data quality enhancement method and device based on a large prediction model, which comprehensively improves the quality of power grid source-load data.

[0005] According to a first aspect of the present disclosure, a time series data quality enhancement method based on a large prediction model is provided, comprising: performing data preprocessing on power grid source-load time series data through multi-scale feature analysis to obtain a feature sequence set; performing time series modeling on the feature sequence set through a prediction model to obtain a prediction sequence result; calculating the deviation features of actual observation data using a sliding window method based on the prediction sequence result to obtain a time series deviation feature vector; performing data analysis based on the time series deviation feature vector through confidence evaluation to generate a data anomaly label; positioning the missing values of the time series according to the data anomaly label to obtain a null index set; and performing repair processing on the power grid source-load data based on the null index set and a data completion model to obtain a complete time series data set.

[0006] According to a second aspect of the present disclosure, a time series data quality enhancement device based on a large prediction model is provided, comprising:

[0007] A collection module is configured to perform data preprocessing on power grid source-load time series data through multi-scale feature analysis to obtain a feature sequence set;

[0008] A modeling module is configured to perform time series modeling on the feature sequence set through a prediction model to obtain a prediction sequence result;

[0009] A calculation module is configured to calculate bias features of actual observation data according to the prediction sequence result using a sliding window method to obtain a time series bias feature vector;

[0010] An analysis module is configured to perform data analysis based on the time series bias feature vector through confidence evaluation to generate a data anomaly label;

[0011] A positioning module is configured to position missing values in time series according to the data anomaly label to obtain a null value index set;

[0012] A processing module is configured to repair power grid source-load data according to the null value index set in combination with a data completion model to obtain a complete time series data set.

[0013] According to a third aspect of the present disclosure, an electronic device is also provided, comprising a processor, a memory and a bus, the memory stores machine readable instructions executable by the processor, when the power sample data quality evaluation device is running, the processor and the memory communicate through the bus, and the machine readable instructions are executed by the processor to perform the steps of a time series data quality enhancement method based on a large prediction model as described in the first aspect.

[0014] According to a fourth aspect of the present disclosure, a computer readable storage medium is also provided, the computer readable storage medium stores instructions, characterized in that the instructions are executed by the processor to implement a time series data quality enhancement method based on a large prediction model as described in the first aspect.

[0015] The present disclosure effectively captures the feature information of power grid source and load time series data at different time scales through multi-scale feature analysis for data preprocessing, improves the comprehensiveness and accuracy of feature extraction, and lays a solid foundation for subsequent data quality enhancement. A prediction model is used to model the time series of the feature sequence set, fully utilizes the time series dependence of the data, accurately predicts the trend of data change, and significantly improves the reliability of anomaly detection. The deviation features of the prediction sequence results and the actual observation data are calculated by the sliding window method, the local change characteristics of the data are dynamically captured, and the anomaly detection is more sensitive and accurate. Based on the confidence evaluation mechanism of the time series deviation feature vector, an objective data anomaly judgment standard is established, which effectively reduces the false positive rate and the false negative rate. The identified abnormal data is positioned for missing values, and the null index set is accurately obtained, providing accurate position guidance for data repair. Finally, the data is repaired by the data completion model, which maintains the time series characteristics of the data while realizing high-quality data completion, ensuring that the repaired data meets the physical law and maintains the time series continuity. The quality of the power grid source and load data is improved, and the efficiency and accuracy of data processing are improved.

[0016] It should be understood that the content described in the summary section is not intended to limit the key or important features of the embodiments of the present disclosure, nor to limit the scope of the present disclosure. Other features of the present disclosure will become apparent through the following description. BRIEF DESCRIPTION OF DRAWINGS

[0017] The above and other features, advantages, and aspects of embodiments of the present disclosure will become more apparent by describing in detail the following embodiments with reference to the attached drawings. The attached drawings are intended to better understand the present disclosure and do not limit the present disclosure. In the drawings, the same or similar elements are denoted by the same or similar reference numerals, and:

[0018] Figure 1 A flowchart of a time series data quality enhancement method based on a large prediction model according to an embodiment of the present disclosure is shown;

[0019] Figure 2 A block diagram of a time series data quality enhancement device based on a large prediction model according to an embodiment of the present disclosure is shown;

[0020] Figure 3 A structural schematic diagram of an electronic device according to an embodiment of the present disclosure is shown. DETAILED DESCRIPTION

[0021] In order to make the purposes, technical solutions and advantages of the embodiments of the present disclosure clearer, the technical solutions in the embodiments of the present disclosure will be described clearly and completely below with reference to the drawings in the embodiments of the present disclosure. Obviously, the described embodiments are only some of the embodiments of the present disclosure, rather than all the embodiments. Based on the embodiments in the present disclosure, all other embodiments obtained by a person of ordinary skill in the art without creative work fall within the protection scope of the present disclosure.

[0022] In addition, the term "and / or" herein is only to describe the association relationship of the associated objects, which means that there can be three relationships, for example, A and / or B can represent the three cases of A alone, A and B together, and B alone. In addition, the character " / " herein generally represents an "or" relationship between the front and rear associated objects.

[0023] Figure 1 A flowchart of a time series data quality enhancement method 100 based on a prediction large model in the embodiments of the present disclosure is shown, as shown in Figure 1 The method 100 includes the following steps:

[0024] S110: performing data preprocessing on the power grid source and load time series data through multi-scale feature analysis to obtain a feature sequence set;

[0025] Optionally, the power grid source and load time series data is segmented by a time window and normalized, and data quality is evaluated and cleaned in combination with time series continuity to obtain normalized time series data; multi-dimensional feature extraction and feature correlation analysis are performed on the basis of the normalized time series data to obtain a multi-scale feature matrix, and feature quantization and weight calculation are performed in combination with data distribution characteristics; data dimension reduction and dimension conversion are performed on the multi-scale feature matrix, and feature screening and importance sorting are performed in combination with data integrity inspection to form reduced dimension feature data; data preprocessing is performed on the basis of the reduced dimension feature data through time dimension mapping and sequence reconstruction, and data integration and normalization are performed in combination with feature effectiveness verification to generate a feature mapping set; time series correlation analysis and feature combination are performed on the feature mapping set, and feature fusion and sequence splicing are performed in combination with data distribution rules to obtain a feature combination sequence; the feature combination sequence is processed through data format conversion and dimension regularization, and sequence merging and normalization are performed in combination with integrity verification to obtain a feature sequence set.

[0026] Specifically, time window segmentation and data normalization processing are performed. The power grid source and load time series data contain measurement values of multiple physical quantities such as voltage, current and power. These data are collected at fixed time intervals. Time window segmentation divides the continuous time series data by setting a fixed size time window, and each window contains a certain number of data points. Data normalization processing uses standardization methods to unify data of different physical quantities into the same numerical range, eliminating dimensional differences. Time series continuity analysis focuses on the time correlation between data points. By checking the time interval of data collection, data change trend and other characteristics, the data quality is evaluated and the obviously abnormal data points are removed to form normalized time series data. When extracting multi-dimensional features from normalized time series data, features are extracted from multiple dimensions such as time domain and frequency domain. Time domain features include statistical features such as mean, variance and kurtosis of data. Frequency domain features are obtained by Fourier transform to obtain frequency composition features of data. Feature correlation analysis calculates the correlation coefficient between different features to identify the dependency between features. Multi-scale feature analysis decomposes data at different time scales, including hourly, daily, weekly and other scales, to extract features at each scale. Data distribution characteristic analysis includes probability distribution characteristics and abnormal value distribution of data. Based on these characteristics, feature quantization is performed, and weights are assigned according to the importance of features to form a multi-scale feature matrix. The multi-scale feature matrix is processed by data dimension reduction, and the feature dimension is reduced by principal component analysis and other methods to retain the main feature information. Dimension conversion maps high-dimensional feature space to low-dimensional feature space while maintaining the main feature structure of the data. Data integrity test ensures the integrity of the data during feature extraction and dimension reduction. The features are sorted by feature importance evaluation, and the most representative feature subset is selected to obtain the reduced feature data. Time dimension mapping is performed on the reduced feature data to remap the feature data to the time axis, maintaining the time series characteristics of the data. During sequence reconstruction, the mapped data is reorganized to ensure the continuity and integrity of the data. Feature effectiveness verification evaluates the effectiveness of the reconstructed features, and integrates related features through data integration to form a feature mapping set after normalization processing.

[0027] Time series correlation analysis is performed on the feature mapping set to study the correlation between data at different time points and identify the time series dependency pattern of the data. During feature combination, related features are combined according to the time series correlation, and feature fusion is performed based on the data distribution law. Sequence splicing splices different feature sequences in chronological order to form a unified feature combination sequence. Data format conversion is performed on the feature combination sequence to unify the data format and normalize the dimensions. Integrity verification ensures the integrity and consistency of the data during processing, and related sequences are integrated through sequence merging to form a feature sequence set.

[0028] For example, the raw data contains voltage, current, active power and other measurement data of the substation. The time window segmentation divides the 24-hour continuous data into 24 time windows according to the window size of 1 hour. The data in each window is normalized to convert the voltage data from kilovolts and the current data from kiloamperes to a unified numerical interval. Through the analysis of time sequence continuity, the breakpoints or abnormal jumps in the data acquisition process are found, and these abnormal points are cleaned. In the multi-dimensional feature extraction stage, the statistical features of the data in each time window are calculated, such as the mean and standard deviation of the voltage, and the frequency characteristics of the data are analyzed to identify periodic fluctuation patterns. The data features are analyzed at multiple time scales, such as hourly load fluctuations, daily electricity usage patterns, and weekly periodic changes. The correlation analysis between voltage and current and the dependence between power and load are found, and the feature quantization and weight distribution are performed. The data dimension reduction process retains the main feature dimensions, such as the load change trend and voltage fluctuation characteristics. The time dimension mapping maps these features to the time sequence, and the feature sequence is formed through feature combination.

[0029] S120: Time series modeling of the feature sequence set through the prediction model to obtain a prediction sequence result;

[0030] Optionally, according to the data structure and time sequence characteristics of the feature sequence set, a prediction model is constructed through multi-layer design, the prediction model including an input layer, a data coding layer, a time sequence feature extraction layer, a prediction output layer and a feedback adjustment layer; the feature sequence set is subjected to data format conversion and dimension reconstruction through the input layer, and data alignment and sequence splicing are performed through time correlation analysis to obtain an input feature matrix; the input feature matrix is subjected to feature mapping and dimension transformation through the data coding layer, feature space reconstruction and data compression are performed through a nonlinear transformation function, and an encoded feature vector is obtained; time dependence analysis and long short-term memory unit processing are performed on the encoded feature vector in the time sequence feature extraction layer, time sequence relationship modeling is performed through multi-scale feature fusion, and a time sequence feature representation is generated; prediction calculation and result generation are performed on the time sequence feature representation in the prediction output layer, prediction error analysis and result correction are performed through historical data, and an initial prediction result is formed; the initial prediction result is subjected to prediction bias calculation and correction parameter update through the feedback adjustment layer, and the result is optimized and the sequence is reorganized through historical prediction accuracy, and a prediction sequence result is obtained.

[0031] Specifically, based on the data structure of the feature sequence set and the timing characteristics, the constructed prediction model adopts a multi-layer design structure, each layer having a specific data processing function. The prediction model includes five main functional layers: the input layer is responsible for initial processing and format conversion of data; the data encoding layer performs conversion and compression of the feature space; the timing feature extraction layer analyzes the time dependence of the data; the prediction output layer generates the prediction result; and the feedback adjustment layer optimizes the prediction result. The input layer uniformly converts the data format of the feature sequence set, converting different forms of feature data into a standard format. The dimension reconstruction process reorganizes the original features in chronological order, ensuring that the timestamps of the data are aligned. The time correlation analysis focuses on the time dependence between data points, identifying the time correlation patterns of the feature sequence by calculating the time correlation coefficient. The data alignment process ensures that different feature sequences are completely matched in the time dimension, and the missing time points of the data are interpolated or filled. The sequence splicing combines the aligned multiple feature sequences in chronological order to form a unified input feature matrix.

[0032] The data encoding layer receives the input feature matrix and converts the original feature space to a new feature space through feature mapping. Dimension transformation uses dimension reduction or dimension elevation operations to adjust the dimension of the feature space to adapt to the subsequent processing requirements. Nonlinear transformation functions (such as ReLU, sigmoid, etc.) are used to capture the nonlinear relationship between features, enhancing the expression ability of the model. Feature space reconstruction reorganizes the transformed features to establish a new feature representation. Data compression removes redundant information and retains key feature information to obtain compressed encoded feature vectors. The timing feature extraction layer processes the encoded feature vectors, focusing on analyzing the time dependence of the data. Time dependence analysis includes identification of short-term dependence and long-term dependence, capturing timing patterns by calculating feature correlation at different time scales. Long Short-Term Memory units (LSTM units) specifically handle long-term dependencies by controlling information transmission and retention through a gating mechanism. Multi-scale feature fusion integrates features at different time scales to establish a timing relationship model, generating a comprehensive timing feature representation.

[0033] The prediction output layer performs prediction calculation based on the timing feature representation. The prediction calculation process considers both short-term trends and long-term regularities to generate preliminary prediction values. Historical data are used to evaluate the rationality of the prediction results, and prediction error analysis is performed by comparing the change trend of the historical data. Result correction adjusts the prediction values according to the results of error analysis to form the initial prediction results.

[0034] The feedback adjustment layer further optimizes the initial prediction result. The prediction bias calculation compares the difference between the predicted value and the actual value to quantify the prediction accuracy. The correction parameter update adjusts the model parameters based on the prediction bias to improve the prediction accuracy. The historical prediction accuracy serves as an important reference to guide the direction and amplitude of parameter adjustment. The result optimization process considers multiple factors to fine-tune the prediction result. The sequence reconstruction organizes the optimized prediction result into a standard format prediction sequence result. In the power grid load data prediction, the original feature sequence set contains multiple characteristic time series such as electricity load, voltage, power factor, etc. The input layer uniformly converts these feature data into matrix format, and after aligning the time stamp, it is found that the power factor data is missing at some time points, which is supplemented by linear interpolation method. The data encoding layer uses a nonlinear activation function to process the feature data, capturing the nonlinear relationship between the load and the voltage. The time series feature extraction layer identifies that the power load has obvious daily and weekly periodic characteristics, and the LSTM unit effectively captures this long-term dependence. The prediction output layer generates the load prediction value for the next 24 hours, and the feedback adjustment layer corrects the prediction result by analyzing the historical prediction error to improve the prediction accuracy.

[0035] S130: According to the prediction sequence result, the deviation characteristics of the actual observation data are calculated by using the sliding window method, and the time series deviation characteristic vector is obtained;

[0036] Optionally, a difference matrix is constructed according to the prediction sequence result and the actual observation data according to the time sequence relationship, and time stamp alignment and data matching are performed through comparison and analysis to obtain sequence deviation data; according to the sequence deviation data, sliding window segmentation is performed, and window boundary determination and data completion are performed in combination with time series continuity and data integrity, to generate multiple groups of time window analysis sequences; the multiple groups of time window analysis sequences are subjected to data feature calculation through numerical range statistics and distribution characteristic analysis, and feature quantization is performed in combination with linear correlation evaluation, to obtain a window feature set; difference quantization and variance analysis are performed on the window feature set, and feature importance sorting and weight distribution are performed in combination with data fluctuation law, to form a weighted deviation feature; the weighted deviation feature is subjected to time dimension mapping and sequence reconstruction, and feature combination and correlation analysis are performed in combination with data integrity test, to obtain a feature mapping sequence; the feature mapping sequence is subjected to vectorization conversion and dimension regularization, and feature fusion and sequence merging are performed in combination with data effectiveness verification, to obtain a time series deviation characteristic vector.

[0037] Specifically, the difference matrix calculates the numerical difference at each time point by pairing the predicted sequence results with the actual observation data by timestamp. Timestamp alignment ensures that the two sets of data are completely matched in the time dimension, and the data matching process unifies the data format, making the difference calculation comparable, so as to obtain accurate sequence deviation data. The sequence deviation data is segmented by the sliding window method. The sliding window moves on the time series with a fixed size, and each movement produces a data subset of a time window. Time series continuity analysis focuses on the continuity characteristics of the data in the window, ensuring that the division of the window boundary does not destroy the time correlation of the data. Data integrity check identifies missing values or outliers in the window, and through data completion processing, ensures the data integrity of each window, forming multiple sets of time window analysis sequences.

[0038] Numerical range statistics are performed on each set of time window analysis sequences, and statistical characteristics of the data in the window, such as mean, standard deviation, etc. are calculated. Distribution characteristic analysis includes statistical quantities such as probability distribution characteristics, skewness, kurtosis, etc. of the data. Linear correlation evaluation calculates the correlation coefficient between the data points in the window, quantifying the linear dependence relationship of the data. Feature quantization converts these statistical characteristics and correlation indicators into numerical representations to form a window feature set. The window feature set is processed by difference quantization to calculate the feature difference between different windows. Variance analysis evaluates the fluctuation of the features and identifies significant features. Data fluctuation rule analysis focuses on the trend and periodic characteristics of the feature values. Feature importance sorting sorts the features according to their significance, and weight allocation allocates weight coefficients according to the importance of the features, forming weighted deviation features.

[0039] The weighted deviation features are mapped in the time dimension, and the features are re-mapped to the time series. The sequence reconstruction process reorganizes the mapped features in chronological order, maintaining the time series characteristics of the data. Data integrity check ensures that important information is not lost during the reconstruction process. Feature combination combines related features, correlation analysis studies the dependence between features, and generates a feature mapping sequence. The feature mapping sequence is converted into a standard vector format through vectorization. Dimension regularization unifies the vector dimension, ensuring the consistency of the data format. Data validity verification evaluates the effectiveness and representativeness of the features, feature fusion integrates multiple related features, and sequence merging forms a time series deviation feature vector.

[0040] In the analysis of grid source and load data, the difference between the predicted value and the actual observation value is analyzed as follows: a difference matrix is constructed based on the 24-hour load prediction result and the actual load data, data segmentation is performed through a 1-hour sliding window to obtain 24 analysis windows. The statistical characteristics of the load deviation are calculated in each window, and the load fluctuation law is analyzed. The prediction deviation in the peak period is focused on, and the characteristics of these key periods are highlighted through weight allocation. The obtained time series deviation feature vector reflects the accuracy and reliability of the prediction result, providing a basis for subsequent anomaly detection.

[0041] S140: data analysis is performed based on the time series deviation feature vector through confidence evaluation to generate data anomaly labels;

[0042] Optionally, confidence interval calculation and probability distribution analysis are performed on the time series deviation feature vector, and deviation threshold determination and boundary division are performed in combination with historical data to obtain a confidence evaluation standard; statistical significance analysis and volatility test are performed on the deviation value according to the confidence evaluation standard, and the degree of abnormality is quantified and scored through data distribution characteristics to obtain an abnormality quantification result; feature fusion is performed through multi-dimensional feature combination and correlation analysis on the abnormality quantification result, and abnormality pattern recognition and classification are performed in combination with time series continuity to generate an abnormality feature matrix; time dimension mapping and sequence reconstruction are performed based on the abnormality feature matrix, and feature combination and correlation verification are performed in combination with data integrity test to form an abnormality label candidate set; label screening is performed through confidence level determination and abnormality degree evaluation according to the abnormality label candidate set, and label optimization and adjustment are performed in combination with time series continuity to obtain an abnormality label sequence; data format conversion and dimension regularization are performed on the abnormality label sequence, and label merging and sequence integration are performed in combination with effectiveness verification to obtain data anomaly labels.

[0043] Specifically, anomaly detection on the time series deviation feature vector first performs confidence interval calculation. The confidence interval is based on the statistical distribution characteristics of historical data, and the upper and lower limits of the normal fluctuation range are determined by calculating the mean and standard deviation of the data. Probability distribution analysis studies the distribution patterns of data, such as normal distribution and skew distribution, to help understand the variation law of data. In the process of determining the deviation threshold, the fluctuation law of historical data is combined to set a reasonable fluctuation boundary to form a confidence evaluation standard. Statistical significance analysis is performed based on the confidence evaluation standard, and hypothesis testing method is used to judge whether the data deviation is significant. Volatility test focuses on the change amplitude and frequency of data to identify abnormal fluctuation patterns. Data distribution characteristic analysis includes the calculation of skewness, kurtosis and other statistical quantities to evaluate the degree of abnormality. Abnormality quantification converts qualitative abnormality features into quantitative score results to form abnormality quantification results.

[0044] The abnormality quantification result is combined with multiple dimensions to integrate abnormal features of different dimensions. The correlation analysis studies the mutual relationship between abnormal features and identifies the dependency pattern between features. The time sequence continuity analysis focuses on the time continuity of abnormal occurrence, which helps to distinguish isolated anomalies from continuous anomalies. The abnormal pattern recognition classifies similar abnormal patterns to form an abnormal feature matrix. The abnormal feature matrix is mapped to the time dimension to re-couple the abnormal features to the time series. The sequence reconstruction process maintains the time sequence relationship of the abnormal features and ensures the time continuity of the abnormal detection result. The data integrity test ensures that important abnormal information is not missed in the reconstruction process. The feature combination and correlation verification further confirm the effectiveness of the abnormal features to generate an abnormal marker candidate set.

[0045] The abnormal marker candidate set is screened by confidence level determination, and a suitable confidence level threshold is set to distinguish normal and abnormal data. The abnormality degree evaluation considers the influence of multiple abnormal features to determine the abnormal marker. The time sequence continuity analysis guides the optimization and adjustment of the marker to avoid fragmentation of the marker result and forms a continuous abnormal marker sequence. The abnormal marker sequence is converted into a standard data format. The dimension regularization ensures the dimension consistency of the marker data. The effectiveness verification evaluates the accuracy and reliability of the marker result. The marker merging merges adjacent or related abnormal markers to form data abnormal markers.

[0046] In the power grid source-load data anomaly detection, the time sequence deviation feature vector reflects the change characteristics of the load prediction deviation. By analyzing the distribution law of historical load data, reasonable confidence interval and deviation threshold are set. Statistical analysis is performed on the prediction deviation, focusing on the abnormal values that exceed the normal fluctuation range. Abnormal features include sudden increase, sudden decrease, and sustained deviation, and these abnormal patterns are identified by time sequence analysis. The generated data abnormal markers clearly indicate the abnormal data points that need to be focused on, providing accurate location information for subsequent data repair.

[0047] S150: According to the data abnormal marker, the time sequence is positioned for missing value, and an empty value index set is obtained;

[0048] Optionally, the time sequence position analysis and data integrity test are performed for the data anomaly mark, sequence alignment and missing point identification are performed in combination with the timestamp information to obtain an initial missing position set; the missing interval sequence is obtained by performing data continuity analysis and time sequence correlation verification according to the initial missing position set, and interval boundary determination and adjustment are performed in combination with data distribution characteristics; the missing interval sequence is subjected to time dimension mapping and feature reconstruction, and interval merging and segmentation are performed in combination with data validity test to form a missing feature matrix; data positioning is performed based on the missing feature matrix through position index generation and coordinate mapping, and index verification and correction are performed in combination with time sequence continuity to generate a position index sequence; the position index sequence is subjected to data format conversion and dimension regularization, and index optimization and adjustment are performed in combination with integrity test to obtain an index mark set; the index mark set is integrated through time sequence correlation analysis and position mapping, and sequence merging and normalization processing are performed in combination with validity verification to obtain a null value index set.

[0049] Specifically, the data anomaly mark is subjected to in-depth time sequence position analysis. The time sequence position analysis starts from the time dimension of data, sequentially checks the data state of each time point, and determines the specific position of the anomaly mark in the time sequence. The data integrity test scans the entire time sequence to identify the missing position of data. The timestamp information is a key reference, and sequence alignment is performed by comparing the timestamp of each data point to ensure the consistency of the time dimension. In the missing point identification process, attention is focused on the positions where the data value is empty, abnormal or does not conform to the expected format, and the timestamp and position information of these positions are recorded to form an initial missing position set. Based on the initial missing position set, deeper data continuity analysis is performed. Continuity analysis focuses on the time interval between missing points to determine whether the missing is a single point or continuous. Time sequence correlation verification confirms the range and boundary of the missing by analyzing the data features of adjacent time points. In the missing interval division process, the distribution characteristics of data need to be considered, including periodicity, trend and other features. When determining the interval boundary, the change law of data is combined to avoid misjudging normal data fluctuations as missing. Boundary adjustment ensures that the divided interval can completely cover the missing data without excessively expanding the missing range, forming an accurate missing interval sequence.

[0050] The sequence of the missing interval needs to be mapped in the time dimension, and the interval information is re-mapped to the time axis to maintain the time sequence characteristics of the data. In the feature reconstruction process, feature descriptions are extracted for each missing interval, including interval length, start time, end time, and other information. Data validity verification ensures that the reconstructed features accurately reflect the missing situation. Interval merging processes adjacent or overlapping missing intervals to avoid redundant marking. Interval segmentation processes oversized missing intervals to ensure the rationality of interval division and constructs a missing feature matrix. The missing feature matrix is the basis for accurate positioning. Position index generation assigns a unique index identifier to each missing position, and coordinate mapping corresponds these indexes to specific positions in the time series. Time sequence continuity plays a role again, and the accuracy of the positioning results is verified by checking the continuity of the indexes. In the index correction process, inaccurate indexes are corrected to ensure the accuracy and integrity of the position index sequence.

[0051] The position index sequence is standardized to unify the data format and make the index representation consistent. Dimension regularization ensures that the dimension of the index sequence is consistent with the original data. Integrity verification verifies the coverage of the index to avoid missing missing positions. In the index optimization process, the representation or organization structure of the index may need to be adjusted to ensure the effectiveness and usability of the index, forming an index marker set. Time sequence correlation analysis studies the correlation between indexes and identifies index groups with time sequence dependence. Position mapping confirms the correspondence between indexes and actual positions again. Effectiveness verification is the last quality control to ensure that the integrated indexes accurately reflect the missing positions. Sequence merging integrates related indexes, normalizes the representation format of the indexes, and forms a null value index set.

[0052] In the processing of power grid source-load data, this process is as follows: First, identify the abnormal value positions in the load data, such as data interruption caused by sudden power failure and data anomalies caused by equipment failure. By analyzing the time distribution of these abnormal points, the specific time period of missing data is determined. For example, a power line appears data anomalies in continuous monitoring, and through time sequence analysis, it is found that these anomalies are concentrated in a specific period, forming a missing interval. Further analysis shows that this interval coincides with the equipment maintenance time, confirming the rationality of the missing. Record these missing positions through position index, and merge adjacent missing points into continuous intervals to form an accurate null value index set, providing accurate position guidance for subsequent data repair. The whole positioning process is rigorous and meticulous, ensuring the accuracy and integrity of the missing position identification.

[0053] S160: Repair the power grid source-load data according to the null value index set combined with the data completion model to obtain a complete time series data set.

[0054] Optionally, the data completion model is assembled by hierarchical construction, and the data completion model comprises a data input layer, a missing value encoding layer, a feature reconstruction layer, a data generation layer, and a result verification layer; the missing value index set is used for position positioning and data format conversion in the data input layer, and the power grid source and load data are used for time sequence alignment and range marking, to obtain an input feature matrix; the input feature matrix is mapped to a missing position and encoded by the missing value encoding layer, and the encoding is standardized and the dimension is adjusted in combination with time sequence continuity, to obtain a missing feature vector; the missing feature vector is used for time sequence correlation analysis and feature reconstruction in the feature reconstruction layer, and feature fusion and weight allocation are performed in combination with data distribution rules, to generate a reconstructed feature matrix; the reconstructed feature matrix is used for missing value filling and data generation by the data generation layer, and the generated result is optimized and corrected in combination with historical data distribution, to form a completed data sequence; the completed data sequence is used for effectiveness test and rationality evaluation in the result verification layer, and sequence integration and standardization processing are performed in combination with data integrity, to obtain a complete time sequence data set.

[0055] Specifically, the data completion model adopts a multi-layer structure design, and each layer bears a specific data processing function. When the model is constructed, a processing architecture is first formed by hierarchical assembly. The data completion model comprises five functional layers: the data input layer processes original data conversion, the missing value encoding layer maps missing positions, the feature reconstruction layer reconstructs data features, the data generation layer fills missing values, and the result verification layer ensures data quality.

[0056] The data input layer accurately positions the missing positions based on the missing value index set, and simultaneously performs format standardization processing on the power grid source and load data. For filling of the missing values, the reconstruction process of the input feature matrix X is first defined:

[0057]

[0058] Wherein, R(X) represents a reconstruction function, n is a feature dimension, a i is a feature weight coefficient, F i (X m ) is a mapping function of the i-th feature, X m is a missing position matrix, β is a time sequence weight coefficient, and D(X t ) is a time sequence dependent function.

[0059] The missing value encoding layer converts the input feature matrix into a standard encoding format, while performing dimension standardization. The feature reconstruction layer deeply analyzes the missing feature vector and reconstructs the data feature structure. In the data generation process, the missing values are calculated by using a weighted reconstruction method:

[0060] V fill (t)=ω1H(t-k)+ω2P(t)+ω3S(t)

[0061] wherein, V fill (t) is the filling value at time t, ω1, ω2, ω3 are weight coefficients, H(t-k) is the influence factor of historical data, k is the time delay, P(t) is the periodic characteristic function, and S(t) is the seasonal adjustment factor. The data generation layer generates the filling value of the missing position according to the reconstructed feature matrix, and optimizes the generation result to make it conform to the data distribution law. The result verification layer comprehensively verifies the completed data sequence to ensure the effectiveness and rationality of the generated data, forming a time series data set.

[0062] In the power grid source-load data completion, the model processing process is embodied as follows: firstly, the missing positions in the load data are accurately located, and the periodic characteristics of the load curve are combined for coding. The feature reconstruction considers the time series correlation and fluctuation law of the load to generate reasonable filling values. The accuracy of the filling result is verified through the distribution characteristics of the historical load data, so as to ensure that the completed data not only maintains the original time series characteristics, but also meets the change law of the power grid load. The entire completion process forms a data processing chain, and each layer works cooperatively to realize high-quality data repair.

[0063] Figure 2 A block diagram of a time series data quality enhancement device 200 based on a large prediction model according to an embodiment of the present disclosure is shown. As shown in Figure 2 the device 200 includes:

[0064] The acquisition module 210 is configured to perform data preprocessing on the power grid source-load time series data through multi-scale feature analysis to obtain a feature sequence set;

[0065] The modeling module 220 is configured to perform time series modeling on the feature sequence set through a prediction model to obtain a prediction sequence result;

[0066] The calculation module 230 is configured to calculate the deviation features of the actual observation data according to the prediction sequence result by using a sliding window method to obtain a time series deviation feature vector;

[0067] The analysis module 240 is configured to perform data analysis based on the time series deviation feature vector by confidence evaluation to generate a data anomaly label;

[0068] The positioning module 250 is configured to locate missing values in the time series according to the data anomaly label to obtain a null index set;

[0069] The processing module 260 is configured to perform repair processing on the power grid source-load data according to the null index set in combination with a data completion model to obtain a complete time series data set.

[0070] Those skilled in the art can clearly understand that, for the convenience and brevity of description, the specific working processes of the described modules can refer to the corresponding processes in the foregoing method embodiments, which will not be described here.

[0071] Based on the same technical concept, the present application also provides an electronic device. Referring to Figure 3 The structure schematic diagram of the electronic device 300 provided by the embodiment of the present application is shown in FIG. 3, which includes a processor 301, a memory 302, and a bus 303. The memory 302 is used to store execution instructions, including an internal memory 3021 and an external memory 3022. The internal memory 3021 is also called an internal storage, which is used to temporarily store operation data in the processor 301 and exchange data with the external memory 3022 such as a hard disk. The processor 301 exchanges data with the external memory 3022 through the internal memory 3021. When the electronic device 300 is running, the processor 301 communicates with the memory 302 through the bus 303. The machine readable instructions are executed by the processor to perform the steps of the multivariate time series prediction method based on a prediction large model as described in the first aspect.

[0072] The present application also provides a computer readable storage medium, which can be a non-volatile computer readable storage medium or a volatile computer readable storage medium. The computer readable storage medium stores instructions, which, when executed on a computer, cause the computer to perform the steps of the multivariate time series prediction method based on a prediction large model.

[0073] Those skilled in the art can clearly understand that, for the convenience and brevity of description, the specific working processes of the above-described system, system and unit can refer to the corresponding processes in the foregoing method embodiments, which will not be described here.

[0074] The integrated unit, if realized in the form of a software function unit and sold or used as an independent product, can be stored in a computer readable storage medium. Based on such understanding, the technical solutions of the present application essentially or the part that contributes to the prior art or the whole or part of the technical solutions can be embodied in the form of a software product. The computer software product is stored in a storage medium, including a plurality of instructions to make an electric power sample data quality evaluation device (which can be a personal computer, a server, or a network device, etc.) execute all or part of the steps of the method described in each embodiment of the present application. The foregoing storage medium includes a U disk, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk, and various media that can store program codes.

[0075] The above-described embodiments are only used to illustrate the technical solutions of the present application, but not limit the present application; although the present application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that: it can still modify the technical solutions recorded in the foregoing embodiments, or make equivalent replacement for part of the technical features; and these modifications or replacements do not make the essence of the corresponding technical solutions deviate from the spirit and scope of the technical solutions of the embodiments of the present application.

[0076] The above specific embodiments do not constitute a limitation on the protection scope of the present disclosure. Those skilled in the art should understand that various modifications, combinations, sub-combinations, and replacements can be made according to design requirements and other factors. Any modification, equivalent replacement, and improvement within the spirit and principles of the present disclosure should be included in the protection scope of the present disclosure.

Claims

1. A method for enhancing the quality of time series data based on a large prediction model, characterized in that, include: Based on the power grid source-load time series data, data preprocessing is performed through multi-scale feature analysis to obtain a feature sequence set. A prediction model is then used to perform time series modeling on the feature sequence set to obtain predicted sequence results. Based on the predicted sequence results, a sliding window method is used to calculate the deviation characteristics of the actual observed data, resulting in a time series deviation feature vector. Based on the time series deviation feature vector, data analysis is performed through confidence assessment to generate data anomaly markers. Missing values ​​in the time series are located based on the data anomaly markers to obtain a null value index set. Finally, based on the null value index set and a data completion model, the power grid source-load data is repaired to obtain a complete time series dataset.

2. The method for enhancing the quality of time series data based on a large prediction model according to claim 1, characterized in that, The process of preprocessing data based on power grid source-load time series data through multi-scale feature analysis yields a feature sequence set, including: Based on the power grid source-load time series data, time window segmentation and data normalization are performed, and data quality assessment and cleaning are combined with time series continuity to obtain normalized time series data. Based on the standardized time series data, multi-scale feature analysis is performed through multi-dimensional feature extraction and feature correlation analysis, and feature quantization and weight calculation are combined with data distribution characteristics to obtain a multi-scale feature matrix; The multi-scale feature matrix is ​​subjected to data dimensionality reduction and dimensionality transformation, and feature selection and importance ranking are performed in conjunction with data integrity checks to form dimensionality-reduced feature data; Based on the dimensionality-reduced feature data, data preprocessing is performed through time dimension mapping and sequence reconstruction, and data integration and normalization are combined with feature validity verification to generate a feature mapping set; Temporal correlation analysis and feature combination are performed on the feature mapping set, and feature fusion and sequence splicing are performed in combination with data distribution patterns to obtain the feature combination sequence; The feature combination sequence is processed through data format conversion and dimension normalization, and sequence merging and normalization are combined with integrity verification to obtain the feature sequence set.

3. The method for enhancing the quality of time series data based on a large prediction model according to claim 1, characterized in that, The step of performing time-series modeling on the feature sequence set using a prediction model to obtain the predicted sequence result includes: Based on the data structure and temporal characteristics of the feature sequence set, a prediction model is constructed through a multi-layer design. The prediction model includes an input layer, a data encoding layer, a temporal feature extraction layer, a prediction output layer, and a feedback adjustment layer. The feature sequence set is processed through the input layer for data format conversion and dimension reconstruction, and combined with time correlation analysis for data alignment and sequence concatenation to obtain the input feature matrix; The input feature matrix is ​​subjected to feature mapping and dimension transformation through a data encoding layer, and feature space reconstruction and data compression are performed by combining a nonlinear transformation function to obtain the encoded feature vector. Based on the encoded feature vector, time dependency analysis and long short-term memory unit processing are performed in the temporal feature extraction layer, and temporal relationship modeling is performed through multi-scale feature fusion to generate temporal feature representation; Based on the temporal feature representation, prediction calculation and result generation are performed in the prediction output layer. At the same time, historical data is combined to perform prediction error analysis and result correction to form an initial prediction result. The initial prediction results are processed through a feedback adjustment layer to calculate prediction bias and update correction parameters. The results are then optimized and the sequence is recombined by combining historical prediction accuracy to obtain the predicted sequence results.

4. The method for enhancing the quality of time series data based on a large prediction model according to claim 1, characterized in that, The deviation characteristics of the actual observed data are calculated using the sliding window method based on the predicted sequence results to obtain a time-series deviation feature vector, including: Based on the predicted sequence results and actual observation data, a difference matrix is ​​constructed according to the time series relationship. Then, time stamp alignment and data matching are performed through comparative analysis to obtain the sequence bias data. Based on the sequence deviation data, a sliding window segmentation is performed. At the same time, the window boundaries are determined and data is filled in by combining temporal continuity and data integrity, generating multiple sets of time window analysis sequences. The data features of the multiple time window analysis sequences are calculated through numerical range statistics and distribution characteristic analysis, and feature quantification is performed in combination with linear correlation evaluation to obtain a window feature set. The difference measurement and variance analysis are performed on the window feature set, and the importance of features is ranked and weights are assigned in combination with the data fluctuation pattern to form a weighted bias feature. By mapping the weighted bias features to the time dimension and reconstructing the sequence, and combining the data integrity check with feature combination and correlation analysis, a feature mapping sequence is obtained; Based on the feature mapping sequence, vectorization and dimension normalization are performed, and feature fusion and sequence merging are combined with data validity verification to obtain the time series deviation feature vector.

5. The method for enhancing the quality of time series data based on a large prediction model according to claim 1, characterized in that, The step of performing data analysis based on the time-series deviation feature vector through confidence assessment to generate data anomaly markers includes: Confidence intervals and probability distribution analyses are performed on the time-series deviation feature vectors. Simultaneously, deviation thresholds and boundary divisions are determined by combining historical data to obtain confidence evaluation criteria. The deviation values ​​are statistically significant and tested for volatility according to the confidence assessment criteria, and the degree of anomaly is quantified and scored by the data distribution characteristics to obtain the anomaly measurement results. The anomaly measurement results are fused through multi-dimensional feature combination and correlation analysis, and anomaly pattern recognition and classification are performed by combining temporal continuity to generate an anomaly feature matrix. Based on the aforementioned anomaly feature matrix, time-dimensional mapping and sequence reconstruction are performed, and feature combination and correlation verification are combined with data integrity checks to form an anomaly label candidate set; Based on the candidate set of anomaly labels, labels are screened by confidence level determination and anomaly degree assessment, and labels are optimized and adjusted by combining temporal continuity to obtain an anomaly label sequence. The abnormal marker sequence is subjected to data format conversion and dimension normalization, and simultaneously marker merging and sequence integration are performed in conjunction with validity verification to obtain the data abnormal marker.

6. The method for enhancing the quality of time series data based on a large prediction model according to claim 1, characterized in that, The step of locating missing values ​​in the time series based on the data anomaly markers to obtain a null value index set includes: Temporal location analysis and data integrity verification are performed on the data anomaly markers. At the same time, sequence alignment and missing point identification are performed in combination with timestamp information to obtain an initial set of missing locations. Based on the initial set of missing locations, missing intervals are divided through data continuity analysis and temporal correlation verification. The interval boundaries are determined and adjusted in conjunction with the data distribution characteristics to obtain a sequence of missing intervals. The missing interval sequence is mapped and reconstructed in the time dimension, and interval merging and segmentation are performed in combination with data validity checks to form a missing feature matrix. Based on the missing feature matrix, data is located through location index generation and coordinate mapping, and index verification and correction are performed in combination with temporal continuity to generate a location index sequence. The location index sequence is subjected to data format conversion and dimension normalization, and index optimization and adjustment are performed in conjunction with integrity checks to obtain an index tag set; The set of index tags is indexed and integrated through time-series correlation analysis and position mapping, and then combined with validity verification for sequence merging and normalization to obtain the set of null value indexes.

7. The method for enhancing the quality of time series data based on a large prediction model according to claim 1, characterized in that, The process of repairing the power grid source-load data based on the null value index set and the data completion model yields a complete time-series dataset, including: A hierarchical data completion model is constructed and assembled, which includes a data input layer, a missing value encoding layer, a feature reconstruction layer, a data generation layer, and a result verification layer. Based on the null value index set, the data input layer performs location positioning and data format conversion, and simultaneously combines the power grid source-load data for time-series alignment and range marking to obtain the input feature matrix; The input feature matrix is ​​passed through a missing value encoding layer to map missing positions and encode features, and encoding normalization and dimension adjustment are performed in combination with temporal continuity to obtain missing feature vectors; Based on the missing feature vector, temporal correlation analysis and feature reconstruction are performed in the feature reconstruction layer. At the same time, feature fusion and weight allocation are performed in combination with data distribution patterns to generate a reconstructed feature matrix. The reconstructed feature matrix is ​​filled with missing values ​​and generated through a data generation layer. The generated results are then optimized and corrected by combining them with historical data distribution to form a complete data sequence. Based on the completed data sequence, validity and rationality are verified at the result verification layer. At the same time, sequence integration and normalization are performed in conjunction with data integrity to obtain the complete time series dataset.

8. A time series data quality enhancement device based on a large predictive model, used to implement the time series data quality enhancement method based on a large predictive model as described in any one of claims 1-7, characterized in that, The time series data quality enhancement device based on the large prediction model includes: The acquisition module is used to preprocess the power grid source-load time series data through multi-scale feature analysis to obtain a feature sequence set; The modeling module is used to perform time-series modeling on the feature sequence set through a prediction model to obtain the prediction sequence result; The calculation module is used to calculate the deviation characteristics of the actual observed data based on the predicted sequence results using the sliding window method, and obtain the time-series deviation feature vector; The analysis module is used to perform data analysis based on the time-series deviation feature vector through confidence assessment and generate data anomaly markers; The positioning module is used to locate missing values ​​in the time series based on the data anomaly markers and obtain a null value index set. The processing module is used to repair the power grid source-load data based on the null value index set and the data completion model to obtain a complete time-series dataset.

9. An electronic device, characterized in that, include: The device includes a processor, a memory, and a bus. The memory stores machine-readable instructions executable by the processor. When the power sample data quality assessment device is running, the processor communicates with the memory via the bus. When the machine-readable instructions are executed by the processor, they perform the steps of the time series data quality enhancement method based on a large prediction model as described in any one of claims 1 to 7.

10. A computer-readable storage medium storing instructions thereon, characterized in that, When the instructions are executed by the processor, they implement the time series data quality enhancement method based on the large prediction model as described in any one of claims 1-7.

Images