Data cleaning method and system for digital twin data transmission of power transmission and transformation equipment

Abstract

The invention discloses a data cleaning method for digital twinborn data transmission of power transmission and transformation equipment. The method comprises the following steps: acquiring digital twinborn data and preprocessing the digital twinborn data; performing multi-dimensional feature analysis and dimension reduction according to the preprocessed digital twin data; further performing noise filtering and smoothing processing on the dimension-reduced digital twin data; carrying out abnormal value detection and automatic error correction on the digital twin data; and checking the continuity and logicality of the processed data and feeding back the cleaning data. The method has the advantages that efficient cleaning of the digital twin data of the power transmission and transformation equipment is achieved through the strict steps of data preprocessing, feature analysis, dimension reduction, noise filtering, abnormal value detection and data verification, the data processing efficiency and accuracy are improved through the steps, the stability and reliability of the data are enhanced, and the method is suitable for popularization and application. And finally, the purpose of improving the equipment state monitoring and fault prediction capabilities is achieved.

Description

Technical Field

[0001] The present invention relates to the field of state monitoring and data analysis of power transmission and transformation equipment in the electric power industry, and in particular to a data cleaning method and system for digital twin data transmission of power transmission and transformation equipment. Background Art

[0002] With the rapid development of smart grids, the demand for operation monitoring of power transmission and transformation equipment is growing, and the power industry has begun to widely adopt big data technology to analyze and process equipment status information. As a new path to achieve intelligent management of the entire life cycle of equipment, digital twin technology can accurately simulate, monitor, diagnose, predict and control power transmission and transformation equipment through a data closed-loop empowerment system. This technology integrates multi-source heterogeneous data, including equipment design data, manufacturing data, operation and maintenance perception data, etc., with physical mechanism simulation models and data-driven analysis models as the core, and monitors and intelligently controls the entity decision of power transmission and transformation equipment through cloud-edge collaborative computing. However, data cleaning, as a key step in preprocessing in digital twin technology, directly affects the accuracy of equipment monitoring and the reliability of system decision-making. Faced with the huge and multi-dimensional data volume collected by power transmission and transformation equipment, traditional data cleaning methods face challenges in processing such data. How to effectively clean and optimize equipment operation data and improve data quality and consistency has become a technical problem that needs to be solved urgently.

[0003] Existing data cleaning technologies for power transmission and transformation equipment mainly include sample data cleaning methods based on time series analysis and weighted iteration. The time series analysis method performs well in processing time series data with trends or periodicity, but it does not provide sufficient support for the correlation between multi-dimensional features, and it is difficult to accurately process the multi-dimensional feature information of the equipment status. In addition, this method may remove abnormal points when dealing with noisy data, resulting in data chain interruption, reducing the comprehensiveness and accuracy of fault prediction. Although the weighted iteration method can remove low-weight samples through weight update iteration and effectively reduce the erroneous information in the data set, it does not provide sufficient support for data with high real-time and time series requirements, especially in multi-dimensional equipment data. It is difficult to identify complex abnormal patterns, and it is easy to cause some useful data to be deleted by mistake, affecting the continuity and accuracy of subsequent analysis. Summary of the invention

[0004] In view of the problems existing in the prior art, the present invention is proposed.

[0005] Therefore, the problem to be solved by the present invention is to solve the problems of data cleaning efficiency and accuracy in the prior art, especially in processing multi-dimensional feature data and real-time data streams.

[0006] In order to solve the above technical problems, the present invention provides the following technical solutions:

[0007] In the first aspect, an embodiment of the present invention provides a data cleaning method for digital twin data transmission of power transmission and transformation equipment, which includes acquiring data and performing preprocessing; performing multi-dimensional feature analysis and dimensionality reduction based on the preprocessed digital twin data; further performing noise filtering and smoothing on the reduced dimensionality digital twin data; performing outlier detection and automatic error correction on the digital twin data; verifying the continuity and logic of the processed data and providing feedback on the cleaned data.

[0008] As a preferred solution of the data cleaning method for digital twin data transmission of power transmission and transformation equipment described in the present invention, wherein: data is acquired and preprocessed, including unification of data format, filling of missing values, and standardization of missing values;

[0009] The unification of data formats includes normalizing features with large value range differences. Through normalization, each feature is converted into a value in the interval [0, 1] to ensure that each feature is processed on a unified scale;

[0010] Missing value filling includes using linear interpolation to fill in data points for missing data caused by short-term sampling failures;

[0011] Standardization of missing values ​​further eliminates the influence of outliers and makes the data follow a standard normal distribution.

[0012] As a preferred solution of the data cleaning method for digital twin data transmission of power transmission and transformation equipment described in the present invention, wherein: multi-dimensional feature analysis and dimensionality reduction are performed based on pre-processed digital twin data, including

[0013] Multidimensional feature analysis uses the principal component analysis (PCA) method to reduce the high-dimensional data to a low-dimensional space and extract the key features that describe the operating status of the equipment. Suppose the original data matrix is Where m represents the number of samples and n represents the number of features.

[0014] As a preferred solution of the data cleaning method for digital twin data transmission of power transmission and transformation equipment described in the present invention, wherein: multi-dimensional feature analysis and dimensionality reduction are performed based on the pre-processed digital twin data, including covariance matrix construction, eigenvalue decomposition and principal component mapping;

[0015] The construction of the covariance matrix C quantifies the correlation between features. Suppose the original data matrix is Where m represents the number of samples, n represents the number of features, and the covariance matrix C is constructed as follows:

[0016]

[0017] Among them, X T is the transpose of the data matrix X, the covariance matrix Element C ij represents the covariance between the i-th and j-th features;

[0018] Eigenvalue decomposition is to perform eigenvalue decomposition on the covariance matrix C to obtain the eigenvalue λ i and the corresponding eigenvector v i , we sort the eigenvectors according to the size of the eigenvalues, select the eigenvectors corresponding to the largest k eigenvalues, and construct the principal component matrix V k :

[0019] V k =[v1,v2,...,v k

[0020] in, Represents the feature matrix after dimensionality reduction, k≤n, the more eigenvectors there are, the greater the variance of the data retained;

[0021] Through eigenvalue decomposition, the features that have the greatest impact on the change of equipment status are screened out;

[0022] The principal component mapping is done using the principal component matrix V k Project the original data into the dimensionality reduction space to obtain the dimensionality reduced data X′

[0023] X ′ =X·V k

[0024] in, Represents the data matrix after dimensionality reduction.

[0025] As a preferred solution of the data cleaning method for digital twin data transmission of power transmission and transformation equipment described in the present invention, the digital twin data with reduced dimension is further subjected to noise filtering and smoothing, including moving average smoothing, exponential weighted smoothing and low-pass filtering;

[0026] The moving average smoothing adopts the moving average method to smooth the continuous temperature and current related data;

[0027] The exponential weighted smoothing is to use exponential weighted smoothing for data with large fluctuations;

[0028] The low-pass filtering is to obtain the frequency distribution of features with high frequency fluctuations (such as vibration) through Fourier transform, and set a low-pass filter to filter out frequency components higher than a set threshold to eliminate environmental interference.

[0029] As a preferred solution of the data cleaning method for digital twin data transmission of power transmission and transformation equipment described in the present invention, wherein: outlier detection and automatic error correction are performed on the digital twin data, including a threshold detection method, an isolation forest algorithm and automatic error correction;

[0030] The threshold detection method sets the upper threshold U and lower threshold L for data with physical limitations in the characteristics (such as voltage and current):

[0031]

[0032] The Isolation Forest Algorithm is an unsupervised algorithm specifically used for outlier detection. It calculates the isolation score s(x) of each data point by building multiple decision trees:

[0033]

[0034] Among them, E(h(x)) represents the expected value of the path length of data point x, c(n) is the normalization coefficient, n is the sample size, and if s(x) exceeds the set threshold, the point is considered abnormal;

[0035] Automatic error correction refers to the use of interpolation or prediction methods to repair the marked abnormal points to ensure the continuity and accuracy of the data, and the use of linear interpolation to repair missing data points x t for

[0036]

[0037] As a preferred solution of the data cleaning method for digital twin data transmission of power transmission and transformation equipment described in the present invention, wherein: verifying the continuity and logic of the processed data and providing feedback of the cleaned data, including time series continuity verification, logical relationship verification and time series model verification;

[0038] The continuity check of time series is to determine whether there is a sudden change or anomaly by calculating the increments of adjacent data points;

[0039] By calculating the difference Δx between adjacent points i =x i -x i-1 To determine data continuity,

[0040] if|Δx i |>Δ max

[0041] It is marked as abnormal;

[0042] Where Δ max is the maximum allowed increment. If the increment exceeds this threshold, the data point is considered to not meet the continuity requirement.

[0043] Logical relationship verification is to verify the logical relationship between features;

[0044] Taking the linear relationship between current I and temperature T as an example, the regression equation is:

[0045] T=β0+β1I+∈

[0046] Where β0 and β1 are regression coefficients, and if a data point deviates from this relationship, it is marked as an anomaly;

[0047] If the verified data point does not meet the prediction range of the model, it is marked as an outlier;

[0048] Time series model verification is to check the trend consistency of time series data by using ARIMA model;

[0049] If the trend of the equipment data changes dramatically, the ARIMA model is used to perform trend analysis and find abnormal trend points. i , and its prediction model is:

[0050] x t =φ1x t-1 +...+φ p x t-p +∈ t

[0051] Among them, x t is the eigenvalue at the current moment, φ1,...,φ p is the autoregressive model parameter, ∈ t is the noise term;

[0052] If the actual value deviates significantly from the predicted value, it is considered abnormal;

[0053] After the data verification is passed, the cleaned data D′={d′1,d′2,...,d′ n}Feedback to the digital twin system to provide high-quality data support for equipment operation monitoring, status assessment and predictive analysis.

[0054] In a second aspect, an embodiment of the present invention provides a multi-resource inertia and primary frequency modulation auxiliary service sequential acquisition system, which includes a data acquisition and preprocessing module, a multi-dimensional feature analysis and dimension reduction module, a noise filtering and smoothing processing module, an outlier detection and automatic error correction module, and a data verification and feedback module;

[0055] The data preprocessing and standardization module is used to preliminarily process the raw data collected by power transmission and transformation equipment to ensure the consistency and analyzability of the data;

[0056] The multi-dimensional feature analysis and dimensionality reduction module is used to extract key features from high-dimensional data, simplify data structure, and improve data processing efficiency;

[0057] The noise filtering and outlier detection module is used to eliminate high-frequency noise and random fluctuations in the data to ensure the stability and continuity of the data;

[0058] The outlier detection and automatic error correction module is used to identify and process abnormal fluctuations or fault points in the data to ensure the accuracy of the data;

[0059] The data verification and feedback module is used to verify the continuity and logical relationship of the processed data to ensure that the cleaned data conforms to the operating rules of the power transmission and transformation equipment.

[0060] In a third aspect, an embodiment of the present invention provides a computer device comprising a memory and a processor, wherein the memory stores a computer program, wherein: when the computer program instructions are executed by the processor, the steps of the data cleaning method for digital twin data transmission of power transmission and transformation equipment as described in the first aspect of the present invention are implemented.

[0061] In a fourth aspect, an embodiment of the present invention provides a computer-readable storage medium having a computer program stored thereon, wherein: when the computer program instructions are executed by a processor, the steps of the data cleaning method for digital twin data transmission of power transmission and transformation equipment as described in the first aspect of the present invention are implemented.

[0062] The beneficial effects of the present invention are as follows: through the unification of data formats, filling of missing values ​​and standardization of missing values, data normalization and integrity are achieved, laying a solid foundation for subsequent analysis and processing; the principal component analysis (PCA) method is used to reduce the dimension of high-dimensional data to a low-dimensional space, and key features describing the operating status of the equipment are extracted, which not only reduces the complexity of the data, but also retains the most important variance information in the data, thereby improving the efficiency and accuracy of data processing; the correlation between features is quantified by the covariance matrix, the eigenvalue decomposition obtains the eigenvalue and the corresponding eigenvector, and the principal component mapping projects the original data to the reduced-dimensional space, and the principal component that best represents the data features is screened out, providing a basis for subsequent anomaly detection and prediction. More accurate data foundation; smoothing the data by moving average smoothing, exponential weighted smoothing, and low-pass filtering to eliminate noise and environmental interference, effectively reducing random fluctuations and noise in the data, improving the stability and reliability of the data, and providing a clearer data signal for subsequent anomaly detection; detecting and repairing outliers in the data by threshold detection, isolation forest algorithm, and automatic error correction method, not only improves the accuracy of the data, but also ensures the continuity and integrity of the data; ensuring the continuity and logic of the data by time series continuity verification, logical relationship verification, and time series model verification, thereby improving the accuracy of equipment operation monitoring, status evaluation, and predictive analysis. The present invention achieves efficient cleaning of digital twin data of power transmission and transformation equipment through a series of rigorous data preprocessing, feature analysis, dimensionality reduction, noise filtering, outlier detection, and data verification steps. These steps not only improve the efficiency and accuracy of data processing, but also enhance the stability and reliability of the data, and ultimately achieve the purpose of improving equipment status monitoring and fault prediction capabilities. BRIEF DESCRIPTION OF THE DRAWINGS

[0063] In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the following briefly introduces the drawings required for use in the description of the embodiments. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.

[0064] Figure 1 A flow chart of a data cleaning method for digital twin data transmission of power transmission and transformation equipment;

[0065] Figure 2 Computer equipment diagram for data cleaning method for digital twin data transmission of power transmission and transformation equipment;

[0066] Figure 3 Schematic diagram of smoothing processing of data cleaning method for digital twin data transmission of power transmission and transformation equipment.

[0067] Figure 4 Schematic diagram of the isolation forest algorithm for the data cleaning method for digital twin data transmission of power transmission and transformation equipment. DETAILED DESCRIPTION

[0068] In order to make the above-mentioned objects, features and advantages of the present invention more obvious and easy to understand, the specific implementation methods of the present invention are described in detail below in conjunction with the accompanying drawings.

[0069] In the following description, many specific details are set forth to facilitate a full understanding of the present invention, but the present invention may also be implemented in other ways different from those described herein, and those skilled in the art may make similar generalizations without violating the connotation of the present invention. Therefore, the present invention is not limited to the specific embodiments disclosed below.

[0070] Secondly, the term "one embodiment" or "embodiment" as used herein refers to a specific feature, structure, or characteristic that may be included in at least one implementation of the present invention. The term "in one embodiment" that appears in different places in this specification does not necessarily refer to the same embodiment, nor does it refer to a separate or selective embodiment that is mutually exclusive with other embodiments.

[0071] Example 1

[0072] Reference Figures 1 to 4 , which is the first embodiment of the present invention, and provides a data cleaning method for digital twin data transmission of power transmission and transformation equipment, comprising:

[0073] S1: Obtain data and perform preprocessing.

[0074] Furthermore, the data is obtained and preprocessed, including the unification of data formats, filling of missing values, and standardization of missing values;

[0075] Furthermore, the unification of data formats includes normalizing features with large differences in value ranges. Through normalization, each feature is converted into a value in the interval [0, 1] to ensure that each feature is processed on a unified scale;

[0076] Specifically, the normalization formula is as follows:

[0077]

[0078] Where x represents the original value of the feature, X represents the feature data set, x′ is the normalized value, min(X) and max(X) are the minimum and maximum values, respectively;

[0079] Missing value filling includes using linear interpolation to fill in data points for missing data caused by short-term sampling failures;

[0080] Specifically, let the missing value position be xmiss , the adjacent data point is x t-1 and x t+1 , then the interpolation formula is:

[0081]

[0082] Standardization of missing values ​​further eliminates the impact of outliers and makes the data follow a standard normal distribution;

[0083] Specifically, the standardized formula is:

[0084]

[0085] Among them, z is the standardized value, max(X) is the mean value of feature X, and σ(X) is its standard deviation;

[0086] The standardization operation makes the mean of each feature equal to 0 and the variance equal to 1.

[0087] It should be noted that the data preprocessing step is the basis of data processing, ensuring the consistency and integrity of the data and providing stable input data for subsequent analysis.

[0088] It should also be noted that the normalization process allows each feature to be processed on a unified scale to reduce the magnitude differences between different features and improve the robustness of the algorithm; missing value filling uses linear interpolation to fill in the data missing caused by short-term sampling failures, ensuring the continuity of the data and avoiding analysis bias caused by missing data; standardization operations eliminate the influence of outliers through formulas, making the data obey the standard normal distribution and improving the stability of the model analysis.

[0089] S2: Perform multi-dimensional feature analysis and dimensionality reduction based on the pre-processed digital twin data.

[0090] Furthermore, multi-dimensional feature analysis and dimensionality reduction are performed on the pre-processed digital twin data, including multi-dimensional feature analysis that uses the principal component analysis (PCA) method to reduce the high-dimensional data to a low-dimensional space and extract key features that describe the operating status of the equipment.

[0091] Furthermore, multi-dimensional feature analysis and dimensionality reduction are performed based on the preprocessed digital twin data, including covariance matrix construction, eigenvalue decomposition, and principal component mapping;

[0092] The construction of the covariance matrix C quantifies the correlation between features. Let the original data matrix be Where m represents the number of samples, n represents the number of features, and the covariance matrix C is constructed as follows:

[0093]

[0094] Among them, X T is the transpose of the data matrix X, the covariance matrix Element C ij represents the covariance between the i-th and j-th features;

[0095] Eigenvalue decomposition is to perform eigenvalue decomposition on the covariance matrix C to obtain the eigenvalue λ i and the corresponding eigenvector v i , we sort the eigenvectors according to the size of the eigenvalues, select the eigenvectors corresponding to the largest k eigenvalues, and construct the principal component matrix V k :

[0096] V k =[v1,v2,...,v k ]

[0097] in, Represents the feature matrix after dimensionality reduction, k≤n, the more eigenvectors there are, the greater the variance of the data retained;

[0098] Through eigenvalue decomposition, the features that have the greatest impact on the change of equipment status are screened out;

[0099] The principal component mapping is done using the principal component matrix V k Project the original data into the dimensionality reduction space to obtain the dimensionality reduced data X′

[0100] X′=X·V k

[0101] in, Represents the data matrix after dimensionality reduction.

[0102] It should be noted that multi-dimensional feature analysis is used to identify the main change patterns in equipment data, simplify the subsequent data cleaning process, construct a covariance matrix, quantify the correlation between different features, and provide a basis for subsequent eigenvalue decomposition; eigenvalue decomposition, by selecting eigenvectors corresponding to larger eigenvalues, construct a principal component matrix, achieve data dimensionality reduction, and retain the main information; principal component mapping, projects the original data into a new low-dimensional space to obtain a data matrix after dimensionality reduction.

[0103] It should also be noted that the operating data of power transmission and transformation equipment is usually high-dimensional and contains redundant information. The PCA method is used for dimensionality reduction. Different from the traditional single feature analysis, the PCA method retains the most important variance information in the data through dimensionality reduction. By constructing the covariance matrix and decomposing the eigenvalues, the correlation between features is quantified, and the features that have the greatest impact on the equipment status change are screened out. This solves the computational burden and information redundancy problems brought by high-dimensional data, effectively reduces the data dimension, and reduces the computational complexity. It not only provides a more accurate data basis for equipment status monitoring and fault prediction, but also improves the efficiency and accuracy of subsequent data processing.

[0104] S3: Further perform noise filtering and smoothing on the reduced-dimensional digital twin data.

[0105] Furthermore, the dimension-reduced digital twin data is further subjected to noise filtering and smoothing, including moving average smoothing, exponentially weighted smoothing, and low-pass filtering;

[0106] Moving average smoothing uses the moving average method to smooth continuous temperature and current related data;

[0107] Specifically, the smoothing formula is:

[0108]

[0109] Among them, y i represents the smoothed data value, x i is the original value of the ith data point, and M is the smoothing window size.

[0110] Exponential weighted smoothing is used for data with large fluctuations;

[0111] Specifically, the update formula is:

[0112] y i =αx i +(1-α)y i-1

[0113] Among them, α∈(0,1) is the smoothing coefficient, y i-1 is the smoothed value at the previous moment.

[0114] Low-pass filtering is used to obtain the frequency distribution of features with high frequency fluctuations (such as vibration) through Fourier transform, and a low-pass filter is set to filter out frequency components higher than the set threshold to eliminate environmental interference.

[0115] It should be noted that the moving average reduces short-term fluctuation noise by calculating the mean of adjacent data, making the data curve smoother; exponential weighted smoothing can adjust the emphasis on the latest data according to the smoothing coefficient, and is suitable for processing rapidly fluctuating data characteristics; the frequency distribution is obtained through Fourier transform, and a low-pass filter is set to filter out frequency components above the set threshold to eliminate environmental interference.

[0116] It should also be noted that the operating environment of power transmission and transformation equipment is complex, and the collected data inevitably contains noise. Different smoothing methods are used for different types of data. The moving average method is suitable for smoothing continuous temperature and current data; exponential weighted smoothing is suitable for data with large fluctuations; low-pass filtering is suitable for high-frequency data such as vibration. This differentiated processing method can effectively remove high-frequency interference and random fluctuations, retain the real signal, and not only ensure the stability and continuity of the data, but also improve the data quality and reliability.

[0117] S4: Perform outlier detection and automatic error correction on digital twin data.

[0118] Furthermore, outlier detection and automatic error correction are performed on digital twin data, including threshold detection method, isolation forest algorithm and automatic error correction;

[0119] The threshold detection method sets an upper threshold U and a lower threshold L for data with physical limitations in the features (such as voltage and current):

[0120]

[0121] The isolation forest algorithm is an unsupervised algorithm specifically used for outlier detection. It calculates the isolation score s(x) of each data point by constructing multiple decision trees:

[0122]

[0123] Among them, E(h(x)) represents the expected value of the path length of data point x, c(n) is the normalization coefficient, n is the sample size, and if s(x) exceeds the set threshold, the point is considered abnormal;

[0124] The automatic error correction refers to the use of interpolation or prediction methods to repair the marked abnormal points to ensure the continuity and accuracy of the data, and the use of linear interpolation to repair the missing data points x t for

[0125]

[0126] It should be noted that the threshold detection method quickly determines whether the data is abnormal through preset upper and lower thresholds; the isolation forest algorithm determines the anomaly by calculating the isolation score of the data point; the linear interpolation method is used to repair missing or abnormal data points to ensure data continuity.

[0127] It should also be noted that outlier detection combines the threshold detection method and the isolation forest algorithm. The threshold detection method uses the physical limitations of the parameters of the power transmission and transformation equipment. It is efficient and reliable in removing obvious outliers and is an effective means of dealing with large-scale outliers. The isolation forest algorithm is an unsupervised algorithm that can effectively identify outliers hidden in high-dimensional data, making up for the shortcomings of the threshold detection method. The automatic error correction mechanism repairs outliers through interpolation or prediction methods to ensure the continuity and accuracy of the data and provide higher quality input for data monitoring and subsequent predictions. The overall detection process solves the impact of data anomalies on system reliability and stability and improves the accuracy and integrity of data.

[0128] S5: Verify the continuity and logic of the processed data and provide cleansed data feedback.

[0129] Furthermore, the continuity and logic of the processed data are verified and the cleansed data feedback is performed, including time series continuity verification, logical relationship verification, and time series model verification;

[0130] The time series continuity check is to determine whether there is a mutation or anomaly by calculating the increments of adjacent data points;

[0131] By calculating the difference Δx between adjacent points i =x i -x i-1 To determine data continuity,

[0132] if|Δx i |>Δ max

[0133] It is marked as abnormal;

[0134] where Δ max is the maximum allowed increment. If the increment exceeds this threshold, the data point is considered to not meet the continuity requirement.

[0135] The logic relationship check is to check the logic relationship between features;

[0136] Taking the linear relationship between current I and temperature T as an example, the regression equation is:

[0137] T=β0+β1I+∈

[0138] Where β0 and β1 are regression coefficients, and if a data point deviates from this relationship, it is marked as an anomaly;

[0139] If the verified data point does not meet the prediction range of the model, it is marked as an outlier;

[0140] The time series model verification is to use the ARIMA model to test the trend consistency of the time series data;

[0141] If the trend of the equipment data changes dramatically, the ARIMA model is used to perform trend analysis and find abnormal trend points. i , and its prediction model is:

[0142] x t =φ1x t-1 +...+φ p x t-p +∈ t

[0143] Among them, x t is the eigenvalue at the current moment, φ1,...,φ p is the autoregressive model parameter, ∈ t is the noise term;

[0144] If the actual value deviates significantly from the predicted value, it is considered abnormal;

[0145] After the data verification is passed, the cleaned data D′={d′1,d′2,...,d′ n}Feedback to the digital twin system to provide high-quality data support for equipment operation monitoring, status assessment and predictive analysis.

[0146] It should be noted that the time series continuity check determines the data continuity by calculating the increments of adjacent data points; the logical relationship check is carried out by taking the linear relationship between current and temperature as an example; the time series model check uses the ARIMA model to perform trend analysis and discover abnormal trends.

[0147] It should also be noted that during the overall inspection process, the time series continuity check can identify data mutations and is an important prerequisite for maintaining the real-time performance of the system; the logical relationship check uses the physical laws between features to determine whether the data is logically consistent, and can effectively identify abnormal logical fluctuations in the data, which helps to improve the accuracy of equipment status assessment; the time series model check analyzes data trends through the ARIMA model to discover abnormal trend points, which helps to identify trend deviations and provide high-quality data support for equipment operation monitoring, status assessment and predictive analysis; this multi-angle verification mechanism can effectively identify various types of data anomalies, solve the problem of data inconsistency, ensure the integrity and accuracy of data in time and logic, and provide reliable data support for the digital twin system.

[0148] It should also be noted that by verifying the processed data, it is ensured that the cleaned data conforms to the operating rules of the power transmission and transformation equipment, so that it has continuous time logic, providing high-quality data guarantee for subsequent storage and feedback. After the verification is completed, the cleaned data will be stored and fed back to the digital twin system to support real-time monitoring, fault prediction and status evaluation of the equipment.

[0149] Furthermore, the present embodiment also provides a multi-resource inertia and primary frequency modulation auxiliary service sequence acquisition system, including a data acquisition and preprocessing module, a multi-dimensional feature analysis and dimension reduction module, a noise filtering and smoothing processing module, an outlier detection and automatic error correction module, and a data verification and feedback module;

[0150] The data preprocessing and standardization module is used to preliminarily process the raw data collected by power transmission and transformation equipment to ensure the consistency and analyzability of the data;

[0151] The multi-dimensional feature analysis and dimensionality reduction module is used to extract key features from high-dimensional data, simplify data structure, and improve data processing efficiency;

[0152] The noise filtering and outlier detection module is used to eliminate high-frequency noise and random fluctuations in the data to ensure the stability and continuity of the data;

[0153] The outlier detection and automatic error correction module is used to identify and process abnormal fluctuations or fault points in the data to ensure the accuracy of the data;

[0154] The data verification and feedback module is used to verify the continuity and logical relationship of the processed data to ensure that the cleaned data conforms to the operating rules of the power transmission and transformation equipment.

[0155] Specifically, the data acquisition and preprocessing module is used to acquire the digital twin data of power transmission and transformation equipment, and to unify the data format, fill in missing values ​​and perform standardization processing; among them, the data format unification uses the normalization formula to convert the features with large differences in value ranges into numerical values ​​within a specific interval to reduce the magnitude difference; the missing value filling uses the linear interpolation method to deal with the data missing caused by short-time sampling failure; the standardization processing uses the standardization formula to further eliminate the influence of outliers, so that the data obeys the standard normal distribution, ensuring that each feature is subsequently analyzed on a unified scale, and providing stable input for data cleaning.

[0156] Specifically, the multi-dimensional feature analysis and dimensionality reduction module uses the principal component analysis (PCA) method to construct a covariance matrix to quantify feature correlations, selects key eigenvectors through eigenvalue decomposition to construct a principal component matrix, projects the original high-dimensional data into a low-dimensional space, removes redundant information, condenses key data information, enhances data processability, and provides a simplified and effective data structure for subsequent steps such as noise filtering and anomaly detection.

[0157] Specifically, the noise filtering and smoothing processing module adopts moving average smoothing, exponential weighted smoothing and low-pass filtering methods for the data after dimensionality reduction; moving average smoothing reduces short-term fluctuation noise by calculating the mean of adjacent data; exponential weighted smoothing adjusts the emphasis on the latest data according to the smoothing coefficient to process data with large fluctuations; low-pass filtering eliminates high-frequency interference with the help of Fourier transform and filter, effectively removes high-frequency noise in the data, improves data stability, and provides a more accurate data basis for anomaly detection;

[0158] Specifically, the outlier detection and automatic error correction module combines the threshold detection method and the isolation forest algorithm to perform outlier detection. The threshold detection method sets upper and lower thresholds for physical restriction features to eliminate obvious outliers; the isolation forest algorithm constructs multiple decision trees to calculate the isolation scores of data points to determine anomalies. For marked outliers, interpolation or prediction methods are used to automatically correct errors to ensure data continuity and accuracy, providing reliable data for data monitoring and prediction.

[0159] Specifically, the data verification and feedback module performs time series continuity verification, logical relationship verification and time series model verification on the cleaned data. By calculating the increments of adjacent data points, comparing the logical relationships between features and ARIMA model trend analysis, the data continuity, logic and trend consistency are judged, and abnormal data points are identified and processed. After the verification is passed, the cleaned data is fed back to the digital twin system to provide high-quality data support for equipment operation monitoring, status assessment and predictive analysis, and to ensure the efficient operation of the digital twin system.

[0160] This embodiment also provides a computer device, which is suitable for the data cleaning method for digital twin data transmission of power transmission and transformation equipment, including a memory and a processor; the memory is used to store computer executable instructions, and the processor is used to execute computer executable instructions to implement the data cleaning method for digital twin data transmission of power transmission and transformation equipment proposed in the above embodiment.

[0161] The computer device may be a terminal, and the computer device includes a processor, a memory, a communication interface, a display screen and an input device connected via a system bus. The processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system and a computer program. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The communication interface of the computer device is used to communicate with an external terminal in a wired or wireless manner, and the wireless manner can be achieved through WIFI, an operator network, NFC (near field communication) or other technologies. The display screen of the computer device may be a liquid crystal display screen or an electronic ink display screen, and the input device of the computer device may be a touch layer covering the display screen, or a key, trackball or touchpad provided on the housing of the computer device, or an external keyboard, touchpad or mouse, etc.

[0162] This embodiment also provides a storage medium on which a computer program is stored. When the program is executed by a processor, the data cleaning method for digital twin data transmission of power transmission and transformation equipment proposed in the above embodiment is implemented.

[0163] In summary, the present invention provides a data cleaning method for digital twin data transmission of power transmission and transformation equipment. By integrating multi-dimensional feature analysis, noise filtering, outlier detection and logic verification and other technologies, the data is effectively detected and processed according to the characteristics of power transmission and transformation equipment data, and the integrity and logic of the data are ensured before entering the digital twin system. The overall method not only significantly improves the real-time and decision-making reliability of equipment status monitoring, but also reduces the impact of factors such as sensor failure and transmission delay on the data by verifying the continuity and logic of time series data. By integrating low-pass filtering, smoothing processing and machine learning methods, a set of automated cleaning processes is formed, so that the digital twin model can obtain more accurate equipment operation information, and provide reliable data support for equipment fault prediction, status assessment and operation optimization, thereby improving the overall performance of the digital twin system.

[0164] Example 2

[0165] The second embodiment of the present invention provides a data cleaning method for digital twin data transmission of power transmission and transformation equipment. In order to verify the effectiveness of the data cleaning method of the present invention in improving data quality, accuracy and system real-time performance, this experiment takes the operating data of power transmission and transformation equipment as the experimental object, constructs a multi-dimensional, time series feature matrix, and simulates various noises and abnormal conditions.

[0166] The experimental data covers the key characteristics of the equipment (such as voltage, current, temperature, vibration, etc.). Through multi-dimensional feature analysis and machine learning algorithms, noise, missing values, outliers and fault simulation data are detected and cleaned.

[0167] The experiment compares the cleaning effect and performance of the method of the present invention with the existing time series cleaning method and weighted iteration cleaning method. The main indicators include the accuracy of the cleaned data, the data recovery rate, the anomaly detection rate, the cleaning efficiency, and the misdiagnosis rate in the equipment status monitoring. For the same group of digital twin data of power transmission and transformation equipment, the digital twin data cleaning method based on multi-dimensional feature analysis (the method of the present invention), time series analysis, and weighted iteration are used for data cleaning, and various indicators are obtained according to the samples. Table 1 is a comparison of the implementation effects of different methods;

[0168] Table 1

[0169]

[0170]

[0171] Experimental results show that the method of the present invention is significantly superior to existing methods in terms of data accuracy, data recovery rate, anomaly detection rate and misdiagnosis rate for digital twin data of power transmission and transformation equipment, especially in processing real-time monitoring data of equipment, showing higher efficiency and accuracy. This makes the cleaning method have significant advantages in improving the monitoring accuracy of power transmission and transformation equipment, reducing the risk of misdiagnosis, and optimizing equipment failure prediction.

[0172] It should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention rather than to limit it. Although the present invention has been described in detail with reference to the preferred embodiments, those skilled in the art should understand that the technical solutions of the present invention may be modified or replaced by equivalents without departing from the spirit and scope of the technical solutions of the present invention, which should all be included in the scope of the claims of the present invention.

Claims

1. A data cleaning method for digital twin data transmission of power transmission and transformation equipment, characterized by: include, Acquire digital twin data and preprocess it; Perform multi-dimensional feature analysis and dimensionality reduction based on pre-processed digital twin data; Further noise filtering and smoothing are performed on the dimension-reduced digital twin data; Perform outlier detection and automatic error correction on digital twin data; Verify the continuity and logic of processing data and provide cleansing data feedback.

2. The data cleaning method for digital twin data transmission of power transmission and transformation equipment according to claim 1, characterized in that: The data is obtained and preprocessed, including unification of data formats, filling of missing values, and standardization of missing values; The unification of the data format includes normalizing the features with large differences in value ranges. Through the normalization process, each feature is converted into a value in the interval [0, 1], ensuring that each feature is processed on a unified scale; The filling of missing values ​​includes using linear interpolation to fill in data points for data missing caused by short-term sampling failure; The standardization of missing values ​​further eliminates the influence of outliers and makes the data follow a standard normal distribution.

3. The data cleaning method for digital twin data transmission of power transmission and transformation equipment according to claim 2, characterized in that: The multi-dimensional feature analysis and dimensionality reduction are performed based on the pre-processed digital twin data, including the multi-dimensional feature analysis using a principal component analysis (PCA) method, which reduces the high-dimensional data to a low-dimensional space through the principal component analysis (PCA) method to extract key features that describe the operating status of the equipment.

4. The data cleaning method for digital twin data transmission of power transmission and transformation equipment according to claim 3 is characterized in that: The multi-dimensional feature analysis and dimensionality reduction are performed based on the pre-processed digital twin data, including covariance matrix construction, eigenvalue decomposition, and principal component mapping; The construction of the covariance matrix C quantifies the correlation between features. Suppose the original data matrix is Where m represents the number of samples, n represents the number of features, and the covariance matrix C is constructed as follows: Among them, X T is the transpose of the data matrix X, the covariance matrix Element C ij represents the covariance between the i-th and j-th features; The eigenvalue decomposition is to perform eigenvalue decomposition on the covariance matrix C to obtain the eigenvalue λ i and the corresponding eigenvector v i , we sort the eigenvectors according to the size of the eigenvalues, select the eigenvectors corresponding to the largest k eigenvalues, and construct the principal component matrix V k : V k =[v1,v2,...,v k ] in, Represents the feature matrix after dimensionality reduction, k≤n, the more eigenvectors there are, the greater the variance of the data retained; Through eigenvalue decomposition, the features that have the greatest impact on the change of equipment status are screened out; The principal component mapping is done using the principal component matrix V k Project the original data into the dimensionality reduction space to obtain the dimensionality reduced data X′ X′=X·V k in, Represents the data matrix after dimensionality reduction.

5. The data cleaning method for digital twin data transmission of power transmission and transformation equipment according to claim 4, characterized in that: The further noise filtering and smoothing of the dimension-reduced digital twin data includes moving average smoothing, exponential weighted smoothing and low-pass filtering; The moving average smoothing adopts the moving average method to smooth the continuous temperature and current related data; The exponential weighted smoothing is to use exponential weighted smoothing for data with large fluctuations; The low-pass filtering is to obtain the frequency distribution of features with high frequency fluctuations (such as vibration) through Fourier transform, and set a low-pass filter to filter out frequency components higher than a set threshold to eliminate environmental interference.

6. The data cleaning method for digital twin data transmission of power transmission and transformation equipment according to claim 5, characterized in that: The outlier detection and automatic error correction of the digital twin data include a threshold detection method, an isolation forest algorithm, and automatic error correction; The threshold detection method sets an upper threshold U and a lower threshold L for data with physical limitations in the features (such as voltage and current): The isolation forest algorithm is an unsupervised algorithm specifically used for outlier detection. It calculates the isolation score s(x) of each data point by constructing multiple decision trees: Among them, E(h(x)) represents the expected value of the path length of data point x, c(n) is the normalization coefficient, n is the sample size, and if s(x) exceeds the set threshold, the point is considered abnormal; The automatic error correction refers to the use of interpolation or prediction methods to repair the marked abnormal points to ensure the continuity and accuracy of the data, and the use of linear interpolation to repair the missing data points x t for 7. The data cleaning method for digital twin data transmission of power transmission and transformation equipment according to claim 6, characterized in that: The verification process processes data continuity and logic and performs cleansing data feedback, including time series continuity verification, logical relationship verification, and time series model verification; The time series continuity check is to determine whether there is a mutation or anomaly by calculating the increments of adjacent data points; By calculating the difference Δx between adjacent points i =x i -x i-1 To determine data continuity, if|Δx i |>D max It is marked as abnormal; Where Δ max is the maximum allowed increment. If the increment exceeds this threshold, the data point is considered to not meet the continuity requirement. The logic relationship check is to check the logic relationship between features; Taking the linear relationship between current I and temperature T as an example, the regression equation is: T=β0+β1I+∈ Where β0 and β1 are regression coefficients, and if a data point deviates from this relationship, it is marked as an anomaly; If the verified data point does not meet the prediction range of the model, it is marked as an outlier; The time series model verification is to use the ARIMA model to test the trend consistency of the time series data; If the trend of the equipment data changes dramatically, the ARIMA model is used to perform trend analysis and find abnormal trend points. i , and its prediction model is: x t =φ1x t-1 +...+φ p x t-p +∈ t Among them, x t is the eigenvalue at the current moment, φ1,...,φ p is the autoregressive model parameter, ∈ t is the noise term; If the actual value deviates significantly from the predicted value, it is considered abnormal; After the data verification is passed, the cleaned data D′={d′1,d′2,...,d′ n }Feedback to the digital twin system to provide high-quality data support for equipment operation monitoring, status assessment and predictive analysis.

8. A data cleaning system for digital twin data transmission of power transmission and transformation equipment, based on the data cleaning method for digital twin data transmission of power transmission and transformation equipment according to any one of claims 1 to 7, characterized in that: It also includes data acquisition and preprocessing module, multi-dimensional feature analysis and dimension reduction module, noise filtering and smoothing processing module, outlier detection and automatic error correction module, data verification and feedback module; The data preprocessing and standardization module is used to perform preliminary processing on the raw data collected by the power transmission and transformation equipment to ensure the consistency and analyzability of the data; The multi-dimensional feature analysis and dimensionality reduction module is used to extract key features from high-dimensional data, simplify data structure, and improve data processing efficiency; The noise filtering and outlier detection module is used to eliminate high-frequency noise and random fluctuations in the data to ensure the stability and continuity of the data; The outlier detection and automatic error correction module is used to identify and process abnormal fluctuations or fault points in the data to ensure the accuracy of the data; The data verification and feedback module is used to verify the continuity and logical relationship of the processed data to ensure that the cleaned data complies with the operating rules of the power transmission and transformation equipment.

9. A computer device comprising a memory and a processor, wherein the memory stores a computer program, wherein: When the processor executes the computer program, the steps of the data cleaning method for digital twin data transmission of power transmission and transformation equipment described in any one of claims 1 to 7 are implemented.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the data cleaning method for digital twin data transmission of power transmission and transformation equipment described in any one of claims 1 to 7 are implemented.

Images