Data cleaning method and system for power transmission and transformation equipment digital twin data transmission

Abstract

The application discloses a kind of data cleaning methods for power transmission and transformation equipment digital twin data transmission, including obtaining digital twin data and pre-processing;According to the multi-dimensional feature analysis and dimension reduction of pre-processing digital twin data;Further, the noise filtering and smoothing processing of dimension reduction digital twin data;Digital twin data is detected and automatically corrected with abnormal value;Check data continuity and logicality and do cleaning data feedback.The application has beneficial effects for through rigorous data preprocessing, feature analysis, dimension reduction, noise filtering, abnormal value detection and data checking steps, efficient cleaning of power transmission and transformation equipment digital twin data is realized, these steps not only improve the efficiency and accuracy of data processing, but also enhance the stability and reliability of data, ultimately achieve the purpose of improving equipment condition monitoring and fault prediction capability.

Description

TECHNICAL FIELD

[0001] The present application relates to the field of power industry transmission and transformation equipment state monitoring and data analysis, and particularly relates to a data cleaning method and system for digital twin data transmission of transmission and transformation equipment. BACKGROUND

[0002] With the rapid development of smart grids, the operation monitoring demand of transmission and transformation equipment is increasing, and the power industry has begun to widely adopt big data technology to analyze and process equipment state information. Digital twin technology, as a new path to realize intelligent management of equipment throughout its life cycle, realizes accurate simulation, monitoring, diagnosis, prediction and control of transmission and transformation equipment through a data closed-loop empowerment system. This technology integrates multi-source heterogeneous data, including equipment design data, manufacturing data, operation and maintenance sensing data, etc., with physical mechanism simulation models and data-driven analysis models as the core, and through cloud-edge collaborative computing, monitors and intelligently controls transmission and transformation equipment entity decision-making. However, data cleaning, as a key step in the preprocessing of digital twin technology, directly affects the accuracy of equipment monitoring and the reliability of system decision-making. In the face of huge and multi-dimensional data collected by transmission and transformation equipment, traditional data cleaning methods face challenges in processing such data. How to effectively clean and optimize equipment operation data and improve the quality and consistency of data has become a technical problem that needs to be solved.

[0003] The existing data cleaning technology for transmission and transformation equipment mainly includes a time series analysis-based method and a sample data cleaning method based on weight iteration. The time series analysis method performs well in processing time series data with trends or periodicity, but it lacks support for the correlation between multi-dimensional features and is difficult to accurately process multi-dimensional feature information of equipment state. In addition, when dealing with noisy data, this method may exclude abnormal points, causing data chain interruption and reducing the comprehensiveness and accuracy of fault prediction. The weight iteration method can effectively reduce the error information in the data set by updating and iterating the weights to exclude low-weight samples, but it lacks support for data with high real-time and time sequence requirements, especially in multi-dimensional equipment data, it is difficult to identify complex abnormal patterns, and it is easy to cause some useful data to be mistakenly deleted, affecting the continuity and accuracy of subsequent analysis. SUMMARY

[0004] In view of the problems existing in the prior art, the present application is proposed.

[0005] Therefore, the problem to be solved by the present application is to solve the data cleaning efficiency and accuracy problem in the prior art, especially in processing multi-dimensional feature data and real-time data streams.

[0006] To solve the above technical problems, the present application provides the following technical solutions:

[0007] In a first aspect, the embodiments of the present application provide a data cleaning method for power transmission and transformation equipment digital twin data transmission, which comprises: obtaining data and preprocessing; performing multi-dimensional feature analysis and dimension reduction on the preprocessed digital twin data; further filtering and smoothing the noise of the dimension-reduced digital twin data; detecting outliers and automatically correcting the digital twin data; checking the continuity and logicality of the processed data and feeding back the cleaned data.

[0008] As a preferred scheme of the data cleaning method for power transmission and transformation equipment digital twin data transmission, the obtaining of the digital twin data and the preprocessing thereof comprises: unifying the data format, filling in the missing values, and standardizing the missing values.

[0009] The unification of the data format comprises normalizing the features with large value range differences, so that each feature is converted into a value in the interval [0, 1], ensuring that each feature is processed on a unified scale.

[0010] The filling in of the missing values comprises using linear interpolation to fill in the data points caused by short-time sampling failure.

[0011] The standardization of the missing values further eliminates the influence of outliers and makes the data conform to the standard normal distribution.

[0012] As a preferred scheme of the data cleaning method for power transmission and transformation equipment digital twin data transmission, the multi-dimensional feature analysis and dimension reduction based on the preprocessed digital twin data comprises:

[0013] The multi-dimensional feature analysis comprises using principal component analysis (PCA) to reduce the high-dimensional data to a low-dimensional space and extract key features describing the running state of the equipment; the original data matrix is denoted as X where m represents the number of samples, and n represents the number of features.

[0014] As a preferred scheme of the data cleaning method for power transmission and transformation equipment digital twin data transmission, the multi-dimensional feature analysis and dimension reduction based on the preprocessed digital twin data comprises covariance matrix construction, eigenvalue decomposition, and principal component mapping.

[0015] The construction of the covariance matrix C quantifies the correlation between features, and the original data matrix is denoted as X where m represents the number of samples, and n represents the number of features, and the covariance matrix C is constructed according to the formula:

[0016]

[0017] where X T is the transpose of the data matrix X, and the covariance matrix C is Element C of ij represents the covariance between the i-th and j-th features;

[0018] Eigenvalue decomposition is to perform eigenvalue decomposition on the covariance matrix C to obtain eigenvalues λ i and corresponding eigenvectors v i We sort the eigenvectors according to the size of the eigenvalues, select the eigenvectors corresponding to the largest k eigenvalues, and construct the principal component matrix V k :

[0019] V k = [v1, v2,..., vk] k

[0020] wherein, represents the feature matrix after dimensionality reduction, k≤n, the more eigenvectors, the greater the variance of data retention;

[0021] Through eigenvalue decomposition, the features that have the greatest impact on the change of the equipment state are screened out;

[0022] Principal component mapping is to project the original data to the dimensionality reduction space using the principal component matrix V k to obtain the data X′

[0023] X ′ = X·V k

[0024] wherein, represents the data matrix after dimensionality reduction.

[0025] As a preferred scheme of the data cleaning method for the digital twin data transmission of the power transmission and transformation equipment, wherein: the dimensionality-reduced digital twin data is further subjected to noise filtering and smoothing processing, including moving average smoothing, exponential weighted smoothing and low-pass filtering;

[0026] The moving average smoothing uses the moving average method to smooth the continuous temperature and current related data;

[0027] The exponential weighted smoothing is used for data with large fluctuations.

[0028] The low-pass filtering is for features with high frequency fluctuations (such as vibration), and the frequency distribution is obtained through Fourier transform, and a low-pass filter is set to filter out frequency components higher than the set threshold to eliminate environmental interference.

[0029] As a preferred scheme of the data cleaning method for the digital twin data transmission of the power transmission and transformation equipment, wherein: the digital twin data is subjected to outlier detection and automatic error correction, including threshold detection method, isolated forest algorithm and automatic error correction.​

[0030] Threshold detection method sets upper threshold U and lower threshold L for data with physical limits in features (such as voltage, current):

[0031]

[0032] Isolation forest algorithm is a special unsupervised algorithm for anomaly detection, which calculates the isolation score s(x) of each data point by building multiple decision trees:

[0033]

[0034] Where E(h(x)) represents the expected path length of data point x, c(n) is the normalization coefficient, n is the sample size, and if s(x) exceeds the set threshold, the point is considered abnormal;

[0035] Automatic error correction refers to repairing marked abnormal points using interpolation or prediction method to ensure data continuity and accuracy, and using linear interpolation method to repair missing data point x t For

[0036]

[0037] As a preferred scheme of the data cleaning method for power transmission and transformation equipment digital twin data transmission, the verification process data continuity and logicality and cleaning data feedback, including time series continuity verification, logical relationship verification and time series model verification;

[0038] Time series continuity verification is to judge whether there is mutation or abnormality by calculating the increment of adjacent data points;

[0039] By calculating the difference Δx i = x i -x i-1 to judge the data continuity,

[0040] if|Δx i |>Δ max

[0041] It is marked as abnormal;

[0042] Where Δ m ax is the maximum allowable increment, if the increment exceeds this threshold, it is considered that the data point does not meet the continuity requirement;

[0043] Logical relationship verification is to verify the logical relationship between features;

[0044] Taking the linear relationship between current I and temperature T as an example, the regression equation is:

[0045] T = β0 + β1I + ∈

[0046] Where β0 and β1 are regression coefficients, and the data points are marked as abnormal if they deviate from this relationship;

[0047] If the checked data points do not meet the prediction range of the model, they are marked as abnormal points;

[0048] The time series model verification is to verify the trend consistency of the time series data by using the ARIMA model;

[0049] If the trend of the equipment data changes dramatically, the ARIMA model is used for trend analysis to find abnormal trend points. For the value x of time point t i , the prediction model is:

[0050] x t = φ1x t-1 +... + φ p x t-p + ∈ t

[0051] Where x t is the feature value at the current time, φ1,..., φ p are autoregressive model parameters, and ∈ t is a noise term;

[0052] If the actual value deviates significantly from the predicted value, it is determined to be abnormal;

[0053] After the data verification, the cleaned data D ′ = {d ′ 1, d ′ 2,..., d ′ n} is fed back to the digital twin system, providing high-quality data support for equipment operation monitoring, state evaluation and prediction analysis.

[0054] In the second aspect, the embodiments of the present application provide a multi-resource inertia and primary frequency modulation auxiliary service sequence acquisition system, which includes a data acquisition and preprocessing module, a multi-dimensional feature analysis and dimension reduction module, a noise filtering and smoothing processing module, an abnormal value detection and automatic error correction module, and a data verification and feedback module.

[0055] The data preprocessing and standardization module is used to preliminarily process the raw data collected by the power transmission and transformation equipment to ensure the consistency and analyzability of the data;

[0056] The multi-dimensional feature analysis and dimension reduction module is used to extract key features from high-dimensional data, simplify data structure, and improve data processing efficiency;

[0057] The noise filtering and outlier detection module is used to eliminate high-frequency noise and random fluctuations in the data, ensuring the stability and continuity of the data.

[0058] The outlier detection and automatic correction module is used to identify and process abnormal fluctuations or fault points in the data, ensuring the accuracy of the data.

[0059] The data verification and feedback module is used to verify the continuity and logical relationship of the processed data, ensuring that the cleaned data conforms to the operation rules of the power transmission and transformation equipment.

[0060] In a third aspect, an embodiment of the present application provides a computer device, comprising a memory and a processor, the memory storing a computer program, wherein the computer program instructions are executed by the processor to implement the steps of the data cleaning method for power transmission and transformation equipment digital twin data transmission as described in the first aspect of the present application.

[0061] In a fourth aspect, an embodiment of the present application provides a computer readable storage medium having a computer program stored thereon, wherein the computer program instructions are executed by the processor to implement the steps of the data cleaning method for power transmission and transformation equipment digital twin data transmission as described in the first aspect of the present application.

[0062] The application has the beneficial effects that: through the unification of data formats, the filling of missing values and the standardization of missing values, the normalization and integrity of data are realized, laying a solid foundation for subsequent analysis and processing; the principal component analysis (PCA) method is adopted to reduce high-dimensional data to low-dimensional space, extract key features describing the running state of the equipment, not only reduce the complexity of data, but also retain the most important variance information in the data, improve the efficiency and accuracy of data processing; the correlation between features is quantified through the covariance matrix, the eigenvalues and corresponding eigenvectors are obtained through eigenvalue decomposition, and the original data is projected to the reduced dimension space through principal component mapping, so that the principal components representing the data features are screened out, and a more accurate data basis is provided for subsequent anomaly detection and prediction; the data is smoothed by moving average smoothing, exponential weighted smoothing and low-pass filtering, etc., to eliminate noise and environmental interference, effectively reduce random fluctuations and noise in the data, improve the stability and reliability of the data, and provide a clearer data signal for subsequent anomaly detection; the threshold detection method, the isolation forest algorithm and the automatic error correction method are used to detect and repair the abnormal values in the data, which not only improves the accuracy of the data, but also ensures the continuity and integrity of the data; the continuity and logic of the data are ensured through time series continuity verification, logical relationship verification and time series model verification, so as to improve the accuracy of equipment operation monitoring, state evaluation and prediction analysis. Through a series of rigorous data preprocessing, feature analysis, dimension reduction, noise filtering, abnormal value detection and data verification steps, the application realizes efficient cleaning of the digital twin data of power transmission and transformation equipment. These steps not only improve the efficiency and accuracy of data processing, but also enhance the stability and reliability of the data, and ultimately achieve the purpose of improving the equipment state monitoring and fault prediction capability. BRIEF DESCRIPTION OF DRAWINGS

[0063] In order to more clearly illustrate the technical solutions of the embodiments of the present application, the drawings needed in the embodiment description will be briefly introduced. Obviously, the drawings in the following description are only some embodiments of the present application, and other drawings can be obtained by those skilled in the art without creative labor.

[0064] Fig. 1 Flow chart of the data cleaning method for power transmission and transformation equipment digital twin data transmission;

[0065] Fig. 2 Computer device diagram of the data cleaning method for power transmission and transformation equipment digital twin data transmission;

[0066] Fig. 3 Smooth processing schematic diagram of the data cleaning method for power transmission and transformation equipment digital twin data transmission.

[0067] Fig. 4 FIG. 1 is a schematic diagram of an isolation forest algorithm for a data cleaning method for digital twin data transmission of power transmission and transformation equipment. DETAILED DESCRIPTION

[0068] In order to make the above-mentioned purposes, features and advantages of the present application more obvious and easy to understand, the specific embodiments of the present application will be described in detail below with reference to the accompanying drawings.

[0069] In the following description, a large number of specific details are set forth in order to facilitate a thorough understanding of the present application, but the present application can also be implemented in other ways different from the description, and those skilled in the art can make similar generalizations without departing from the concept of the present application, therefore the present application is not limited to the specific embodiments disclosed below.

[0070] Secondly, the "one embodiment" or "embodiment" referred to herein means that the specific features, structures or characteristics can be included in at least one implementation of the present application. "In one embodiment" appearing in different places in the specification does not mean the same embodiment, nor is it an independent or alternative embodiment that excludes other embodiments.

[0071] Embodiment 1

[0072] Reference Figs. 1-4 For the first embodiment of the present application, the embodiment provides a data cleaning method for digital twin data transmission of power transmission and transformation equipment, comprising,

[0073] S1: Obtain data and perform preprocessing.

[0074] Further, the digital twin data is obtained and preprocessed, including data format unification, missing value filling and missing value standardization;

[0075] Further, the data format unification includes normalizing the features with large value range difference, through the normalization processing, each feature is converted into a value in the interval [0, 1], ensuring that each feature is processed on a unified scale;

[0076] Specifically, the normalization formula is as follows:

[0077]

[0078] Where x represents the original value of the feature, X represents the feature data set, x' is the normalized value, min(X) and max(X) are the minimum and maximum values respectively;

[0079] The missing value filling includes filling the data points by using linear interpolation method for data missing caused by short-time sampling failure;

[0080] Specifically, the missing value position is x miss , the adjacent data points are x t-1 and x t+1 , and the interpolation formula is:

[0081]

[0082] The standardization of the missing value further eliminates the influence of outliers and makes the data conform to the standard normal distribution.

[0083] Specifically, the standardization formula is:

[0084]

[0085] Where z is the standardized value, max(X) is the mean of feature X, and σ(X) is its standard deviation.

[0086] The standardization operation makes the mean of each feature 0 and the variance 1.

[0087] It should be noted that the data preprocessing step is the basis of data processing, ensuring the consistency and integrity of the data, and providing stable input data for subsequent analysis.

[0088] It should also be noted that the normalization processing makes each feature processed on a unified scale to reduce the magnitude difference between different features and improve the robustness of the algorithm; the missing value filling fills the data missing caused by short-time sampling failure through linear interpolation method, ensuring the continuity of the data and avoiding the analysis deviation caused by data missing; the standardization operation eliminates the influence of outliers through the formula, making the data conform to the standard normal distribution and improving the stability of model analysis.

[0089] S2: Multi-dimensional feature analysis and dimensionality reduction according to the preprocessed digital twin data.

[0090] Further, the multi-dimensional feature analysis and dimensionality reduction according to the preprocessed digital twin data include using the principal component analysis (PCA) method to reduce high-dimensional data to low-dimensional space and extract key features describing the equipment running state.

[0091] Further, the multi-dimensional feature analysis and dimensionality reduction according to the preprocessed digital twin data include covariance matrix construction, eigenvalue decomposition, and principal component mapping.

[0092] The construction of the covariance matrix C quantifies the correlation between features, and the original data matrix is Where m represents the number of samples, n represents the number of features, and the covariance matrix C construction formula is:

[0093]

[0094] Among them, X T Let X be the transpose of the data matrix, and let X be the covariance matrix. element C ij This represents the covariance between the i-th and j-th features;

[0095] Eigenvalue decomposition is the process of performing eigenvalue decomposition on the covariance matrix C to obtain the eigenvalues ​​λ. i and the corresponding feature vector v i We sort the eigenvectors according to their eigenvalues, select the eigenvectors corresponding to the k largest eigenvalues, and construct the principal component matrix V. k :

[0096] V k =[v1,v2,...,v k ]

[0097] in, This represents the feature matrix after dimensionality reduction, where k ≤ n. The more eigenvectors there are, the greater the variance of the data is preserved.

[0098] By using eigenvalue decomposition, the features that have the greatest impact on changes in equipment status are selected.

[0099] Principal component mapping uses the principal component matrix V k Projecting the original data onto the reduced space yields the reduced-dimensional data X′.

[0100] X ′ =X·V k

[0101] in, This represents the data matrix after dimensionality reduction.

[0102] It should be noted that multidimensional feature analysis is used to identify the main change patterns in equipment data, simplify the subsequent data cleaning process, construct the covariance matrix, quantify the correlation between different features, and provide a basis for subsequent eigenvalue decomposition; eigenvalue decomposition, by selecting the eigenvectors corresponding to larger eigenvalues, constructs the principal component matrix to achieve data dimensionality reduction and retain the main information; principal component mapping projects the original data to a new low-dimensional space to obtain the dimensionality-reduced data matrix.

[0103] It should be noted that the power transmission and transformation equipment operation data is usually high-dimensional and has redundant information. The PCA method is used for dimension reduction. Unlike traditional single feature analysis, the PCA method retains the most important variance information in the data. Through covariance matrix construction and eigenvalue decomposition, the correlation between features is quantified, and the features that have the greatest impact on equipment state changes are selected, solving the problems of computational burden and information redundancy caused by high-dimensional data, effectively reducing the data dimension and reducing the computational complexity. Not only does it provide a more accurate data basis for equipment state monitoring and fault prediction, but it also improves the efficiency and accuracy of subsequent data processing.

[0104] S3: Further noise filtering and smoothing processing is performed on the dimension-reduced digital twin data.

[0105] Further, further noise filtering and smoothing processing is performed on the dimension-reduced digital twin data, including moving average smoothing, exponential weighted smoothing, and low-pass filtering.

[0106] Moving average smoothing uses the moving average method to smooth continuous temperature and current-related data.

[0107] Specifically, the smoothing formula is:

[0108]

[0109] where y i represents the smoothed data value, x i is the original value of the i-th data point, and N is the smoothing window size.

[0110] Exponential weighted smoothing is used for data with large fluctuations.

[0111] Specifically, the update formula is:

[0112] y i = αx i + (1-α)y i-1

[0113] where α ∈ (0, 1) is the smoothing coefficient, and y i-1 is the smoothed value at the previous time.

[0114] Low-pass filtering is used for features with high frequency fluctuations (such as vibration). Fourier transform is used to obtain the frequency distribution, and a low-pass filter is set to filter out frequency components above the set threshold to eliminate environmental interference.

[0115] It should be noted that the moving average reduces short-term fluctuation noise by calculating the mean of adjacent data, making the data curve smoother; the exponential smoothing can adjust the importance of the latest data according to the smoothing coefficient, and is suitable for processing data characteristics with rapid fluctuations; the frequency distribution is obtained by Fourier transform, and a low-pass filter is set to filter out frequency components higher than the set threshold to eliminate environmental interference.

[0116] It should also be noted that the operation environment of power transmission and transformation equipment is complex, and the collected data inevitably contains noise. Different smoothing methods are used for different types of data. Moving average method is suitable for smoothing continuous temperature and current data; exponential smoothing is suitable for data with large fluctuations; low-pass filtering is suitable for high-frequency data such as vibration. This differentiated processing method can effectively remove high-frequency interference and random fluctuations, retain the true signal, not only ensure the stability and continuity of the data, but also improve the data quality and reliability.

[0117] S4: Abnormal value detection and automatic correction of digital twin data.

[0118] Further, the abnormal value detection and automatic correction of the digital twin data includes threshold detection method, isolated forest algorithm and automatic correction;

[0119] The threshold detection method sets an upper threshold U and a lower threshold L for data with physical limits in the features (such as voltage and current):

[0120]

[0121] The isolated forest algorithm is an unsupervised algorithm specifically used for anomaly detection, which calculates the isolation score s(x) of each data point by constructing multiple decision trees:

[0122]

[0123] Where E(h(x)) represents the expected path length of data point x, c(n) is the normalization coefficient, n is the sample size, and if s(x) exceeds the set threshold, the point is considered abnormal.

[0124] The automatic correction refers to repairing the marked abnormal points using interpolation or prediction method to ensure the continuity and accuracy of the data, and using linear interpolation method to repair the missing data point x t For

[0125]

[0126] It should be noted that the threshold detection method quickly determines whether the data is abnormal through the preset upper and lower thresholds; the isolation forest algorithm determines the abnormality by calculating the isolation score of the data points; the linear interpolation method is used to repair missing or abnormal data points to ensure data continuity.

[0127] It should also be noted that the outlier detection combines threshold detection and isolation forest algorithm. Threshold detection utilizes the physical limitations of power transmission equipment parameters, is efficient and reliable in removing obvious outliers, and is an effective means to handle large-scale outliers. Isolation forest algorithm is an unsupervised algorithm that can effectively identify abnormal points hidden in high-dimensional data, making up for the shortcomings of threshold detection. The automatic error correction mechanism repairs outliers through interpolation or prediction methods to ensure data continuity and accuracy, providing higher quality input for data monitoring and subsequent prediction. The overall detection process solves the influence of data anomalies on system reliability and stability, improving the accuracy and integrity of the data.

[0128] S5: Check data continuity and logicality and do data cleaning feedback.

[0129] Further, checking data continuity and logicality and doing data cleaning feedback includes time series continuity checking, logical relationship checking, and time series model checking.

[0130] The time series continuity checking is to judge whether there is a mutation or anomaly by calculating the increment of adjacent data points.

[0131] The difference Δx between adjacent points is calculated i = x i -x i-1 to judge the data continuity,

[0132] if |Δx i |>Δ max

[0133] then mark it as abnormal;

[0134] where Δ m ax is the maximum allowable increment. If the increment exceeds this threshold, the data point is considered not to meet the continuity requirement.

[0135] The logical relationship checking is to check the logical relationship between features.

[0136] Taking the linear relationship between current I and temperature T as an example, the regression equation is:

[0137] T = β0 + β1I + ∈

[0138] where β0 and β1 are regression coefficients. If the data point deviates from this relationship, it is marked as abnormal.

[0139] If the data points checked do not conform to the prediction range of the model, they are marked as abnormal points;

[0140] The time series model verification is to verify the trend consistency of the time series data by using the ARIMA model;

[0141] If the trend of the equipment data changes dramatically, trend analysis is performed by the ARIMA model to find abnormal trend points. For the value x i , the prediction model is:

[0142] x t = φ1x t-1 +... + φ p x t-p + ∈ t

[0143] where x t is the feature value at the current time, φ1,..., φ p are autoregressive model parameters, and ∈ t is a noise term;

[0144] If the actual value deviates significantly from the predicted value, it is determined to be abnormal;

[0145] After the data verification, the cleaned data D ′ = {d ′ 1, d ′ 2,..., d ′ n} is fed back to the digital twin system, providing high-quality data support for equipment operation monitoring, state evaluation and prediction analysis.

[0146] It should be noted that the time series continuity verification judges the data continuity by calculating the increment of adjacent data points; the logical relationship verification checks the linear relationship between current and temperature; and the time series model verification analyzes the trend by using the ARIMA model to find abnormal trends.

[0147] It should also be noted that in the overall verification process, the time series continuity verification can identify data mutations and is an important prerequisite for maintaining system real-time performance; the logical relationship verification uses the physical laws between features to determine whether the data conforms to logic and can effectively identify abnormal fluctuations in data logic, helping to improve the accuracy of equipment state evaluation; the time series model verification analyzes the data trend by using the ARIMA model to find abnormal trend points, which helps to identify trend deviations and provides high-quality data support for equipment operation monitoring, state evaluation and prediction analysis; this multi-angle verification mechanism can effectively identify various types of data abnormalities, solve the problem of data inconsistency, ensure the integrity and accuracy of data in time and logic, and provide reliable data support for the digital twin system.

[0148] It should also be noted that the cleaning data is ensured to comply with the operation law of the power transmission and transformation equipment by verifying the processing data, so that it has continuous time logic, and high-quality data guarantee is provided for subsequent storage and feedback. After the verification is completed, the cleaned data is stored and fed back to the digital twin system to support real-time monitoring, fault prediction and state evaluation of the equipment.

[0149] Further, the embodiment also provides a multi-resource inertia and primary frequency modulation auxiliary service sequence acquisition system, comprising a data acquisition and preprocessing module, a multi-dimensional feature analysis and dimension reduction module, a noise filtering and smoothing processing module, an abnormal value detection and automatic error correction module, a data verification and feedback module.

[0150] The data preprocessing and standardization module is used for preliminary processing of the original data collected by the power transmission and transformation equipment to ensure the consistency and analyzability of the data.

[0151] The multi-dimensional feature analysis and dimension reduction module is used for extracting key features from high-dimensional data, simplifying data structure and improving data processing efficiency.

[0152] The noise filtering and abnormal value detection module is used to eliminate high-frequency noise and random fluctuations in the data to ensure the stability and continuity of the data.

[0153] The abnormal value detection and automatic error correction module is used to identify and process abnormal fluctuations or fault points in the data to ensure the accuracy of the data.

[0154] The data verification and feedback module is used to verify the continuity and logical relationship of the processed data to ensure that the cleaned data complies with the operation law of the power transmission and transformation equipment.

[0155] Specifically, the data acquisition and preprocessing module is used to acquire digital twin data of power transmission and transformation equipment, and perform data format unification, missing value filling and standardization processing; wherein, the data format unification converts features with large value range difference into values within a specific interval through a normalization formula to reduce the magnitude difference; the missing value filling uses linear interpolation method to process data missing caused by short-time sampling failure; the standardization processing uses a standardization formula to further eliminate the influence of abnormal values, so that the data conforms to the standard normal distribution, ensuring that each feature is analyzed on a unified scale, and providing stable input for data cleaning.

[0156] Specifically, the multi-dimensional feature analysis and dimension reduction module uses principal component analysis (PCA) method to construct a covariance matrix to quantify feature correlation, selects key feature vectors by eigenvalue decomposition to construct a principal component matrix, projects the original high-dimensional data to a low-dimensional space, removes redundant information, condenses key data information, and enhances data processability, providing a simplified and effective data structure for subsequent noise filtering and abnormal detection steps.

[0157] Specifically, the noise filtering and smoothing module adopts moving average smoothing, exponential weighted smoothing and low-pass filtering methods for the reduced dimension data; moving average smoothing reduces short-term fluctuation noise by calculating the mean of adjacent data; exponential weighted smoothing adjusts the degree of attention to the latest data according to the smoothing coefficient, and processes data with large fluctuations; low-pass filtering eliminates high-frequency interference by means of Fourier transform and filter, effectively removes high-frequency noise in the data, improves data stability, and provides more accurate data basis for anomaly detection.

[0158] Specifically, the anomaly value detection and automatic correction module combines threshold detection method and isolated forest algorithm for anomaly value detection; the threshold detection method sets upper and lower threshold values for physical limit features to remove obvious abnormal values; the isolated forest algorithm constructs multiple decision trees to calculate the isolation score of data points to judge anomalies; for marked abnormal points, interpolation or prediction method is used for automatic correction to ensure data continuity and accuracy, and to provide reliable data for data monitoring and prediction.

[0159] Specifically, the data verification and feedback module verifies the time series continuity, logical relationship and time series model of the cleaned data. By calculating the increment of adjacent data points, comparing the logical relationship between features and analyzing the trend of ARIMA model, the data continuity, logicality and trend consistency are judged, and the abnormal data points are identified and processed. After verification, the cleaned data is fed back to the digital twin system to provide high-quality data support for equipment operation monitoring, state evaluation and prediction analysis, and to ensure the efficient operation of the digital twin system.

[0160] The embodiment also provides a computer device suitable for the data cleaning method for power transmission and transformation equipment digital twin data transmission, which comprises a memory and a processor; the memory is used to store computer executable instructions, and the processor is used to execute the computer executable instructions to realize the data cleaning method for power transmission and transformation equipment digital twin data transmission as proposed in the above embodiment.

[0161] The computer device can be a terminal, and the computer device includes a processor, a memory, a communication interface, a display screen and an input device connected by a system bus. The processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system and a computer program. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The communication interface of the computer device is used for wired or wireless communication with external terminals. The wireless mode can be achieved through WIFI, operator network, NFC (near field communication) or other technologies. The display screen of the computer device can be a liquid crystal display screen or an electronic ink display screen. The input device of the computer device can be a touch layer overlaid on the display screen, or a key, trackball or touchpad arranged on the shell of the computer device, or an external keyboard, touchpad or mouse, etc.

[0162] The embodiment also provides a storage medium having a computer program stored thereon, the program being executed by a processor to implement the data cleaning method for digital twin data transmission of power transmission and transformation equipment proposed in the above embodiment.

[0163] In summary, the present application provides a data cleaning method for digital twin data transmission of power transmission and transformation equipment. By integrating multi-dimensional feature analysis, noise filtering, outlier detection and logical verification technologies, the method effectively detects and processes noise, outliers and missing values in the data according to the characteristics of power transmission and transformation equipment data, ensuring the integrity and logicality of the data before entering the digital twin system. The overall method not only significantly improves the real-time performance of equipment state monitoring and the reliability of decision-making, but also reduces the impact of factors such as sensor failure and transmission delay on the data through continuous and logical verification of time series data. By integrating low-pass filtering, smoothing processing and machine learning methods, an automated cleaning process is formed, which enables the digital twin model to obtain more accurate equipment operation information, providing reliable data support for equipment fault prediction, state evaluation and operation optimization, thereby improving the overall performance of the digital twin system.

[0164] Embodiment 2

[0165] In the second embodiment of the present application, a data cleaning method for digital twin data transmission of power transmission and transformation equipment is provided. In order to verify the effectiveness of the data cleaning method in improving data quality, accuracy and system real-time performance, the experiment takes the operation data of power transmission and transformation equipment as the experimental object, constructs a multi-dimensional, time series feature matrix, and simulates various noise and abnormal situations.

[0166] The experimental data covers key characteristics of the equipment (such as voltage, current, temperature, vibration, etc.). Through multidimensional feature analysis and machine learning algorithms, noise, missing values, outliers, and fault simulation data are detected and cleaned.

[0167] The experiment compared the cleaning effect and performance of the method of this invention with existing time series cleaning methods and weighted iterative cleaning methods. The main indicators included the accuracy of the cleaned data, data recovery rate, anomaly detection rate, cleaning efficiency, and the false diagnosis rate in equipment status monitoring. For the same set of digital twin data of power transmission and transformation equipment, the digital twin data cleaning method based on multi-dimensional feature analysis (the method of this invention), time series analysis, and weighted iteration were applied for data cleaning, and various indicators were obtained based on the samples. Table 1 shows a comparison of the implementation effects of different methods;

[0168] Table 1

[0169]

[0170]

[0171] Experimental results show that the method of this invention significantly outperforms existing methods in terms of data accuracy, data recovery rate, anomaly detection rate, and false diagnosis rate for digital twin data of power transmission and transformation equipment, especially demonstrating higher efficiency and accuracy when processing real-time monitoring data. This makes the cleaning method significantly advantageous in improving the monitoring accuracy of power transmission and transformation equipment, reducing the risk of false diagnosis, and optimizing equipment fault prediction.

[0172] It should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention and are not intended to limit it. Although the present invention has been described in detail with reference to preferred embodiments, those skilled in the art should understand that modifications or equivalent substitutions can be made to the technical solutions of the present invention without departing from the spirit and scope of the technical solutions of the present invention, and all such modifications or substitutions should be covered within the scope of the claims of the present invention.

Claims

1. A data cleaning method for digital twin data transmission of power transmission and transformation equipment, characterized in that: include, Acquire digital twin data and preprocess it; Multi-dimensional feature analysis and dimensionality reduction are performed based on preprocessed digital twin data; Further noise filtering and smoothing processing is performed on the dimensionality-reduced digital twin data; Outlier detection and automatic error correction for digital twin data; Verify the continuity and logic of the processed data and provide feedback on the cleaned data; The outlier detection and automatic error correction of digital twin data includes threshold detection, isolated forest algorithm, and automatic error correction. The threshold detection method sets an upper threshold U and a lower threshold L for data with physical limitations in its features: The isolated forest algorithm is an unsupervised algorithm specifically designed for outlier detection. It calculates the isolated score s(x) for each data point by constructing multiple decision trees. Where E(h(x)) represents the expected path length of data point x, c(n) is the normalization coefficient, n is the sample size, and if s(x) exceeds the set threshold, the point is considered abnormal. The automatic error correction refers to the use of interpolation or prediction methods to repair marked outliers, ensuring data continuity and accuracy. Linear interpolation is used to repair missing data points x. t for The verification process checks the continuity and logic of the data and provides cleaned data feedback, including time series continuity verification, logical relationship verification, and time series model verification. The time series continuity check is performed by calculating the increment of adjacent data points to determine whether there are sudden changes or anomalies. By calculating the difference Δx between adjacent points i =x i -x i-1 To determine the continuity of data, if|Δx i |>D max Then mark it as an exception; Where Δ m ax is the maximum allowable increment. If the increment exceeds this threshold, the data point is considered to not meet the continuity requirement. The logical relationship verification is performed on the logical relationships between features; Taking the linear relationship between current I and temperature T as an example, let the regression equation be: T=β0+β1I+∈ Where β0 and β1 are regression coefficients, and data points that deviate from this relationship are marked as outliers; If the data points detected do not conform to the model's prediction range, they are marked as outliers. The time series model validation is performed by using an ARIMA model to examine the trend consistency of the time series data. If the trend of the device data changes drastically, trend analysis is performed using the ARIMA model to identify abnormal trend points; for the value x at time point t... i Its prediction model is: x t =φ1x t-1 +...+φ p x t-p +∈ t Where, x t Let φ1,...,φ be the characteristic values ​​at the current time. p For the parameters of the autoregressive model, ∈ t Noise term; If the actual value deviates significantly from the predicted value, it is considered abnormal; After the data verification is passed, the cleaned data D′={d′1,d′2,...,d′ will be processed. n The data is fed back to the digital twin system, providing high-quality data support for equipment operation monitoring, status assessment, and predictive analysis.

2. The data cleaning method for digital twin data transmission of power transmission and transformation equipment as described in claim 1, characterized in that: The acquisition and preprocessing of digital twin data includes unifying the data format, filling in missing values, and standardizing missing values. The standardization of the data format includes normalizing features with large differences in value range. Through normalization, each feature is converted into a value within the interval [0, 1], ensuring that each feature is processed on a uniform scale. The filling of missing values ​​includes filling data points with linear interpolation for data missing caused by short-term sampling failures; The standardization of missing values ​​further eliminates the influence of outliers and makes the data follow a standard normal distribution.

3. The data cleaning method for digital twin data transmission of power transmission and transformation equipment as described in claim 2, characterized in that: The process of performing multi-dimensional feature analysis and dimensionality reduction based on preprocessed digital twin data includes... The multi-dimensional feature analysis employs principal component analysis, which reduces high-dimensional data to a low-dimensional space and extracts key features describing the operating status of the equipment.

4. The data cleaning method for digital twin data transmission of power transmission and transformation equipment as described in claim 3, characterized in that: The process involves multi-dimensional feature analysis and dimensionality reduction based on preprocessed digital twin data, including covariance matrix construction, eigenvalue decomposition, and principal component mapping. The construction of the covariance matrix C quantifies the correlation between features. Let the original data matrix be... Where m represents the number of samples and n represents the number of features, the formula for constructing the covariance matrix C is: Among them, X T Let X be the transpose of the data matrix, and let X be the covariance matrix. element C ij This represents the covariance between the i-th and j-th features; The eigenvalue decomposition is performed on the covariance matrix C to obtain the eigenvalues ​​λ. i and the corresponding feature vector v i We sort the eigenvectors according to their eigenvalues, select the eigenvectors corresponding to the k largest eigenvalues, and construct the principal component matrix V. k : V k =[v1,v2,...,v k ] in, This represents the feature matrix after dimensionality reduction, where k ≤ n. The more eigenvectors there are, the greater the variance of the data is preserved. By using eigenvalue decomposition, the features that have the greatest impact on changes in equipment status are selected. The principal component mapping uses the principal component matrix V. k Projecting the original data onto the reduced space yields the reduced-dimensional data X′. X ′ =X·V k in, This represents the data matrix after dimensionality reduction.

5. The data cleaning method for digital twin data transmission of power transmission and transformation equipment as described in claim 4, characterized in that: The further steps involve noise filtering and smoothing of the dimensionality-reduced digital twin data, including moving average smoothing, exponential weighted smoothing, and low-pass filtering. The moving average smoothing method uses the moving average method to smooth continuous temperature and current related data. The exponentially weighted smoothing is used for data with large fluctuations. The low-pass filtering method is used to obtain the frequency distribution through Fourier transform for features with high frequency fluctuations, and then set a low-pass filter to filter out frequency components that are higher than a set threshold in order to eliminate environmental interference.

6. A system for data cleaning of digital twin data transmission for power transmission and transformation equipment, based on the data cleaning method for digital twin data transmission for power transmission and transformation equipment as described in any one of claims 1 to 5, characterized in that: It also includes modules for data acquisition and preprocessing, multi-dimensional feature analysis and dimensionality reduction, noise filtering and smoothing, outlier detection and automatic error correction, and data verification and feedback. The data preprocessing and standardization module is used to perform preliminary processing on the raw data collected from power transmission and transformation equipment to ensure data consistency and analyzability. The multi-dimensional feature analysis and dimensionality reduction module is used to extract key features from high-dimensional data, simplify the data structure, and improve data processing efficiency. The noise filtering and outlier detection module is used to eliminate high-frequency noise and random fluctuations in the data, ensuring the stability and continuity of the data. The outlier detection and automatic error correction module is used to identify and process abnormal fluctuations or fault points in the data to ensure the accuracy of the data. The data verification and feedback module is used to verify the continuity and logical relationship of the processed data to ensure that the cleaned data conforms to the operating rules of the power transmission and transformation equipment.

7. A computer device comprising a memory and a processor, wherein the memory stores a computer program, characterized in that: When the processor executes the computer program, it implements the steps of the data cleaning method for digital twin data transmission of power transmission and transformation equipment as described in any one of claims 1 to 5.

8. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by the processor, it implements the steps of the data cleaning method for digital twin data transmission of power transmission and transformation equipment as described in any one of claims 1 to 5.

Images