Method, system and equipment for improving multi-source data quality of transformer and medium

Abstract

The invention belongs to the technical field of data processing, and discloses a transformer multi-source data quality improvement method, system and device and a medium, and the method comprises the following steps: collecting monitoring data of a distribution transformer in real time through a plurality of types of monitoring devices deployed on the distribution transformer, forming multi-source data, and carrying out the data alignment processing of the multi-source data; performing data quality processing on the time-aligned monitoring data by using a multi-source collaborative anomaly detection method and combining a missing duration difference judgment mechanism; and performing data quality evaluation on the quality improvement data to obtain a data quality evaluation result, and performing data quality grade division on the quality improvement data according to the data quality evaluation result to obtain a data quality division result. According to the method, a reliable data basis is provided for sensing, analyzing and managing the running state of the power grid, analysis deviation caused by data inconsistency is reduced, and the accuracy and efficiency of data processing are remarkably improved.

Description

Technical Field

[0001] This invention relates to the field of data processing technology, and in particular to a method, system, device and medium for improving the quality of multi-source data from transformers. Background Technology

[0002] With the continuous acceleration of smart grid construction, distribution transformers, as core equipment in the distribution network, play an irreplaceable role in ensuring the safety and efficiency of the power grid through accurate monitoring of their operating status. Currently, distribution transformers are widely equipped with various intelligent monitoring devices that can collect multi-dimensional operating information such as three-phase voltage, current, active power, reactive power, and temperature in real time, providing solid data support for equipment fault diagnosis, load forecasting, and condition assessment.

[0003] However, in the data processing stage, the existing system still faces several prominent challenges: (1) Strong heterogeneity of multi-source data: There are significant differences in data accuracy, sampling format, and timestamp synchronization among different manufacturers and models of equipment, which makes data fusion and unified modeling difficult and restricts the effectiveness of subsequent analysis; (2) Frequent occurrence of abnormal data: Affected by factors such as sensor aging and communication interference, key parameters such as voltage and current are prone to sudden changes and drifts. If they are not identified and corrected in time, they will seriously interfere with the accuracy of diagnosis and prediction; (3) Crude handling of missing values: Data missing problems caused by equipment failure or communication interruption are common. Existing processing methods mostly use mean substitution or linear interpolation, ignoring the physical coupling relationship between electrical quantities, making it difficult to accurately restore the actual operating state of the equipment; (4) Insufficient expression of fluctuation characteristics: Power grid operation data naturally has dynamic fluctuations. Traditional smoothing algorithms often have the problem of "filtering waves", which easily weakens the expression of key dynamic features such as load changes; (5) Imperfect quality assessment system: Existing methods mostly rely on single indicators such as missing rate and abnormal rate, lacking a systematic and quantitative multi-dimensional assessment framework, making it difficult to comprehensively depict the data quality level.

[0004] The aforementioned problems significantly reduce the accuracy and reliability of distribution transformer monitoring data, thereby restricting the depth and breadth of smart grid application in intelligent operation and maintenance and digital assessment.

[0005] Therefore, how to provide a method, system, equipment, and medium for improving the quality of multi-source data from transformers is an urgent problem to be solved. Summary of the Invention

[0006] This invention provides a method, system, device, and medium for improving the quality of multi-source data in transformers, in order to solve the problems mentioned above in the prior art.

[0007] According to a first aspect of the present invention, a method for improving the quality of multi-source data from transformers is provided.

[0008] In one embodiment, the method for improving the quality of multi-source data from transformers includes the following steps:

[0009] By deploying various monitoring devices on distribution transformers, monitoring data of distribution transformers are collected in real time to form multi-source data. Through a multi-source data alignment mechanism, the multi-source data is aligned to obtain time-aligned monitoring data.

[0010] By using a multi-source collaborative anomaly detection method and combining it with a missing duration difference judgment mechanism, data quality processing is performed on time-aligned monitoring data to obtain data with improved quality.

[0011] Based on a pre-defined data quality assessment system, the data quality of the data for quality improvement is evaluated to obtain data quality assessment results. Then, based on the data quality assessment results, the data quality of the data for quality improvement is classified into data quality levels to obtain data quality classification results.

[0012] In one embodiment, the process of collecting monitoring data of the distribution transformer in real time through multiple types of monitoring devices deployed on the distribution transformer to form multi-source data, and then performing data alignment processing on the multi-source data through a multi-source data alignment mechanism to obtain time-aligned monitoring data includes the following steps:

[0013] By integrating and processing the monitoring data of distribution transformers collected in real time by various monitoring devices, multi-source data is obtained;

[0014] Based on a preset scanning cycle, the time information of the data sources is scanned from multiple sources to obtain the time information of each data source.

[0015] Based on the time information of each data source, the sampling interval differences of each data source in the multi-source data are determined. Based on the sampling interval differences, the time window sliding and interpolation fusion method is used to align the monitoring data of each data source to obtain time-aligned monitoring data.

[0016] In one embodiment, the process of using a multi-source collaborative anomaly detection method, combined with a missing duration difference judgment mechanism, to perform data quality processing on time-aligned monitoring data to obtain improved data includes the following steps:

[0017] A multi-source collaborative anomaly detection method is used to detect anomalies in time-aligned monitoring data, identify anomalous data points in the time-aligned monitoring data, and remove these anomalous data points.

[0018] For the monitoring data after removing outlier data points, missing value detection is performed, and the duration of missing values ​​is determined.

[0019] The missing duration is compared with a preset missing duration threshold. If the missing duration is greater than the preset missing duration threshold, the missing value is filled using a long-term missing value filling method to obtain the filled monitoring data. Otherwise, the missing value is filled using a short-term missing value filling method to obtain the filled monitoring data.

[0020] The fluctuation of the supplemented monitoring data is monitored, and based on the test results, the supplemented monitoring data is smoothed to obtain data with improved quality.

[0021] In one embodiment, the multi-source collaborative anomaly detection method includes univariate detection, multivariate collaborative detection, and physical constraint detection;

[0022] The univariate detection method is used to detect abnormal data in time-aligned monitoring data using box plots.

[0023] The multivariate collaborative detection is used to detect abnormal data in time-aligned monitoring data using a multivariate detection algorithm based on ensemble learning.

[0024] The physical constraint detection is used to detect anomalies in time-aligned monitoring data by utilizing the physical laws of the power system.

[0025] In one embodiment, the short-term missing value filling method uses interpolation to fill in missing values.

[0026] In one embodiment, the long-term missing value imputation method uses a multivariate regression model to impute missing values. The multivariate regression model is trained using monitoring data as independent variables and missing data as dependent variables.

[0027] In one embodiment, the step of monitoring the volatility of the supplemented monitoring data and smoothing the data based on the detection results to obtain improved quality data includes:

[0028] The standard deviation of the supplemented monitoring data is calculated and compared with the preset standard deviation threshold for judgment.

[0029] If the standard deviation is less than the preset standard deviation threshold, the moving average algorithm is used to smooth the filled monitoring data to obtain the quality improvement data.

[0030] If the standard deviation is greater than or equal to the preset standard deviation threshold, wavelet denoising is used to smooth the filled monitoring data to obtain improved quality data.

[0031] In one embodiment, the calculation formula for smoothing the filled monitoring data using the moving average algorithm is as follows:

[0032]

[0033] In the formula, This represents the improved data obtained after smoothing the imputed monitoring data using a moving average algorithm, where n represents the sliding window and x represents the sliding window. i Let x represent the i-th monitoring data point after filling in the missing data. i-1 Let x represent the (i-1)th monitoring data point after filling in the missing data. i-n+1 This represents the (i-n+1)th monitoring data point after filling in the gaps.

[0034] In one embodiment, the data smoothing process of the filled monitoring data using wavelet denoising includes: using wavelet transform to decompose the filled monitoring data into different frequency sub-bands, removing high-frequency noise through threshold processing, and then performing inverse transform to reconstruct the smoothed data to obtain improved data.

[0035] In one embodiment, the process of evaluating the quality of the improved data based on a preset data quality assessment system, obtaining data quality assessment results, and classifying the improved data into data quality levels based on the data quality assessment results to obtain data quality classification results includes the following steps:

[0036] Based on a pre-defined data quality assessment system, multi-dimensional assessment indicators are calculated based on the data showing quality improvement.

[0037] Based on preset indicator weights and combined with multi-dimensional evaluation indicators, the total score of quality improvement data is calculated.

[0038] Based on the overall score of the quality improvement data, the quality improvement data is divided into corresponding data quality levels, resulting in the data quality classification results.

[0039] In one embodiment, the preset data quality assessment system includes: integrity indicators, consistency indicators, accuracy indicators, volatility indicators, and anomaly rate indicators.

[0040] According to a second aspect of the present invention, a transformer multi-source data quality improvement system is provided.

[0041] In one embodiment, a transformer multi-source data quality improvement system includes:

[0042] The data acquisition and alignment module is used to collect monitoring data of the distribution transformer in real time through various monitoring devices deployed on the distribution transformer, forming multi-source data, and perform data alignment processing on the multi-source data through a multi-source data alignment mechanism to obtain time-aligned monitoring data;

[0043] The data quality improvement module is used to process time-aligned monitoring data using a multi-source collaborative anomaly detection method and a missing duration difference judgment mechanism to obtain improved data.

[0044] The data segmentation module is used to evaluate the data quality of the quality improvement data based on a preset data quality assessment system, obtain the data quality assessment results, and classify the data quality improvement data into data quality levels based on the data quality assessment results, thus obtaining the data quality segmentation results.

[0045] In one embodiment, the real-time acquisition of monitoring data from the distribution transformer by various monitoring devices deployed on the distribution transformer forms multi-source data. A multi-source data alignment mechanism is then used to align the multi-source data to obtain time-aligned monitoring data, including:

[0046] By integrating and processing the monitoring data of distribution transformers collected in real time by various monitoring devices, multi-source data is obtained;

[0047] Based on a preset scanning cycle, the time information of the data sources is scanned from multiple sources to obtain the time information of each data source.

[0048] Based on the time information of each data source, the sampling interval differences of each data source in the multi-source data are determined. Based on the sampling interval differences, the time window sliding and interpolation fusion method is used to align the monitoring data of each data source to obtain time-aligned monitoring data.

[0049] In one embodiment, the multi-source collaborative anomaly detection method, combined with a missing duration difference judgment mechanism, is used to perform data quality processing on time-aligned monitoring data to obtain quality-improved data, including:

[0050] A multi-source collaborative anomaly detection method is used to detect anomalies in time-aligned monitoring data, identify anomalous data points in the time-aligned monitoring data, and remove these anomalous data points.

[0051] For the monitoring data after removing outlier data points, missing value detection is performed, and the duration of missing values ​​is determined.

[0052] The missing duration is compared with a preset missing duration threshold. If the missing duration is greater than the preset missing duration threshold, the missing value is filled using a long-term missing value filling method to obtain the filled monitoring data. Otherwise, the missing value is filled using a short-term missing value filling method to obtain the filled monitoring data.

[0053] The fluctuation of the supplemented monitoring data is monitored, and based on the test results, the supplemented monitoring data is smoothed to obtain data with improved quality.

[0054] According to a third aspect of the present invention, a computer device is provided.

[0055] In some embodiments, the computer device includes a memory and a processor, the memory storing a computer program, and the processor executing the computer program to implement the steps of the method described above.

[0056] According to a fourth aspect of the present invention, a computer-readable storage medium is provided.

[0057] In one embodiment, a computer program is stored on the computer-readable storage medium, which, when executed by a processor, implements the steps of the above method.

[0058] The technical solutions provided by the embodiments of the present invention may include the following beneficial effects:

[0059] (1) By constructing a precise time synchronization mechanism, this invention effectively solves the inconsistency problem of data format, sampling accuracy and timestamp differences among multi-source monitoring devices, realizes the unified integration of cross-device and cross-source data, thereby forming a complete and standardized multi-source dataset, providing a reliable data foundation for the perception, analysis and management of power grid operation status, reducing analysis deviations caused by data inconsistency, and significantly improving the accuracy and efficiency of data processing.

[0060] (2) The collaborative anomaly detection method proposed in this invention organically combines univariate detection, multivariate collaborative detection, and power physical constraint detection, significantly improving the accuracy and robustness of anomaly identification compared to traditional methods, and effectively reducing the false alarm rate. This method can quickly identify single-parameter anomalies and reveal potential global or local anomalies based on multivariate correlations. Simultaneously, it utilizes the physical laws of the power system for cross-verification, enhancing the reliability of the detection results. This enables accurate identification of potential faults or abnormal states in the power grid, improving the safety and stability of power grid operation, reducing maintenance and repair costs, and extending the service life of critical equipment.

[0061] (3) This invention proposes a differentiated incompleteness strategy based on physical constraints and intelligent prediction, addressing the different characteristics of short-term and long-term missing values. By combining spline interpolation with a multivariate time-series prediction model, the physical coupling relationship and historical evolution law between power system parameters are fully utilized to achieve high-precision recovery of missing values. This method can maintain the authenticity and consistency of data, improve the reliability of subsequent modeling and prediction, and enhance the scientific and economical operation of the power grid.

[0062] (4) The adaptive smoothing method proposed in this invention can intelligently select the optimal smoothing algorithm based on the fluctuation characteristics of the monitored data. While effectively suppressing noise interference, it retains key dynamic characteristics such as load mutations to the maximum extent, which helps to capture minute changes and abnormal signals in the operation of the power grid in a timely manner. This method ensures the authenticity and sensitivity of power data, thereby improving the ability to monitor power quality and ensure the stable operation of the power grid.

[0063] (5) This invention constructs a comprehensive quality assessment system covering multiple dimensions such as completeness, consistency, accuracy, volatility, and anomaly rate, and generates a total score through weighted calculation, classifying data quality into three levels: high, medium, and low. This system achieves quantitative and refined assessment of monitoring data quality, providing a clear basis for intelligent power grid analysis and operation and maintenance decisions, improving the scientific nature of data management, reducing operation and maintenance risks caused by insufficient data quality, and enhancing the overall efficiency and reliability of power grid operation.

[0064] It should be understood that the above general description and the following detailed description are exemplary and explanatory only, and are not intended to limit the invention. Attached Figure Description

[0065] The accompanying drawings, which are incorporated in and form part of this specification, illustrate embodiments consistent with the invention and, together with the description, serve to explain the principles of the invention.

[0066] Figure 1 This is one of the flowcharts illustrating a method for improving the quality of multi-source data from a transformer according to an exemplary embodiment;

[0067] Figure 2 This is a schematic diagram illustrating a transformer multi-source data quality improvement system according to an exemplary embodiment;

[0068] Figure 3 This is a schematic diagram of the structure of a computer device according to an exemplary embodiment.

[0069] Figure 4 This is a second method for improving the quality of multi-source data in a transformer, as illustrated in an exemplary embodiment.

[0070] Figure 5 This is a flowchart illustrating the multi-source data alignment process in a transformer multi-source data quality improvement method according to an exemplary embodiment;

[0071] Figure 6 This is a flowchart illustrating a multi-source collaborative anomaly detection process in a transformer multi-source data quality improvement method according to an exemplary embodiment.

[0072] Figure 7This is a flowchart illustrating the intelligent missing value filling process in a transformer multi-source data quality improvement method according to an exemplary embodiment.

[0073] Figure 8 This is a flowchart illustrating the data smoothing process in a transformer multi-source data quality improvement method according to an exemplary embodiment;

[0074] Figure 9 This is a flowchart illustrating the data quality assessment process in a transformer multi-source data quality improvement method according to an exemplary embodiment;

[0075] Figure 10 This is a schematic diagram of the original data in a transformer multi-source data quality improvement method according to an exemplary embodiment;

[0076] Figure 11 This is a schematic diagram of the improved data in a transformer multi-source data quality improvement method according to an exemplary embodiment. Detailed Implementation

[0077] The following description and accompanying drawings fully illustrate specific embodiments described herein to enable those skilled in the art to practice them. Some portions and features of certain embodiments may be included in or replace portions and features of other embodiments. The scope of the embodiments herein includes the entire scope of the claims and all available equivalents thereof. The various embodiments described herein are presented in a progressive manner, with each embodiment focusing on its differences from other embodiments; similar or identical parts between embodiments can be referred to interchangeably.

[0078] The modules in the apparatus or system of this application can be implemented entirely or partially through software, hardware, or a combination thereof. These modules can be embedded in or independent of the processor in a computer device in hardware form, or stored in the memory of a computer device in software form, so that the processor can call and execute the operations corresponding to each module.

[0079] Where there is no conflict, the embodiments and features in the embodiments of the present invention can be combined with each other.

[0080] Figure 1 and Figure 4-9 An embodiment of a transformer multi-source data quality improvement method of the present invention is shown.

[0081] In this optional embodiment, the method for improving the quality of multi-source data from transformers includes the following steps:

[0082] Step S101: Real-time monitoring data of the distribution transformer is collected by multiple monitoring devices deployed on the distribution transformer to form multi-source data. The multi-source data is then aligned using a multi-source data alignment mechanism to obtain time-aligned monitoring data.

[0083] Step S102: Using a multi-source collaborative anomaly detection method and combined with a missing duration difference judgment mechanism, the time-aligned monitoring data is processed to obtain improved data.

[0084] Step S103: Based on the preset data quality assessment system, perform data quality assessment on the quality improvement data to obtain data quality assessment results, and classify the data quality improvement data into data quality levels according to the data quality assessment results to obtain data quality classification results.

[0085] In this optional embodiment, when monitoring data of the distribution transformer is collected in real time by multiple monitoring devices deployed on the distribution transformer to form multi-source data, and the multi-source data is aligned through a multi-source data alignment mechanism to obtain time-aligned monitoring data, the monitoring data of the distribution transformer collected in real time by multiple monitoring devices can be integrated to obtain multi-source data; the time information of the data sources is scanned based on a preset scanning cycle to obtain the time information of each data source; the sampling interval difference of each data source in the multi-source data is determined according to the time information of each data source, and based on the sampling interval difference, the monitoring data of each data source is time-aligned using a time window sliding and interpolation fusion method to obtain time-aligned monitoring data.

[0086] In this optional embodiment, when using a multi-source collaborative anomaly detection method combined with a missing duration difference judgment mechanism to perform data quality processing on time-aligned monitoring data to obtain quality-improved data, the multi-source collaborative anomaly detection method can be used to detect anomalies in the time-aligned monitoring data, identify abnormal data points in the time-aligned monitoring data, and remove abnormal data points; for the monitoring data after the abnormal data point removal process, missing value detection is performed, and the missing duration is determined; the missing duration is compared with a preset missing duration threshold. If the missing duration is greater than the preset missing duration threshold, a long-term missing value imputation method is used to imput the missing value to obtain imputed monitoring data; otherwise, a short-term missing value imputation method is used to imput the missing value to obtain imputed monitoring data; the imputed monitoring data is subjected to fluctuation monitoring, and based on the detection results, the imputed monitoring data is smoothed to obtain quality-improved data.

[0087] In this optional embodiment, the multi-source collaborative anomaly detection method includes univariate detection, multivariate collaborative detection, and physical constraint detection; the univariate detection is used to detect anomalies in time-aligned monitoring data using box plots; the multivariate collaborative detection is used to detect anomalies in time-aligned monitoring data using an ensemble learning multivariate detection algorithm; and the physical constraint detection is used to detect anomalies in time-aligned monitoring data using the physical laws of the power system.

[0088] In this optional embodiment, the short-term missing value filling method uses interpolation to fill in missing values.

[0089] In this optional embodiment, the long-term missing value imputation method uses a multivariate regression model to impute missing values. The multivariate regression model is trained using monitoring data as independent variables and missing data as dependent variables.

[0090] In this optional embodiment, the step of monitoring the fluctuation of the filled monitoring data and smoothing the filled monitoring data according to the detection results to obtain quality-improved data includes: calculating the standard deviation of the filled monitoring data and comparing the standard deviation with a preset standard deviation threshold; if the standard deviation is less than the preset standard deviation threshold, then the filled monitoring data is smoothed using a moving average algorithm to obtain quality-improved data; if the standard deviation is greater than or equal to the preset standard deviation threshold, then the filled monitoring data is smoothed using a wavelet denoising method to obtain quality-improved data.

[0091] In this optional embodiment, the data smoothing process of the filled monitoring data using wavelet denoising includes: using wavelet transform to decompose the filled monitoring data into different frequency sub-bands, removing high-frequency noise through threshold processing, and then performing inverse transform to reconstruct smooth data to obtain improved data.

[0092] In this optional embodiment, when assessing the quality of the improved data based on a preset data quality assessment system to obtain a data quality assessment result, and classifying the improved data into data quality levels based on the data quality assessment result to obtain a data quality classification result, multi-dimensional assessment indicators can be calculated based on the improved data according to the preset data quality assessment system; based on preset indicator weights and combined with the multi-dimensional assessment indicators, a total score for the improved data can be calculated; and based on the total score of the improved data, the improved data can be classified into corresponding data quality levels to obtain a data quality classification result.

[0093] In this optional embodiment, the preset data quality assessment system includes: integrity index, consistency index, accuracy index, volatility index, and anomaly rate index.

[0094] To facilitate understanding of the above technical solutions of the present invention, the following further describes the above technical solutions of the present invention from the perspectives of architecture and principle, as follows:

[0095] (1) Multi-source data alignment;

[0096] To address the issue of inconsistent timestamps in data collected by different monitoring devices for distribution transformers, this invention designs a multi-source data alignment mechanism. After system startup, it automatically scans the time information of each data source. For cases where sampling intervals differ, alignment is achieved using a time window sliding and interpolation fusion method. For example, if device A collects data every 5 minutes and device B collects data every 10 minutes, the longer time interval (10 minutes) is used as the baseline time window. Within this time window, the data from device A is interpolated to unify its sampling interval to 10 minutes, thus achieving time scale consistency. For cases where start times differ, the earliest global start time is determined, and the time axes of all data streams are uniformly shifted to this baseline point, achieving time alignment of the monitoring data.

[0097] (2) Multi-source collaborative anomaly detection;

[0098] To address the shortcomings of existing anomaly detection methods, which rely on single features and suffer from insufficient detection accuracy, this invention proposes a multi-source collaborative anomaly detection method that integrates three mechanisms: univariate detection, multivariate collaborative detection, and physical constraint detection, thereby improving the comprehensiveness and reliability of anomaly identification.

[0099] 1) Univariate detection;

[0100] This invention employs a statistical feature-based method for anomaly detection in a single data stream. Taking the box plot method as an example, for the collected current data, the first quartile (Q1), the second quartile (Q2, median), and the third quartile (Q3) are first calculated, and the interquartile range IQR = Q3 - Q1 is obtained. If the data value is less than Q1 - 1.5 × IQR or greater than Q3 + 1.5 × IQR, it is determined to be an anomaly.

[0101] 2) Multivariate collaborative detection;

[0102] To address the inherent correlations among multidimensional parameters such as voltage, current, active power, and reactive power, a multivariate detection algorithm based on ensemble learning is employed to identify global or local anomalies. For example, a decision tree model is constructed multiple times by randomly selecting features and segmentation values, and the average path length of samples in the forest is calculated. If a sample's path is significantly shorter, it is identified as an anomaly. This method can effectively identify complex anomaly patterns under the coupling relationships between multiple parameters.

[0103] 3) Anomaly detection based on physical constraints;

[0104] This invention utilizes the physical laws of power systems to detect anomalies in monitoring data, including but not limited to:

[0105] Three-phase current vector sum detection: Real-time calculation of the vector sum of three-phase currents I1, I2, and I3, I = I1 + I2 + I3. If |I| exceeds a preset threshold, the current data is determined to be abnormal.

[0106] Power factor detection: Based on the power factor formula PF=P / √(P 2 +Q 2 The power factor is calculated by combining measured current, voltage, and power data, where P is active power and Q is reactive power. If the deviation between the calculated value and the actual measured value exceeds a set threshold (e.g., 5%), the relevant data is considered abnormal.

[0107] (3) Intelligent filling of missing values;

[0108] This invention addresses the problem of missing monitoring data by proposing an intelligent imputation method based on differences in the duration of missing data, specifically including:

[0109] 1) Short-term missing data filling;

[0110] When the detected missing duration is no more than 10 minutes, interpolation is used for data recovery. For example, if the missing data occurs between time points t1 and t2, and the voltage at time t1 is V1 and the voltage at time t2 is V2, then the voltage value at the missing time t is:

[0111]

[0112] 2) Long-term missing data filling;

[0113] For data missing for more than 10 minutes, a multivariate regression model is constructed, using voltage, current, active power, and reactive power as independent variables and the missing variable as the dependent variable. The model is trained using historical data. When data is missing, the current independent variable is input into the model to predict the missing value.

[0114] (4) Data smoothing processing;

[0115] This invention addresses the problems of high volatility and noise in power grid monitoring data by proposing an adaptive data smoothing processing method, including:

[0116] 1) Volatility monitoring;

[0117] The system calculates the standard deviation or coefficient of variation of the data in real time to determine the degree of fluctuation. When the standard deviation is below a threshold, a moving average algorithm is used. Assuming the window size is n, the smoothed value of the current data point xi is:

[0118]

[0119] In the formula, This represents the improved data obtained after smoothing the imputed monitoring data using a moving average algorithm, where n represents the sliding window and x represents the sliding window. i Let x represent the i-th monitoring data point after filling in the missing data. i-1 Let x represent the (i-1)th monitoring data point after filling in the missing data. i-n+1 This represents the (i-n+1)th monitoring data point after filling in the gaps.

[0120] 2) Wavelet denoising;

[0121] When the standard deviation is higher than the threshold, wavelet transform is used to decompose the data into different frequency sub-bands. After removing high-frequency noise through threshold processing, inverse transform is performed to reconstruct smooth data.

[0122] (5) Data quality assessment;

[0123] To comprehensively assess data quality, this invention proposes a multi-dimensional and quantifiable data quality assessment system, including integrity indicators, consistency indicators, accuracy indicators, volatility indicators, and anomaly rate indicators.

[0124] 1) Completeness metrics; Missing rate = (Number of missing data points / Total number of data points) × 100%; Duration of consecutive missing data points: Records the length of time that data points are consecutively missing.

[0125] 2) Consistency index; Multi-source data consistency rate = (Number of consistent data points from multiple sources / Total number of data points) × 100%;

[0126] 3) Accuracy index; Difference rate from predicted value = |actual value - predicted value| / predicted value × 100%;

[0127] 4) Volatility indicators; Standard deviation: reflects the degree to which data deviates from the mean; Coefficient of variation = Standard deviation / Mean;

[0128] 5) Abnormality rate indicator; Abnormality rate = (Number of abnormal data points / Total number of data points) × 100%;

[0129] This invention assigns weights to various indicators, such as: completeness 0.2, consistency 0.2, accuracy 0.3, volatility 0.1, and anomaly rate 0.2. The total score is calculated as follows:

[0130] Total score = Completeness score × 0.2 + Consistency score × 0.2 + Accuracy score × 0.3 + Volatility score × 0.1 + Abnormality rate score × 0.2;

[0131] Based on the overall score, data quality is categorized as follows: High: Overall score ≥ 80; Medium: 60 ≤ Overall score < 80; Low: Overall score < 60. This invention, through the aforementioned multi-dimensional evaluation system, achieves quantitative and graded assessment of the quality of distribution transformer monitoring data, providing a refined and reliable data foundation for intelligent power grid analysis and operation and maintenance decisions.

[0132] The following is a supplementary explanation with reference to an example. A distribution transformer in a certain urban residential area is equipped with various monitoring devices, but the sampling times of these devices are inconsistent, and the data contains gaps and noise. Applying the method of this invention, multi-source data alignment is first achieved through time window sliding and interpolation, and then multiple methods, including univariate detection, multivariate detection, and physical constraint detection, are used collaboratively to identify abnormal data.

[0133] For missing data, the system distinguishes between short-term missing data, using interpolation, and long-term missing data, using multivariate regression for imputation. It also smooths fluctuating data using moving averages or wavelet denoising. Combining scores from multiple indicators such as completeness, consistency, and accuracy, the system grades and evaluates data quality, providing reliable data support for operations and maintenance. Figure 10-11 As shown, after implementation, the measurement data quality score reached 86 points, which was judged as high quality, effectively improving the accuracy of intelligent operation and maintenance.

[0134] Figure 2 An embodiment of a transformer multi-source data quality improvement system of the present invention is shown.

[0135] In this optional embodiment, a transformer multi-source data quality improvement system includes:

[0136] The data acquisition and alignment module 201 is used to collect monitoring data of the distribution transformer in real time through various monitoring devices deployed on the distribution transformer, form multi-source data, and perform data alignment processing on the multi-source data through a multi-source data alignment mechanism to obtain time-aligned monitoring data.

[0137] The data quality improvement module 202 is used to perform data quality processing on time-aligned monitoring data by using a multi-source collaborative anomaly detection method and combining it with a missing duration difference judgment mechanism to obtain quality-improved data.

[0138] The data segmentation module 203 is used to evaluate the quality of the data to be improved based on a preset data quality assessment system, obtain the data quality assessment results, and classify the data quality of the data to be improved based on the data quality assessment results, thereby obtaining the data quality segmentation results.

[0139] In this optional embodiment, the step of collecting monitoring data of the distribution transformer in real time through multiple types of monitoring devices deployed on the distribution transformer to form multi-source data, and performing data alignment processing on the multi-source data through a multi-source data alignment mechanism to obtain time-aligned monitoring data includes: integrating and processing the monitoring data of the distribution transformer collected in real time by multiple types of monitoring devices to obtain multi-source data; scanning the data source time information of the multi-source data based on a preset scanning cycle to obtain the time information of each data source; determining the sampling interval difference of each data source in the multi-source data according to the time information of each data source; and based on the sampling interval difference, using a time window sliding and interpolation fusion method to perform time alignment processing on the monitoring data of each data source to obtain time-aligned monitoring data.

[0140] In this optional embodiment, the step of using a multi-source collaborative anomaly detection method, combined with a missing duration difference judgment mechanism, to perform data quality processing on time-aligned monitoring data to obtain quality-improved data includes: using the multi-source collaborative anomaly detection method to detect anomalies in time-aligned monitoring data, identifying anomaly data points in the time-aligned monitoring data, and removing anomaly data points; performing missing value detection on the monitoring data after anomaly point removal and determining the missing duration; comparing the missing duration with a preset missing duration threshold, if the missing duration is greater than the preset missing duration threshold, then using a long-term missing value imputation method to imput the missing values ​​to obtain imputed monitoring data; otherwise, using a short-term missing value imputation method to imput the missing values ​​to obtain imputed monitoring data; performing volatility monitoring on the imputed monitoring data, and based on the detection results, performing data smoothing processing on the imputed monitoring data to obtain quality-improved data.

[0141] In one embodiment, a computer device is provided, which may be a server, and its internal structure diagram may be as follows: Figure 3 As shown, the computer device includes a processor, memory, and a network interface connected via a system bus. The processor provides computing and control capabilities. The memory includes a non-volatile storage medium and internal memory. The non-volatile storage medium stores an operating system, computer programs, and a database. The internal memory provides an environment for the operation of the operating system and computer programs in the non-volatile storage medium. The database stores static and dynamic information data. The network interface communicates with external terminals via a network connection. When the computer program is executed by the processor, it implements the steps in the above method embodiments.

[0142] Those skilled in the art will understand that Figure 3The structure shown is merely a block diagram of a portion of the structure related to the present invention and does not constitute a limitation on the computer device to which the present invention is applied. A specific computer device may include more or fewer components than those shown in the figure, or combine certain components, or have different component arrangements.

[0143] In addition, the present invention also provides a computer device, including a memory and a processor, wherein the memory stores a computer program, and the processor executes the computer program to implement the steps in the above method embodiments.

[0144] In addition, the present invention also provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the steps in the above method embodiments.

[0145] Those skilled in the art will understand that all or part of the processes in the methods of the above embodiments can be implemented by a computer program instructing related hardware. The computer program can be stored in a non-volatile computer-readable storage medium, and when executed, it can include the processes of the embodiments of the methods described above. Any references to memory, storage, databases, or other media used in the embodiments provided by this invention can include at least one of non-volatile and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, or optical storage, etc. Volatile memory can include random access memory (RAM) or external cache memory. By way of illustration and not limitation, RAM can be in various forms, such as static random access memory (SRAM) or dynamic random access memory (DRAM), etc.

[0146] This invention is not limited to the structures described above and shown in the accompanying drawings, and various modifications and changes can be made without departing from its scope. The scope of this invention is limited only by the appended claims.

Claims

1. A transformer multi-source data quality improvement method, characterized in that, The method comprises the following steps: Real-time collection of monitoring data of the power distribution transformer by a plurality of monitoring devices deployed on the power distribution transformer to form multi-source data, and data alignment processing of the multi-source data by a multi-source data alignment mechanism to obtain time-aligned monitoring data; Data quality processing of the time-aligned monitoring data by a multi-source collaborative anomaly detection method combined with a missing time length difference judgment mechanism to obtain quality-improved data; Data quality assessment of the quality-improved data based on a preset data quality assessment system to obtain a data quality assessment result, and data quality level division of the quality-improved data according to the data quality assessment result to obtain a data quality division result.

2. The transformer multi-source data quality improvement method of claim 1, wherein, The real-time collection of monitoring data of the power distribution transformer by a plurality of monitoring devices deployed on the power distribution transformer to form multi-source data, and data alignment processing of the multi-source data by a multi-source data alignment mechanism to obtain time-aligned monitoring data comprises the following steps: Obtaining multi-source data by integrating the real-time collection of monitoring data of the power distribution transformer by a plurality of monitoring devices; Scanning of data source time information of the multi-source data based on a preset scanning period to obtain time information of each data source; Determining sampling interval differences of each data source in the multi-source data according to the time information of each data source, and time alignment processing of monitoring data of each data source by a time window sliding and interpolation fusion method based on the sampling interval differences to obtain time-aligned monitoring data.

3. The transformer multi-source data quality improvement method of claim 1, wherein, The data quality processing of the time-aligned monitoring data by a multi-source collaborative anomaly detection method combined with a missing time length difference judgment mechanism to obtain quality-improved data comprises the following steps: Anomaly data detection of the time-aligned monitoring data by a multi-source collaborative anomaly detection method to identify anomaly data points of the time-aligned monitoring data and perform anomaly data point elimination processing; Missing value detection of the monitoring data after the anomaly data point elimination processing and determination of a missing time length; Judgment of the missing time length and a preset missing time length threshold value, if the missing time length is greater than the preset missing time length threshold value, long missing value filling processing is adopted to fill the missing values to obtain filled monitoring data, otherwise, short missing value filling processing is adopted to fill the missing values to obtain filled monitoring data; Fluctuation monitoring of the filled monitoring data, and data smoothing processing of the filled monitoring data according to the detection result to obtain quality-improved data.

4. The transformer multi-source data quality improvement method of claim 2, wherein, The multi-source collaborative anomaly detection method comprises single variable detection, multi-variable collaborative detection and physical constraint detection; The single variable detection is used for anomaly data detection of the time-aligned monitoring data by a box plot method; The multi-variable collaborative detection is used for anomaly data detection of the time-aligned monitoring data by an integrated learning multi-variable detection algorithm; The physical constraint detection is used for anomaly detection of the time-aligned monitoring data by a physical law of the power system.

5. The transformer multi-source data quality improvement method of claim 3, wherein, The short missing value filling processing adopts an interpolation method to fill the missing values.

6. The transformer multi-source data quality improvement method of claim 3, wherein, The long-time missing value filling method adopts a multivariate regression model to fill in the missing values, and the multivariate regression model adopts monitoring data as independent variables and missing data as dependent variables to obtain.

7. The transformer multi-source data quality improvement method of claim 3, wherein, The filled monitoring data is subjected to volatility monitoring, and the filled monitoring data is subjected to data smoothing processing according to the detection result to obtain quality improvement data, which includes: The standard deviation of the filled monitoring data is calculated, and the standard deviation is compared with a preset standard deviation threshold to determine; If the standard deviation is less than the preset standard deviation threshold, the filled monitoring data is subjected to data smoothing processing using a sliding average algorithm to obtain quality improvement data; If the standard deviation is greater than or equal to the preset standard deviation threshold, the filled monitoring data is subjected to data smoothing processing using a wavelet denoising method to obtain quality improvement data.

8. The transformer multi-source data quality improvement method of claim 7, wherein, The calculation formula for the filled monitoring data subjected to data smoothing processing using the sliding average algorithm is: In the formula, represents the quality improvement data obtained by performing data smoothing on the filled monitoring data using a sliding average algorithm, n represents a sliding window, x i represents the i-th filled monitoring data point, x i-1 represents the i-1-th filled monitoring data point, x i-n+1 represents the i-n+1-th filled monitoring data point.

9. The transformer multi-source data quality improvement method of claim 7, wherein, The filled monitoring data subjected to data smoothing processing using the wavelet denoising method includes: using wavelet transform to decompose the filled monitoring data into different frequency subbands, removing high-frequency noise through threshold processing, and then performing inverse transform to reconstruct the smoothed data to obtain quality improvement data.

10. The transformer multi-source data quality improvement method of claim 1, wherein, Based on the preset data quality evaluation system, the quality improvement data is subjected to data quality evaluation to obtain data quality evaluation results, and the quality improvement data is subjected to data quality level division according to the data quality evaluation results to obtain data quality division results, which include the following steps: Based on the preset data quality evaluation system, multi-dimensional evaluation indexes are calculated according to the quality improvement data; Based on the preset index weight, the total score of the quality improvement data is calculated in combination with the multi-dimensional evaluation indexes; According to the total score of the quality improvement data, the quality improvement data is divided into corresponding data quality levels to obtain data quality division results.

11. The transformer multi-source data quality improvement method of claim 10, wherein, The preset data quality evaluation system includes: integrity index, consistency index, accuracy index, volatility index and abnormal rate index.

12. A transformer multi-source data quality improvement system, characterized in that, It includes: A data acquisition alignment module is configured to acquire monitoring data of a power distribution transformer in real time through a plurality of monitoring devices deployed on the power distribution transformer, form multi-source data, and perform data alignment processing on the multi-source data through a multi-source data alignment mechanism to obtain time-aligned monitoring data. A data quality improvement module is configured to use a multi-source collaborative anomaly detection method and combine a missing time difference judgment mechanism to process the time-aligned monitoring data to obtain quality improvement data. A data division module is configured to evaluate the quality improvement data based on a preset data quality evaluation system to obtain data quality evaluation results, and divide the quality improvement data into data quality levels according to the data quality evaluation results to obtain data quality division results.

13. The transformer multi-source data quality elevation system of claim 12, wherein, The multi-source data is obtained by integrating the monitoring data of the power distribution transformer collected in real time by the plurality of monitoring devices. ​ The time information of each data source is obtained by scanning the multi-source data based on a preset scanning period; According to the time information of each data source, the sampling interval difference of each data source in the multi-source data is determined, and the monitoring data of each data source is time-aligned by using a time window sliding and interpolation fusion method based on the sampling interval difference, to obtain time-aligned monitoring data.

14. The transformer multi-source data quality elevation system of claim 13, wherein, The time-aligned monitoring data is processed to obtain quality-improved data by using the multi-source collaborative anomaly detection method and combining a missing time length difference judgment mechanism, including: The time-aligned monitoring data is detected for abnormal data by using the multi-source collaborative anomaly detection method, to identify abnormal data points of the time-aligned monitoring data and perform abnormal data point elimination processing; The monitoring data after the abnormal data point elimination processing is detected for missing values, and the missing time length is determined; The missing time length is compared with a preset missing time length threshold value, if the missing time length is greater than the preset missing time length threshold value, a long-time missing value filling method is used to fill the missing values, to obtain filled monitoring data, otherwise, a short-time missing value filling method is used to fill the missing values, to obtain filled monitoring data; The filled monitoring data is monitored for volatility, and the filled monitoring data is smoothed according to the detection result, to obtain quality-improved data.

15. A computer device comprising a memory and a processor, the memory storing a computer program, characterized in that, The processor executes the computer program to implement the steps of the method in any one of claims 1 to 11.

16. A computer readable storage medium having stored thereon a computer program, characterized in that, The computer program is executed by the processor to implement the steps of the method in any one of claims 1 to 11.

Images