High-dimensional space abnormal data optimization identification method

Abstract

The invention discloses a high-dimensional space abnormal data optimization identification method comprising the steps: firstly, obtaining industrial process multi-dimensional variable operation data,acquiring and storing the industrial process multi-dimensional variable operation data, then performing mechanism elimination, and then performing first clustering analysis on the multi-dimensional variable operation data through a high-dimensional space clustering analysis method to obtain a plurality of groups of first data classes Ci; then, carrying out second clustering analysis on all groupsof first data classes Ci to obtain a plurality of groups of second data classes Cij; then, in a Copula high-dimensional probability space, establishing joint probability distribution formed by the multi-dimensional variables, and in each dimension, evenly dividing the value range of the joint probability distribution of the multi-dimensional variables into a plurality of probability intervals; and finally, based on a multidirectional quartile algorithm and an intelligent or numeric type optimization algorithm, realizing abnormal data optimization identification of the operation data sample ineach probability interval.

Description

technical field [0001] The invention relates to a data preprocessing method, in particular to a high-dimensional space abnormal data optimization identification method and device. Background technique [0002] With the rapid development of the Industrial Internet of Things, massive industrial process operation data can be collected and stored. Taking the wind power industry as an example, due to the widespread existence of wind curtailment, environmental interference, measurement noise, and transmission and storage errors, a large amount of abnormal data has been generated in the records of the wind turbine data acquisition and monitoring (SCADA) system, mainly including accumulation and Sparse abnormal data. The amount of these abnormal data is huge and it is easy to have a negative impact on applications such as data-driven wind power curve modeling, wind power theoretical power calculation, wind turbine operating performance evaluation, operating status, and fault diagno...

AI Technical Summary

Problems solved by technology

At present, the clustering method is a feasible solution to realize the identification of a large amount of accumulated abnormal data. However, due to the large number of irrelevant attributes and sparse distribution of the data in the high-dimensional space, the data identification of multivariate data in the high-dimensional space is often difficult. difficult to achieve good results

In addition, there are few effective algorithms for the identification of sparse abnormal data, especially in high-dimensional space, it is more difficult to realize the effective identification and accuracy evaluation of sparse abnormal data

Images