Data processing method, device and equipment

Abstract

The embodiment of the invention discloses a data processing method, device and equipment, which are used for carrying out dimensionality reduction on high-dimensionality load data of a user so as to solve the problem of dimensionality disasters. The method provided by the embodiment of the invention comprises the following steps: acquiring initial load data of a target user, wherein the initial load data comprises load data of an initial dimension; a target auto-encoder obtained through training of a plurality of training samples is obtained, and the target auto-encoder comprises an encoder composed of N convolution layers and M pooling layers; each training sample comprises historical load data of an initial dimension and historical load data of a target dimension, the initial dimension is greater than the target dimension, and the target auto-encoder stores a dimension reduction rule for reducing the initial dimension to the target dimension; and inputting the initial load data into the target auto-encoder, so that the target auto-encoder reduces the dimension of the load data of the initial dimension into the load data of the target dimension according to the dimension reduction rule.

Description

technical field [0001] The embodiments of the present application relate to the field of machine learning, and in particular, to a data processing method, apparatus, and device thereof. Background technique [0002] Load curve clustering is an important topic and useful tool in many fields of energy research, and it is also a research hotspot in power data mining. [0003] In load prediction, the load curve is normalized before clustering, and a general model that can be used to predict the load curve is established, so as to realize the load prediction. [0004] But due to the curse of dimensionality in load curves, it is difficult to build a general model that can accurately predict load curves under all conditions. Distance-based methods such as K-means clustering exhibit instability in high-dimensional data. While traditional dimensionality reduction methods such as principal component analysis (PCA) can be used to alleviate the curse of dimensionality, these methods c...

AI Technical Summary

Problems solved by technology

While traditional dimensionality reduction methods such as principal component analysis (PCA) can be used to mitigate the curse of dimensionality, these methods can only generate linear maps and cannot solve complex problems

Images