Data processing method and system based on electric power big data platform

Abstract

The invention relates to the technical field of big data, and discloses a data processing method based on an electric power big data platform, which comprises the following steps: acquiring electric power data, and carrying out missing value processing and noise data processing on the electric power data; according to the processed electric power data, calculating to obtain an electric power data index, and assigning an electric power data index weight by using an electric power data index weighting method; detecting the weighted power data indexes by using an abnormal power utilization detection method based on machine learning; clustering the power data index data by using an improved K-means algorithm, and taking a clustered power data index vector as a power data feature vector; and taking the electric power data feature vector as model input, and performing fault diagnosis of the electric power system on the electric power data feature vector by using an improved LSTM network model. The invention further provides a data processing system based on the electric power big data platform. According to the invention, power data processing is realized.

Description

technical field [0001] The invention relates to the technical field of big data, in particular to a data processing method and system based on an electric power big data platform. Background technique [0002] With the advent of the intelligent age, the amount of data in various industries is increasing rapidly. At the same time, the scale of power grid construction is gradually expanding and the speed of construction is accelerating, making the amount of power data more and more large. How to quickly process large-scale power Data is a hot topic of current research. [0003] Most clustering algorithms for traditional power data choose the K-means algorithm. The K value in the K-means algorithm is randomly selected, which will have a great impact on the classification results; Different classification results are obtained, and finally the most suitable K value is determined by comparison. However, such a method is not suitable for power data. The amount of power load genera...

AI Technical Summary

Problems solved by technology

[0003] Most clustering algorithms for traditional power data choose the K-means algorithm. The K value in the K-means algorithm is randomly selected, which will have a great impact on the classification results; Different classification results are obtained, and finally the most suitable K value is determined by comparison. However, such a method is not suitable for power data. The amount of power load generated every day is huge. The K value is selected by comparison, so that Data processing efficiency will be very low

[0004] At the same time, when the traditional RNN model processes long-term power data sequences, there will be a gradient disappearance problem, that is, as the information is transmitted in the time dimension, the perception of the neurons behind the neurons in the front will decrease, and the information will gradually be lost. , thereby reducing the accuracy of power system fault diagnosis based on power data

Images