BP neural-network classification method based on Hadoop

Abstract

The invention discloses a BP neural-network classification method based on a Hadoop. The BP neural-network classification method based on the Hadoop comprises the following steps that data are preprocessed, Map tasks are started at Mapper ends of all nodes on a Hadoop platform, a training datum is obtained at each Mapper end and the training data serve as weight calculation modification values of the current network and the modification valves are transmitted to Reducer ends; Reduce tasks are started on the Reducer ends of all nodes on the Hadoop platform, all modification values of one weight are obtained by each Reducer end and the average value of the modification values is calculated out and serves as the output; the manner of batch training is adopted and the weight values of all layers are modified; the steps are repeated until the error reaches the preset precision or the frequency of study is larger than the preset maximum frequency and a BP neural-network model is obtained. Otherwise, iteration continues. Parallel computing can be achieved according to the BP neural-network classification method based on the Hadoop.

Description

technical field [0001] The present invention relates to a Hadoop-based BP neural network (referred to as "BP network") classification method. Background technique [0002] The BP (Back Propagation) network was proposed by a team of scientists headed by Rumelhart and McCelland in 1986. It is a multi-layer feed-forward network trained by the error back propagation algorithm and is one of the most widely used neural network models. The BP network can learn and store a large number of input-output pattern mapping relationships without revealing the mathematical equations describing the mapping relationship in advance. Its learning rule is to use the steepest descent method to continuously adjust the weights and thresholds of the network through backpropagation to minimize the sum of squared errors of the network. The topological structure of BP neural network model includes input layer, hidden layer and output layer. Error Backpropagation The learning process of the error back...

AI Technical Summary

Problems solved by technology

[0003] The traditional BP neural network training method is to serially process data sets on a single machine, but with the rapid development of the information society, the amount of data that needs to be mined has increased sharply, reaching the level of massive data, so the traditional BP neural network The neural network training method will have big problems when dealing with massive data sets, such as taking a very long time, or even insufficient memory to train, etc.

Images