A method, system and device for training a multi-layer output neural network

Abstract

This invention provides a training method for a multi-layer output neural network, comprising: S1, dividing the neural network into N sub-networks connected sequentially, where N is a positive integer, and each sub-network having an output layer; S2, obtaining the gradient of the corresponding sub-network through the output layer of each sub-network and updating the parameters of the corresponding sub-network based on the obtained gradient value, wherein the iterative training of the N sub-networks is independent of each other until all N sub-networks converge; S3, stopping the output layer of the i-th sub-network (1≤i≤N-1) sequentially according to the connection order of the sub-networks; S4, after the output layer of the i-th sub-network stops working, obtaining the gradient of the overall network composed of the first i+1 sub-networks using the output layer of the (i+1)-th sub-network, and updating the parameters of the first i+1 sub-networks based on the obtained gradient value; S5, repeating step S4 to iteratively train the overall network until the overall network converges; this improves the efficiency of updating the parameters of the first layer of the neural network.

Images