Deep neural network parameter optimization method and system based on distributed estimation algorithm

Abstract

The invention discloses a deep neural network parameter optimization method and system based on a distributed estimation algorithm, and the method comprises the steps: initializing a population, and decoding each individual in the population into a deep neural network; evaluating the classification capability of the deep neural network; sorting the advantages of the individuals in the population;randomly generating a mask vector, and determining a probability distribution model of the current iteration of the to-be-solved variable according to the mask vector and the statistical parameters; performing sampling according to the mask vector and the probability distribution model to generate a new population individual; obtaining an optimal distributed estimation algorithm individual; and utilizing a gradient optimization algorithm to finely adjust the deep neural network model obtained by the distributed estimation algorithm to obtain optimal deep neural network parameters. According tothe method, the distributed estimation algorithm is combined with the optimization of the deep neural network, and the dependence on gradient information in the neural network optimization process isreduced by utilizing the global search capability of the distributed estimation algorithm.

Description

technical field [0001] The invention belongs to the technical field of information processing, and in particular relates to a method and system for optimizing parameters of a deep neural network based on a distributed estimation algorithm. Background technique [0002] The statements in this section merely provide background information related to the present invention and do not necessarily constitute prior art. [0003] In recent years, deep neural network, as an important branch of artificial intelligence, has achieved rapid development. Deep neural network parameter optimization is a large-scale parameter optimization problem. Neural networks rely on training to correct connection parameters. The current deep neural network parameter optimization method is mainly based on the backpropagation algorithm of gradient information. The gradient algorithm has certain limitations and is easy to fall into Problems such as local minima, vanishing or exploding gradients. [0004]...

AI Technical Summary

Problems solved by technology

[0005] In order to solve the above problems, the present invention proposes a deep neural network parameter optimization method and system based on a distributed estimation algorithm, which can overcome the unsatisfactory shortcomings of the gradient-based neural network optimization method and the traditional distributed estimation algorithm in dealing with large-scale parameter optimization problems

Images