Local connection communication based deep learning network structure algorithm

Abstract

The invention relates to the technical field of deep learning network structure algorithms, in particular to a deep learning network structure algorithm based on local connection communication; The deep learning network structure algorithm is based on the deep learning network structure algorithm; the adopted technical solution is: comprising the following steps: S101, determining a network connection model between each network layer according to a computing problem; S102, allocating each network layer to each computing node according to a parallel processing method S103, determine the communication relationship between each computing node; S104, keep the communication between adjacent nodes, for the communication across nodes, delete directly; S105, generate new model according to the connection relationship after the deletion operation in step S104; S106, using the new model to perform parallel computing; the present invention is applicable to the field of deep learning algorithms.

Description

technical field [0001] The invention relates to the technical field of deep learning network structure algorithms, in particular to a deep learning network structure algorithm based on local connection communication. Background technique [0002] Deep learning is a new field in machine learning research. Its motivation is to establish and simulate the neural network of human brain for analysis and learning. It imitates the mechanism of human brain to explain data, such as images, sounds and texts. Its concept is proposed by Hinton et al. was proposed in 2006. Based on the deep belief network (DBN), a non-supervised greedy layer-by-layer training algorithm is proposed, which brings hope to solve the optimization problems related to the deep structure, and then a multi-layer autoencoder deep structure is proposed. In addition, the convolutional neural network proposed by Lecun et al. is the first real multi-layer structure learning algorithm, which uses the spatial relative r...

AI Technical Summary

Problems solved by technology

However, the biggest drawback of this connection method is: the amount of calculation and memory consumption are huge, and the efficiency is low

image 3 It is a schematic diagram of model parallelism in deep learning algorithms. Obviously, the denser the connection table between layers, the greater the communication traffic. In extreme cases such as fully connected neural networks, it means all-to-all communication. This characteristic greatly limits the scalability of the algorithm

Images