Configuration method of fault-tolerant mechanism based on complete binary tree

Abstract

The invention discloses a configuration method of a fault-tolerant mechanism based on a complete binary tree. The configuration method comprises the following steps of: setting a manager manger, creating n containers, deploying the n containers on different machines, marking the containers as nodes, equally dividing a data set into n parts, and setting a node control table; sorting the data sets,respectively sending the first n data set subsets into the nodes, and setting the time slice T of a period; building a neural network, obtaining parameters through forward propagation, updating the parameters through a stochastic gradient descent algorithm, building the nodes into the form of a complete binary tree, enabling each tree node to be of the same data structure; establishing communication protocols are on different hosts, enabling child nodes to carry out message transmission and model transmission in different periods respectively, and carrying out safety detection; and combining segmented models into a complete model after confirming safety, taking the complete model as a hidden layer of the current neural network to perform transfer learning training. According to the methodof the technical schemes of the invention, the error-tolerant rate of model training can be improved, and the data processing capacity of the model is improved.

Description

technical field [0001] The invention relates to the field of deep learning, in particular to a configuration method based on a complete binary tree fault-tolerant mechanism. Background technique [0002] A binary tree with a depth of k and n nodes, the nodes in the tree are numbered from top to bottom and from left to right, if the node with the number i (1≤i≤n) and If the node number i in the full binary tree has the same position in the binary tree, then this binary tree is called a complete binary tree. The leaf nodes of a complete binary tree can only appear in the bottom and second bottom layers, and the leaf nodes in the bottom layer are concentrated in the left part of the tree. [0003] A deep neural network usually contains a large number of trainable parameters, so it takes a lot of time to train a neural network with good performance. On the other hand, in order to learn more valuable features from massive amounts of data, the layers of deep neural networks are ...

AI Technical Summary

Problems solved by technology

[0003] Deep neural networks usually contain a large number of trainable parameters, so it takes a lot of time to train a neural network with good performance

On the other hand, in order to learn more valuable features from massive data, the level of deep neural network is deepening, further increasing the time-consuming training

The traditional distributed architecture model parallel and data parallel training methods do not implement model parallelism, which will lead to insufficient utilization of GPU clusters, insufficient loose coupling between nodes, and insufficient ability to handle large data

Images