Looking for breakthrough ideas for innovation challenges? Try Patsnap Eureka!

Deep neural network model parallel fully connected layer data exchange method and system

A deep neural network, fully connected layer technology, applied in the field of fully connected layer data exchange, can solve problems such as high communication overhead

Active Publication Date: 2019-09-24
HUAZHONG UNIV OF SCI & TECH
View PDF6 Cites 0 Cited by
  • Summary
  • Abstract
  • Description
  • Claims
  • Application Information

AI Technical Summary

Problems solved by technology

This solves the technical problem of large communication overhead in the prior art

Method used

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
View more

Image

Smart Image Click on the blue labels to locate them in the text.
Viewing Examples
Smart Image
  • Deep neural network model parallel fully connected layer data exchange method and system
  • Deep neural network model parallel fully connected layer data exchange method and system
  • Deep neural network model parallel fully connected layer data exchange method and system

Examples

Experimental program
Comparison scheme
Effect test

Embodiment Construction

[0044] In order to make the object, technical solution and advantages of the present invention more clear, the present invention will be further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described here are only used to explain the present invention, not to limit the present invention. In addition, the technical features involved in the various embodiments of the present invention described below can be combined with each other as long as they do not constitute a conflict with each other.

[0045] The present invention is made up of two parts, respectively is the forward propagation method such as half-stop and the backward propagation method such as fixed stop, wherein, the core idea of ​​the forward propagation method such as half-stop is as follows:

[0046] (1) Partial calculation: the training unit first calculates the output data (Input Data, ID) of the previous layer that...

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to View More

PUM

No PUM Login to View More

Abstract

The invention discloses a deep neural network model parallel fully connected layer data exchange method and system. The fully connected layer of the deep neural network is evenly divided into N training units according to the number of neurons to form a deep neural network. A network model in which the fully connected layer model is parallel; in the forward propagation process of the fully connected layer, the forward propagation method such as half-stop is used to process the input data of the front layer by partial arrival, partial calculation, overall output and overall propagation method; in the backward propagation process of the fully connected layer, the residual data of the rear layer is processed by using the backward propagation method such as stop and stop, and the processing methods of quantitative achievement, quantitative calculation and quantitative propagation are adopted; in a forward and backward propagation After completion, the weight data and threshold data of each layer are updated in parallel according to the obtained weight gradient and threshold gradient. It can overlap the data communication and data calculation of the fully connected layer, and accelerate the convergence of the model under the premise of ensuring the correct rate.

Description

technical field [0001] The invention belongs to the technical field of deep learning, and more specifically relates to a method and system for exchanging fully connected layer data parallel to a model in a deep neural network. Background technique [0002] Deep Neural Network (DNN) is an artificial neural network (Artificial Neural Network, ANN) composed of an input layer, multiple hidden layers, and an output layer. Each layer is composed of multiple neuron nodes. The neuron nodes of the front layer and the back layer are connected to each other, such as figure 1 as shown, figure 1 All layers in are on the same training unit, I represents the input layer, H represents the hidden layer (the hidden layer needs to have more than one), O represents the hidden layer, the thin line represents the connection between neurons and neurons, and the thick line represents the component Connect with a component (here a layer). In the neural network model, a fully-connected layer (Full...

Claims

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to View More

Application Information

Patent Timeline
no application Login to View More
Patent Type & Authority Patents(China)
IPC IPC(8): G06N3/04G06N3/08
CPCG06N3/08G06N3/045
Inventor 蒋文斌金海张杨松叶阁焰马阳祝简刘湃
Owner HUAZHONG UNIV OF SCI & TECH
Who we serve
  • R&D Engineer
  • R&D Manager
  • IP Professional
Why Patsnap Eureka
  • Industry Leading Data Capabilities
  • Powerful AI technology
  • Patent DNA Extraction
Social media
Patsnap Eureka Blog
Learn More
PatSnap group products

Browse by: Latest US Patents, China's latest patents, Technical Efficacy Thesaurus, Application Domain, Technology Topic, Popular Technical Reports.

© 2024 PatSnap. All rights reserved.Legal|Privacy policy|Modern Slavery Act Transparency Statement|Sitemap|About US| Contact US: help@patsnap.com