Training system of back propagation neural network DNN (Deep Neural Network)

A neural network and backpropagation technology, applied in biological neural network models, neural learning methods, etc., can solve the problems of hundreds of milliseconds, high data synchronization overhead, etc., to improve training speed, increase concurrency, and reduce transmission overhead Effect

Active Publication Date: 2013-06-12
BAIDU ONLINE NETWORK TECH (BEIJIBG) CO LTD
View PDF2 Cites 32 Cited by
  • Summary
  • Abstract
  • Description
  • Claims
  • Application Information

AI Technical Summary

Problems solved by technology

However, the above method has the following disadvantages: the data synchronization overhead is large
At present, the PCI-E2.0 bus used between multiple GPUs has a one-way transmission bandwidth of 4 to 5GB/S. If 4 GPUs need to synchronize 200MB of weight data respectively, the method of group synchronization (GPU 1-2 synchronization, 3-4 synchron

Method used

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
View more

Image

Smart Image Click on the blue labels to locate them in the text.
Viewing Examples
Smart Image
  • Training system of back propagation neural network DNN (Deep Neural Network)
  • Training system of back propagation neural network DNN (Deep Neural Network)
  • Training system of back propagation neural network DNN (Deep Neural Network)

Examples

Experimental program
Comparison scheme
Effect test

Embodiment Construction

[0026] Embodiments of the present invention are described in detail below, examples of which are shown in the drawings, wherein the same or similar reference numerals designate the same or similar elements or elements having the same or similar functions throughout. The embodiments described below by referring to the figures are exemplary only for explaining the present invention and should not be construed as limiting the present invention.

[0027] In the description of the present invention, the terms "longitudinal", "transverse", "upper", "lower", "front", "rear", "left", "right", "vertical", "horizontal", " The orientations or positional relationships indicated by "top", "bottom", etc. are based on the orientations or positional relationships shown in the drawings, and are only for the convenience of describing the present invention and do not require that the present invention must be constructed and operated in a specific orientation, so they cannot be understood as Lim...

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to view more

PUM

No PUM Login to view more

Abstract

The invention provides a training system of back propagation neural network DNN (Deep Neural Network). The training system of the back propagation neural network DNN comprises a first graphics processor assembly, a second graphics processor assembly and a controller assembly, wherein the first graphics processor assembly is used for performing DNN forward calculation and weight update calculation; the second graphics processor assembly is used for perform DNN forward calculation and DNN back calculation; the controller assembly is used for controlling the first graphics processor assembly and the second graphics processor assembly to perform Nth-layer DNN forward calculation respectively according to respective input data, and after the completion of the forward calculation, controlling the first graphics processor assembly to perform the weight update calculation and controlling the second graphics processor assembly to perform the DNN back calculation; and N is a positive integer. The training system provided by the invention has the advantages of high training speed and low data transmission cost, so that the training speed of the back propagation neural network DNN is promoted.

Description

technical field [0001] The invention relates to the technical field of backpropagation neural network, in particular to a training system of backpropagation neural network DNN. Background technique [0002] Backpropagation neural network (DNN) is widely used in many important Internet applications, such as speech recognition, image recognition, natural language processing, etc. It can greatly improve the accuracy of speech recognition, so it is widely used in speech recognition products of many companies. [0003] DNN training calculation has two characteristics: high computational complexity and incapable of large-scale parallelism. The calculation of DNN training is mainly the multiplication of floating-point matrix, and the calculation complexity is O(N3), which is a typical calculation-intensive type. Due to the limitations of the algorithm, the DNN training calculation cannot be parallelized on a large scale, so the traditional multi-server method cannot be used to im...

Claims

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to view more

Application Information

Patent Timeline
no application Login to view more
IPC IPC(8): G06N3/08
Inventor 欧阳剑
Owner BAIDU ONLINE NETWORK TECH (BEIJIBG) CO LTD
Who we serve
  • R&D Engineer
  • R&D Manager
  • IP Professional
Why Eureka
  • Industry Leading Data Capabilities
  • Powerful AI technology
  • Patent DNA Extraction
Social media
Try Eureka
PatSnap group products

Browse by: Latest US Patents, China's latest patents, Technical Efficacy Thesaurus, Application Domain, Technology Topic.

© 2024 PatSnap. All rights reserved.Legal|Privacy policy|Modern Slavery Act Transparency Statement|Sitemap