Looking for breakthrough ideas for innovation challenges? Try Patsnap Eureka!

Tensor processing using low precision format

A floating-point format, low-threshold technology, used in physical implementation, complex mathematical operations, biological neural network models, etc., and can solve problems such as accuracy loss

Pending Publication Date: 2017-12-29
NVIDIA CORP
View PDF5 Cites 7 Cited by
  • Summary
  • Abstract
  • Description
  • Claims
  • Application Information

AI Technical Summary

Problems solved by technology

Both overflow and underflow also cause a loss of accuracy during the weight update step of the second phase of the training cycle

Method used

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
View more

Image

Smart Image Click on the blue labels to locate them in the text.
Viewing Examples
Smart Image
  • Tensor processing using low precision format
  • Tensor processing using low precision format
  • Tensor processing using low precision format

Examples

Experimental program
Comparison scheme
Effect test

Embodiment Construction

[0018] Reference will now be made in detail to the preferred embodiments of the invention. While the invention will be described in conjunction with the preferred embodiments, it will be understood that they are not intended to be limited to these embodiments. On the contrary, the invention is intended to cover alternatives, modifications and equivalents, which may be included within the spirit and scope as defined by the appended claims.

[0019] Furthermore, in the following detailed description of embodiments of the invention, numerous specific details are set forth in order to provide a thorough understanding of the invention. However, one skilled in the art will recognize that the present invention may be practiced without these specific details. In other instances, well-known methods, procedures, components, and circuits have not been described in detail so as not to unnecessarily obscure aspects of the invention.

[0020] Some portions of the following detailed descri...

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to View More

PUM

No PUM Login to View More

Abstract

The invention relates to tensor processing using low precision format. Aspects of the present invention are directed to computer-implemented techniques for improving the training of artificial neural networks using a reduced precision (e.g., float16) data format. Embodiments of the present invention rescale tensor values prior to performing matrix operations (such as matrix multiplication or matrix addition) to prevent overflow and underflow. To preserve accuracy throughout the performance of the matrix operations, the scale factors are defined using a novel data format to represent tensors, wherein a matrix is represented by the tuple X, where X=(a, v[.]), wherein a is a float scale factor and v[.] are scaled values stored in the float16 format. The value of any element X[i] according to this data format would be equal to a*v[i].

Description

technical field [0001] Embodiments of the invention generally relate to computer-implemented techniques for machine learning. More specifically, embodiments of the present invention relate to a framework for improving the training of neural networks and deep learning of convolutional networks. Background technique [0002] Machine learning is the field of computer science that involves the use of computer-implemented algorithms for problem solving through pattern recognition and adaptive processing of data sets. In contrast to conventional "static" programming, machine learning applications are characterized by the ability to generate predictive data models by iteratively refining the model from a dataset without explicit programming. Artificial neural networks are one of the most pervasive machine learning algorithms, and use distributed parallel processors to process multiple interconnected " A "neuron" (processing unit) performs parametric computations on input data to ...

Claims

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to View More

Application Information

Patent Timeline
no application Login to View More
Patent Type & Authority Applications(China)
IPC IPC(8): G06F17/16G06N3/04G06N3/063G06N3/08
CPCG06F17/16G06N3/063G06N3/08G06N3/084G06N3/045G06N3/04G06N3/088
Inventor 波里斯·金斯伯格塞奇·尼克拉艾艾哈迈德·基斯瓦尼浩·吴阿米尔·吴拉姆纳贾德斯朗瓦莫·基拉特迈克尔·休斯顿亚历克斯·菲特-弗洛雷亚
Owner NVIDIA CORP
Who we serve
  • R&D Engineer
  • R&D Manager
  • IP Professional
Why Patsnap Eureka
  • Industry Leading Data Capabilities
  • Powerful AI technology
  • Patent DNA Extraction
Social media
Patsnap Eureka Blog
Learn More
PatSnap group products

Browse by: Latest US Patents, China's latest patents, Technical Efficacy Thesaurus, Application Domain, Technology Topic, Popular Technical Reports.

© 2024 PatSnap. All rights reserved.Legal|Privacy policy|Modern Slavery Act Transparency Statement|Sitemap|About US| Contact US: help@patsnap.com