Very low precision floating point representation for deep learning acceleration

A low-precision, special-purpose circuit technique used to optimize the computation involved in training neural networks

Pending Publication Date: 2019-11-15
IBM CORP
View PDF10 Cites 0 Cited by
  • Summary
  • Abstract
  • Description
  • Claims
  • Application Information

AI Technical Summary

Problems solved by technology

Also, different floating-point calculations in DNN training may have to be accurate to different numbers of decimal places

Method used

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
View more

Image

Smart Image Click on the blue labels to locate them in the text.
Viewing Examples
Smart Image
  • Very low precision floating point representation for deep learning acceleration
  • Very low precision floating point representation for deep learning acceleration
  • Very low precision floating point representation for deep learning acceleration

Examples

Experimental program
Comparison scheme
Effect test

Embodiment Construction

[0019] FPU has a bit width. In terms of the number of binary bits used to represent numbers in a floating-point format (hereinafter referred to as "format" or "floating-point format"), the bit width is the size. One or more organizations, such as the Institute of Electrical and Electronics Engineers (IEEE), have created standards related to floating point formats. The format currently in use provides a standard method for representing numbers in 16-bit, 32-bit, 64-bit, and 128-bit formats.

[0020] The illustrative embodiments recognize that in terms of the physical size of the semiconductor manufacturing circuit and the amount of power consumed, the larger the bit width, the more complex and larger the FPU. In addition, the larger the FPU, the more time it takes to generate the calculated output.

[0021] Dedicated computing circuits, especially FPUs, are recognized fields of endeavor technology. The current state of the technology in this field of endeavor has certain shortcom...

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to view more

PUM

No PUM Login to view more

Abstract

The invention relates to very low precision floating point representation for deep learning acceleration. A specialized circuit is configured for floating point computations using numbers representedby a very low precision format (VLP format). The VLP format includes less than sixteen bits and is apportion into a sign bit, exponent bits (e), and mantissa bits (p). The configured specialized circuit is operated to store an approximation of a numeric value in the VLP format, where the approximation is represented as a function of a multiple of a fraction, where the fraction is an inverse of a number of discrete values that can be represented using only the mantissa bits.

Description

Technical field [0001] The present invention generally relates to methods, systems, and computer program products for optimizing the calculations involved in training neural networks. More specifically, the present invention relates to a method, system and computer program product for very low precision floating point representation. Background technique [0002] Because computer memory is limited, it is impossible to store numbers with infinite precision regardless of whether the numbers use binary fractions or decimal fractions. At some point, numbers must be cut off or rounded to represent them in computer memory. [0003] How to represent numbers in memory depends on how accurate the representation is expected. Generally speaking, a single fixed way of representing numbers in binary bits is not suitable for various applications that use those numbers. A physicist may need to use a number representing the speed of light (approximately 300 million) and a number representing th...

Claims

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to view more

Application Information

Patent Timeline
no application Login to view more
IPC IPC(8): G06F9/30G06F7/483
CPCG06F9/3001G06F7/483G06F2207/4824G06N3/08G06N3/063G06F7/49947
Inventor 王乃刚K·高帕拉克里斯南崔正旭S·M·穆尔乐A·安哥拉瓦尔D·布兰德
Owner IBM CORP
Who we serve
  • R&D Engineer
  • R&D Manager
  • IP Professional
Why Eureka
  • Industry Leading Data Capabilities
  • Powerful AI technology
  • Patent DNA Extraction
Social media
Try Eureka
PatSnap group products

Browse by: Latest US Patents, China's latest patents, Technical Efficacy Thesaurus, Application Domain, Technology Topic.

© 2024 PatSnap. All rights reserved.Legal|Privacy policy|Modern Slavery Act Transparency Statement|Sitemap