Method and apparatus for neural network quantization

A neural network and equipment technology, applied in the field of neural network quantification, which can solve problems such as the difficulty of deploying deep neural networks

Pending Publication Date: 2018-04-27
SAMSUNG ELECTRONICS CO LTD
View PDF0 Cites 20 Cited by
  • Summary
  • Abstract
  • Description
  • Claims
  • Application Information

AI Technical Summary

Problems solved by technology

[0007] This makes it difficult to deploy deep neural networks o

Method used

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
View more

Image

Smart Image Click on the blue labels to locate them in the text.
Viewing Examples
Smart Image
  • Method and apparatus for neural network quantization
  • Method and apparatus for neural network quantization
  • Method and apparatus for neural network quantization

Examples

Experimental program
Comparison scheme
Effect test

Embodiment Construction

[0030]Hereinafter, embodiments of the present disclosure will be described in detail with reference to the accompanying drawings. It should be noted that the same elements are denoted by the same reference numerals even though they are shown in different drawings. In the following description, specific details such as detailed configuration and components are merely provided to help a comprehensive understanding of the embodiments of the present disclosure. Accordingly, it should be apparent to those skilled in the art that various changes and modifications of the embodiments described herein can be made without departing from the scope of the present disclosure. Also, descriptions of well-known functions and constructions are omitted for clarity and conciseness. Terms described below are terms defined in consideration of functions in the present disclosure, and may vary according to a user, user's intention, or custom. Therefore, definitions of terms should be determined ba...

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to view more

PUM

No PUM Login to view more

Abstract

The invention provides a method and apparatus for neural network quantization. Apparatuses and methods of manufacturing same, systems, and methods for performing network parameter quantization in deepneural networks are described. In one aspect, diagonals of a second-order partial derivative matrix (a Hessian matrix) of a loss function of network parameters of a neural network are determined andthen used to weight (Hessian-weighting) the network parameters as part of quantizing the network parameters. In another aspect, the neural network is trained using first and second moment estimates ofgradients of the network parameters and then the second moment estimates are used to weight the network parameters as part of quantizing the network parameters. In yet another aspect, network parameter quantization is performed by using an entropy-constrained scalar quantization (ECSQ) iterative algorithm. In yet another aspect, network parameter quantization is performed by quantizing the network parameters of all layers of a deep neural network together at once.

Description

[0001] This application claims priority to U.S. Provisional Patent Application No. 62 / 409,961, filed October 19, 2016, and U.S. Nonprovisional Patent Application No. 15 / 433,531, filed February 15, 2017, the entire contents of which applications Incorporated herein by reference. technical field [0002] The present disclosure relates generally to deep neural networks, and more particularly, to a method and apparatus for neural network quantization. Background technique [0003] Machine learning technology is constantly developing and has begun to support many aspects of modern society, from web search, content filtering, automatic recommendation of commercial websites, automated games, object detection, image classification, speech recognition, machine translation, drug discovery and genomics. The current state-of-the-art in the field of machine learning are deep neural networks, which use computational models consisting of multiple processing layers that learn representation...

Claims

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to view more

Application Information

Patent Timeline
no application Login to view more
IPC IPC(8): G06N3/08G06K9/62
CPCG06N3/08G06N3/084G06F18/23213G06N3/063G06N3/082G06N3/06G06F17/16
Inventor 崔柳真李正元穆斯塔法·艾尔可哈米
Owner SAMSUNG ELECTRONICS CO LTD
Who we serve
  • R&D Engineer
  • R&D Manager
  • IP Professional
Why Eureka
  • Industry Leading Data Capabilities
  • Powerful AI technology
  • Patent DNA Extraction
Social media
Try Eureka
PatSnap group products

Browse by: Latest US Patents, China's latest patents, Technical Efficacy Thesaurus, Application Domain, Technology Topic.

© 2024 PatSnap. All rights reserved.Legal|Privacy policy|Modern Slavery Act Transparency Statement|Sitemap