Dynamic mixing precision model construction method and system

A construction method and dynamic mixing technology, applied in the field of deep neural network, can solve problems such as limited convertible states, application of difficult bit scenarios, restrictions on the promotion and use of adaptive quantization models, etc.

Inactive Publication Date: 2021-07-06
SOUTH CHINA UNIV OF TECH
View PDF0 Cites 1 Cited by
  • Summary
  • Abstract
  • Description
  • Claims
  • Application Information

AI Technical Summary

Problems solved by technology

[0008] However, the current adaptive quantization model has problems that are difficult to apply in low-bit scenarios, and

Method used

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
View more

Image

Smart Image Click on the blue labels to locate them in the text.
Viewing Examples
Smart Image
  • Dynamic mixing precision model construction method and system
  • Dynamic mixing precision model construction method and system
  • Dynamic mixing precision model construction method and system

Examples

Experimental program
Comparison scheme
Effect test

Embodiment Construction

[0042] Such as Figure 1-4 As shown, the dynamic mixed precision model construction method described in the present invention is specifically as follows:

[0043] After preprocessing the original data such as zero padding, random cropping, and random flipping, 8-bit quantization is performed to obtain the input data.

[0044] Use cross entropy as loss function and SGD as optimizer.

[0045] To train a full-precision model, first set the parameter matrix w i Zoom to [-1,+1] interval, the formula is Then perform forward propagation, and update the parameter w during backpropagation i . In this embodiment, weights are used as the parameters, and the parameter matrix is ​​the weight matrix.

[0046]Select part of the training data, and calculate the approximate value of the trace of the Hessian matrix of each block parameter of the full-precision model, specifically, including the following sub-steps:

[0047] (1) Randomly sample 2000 pieces of training data, and input th...

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to view more

PUM

No PUM Login to view more

Abstract

The invention discloses a dynamic mixing precision model construction method and system, and relates to a deep neural network technology. Aiming at the convertible state number and precision problems in the prior art, the scheme is provided, and the method is characterized by constructing a mixed precision state conversion table according to an average trace of a Hessian matrix of parameters of different blocks in a full-precision model and an optional parameter precision table S; randomly sampling a plurality of mixed precision sub-models by adopting each training iteration in the training process, and performing quantization operation by using an improved quantization function to obtain a mixed precision model; and forming a mixed bit deployment state table according to the actual deployment requirements to carry out actual deployment. The method has the advantages that the search space and the calculation amount required by training are reduced by utilizing the Hessian matrix average trace and the random sampling, and meanwhile, the improved quantization function can be directly migrated among different quantization bits, and the quantization error is small, so that the mixed precision model can be adaptively deployed in a lower bit scene; and meanwhile, the precision in a higher bit scene is also improved.

Description

technical field [0001] The present invention relates to deep neural network technology, in particular to a dynamic mixed precision model construction method and system. Background technique [0002] In recent years, more and more smart devices have deployed deep neural models, such as mobile phones and VR glasses. Deep neural models usually require large memory footprint and expensive computing operations, and smart devices have limited memory and computing resources. Considering more complex practical application scenarios, even for the same device, the requirements for deep neural models will vary due to changes in battery conditions, hardware aging and other factors. This brings greater difficulty to the deployment of smart devices, which is not conducive to the lightweight application of neural network algorithms. [0003] In order to solve the high memory usage and computing requirements of deep neural networks, researchers proposed a method for quantizing and compres...

Claims

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to view more

Application Information

Patent Timeline
no application Login to view more
IPC IPC(8): G06F30/20
CPCG06F30/20
Inventor 郭锴凌杨弈才徐向民邢晓芬
Owner SOUTH CHINA UNIV OF TECH
Who we serve
  • R&D Engineer
  • R&D Manager
  • IP Professional
Why Eureka
  • Industry Leading Data Capabilities
  • Powerful AI technology
  • Patent DNA Extraction
Social media
Try Eureka
PatSnap group products

Browse by: Latest US Patents, China's latest patents, Technical Efficacy Thesaurus, Application Domain, Technology Topic.

© 2024 PatSnap. All rights reserved.Legal|Privacy policy|Modern Slavery Act Transparency Statement|Sitemap