Looking for breakthrough ideas for innovation challenges? Try Patsnap Eureka!

Method, system and device for processing int4 data type based on Tensorcore and medium

A data type, data technology, applied in the field of neural network, can solve the problem of no way to deal with it, and achieve the effect of large performance improvement

Inactive Publication Date: 2021-01-15
SUZHOU LANGCHAO INTELLIGENT TECH CO LTD
View PDF0 Cites 1 Cited by
  • Summary
  • Abstract
  • Description
  • Claims
  • Application Information

AI Technical Summary

Problems solved by technology

The current fully connected layer can only support the data type of data and weight as int8, and there is no way to process data with the data type of int4

Method used

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
View more

Image

Smart Image Click on the blue labels to locate them in the text.
Viewing Examples
Smart Image
  • Method, system and device for processing int4 data type based on Tensorcore and medium
  • Method, system and device for processing int4 data type based on Tensorcore and medium
  • Method, system and device for processing int4 data type based on Tensorcore and medium

Examples

Experimental program
Comparison scheme
Effect test

Embodiment Construction

[0025] In order to make the objectives, technical solutions and advantages of the present invention more clearly understood, the embodiments of the present invention will be further described in detail below with reference to the specific embodiments and the accompanying drawings.

[0026] It should be noted that all expressions using "first" and "second" in the embodiments of the present invention are for the purpose of distinguishing two entities with the same name but not the same or non-identical parameters. It can be seen that "first" and "second" It is only for the convenience of expression and should not be construed as a limitation to the embodiments of the present invention, and subsequent embodiments will not describe them one by one.

[0027] Based on the above purpose, in the first aspect of the embodiments of the present invention, an embodiment of a method for processing an int4 data type based on Tensorcore is proposed. figure 1 Shown is a schematic diagram of a...

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to View More

PUM

No PUM Login to View More

Abstract

The invention discloses a method, system and device for processing an int4 data type based on Tensorcore and a storage medium, and the method comprises the steps: responding to the received data of which the data type is int4, judging whether the batch processing size, input dimension and output dimension of the data meet the requirements or not according to the input data dimension, weight dimension and bias dimension of the data; writing the input data of the data into the first shared memory from the global memory and writing the weight data of the data into the second shared memory from the global memory in response to that the batch processing size, the input dimension and the output dimension of the data meet the requirements; storing a first calculation result obtained based on thefirst shared memory and the second shared memory into a third shared memory to be added with the offset data to obtain a second calculation result; and returning the second calculation result to the global memory. According to the method, the int4 data type is supported by the TVM full connection layer, and compared with int8, the performance is greatly improved.

Description

technical field [0001] The present invention relates to the field of neural networks, and more particularly, to a method, system, computer device and readable medium for processing int4 data types based on Tensorcore. Background technique [0002] Dense is a fully connected layer in a neural network and plays an important role in neural network feature extraction. Suppose data is the input data of the dense, the shape is (batch, in_dim), the weight is the weight of the dense, the shape is (out_dim, in_dim), the bias is the bias of the dense, the shape is (out_dim, ), here (out_dim,) Represents that this is a one-dimensional vector of length out_dim, such as where batch represents the batch size of the neural network training, in_dim represents the input dimension of the dense, and out_dim represents the output dimension. The current fully connected layer can only support the data type of data and weight as int8, and there is no way to deal with the data whose data type is...

Claims

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to View More

Application Information

Patent Timeline
no application Login to View More
Patent Type & Authority Applications(China)
IPC IPC(8): G06N3/063G06F9/54
CPCG06N3/063G06F9/544
Inventor 宋小妹
Owner SUZHOU LANGCHAO INTELLIGENT TECH CO LTD
Who we serve
  • R&D Engineer
  • R&D Manager
  • IP Professional
Why Patsnap Eureka
  • Industry Leading Data Capabilities
  • Powerful AI technology
  • Patent DNA Extraction
Social media
Patsnap Eureka Blog
Learn More
PatSnap group products

Browse by: Latest US Patents, China's latest patents, Technical Efficacy Thesaurus, Application Domain, Technology Topic, Popular Technical Reports.

© 2024 PatSnap. All rights reserved.Legal|Privacy policy|Modern Slavery Act Transparency Statement|Sitemap|About US| Contact US: help@patsnap.com