Unlock instant, AI-driven research and patent intelligence for your innovation.

A deep learning model inference period acceleration method, device and system

A deep learning and model technology, applied in the field of deep neural network learning, can solve problems such as increasing computing time, response delay and device power consumption, and achieve the effect of reducing computing time, reducing device power consumption, and reducing additional overhead

Active Publication Date: 2021-06-25
BEIJING WENAN INTELLIGENT TECH CO LTD
View PDF2 Cites 0 Cited by
  • Summary
  • Abstract
  • Description
  • Claims
  • Application Information

AI Technical Summary

Problems solved by technology

[0007] Existing techniques retain additional computational overhead during deep learning model inference, increasing computational time, response latency, and device power consumption

Method used

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
View more

Image

Smart Image Click on the blue labels to locate them in the text.
Viewing Examples
Smart Image
  • A deep learning model inference period acceleration method, device and system
  • A deep learning model inference period acceleration method, device and system
  • A deep learning model inference period acceleration method, device and system

Examples

Experimental program
Comparison scheme
Effect test

example 1

[0089] Example 1: Apply the method of accelerating the inference period of the deep learning model to the smart face capture camera; the smart face capture camera realizes face detection, face key point positioning, and face attributes through the embedded deep learning algorithm Recognition and facial recognition. Among them, the face detection model, key point positioning model, attribute and identity recognition model all adopt convolutional neural network with "batch normalization" structure. The following takes the face detection model as an example to illustrate the process of implementing the method for accelerating the inference period of the deep learning model. The implementation of the present invention on other models can be known by analogy.

[0090] First, prepare the training data of the face detection model; and design and build a convolutional neural network with a "batch normalization" structure and a test data set for the detection task;

[0091] S1: Using ...

example 2

[0099] Example 2: Apply the method of accelerating the inference period of the deep learning model to the cloud analysis server. The cloud analysis server can use GPU, FPGA or other computing accelerators to execute deep learning algorithms for large-scale face identity recognition comparison, pedestrian identity re-identification (ReID), target attribute recognition and video structure in intelligent traffic scenarios. and other functions. Different from smart cameras, convolutional neural networks deployed on cloud servers usually have larger parameters and computing scale. When training large-scale convolutional neural networks, "batch normalization" is essential. Taking the large-scale face recognition model as an example, the implementation process of the present invention is described, and the application of the present invention to other algorithm models can be known by analogy.

[0100] First, prepare the training data of the face recognition model; and design and bui...

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to View More

PUM

No PUM Login to View More

Abstract

The invention relates to the technical field of deep neural network learning, and discloses a method, device and system for accelerating the inference period of a deep learning model. The deep learning model reasoning period acceleration method of the present invention obtains the optimized deep learning model and the data to be processed; the optimized deep learning model has optimized merging parameters; the optimized deep learning model with the optimized merging parameters performs the processing on the Perform data processing on the data; output the data after the data processing. The present invention performs data processing on the data to be processed through the optimized deep learning model with the optimized merging parameters; saves extra calculation overhead in the reasoning period of the deep learning model, thereby reducing the reasoning period in the application process of the deep learning model Computation time and response delays reduce device power consumption.

Description

technical field [0001] The invention relates to the technical field of deep neural network learning, in particular to a method, device and system for accelerating the inference period of a deep learning model. Background technique [0002] In recent years, the breakthrough of deep learning technology has greatly promoted the development of the field of computer vision. The accuracy of traditional problems has been continuously improved, approaching the limit, and new application fields are also constantly expanding. [0003] Graphics Processing Unit (GPU for short) is currently the mainstream computing device for cloud and embedded deep learning computing. "NVIDIA TensorRT" (NVIDIA TensorRT) is a high-performance neural network inference engine responsible for converting and optimizing trained models for accelerated execution on NVIDIA GPUs for deploying deep learning applications in production environments. When dealing with "batch normalization" calculations, TensorRT use...

Claims

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to View More

Application Information

Patent Timeline
no application Login to View More
Patent Type & Authority Patents(China)
IPC IPC(8): G06N3/04G06N3/08G06N5/04
CPCG06N3/08G06N5/04G06N3/045
Inventor 曹松魏汉秦林宇陶海
Owner BEIJING WENAN INTELLIGENT TECH CO LTD