Unlock instant, AI-driven research and patent intelligence for your innovation.

Method and apparatus for compressing neural networks

A neural network, summation technology, applied in the field of compressed deep neural networks, which can solve problems such as inappropriateness

Pending Publication Date: 2022-03-08
ROBERT BOSCH GMBH
View PDF0 Cites 0 Cited by
  • Summary
  • Abstract
  • Description
  • Claims
  • Application Information

AI Technical Summary

Problems solved by technology

[0008] Given the fact that all network layers jointly contribute to the learning task, it is inappropriate to remove individual layers independently of each other

Method used

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
View more

Image

Smart Image Click on the blue labels to locate them in the text.
Viewing Examples
Smart Image
  • Method and apparatus for compressing neural networks
  • Method and apparatus for compressing neural networks
  • Method and apparatus for compressing neural networks

Examples

Experimental program
Comparison scheme
Effect test

Embodiment Construction

[0039] For specific learning tasks (eg classification, especially semantic segmentation), the corresponding first cost function is generally defined. learning The neural network or network architecture is defined for this. First cost function L learning It can be any cost function (English: Loss function, loss function), which characterizes the deviation between the output of the neural network and the label consisting of training data. Neural network consists of interconnected layers. Therefore, the neural network is defined by a continuously arranged layer before learning. Each layer is equipped with its input variable, or can be linearly or non-linear conversion thereof. The layer having a weighting summation is referred to as the first layer. Weighted summation can be carried out by means of matrix vectors or by means of convolution. For the matrix vectors, the rows of the matrix correspond to weighted, and for convolution, the filter corresponds to weight. After each layer, a...

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to View More

PUM

No PUM Login to View More

Abstract

A method for compressing a neural network. The method comprises the following steps: defining the maximum complexity of the neural network. A first cost function is determined. A second cost function is determined that characterizes a deviation between a current complexity of the neural network and a defined complexity. Learning a neural network such that a sum of the first and second cost functions is optimized according to parameters of the neural network; and removing the following weighting, the scaling factor assigned to the weighting being less than a predetermined threshold value.

Description

Technical field [0001] The present invention relates to a method, apparatus, computer program, and machine readable storage medium for compressing depth neural networks based on pre-gived maximum complexity. Background technique [0002] Neural networks can be used for various tasks of driver assistance or automatic driving, such as semantic segmentation for video images, wherein each pixel is classified as different categories (pedestrians, vehicles, etc.). [0003] However, systems currently used for drivers to assist or use for automatic driving require special hardware, especially due to safety and efficiency. This, for example, special microcontrollers with limited storage capabilities. However, these restrictions provide special requirements for the development of neural networks because neural networks are usually trained on high performance computers with mathematical optimization methods and floating point numbers. Then, if you simply remove the training of the neural ne...

Claims

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to View More

Application Information

Patent Timeline
no application Login to View More
Patent Type & Authority Applications(China)
IPC IPC(8): G06N3/08G06N3/04G06F17/16
CPCG06N3/08G06F17/16G06N3/045G06N3/082G06N3/048
Inventor F·蒂姆L·恩德里希
Owner ROBERT BOSCH GMBH