Deep Neural Network Inference Using Binary Feature Map Compression

Resolve Bottlenecks,
Find Innovative SolutionsGenerate Solutions

Solution Overview

Problem

Existing deep neural network (DNN) methods that reduce processing load through layer fusion and quantization compression result in insufficient improvements in processing speed and introduce accuracy reductions.

Innovation Solution

An execution method for DNNs that involves converting floating-point or fixed-point intermediate feature maps to binary vectors, compressing these using nonlinear dimensionality reduction layers, storing the compressed maps, decompressing them for reconstruction, and converting back to original form, thereby reducing memory usage and increasing processing speed with minimal accuracy loss.

Engineering Contradictions & Design Principles

VSEngineering Contradiction Analysis

1Quantity of substance

If layer fusion and quantization compression are used to reduce processing load, then memory usage is reduced, but processing speed improvement is insufficient and accuracy drops

Engineering Contradiction:
Improvememory usageVSAvoidprocessing speed
Core Design Contradiction:
Quantity of substanceVSProductivity

Solution Approach 1:

The patent changes the numerical representation parameter from floating-point to binary format for intermediate feature maps. This parameter change enables more aggressive compression (up to 32x reduction) while maintaining computational accuracy, thereby simultaneously improving memory efficiency and processing speed without the accuracy drops associated with conventional quantization methods.

Inventive Principle:
Principle #35Parameter changes

Solution Approach 2:

The patent introduces a new dimension of binary representation between conventional quantization and final output. By transforming feature maps into binary format and using specialized binary convolution operations, the system achieves compression in a different dimensional space, enabling both faster processing and reduced memory usage while preserving accuracy.

Inventive Principle:
Principle #17Another dimension (Dimensionality change)

2Quantity of substance

If conventional quantization is used to reduce memory usage, then processing load is reduced, but accuracy is significantly reduced

Engineering Contradiction:
Improvememory usageVSAvoidaccuracy
Core Design Contradiction:
Quantity of substanceVSMeasurement precision

Solution Approach 1:

The patent changes the numerical representation parameter from floating-point to binary format for intermediate feature maps. This parameter change enables more aggressive compression (up to 32x reduction) while maintaining computational accuracy, thereby simultaneously improving memory efficiency and processing speed without the accuracy drops associated with conventional quantization methods.

Inventive Principle:
Principle #35Parameter changes

3Measurement precision

If floating-point intermediate feature maps are used, then accuracy is maintained, but memory usage and processing load are excessive

Engineering Contradiction:
ImproveaccuracyVSAvoidmemory usage
Core Design Contradiction:
Measurement precisionVSQuantity of substance

Solution Approach 1:

The patent changes the numerical representation parameter from floating-point to binary format for intermediate feature maps. This parameter change enables more aggressive compression (up to 32x reduction) while maintaining computational accuracy, thereby simultaneously improving memory efficiency and processing speed without the accuracy drops associated with conventional quantization methods.

Inventive Principle:
Principle #35Parameter changes

Solution Approach 2:

The patent introduces a new dimension of binary representation between conventional quantization and final output. By transforming feature maps into binary format and using specialized binary convolution operations, the system achieves compression in a different dimensional space, enabling both faster processing and reduced memory usage while preserving accuracy.

Inventive Principle:
Principle #17Another dimension (Dimensionality change)

4Quantity of substance

If binary representation is used for compression, then memory usage is reduced, but processing speed may be affected

Engineering Contradiction:
Improvememory usageVSAvoidprocessing speed
Core Design Contradiction:
Quantity of substanceVSSpeed

Solution Approach 1:

The patent replaces conventional floating-point arithmetic operations with binary convolution operations that are optimized for hardware implementation. This substitution enables faster processing speeds by leveraging bitwise operations and specialized binary neural network hardware, thereby improving processing speed while maintaining the memory efficiency benefits of binary representation.

Inventive Principle:
Principle #28Mechanics substitution (Replace mechanical system)

Data Source

PatentUS11481613B2Execution method, execution device, learning method, learning device, and recording medium for deep neural network
Publication Date: 2022.10.25 PANASONIC INTELLECTUAL PROPERTY CORP OF AMERICA
  • US11481613B2 patent drawing
  • US11481613B2 patent drawing
  • US11481613B2 patent drawing

AI summary

Executing a deep neural network by obtaining, during deep neural network inference, a binary intermediate feature map in binary representation by converting a floating-point or fixed-point intermediate feature map into a binary vector using a first transformation module; generating a compressed feature map by compressing the binary intermediate feature map using a nonlinear dimensionality reduction layer; storing the compressed feature map into memory; reconstructing the binary intermediate feature map by decompressing the compressed feature map read from the memory using a reconstruction layer corresponding to the nonlinear dimensionality reduction layer; and converting the reconstructed binary intermediate feature map into a floating-point or fixed-point intermediate feature map using a second transformation module.