Logical Image Representations for Low-Overhead CNN Inference
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Conventional techniques for compressing machine learning models face a tradeoff between the level of compression and accuracy, with compressed models still being too large or complex for devices with limited resources, and existing compression methods fail to maintain accuracy when compressed beyond a certain point.
Innovation Solution
Generate a simplified representation of a trained machine learning model by determining a set of images associated with an output class and creating a logical or aggregated representation of pixel values, allowing for a mapping between these images and the output class, enabling efficient 'lookup' operations for inference.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Use of energy by moving object
If conventional compression techniques are applied to reduce model size and resource requirements, then resource overhead and latency are reduced, but model accuracy is adversely impacted
Solution Approach 1:
The patent segments the model compression process into two distinct phases: (1) training a full-precision model to achieve high accuracy, and (2) generating a separate integer-valued lookup table for compression. This segmentation allows each component to be optimized independently - the floating-point model maintains accuracy while the integer lookup table reduces resource overhead during inference.
Solution Approach 2:
The patent introduces an intermediary conversion process that transforms floating-point model outputs into integer-valued lookup table keys. This intermediary step enables the system to leverage both floating-point precision during training and integer efficiency during inference, resolving the contradiction between accuracy and resource usage.
2Quantity of substance
If compression ratio is increased to reduce model size for resource-constrained devices, then memory and computational requirements are reduced, but the compressed model remains too large or complex to run efficiently
Solution Approach 1:
The patent changes the parameter representation from floating-point to integer values in the lookup table. This parameter transformation dramatically reduces both model size and complexity - integer values require fewer bits to represent and simpler computational operations, enabling efficient deployment on resource-constrained devices while maintaining reasonable accuracy.
3Quantity of substance
If quantization is applied to use lower precision numbers, then memory and computational resources are reduced, but accuracy is compromised beyond acceptable levels
Solution Approach 1:
The patent implements a dynamic two-stage approach where the system adapts its precision requirements: floating-point precision is used during the training phase to ensure high accuracy, while integer precision is used during the inference phase to reduce memory usage. This dynamic switching resolves the contradiction between memory efficiency and computational accuracy.
Data Source
AI summary
One embodiment of the present invention sets forth a technique for simplifying a trained machine learning model. The technique includes determining a first set of images associated with a first output class predicted by the trained machine learning model. The technique also includes generating a first logical representation of the first set of images, wherein the first logical representation includes one or more conjunctions of a first set of pixel values included in a first image and a disjunction of the first set of pixel values and a second set of pixel values included in a second image. The technique further includes generating a simplified representation of the trained machine learning model that includes a first mapping of the first logical representation to the first output class, wherein the first mapping indicates that the trained machine learning model predicts the first output class for one or more input images.