CNN Activation Caching for Low-Power Video Inference
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing convolutional neural networks (CNNs) require intensive processing power for accurate inferencing, which is not suitable for low power video applications like battery-powered cameras, due to redundant computations on successive video frames with little or no change.
Innovation Solution
Cache the activations produced by one or more layers of a CNN and substitute them for redundant computations on static pixels between successive video frames, performing neural network operations only on motion pixels.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If a full CNN is applied to every video frame, then accurate inferencing is achieved, but power consumption and computational load increase significantly
Solution Approach 1:
The video processing is segmented into motion detection and CNN inferencing components. Only frames containing motion (detected via difference from reference frame) are processed by the power-intensive CNN, while static frames are skipped. This segmentation resolves the contradiction by applying full processing only where necessary for accuracy while conserving power on redundant frames.
Solution Approach 2:
Instead of applying the full CNN to every frame (excessive action), the system applies partial processing by first performing lightweight motion detection and then applying the CNN only to frames with detected motion. This partial action approach maintains accuracy for frames that need it while avoiding unnecessary power consumption on static frames.
2Measurement precision
If a full CNN is applied to every video frame, then accurate inferencing is achieved, but computational load increases significantly
Solution Approach 1:
The processing pipeline is segmented into two stages: a lightweight motion detection stage that processes all frames, and a CNN inferencing stage that processes only motion-containing frames. This segmentation improves computational efficiency by eliminating redundant full-CNN applications on static frames while preserving accuracy through selective processing.
Solution Approach 2:
Motion detection is performed as a preliminary action before CNN inferencing. This preliminary filtering identifies which frames require full processing, allowing the system to avoid unnecessary computational load on static frames while ensuring accurate inferencing on frames that contain relevant changes.
3Productivity
If motion detection is performed to identify changes between frames, then redundant computations are reduced, but additional processing steps are required
Solution Approach 1:
The motion detection step extracts only the essential information needed to determine whether a frame requires CNN processing. By extracting and acting on this simple motion indicator, the system achieves significant computational efficiency gains without adding substantial complexity, as the motion detection logic is straightforward frame-difference comparison.
Data Source
AI summary
This disclosure provides methods, devices, and systems for machine learning. The present implementations more specifically relate to techniques for reducing the computational load of a convolutional neural network (CNN) when processing successive video frames. In some aspects, a machine learning system may cache or store the outputs (also referred to as “activations”) produced by one or more layers of a CNN so that one or more cached activations can be substituted for respective activations that would otherwise be computed by the CNN when processing a subsequent video frame. For example, the machine learning system may compare each video frame with a preceding frame of the video to detect pixels that undergo significant changes between successive frames (also referred to as “motion pixels”). In some aspects, the CNN may only perform neural network operations that involve one or more motion pixels or features derived from a motion pixel.