Dynamic Vision Sensor Heat Maps for Video Encoding

Resolve Bottlenecks,
Find Innovative Solutions
Generate Solutions

Solution Overview

Problem

Computer vision systems face challenges in accurately detecting objects in video frames due to bandwidth consumption and visual artifacts introduced by video compression, which affects the accuracy of object recognition algorithms.

Innovation Solution

The system generates heat maps using dynamic vision sensor events, which allows for low-latency, high-accuracy video encoding by identifying regions of interest and adjusting encoding parameters, thereby reducing data rate and power consumption while maintaining image quality.

Engineering Contradictions & Design Principles

VSEngineering Contradiction Analysis

1Loss of energy

If video compression is applied to reduce bandwidth consumption, then data transmission efficiency is improved, but visual artifacts are introduced that degrade object recognition accuracy

Engineering Contradiction:
Improvebandwidth consumptionVSAvoidobject recognition accuracy
Core Design Contradiction:
Loss of energyVSMeasurement precision

Solution Approach 1:

The patent applies different encoding qualities to different regions of the video frame based on a heat map. Regions with high temporal activity (identified by the heat map) are encoded with higher quality to preserve object details, while regions with low activity are encoded with lower quality to reduce bandwidth. This local differentiation resolves the contradiction by maintaining recognition accuracy in critical areas while reducing overall bandwidth consumption.

Inventive Principle:
Principle #3Local quality

Solution Approach 2:

The video frame is segmented into multiple regions based on temporal activity levels derived from the heat map. Each segment is then encoded independently with appropriate quality parameters. This segmentation allows the system to allocate bandwidth efficiently - preserving detail in regions containing moving objects while compressing static background areas, thus resolving the bandwidth vs. accuracy trade-off.

Inventive Principle:
Principle #1Segmentation

2Measurement precision

If standard video encoding is used to maintain image quality, then object detection accuracy is improved, but bandwidth consumption and power usage increase

Engineering Contradiction:
Improveobject detection accuracyVSAvoidpower consumption
Core Design Contradiction:
Measurement precisionVSLoss of energy

Solution Approach 1:

The patent dynamically adjusts encoding parameters based on real-time heat map data that reflects temporal activity in different frame regions. Rather than using static encoding settings, the system adapts the compression level and quality for each region according to its activity level. This dynamic approach maintains detection accuracy in active regions while reducing power consumption through aggressive compression in inactive regions.

Inventive Principle:
Principle #15Dynamics

Solution Approach 2:

The encoding parameters (such as quantization level, compression ratio, and quality settings) are changed based on the heat map information. Regions with high temporal activity receive encoding parameters that preserve detail, while regions with low activity receive parameters that maximize compression. This parameter adaptation resolves the contradiction by optimizing the balance between detection accuracy and power consumption on a region-by-region basis.

Inventive Principle:
Principle #35Parameter changes

Data Source

PatentUS10582196B2Generating heat maps using dynamic vision sensor events
Publication Date: 2020.03.03 INTEL CORP
  • US10582196B2 patent drawing
  • US10582196B2 patent drawing
  • US10582196B2 patent drawing

AI summary

An example apparatus for encoding video frames includes a receiver to receive events from a dynamic vision sensor and a video frame from an image sensor. The apparatus also includes a heat map generator to generate a heat map based on the received events. The apparatus further includes a region of interest (ROI) map generator generate a ROI map based on the heat map. The apparatus includes a parameter adjuster to adjust an encoding parameter based on the ROI map. The apparatus also further includes a video encoder to encode the video frame using the adjusted parameter.