Perception-Based Image Processing for Adaptive Video Encoding

Resolve Bottlenecks,
Find Innovative Solutions
Generate Solutions

Solution Overview

Problem

Current smartphone video processing technologies face challenges in optimizing power consumption while maintaining perceived visual quality, particularly in performing auto-focus and auto-exposure functions with minimal user intervention.

Innovation Solution

A perception-based image processing apparatus and method that utilizes an image analyzing circuit to obtain training data, set a perception model, perform object detection, and generate an object detection information signal, which is used to control an application circuit for adaptive video encoding and camera functions, such as auto-focus and auto-exposure, based on predicted user attention regions.

Engineering Contradictions & Design Principles

VSEngineering Contradiction Analysis

1Manufacturing precision

If video frames are encoded with high quality settings, then perceived visual quality is improved, but power consumption increases

Engineering Contradiction:
Improvevideo encoding qualityVSAvoidpower consumption
Core Design Contradiction:
Manufacturing precisionVSUse of energy by moving object

Solution Approach 1:

The patent applies different encoding qualities to different regions of the video frame based on predicted user attention. High-quality encoding is applied only to attention regions where users are likely to look, while non-attention regions use lower quality settings. This resolves the contradiction by maintaining perceived visual quality through selective high-quality encoding rather than uniform high-quality encoding across the entire frame.

Inventive Principle:
Principle #3Local quality

Solution Approach 2:

The system dynamically changes encoding parameters (such as bitrate, resolution, or compression level) based on the predicted user attention regions. By adjusting these parameters locally according to attention predictions, the system achieves high perceived quality where needed while reducing overall power consumption through selective parameter optimization.

Inventive Principle:
Principle #35Parameter changes

2Measurement precision

If auto-focus and auto-exposure functions are implemented with manual user selection, then measurement precision is improved, but ease of operation deteriorates

Engineering Contradiction:
Improvefocus and exposure accuracyVSAvoiduser intervention requirement
Core Design Contradiction:
Measurement precisionVSEase of operation

Solution Approach 1:

The system performs auto-focus and auto-exposure automatically based on predicted user attention regions without requiring manual user selection. The attention prediction model identifies what the user is likely to look at, and the camera automatically adjusts focus and exposure for those regions, making the system self-sufficient and eliminating the need for user intervention while maintaining accuracy.

Inventive Principle:
Principle #25Self-service

Solution Approach 2:

The system uses feedback from the attention prediction model to automatically adjust camera settings. The model continuously predicts user attention regions, and this information feeds back to the auto-focus and auto-exposure algorithms, enabling them to make accurate adjustments without user input. This creates a closed-loop system that maintains measurement precision while improving ease of operation.

Inventive Principle:
Principle #23Feedback

Data Source

PatentUS10805519B2Perception-based image processing apparatus and associated method
Publication Date: 2020.10.13 MEDIATEK INC
  • US10805519B2 patent drawing
  • US10805519B2 patent drawing
  • US10805519B2 patent drawing

AI summary

A perception-based image processing apparatus includes an image analyzing circuit and an application circuit. The image analyzing circuit obtains training data, sets a perception model according to the training data, performs an object detection of at least one frame, and generates an object detection information signal based at least partly on a result of the object detection of said at least one frame. The application circuit operates in response to the object detection information signal.