Machine Learning Image Adjustment Maps

Resolve Bottlenecks,
Find Innovative SolutionsGenerate Solutions

Solution Overview

Problem

Traditional image signal processors rely on numerous hand-tuned parameters for image processing, which is time-consuming and expensive, and fail to provide dynamic adjustments based on image content, leading to suboptimal image enhancement.

Innovation Solution

A machine learning-based system that generates spatially varying maps for image processing functions like noise reduction, sharpening, and color saturation, allowing for pixel-level adjustments and dynamic tuning based on image content, using trained neural networks to apply affine coefficients and ensure local linearity constraints for improved image quality.

Engineering Contradictions & Design Principles

VSEngineering Contradiction Analysis

1Adaptability or versatility

If traditional ISPs use separate discrete blocks with hand-tuned parameters for image processing, then specific image processing functions can be applied, but the system requires over 10,000 hand-tuned parameters that must be re-tuned for each customer preference, which is very time-consuming and expensive

Engineering Contradiction:

Improveability to re-tune parameters for different customer preferencesVSAvoidtime required for hand-tuning parameters

Core Design Contradiction:

Adaptability or versatilityVSLoss of time

Solution Approach 1:

The patent replaces the manual hand-tuning mechanism with a machine learning model that automatically generates processing parameters. The ML model takes image data as input and outputs optimized parameter settings for various image processing functions, eliminating the need for manual adjustment of over 10,000 parameters while maintaining adaptability to different customer preferences and image characteristics.

Inventive Principle:

Principle #28Mechanics substitution (Replace mechanical system)

Solution Approach 2:

The system enables self-service by allowing the image processing parameters to be automatically determined by the machine learning model based on the input image characteristics. The model autonomously adjusts parameters for noise reduction, sharpening, color processing, and other functions without requiring manual intervention, thus resolving the time-consuming hand-tuning process.

Inventive Principle:

Principle #25Self-service

2Manufacturing precision

If traditional ISPs use hand-tuned parameters for image processing, then consistent processing can be applied, but the same parameters are used for every image regardless of content, leading to suboptimal image enhancement

Engineering Contradiction:

Improveconsistency of image processing parametersVSAvoidability to adjust parameters based on image content

Core Design Contradiction:

Manufacturing precisionVSAdaptability or versatility

Solution Approach 1:

The patent implements local quality by generating spatially varying parameter maps instead of uniform parameters across the entire image. The machine learning model produces multiple maps (e.g., noise map, sharpness map, color map) where each pixel or region has customized processing parameters tailored to its local characteristics, enabling precise adaptation to different image content while maintaining overall processing consistency.

Inventive Principle:

Principle #3Local quality

3Device complexity

If uniform image processing parameters are applied to all regions, then processing simplicity is maintained, but halo effects occur at boundaries and image quality deteriorates

Engineering Contradiction:

Improvesimplicity of processing parameter applicationVSAvoidimage quality and absence of halo effects

Core Design Contradiction:

Device complexityVSManufacturing precision

Solution Approach 1:

The patent applies segmentation by dividing the image processing into multiple spatial regions with distinct parameter sets. The machine learning model generates separate parameter maps for different image regions, allowing independent optimization of processing parameters for each region. This segmented approach eliminates halo effects at boundaries by ensuring smooth transitions between regions with different processing characteristics.

Inventive Principle:

Principle #1Segmentation

Data Source

PatentUS11983846B2Machine learning based image adjustment

Publication Date: 2024.05.14 QUALCOMM INC

AI summary

An imaging system can obtain image data, for instance from an image sensor. The imaging system can supply the image data as input data to a machine learning system, which can generate one or more maps based on the image data. Each map can identify strengths at which a certain image processing function is to be applied to each pixel of the image data. Different maps can be generated for different image processing functions, such as noise reduction, sharpening, or color saturation. The imaging system can generate a modified image based on the image data and the one or more maps, for instance by applying each of one or more image processing functions in accordance with each of the one or more maps. The imaging system can supply the image data and the one or more maps to a second machine learning system to generate the modified image.