Multi-Camera Depth-Based Zoom Spatial Transform

Resolve Bottlenecks,
Find Innovative Solutions
Generate Solutions

Solution Overview

Problem

Multi-camera devices face issues with seamless transitions between cameras due to differences in field of view and color, leading to undesirable shifts in the preview screen during zoom operations, which can distort images and disrupt the user's composition experience.

Innovation Solution

The implementation of a camera system that computes an error-free transformation using depth information, particularly at user- or machine-determined regions of interest, to align pixels from one camera to another, ensuring a smooth and distortion-free transition between cameras.

Engineering Contradictions & Design Principles

VSEngineering Contradiction Analysis

1Manufacturing precision

If a global transformation is applied between multiple cameras without considering depth information, then the transformation process is simple and fast, but the resulting image becomes distorted due to parallax

Engineering Contradiction:
Improveimage alignment precisionVSAvoidtransformation computation complexity
Core Design Contradiction:
Manufacturing precisionVSDevice complexity

Solution Approach 1:

The patent applies local quality by transitioning from a global transformation approach to a local, depth-aware transformation approach. Instead of applying a single transformation to the entire image, the system computes depth information for different regions and applies localized transformations that account for varying depth distances. This ensures that pixels from different depth planes are transformed appropriately, eliminating parallax-induced distortions while maintaining computational feasibility through region-based processing.

Inventive Principle:
Principle #3Local quality

2Adaptability or versatility

If the imaging device switches between cameras based on zoom commands, then the zoom function is achieved, but the user perceives a sudden change in the preview screen

Engineering Contradiction:
Improvezoom capabilityVSAvoidpreview screen stability
Core Design Contradiction:
Adaptability or versatilityVSStability of the object's composition

Solution Approach 1:

The patent applies preliminary action by performing depth-based spatial transformation on images from the second camera before switching occurs. The system pre-computes the transformation using depth information from the first camera's depth map, so that when the camera switch happens, the transformed image from the second camera already matches the spatial characteristics of the first camera. This preliminary transformation ensures a smooth, imperceptible transition that maintains preview stability during zoom operations.

Inventive Principle:
Principle #10Preliminary action

3Measurement precision

If depth information is used to compute spatial transformation between cameras, then image alignment precision is improved, but computational requirements increase

Engineering Contradiction:
Improvepixel alignment precisionVSAvoidprocessing power consumption
Core Design Contradiction:
Measurement precisionVSPower

Solution Approach 1:

The patent applies segmentation by dividing the image processing into distinct stages: first capturing depth information and generating a depth map from the first camera, then using this depth map to guide the transformation of the second camera's image. The system segments the computation into depth estimation, spatial transform calculation, and image transformation phases. This segmented approach enables precise depth-aware alignment while managing computational load through efficient use of the depth map for guiding transformations.

Inventive Principle:
Principle #1Segmentation

Data Source

PatentUS10389948B2Depth-based zoom function using multiple cameras
Publication Date: 2019.08.20 QUALCOMM INC
  • US10389948B2 patent drawing
  • US10389948B2 patent drawing
  • US10389948B2 patent drawing

AI summary

A method for displaying preview images is disclosed. In one aspect, the method includes: receiving first images captured by a first camera having a first field-of-view (FOV), receiving second images captured by a second camera having a second FOV that is different than the first FOV, and displaying preview images generated based on the first and second images. The method may further include determining a spatial transform based on depth information associated with individual pixels in the first and second images, and upon receiving instructions to zoom in or out beyond a camera switching threshold, modifying the second image using the spatial transform and displaying the first image and the modified second image consecutively.