Imaging Device Nested Picture-in-Picture Compositing
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing imaging systems struggle to simultaneously capture and display both real and virtual objects within and outside the angle of view, leading to exclusion of main objects from the screen when multiple objects are present.
Innovation Solution
An imaging device that includes an image sensor and processors to capture real objects, acquire virtual object information, generate virtual images, and composite real and virtual images, allowing for the generation of composited images that include both in-angle and out-of-angle views.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Loss of information
If picture-in-picture technology is used to display multiple objects, then information within the angle of view can be displayed, but information outside the angle of view cannot be displayed
Solution Approach 1:
The patent implements a nested picture-in-picture structure where a first composited image (containing virtual object and first real image) is embedded within a second composited image (containing second real image). This nested arrangement allows multiple layers of information to be displayed simultaneously, with the inner picture showing out-of-angle-view information and the outer picture showing additional contextual information, thereby overcoming the limitation of conventional single-layer PiP technology
Solution Approach 2:
The patent transitions from conventional two-dimensional display to a multi-dimensional composited image structure by integrating virtual objects with real images across multiple layers. The first composited image layer combines virtual object information with first real image, while the second composited image layer combines second real image with the first composited image, creating a layered dimensional structure that displays information both within and outside the original angle of view
2Ease of operation
If pan/tilt/zoom control is performed to include main object in angle of view, then composition is optimized, but other main objects may be excluded from screen
Solution Approach 1:
The patent segments the display into multiple composited image layers, with each layer capturing different objects and information. The first composited image captures the virtual object and first real image, while the second composited image captures the second real image. This segmentation allows the imaging device to maintain optimized composition for each segment while preserving information about multiple main objects that would otherwise be excluded
Solution Approach 2:
The patent introduces composited images as intermediary elements that bridge the gap between the limited angle of view and the need to display multiple objects. By compositing virtual objects with real images and nesting these composited images within each other, the system creates intermediary visual representations that convey information about objects both within and outside the original angle of view without requiring physical repositioning of the imaging device
Data Source
AI summary
An imaging device according to the present invention includes: an image sensor configured to capture a real object and output a real image; and one or more processors and/or circuitry configured to execute acquisition processing of acquiring information of a virtual object, execute generating processing of generating a virtual image on a basis of the information of the virtual object acquired in the acquisition processing, and execute compositing processing of compositing the real image and the virtual image to generate a composited image, wherein in the compositing processing, a first composited image in which part of a first real image and the virtual image are composited is generated, and a second composited image in which an image based on a second real image and the first composited image are composited is generated.