VR Image Blending via Shape Warping and Color Morphing

Resolve Bottlenecks,
Find Innovative SolutionsGenerate Solutions

Solution Overview

Problem

Existing VR systems face issues with jerky transitions and double vision when combining images from different cameras, particularly when switching between binocular and monocular views, leading to a degraded user experience.

Innovation Solution

An image processing system that blends top/bottom images with synthetic images using shape warping and color morphing techniques, adjusting the interpupillary distance based on the vertical angle to create seamless transitions between binocular and monocular views, and addresses occlusions by merging secondary camera views with primary views.

Engineering Contradictions & Design Principles

VSEngineering Contradiction Analysis

1Adaptability or versatility

If images from different cameras are combined to create canvas views, then virtual scenes can be provided for VR experience, but double vision occurs in the combined image

Engineering Contradiction:

ImproveVR virtual scene creationVSAvoidimage combination accuracy

Core Design Contradiction:

Adaptability or versatilityVSManufacturing precision

Solution Approach 1:

The patent introduces an intermediary blending process between images from different cameras. Instead of directly combining images which causes double vision, the system uses optical flow to calculate pixel-level correspondence and alpha blending to smoothly transition between images, eliminating the harmful double vision effect while maintaining VR scene creation capability

Inventive Principle:

Principle #24Intermediary (Mediator)

Solution Approach 2:

The patent dynamically adjusts blending parameters based on the vertical angle of the view. By changing the alpha blending factor and optical flow weights as a function of vertical angle, the system adapts the image combination process to maintain precision across different viewing directions, resolving the double vision issue while preserving VR adaptability

Inventive Principle:

Principle #35Parameter changes

2Adaptability or versatility

If top/bottom and side camera images are blended to simulate user views, then binocular and monocular views can be created, but the transition between views is jerky with seams

Engineering Contradiction:

Improvebinocular and monocular view simulationVSAvoidview transition smoothness

Core Design Contradiction:

Adaptability or versatilityVSEase of operation

Solution Approach 1:

The patent implements dynamic view transition by adjusting the blending between top/bottom and side camera images based on the vertical angle. The system dynamically modifies the contribution of each camera's image as the user's gaze moves vertically, creating a smooth continuous transition between binocular and monocular views rather than abrupt switches

Inventive Principle:

Principle #15Dynamics

Solution Approach 2:

The patent ensures continuous smooth transition by using optical flow to maintain pixel correspondence across the transition boundary and alpha blending to smoothly interpolate between different camera views. This eliminates seams and jerky transitions, providing continuous visual experience as the user moves their gaze vertically

Inventive Principle:

Principle #20Continuity of useful action

3Adaptability or versatility

If synthetic images are created from side images to simulate user eye views, then binocular vision can be provided, but distortion occurs in the created view

Engineering Contradiction:

Improvebinocular vision simulationVSAvoidview accuracy

Core Design Contradiction:

Adaptability or versatilityVSManufacturing precision

Solution Approach 1:

The patent replaces direct geometric projection methods with optical flow-based pixel correspondence. Instead of mechanically transforming images which causes distortion, the system uses optical flow to calculate how pixels move between frames, preserving the natural appearance and reducing distortion in the synthesized binocular views

Inventive Principle:

Principle #28Mechanics substitution (Replace mechanical system)

Applied Scientific Principles

This section explains which scientific principles are used to turn an abstract innovation direction into a practical engineering solution.

Function Achieved in This Case

The system achieves smooth transitions between binocular and monocular views, reducing distortion and double vision, thereby enhancing the VR user experience by creating a seamless and accurate virtual scene.

Implementation Method 1

optical flow is used for the shape warping

Methodology Applied

Scientific EffectOptical flow:

Implementation Method 2

alpha blending is used for the color morphing

Methodology Applied

Scientific EffectAlpha blending:

Data Source

PatentUS10460521B2Transition between binocular and monocular views

Publication Date: 2019.10.29 META PLATFORMS INC

AI summary

An image processing system is designed to generate a canvas view that transitions between binocular views and monocular views. Initially, the image processing system receives top/bottom images and side images of a scene and calculates offsets to generate synthetic side images for left and right view of a user. To transition between binocular views and monocular views, the image processing system first warps top/bottom images onto corresponding synthetic side images to generate warped top/bottom images, which realizes the transition in terms of shape. The image processing system then morphs the warped top/bottom images onto the corresponding synthetic side images to generate blended images for left and right eye views with the blended images. The image processing system creates the canvas view which transitions between binocular views and monocular views in terms of image shape and color based on the blended images.