Mobile AR Perspective Alignment via Back Projection
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Augmented reality systems suffer from obstructed visual flow due to the mismatch in perspectives between the user and the camera in mobile computing devices, causing real-world objects at the periphery to be duplicated in augmented reality renderings.
Innovation Solution
A mobile computing device that captures images of the user and the real-world environment, determines the position of the user's eyes and objects relative to the device, and generates a back projection of the real-world environment to the display, aligning the image with the user's perspective to create a seamless visual experience by projecting the occluded content as if viewed through a window.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If the camera captures images from its fixed position, then the captured image contains the real-world environment, but the user's perspective does not match the camera's perspective causing visual discontinuity and object duplication
Solution Approach 1:
Instead of projecting virtual objects onto the captured image from the camera's perspective, the patent inverts the approach by back-projecting the captured image content from the display to the user's eye position. This reversal of projection direction ensures that the displayed image aligns with the user's perspective rather than the camera's perspective, eliminating object duplication and achieving visual continuity.
Solution Approach 2:
The patent introduces a back-projection computation as an intermediary process between the captured image and the display output. This intermediary step calculates the geometric transformation needed to map image content from camera coordinates to user-eye coordinates, serving as a mediator that reconciles the perspective mismatch between camera capture and user viewing.
2Adaptability or versatility
If virtual objects are overlaid on captured images without perspective adjustment, then augmented reality functionality is achieved, but real-world objects appear duplicated in the periphery reducing immersion
Solution Approach 1:
The patent applies back-projection from display to user eye, inverting the traditional projection direction. This ensures that both the captured image content and overlaid virtual objects are rendered from the user's perspective, maintaining visual continuity while preserving augmented reality functionality.
Solution Approach 2:
The patent applies different processing to different parts of the image: the captured image content undergoes back-projection transformation, while virtual objects are rendered directly in user-space coordinates. This local differentiation ensures that each element is displayed from the appropriate perspective, maintaining overall visual continuity.
Data Source
Figure 1
Figure 2
Figure 3
AI summary
Technologies for adjusting a perspective of a captured image for display on a mobile computing device include capturing a first image of a user by a first camera and a second image of a real-world environment by a second camera. The mobile computing device determines a position of an eye of the user relative to the mobile computing device based on the first captured image and a distance of an object in the real-world environment from the mobile computing device based on the second captured image. The mobile computing device generates a back projection of the real-world environment captured by the second camera to the display based on the determined distance of the object in the real-world environment relative to the mobile computing device, the determined position of the user's eye relative to the mobile computing device, and at least one device parameter of the mobile computing device.