AR Image Compositing via Cloud Rendering Segmentation
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current augmented reality systems face challenges in seamlessly integrating computer-generated content with real-time physical environments, particularly in maintaining synchronization and realism, due to limitations in processing power and memory on mobile devices.
Innovation Solution
A method that involves a mobile device capturing a physical environment image and receiving a corresponding image from a content provider system, which renders a virtual environment view, allowing for real-time compositing and display of computer-generated objects that match the physical environment's lighting and geometry, using alpha channels and motion capture for accurate positioning and interaction.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Ease of operation
If computer-generated content is integrated with real-time physical environments using mobile devices, then augmented reality experience quality is improved, but processing power and memory constraints are exceeded
Solution Approach 1:
The system divides the image processing task into two segments: the content provider system performs rendering of computer-generated objects in a virtual environment, while the mobile device performs capture of physical environment images and compositing operations. This segmentation allows each system to operate within its computational constraints while achieving high-quality augmented reality.
Solution Approach 2:
The content provider system acts as an intermediary that receives mobile device location and camera parameters, renders corresponding virtual environment images with computer-generated content, and transmits these images back to the mobile device for compositing with captured physical environment images. This intermediary approach offloads heavy rendering computations from the mobile device.
2Reliability
If real-time compositing is performed on mobile devices, then synchronization with physical environment is improved, but processing time and energy consumption increase
Solution Approach 1:
The content provider system performs preliminary rendering actions by pre-computing virtual environment images with computer-generated content positioned according to the mobile device's location and camera parameters. This preliminary rendering before transmission to the mobile device reduces the processing burden and energy consumption during real-time compositing operations.
3Measurement precision
If virtual camera location corresponds to physical camera location, then spatial accuracy is improved, but data transmission requirements increase
Solution Approach 1:
The system transmits key parameters including mobile device location, camera parameters, and virtual environment rendering instructions rather than transmitting complete high-resolution images. The content provider system renders virtual images with alpha channels indicating transparency regions, allowing efficient transmission and processing while maintaining spatial accuracy through parameter-based positioning.
Data Source
AI summary
A method may include capturing a first image of a physical environment using a mobile device. The mobile device may include a physical camera and a display. The method may also include receiving a second image from a content provider system. The second image may be generated by the content provider system by rendering a view from a virtual camera in a virtual environment. The virtual environment may represent at least a portion of the physical environment. A location of the virtual camera in the virtual environment may correspond to a location of the physical camera in the physical environment. The second image may include a view of a computer-generated object. The method may additionally include generating a third image by compositing the first image and the second image, and causing the third image to be displayed on the display of the mobile device.