Virtual Display for 2D Apps in VR

Resolve Bottlenecks,
Find Innovative SolutionsGenerate Solutions

Solution Overview

Problem

Users of virtual reality (VR) headsets face challenges in accessing two-dimensional (2D) mobile application interfaces within a three-dimensional (3D) VR environment without removing the headset, as existing solutions do not efficiently utilize computing resources to provide seamless interaction with mobile devices for tasks like texting or making calls.

Innovation Solution

A system and method that redirect the main display of a mobile device to a virtual display in volatile memory, allowing 2D graphic content to be rendered and integrated into the 3D VR environment, enabling unmodified 2D mobile apps to operate within the VR environment by using hardware composers and memory buffers to assist processors and GPUs in rendering 2D and 3D images.

Engineering Contradictions & Design Principles

VSEngineering Contradiction Analysis

1Adaptability or versatility

If the mobile device uses its computing resources to render both VR environment and 2D mobile applications simultaneously, then the user can access 2D apps within VR environment, but the computing power and resources are insufficient to provide satisfactory performance for both VR and 2D applications

Engineering Contradiction:

Improveability to run 2D apps in VR environmentVSAvoidcomputing power and resources

Core Design Contradiction:

Adaptability or versatilityVSPower

Solution Approach 1:

The patent creates a virtual display in volatile memory that copies the functionality of the physical display. The main display is redirected to this virtual display, allowing the system to render 2D mobile applications within the VR environment without requiring additional physical display hardware. This virtual copy enables 2D apps to operate seamlessly within the 3D VR space.

Inventive Principle:

Principle #26Copying

Solution Approach 2:

The patent transitions from a single physical display dimension to a multi-dimensional display architecture by introducing a virtual display layer. The virtual display in volatile memory acts as an intermediate layer between the mobile device's main display and the VR headset's display, enabling 2D content to be rendered in the 3D VR environment without consuming additional physical display resources.

Inventive Principle:

Principle #17Another dimension (Dimensionality change)

2Adaptability or versatility

If the main display is redirected to a virtual display in volatile memory, then 2D graphic content can be integrated into 3D VR environment, but the system complexity increases due to additional hardware composers and memory buffers

Engineering Contradiction:

Improveintegration of 2D interfaces into 3D VR sceneVSAvoidhardware composers and memory buffers

Core Design Contradiction:

Adaptability or versatilityVSDevice complexity

Solution Approach 1:

The virtual display in volatile memory serves multiple functions: it acts as the main display when in VR mode, provides a canvas for 2D mobile applications, and integrates with both the VR rendering pipeline and mobile application framework. This multi-functional approach consolidates several components into a single versatile element, reducing overall system complexity despite the added capability.

Inventive Principle:

Principle #6Universality (Multi-functionality)

Solution Approach 2:

The virtual display acts as an intermediary layer between the mobile device's main display hardware and the VR headset's display system. It mediates the redirection of main display content, allowing 2D applications to be composited into the 3D VR environment without requiring direct modifications to either the physical display or the VR rendering pipeline.

Inventive Principle:

Principle #24Intermediary (Mediator)

3Ease of operation

If the mobile device renders 3D VR environment and streams 2D mobile applications simultaneously, then the user experience is enhanced, but the computing resources are not efficiently utilized

Engineering Contradiction:

Improveuser experience in accessing 2D apps during VRVSAvoidcomputing resources efficiency

Core Design Contradiction:

Ease of operationVSUse of energy by moving object

Solution Approach 1:

The system performs preliminary redirection of the main display to the virtual display in volatile memory before launching 2D mobile applications within the VR environment. This preliminary setup ensures that the display pathway is already optimized for compositing 2D content into the 3D VR scene, improving resource efficiency by avoiding real-time display pathway changes during application execution.

Inventive Principle:

Principle #10Preliminary action

Data Source

PatentEP3580638B1System and method for two dimensional application usage in three dimensional virtual reality environment

Publication Date: 2023.10.25 SAMSUNG ELECTRONICS CO LTD

Figure 1
Figure 2
Figure 3

AI summary

An electronic mobile device having a physical display, at least one processor, a volatile memory, and non-volatile memory. Stored in the volatile memory are first instructions for generating a 2D user interface and second instructions for generating a 3D VR environment, and system instructions. In a first operation, the first instructions are executed to generate the 2D user interface and provide the 2D user interface to the physical display. In a second operation, the first instructions are executed to generate the 2D user interface and provide the 2D user interface to a virtual display in the volatile memory. Also, in the second operation, the second instructions are executed to generate the 3D VR environment so as to obtain the 2D user interface from the virtual display and render a 3D image using at least part of the 2D user interface, and provide the 3D image to the physical display.