Head Mount Display Screen Overlay for External Terminal Interaction
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Users wearing head mount displays without cameras face challenges in operating external information terminals, as they cannot directly view their surroundings or perform tasks like answering calls or inputting on screens without removing the device, due to obstructed vision and lack of camera functionality.
Innovation Solution
An information processing apparatus that includes an image capturing unit, a position/posture calculation unit, and a position/posture transform unit, which calculates and transforms the position and posture of the head mount display to align screen information from an external terminal, allowing users to view and interact with the terminal's screen without removing the head mount display by capturing images of the terminal and transforming its display on the head mount display's screen.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If a light-shielding head mount display is used to increase immersive feeling, then the viewing experience is improved, but the user cannot observe surrounding scenery or perform operations to the outside
Solution Approach 1:
The patent displays a copied image of the external information terminal's screen on the head mount display's own display unit. The image capturing unit captures the terminal screen, and this captured image is then displayed on the head mount display, allowing the user to interact with the terminal without removing the device. This copying approach resolves the contradiction by providing operational capability while maintaining the light-shielding immersive environment.
Solution Approach 2:
The head mount display acts as an intermediary between the user and the external information terminal. Instead of directly viewing and operating the terminal, the user interacts with a reproduced image of the terminal screen displayed on the head mount display. This intermediary mechanism allows operation of external devices while maintaining the light-shielding display mode.
2Reliability
If the head mount display restricts the sense of sight to provide immersive viewing, then the viewing experience is improved, but the user cannot directly view or interact with external devices
Solution Approach 1:
The system captures and reproduces the screen information from the external information terminal on the head mount display's display unit. This copying process transfers information from the external terminal to the head mount display, allowing the user to access external information without breaking the immersive viewing environment. The captured image is then displayed, preserving both the immersive experience and access to external information.
3Device complexity
If no camera is equipped on the head mount display, then the device complexity is reduced, but the user cannot capture external images for operation
Solution Approach 1:
Instead of equipping the head mount display with a camera to capture external images, the patent inverts the approach by using the external information terminal's display screen as the imaging target. The head mount display's display unit shows a reproduced image of the terminal screen, allowing operation without requiring the head mount display to have camera functionality. This inversion resolves the contradiction by achieving operational capability without adding camera equipment.
Data Source
AI summary
Provided is an information processing apparatus that provides screen information to be displayed on an image display apparatus that is worn by a user on the head or face and used for viewing an image. On the basis of an image of a head mount display (HMD) captured by a camera mounted on an information terminal, the mutual positional relationship and posture between the HMD and the information terminal are grasped, and screen information transmitted from the information terminal is superimposed and displayed in a contents reproducing screen of the HMD being viewed with an appropriate posture at an appropriate position on the reproducing screen. Since only the information terminal is present with the same position and posture as in the real world in the contents reproducing screen, a user can perform a screen operation of the information terminal with the HMD on.