Pose-Based User Image Scaling for Virtual Reality Backgrounds
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
In virtual reality environments, the upper part of a user's face is blocked by a headset, preventing full visibility of their 3D face, and existing methods for determining the scale of a user's image in a virtual environment often result in inaccurate sizing due to incorrect assumptions about the user's posture.
Innovation Solution
An image processing apparatus and method that uses a machine learning algorithm to accurately determine the scale of a user's image in a virtual environment by extracting human pose landmarks, preprocessing the data, and applying a neural network to infer the correct scale based on a user's actual height, ensuring proper sizing and visibility in the virtual environment.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Ease of operation
If a headset is positioned on the user's face to enable virtual reality viewing, then the user can access virtual content, but the upper part of the face is blocked and cannot be seen
Solution Approach 1:
The patent extracts the blocked upper face region from the captured image and generates a synthetic completion of this region. The system removes the headset obstruction by separating the visible face region from the blocked region, then fills in the missing upper face portions using generative models trained on facial data, resulting in a complete 3D face representation without the headset blocking the view.
2Measurement precision
If manual input of user height is required to determine scale in virtual environment, then scaling can be achieved, but user input time is needed and may not be available in real-time scenarios
Solution Approach 1:
The system automatically determines user scale by analyzing the captured image itself. The processor detects the user's pose, identifies reference objects or landmarks within the image, and calculates the user's dimensions relative to these references. This self-service approach eliminates the need for manual height input while maintaining accurate scaling in the virtual environment.
3Productivity
If existing scale determination methods are used in virtual reality, then image scaling can be performed, but inaccurate sizing occurs due to incorrect assumptions about user posture
Solution Approach 1:
The patent changes the parameters used for scale determination by incorporating pose information and multiple reference points into the calculation. Instead of using fixed assumptions about user posture, the system adapts the scaling parameters based on the actual detected pose and spatial relationships in the captured image, leading to more accurate scale measurements that account for various user positions and orientations.
Data Source
AI summary
An information processing apparatus is provided and includes one or more memories storing instructions; and one or more processors configured to execute the instructions stored in the memory to perform operations including receiving a captured image of a user at a first pose, extracting information of landmarks of the user in the captured image, obtaining information indicating a size of a user at a predetermined pose, based on the extracted information, determining a scale of an image of the user at the first pose based on the obtained information, and locating the determined scale of the image of the user at the first pose in a background image.