Facial Landmark Processing for Sensor-Free Virtual Camera Control
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing remote collaboration systems require specialized equipment like VR headsets and powerful computing devices to track head movements, creating a gap between real and virtual worlds, which limits immersive collaboration experiences.
Innovation Solution
An image processing method and electronic device that analyze facial landmarks to control a virtual camera's object distance and viewing angle, converting a three-dimensional virtual scene to a two-dimensional image without additional sensors, using a neural network to estimate head poses and adjust the virtual camera's position based on facial features.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If VR headsets and tracking sensors are used to detect head pose, then head movement tracking precision is improved, but device complexity and cost increase
Solution Approach 1:
The patent uses a camera to capture images of the user's face and creates a digital model of facial landmarks, replacing the need for physical tracking sensors. The system copies the visual information from the real world into a digital representation that can be processed to determine head pose, thereby eliminating specialized tracking equipment while maintaining detection capability.
Solution Approach 2:
The patent replaces mechanical tracking sensors with an optical system using a camera and image processing algorithms. Instead of using physical sensors to detect head movements, the system uses visual information from captured images, processed through neural networks, to infer head pose and movement, substituting a mechanical detection system with an optical-computational one.
2Power
If powerful computing devices are used to ensure virtual application performance, then processing capability is improved, but device accessibility and ease of operation worsen
Solution Approach 1:
The patent performs head pose detection and virtual camera control on a frame-by-frame basis, processing only the necessary visual information rather than continuously analyzing all sensor data. This partial processing approach reduces computational requirements while maintaining functional effectiveness, making the system accessible on devices with moderate computing power.
Solution Approach 2:
The system uses the device's existing camera and processor to perform head pose detection and virtual camera control, leveraging resources already present in conventional devices rather than requiring additional specialized hardware. The device serves its own computational needs using built-in components, improving accessibility.
3Measurement precision
If specialized equipment is required for head pose detection, then detection accuracy is improved, but ease of operation and user accessibility deteriorate
Solution Approach 1:
The patent makes the head pose detection system universal by using a standard camera that is already present in most devices, rather than requiring specialized tracking equipment. The same camera used for general purposes is leveraged for head pose detection, allowing any device with a camera to participate in the virtual collaboration system without additional specialized equipment.
Data Source
AI summary
An image processing method includes the following steps. A plurality of facial landmarks of a face frame are analyzed. A feature width is calculated according to the facial landmarks, and a head pose is analyzed according to the facial landmarks. The head pose is utilized to update the feature width to generate an updated width. A scale ratio of the updated width to an initial width is calculated. An object distance of a virtual camera is controlled according to the scale ratio. A two-dimensional image is captured from a virtual scene according to the object distance of the virtual camera.