Mixed Reality Light Source Detection via 3D Vector Clustering
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
In mixed reality (MR) technology, existing methods for generating real-world light data fail to accurately represent light conditions, leading to unrealistic virtual objects when they are close to light sources, as they receive inconsistent lighting and cannot generate shadows matching actual light environments.
Innovation Solution
A method for detecting real-world light sources in MR systems involves analyzing light regions in continuous images, projecting them into three-dimensional vectors, clustering to predict directional light, and calculating intersection points to identify point lights, allowing for accurate light data generation that mimics real-world conditions.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If real-world light data is generated based on average image intensities and color temperatures of the current frame, then the computation is simple and fast, but the virtual object receives the same light at any position and cannot generate realistic shadows
Solution Approach 1:
The patent segments the image into multiple light regions by analyzing brightness values and identifying continuous bright areas. This segmentation allows the system to treat different light sources separately rather than using a single average value, enabling accurate determination of multiple light sources including directional lights and point lights while maintaining computational efficiency
Solution Approach 2:
The patent projects two-dimensional light regions from images into three-dimensional space by calculating three-dimensional vectors for each light region. This dimensional transformation allows the system to determine spatial positions and directions of light sources in 3D space, enabling realistic lighting effects that account for object positions and shadow directions
2Device complexity
If light data is determined using average intensities and color temperatures, then the processing is simple, but the virtual object cannot receive position-dependent lighting and shadow generation is inaccurate
Solution Approach 1:
The system segments light regions by analyzing brightness values and identifying continuous bright areas in the image. This segmentation enables the detection of multiple distinct light sources and their properties, allowing the virtual object to receive position-dependent lighting and generate accurate shadows while keeping processing relatively simple
Solution Approach 2:
The patent changes the lighting parameters from a single average intensity and color temperature to multiple light source parameters including position, direction, intensity, and color temperature for each detected light source. This parameter transformation enables realistic position-dependent lighting and accurate shadow generation
Data Source
AI summary
A method, a mixed reality (MR) system, and a recording medium for detecting a real-world light source in MR are provided. The method is applicable to the MR system having a computing apparatus and a head-mounted display (HMD). In the method, a plurality of light regions of a plurality of continuous images is analyzed, and each of the light regions is projected according to a coordinate position of the HMD corresponding to the plurality of continuous images to generate three-dimensional vectors. Then, the three-dimensional vectors are clustered to predict a directional light in a current image. Finally, a plurality of intersection points between the three-dimensional vectors is calculated, and a point light in the current image is predicted according to the intersection points.