Virtual Background Lighting Mapping for Realistic Video Call Portraits
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Conventional video call scenes fail to provide realistic lighting and shadow effects on participants within virtual backgrounds, as they merely preserve lighting from the participant's actual background, leading to unrealistic appearances.
Innovation Solution
A scene generation system identifies light sources in a 360-degree virtual background, applies lighting and shadow effects to the participant's body and background based on their orientation and position relative to these sources, using 3D graphics models to create a realistic virtual environment.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Illumination intensity
If conventional video call scenes preserve lighting from the participant's actual background, then the participant's appearance is maintained, but the lighting and shadow effects are unrealistic within the virtual background
Solution Approach 1:
The system creates a copy of the participant's video feed and applies virtual lighting effects to this copy rather than modifying the original. This allows realistic lighting and shadow effects to be superimposed on the participant's appearance without altering the actual captured image, resolving the contradiction between maintaining appearance fidelity and achieving realistic lighting
Solution Approach 2:
The system introduces an intermediary processing layer that analyzes the virtual background's lighting conditions and applies corresponding lighting effects to the participant's video feed. This intermediary process enables the integration of realistic lighting from the virtual environment onto the participant without directly modifying the original video capture
2Manufacturing precision
If a 360-degree virtual background with light sources is implemented, then realistic lighting effects are achieved, but the system complexity increases
Solution Approach 1:
The system performs preliminary analysis of the 360-degree virtual background to identify light source positions, intensities, and characteristics before processing the participant's video feed. This pre-processing step allows the complex lighting calculations to be prepared in advance, reducing real-time computational complexity while maintaining realistic lighting effects
Solution Approach 2:
The system segments the virtual background processing into distinct components: light source identification, lighting effect generation, and application to video feed. This segmentation allows each complex task to be handled by specialized processing modules, managing overall system complexity through modular architecture
Data Source
AI summary
A system applies an object detection model to a mapped graphical spherical object including a 360 degree virtual background image mapped to a spherical graphical model to identify a light source within the mapped graphical spherical object. The system generates a light source mapped graphical spherical object by replacing, in the mapped graphical spherical object, the light source with a light source object associated with one or more light properties. The system extracts, during a video call, a human body object from a live video feed captured via a user computing device. The system generates, based on the light source mapped graphical spherical object and the human body object, a finalized spherical live image including applying one or more lighting and/or shadow effects caused by the light source object within the light source mapped graphical spherical object. The system generates, from the finalized spherical image, a finalized rectangular live image.