Portrait Relighting Using Scene Light and User Location
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
In video conferences, participants' diverse lighting conditions due to different environments create a strong sense of incongruity when synthesizing portraits with a background, undermining the realism of the online conference experience.
Innovation Solution
A relighting method that generates images based on user location and light source information, using neural networks to determine optical reflection and render images with accurate lighting, ensuring the lighting result conforms to the actual situation.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If portraits of participants are directly extracted and synthesized with a background scenario, then the synthesis process is simple and fast, but the lighting conditions are inconsistent creating a strong sense of incongruity
Solution Approach 1:
The patent applies preliminary action by pre-calculating and storing light reflection information (including normal vectors and reflectivity) for each participant's portrait before synthesis. This pre-processing enables the system to quickly apply appropriate lighting during synthesis while maintaining realistic lighting consistency, thus resolving the contradiction between synthesis speed and lighting consistency.
Solution Approach 2:
The patent applies local quality by adjusting lighting parameters specifically for each participant based on their location in the virtual scene. Different participants receive customized lighting adjustments according to their positions relative to virtual light sources, while the background remains unchanged. This localized approach maintains overall synthesis efficiency while achieving realistic lighting consistency for each individual.
2Manufacturing precision
If lighting parameters are adjusted for each participant based on location, then lighting realism is improved, but the computational complexity increases
Solution Approach 1:
The patent reduces computational complexity during synthesis by pre-calculating light reflection information (normal vectors, reflectivity) and storing them in advance. When synthesizing multiple participants, the system only needs to retrieve and apply these pre-computed values based on participant locations, avoiding repeated complex calculations and thus maintaining lighting realism while reducing real-time computational burden.
Solution Approach 2:
The patent uses copying by creating and storing light reflection information maps for each participant portrait. These pre-computed reflection maps are then copied and applied to the corresponding participants in the synthesized scene based on their locations. This copying approach avoids recalculating complex lighting interactions for each participant, reducing computational complexity while preserving lighting realism.
Data Source
Figure 1
Figure 2
Figure 3~4
AI summary
Embodiments of this application provide a relighting method. The method includes: obtaining a first image, a first location, and light source information (310), where the first image includes a portrait of a first user, the first location indicates location information of the first user in a preset scene, and the light source information includes a location of a light source in the scene and lighting information of the light source; determining first optical reflection information of the first user based on the first image and the first location (320), where the first optical reflection information indicates a light reflection status of a body surface of the first user at the first location; and obtaining a second image through rendering based on the first image, the light source information, and the first optical reflection information (330), where the second image includes a portrait obtained through relighting on the first user at the first location in the scene based on the light source information. In the technical solutions of this application, a relighted portrait of a user can be generated based on location information of the user, so that a lighting result of the generated image conforms to an actual situation, and a sense of reality of an online conference is improved.