Normal Image Data Integration for Natural Lighting Correction
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing methods for generating images under desired illumination conditions, such as those using 3D models, often result in unnatural outcomes due to the need for pre-prepared models that match various poses and expressions, leading to large data requirements and difficulty in selecting appropriate models.
Innovation Solution
An image processing apparatus that generates normal image data with varying smoothness degrees, integrates this data, and performs lighting processing to simulate desired illumination conditions, allowing for natural image correction without the need for extensive 3D model preparation.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Adaptability or versatility
If 3D models are prepared in advance to cover various poses and expressions, then the image can be generated under desired illumination conditions, but the amount of data becomes very large and it is difficult to select appropriate models
Solution Approach 1:
The patent extracts only the essential normal image data from 3D models rather than using complete 3D models. By taking out and utilizing only the normal information (surface orientation data) needed for lighting calculation, the system achieves illumination adaptation without requiring full 3D model data, thus reducing data quantity while maintaining functionality.
Solution Approach 2:
The patent creates a universal approach by generating normal image data from a single 3D model that can be applied to multiple subjects and poses. The normal data generation process is designed to be pose-invariant, allowing the same processed normal data to serve various illumination correction scenarios without requiring separate models for each pose or subject.
2Manufacturing precision
If 3D models are replaced with subject images, then the subject image can be used for lighting correction, but the pose and expression of the subject are replaced with the 3D model, resulting in unnatural results
Solution Approach 1:
The patent applies local quality by processing normal image data differently for different regions. It generates normal data specifically for the subject area while maintaining the original subject image for areas where 3D model data is unavailable or inappropriate. This allows lighting correction to be applied locally where needed while preserving the natural appearance of the subject in other regions.
Solution Approach 2:
The patent introduces normal image data as an intermediary between the 3D model and the final image. Instead of directly replacing the subject image with 3D model renders, the system uses normal data from 3D models as a mediator to calculate lighting effects while keeping the original subject image intact, thus avoiding the unnatural appearance problem.
3Adaptability or versatility
If multiple 3D models with different poses and expressions are prepared, then various illumination conditions can be simulated, but the device complexity increases
Solution Approach 1:
The patent merges multiple 3D models into a single unified approach by extracting and processing normal image data from all models, then integrating this data into a comprehensive normal image database. This consolidation allows the system to handle various poses and expressions through a single integrated structure rather than managing multiple separate models, reducing device complexity while maintaining versatility.
Data Source
AI summary
An image processing apparatus according to the present invention includes: a generation unit configured to generate normal image data recording normal information for each of a plurality of areas at least part of which is different from one another in image data and for each pixel of the area; a setting unit configured to set one of at least two or more smoothness degrees different from one another to each of a plurality of pieces of generated normal image data; a smoothing unit configured to smooth each of the plurality of pieces of normal image data in the set smoothness degree; an integration unit configured to generate integrated normal image data corresponding to pixel arrangement of the image data by integrating the plurality of pieces of smoothed normal image data; and a lighting processing unit configured to perform lighting processing for the image data based on the integrated normal image data.