Photorealistic Cosmetic Rendering via Neural Parameter Extraction
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current methods for generating photorealistic renderings of cosmetic products in augmented reality are resource-intensive, limited to predefined products, and struggle with real-time rendering on portable devices, often failing with unusual colors and requiring complex neural networks or tedious configuration of physically based rendering engines.
Innovation Solution
A method using an encoding artificial neural network to determine characterizing parameters of cosmetic products from reference images, combined with a realistic physically based rendering engine, allows for photorealistic rendering of various cosmetic products on users, even with limited computing resources, enabling real-time rendering on devices like smartphones.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Adaptability or versatility
If complex artificial neural networks are used to extract cosmetic product appearance from reference images, then the ability to handle unusual colors and product varieties is improved, but computing resources and execution time increase significantly
Solution Approach 1:
The system segments the complex rendering task into two parts: (1) an encoding neural network that extracts characterizing parameters from reference images, and (2) a physics-based rendering engine that generates photorealistic renderings using these parameters. This segmentation allows the neural network to be simple and fast while maintaining versatility through the physics-based renderer.
Solution Approach 2:
The patent introduces characterizing parameters as an intermediary between the reference image and the photorealistic rendering. The encoding neural network extracts these parameters from the reference image, and the physics-based rendering engine uses them to generate the final rendering. This intermediary approach decouples the complexity of handling diverse cosmetic products from the rendering process itself.
2Manufacturing precision
If rendering engines based on physical principles are used to generate realistic renderings, then rendering quality is improved, but configuration complexity and difficulty increase due to needing to define characterizing parameters
Solution Approach 1:
The system makes the rendering engine self-configuring by using the encoding neural network to automatically extract characterizing parameters from reference images. Users simply provide reference images, and the system automatically determines the appropriate parameters for the physics-based rendering engine, eliminating manual configuration complexity.
Solution Approach 2:
The encoding neural network performs preliminary extraction of characterizing parameters from reference images before the rendering process begins. This preliminary action prepares the necessary configuration data for the physics-based rendering engine, so that the actual rendering can proceed without manual parameter definition.
3Productivity
If complex artificial neural networks are implemented on portable appliances, then real-time rendering capability is improved, but device resource requirements and power consumption increase
Solution Approach 1:
The patent replaces the need for complex, resource-intensive neural networks with a simpler encoding network that extracts parameters quickly, combined with a pre-configured physics-based rendering engine. This approach achieves real-time rendering on portable devices without requiring powerful hardware, effectively using a simpler, more energy-efficient solution.
Data Source
AI summary
According to one aspect, what is proposed is a method for generating a photorealistic rendering of a cosmetic product, comprising: —obtaining (10, 12) a reference image (Xref) of a real cosmetic product (PC) applied to a first person (P1) and at least one source image (Xjsource) of a second person (P2), —implementing (13) an encoding artificial neural network (E) configured to determine characterizing parameters (E(Xref)) of the cosmetic product (PC) from the reference image (Xref), and then —implementing (14) a realistic physically based rendering engine (R) configured to generate a transformed image (R (Xjsource, E(Xref))) in which a photorealistic rendering of the cosmetic product (PC) is applied to the person (P2) from said at least one source image (Xjsource) based on the characterizing parameters (E (Xref)) of the cosmetic product (PC) that are determined by the encoding artificial neural network (E).