Personalized Try-On Ads Using 2D Image Segmentation
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current augmented reality systems in messaging applications face limitations due to the need for depth sensors, which increase costs and complexity, and struggle to accurately replace backgrounds in images that include the whole body of a user or multiple users, leading to poor image quality and failure in applying visual effects to clothing articles.
Innovation Solution
The system processes images without depth sensors by identifying and segmenting fashion items worn by users, replacing them with corresponding products from a database, and displaying personalized advertisements that depict the user wearing the advertised product, enhancing engagement and reducing system resource usage.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If depth sensors are used in augmented reality systems, then background replacement accuracy is improved, but device cost and complexity increase
Solution Approach 1:
The patent extracts and removes the depth sensor component from the AR system, replacing it with a solution that uses only standard 2D camera imaging. The invention achieves background replacement without relying on depth information, thereby simplifying the device while maintaining functionality.
Solution Approach 2:
The patent creates a virtual copy of the user's body and clothing from 2D images, then applies virtual clothing to this copied model. This copying approach eliminates the need for depth sensors while still achieving realistic virtual try-on effects through image processing and rendering.
2Manufacturing precision
If depth sensors are used in augmented reality systems, then image processing accuracy is improved, but manufacturing cost increases
Solution Approach 1:
The patent replaces expensive depth sensing hardware with inexpensive 2D camera imaging and software-based processing. The solution uses computationally intensive but hardware-simple approaches, trading processing power for hardware simplicity to reduce manufacturing costs.
Solution Approach 2:
The patent substitutes the mechanical depth sensing system with a software-based image processing system. Instead of using physical depth sensors to capture 3D information, the system uses 2D images combined with virtual modeling and rendering algorithms to achieve the same effect.
3Device complexity
If traditional background replacement methods are used without depth sensors, then device complexity is reduced, but image quality deteriorates
Solution Approach 1:
The patent transitions from 2D image processing to a pseudo-3D representation by creating virtual 3D models of the user and clothing from 2D inputs. This dimensional transformation allows the system to achieve realistic depth and lighting effects without requiring actual depth sensing hardware.
Solution Approach 2:
The patent introduces a virtual clothing model and virtual body model as intermediary elements between the user and the final rendered image. These virtual models serve as mediators that enable realistic clothing visualization without requiring direct depth measurement of the user.
4Ease of manufacture
If traditional background replacement methods are used without depth sensors, then manufacturing cost is reduced, but advertising personalization effectiveness deteriorates
Solution Approach 1:
The patent performs preliminary actions by creating and storing virtual clothing models and body models in advance. These pre-prepared virtual assets enable rapid personalization of advertisements without requiring complex real-time processing, thus maintaining effectiveness while reducing costs.
Data Source
AI summary
Methods and systems are disclosed for performing operations comprising: accessing content received from a first client device associated with a first user; processing the content to identify a first image that depicts the first user wearing a first fashion item; determining a first pose of the first user depicted in the first image; searching a plurality of products to identify a first product that corresponds to the first pose of the first user depicted in the first image and that includes one or more attributes associated with the first fashion item; modifying the first image to generate an advertisement that depicts the first user wearing the first product; and during a content browsing session being accessed by the first client device, causing the advertisement that depicts the first user wearing the first product to be displayed automatically on the first client device.