Layered Image Generation Using Object Masks and Text Regeneration
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing text-to-image models generate flat images that lack transparency and require manual editing to create layered images, leading to inefficiencies and resource-intensive training when handling alpha-channel images.
Innovation Solution
A method using two trained machine learning models to generate and segment a single-layer image into multiple layers, ensuring consistency and transparency without requiring alpha-channel training, by segmenting objects and generating masks to combine images into multi-layer images.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Ease of manufacture
If existing text-to-image models generate flat images, then the models can be trained with standard images without alpha-channel requirements, but the images require manual segmentation and editing to create layered images
Solution Approach 1:
The patent segments the generated image into multiple layers by identifying different objects and their spatial relationships. The system divides the single-layer image into separate transparent layers for each object, enabling independent manipulation and compositing while maintaining the original generation simplicity.
Solution Approach 2:
The patent transitions from two-dimensional flat images to multi-layered transparent images by adding the alpha channel dimension. This allows the images to be composed of multiple transparent layers that can be independently manipulated, effectively adding a new dimension of control without requiring retraining of the base generation model.
2Productivity
If models are trained on alpha-channel images to generate transparent pixels, then layered images can be generated directly, but the training becomes resource-intensive and complex
Solution Approach 1:
The patent extracts the transparency generation task from the main image generation model. Instead of training the base model to handle alpha channels, the system extracts objects from generated images and creates separate transparent layers, separating the generation and segmentation functions to reduce training complexity.
Solution Approach 2:
The patent introduces an intermediary segmentation process between image generation and final layered output. This intermediary step processes the generated images to create masks and separate layers, acting as a mediator that converts flat images into layered format without requiring the generation model itself to handle transparency.
3Ease of operation
If objects are segmented from generated images to create layers, then transparency and layering are achieved, but consistency between layers may be compromised
Solution Approach 1:
The patent performs preliminary actions by generating the complete single-layer image first, then systematically segments it into layers. This ensures that all objects are consistently generated in their final form before segmentation, maintaining appearance consistency while enabling easy layer creation through subsequent masking and separation operations.
Data Source
AI summary
Systems and methods are described for generating, using the first trained machine learning model and based on text input, a single-layer image comprising a plurality of objects; generating a plurality of masks associated with the plurality of objects; determining a plurality of attributes associated with the plurality of objects; generating, using a second trained machine learning model, a plurality of textual descriptions respectively corresponding to the plurality of objects; inputting the plurality of textual descriptions, and the plurality of attributes, to the first trained machine learning model; generating, using the first trained machine learning model, a plurality of images respectively corresponding to the plurality of textual descriptions; and generating the multi-layer image by combining the plurality of images and by using the plurality of masks, wherein the plurality of images respectively correspond to a plurality of layers of the multi-layer image.