Debiasing Text-to-Image Diffusion Models via Attribute Distribution Feedback
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current Text-to-Image (TTI) models exhibit bias in generated images, leading to imbalanced distributions of attributes such as color, which deviate from natural occurrences.
Innovation Solution
The method involves obtaining multiple images from an image generating model based on a prompt, determining the attributes of these images, and updating the model to align with a predetermined distribution of attributes, thereby reducing bias.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If Text-to-Image models are used for image generation, then image generation capability is achieved, but attribute distribution bias occurs
Solution Approach 1:
The patent implements a feedback mechanism where the generated images are evaluated for attribute distribution, and the model parameters are updated based on the evaluation results. The loss function computes the difference between predicted and target attribute distributions, guiding the model to generate images with more accurate attribute distributions while maintaining generation capability.
Solution Approach 2:
The patent modifies the model parameters through iterative optimization based on attribute distribution evaluation. By adjusting the parameters to minimize the loss function that measures distribution discrepancy, the model transforms from generating biased images to generating images with balanced attribute distributions, resolving the contradiction between generation capability and distribution accuracy.
2Measurement precision
If multiple images are generated to evaluate attributes, then attribute distribution can be assessed, but computational cost increases
Solution Approach 1:
The patent generates multiple images per iteration to sufficiently assess attribute distribution, accepting increased computational cost as necessary to achieve accurate measurement. This partial excessive action ensures that the attribute distribution evaluation is based on sufficient sample size, enabling precise assessment and effective model optimization.
3Manufacturing precision
If the model is updated to reduce bias, then attribute distribution accuracy improves, but model complexity increases
Solution Approach 1:
The patent incorporates attribute distribution evaluation and loss computation as preliminary steps in the training process. By pre-defining the loss function that measures distribution discrepancy and integrating it into the optimization objective, the system systematically reduces bias through structured model updates, improving attribute distribution accuracy through a methodical approach that manages complexity.
Data Source
AI summary
There are provided methods, devices, and computer program products for image generation, particularly to debiasing text-to-image diffusion models. In a method, a plurality of images are obtained by an image generating model based on a prompt. The plurality of images comprises a plurality of instances of an object, respectively and the object is specified by the prompt. A plurality of attributes of the plurality of instances of the object are determined respectively. The image generating model is updated based on the plurality of attributes and a predetermined distribution of a plurality of predetermined attributes related to the object. With the above method, the images generated by the updated image generating model may follow the predetermined distribution, and the updated image generating model may output debiased results.