Text-to-Image Model Fine-Tuning With Performance-Aware Captions

The patent changes the training parameters and objectives of the text-to-image model by incorporating performance metrics (click-through rates, conversion rates) alongside visual quality metrics. This allows the model to learn to generate images that satisfy both aesthetic requirements and performance goals simultaneously, resolving the contradiction between visual quality and performance metrics.

The patent introduces an intermediary component that bridges visual quality and performance metrics. This intermediary mechanism enables the model to understand the relationship between aesthetic attributes and performance outcomes, allowing it to generate images that achieve both visual appeal and desired performance metrics rather than treating them as opposing objectives.

The patent implements self-service by using an image-to-text model to automatically generate captions for images in the training pool. This eliminates the need for manual captioning or pre-existing captions, allowing the system to independently create its own training data annotations and significantly expand the available training data without external assistance.

The patent applies preliminary action by pre-generating captions for all training images using the image-to-text model before the actual model training begins. This preliminary caption generation prepares the training data in advance, ensuring that both visual and textual data are ready for efficient model training without requiring real-time caption generation during the training process.