Multilingual Image Generation Fine-Tuning With Reward Models

Resolve Bottlenecks,
Find Innovative Solutions
Generate Solutions

Solution Overview

Problem

Existing image generation models struggle to effectively train on non-specific languages, leading to suboptimal performance when user input text is not in a supported language.

Innovation Solution

A method involving a reference text set in multiple languages, conversion of text to different languages, and a reward model trained on labeled image quality information to fine-tune the image generation model, enhancing its performance across languages.

Engineering Contradictions & Design Principles

VSEngineering Contradiction Analysis

1Productivity

If the image generation model is trained on a single language, then the training process is simple and fast, but the model performance degrades when processing texts in unsupported languages

Engineering Contradiction:
Improvetraining efficiencyVSAvoidlanguage support capability
Core Design Contradiction:
ProductivityVSAdaptability or versatility

Solution Approach 1:

The model is trained to handle multiple languages simultaneously by incorporating texts from different languages during the training phase. This enables the single model to function universally across multiple languages without requiring separate models for each language, thus improving adaptability while maintaining training efficiency.

Inventive Principle:
Principle #6Universality (Multi-functionality)

Solution Approach 2:

The model performs preliminary learning of language-specific features and patterns during the training phase by exposing it to diverse language texts. This preliminary action equips the model with the ability to handle various languages before actual deployment, preventing performance degradation when encountering unsupported languages.

Inventive Principle:
Principle #10Preliminary action

2Adaptability or versatility

If the model is trained on multiple languages from the beginning, then language support is comprehensive, but the training complexity and resource requirements increase

Engineering Contradiction:
Improvemulti-language supportVSAvoidtraining system complexity
Core Design Contradiction:
Adaptability or versatilityVSDevice complexity

Solution Approach 1:

The training process is segmented into distinct phases: initial pre-training on diverse language data to learn general patterns, followed by fine-tuning on specific language pairs. This segmentation reduces the complexity of training a multi-language model from scratch by breaking it into manageable stages with different complexity requirements.

Inventive Principle:
Principle #1Segmentation

Solution Approach 2:

The training approach dynamically adjusts the complexity based on the stage: starting with simpler pre-training on diverse data, then progressing to more complex fine-tuning on specific language pairs. This dynamic approach allows the system to achieve multi-language support while managing computational resources efficiently at each stage.

Inventive Principle:
Principle #15Dynamics

Data Source

PatentUS20260065651A1Method, device, and storage medium for image generation
Publication Date: 2026.03.05 BEIJING ZITIAO NETWORK TECH CO LTD
  • US20260065651A1 patent drawing
  • US20260065651A1 patent drawing
  • US20260065651A1 patent drawing

AI summary

The embodiment of the invention provides a method and device for image generation, equipment and a storage medium. The method includes obtaining a reference text set indicating an image generation objective, the reference text set including text in multiple languages. Generating at least one reference image based on the first text in the first language in the reference text set by using the image generation model. The first text is converted to a second text in the second language, the second language being different from the first language. The reward model is trained based on the first text, the second text, the at least one reference image, and the labeled information for the at least one reference image, the labeled information indicates an image quality of the at least one reference image, and the reward model is configured to fine tune the image generation model.