Adversarial Image Generation Model Using Weakly Correlated Data

Resolve Bottlenecks,
Find Innovative SolutionsGenerate Solutions

Solution Overview

Problem

Existing text-to-image generation methods struggle to produce images that closely resemble real-life scenarios due to their reliance on strongly correlated image-text data and multistage generators, which are difficult to implement in practical settings.

Innovation Solution

The proposed method uses weakly correlated image-text data pairs to train an image generation model based on an adversarial network, allowing for the generation of images that better match real-life scenarios through a single-stage end-to-end training process.

Engineering Contradictions & Design Principles

VSEngineering Contradiction Analysis

1Manufacturing precision

If multistage text-to-image generation methods are used with strongly correlated image-text data, then fine-grained alignment between text and image is improved, but the complexity of implementation and requirement for fine-grained semantic object labels increases significantly

Engineering Contradiction:

Improvefine-grained alignment precisionVSAvoidimplementation complexity

Core Design Contradiction:

Manufacturing precisionVSDevice complexity

Solution Approach 1:

The patent segments the image generation process into multiple stages (semantic layout inference stage and image generation stage), where each stage focuses on specific tasks. The semantic layout module first infers object bounding boxes and segmentation masks, then the image generation module creates the final image based on these layouts, achieving fine-grained alignment without requiring complex end-to-end training

Inventive Principle:

Principle #1Segmentation

Solution Approach 2:

The patent performs preliminary semantic layout inference before image generation. The semantic layout module pre-processes the text description to obtain object bounding boxes, segmentation masks, and hierarchical relationships, which then guide the subsequent image generation process. This preliminary action simplifies the overall system by breaking down the complex task into manageable preprocessing and generation steps

Inventive Principle:

Principle #10Preliminary action

2Manufacturing precision

If strongly correlated image-text data pairs are used for training, then semantic alignment between text and image is improved, but the applicability to real-life scenarios deteriorates due to the artificial nature of such data

Engineering Contradiction:

Improvesemantic alignment precisionVSAvoidreal-life scenario applicability

Core Design Contradiction:

Manufacturing precisionVSAdaptability or versatility

Solution Approach 1:

The patent inverts the traditional approach by using weakly correlated image-text data pairs instead of strongly correlated ones. Rather than forcing the model to learn from artificially aligned data, the system is trained on natural, diverse data from the web where text and image relationships are more varied and realistic, improving adaptability to real-life scenarios

Inventive Principle:

Principle #13The other way round (Inversion)

Solution Approach 2:

The patent changes the data correlation parameter from strong to weak correlation. By training on weakly correlated data pairs collected from web sources, the model learns to handle more diverse and realistic text-image relationships, making it more adaptable to real-life applications while maintaining acceptable alignment quality through the multistage architecture

Inventive Principle:

Principle #35Parameter changes

3Reliability

If conventional text-to-image generation methods are used, then semantic consistency is improved, but the images generated are not close to real-life scenarios due to the artificial training data

Engineering Contradiction:

Improvesemantic consistencyVSAvoidrealism of generated images

Core Design Contradiction:

ReliabilityVSManufacturing precision

Solution Approach 1:

The patent employs a GAN-based image generation module that serves multiple functions: it generates images based on semantic layouts, learns from diverse weakly correlated data, and produces realistic outputs suitable for various applications. This universal module handles both semantic consistency and visual realism through adversarial training, making the system adaptable to different real-life scenarios

Inventive Principle:

Principle #6Universality (Multi-functionality)

Data Source

PatentUS20250069280A1Image generation method and apparatus, and device and medium

Publication Date: 2025.02.27 LANGCHAO ELECTRONIC INFORMATION IND CO LTD

AI summary

An image generating method and apparatus, and a device and a medium are disclosed. The method comprises: acquiring weakly correlated image-text data pairs, and creating an image-text data set according to the weakly correlated image-text data pairs, wherein the weakly correlated image-text data pairs are image-text data pairs in which images and texts have weak correlations (S11); training, by using the image-text data set, an image generation model which is preconstructed on the basis of an adversarial network, so as to obtain a trained image generation model, wherein the image generation model includes a generator for generating an image, and a discriminator for identifying the authenticity of the image and calculating a corresponding loss value (S12); and after when text data to be processed has been acquired, generating, by using the trained image generation model, an image corresponding to the said text data (S13).