Personalized Image Generation With Cross-Attention Identity Preservation

Resolve Bottlenecks,
Find Innovative SolutionsGenerate Solutions

Solution Overview

Problem

Existing technologies face challenges in addressing the limitations of existing technologies in preserving the identity of subjects in images while incorporating textual prompts and enhancing the diversity of applications in digital art, such as the challenges of existing technologies in the field of digital art, design, and entertainment.

Innovation Solution

The proposed solution introduces a method for generating a target image from a reference image, such as the proposed method for generating a reference image, such as the proposed method for generating a target image from a reference image, such as the proposed method for generating a reference image from a reference image, such as the proposed method for generating a reference image from a reference image from a textual description, such as the proposed method for generating a text-to-image model from a reference image from a reference image.

Engineering Contradictions & Design Principles

VSEngineering Contradiction Analysis

1Measurement precision

If personalization models become more specific to a corresponding identity, then identity preservation is improved, but the model's ability to generalize to new identities deteriorates

Engineering Contradiction:

Improveidentity preservationVSAvoidgeneralization to new identities

Core Design Contradiction:

Measurement precisionVSAdaptability or versatility

Solution Approach 1:

The model is segmented into a generalization component and a personalization component. The personalization component is trained on reference images of a specific identity to preserve identity characteristics, while the generalization component handles diverse identities. This segmentation allows the model to specialize in identity preservation without sacrificing generalization capability.

Inventive Principle:

Principle #1Segmentation

Solution Approach 2:

A cross-attention mechanism serves as an intermediary between the reference image features and the text prompt features. This intermediary allows the model to selectively attend to identity-relevant features from the reference image while still incorporating information from the text prompt, thereby preserving identity without overfitting to a single identity.

Inventive Principle:

Principle #24Intermediary (Mediator)

2Stability of the object's composition

If the model preserves identity from reference images, then identity consistency is improved, but the diversity of generated images deteriorates

Engineering Contradiction:

Improveidentity consistencyVSAvoidimage diversity

Core Design Contradiction:

Stability of the object's compositionVSAdaptability or versatility

Solution Approach 1:

The model applies different quality requirements to different parts of the generated image. Identity-critical regions (such as facial features) maintain high consistency with the reference image, while other regions (such as background, clothing, pose) allow for greater diversity based on the text prompt. This local quality differentiation resolves the contradiction between identity consistency and image diversity.

Inventive Principle:

Principle #3Local quality

Solution Approach 2:

The attention weights in the cross-attention mechanism are dynamically adjusted based on the input text prompt and reference image. This allows the model to flexibly balance between identity preservation and diversity generation depending on the specific requirements of each generation task, rather than applying a fixed level of identity constraint.

Inventive Principle:

Principle #15Dynamics

3Measurement precision

If the model is tuned for specific identities, then personalization accuracy is improved, but the complexity of deployment deteriorates

Engineering Contradiction:

Improvepersonalization accuracyVSAvoiddeployment complexity

Core Design Contradiction:

Measurement precisionVSDevice complexity

Solution Approach 1:

The model architecture is designed to be universal and multi-functional, capable of handling multiple identities without requiring separate model instances. The personalization is achieved through input references rather than model parameters, allowing the same model to be deployed for generating images of any identity by simply providing the appropriate reference image, thereby reducing deployment complexity.

Inventive Principle:

Principle #6Universality (Multi-functionality)

Solution Approach 2:

Instead of copying and fine-tuning the entire model for each identity, the system copies only the necessary identity characteristics from reference images through feature extraction and cross-attention mechanisms. This approach maintains personalization accuracy while avoiding the computational overhead and complexity of multiple model copies.

Inventive Principle:

Principle #26Copying

Data Source

PatentUS20260004489A1Methods, apparatuses and computer program products for providing tuning-free personalized image generation

Publication Date: 2026.01.01 META PLATFORMS INC

AI summary

A system and method to generate a target image from a reference image are provided. The system may receive, via a LDM, a reference image and a text prompt. The system may extract, via a trained vision encoder in the LDM, a vision control signal from an object in the reference image. The vision control signal indicates an identity of the object. The system may extract, via trained text encoders in the LDM, text control signals associated with the text prompt. The system may generate, via cross attention summation of an output of a vision cross attention unit(s) associated with the vision control signal and an output of text cross attention units associated with the text control signals, spatial features indicative of the reference image and the text prompt. The system may output, via a decoder in communication with the LDM, a target image based on the generated spatial features.