A fine-grained visual illusion image generation method based on region-aware phase injection

Abstract

The application discloses a kind of based on regional perception phase injection fine-grained visual illusion image generation method, the method comprises the following steps: step 1, the automatic construction of adaptive phase bank;Step 2, text-driven natural control interface analysis and instruction generation;Step 3, phase query attention execution and structure injection.The method supports accurate control of multiple area content in a map, realizes the multi-area embedding image generation of specified content, and the generated image meets the following conditions: (1) the overall content strictly conforms to the given text prompt;(2) in the region specified by the user, the visual structure of the reference image can be naturally embedded in a fine-grained manner, thereby forming an effective visual illusion effect while maintaining the accuracy of local control.The present application relates to diffusion model, image-image translation, phase injection mechanism and regional control generation technology, and is suitable for controllable multi-faceted image generation, illusion art synthesis and other application scenarios.

Description

Technical Field

[0001] This invention relates to the fields of computer vision and image generation, specifically to a text-guided image-to-image (T2I-I2I) conversion method. Background Technology

[0002] T2I-I2I conversion technology aims to generate visual illusion hidden images with strong visual confusion characteristics, hereinafter referred to as "illusion images". The core feature of such images is that visual elements (such as shapes, patterns, and colors) of a reference image are seamlessly embedded into the complex details of the target scene. This embedding method makes it difficult for the observer to directly or immediately perceive the presence of the reference image while perceiving the semantic content of the target scene. Visual illusion image technology has shown broad potential in many application scenarios, including high-end artistic creation, where artists can create multi-layered visual works with strong narratives, such as a landscape painting where mountains are contemplative philosophers and rivers are galloping horses; commercial brands and advertising, where brand logos, product outlines, or founder portraits are integrated into promotional posters and videos in the form of hidden art, creating "visual Easter eggs" with high dissemination value and fun; and personalization and entertainment, where users can combine the images of family members, friends, and pets into a unique artwork.

[0003] Throughout various art movements and styles, the art technique of using optical illusions to hide multi-layered images has always been a timeless creative theme. This form of expression, known as "picture within a picture" or "hidden image art," allows viewers to perceive progressively layered visual information within a single image through ingenious visual arrangement.

[0004] Master artists in this field, such as Oleg Shupliak, Salvador Dalí, Rob Gonsalves, and Octavio Ocampo, excel at meticulously embedding one or more hidden figures within seemingly ordinary scenes. These works can be categorized based on the complexity of their visual layers: some employ precise local reconstruction to seamlessly merge two independent images; others, through subtle adjustments to elements, allow for multiple interpretations of a single image.

[0005] The generation of visual illusions has long faced computational challenges in the fields of computer vision and graphics. Early research focused on utilizing the human brain's visual processing mechanisms to generate basic illusion types. With technological advancements, related research has gradually expanded into the field of image processing: early works employed a texture synthesis paradigm, such as Chu et al.'s retexturing pipeline, which achieves visual camouflage by embedding the structural adaptability of foreground image patches into background textures. Optimization-driven style transfer methods achieve camouflage effects by designing objective functions, with representative works such as Zhang et al.'s series of optimization constraints, deriving a structured hiding mechanism from the style transfer framework. The rise of Generative Adversarial Networks (GANs) has driven data-driven solutions, with Lamdouar et al. achieving end-to-end camouflage image synthesis for the first time based on the StyleGAN architecture.

[0006] In recent years, breakthroughs in diffusion models in generative AI (AIGC) have opened new paths for visual illusion generation: In functional fusion, Text2QR leverages ControlNet's fine-grained control capabilities to seamlessly integrate scannable QR codes into aesthetic images. For dynamic perceptual illusions, Diffusion Illusions combines T2I diffusion models with fractional distillation sampling (SDS) to generate multi-stable perceptual images with motion blur or light and shadow overlay effects. In the field of multi-view transformation, Visual Anagrams innovatively integrates multi-view noise estimation to generate images that exhibit semantic switching under specific geometric transformation operations (such as mirror flipping, rotation transformation, or jigsaw puzzle deconstruction). Recent advancements focus on implicit structural embedding; PTDiffusion proposes a phase spectrum dynamic transplantation technique, which achieves dual discriminative embedding of the target scene and the reference image structure by transplanting the phase spectrum of the reference image into the latent space diffusion features.

[0007] While existing methods offer various technical pathways for generating optical illusion images, their limitations in key dimensions still fail to meet the following core requirements: First, the imbalance in visual recognizability is prominent. Traditional methods for generating optical illusions (such as retexturing pipelines, optimization-driven style transfer, and end-to-end GAN synthesis) aim for extreme concealment, completely dissolving the content into the background texture, making it difficult for the host scene and hidden content to coexist and be recognized. Second, the dynamics of background generation are severely limited. Existing optical illusion image synthesis schemes generally rely on predefined static background textures as embedding carriers, lacking the ability to dynamically construct adapted scenes based on semantic descriptions. Finally, the illusion triggering mechanism has inherent constraints. Current optical illusion techniques based on diffusion models (such as Diffusion Illusions and Visual Anagrams) rely on external geometric transformation operations (such as mirror flipping, rotation transformation, or jigsaw puzzle deconstruction) as triggering conditions for illusion manifestation, essentially belonging to transformation-induced optical illusions.

[0008] It is worth noting that although recent phase spectrum transplantation technology attempts to achieve implicit structural embedding in latent space, it is limited by the global phase transplantation mechanism. The reference image structure is globally permeated into the generated image in an uncontrollable manner. It is impossible to accurately locate the hidden content to a specific area of ​​the host scene according to semantic instructions. In other words, it is impossible to achieve local area image synthesis with clear spatial correspondence and higher fine-grainedness, and it is also impossible to achieve visual illusion effect synthesis based on multi-source reference images.

[0009] Currently, image translation methods based on diffusion models, such as PTDiffusion, have made some progress in style transfer for face images.

[0010] However, the aforementioned methods mostly use the entire image structure as the injection scope, lacking the ability to perform fine-grained control over local areas of the image (such as multiple faces or local styles). They cannot effectively meet the creative needs of multi-source embedding, artistic style fusion, and controllable region awareness. Furthermore, existing technologies can only utilize information provided by the reference image itself, limiting their integration with text prompts. Currently, there is a lack of mature technologies capable of simultaneously embedding feature image information from multiple different sources at a fine-grained level in the generated image while maintaining the overall visual consistency and structural integrity of the image. Summary of the Invention

[0011] To address the shortcomings of current image generation models, such as the lack of interactive, fine-grained local control, the inability to inject reference images at multiple locations within a single image, and the inability to generate an overall image that conforms to the text prompts, this invention provides a fine-grained optical illusion image generation method based on region-aware phase injection, starting from the perspective of text-guided image-to-image translation. This method aims to generate artistic optical illusion hidden images. It supports precise control of multiple regions within a single image, enabling multi-region embedding of specified content into images. The generated images meet the following conditions: (1) the overall content strictly conforms to the given text prompts; (2) in the user-specified regions, the visual structure of the reference image can be naturally embedded in a fine-grained manner, thereby forming an effective optical illusion effect while maintaining the precision of local control. This invention relates to diffusion models, image-to-image translation, phase injection mechanisms, and region-controlled generation techniques, and is applicable to scenarios such as controllable multi-face image generation and optical illusion art synthesis.

[0012] The objective of this invention is achieved through the following technical solution:

[0013] A fine-grained visual illusion image generation method based on region-aware phase injection includes the following steps:

[0014] Step 1: Automated Construction of Adaptive Phase Bank:

[0015] An adaptive phase banking module is introduced to organize the structural information of the reference image in a fine-grained manner. The specific steps are as follows:

[0016] Step 1-1: Basic Encoding and Global Information Extraction:

[0017] Step 1-1-1: Input the reference image into the pre-trained VAE encoder to obtain its clear initial latent space representation. ;

[0018] Step 1-1-2, for Performing a two-dimensional fast Fourier transform yields the complete amplitude spectrum Mag( ) and phase spectrum ( );

[0019] Step 1-1-3, for Phase ( By applying filters and modifying the filters and their parameters, overall image feature information in different frequency domains can be extracted.

[0020] Step 1-2: Call the pre-trained open vocabulary segmentation model, use text prompts to perform fine-grained automatic segmentation, recognition and description tasks on the reference image, generate region control mask, semantic feature vector, and text description for all salient objects / parts, and store them in the form of tuple (text_label, feature_vector, mask) in the adaptive phase bank, and combine them with the overall image in Step 1-1 to achieve feature binding from whole to region awareness;

[0021] Step 2: Text-driven natural control interface parsing and instruction generation:

[0022] The system parser uses the user's input text prompts and declarations to perform fine-grained semantic interpretation and instruction conversion. The specific steps are as follows:

[0023] Step 2-1: Automatically identify host object keywords to locate the target fusion region with region awareness;

[0024] Step 2-2: The Parser module parses the parameterized control commands, including: ref: parameter associated with the pre-registered reference image label and its corresponding source reference image; w: parameter setting of fine-grained phase injection intensity; d: parameter configuration of the asynchronous distance of the region-aware phase flow across time steps.

[0025] Steps 2-3: The parser converts semantic elements and parameter constraints into corresponding operation parameters to control the injection process.

[0026] Step 3: Phase Query Attention Execution and Structure Injection:

[0027] Step 3-1: Design phase query attention within the MM-DiT architecture. Through attention boundary constraints, realize the query and fusion of structural information from image patch to APB, ensuring the directional embedding of fine-grained features. The specific steps are as follows:

[0028] Step 3-1-1: In the MM-DiT attention layer, for each image block token... :analyze The weight distribution in standard text-image cross-attention determines the dominant text word;

[0029] Step 3-1-2, only when When the dominant word is the currently parsed host object word that requires structure injection, activate that... Phase query permissions for a specified reference label;

[0030] Step 3-1-3: For those who have obtained query permissions The feature vector is used as the Query, and the cosine similarity is calculated with all registered feature vectors in the target APB.

[0031] Step 3-1-4: Based on the similarity weight, the phase information in the APB is aggregated in a region-aware weighted manner to generate the... Customized target phase spectrum ;

[0032] Step 3-2: Dynamically inject the target phase using cross-time-step phase flow technology. The specific steps are as follows:

[0033] Step 3-2-1, Current Time Step The final injection phase is determined by the previous time step. The injected phase is obtained by weighted fusion with the current target phase, and the calculation formula is as follows:

[0034]

[0035] in, It is the previous time step The injected phase, These are dynamically updated coefficients; At the current time step It has already fused all the local phases of the region perception, and is the phase that will eventually be injected into the target generated image;

[0036] Step 3-2-2: Smooth the... The current latent variables are fused in the frequency domain using region-aware fusion, and then transformed back to the spatial domain using inverse FFT to continue the subsequent diffusion generation process.

[0037] Compared with the prior art, the present invention has the following advantages:

[0038] 1. This invention aims to make the structural clues of the hidden reference image recognizable under active observation, while ensuring that the semantics of the target scene are clear and discernible, thereby achieving dual visual recognizability.

[0039] 2. This invention requires the generation of new scene elements that conform to semantic constraints through text prompts, and the seamless integration of reference images in the process, breaking through the bottleneck of static background dependence.

[0040] 3. This invention focuses on the ability to naturally embed reference structural cues into the visual content generated by text in a static single-view scene without any external intervention, thus achieving autonomously manifested embedding.

[0041] 4. This invention achieves fine-grained region-aware, multi-region style independent control while maintaining the overall structural consistency of the image.

[0042] 5. This invention can accomplish complex creative tasks such as Oleg Shuplyak-style illusion images and multi-face fusion art images.

[0043] 6. Compared with existing technologies such as PTDiffusion, this invention supports multi-source input and region-aware fine-grained injection, significantly enhancing the degree of freedom and fine-grained style fusion capability of the generated image. Attached Figure Description

[0044] Figure 1 Generate an architecture diagram by injecting fine-grained phase into the region-aware phase;

[0045] Figure 2 For comparison: Single-phase injection (first two images) vs. multi-region fine-grained injection generation results;

[0046] Figure 3 Generate the APB module structure;

[0047] Figure 4 Images generated for an example. Detailed Implementation

[0048] The technical solution of the present invention will be further described below with reference to the accompanying drawings, but it is not limited thereto. Any modifications or equivalent substitutions to the technical solution of the present invention that do not depart from the spirit and scope of the technical solution of the present invention should be covered within the protection scope of the present invention.

[0049] This invention provides a fine-grained method for generating visual illusion images based on region-aware phase injection. This method, through three core innovations, reshapes the technological paradigm of the field of controllable generation:

[0050] Firstly, an "Adaptive Phase Bank (APB)" is introduced—a lightweight, structured "phase-semantic" information system that organizes the structural information of the reference image in a fine-grained manner across multiple dimensions, including macroscopic, microscopic, and stylistic dimensions, and supports multiple "access" operations within the attention layer. By combining cross-time-step phase flow and hierarchical design, dynamic, smooth, and multi-scale injection of structural information is achieved, providing precise underlying support for local control.

[0051] Secondly, we designed "Phase Query Attention (PQA)"—a cross-domain attention mechanism natively implemented within the MM-DiT architecture. Through attention-based bounding constraints, it enables precise, sparse, and leak-free structural information querying and fusion from image patches to APB, ensuring the targeted embedding of fine-grained features.

[0052] Thirdly, a "text-driven natural control interface" is established, allowing users to intuitively schedule and combine multiple hidden structures through specially tagged natural language prompts, finely adjust their generation parameters, and control the region perception capability of the image through influence masks. This method abandons the resource-intensive parallel inversion path, achieving a leap from physical region control to semantic object control, providing better flexibility, efficiency, and artistic expression for complex, composable, and highly controllable optical illusion art creation. If the user inputs only one reference image and the injection strength is not defined, the default setting for hidden structure is full-image injection. Figure 2 The two images on the left (single-phase injection); other cases should be adjusted according to user settings, see reference. Figure 2 Right side (multi-phase injection).

[0053] The core of this invention lies in the construction and application of adaptive phase banking, the execution of phase query attention, and the parsing and implementation of a text-driven natural control interface, such as... Figure 1 As shown, the specific implementation steps are as follows:

[0054] Step 1: Automated Construction of the Adaptive Phase Bank (APB):

[0055] like Figure 1As shown, when a user provides a reference image (such as dragon_ref.jpg) and specifies a tag (such as dragon_ref), the system automatically executes the following build process:

[0056] Step 1-1: Basic Encoding and Global Information Extraction: Input the reference image into the pre-trained VAE encoder E to obtain its clear initial latent space representation. .right Performing a two-dimensional fast Fourier transform (2D-FFT) yields the complete amplitude spectrum Mag( ) and phase spectrum ( Subsequently, regarding Phase ( By applying filters and modifying the filters and their parameters, overall image feature information in different frequency domains can be extracted.

[0057] To effectively capture the global and local structural features of the reference image at different frequency scales, this invention focuses on the phase spectrum. A fusion strategy for multiple filter types is introduced. Specifically, this includes:

[0058] (1) Low-pass filter (LPF): preserves low-frequency components and is used to extract the overall contour, illumination distribution and large-scale structure.

[0059] (2) High-pass filter (HPF): Emphasizes high-frequency components, highlights edge and detail texture, and provides structural support for fine-grained injection of local areas.

[0060] (3) Wavelet filtering: decomposes the phase spectrum in a multi-scale space, taking into account both frequency information and spatial positioning capability, and can extract local multi-scale features under the constraint of a region mask.

[0061] In practical implementation, this invention extracts global and detailed features through a hierarchical combination of low-pass and high-pass filters, and also combines wavelet decomposition to achieve more refined multi-scale region-aware feature acquisition. The above filtering results are ultimately stored and organized in an adaptive phase bank (APB), providing hierarchical and controllable structural information for subsequent region-aware phase injection.

[0062] For example, a Gaussian low-pass filter can be used, employing a two-dimensional Gaussian kernel with σ=0.5. Extract the low-frequency-dominated macro phase, which represents the overall outline and shape of the object while preserving fine-grained local structural features.

[0063] Step 1-2: Call the pre-trained open vocabulary segmentation model (here, the TAP model is used) to perform fine-grained automatic segmentation, recognition, and description tasks on the reference image using text prompts. Generate region control masks, semantic feature vectors, and text descriptions for all salient objects / parts. Store these in the form of tuples (text_label, feature_vector, mask) in the adaptive phase bank. Combine this with the overall image from Step 1-1 to achieve feature binding from whole to region awareness.

[0064] Step 2: Text-driven natural control interface parsing and instruction generation:

[0065] The system parser uses the user's input text prompts and declarations to perform fine-grained semantic interpretation and instruction conversion. The specific steps are as follows:

[0066] Step 2-1: Automatically identify host object keywords (target_object, such as "mountain") to locate the target fusion region in a region-aware manner;

[0067] Step 2-2: The Parser module parses parameterized control commands, including:

[0068] ref: The parameter is associated with a pre-registered reference image label (such as dragon_ref) and its corresponding source reference image;

[0069] w: Parameter setting for fine-grained phase injection intensity (weight value, floating point, domain [0,1]);

[0070] d: Parameter configuration region-aware asynchronous distance of cross-time step phase flow (CTPF) (integer variable, controlling the initial injection offset of phase information in the diffusion sequence);

[0071] Steps 2-3: The parser transforms semantic elements and parameter constraints into corresponding operation parameters to control the injection process.

[0072] Step 3: Phase Query Attention Execution and Structure Injection:

[0073] Step 3-1: Design phase query attention within the MM-DiT architecture. Through attention boundary constraints, realize the query and fusion of structural information from image patch to APB, ensuring the directional embedding of fine-grained features. The specific steps are as follows:

[0074] Step 3-1-1: In the MM-DiT attention layer, for each image block token... :analyze The weight distribution in standard text-image cross-attention determines the dominant text word;

[0075] Step 3-1-2, only when When the dominant word is the currently parsed host object word that requires structure injection, activate that... Phase query permissions for a specified reference label;

[0076] Step 3-1-3: For those who have obtained query permissions The feature vector is used as the Query, and the cosine similarity is calculated with all registered feature vectors in the target APB.

[0077] Step 3-1-4: Based on the similarity weight, the phase information in the APB is aggregated in a region-aware weighted manner to generate the... Customized target phase spectrum .

[0078] Step 3-2, Structure Injection:

[0079] To achieve progressive fusion of the hidden structure, the cross-time-step phase flow (CTPF) technique is used to dynamically inject the target phase. The specific steps are as follows:

[0080] Step 3-2-1, Current Time Step The final injection phase is determined by the previous time step. The dynamic weighting coefficient is obtained by weighted fusion of the injected phase and the current target phase, and its calculation follows the exponential moving average criterion. The phase fusion strategy decays over time, initially assigning a larger weight to quickly establish the main structural features, and then gradually reducing the weight as the generation process progresses for fine-grained fine-tuning. This coefficient is typically set as a linear decay function based on the total number of steps T, ensuring a base update strength of at least 0.1 in the early stages of generation. This time-aware phase fusion strategy guarantees the natural evolution of the hidden structure while avoiding image distortion caused by excessive injection in later stages.

[0081]

[0082] in, It is the previous time step The injected phase (initialized to zero or default spectrum). These are dynamically updated coefficients. At the current time step The phase that has been fused with all the local phases of the region perception and will eventually be injected into the target-generated image has a size that is consistent with the size of the target-generated image in the latent space.

[0083] Step 3-2-2: Smooth the... The current latent variables are fused in the frequency domain using region-aware fusion, and then transformed back to the spatial domain using inverse FFT to continue the subsequent diffusion generation process.

[0084] Example:

[0085] I. Task Description

[0086] The user wants to generate an image with the global description "A grand painting of a valley at sunset, oil painting style". Simultaneously, the user wants to cleverly embed two hidden structures within this painting: the outline of the mountains should resemble "a man's profile", and the rolling clouds in the sky should resemble "a dragon".

[0087] II. User Input

[0088] Global text hint: "A grand painting of a valley at sunset, oil painting style."

[0089] 1. Local text hints:

[0090] [mountain|ref: mans_face, w:0.7]

[0091] [clouds|ref: dragon_form,w:0.6]

[0092] 2. Reference Resource Declaration:

[0093] \def_ref{mans_face,path:'. / refs / mans_profile.png'}

[0094] \def_ref{dragon_form,path:'. / refs / dragon.jpg'}

[0095] These two statements correspond to respectively Figure 4 (a) and Figure 4 (b).

[0096] III. Implementation Process

[0097]

[0098] Table parameter description: P - Text prompt for user input; - Reference images; - Reference words; Total number of denoising steps in the T-diffusion model. - Latent representation of the reference image obtained through the VAE encoder; VAE_encoder - Encoder part of the Variational Autoencoder; - The phase spectrum of the latent variables of the reference image after 2D Fourier transform (FFT2D); FFT2D and 2D-FFT have the same meaning and can be equivalently substituted, two-dimensional fast Fourier transform (2D Fast Fourier Transform). -Attention mask from TAP obtained from the reference image; - Attention values ​​from TAP obtained from the reference image; TAP - Refer to the main module of the paper Tokenize Anything via Prompting; APB - Adaptive Phase Bank. - Initial random noise (sampled from Gaussian distribution); - Intermediate feature at step t during denoising; MM-DiT - Multimodal Diffusion Transformer; TextTokens - Tokenized text prompt P, i.e., words in the prompt related to the target image region; S - Set of pixels / regions with high attention to W_ref; p - Local patch in the image; τ - Attention threshold for filtering; - Target phase spectrum (weighted phase spectrum for injection); -Weight for phase combination; - Phase spectrum from APB in the repository; - Phase spectrum to be injected; α- Phase fusion factor. - The injection phase from the previous step; CTPF - Cross-Time Phase Fusion strategy. - Combined phase spectrum; - Original phase from h_t; Magnitude spectrum from Mag Fourier transform; IFFT2D - 2D Inverse Fast Fourier Transform; Denoise - Denoising function at step t for diffusion model. - The final latent after denoising; VAEDecoder - the decoder part of the Variational Autoencoder.

[0099] (a) Static Relationship Construction: Real-time Construction of APB (Adaptive Phase Bank)

[0100] Before the diffusion generation process begins, the system performs the following one-time preprocessing operation to construct the APB. This is the static relational part of the present invention.

[0101] 1. Reference Image Processing - mans_face:

[0102] First, the input image mans_profile.png is encoded using a pre-trained VAE to obtain its latent space representation. ,0), and perform a two-dimensional fast Fourier transform on the latent variable to extract the global phase spectrum Phase( Subsequently, using the SAM training dataset SA-1B and the joint framework TAP of EVA-CLIP, semantically clear "face contour" regions were segmented from the original image, and the corresponding binary mask was downsampled to 64×64 resolution, denoted as mask_mans_face. This mask was then used to spatially filter the phase spectrum, yielding the local phase feature Phase( The EVA-CLIP image embedding vector V_mans_face is extracted from the masked region using the 'mans_face', 'v_mans_face', and 'v_mans_face'. Finally, the semantic identifier, visual feature vector, and local phase spectrum triple ('mans_face', V_mans_face, Phase(0)) are combined. ,0)[mask_mans_face]) is stored in APB to complete the standardized encapsulation of the cell.

[0103] 2. Reference Image Processing - dragon_form:

[0104] For the input image dragon.jpg, the same processing flow is used: first, its latent space representation is obtained through the VAE encoder ( The global phase spectrum (Phase(0)) is obtained after 2D-FFT transformation. Then, TAP is used to accurately segment semantic regions such as "dragon form" and generate the corresponding mask mask_dragon_form. Based on this mask, the local phase spectrum Phase( ,0) representing the structural features of the dragon form is extracted from the global phase spectrum. The region is obtained by first generating a tuple of dragon-shaped features ('dragon_form', V_dragon_form, Phase(0)[mask_dragon_form], and then using the EVA-CLIP encoder to obtain the depth visual feature vector V_dragon_form for that region. Finally, the dragon-shaped feature triple with clear semantic identification ('dragon_form', V_dragon_form, Phase(0)[mask_dragon_form], and Phase(0)[mask_dragon_form] ... The 0)[mask_dragon_form]) is stored in the APB as prior structural knowledge for subsequent cross-modal generation. This process not only realizes the hierarchical decomposition of visual features (spatial structure → frequency domain phase → semantic embedding), but also establishes a scalable cross-domain feature association mechanism.

[0105] At this point, the APB construction is complete, and the stored content is as follows: Figure 3 As shown.

[0106] (II) Dynamic Relationships and Effects: Iterative Denoising Based on PQA

[0107] The system starts from random Gaussian noise We begin with T-step iterative denoising. At each time step t, the following dynamic interactions and effects occur:

[0108] 1. MM-DiT forward propagation:

[0109] Use the current noise latent variable The tokens of the time step t and the parsed global text hints (with the plain text portion of the control block removed) are fed into the cross-attention and self-attention computation standard of MM-DiT.

[0110] 2. Trigger words and attention boundary constraints:

[0111] During cross-attention computation, the system identified the target words "mountains" and "clouds" in the text. By analyzing the cross-attention map, the system determined the image... Within this framework, we determine which patch tokens are strongly activated by the word "mountains" (forming the set S_mountains) and which are strongly activated by the word "clouds" (forming the set S_clouds). This constitutes a dynamic, semantically driven soft boundary.

[0112] 3. Phase query attention and dynamic phase injection:

[0113] For mountainous regions: For each patch token t_p belonging to S_mountains, its feature vector is used as the query, and cosine similarity is calculated with all keys (V_mans_face, V_dragon_form, etc.) in the APB. Since... The semantic meaning of is "mountain", and its similarity with V_mans_face (human face) is higher than that with V_dragon_form (dragon). Based on this similarity, PQA extracts the local phase spectrum corresponding to mans_face from APB in a weighted manner, which is used as the target phase phase_target_mans for this patch.

[0114] For cloud regions: Similarly, for patch tokens belonging to S_clouds, they will query and extract the target phase phase_target_dragon associated with dragon_form.

[0115] Outside the boundary: For patches that do not belong to S_mountains or S_clouds (such as rivers in valleys), they do not trigger PQA, or their query weight is extremely low, and their phase remains unchanged.

[0116] 4. Phase flow across time steps and frequency domain reconstruction:

[0117] For each modified patch The final injected phase It is the phase that has been injected in the previous step. Phase with the current target The exponential moving average ensures the smooth "growth" of the hidden structure. A modified composite phase spectrum is generated. The amplitude spectrum predicted by the original MM-DiT is compared with... By combining these methods, new latent space features with injected hidden structures can be obtained through IFFT. .

[0118] 5. Iteration:

[0119] use As input, calculate the noise for the next step. Repeat this process until t=0.

[0120] 6. Final Decoding:

[0121] The final result The final image is generated using the VAE decoder.

[0122] Effect: The final generated image is as follows Figure 4 As shown in (c), from a macroscopic perspective, it is a landscape painting that fits the description of "sunset valley, oil painting style". However, upon closer observation or viewing from a distance, it can be seen that the undulating contours of the mountains cleverly form the profile of a man's face, while the clouds in the sky naturally converge to form a faintly visible dragon. The structure of the man's profile and the dragon matches the input reference image, and each hidden structure is precisely confined within its semantic host (mountains, clouds) without leaking into other areas.

Claims

1. A fine-grained method for generating visual illusion images based on region-aware phase injection, characterized in that... The method includes the following steps: Step 1: Automated Construction of Adaptive Phase Bank: An adaptive phase banking module (APB) is introduced to organize the structural information of the reference image in a fine-grained manner. The specific steps are as follows: Step 1-1: Basic Encoding and Global Information Extraction: Step 1-1-1: Input the reference image into the pre-trained VAE encoder to obtain its clear initial latent space representation. ; Step 1-1-2, for Performing a two-dimensional fast Fourier transform yields the complete amplitude spectrum Mag( ) and phase spectrum (Phase) ); Step 1-1-3, for Phase ( By applying filters and modifying the filters and their parameters, overall image feature information in different frequency domains can be extracted. Step 1-2: Call the pre-trained open vocabulary segmentation model, use text prompts to perform fine-grained automatic segmentation, recognition and description tasks on the reference image, generate region control mask, semantic feature vector, and text description for all salient objects / parts, and store them in the form of tuple (text_label, feature_vector, mask) in the adaptive phase bank, and combine them with the overall image in Step 1-1 to achieve feature binding from whole to region awareness; Step 2: Text-driven natural control interface parsing and instruction generation: The system parser uses the user's input text prompts and declarations to perform fine-grained semantic interpretation and instruction conversion. The specific steps are as follows: Step 2-1: Automatically identify host object keywords to locate the target fusion region with region awareness; Step 2-2: The Parser module parses the parameterized control commands, including: ref: parameter associated with the pre-registered reference image label and its corresponding source reference image; w: parameter setting of fine-grained phase injection intensity; d: parameter configuration of the asynchronous distance of the region-aware phase flow across time steps. Steps 2-3: The parser converts semantic elements and parameter constraints into corresponding operation parameters to control the injection process. Step 3: Phase Query Attention Execution and Structure Injection: Step 3-1: Design phase query attention within the MM-DiT architecture. Through attention boundary constraints, realize the query and fusion of structural information from image patch to APB, ensuring the directional embedding of fine-grained features. The specific steps are as follows: Step 3-1-1: In the MM-DiT attention layer, for each image block token... :analyze The weight distribution in standard text-image cross-attention determines the dominant text word; Step 3-1-2, only when When the dominant word is the currently parsed host object word that requires structure injection, activate that... Phase query permissions for a specified reference label; Step 3-1-3: For those who have obtained query permissions The feature vector is used as the Query, and the cosine similarity is calculated with all registered feature vectors in the target APB. Step 3-1-4: Based on the similarity weight, the phase information in the APB is aggregated in a region-aware weighted manner to generate the... Customized target phase spectrum ; Step 3-2: Dynamically inject the target phase using cross-time step phase flow technology.

2. The fine-grained visual illusion image generation method based on region-aware phase injection according to claim 1, characterized in that... In step 1-1-3, a Gaussian low-pass filter is used, employing a two-dimensional Gaussian kernel with σ=0.

5. We extract the low-frequency-dominated macroscopic phase, which represents the overall outline and shape of an object, while preserving fine-grained local structural features.

3. The fine-grained visual illusion image generation method based on region-aware phase injection according to claim 1, characterized in that... The specific steps of step 3-2 are as follows: Step 3-2-1, Current Time Step The final injection phase is determined by the previous time step. The injected phase is obtained by weighted fusion with the current target phase, and the calculation formula is as follows: in, It is the previous time step The injected phase, These are dynamically updated coefficients; At the current time step It has already fused all the local phases of the region perception, and is the phase that will eventually be injected into the target generated image; Step 3-2-2: Smooth the... The current latent variables are fused in the frequency domain using region-aware fusion, and then transformed back to the spatial domain using inverse FFT to continue the subsequent diffusion generation process.

Images