Diffusion style migration method and system based on subsurface distribution reanchoring dynamic injection

Abstract

The invention relates to the technical field of image style migration, and provides a diffusion style migration method and system based on latent distribution re-anchoring dynamic injection, and the method comprises the steps: carrying out the second-order statistical alignment of a content hidden variable and a style hidden variable through a potential covariance re-coloring method; automatically positioning an injection starting point of the style according to the edge intensity of the content image, and injecting hidden distribution reanchoring noise into the initialized mixed noise hidden variable to realize hidden distribution reanchoring; the content self-attention features and the style self-attention features are fused through a soft + dynamic style injection module, in the reverse sampling process of the calibrated hidden variables, the fused attention features are gradually injected into an attention layer in the diffusion process at the injection starting point, diffusion denoising is carried out, and a style migration image is generated. According to the method, the image with rich structure details is protected, and the excellent content structure maintaining capability and the accurate style reappearance are realized at the same time.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of image style transfer, in particular to a diffusion style transfer method and system based on latent distribution reanchoring dynamic injection. BACKGROUND

[0002] Existing style transfer related researches usually focus on explicitly decoupling style and content to achieve interpretable and controllable style transfer. This kind of method mainly utilizes the generation ability of pre-trained DM (Diffusion Model), but needs additional gradient optimization to fine-tune the model, so it is time-consuming. For the above challenges, some researches propose a training-free style transfer scheme. This kind of method transfers the texture of style images to content images by manipulating the self-attention features of pre-trained large-scale DM. Although these methods have shown promising results, they still encounter the problems of naturalness and color of generated results and content leakage of transferred images. The problem of naturalness and color of generated results is reflected in that part of the generated images often present unnatural visual effects. Not only is there a lack of organic integration between style and content, there is a clear sense of disconnection, but there is also a situation of unstable color tone fluctuation. The core reason for these problems is that the AdaIN method which only relies on channel mean and variance matching on latent variables often has limitations. It can only achieve alignment at the first-order statistical level and cannot capture more critical correlation information between style and content latent variables, making it difficult to support the needs of generated results in style consistency, content fusion and color stability. The problem of content leakage of transferred images is specifically manifested in that the structure of the content is destroyed in the generated image after style injection, such as face distortion when generating a person and object losing original morphological characteristics when generating a still life. The core reason for this problem is that the existing style injection method has many defects: first, although the self-attention layer is crucial for style injection, it needs to be optimized and directly overturns the original structure of the content in the early stage of injection, lacking protection of the content structure; second, the timing of style injection is difficult to accurately control. Early injection will further damage the integrity of the content structure, and late injection will not allow the style to be fully integrated, resulting in weak style expression. SUMMARY

[0003] The present application aims to at least solve one of the technical problems in the related art. To this end, the present application provides a diffusion style transfer method and system based on latent distribution reanchoring dynamic injection, which avoids the destructive effect of style information on content structure in the early sampling stage through progressive style injection, adaptively determines the injection starting point according to the edge complexity of the content image, protects the image with rich structural details, and thus realizes excellent content structure preservation ability and accurate style reproduction.

[0004] The application provides a diffusion style migration method based on latent distribution re-anchor dynamic injection, comprising: S1: obtaining a content image and a style image, reversing the content image to a high-noise state in a hidden space through a denoising diffusion implicit model inverse diffusion process to obtain a content hidden variable and a content self-attention feature, reversing the style image to a high-noise state in the hidden space through the denoising diffusion implicit model inverse diffusion process to obtain a style hidden variable and a style self-attention feature; S2: aligning the content hidden variable and the style hidden variable through a latent covariance recoloring method to obtain an initialized mixed noise hidden variable; S3: calculating a grayscale image of the content image through a Sobel gradient operator, calculating an edge intensity through the grayscale image of the content image, and automatically positioning an injection starting point of the style according to the edge intensity; S4: injecting the initialized mixed noise hidden variable through latent distribution re-anchor noise injection to obtain a calibrated hidden variable; S5: fusing the content self-attention feature and the style self-attention feature through a soft+dynamic style injection module to obtain a fused attention feature; S6: in the process of reverse sampling of the calibrated hidden variable, gradually injecting the fused attention feature into an attention layer of a diffusion process at the injection starting point to generate a style migration image through diffusion denoising.

[0005] Further, the S2 step comprises: S21: flattening the content hidden variable and the style hidden variable in a spatial dimension; S22: calculating a decentered feature of the flattened content hidden variable and the flattened style hidden variable; S23: calculating a content covariance matrix according to the decentered feature of the flattened content hidden variable, and calculating a style covariance matrix according to the decentered feature of the flattened style hidden variable; S24: performing eigenvalue decomposition on the content covariance matrix and the style covariance matrix to obtain a content eigenvalue and a style eigenvalue; S25: constructing a whitening matrix according to the content eigenvalue, and constructing a coloring matrix according to the style eigenvalue; constructing a latent covariance recoloring transformation matrix according to the whitening matrix and the coloring matrix, and aligning the content hidden variable and the style hidden variable in a second-order statistics according to the latent covariance recoloring transformation matrix to obtain an initialized mixed noise hidden variable.

[0006] Further, the S3 step comprises: S31: normalizing the content image, and calculating a grayscale image of the normalized content image; S32: calculating a horizontal direction gradient image and a vertical direction gradient image of the grayscale image of the normalized content image through a Sobel gradient operator. S33: calculating edge strength according to the gradient map in the horizontal direction and the gradient map in the vertical direction; S34: normalizing and linearly mapping the edge strength to an injection starting point interval to obtain an injection starting point.

[0007] Further, the calculation expression of the calibrated latent variable is: wherein, is the calibrated latent variable at the moment, is the latent variable of the diffusion model at the moment, is the noise intensity, is the latent distribution reanchoring noise, is a standard normal distribution. Further, the soft + dynamic style injection module includes key-value soft injection and dynamic parameter adjustment.

[0008] Further, the key-value soft injection includes: S511: obtaining a sampling progress and normalizing the sampling progress;

[0009] S512: constructing a mixing coefficient according to the normalized sampling progress; S513: linearly interpolating the keys in the content self-attention feature and the keys in the style self-attention feature according to the mixing coefficient to obtain the keys of the fused attention; S514: linearly interpolating the values in the content self-attention feature and the values in the style self-attention feature according to the mixing coefficient to obtain the values of the fused attention. Further, the dynamic parameter adjustment includes: S521: constructing a query retention rate hyperparameter by using cosine scheduling to smoothly transition from a high initial value to a low termination value;

[0010] S522: mixing the content query and the intermediate result of the migration diffusion model according to the query retention rate hyperparameter to obtain the final query in the self-attention. Further, the dynamic parameter adjustment further includes: constructing a temperature scaling parameter according to the normalized sampling progress;

[0011] scaling up the attention map before softmax in the self-attention layer according to the temperature scaling parameter. Further, the key-value soft injection and the dynamic parameter adjustment only take effect in the N-layer attention layer within the diffusion sampling loop.

[0012] Further, the key-value soft injection and the dynamic parameter adjustment only take effect in the N-layer attention layer within the diffusion sampling loop.

[0013] ​The application also provides a diffusion style migration based on latent distribution re-anchor dynamic injection, which is used to execute the diffusion style migration based on latent distribution re-anchor dynamic injection. The feature extraction module acquires the content image and the style image, reverses the content image to a high-noise state in the latent space through a denoising diffusion implicit model reverse diffusion process, obtains content latent variables and content self-attention features, reverses the style image to a high-noise state in the latent space through the denoising diffusion implicit model reverse diffusion process, and obtains style latent variables and style self-attention features. The alignment module aligns the content latent variables and the style latent variables through a latent covariance re-coloring method to obtain initialized mixed noise latent variables. The automatic positioning module calculates a gray image of the content image through a Sobel gradient operator, calculates edge intensity through the gray image of the content image, and automatically positions an injection starting point of the style according to the edge intensity. The re-anchor module injects latent distribution re-anchor noise into the initialized mixed noise latent variables to obtain calibrated latent variables. The soft+dynamic style injection module fuses the content self-attention features and the style self-attention features to obtain fused attention features. The migration generation module gradually injects the fused attention features into the attention layer of the diffusion process at the injection starting point in the process of reverse sampling of the calibrated latent variables, and generates a style migration image through diffusion denoising.

[0014] The above one or more technical solutions in the embodiments of the application have at least one of the following technical effects: The application avoids the destructive influence of style information on content structure in the early sampling stage through the progressive style injection, adaptively determines the injection starting point according to the edge complexity of the content image, protects the image with rich structural details, and thus realizes excellent content structure maintaining capability and accurate style reproduction.

[0015] Additional aspects and advantages of the application will be described in the following description, become apparent from the following description, or be learned by practice of the application. BRIEF DESCRIPTION OF DRAWINGS

[0016] In order to more clearly illustrate the technical solutions in the application or prior art, the following will briefly introduce the drawings needed to be used in the embodiments or prior art description. Obviously, the drawings in the following description are some embodiments of the application, and for those skilled in the art, other drawings can also be obtained without creative labor on the basis of these drawings.

[0017] Figure 1 is a flow diagram of a diffusion style transfer method based on latent distribution re-anchoring dynamic injection provided by the present application.

[0018] Figure 2 is a soft+dynamic style injection module schematic diagram provided by the present application.

[0019] Figure 3 is a structural schematic diagram of a diffusion style transfer system based on latent distribution re-anchoring dynamic injection provided by the present application.

[0020] Reference signs: 101, feature extraction module; 102, alignment module; 103, automatic positioning module; 104, re-anchoring module; 105, soft+dynamic style injection module; 106, migration generation module. DETAILED DESCRIPTION

[0021] In order to make the objects, technical solutions and advantages of the present application clearer, the technical solutions in the present application will be clearly and completely described below. Obviously, the described embodiments are some embodiments of the present application, but not all embodiments. Based on the embodiments in the present application, all other embodiments obtained by those skilled in the art without creative labor fall within the scope of protection of the present application. The following embodiments are used to illustrate the present application, but cannot be used to limit the scope of the present application.

[0022] In the description of the present application, the description of the terms "one embodiment", "some embodiments", "example", "specific example", or "some examples" means that the specific features, structures, materials or characteristics described in connection with the embodiment or example are included in at least one embodiment or example of the present application. In the present application, the illustrative description of the above terms does not necessarily refer to the same embodiment or example. Moreover, the specific features, structures, materials or characteristics described can be combined in any appropriate manner in any one or more embodiments or examples. In addition, those skilled in the art can combine and combine the different embodiments or examples described in the present application and the features of the different embodiments or examples without contradiction.

[0023] The present application will be described below in conjunction with Figures 1 to 3 a diffusion style transfer method and system based on latent distribution re-anchoring dynamic injection.

[0024] This invention addresses three core issues in diffusion models: the quality of initial latent variables, attention injection strategy and intensity scheduling, and the accuracy and distribution stability of injection starting points. It proposes a complete train-free style transfer framework. The overall process involves performing a Denoising Diffusion Implicit Model Inverse Diffusion (DDIM) inversion on both content and style images, simultaneously collecting self-attention features and latent variables at each step of the inversion process. The core adopts a training-free paradigm of "query (Q) taken from content, key (K) / value (V) taken from style + query retention rate hyperparameter γ + attention temperature scaling parameter τ," and strengthens the statistical correction of initial latent variables. During the sampling phase, backsampling is performed primarily on content, with style information injected only in the later stages of the decoder (focusing on local texture layers), achieving accurate fusion of style and content.

[0025] To address the issue of inconsistent generated results when relying solely on AdaIN, this invention proposes a Latent Covariance Recoloring (LCR) method. Its core is to achieve second-order statistical renormalization on the initial latent variables: by calculating the covariance matrix of content and style latent variables, a coloring matrix is ​​constructed through eigenvalue decomposition to complete the latent variable remapping, aligning the channel association patterns of content latent variables with the style. Compared to StyleID's "initial latent AdaIN," this invention upgrades "channel statistical alignment" to "second-order statistical alignment." Secondly, this invention injects style information through a self-attention layer, while designing a soft injection and dynamic adjustment strategy for query retention / attention temperature. Borrowing the "progressive hybrid" idea of ​​h-space injection, it achieves progressive soft injection from content to style by linear interpolating the cached K / V tensor. Finally, to address the difficulties in controlling the injection timing (too early destroys the structure, too late weakens the style) and the distribution drift caused by multiple latent variable modifications, this invention proposes a combined scheme of automatic localization and latent distribution re-anchoring module (AS-LDR). By performing Sobel edge statistics on the content graph, the detail richness is linearly mapped to the injection start step, and high-frequency loss caused by repeated modifications in the sampling loop is avoided.

[0026] First, the content image and style images By inverting the high-noise state in the latent space through the reverse diffusion process of the denoising diffusion implicit model, we obtain... and The Latent Covariance Recoloring (LCR) module is used to obtain the initial mixed noise by combining the statistical properties of content noise and style noise. At the same time, through the latent distribution re-anchoring mechanism in Injecting enhanced noise effectively restores high-frequency details. In the... During the forward diffusion sampling process, the automatic localization module adaptively calculates the start time of style injection based on the complexity of the content image and progressively injects the self-attention feature information extracted during the style inversion process into the self-attention layer of the diffusion process, thus completing the entire diffusion denoising process and generating a style transfer image.

[0027] like Figure 1 As shown, a diffusion style transfer method based on latent distribution re-anchoring dynamic injection includes: S1: Obtain the content image and style image. Through the denoising diffusion implicit model inverse diffusion process, the content image is inverted to the high-noise state of the latent space to obtain the content latent variables and content self-attention features. Through the denoising diffusion implicit model inverse diffusion process, the style image is inverted to the high-noise state of the latent space to obtain the style latent variables and style self-attention features. Inversion is performed on the content image and style image respectively, and the self-attention features and latent variables of each step in the inversion process are collected simultaneously.

[0028] S2: The latent variables of content and style are second-order statistically aligned using the latent covariance recoloring method to obtain the initial latent variables; The LCR method performs a complete second-order statistical alignment of content and style latent variables at the initial stage of the diffusion process. Its core idea draws inspiration from the whitening-coloring transformation theory in color science. First, it eliminates the original channel correlations of content latent variables through whitening transformation. Then, it applies the covariance structure of style latent variables for coloring transformation, thereby preserving the spatial layout of the content while fully transferring the color distribution characteristics of the style, resulting in a more harmonious image. Compared to AdaIN, LCR not only matches the mean and variance of the channels, but more importantly, it aligns the complete covariance matrix, enabling the stylization result to more accurately reproduce the color relationships of the target style and reflect greater naturalness.

[0029] S21: Flatten the implicit variables of content and style in the spatial dimension; Given content, implicit variables , ,in, For the sample size, The number of channels in the image. For height, For width, For dimension The real space, style implicit variables , Flattening in the spatial dimension yields the hidden variables of the flattened content. Style hidden variables after flattening , ,in, For spatial length, , For dimension The real number space; S22: Calculate the decentralized characteristics of the flattened content latent variables and style latent variables; The expression for calculating the decentralized feature of the flattened latent variables is as follows: in, To achieve the decentralized characteristics of the latent variables in the flattened content, This is the mean vector of the content latent variables along the channel dimension. , The length of the space after flattening. For dimension The real number space; The expression for calculating the decentralized features of the flattened style latent variables is as follows: in, To achieve the decentralized characteristics of the latent variables in the flattened content, This is the mean vector of the content latent variables along the channel dimension. ; S23: Calculate the content covariance matrix based on the decentralized characteristics of the flattened content latent variables, and calculate the style covariance matrix based on the decentralized characteristics of the flattened style latent variables. The expression for calculating the content covariance matrix is: in, For the content covariance matrix, For dimension The real number space; The expression for calculating the style covariance matrix is: in, The style covariance matrix; S24: Perform eigenvalue decomposition on the content covariance matrix and style covariance matrix to obtain content eigenvalues ​​and style eigenvalues; The covariance matrix of each sample is decomposed into eigenvalues, and the calculation expression is as follows: in, For the first The eigenvalues ​​of the content covariance matrix of each sample For the first The content feature vector matrix of each sample For the first The diagonal eigenvalue matrix of each sample For the first The eigenvalues ​​of the style covariance matrix of each sample. For the first The style feature vector matrix of each sample For the first The style diagonal eigenvalue matrix of each sample. This is the transpose of the matrix; S25: Construct a whitening matrix based on content feature values, and a coloring matrix based on style feature values; The expression for calculating the whitening matrix is: in, For whitening matrix, Covariance structures used to eliminate latent variables in content; The expression for calculating the coloring matrix is: in, The coloring matrix; S26: Construct a latent covariance recoloring transformation matrix based on the whitening matrix and the coloring matrix. Align the content latent variables and style latent variables using the latent covariance recoloring transformation matrix to obtain the initialized mixed noise latent variables. The calculation expression is: in, To initialize the hidden variables of mixed noise.

[0030] Transformed and Having the same covariance matrix while retaining Spatial structure information.

[0031] S3: Calculate the grayscale image of the content image using the Sobel operator, calculate the edge intensity using the grayscale image of the content image, and automatically locate the style injection starting point based on the edge intensity; The injection start point is adaptively determined based on the structural complexity of the content image. The structural complexity of the content image directly affects the choice of style injection timing: images with rich details and complex edges need to be injected later to fully protect the structure, while images with simple structures can be injected earlier to enhance the style effect.

[0032] S31: Normalize the content image and calculate the grayscale image of the normalized content image. The expression for calculating the grayscale image is: in, This is the grayscale image of the content image after normalization. The number of channels in the image. For the first The weighting coefficients for each channel, For content images, S32: Calculate the horizontal and vertical gradient maps of the normalized grayscale image using the Sobel gradient operator. The calculation expression is as follows: in, This is the gradient plot in the horizontal direction. This is the gradient map in the vertical direction. For the horizontal Sobel operator, For the Sobel operator in the vertical direction, This is a two-dimensional convolution operation; S33: Calculate the edge intensity based on the gradient maps in the horizontal and vertical directions. The edge intensity statistic is defined as the spatial average of the gradient magnitude, and the calculation expression is: in, For edge strength, coordinates The gradient plot in the horizontal direction. coordinates The gradient plot in the vertical direction; S34: Normalize the edge intensity and linearly map it to the injection start interval. The calculation expression is as follows: in, As the starting point for the mapped injection, The lower limit of the injection starting interval, To be the upper limit of the starting interval for injection, For the clipping function, This represents the minimum edge strength. This represents the maximum value of the edge strength. The default parameter is set to The greater the edge strength (the richer the structural details), the larger the normalized value E, and the later the injection. The later the injection, the more content-driven the early sampling stage (high signal-to-noise ratio (SNR)) and the more stable the structure; conversely, images with simple structures can be injected earlier, enhancing the style effect while maintaining the structure.

[0033] S4: Obtain the calibrated latent variables by injecting latent distribution reanchoring noise into the initial mixed noise latent variables; Latent distribution re-anchoring is achieved by injecting a tiny amount of Gaussian noise during the initialization phase outside the sampling loop. In the sampling process of the diffusion model, the distribution characteristics of the initial mixed noise latent variables directly affect the quality of the final generated image. The latent variables after LCR statistical alignment may be too deterministic, lacking the necessary randomness to recover high-frequency details. Therefore, LDR-noise is injected into the latent variables after LCR, and the calculation expression is as follows: in, for Latent variables after time-calibration These are latent variables in the diffusion model. For noise intensity, For LDR-noise; The default value is 0.03. The forward process of the diffusion model is defined as follows: At time step T Approximately obey ,in, This represents the conditional probability distribution of the forward process in the diffusion model. In the diffusion model Noisy samples at any given time, This represents the cumulative noise figure for the forward process in the diffusion model. It is an identity matrix. After statistical alignment... Although it matches the statistical properties of the style, it may deviate from the distribution assumptions made during the training of the diffusion model. The injection of LDR-noise will... Migrating from deterministic alignment to a sample point of a Gaussian distribution aligned by covariance enhances the diversity and detail recovery of the distribution while maintaining the effect of statistical alignment.

[0034] like Figure 2 As shown, S5: The content self-attention features and style self-attention features are fused through the Soft & Dynamic Injection inSelf-Attention (SDIS) module to obtain the fused attention features; The soft + dynamic style injection module includes key-value soft injection and dynamic parameter adjustment; Key-value soft injection includes: S511: Obtain the sampling progress and normalize the sampling progress; S512: Construct the mixing coefficients based on the normalized sampling progress; The blending coefficient controls the intensity of style information injection; As the sampling process linearly increases from a small initial value of the mixing coefficient to a larger final value, the expression for calculating the mixing coefficient is: in, for The mixing coefficient at time 10:00 This is the initial value for the mixing coefficient. This is the final value for the mixing coefficient. for Sampling progress after time-normalization; , , Corresponding to the injection start time, The corresponding sampling end time.

[0035] In some specific embodiments of the present invention, the following are set , This design ensures that content is prioritized in the early stages of sampling, while gradually increasing the weight of style information as the sampling process progresses, thus avoiding the destructive impact of hard replacement on the content structure in the early stages.

[0036] S513: Based on the mixing coefficient, perform linear interpolation between the keys in the content self-attention feature and the keys in the style self-attention feature to obtain the keys for fused attention; Linear interpolation is performed on the keys in the content branch and the keys in the style branch, and the expression for calculation is: in, for The key to constantly integrating attention for Attention key to content at any moment for The style attention key of the moment for The mixing coefficient at time; S514: Based on the mixing coefficient, perform linear interpolation between the values ​​in the content self-attention feature and the values ​​in the style self-attention feature to obtain the value of fused attention; Linear interpolation is performed on the values ​​in the content branch and the style branch, and the calculation expression is: in, for The value of attention is constantly integrated. for Attention value of content at any given moment for The style attention value at any given moment.

[0037] Linear interpolation is used to smoothly transition between key and value features of content and style, avoiding the sudden introduction of style information in the early stages.

[0038] Hard replacement is equivalent to performing discontinuous jump operations in the feature space: This discontinuity introduces distribution drift during diffusion sampling, leading to instability in the content structure. Soft injection, however, achieves a continuous transition through linear interpolation, forming a smooth path from content to style in the feature space, maintaining the continuity and stability of the sampling process. This idea echoes the progressive blending and latent calibration mechanisms proposed in the h-space injection method, but this invention applies progressive blending to the self-attention feature level, avoiding the distribution mismatch problem that may result from directly modifying latent variables.

[0039] Dynamic parameter scheduling: The query retention rate and attention temperature are designed as functions of time step t to achieve a smooth transition from structure-preserving to strong style.

[0040] Dynamic parameter adjustment includes: S521: Use cosine scheduling to smoothly transition from a high initial value to a low termination value to construct the query retention rate hyperparameter; The requirements for content retention and style injection differ at different sampling stages. Early stages require stronger content retention to stabilize the structure, while later stages can moderately reduce the retention rate to enhance style transfer. Therefore, the query retention rate hyperparameter... The function is designed as a time step, employing cosine scheduling to smoothly transition from a higher initial value to a lower final value. This is because cosine scheduling is smoother at transition boundaries compared to linear scheduling, helping to avoid visual artifacts caused by abrupt parameter changes. The calculation expression is as follows: in, To query the initial value of the retention rate hyperparameter, To query the termination value of the retention rate hyperparameter, for Sampling progress after time-normalization; S522: Based on the intermediate results of the mixed content query and migration diffusion model using the query retention rate hyperparameter, obtain the final query in the self-attention process.

[0041] In self-attention computation, query features determine the spatial layout of the attention graph; therefore, maintaining query stability is crucial for preserving the content structure. This invention uses intermediate results from a hybrid content query and a migration diffusion model to achieve the final query in self-attention, and the computation expression is as follows: in, for The final query in self-attention at any given moment. for Query the retention rate hyperparameter at all times. To compute the projection layer of the query vector, These are latent variables in the diffusion model. Figure 2 ResbIk is the module for calculating the projection layer of the query vector.

[0042] The need for attention sharpening varies at different sampling stages. In the early stages, moderate sharpening is sufficient to maintain a clear content structure, while later stages require stronger sharpening to highlight stylistic and textural details. Therefore, this invention uses temperature scaling parameters... Extended to a linear scheduling function, this scheduling ensures stronger temperature scaling is applied later in the sampling process (low signal-to-noise ratio stage), synergizing with the increased style injection intensity. The temperature scaling parameter is constructed based on the normalized sampling progress, and its calculation expression is as follows: in, for Temperature scaling parameters at any given time This is the initial value for the temperature scaling parameter. This is the termination value for the temperature scaling parameter; The attention map before softmax in the self-attention layer is magnified according to the temperature scaling parameter; A temperature scaling parameter is added to the calculation of self-attention Atten. Enlarging the attention map before softmax can effectively sharpen the attention distribution; the calculation expression is as follows: in, To integrate attention features, For feature dimension, The softmax activation function is used. This is the transpose of the matrix; Key-value soft injection and dynamic parameter adjustment and scheduling only take effect at designated attention layers within the diffusion sampling loop, and are applied by default to layers 6-11 of the decoder. These layers are mainly responsible for generating local texture details, which is highly consistent with the goals of style transfer. Combined with the second-order statistical alignment of the initial latent variables of LCR, the soft injection and dynamic scheduling mechanism form a complete style transfer process: LCR establishes a priori color and naturalness at the diffusion starting point, soft injection progressively transfers texture features during sampling, and dynamic scheduling ensures the optimal balance at different stages.

[0043] S6: During the backsampling of the calibrated latent variables, the fused attention features are progressively injected into the attention layer of the diffusion process at the injection starting point to perform diffusion denoising and generate a style transfer image.

[0044] LCR, SDIS, and adaptive injection timing work synergistically. The second-order statistical alignment of the initial latent variables establishes a good prior for the entire diffusion process, enabling subsequent feature injection to focus on texture and detail transfer, thus realizing the design concept of naturalness, color, and texture separation optimization.

[0045] In the style transfer framework of the diffusion model, the self-attention layer plays a crucial role in transferring texture features from the style image to the content image. The video generation task leverages self-attention (SA) to ensure consistency of subject matter, background, and action coherence between frames; it preserves details by injecting inverted key and value features from the style image, while retaining overall content through query features extracted from the content inverted image. The core idea of ​​the soft + dynamic scheduling mechanism's self-attention module is to extract key and value features from the self-attention layer of the style branch to adjust corresponding features in the stylization branch, thereby achieving effective transfer of high-level style attributes (such as texture). Linear interpolation facilitates a smooth transition between the key features of content and style, avoiding the sudden introduction of style information in the early stages. The key-value soft injection mechanism, given the content self-attention features and style self-attention features collected during the DDIM inversion process at each time step, achieves progressive style transfer through linear interpolation at the key-value feature tensor level. Simply using the injection method mentioned in SDIS cannot fully guarantee the correct preservation of content structure. The timing of injection is also crucial for balancing content structure preservation and style transfer. Injecting too early will cause the content structure to be damaged by premature style information; injecting too late will result in insufficient style intensity and difficulty in fully transferring texture details. Existing methods usually use a fixed injection starting point, which limits the generalization ability of the method. In addition, multiple unstructured modifications to latent variables will cause distribution drift and loss of high-frequency details. Injecting quality enhancement noise at the end of sampling can effectively restore details, but gradual injection will disturb the stability of the sampling cycle. We adopt two complementary mechanisms: Automatic Localization and Latent Distribution Re-anchoring Module (AS-LDR). The combination of the two mechanisms ensures the stability of content structure in the high SNR stage (early sampling) and gradually introduces style information in the low SNR stage (late sampling), achieving the optimal balance between structure preservation and style intensity.

[0046] To verify the superiority of the present invention, quantitative comparative experiments were conducted with a variety of advanced algorithms, including six diffusion-based methods (StyleID, StyleAlign, InstantStyle, DiffuseIT, InST, DiffStyle) and three traditional style transfer methods (AesPA-Net, AdaAttN, AdaIN). The quantitative experimental results are shown in Table 1.

[0047] Table 1 Quantitative Experimental Results

[0048] In Table 1, LPIPS is an image similarity metric measuring the perceptual level, FID is an index measuring the distribution difference between the generated image and the real image, and ArtFID is a variant of FID for artistic styles. "↓" indicates that the smaller the value, the better. This invention achieves the best score (0.4863) in content fidelity LPIPS, significantly outperforming other methods (StyleID: 0.5055, AesPA-Net: 0.5135, AdaAttN: 0.5439). This advantage is mainly attributed to the soft injection and dynamic scheduling mechanism of this invention. By progressively mixing key-value features instead of hard replacement, it avoids the destructive impact of style information on the content structure in the early sampling stage. The injection starting point is adaptively determined according to the edge complexity of the content image, further protecting images with rich structural details, thereby achieving excellent content structure preservation capabilities.

[0049] In terms of Style Fidelity Index (FID), this invention achieves 18.215, slightly lower than StyleID's 18.131 (the best). This difference mainly stems from the fact that StyleID and AdaIN are more aggressive in style transfer intensity, achieving better results in style feature matching. However, this aggressive style transfer often comes at the cost of content fidelity, resulting in a lower overall performance than this invention.

[0050] Finally, the overall style transfer quality was quantified by jointly evaluating Content Preservation Persistence (LPIPS) and Style Reproduction (FID). Although the present invention performs slightly worse than StyleID in FID, its significant advantage in LPIPS results in the optimal overall ArtFID metric, indicating that the present invention can achieve accurate style reproduction while maintaining the integrity of the content structure in style transfer tasks, providing better overall quality.

[0051] To verify the effectiveness of the proposed component, quantitative ablation experiments were conducted, and the quantitative results of the ablation experiments are shown in Table 2.

[0052] Table 2 Quantitative results of ablation experiments

[0053] Table 2 shows that removing the LCR module (without LCR) increases both FID and ArtFID, indicating a significant decrease in style matching ability and overall quality. This demonstrates that LCR is crucial for capturing and transferring the complex color covariance between channels in style images. Removing the SDIS module (without SDIS) increases both FID and LPIPS, reflecting both the critical role of the SA module in injecting style and the importance of dynamic + soft injection in maintaining content structure. Hard or fixed injection strategies cause style information to prematurely or abruptly intrude into content features, leading to structural loss and instability. Therefore, it can be inferred that SDIS is the key to striking a balance between maintaining structural stability and achieving strong style transfer. Finally, removing the AS-LDR module (without AS-LDR) still results in worse performance than the complete model, but the regression is less than when SDIS is removed. This indicates that the main contribution of the AS-LDR mechanism lies in enhancing detail quality and diversity by adding small perturbations to the initial latent variables, thereby generating richer texture details.

[0054] Compared to diffusion-based methods (such as StyleID, CSGO, DiffuseIT, InST, etc.), these methods generally suffer from serious loss of content structure or unreasonable style transfer, or distort the content form (such as changing character features or scene structure), or the stylization effect is disconnected from the content (such as adding irrelevant elements or color imbalance). This invention, however, firmly maintains the essential structure of the content, accurately conveys the visual characteristics of the style, and the generated results show excellent performance in both structural fidelity and style integration.

[0055] Compared to traditional neural network-based methods (such as AdaAttn, AdaIN, and AesPA-Net), this invention excels in balancing content structure preservation and style feature transfer. Traditional methods often suffer from insufficient structure preservation or abrupt style integration, resulting in either loss of content details or jarring stylization effects. This invention can highly restore the core structure of the content (such as character expressions, scene outlines, and portrait features) while naturally and subtly integrating stylistic elements, achieving a harmonious unity between style and content.

[0056] like Figure 3 As shown, a diffusion style transfer system based on latent distribution re-anchoring dynamic injection is used to execute a diffusion style transfer method based on latent distribution re-anchoring dynamic injection, including: The feature extraction module 101 acquires the content image and style image, and inverts the content image to the high-noise state of the latent space through the inverse diffusion process of the denoising diffusion implicit model to obtain the content latent variables and content self-attention features. The style image is inverted to the high-noise state of the latent space through the inverse diffusion process of the denoising diffusion implicit model to obtain the style latent variables and style self-attention features. Alignment module 102 performs second-order statistical alignment of content latent variables and style latent variables using the latent covariance recoloring method to obtain initial mixed noise latent variables; The automatic localization module 103 calculates the grayscale image of the content image using the Sobel gradient operator, calculates the edge intensity using the grayscale image of the content image, and automatically locates the style injection starting point based on the edge intensity. The re-anchoring module 104 obtains the calibrated latent variables by injecting latent distributed re-anchoring noise into the initial mixed noise latent variables; The soft + dynamic style injection module 105 fuses content self-attention features and style self-attention features to obtain fused attention features; During the backsampling of the calibrated latent variables, the transfer generation module 106 progressively injects the fused attention features into the attention layer of the diffusion process at the injection starting point to generate a style transfer image through diffusion denoising.

[0057] Through the collaborative work of the above modules, progressive style injection avoids the destructive impact of style information on the content structure in the early sampling stage. The injection starting point is adaptively determined according to the edge complexity of the content image, thus protecting images with rich structural details. This achieves excellent content structure preservation capabilities while realizing accurate style reproduction.

[0058] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, and not to limit them; although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features; and these modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A diffusion-style transfer method based on latent distribution re-anchoring dynamic injection, characterized in that, include: S1: Obtain the content image and style image. Through the denoising diffusion implicit model inverse diffusion process, the content image is inverted to the high-noise state of the latent space to obtain the content latent variables and content self-attention features. Through the denoising diffusion implicit model inverse diffusion process, the style image is inverted to the high-noise state of the latent space to obtain the style latent variables and style self-attention features. S2: The latent variables of content and style are second-order statistically aligned using the latent covariance recoloring method to obtain the initial mixed noise latent variables; S3: Calculate the grayscale image of the content image using the Sobel gradient operator, calculate the edge intensity using the grayscale image of the content image, and automatically locate the style injection starting point based on the edge intensity; S4: Obtain the calibrated latent variables by injecting latent distribution reanchoring noise into the initial mixed noise latent variables; S5: The content self-attention features and style self-attention features are fused through the soft + dynamic style injection module to obtain the fused attention features; S6: During the backsampling of the calibrated latent variables, the fused attention features are progressively injected into the attention layer of the diffusion process at the injection starting point to perform diffusion denoising and generate a style transfer image.

2. The diffusion style transfer method based on latent distribution re-anchoring dynamic injection according to claim 1, characterized in that, Step S2 includes: S21: Flatten the implicit variables of content and style in the spatial dimension; S22: Calculate the decentralized characteristics of the flattened content latent variables and style latent variables; S23: Calculate the content covariance matrix based on the decentralized characteristics of the flattened content latent variables, and calculate the style covariance matrix based on the decentralized characteristics of the flattened style latent variables. S24: Perform eigenvalue decomposition on the content covariance matrix and style covariance matrix to obtain content eigenvalues ​​and style eigenvalues; S25: Construct a whitening matrix based on content feature values, and a coloring matrix based on style feature values; S26: Construct a latent covariance recoloring transformation matrix based on the whitening matrix and the coloring matrix. Align the content latent variables and style latent variables with second-order statistics based on the latent covariance recoloring transformation matrix to obtain the initial mixed noise latent variables.

3. The diffusion style transfer method based on latent distribution re-anchoring dynamic injection according to claim 1, characterized in that, Step S3 includes: S31: Normalize the content image and calculate the grayscale image of the normalized content image; S32: Calculate the horizontal and vertical gradient maps of the normalized grayscale image of the content image using the Sobel gradient operator; S33: Calculate the edge strength based on the gradient map in the horizontal direction and the gradient map in the vertical direction; S34: Normalize the edge intensity and linearly map it to the injection start point interval to obtain the injection start point.

4. The diffusion style transfer method based on latent distribution re-anchoring dynamic injection according to claim 1, characterized in that, The expression for calculating the calibrated latent variables is: in, for Latent variables after time-calibration for Latent variables in the time-diffusion model For noise intensity, For latent distribution re-anchoring noise, It follows a standard normal distribution.

5. The diffusion style transfer method based on latent distribution re-anchoring dynamic injection according to claim 1, characterized in that, The soft + dynamic style injection module includes key-value soft injection and dynamic parameter adjustment.

6. The diffusion style transfer method based on latent distribution re-anchoring dynamic injection according to claim 5, characterized in that, Key-value soft injection includes: S511: Obtain the sampling progress and normalize the sampling progress; S512: Construct the mixing coefficients based on the normalized sampling progress; S513: Based on the mixing coefficient, perform linear interpolation between the keys in the content self-attention feature and the keys in the style self-attention feature to obtain the keys for fused attention; S514: Based on the mixing coefficient, perform linear interpolation between the values ​​in the content self-attention feature and the values ​​in the style self-attention feature to obtain the value of fused attention.

7. The diffusion style transfer method based on latent distribution re-anchoring dynamic injection according to claim 5, characterized in that, Dynamic parameter adjustment includes: S521: Use cosine scheduling to smoothly transition from a high initial value to a low termination value to construct the query retention rate hyperparameter; S522: Based on the intermediate results of the mixed content query and migration diffusion model using the query retention rate hyperparameter, obtain the final query in the self-attention process.

8. The diffusion style transfer method based on latent distribution re-anchoring dynamic injection according to claim 7, characterized in that, Dynamic parameter adjustment also includes: Temperature scaling parameters are constructed based on the normalized sampling progress; The attention map before softmax in the self-attention layer is magnified based on the temperature scaling parameter.

9. The diffusion style transfer method based on latent distribution re-anchoring dynamic injection according to claim 5, characterized in that, The key-value soft injection and the dynamic parameter adjustment only take effect in the N attention layers within the diffusion sampling loop.

10. A diffusion style transfer system based on latent distribution re-anchoring dynamic injection, characterized in that, To perform a diffusion style transfer method based on latent distribution re-anchoring dynamic injection as described in any one of claims 1 to 9, comprising: The feature extraction module acquires content images and style images. Through the denoising diffusion implicit model inverse diffusion process, the content image is inverted to a high-noise state in the latent space to obtain content latent variables and content self-attention features. Through the denoising diffusion implicit model inverse diffusion process, the style image is inverted to a high-noise state in the latent space to obtain style latent variables and style self-attention features. The alignment module performs second-order statistical alignment of content latent variables and style latent variables using the latent covariance recoloring method to obtain the initial mixed noise latent variables. An automatic positioning module calculates the grayscale image of the content image using the Sobel gradient operator, calculates the edge intensity using the grayscale image of the content image, and automatically locates the style injection starting point based on the edge intensity. The re-anchoring module obtains the calibrated latent variables by injecting latent distribution re-anchoring noise into the initial mixed noise latent variables; The soft + dynamic style injection module fuses content self-attention features and style self-attention features to obtain fused attention features. The transfer generation module, during the backsampling of the calibrated latent variables, progressively injects the fused attention features into the attention layer of the diffusion process at the injection starting point to generate a style transfer image through diffusion denoising.

Images