A method and system for generating a Q version digital head portrait and a storage medium

Abstract

The application belongs to the field of image processing and computer vision, and discloses a generation method and system of a Q version digital portrait and a storage medium, the method comprising: acquiring a pre-trained image generation model, a user real portrait image and a Q version template image; generating a structure control signal representing the composition and posture of the Q version template image; extracting the identity features of the user portrait to obtain global identity features and local semantic enhancement features, and acquiring text prompt information of the user; inputting the text prompt information, the structure control signal, the global identity features and the local semantic enhancement features into the image generation model integrated with multiple style fine-tuning models; processing and outputting an RGB person image; and finally removing the background to generate a Q version digital portrait with a transparent channel. The application solves the problems of identity feature distortion, uncontrollable posture composition, single style and difficulty in direct industrial application of the output result in the strong stylization migration process.

Description

Technical Field

[0001] This invention belongs to the field of image processing and computer vision technology, specifically relating to a method, system, and storage medium for generating personalized Q-version digital avatars. Background Technology

[0002] In recent years, diffusion model-based image generation technology has made significant progress in fields such as digital art, virtual avatar construction, and personalized content production. Diffusion model-based image generation technologies (such as the Stable Diffusion series), adapter technologies for identity preservation (such as IP-Adapter), ControlNet technologies for structural control, and low-rank fine-tuning technologies (LoRA) have become common knowledge or hot research directions in this field. The technology of automatically converting real portraits into Q-version style images has become a hot research and application area due to its huge commercial value in scenarios such as customized badges, cultural and creative gifts, social avatars, and digital collectibles.

[0003] Current mainstream technical solutions mainly fall into two categories: one is text-driven image generation models, which guide the model to generate cartoon-style images by inputting descriptive words such as "Q version" or "big eyes"; the other is image-guided identity transfer methods, which attempt to preserve the original person's identity features in the generated result by referencing a real human image as input. To improve the controllability of the generation process, structural control techniques have been introduced, using edge maps, depth maps, and other methods to impose spatial constraints on the composition and pose of the generated image.

[0004] However, when the above-mentioned technologies are applied to the generation of chibi-style avatars, which require highly stylized characters and strict requirements for identity recognition and structural consistency, the following shortcomings still exist.

[0005] I. Identity Distortion: In the process of strong style transfer, the pursuit of a "cute" effect can easily lead to distortion of the original person's facial proportions, resulting in a loss of recognizability. Existing identity guidance technologies lack fine semantic perception and control of key facial areas.

[0006] 2. Uncontrollable Pose and Composition: Most methods rely on random sampling to determine the pose and composition of the character, making it impossible to align with the user-specified chibi template with standardized proportions and perspectives. If structural control techniques do not effectively coordinate with the identity preservation module, the generated images often result in "correct structure but lost identity" or "identity preserved but chaotic composition."

[0007] Third, the style is monotonous and lacks template-driven approach: existing solutions mostly rely on fixed model fine-tuning weights or prompt words to achieve style control, which makes it difficult to flexibly adapt to various Q-version sub-styles, and also lacks a template-based parameterized guidance system.

[0008] Fourth, the output results are difficult to apply in industrial applications: the generated images often contain complex backgrounds, have different proportions or blurry edges, and require additional tedious post-processing such as image cutout and cropping, making it impossible to directly and seamlessly connect with downstream industrial production processes.

[0009] Therefore, designing a solution that can simultaneously ensure high identity fidelity, high structural controllability, and multi-style adaptability during Q-version style transfer, and output high-quality transparent background avatars that can be directly used for industrial production, is a technical problem that urgently needs to be solved. Summary of the Invention

[0010] The purpose of this invention is to overcome the above-mentioned defects of the prior art and provide a method and system for generating Q-version avatars that integrates identity preservation and structural control, so as to solve the technical problems of easy distortion of identity features, uncontrollable pose composition of generated images, single style and difficulty in direct industrial application of output results during strong stylization transfer.

[0011] In accordance with the above objectives, a first aspect of the present invention provides a method for generating a Q-version digital avatar, the method comprising:

[0012] Acquire pre-trained image generation models, real user portrait images, and Q-version template images;

[0013] Based on the Q-version template image, generate structural control signals that characterize its composition and posture;

[0014] The user's real portrait image is used to extract identity features to obtain global identity features and local semantic enhancement features, and the user's text prompt information is obtained.

[0015] The text prompt information, structural control signals, global identity features, and local semantic enhancement features are input into the image generation model; wherein, the image generation model integrates multiple style fine-tuning models to collaboratively control the style, identity fidelity, and composition of the generated image;

[0016] The image generation model is used to process and output an RGB portrait image.

[0017] The background of the RGB character image is removed to generate a Q-version digital avatar with an alpha channel.

[0018] Preferably, the image generation model integrates multiple style fine-tuning models, which is achieved by linearly superimposing multiple low-rank weight matrices of LoRA models corresponding to different styles with set style weight coefficients. The style fine-tuning model includes at least two of the following: cartoon style, sticker style, character expression, or age representation.

[0019] Preferably, the processing of the image generation model includes:

[0020] High-frequency detail enhancement processing: By applying a high-frequency enhancement operator to the skip connection path of the U-Net network in the image generation model, the skip connection feature map on the path is subjected to high-pass filtering to enhance the high-frequency components in the feature map, thereby improving the detail clarity of the generated image;

[0021] Batch style alignment processing: When generating multiple Q-version digital avatars in batches, the distribution of intermediate features in the same batch generation process is constrained and aligned to ensure the style consistency of the output avatars.

[0022] Preferably, in the high-frequency detail enhancement processing, the high-frequency enhancement operator is a multi-scale adaptive enhancement operator, and its processing includes:

[0023] At at least two different scales s, the input skip connection feature map Fk is subjected to high-pass filtering to obtain the high-frequency components Hs(Fk) at each scale.

[0024] Global statistical features of the skip connection feature map Fk are extracted and combined with the current generation time step information and style weight coefficients, and input into a multilayer perceptron (MLP) to dynamically predict the enhancement weights λs corresponding to each scale.

[0025] The high-frequency components Hs(Fk) at each scale are multiplied by the corresponding enhancement weights λs and summed to obtain the total high-frequency enhancement amount. This is then added to the input skip connection feature map Fk to obtain the enhanced feature map F'k, i.e.: F'k = Fk + Σ(λs · Hs(Fk)).

[0026] Preferably, the batch style alignment process specifically includes:

[0027] In at least one intermediate layer of the diffusion model denoising process, feature maps of all samples in the same batch are obtained;

[0028] Calculate the mean μ and standard deviation σ of the feature map in the channel dimension for this batch;

[0029] Construct a learnable mask vector M with the same number of channels as the feature map, where the value of M ranges from [0, 1].

[0030] For each sample in the batch, its aligned feature map F align The calculation is as follows:

[0031] F align = M ⊙ [(F - μ) / σ] + (1 - M) ⊙ F

[0032] Where ⊙ represents channel-by-channel multiplication, and F is the original feature map of the sample.

[0033] Preferably, the structure control signal generated based on the Q-version template image is achieved by extracting the edge contour of the Q-version template image through the edge detection module and generating a binarized edge map.

[0034] Preferably, the extraction of identity features from the user's real portrait image includes:

[0035] The portrait image is encoded into a global visual embedding vector using a first identity encoder;

[0036] By combining the second identity encoder with facial key point information, local feature enhancement encoding is performed on the eye, mouth or face shape regions to obtain local semantic enhancement features;

[0037] Based on the global visual embedding vector and local semantic enhancement features, identity conditions are generated to guide the generation process.

[0038] Furthermore, identity conditions are generated to guide the generation process, including:

[0039] Calculate the mutual attention score matrix between the global visual embedding vector and the local semantic enhancement feature; generate an adaptive fusion weight matrix based on the mutual attention score matrix; use the adaptive fusion weight matrix to perform channel weighting on the local semantic enhancement feature, and then concatenate the weighted local semantic enhancement feature with the global visual embedding vector to generate the final identity condition vector.

[0040] Preferably, the text prompt information is formed by concatenating the prompt words entered by the user with preset Q-version style template words, and is converted into a semantic vector by a text encoder.

[0041] Preferably, background removal of the RGB character image is performed by extracting the main character using a semantic segmentation model and outputting it as an image with an alpha channel.

[0042] A second aspect of the present invention provides a system for generating Q-version digital avatars, the system comprising:

[0043] The data acquisition module is used to acquire pre-trained image generation models, real user portrait images, and Q-version template images;

[0044] The structure control signal generation module is used to generate structure control signals that characterize the composition and posture of the Q-version template image based on the Q-version template image.

[0045] The feature extraction module is used to extract identity features from the user's real portrait image to obtain global identity features and local semantic enhancement features, as well as obtain the user's text prompt information;

[0046] The collaborative generation module integrates the image generation model and multiple style fine-tuning models. It is used to receive the text prompt information, structural control signals, global identity features and local semantic enhancement features, and collaboratively control the generation process to output an RGB human image.

[0047] The post-processing module is used to remove the background from the RGB character image and generate a Q-version digital avatar with an alpha channel.

[0048] Preferably, the identity feature extraction module includes:

[0049] The first identity encoding unit is used to encode the portrait image into a global visual embedding vector;

[0050] The second identity encoding unit is used to combine facial key point information to perform local feature enhancement encoding on a preset facial region.

[0051] The post-processing module includes a semantic segmentation unit, which is used to accurately extract the main subject of the person and render a transparent background.

[0052] In a third aspect, the present invention provides a computer storage medium having a computer program stored thereon, which, when executed by a processor, implements the steps of the method for generating Q-version digital avatars as described in the first aspect.

[0053] Compared with the prior art, the method, system, and storage medium for generating Q-version digital avatars disclosed in this invention achieve the following technical effects:

[0054] I. Effective fidelity of original identity features: Through the dual-channel identity guidance mechanism of the first identity encoder and the second identity encoder, on the basis of global feature similarity, the preservation and fine adjustment of key local features such as eyebrows, eyes and mouth shape are strengthened, which effectively solves the identity distortion problem in strong style transfer. The generated Q version avatar has a high degree of recognizability with the person.

[0055] II. Structural controllability: By extracting the edge image of the Q-version template as a structural control signal and using a structural control network to inject it as a strong constraint into the generation process, the pose, head orientation and overall composition of the generated image can be strictly aligned with the template specified by the user, meeting the production requirements of standardized products such as badges and stickers.

[0056] 3. Flexible multi-style fusion capability: By integrating multiple style fine-tuning models and supporting weight coefficient adjustment, it can flexibly mix various style elements such as cartoon, stickers, and specific expressions, realizing refined and parameterized control of Q version sub-styles and breaking through the limitation of a single style.

[0057] IV. End-to-End Industrial Output: The end of the process integrates a high-precision automatic background removal module, directly outputting standard format images with transparency channels. These images feature standardized composition, clear edges, and high resolution, and can be directly applied to downstream industrial production processes such as digital avatars, virtual characters, and peripheral product design without any additional post-processing, thereby improving production efficiency and automation levels.

[0058] V. Improved Detail Quality and Batch Consistency: Through optional high-frequency detail enhancement processing, high-frequency details such as hair strands and textures are effectively preserved during strong stylization, improving the image's refinement and industrial product quality; through optional batch style alignment processing, it is ensured that in batch production scenarios, the avatars generated from different input sources maintain a high degree of consistency in style intensity and performance, meeting the stringent requirements for uniformity in serialized products.

[0059] VI. This invention constructs a synergistic mechanism within an image generation model that integrates multiple style fine-tuning models by constructing a triple guiding condition of "text (style semantics) + image (identity features) + structure (template edges)". This mechanism ensures that the control signals of the three dimensions of style, identity, and structure are comprehensively considered and balanced in every step of image generation, thereby generating high-quality results that simultaneously meet multiple stringent requirements in one go. Attached Figure Description

[0060] Figure 1 This is a flowchart illustrating the method for generating Q-version digital avatars in an embodiment of the present invention;

[0061] Figure 2 This is a schematic diagram of the architecture of the Q-version digital avatar generation system in an embodiment of the present invention;

[0062] Figure 3 This is a schematic diagram of the structure of an electronic device in an embodiment of the present invention. Detailed Implementation

[0063] The following embodiments are only used to more clearly illustrate the technical solutions of the present invention and should not be construed as limiting the scope of protection of the present invention. Certain terms are used in the specification and claims to refer to specific components. Those skilled in the art will understand that hardware or software manufacturers may use different names to refer to the same component. This specification and claims do not distinguish components based on differences in name, but rather on differences in function. The following descriptions of preferred embodiments of the present invention are intended to illustrate the general principles of the invention and are not intended to limit the scope of the invention. The scope of protection of the present invention shall be determined by the appended claims.

[0064] The present invention will now be described in further detail with reference to the accompanying drawings and specific embodiments.

[0065] Please see Figure 1 Embodiment 1 of the present invention provides a method for generating a Q-version digital avatar, which receives three types of input data: first image data (a real portrait image of the user), second image data (a Q-version template image), and text string data (a user style description). Through a series of predefined data conversion and processing units, these input data are processed collaboratively to finally output fourth image data (a Q-version digital avatar file with an alpha channel). The method includes the following:

[0066] Step S101: Obtaining model parameter data.

[0067] The system first reads the pre-trained base diffusion model parameter file, text encoder parameter file, and image decoder parameter file from the storage medium. These parameter files define the computation rules of the core data processing unit. The system loads these parameters into memory, completing the initialization of the main framework for image generation. In this embodiment, the loaded model parameters correspond to a specific diffusion model architecture. DreamShaperXL is selected as the base model, which is based on the Stable DiffusionXL architecture. This architecture is designed to process and generate 1024×1024 pixel RGB image data. This model architecture also includes: a CLIP text encoder to convert user-input prompts into semantic vectors, and a VAE (Variational Autoencoder) decoder to decode the latent variables output by the diffusion model into a visible RGB image. It also loads a subsequent edge-detection-based structural control network and a semantic segmentation model for background removal.

[0068] To enable the basic diffusion model and the newly added learnable components (MLP in the multi-scale adaptive high-frequency enhancement operator, the learnable mask vector M in batch style alignment, and the identity feature fusion module) to collaboratively adapt to the Q-version digital avatar generation task, the above components need to be jointly fine-tuned and trained. The specific process is as follows:

[0069] The training set contains over 50,000 pairs of data (real user portrait images + corresponding Q-version template images + text prompt annotations), covering different ages, genders, and poses; the validation set contains over 10,000 pairs of data, all in a uniform 1024×1024 RGB image format. The portrait images undergo face detection and cropping preprocessing, the Q-version templates are converted to binary edge maps through edge detection, and the text prompts are generated by concatenating "user-defined words + preset Q-version style template words". The training objective is optimized using a multi-objective loss function, with the total loss formula: T = α×S1 + β×S2 + γ×S3 + δ×S4. Here, S1 is the style consistency loss, and its weight coefficient α is configurable, for example, it can be set to 1.0. One implementation of this loss term is to use a VGG19 network to extract features from the generated image and the target style image respectively, and calculate the feature distance between them (such as L2 distance). S2 is the identity preservation loss, with a configurable weight coefficient β, for example, it can be set to 1.2. One implementation of this loss term is to use the ArcFace network to extract features from the generated image and the input portrait image respectively, and calculate the cosine distance between their feature vectors. S3 is the structure alignment loss, with a configurable weight coefficient γ, for example, it can be set to 0.8. One implementation of this loss term is to calculate the L1 distance between the edge map of the generated image (obtained through the Canny operator) and the edge map of the Q-version template image. S4 is the reconstruction loss, with a configurable weight coefficient δ, for example, it can be set to 0.5. One implementation of this loss term is to directly calculate the mean square error (MSE) between the generated image and the target image in the RGB pixel space. The training process is as follows: Freeze the backbone parameters of the pre-trained model loaded in S101, and only unfreeze the parameters of the newly added learnable components; input the preprocessed training data into the ensemble model, and execute the forward pass of feature extraction-multi-conditional collaborative generation-image decoding; calculate the total loss T, and update the parameters of the newly added components through backpropagation using the AdamW optimizer; the hyperparameters of the optimizer can use commonly used values ​​in this field, for example, setting the weight decay (weight_decay) to 0.01, the exponential decay rate (β1) of the first moment estimation to 0.9, and the exponential decay rate (β2) of the second moment estimation to 0.999; iterate training for 30 rounds, and use the validation set to select the optimal parameters with convergent loss; save the optimized parameter file, including: the composite style shift matrix, MLP weights, the learnable mask vector M, and the identity fusion module weights. At the beginning of training, all elements of the learnable mask vector M are initialized to 0.5. During the optimization of the total loss T, there are no additional special constraints on the update of M; its gradient comes from the weighted sum of the style consistency loss and the identity preservation loss, and is learned through the standard backpropagation algorithm.

[0070] Step S102: Dynamic fusion of style control parameters.

[0071] To endow the main framework with multi-dimensional style transformation capabilities, the system further reads the weight matrix files of multiple trained Low-Rank Adaptation (LoRA) models from the storage medium. Each weight matrix file corresponds to a specific style fine-tuning direction, and the style fine-tuning model includes at least two of the following: cartoon style, sticker style, facial expression, or age representation. The system receives a set of weight coefficients from an external source and assigns weights to each LoRA model. For example, cartoon-style=1.0, where cartoon-style is a fine-tuned LoRA model that enhances overall cartoonish features, such as simplifying facial features and softening contours; smiling=0.8, where smiling is a fine-tuned LoRA model that enhances the character's smiling expression and adjusts the curvature of the corners of the mouth and the light in the eyes. The data processing unit, according to a preset linear superposition algorithm, sums the weight matrices of each low-rank adaptation model according to their corresponding coefficients to generate a fused composite style offset matrix W, the formula of which is:

[0072] W=L1×W LoRA1+ L2×W LoRA2+...+ L n ×W LoRAn Among them, W LoRAi Let L1, L2, ..., Ln be the low-rank weight matrix of the i-th LoRA model, 1 ≤ i ≤ n, n ≥ 2, and L1, L2, ..., Ln are the weight matrices. n These are the weight coefficients corresponding to each low-rank weight matrix. Subsequently, this composite style offset matrix is ​​dynamically superimposed onto the weights of specific linear layers within the basic diffusion model, thereby achieving personalized stylistic fine-tuning of the generation logic without altering the model architecture. This step enables refined and quantitative control over the style of the generated images. By linearly superimposing the weights of different style fine-tuning models, it is possible to flexibly integrate various Q-version sub-styles such as cartoons, stickers, and emoticons, solving the problem of style monotony.

[0073] Simultaneously, the system can activate two built-in auxiliary data processing units to further improve the quality of the generated data:

[0074] High-frequency detail enhancement unit: This unit is configured to add high-frequency filters to the skip connection paths of the U-Net diffusion model, or to apply high-pass filtering algorithms to the skip connection feature maps, dynamically enhancing the high-frequency components in the feature maps. Through high-frequency detail enhancement, the clarity and layering of details such as hair strands, collar textures, and image edges in the output image can be significantly improved, overcoming the problem of blurred details that may be caused by strong style transfer, and improving the industrial-grade quality of the generated images. For example, in the decoding path of the U-Net network in the image generation model, each skip connection passes through a high-frequency enhancement operator before passing the encoder features to the decoder. This high-frequency enhancement operator first applies a Laplacian operator or a high-pass filter kernel to the skip connection feature map on this path for convolution, extracting its high-frequency components. Then, through a learnable weight parameter λ (the initial value is usually set to 0.1 to 0.3), the enhancement intensity is controlled, and the high-frequency components are weighted and added back to the original feature map to obtain the enhanced feature map.

[0075] The high-frequency enhancement operator of this invention is a multi-scale adaptive enhancement operator. Its processing is as follows: For an input feature map from a U-Net skip connection path, denoted as the skip connection feature map Fk, where k represents the network layer index, the operator performs the following operations:

[0076] At at least two different preset scales *s*, a high-pass filtering operation is performed on *Fk*. For example, a scale set *S* = {s1, s2, s3} can be defined, and convolution operations are performed using different sizes of Difference of Gaussians (DoG) kernels or Laplacian kernels to obtain a set of high-frequency feature maps *Hs(Fk)* representing details at different granularities. In a specific implementation, the high-pass filtering operation uses a Laplacian operator convolution kernel, and the scale set *S* can be defined as corresponding to different convolution kernel sizes, such as 3×3, 5×5, and 7×7. For the input skip connection feature map *Fk*, at each scale *s*, *Fk* is convolved using a Laplacian kernel of the corresponding size to obtain the high-frequency component *Hs(Fk)* at that scale.

[0077] To dynamically fuse high-frequency information at various scales, a set of enhancement weights λs corresponding to each scale needs to be predicted. First, global statistical features of the input feature map Fk are extracted, such as its spatial mean μk and standard deviation σk. Simultaneously, contextual information representing the current generation state is obtained, including:

[0078] Current generation time step information: The time step t of the diffusion model denoising process is usually encoded as a vector temb (e.g., through sinusoidal position encoding).

[0079] Style weight coefficients: A set of user-defined coefficients {L} that control the strength of different style fine-tuning models (LoRA). i}

[0080] The information obtained above includes: feature map statistics [μk, σk], time step encoding temb, and style coefficients {L}. i These conditions are concatenated into a comprehensive conditional vector. This conditional vector is then input into a lightweight multilayer perceptron (MLP). The output layer of this MLP has the same number of neurons as the number of scales and is activated by a sigmoid function, outputting enhancement weights λs for each scale s, where λs ∈ (0, 1). In this way, the model can adaptively determine which scales of high-frequency details to enhance based on the generation stage (early contours or late details), the desired style intensity, and the characteristics of the feature map itself.

[0081] The high-frequency components Hs(Fk) extracted at each scale are weighted and summed with the corresponding weights λs predicted by the MLP to obtain the total high-frequency enhancement. Finally, this enhancement is added back to the skip connection feature map in the form of residuals to obtain the enhanced feature map F'k, where F'k = Fk + Σ(λs · Hs(Fk)).

[0082] Through the above steps of multi-scale extraction, conditional adaptive weighting, and residual fusion, the enhancement operator of this invention can specifically improve the clarity of key details (such as hair strands and texture edges) in the image according to the changes in the generation stage, style intensity, and feature map characteristics, while avoiding the introduction of irrelevant noise, thereby significantly improving the quality of industrial products generated Q-version avatars.

[0083] Batch Style Alignment Processing Unit: This unit is activated when the system is configured to process multiple sets of input data in batches. It performs statistical analysis on the distribution of intermediate features of all data samples within the same batch during the diffusion generation process and applies distribution alignment constraints. Through batch style alignment, it ensures that even for different input portraits, the generated Q-version avatars in the same batch maintain a high degree of consistency in style intensity and rendering techniques. This is crucial for industrial mass production scenarios such as creating serialized badges and team avatars.

[0084] Specifically, when the system performs parallel generation with a batch size of B, at a specific intermediate layer in the diffusion model denoising process (e.g., the intermediate block output of U-Net), it collects the feature map set of all samples in that batch at that layer. The system calculates the mean μ and standard deviation σ of the features in the channel dimension of that batch. Then, using the calculated batch-level mean μ and standard deviation σ, it standardizes and reparameterizes the feature map of each sample in the batch. Specifically, it subtracts the batch mean μ from the feature map of each sample and divides it by the batch standard deviation σ to align the feature distribution. This operation forces all generation processes in the same batch to share the same high-level feature statistics (mean and variance). These statistics implicitly encode style information (such as brushstrokes, color tendencies, and rendering smoothness), thereby ensuring that even if the input real portraits are different, the generated Q-version avatars maintain a high degree of consistency in style, meeting the needs of serialized product production.

[0085] For example, batch style alignment is performed at specific intermediate layers in the diffusion model denoising process (e.g., the 3rd, 6th, and 9th intermediate blocks of U-Net). For each layer, let the feature maps of all samples in the batch be {F(1), ..., F(B)}, and calculate the mean μ and standard deviation σ of the batch features. To preserve identity information, a learnable mask vector M with the same number of channels as the feature maps is introduced, where each element m c ∈[0,1] (constrained by the Sigmoid function). For each sample n, its aligned feature map F align Calculated as: F align = M ⊙ [ (F - μ) / σ ] + (1 - M) ⊙ F, where ⊙ represents channel-wise multiplication, and F is the original feature map of the sample. The learnable mask vector M, together with learnable components such as the identity feature fusion module and the MLP in the high-frequency enhancement operator, is optimized during the end-to-end fine-tuning training of the image generation model (including the basic diffusion model, the structure control network, and the style fine-tuning model). Through the backpropagation algorithm, joint optimization is performed with the objective of minimizing the weighted sum of style consistency loss and identity preservation loss. The training objective simultaneously minimizes the style consistency loss between the generated image and the target image, and the identity feature preservation loss between the generated image and the input portrait. Under this multi-objective optimization, the mask M can spontaneously learn to assign higher weights (m) to the feature channels encoding style information without explicit supervision. c (close to 1), assigning lower weights (m) to the feature channels encoding identity information. c (close to 0), thereby achieving intelligent channel selective alignment.

[0086] Step S103: Input data reception and structural feature extraction.

[0087] The system receives external input through a data interface:

[0088] Receive the first image data: an RGB image containing a human face (a real and clear portrait image of the user), used for the extraction of identity features.

[0089] Receive second image data: an RGB image (Q-version template image) as a composition reference, which contains the target composition, such as a circular head or a specific pose, for structural guidance.

[0090] The system processes the two types of received image data in parallel. For the first image data, the system inputs it into the face analysis unit. This unit automatically executes a face detection algorithm to locate the face region and further extracts the coordinate data of facial feature points through a key point detection algorithm, forming facial key point information. For the second image data, the system inputs it into the edge detection module. This edge detection module automatically executes a series of image processing algorithms, including grayscale conversion, Gaussian blurring, gradient calculation, and non-maximum suppression, ultimately generating a binary edge map of the same size as the input image. This edge map clearly outlines the template's contour, pose, lines, and other structural information with black and white pixels, serving as the structural control signal for subsequent structural control.

[0091] The user's abstract composition intention (template) is transformed into spatial constraint signals that the machine can accurately understand and execute, laying the foundation for strict control over posture and composition in subsequent steps.

[0092] Step S104: Identity feature extraction and encoding.

[0093] Using the data generated in step S103, identity features are extracted through two parallel encoder units:

[0094] The first identity encoder uses an encoder capable of extracting global semantic features from an image, such as, but not limited to, the IPAdapter model. The system loads pre-trained encoder parameters. This encoder takes the first image data as input, encodes it through a deep convolutional neural network, and outputs a high-dimensional global visual embedding vector (i.e., global identity features). This vector is injected into the cross-attention layer of the diffusion model, continuously guiding the identity features throughout the generation process. This vector represents the overall identity information of the input portrait.

[0095] The second identity encoder uses an encoder capable of extracting local semantic features from an image, such as, but not limited to, the PuLID model. The system synchronously loads pre-trained encoder parameters. This encoder receives the first image data and facial keypoint information (including coordinate information) output by the face analysis unit. Based on the keypoint information, it locates specific semantic regions of the face (such as eyes, mouth, and facial contours) and performs more refined feature extraction and enhancement encoding on these regions, generating local semantic enhancement features. For example, for eyes: increasing eye highlights and adjusting eye roundness; for mouth: enhancing the smile curve; for hairline and face shape: maintaining the original contour proportions to prevent facial distortion caused by cartoonish features. Simultaneously, the encoder internally presets structural alignment constraints, ensuring that the generated features maintain similarity to the original face in key geometric proportions during the encoding process. PuLID internally introduces a structural alignment loss function, which uses L2 distance to constrain the difference in keypoint spacing between the generated image and the original image, constraining the similarity of the generated image and the original image in geometric features such as eyebrow-eye distance and nose-mouth proportions, effectively preventing style drift.

[0096] Based on the dual-channel mechanism, the first identity encoder ensures that the generated result is "similar in spirit" (overall sense of identity) to the input portrait and provides global identity guidance, while the second identity encoder ensures "similar in form" (facial features and proportions) through local enhancement and geometric constraints and provides local style fine-tuning. The two work together to fundamentally solve the problem of identity distortion in the process of Q versioning.

[0097] Subsequently, the system uses a feature fusion unit to fuse the global visual embedding vector with the local semantic enhancement features to generate the final identity condition vector.

[0098] In a preferred embodiment of the present invention, assuming that the first identity encoder extracts a global visual embedding vector VG, and the second identity encoder extracts a local semantic enhancement feature vector VL for the eye and mouth regions, VG and VL are input into an identity feature fusion module, which performs the following operations:

[0099] Calculate the mutual attention score matrix A between VG and VL. Map VG and VL to the same feature space through different linear projection layers to obtain the query vector Q (matrix) and key vector K (matrix). Then, calculate the mutual attention score matrix A = Softmax( (Q · K^T) / sqrt(d) ), where d is the projected feature dimension, sqrt(d) is its square root, and Q · K^T represents the mutual attention score between Q and K. The transpose of the vector is used for matrix multiplication, and the Softmax function is performed along the dimension of the key vector to obtain the attention weight matrix A, which represents the strength of the association between global and local features. Based on the mutual attention score matrix A, the attention score matrix A is flattened into a vector, and an adaptive fusion weight matrix Wf is obtained through a weight generation network. The weight generation network can be composed of a two-layer fully connected network. The first layer maps the flattened attention weight matrix A (dimensionality assumed to be L) to the hidden layer (dimensionality can be set to L / 2 or 128) and uses the ReLU activation function. The second layer maps the hidden layer features to the same dimension as the local semantic enhancement feature VL, and finally outputs a weight matrix with the same number of channels as VL through the Sigmoid activation function. Finally, the weighted local semantic enhancement feature vector and the global visual embedding vector are fused through channel concatenation to generate the final identity condition vector VF, i.e., VF = Concat(VG, Wf ⊙ VL), where Concat(·) represents concatenation along the feature channel dimension. The feature fusion unit is optimized during training, enabling the VF to simultaneously contain enhanced representations of global identity information and local key features. By calculating the mutual attention scores between global identity features and local semantic features, an adaptive fusion weight matrix is ​​generated. This allows the fused identity conditional vector to dynamically adjust the contribution ratio of global and local features according to different chibi-style templates, effectively solving the balance problem between preserving identity features and artistic exaggeration during strong stylization.

[0100] Step S105: Generate text semantic condition vector.

[0101] The system first receives externally input text string data (user description) through a data interface. For example, the user description includes: positive prompts such as "Q-version girl, short pink hair, big eyes, cute smile, light purple background"; and negative prompts such as "blurry, deformed, multiple fingers, text, watermark". Simultaneously, based on the task type, the system automatically retrieves and calls a set of predefined Q-version style template words from its built-in template word library. The system then uses a string concatenation unit to merge the user input text and the style template words according to a preset format, forming a guiding prompt string.

[0102] Next, the prompt string is fed into an initialized text encoder (such as the CLIP text encoder). The encoder converts the string into a machine-readable, high-dimensional text semantic conditional vector.

[0103] By combining users' vague style descriptions with professional style templates, powerful and accurate semantic guidance signals are generated, which work in conjunction with image features to jointly control the generated content and style.

[0104] Step S106: Multimodal conditional collaborative control and image generation.

[0105] The system integrates the various conditional vectors generated in the preceding steps to drive the image generation model to generate the target image. The system loads parameters from a pre-trained structural control network (such as ControlNet-Canny). This network is configured as an additional processing branch running in parallel with the base diffusion model. For example, the Canny edge map generated in the preceding steps is input into the ControlNet model loading module, which loads the pre-trained ControlNet-Canny model. This module constructs an independent structure-aware branch within the diffusion model. In each denoising step, the edge map is used as a conditional signal to guide the generated image's head orientation, shoulder pose, and overall compositional proportions (such as centering and top / bottom margins). The guidance strength of ControlNet can be balanced by adjusting the control weights (e.g., 0.75-0.9) to ensure that structural constraints do not suppress detail generation. In each denoising iteration, the identity conditional vector (VF) and the text semantic conditional vector are concatenated along the channel dimension before being input to the cross-attention layer, forming a combined conditional embedding. The guiding weights of the ControlNet-Canny network are fixed at 0.8 during the training phase, and users can fine-tune them in the range of 0.75 to 0.9 during the inference phase to balance the structural compliance and the degree of freedom in generating details.

[0106] The iterative denoising generation process of the image generation model is initiated, and in each iteration:

[0107] Structural condition injection: The structural control signal (binarized edge map) is input into the structural control network, which extracts its spatial structural features and injects the features into the corresponding computational layer of the main generative model to apply precise spatial structural constraints, ensuring that the composition of the generated image is aligned with the template.

[0108] Identity and textual conditional injection: The identity conditional vector and the textual semantic conditional vector are input together into the cross-attention layer of the image generation model. In each computation step, the model's current state features interact with these two conditional vectors, thereby aligning the generated content with both the specified identity features and the semantic textual description.

[0109] The effect of fine-tuning style: The composite style offset matrix fused in step S102 continues to be applied to the model calculation during this generation process to achieve the preset style offset.

[0110] Optional enhancements: If enabled, high-frequency detail enhancements and batch style alignment will take effect simultaneously during this generation process.

[0111] Through the aforementioned mechanism, text, identity, and structure are coordinated in every computational step within the model, jointly guiding the data transformation from random noise to the target image. The technical effect is the realization of integrated and coordinated control over style, identity, and structure, generating high-fidelity, accurately composed, and stylistically distinct Q-version images in a single operation.

[0112] Step S107: Image decoding and background removal.

[0113] The output of the generation process is a data tensor in the latent space. The specific execution is as follows:

[0114] Data Decoding: The data tensor is input into an initialized image decoder (such as a VAE decoder). The decoder reconstructs the latent data tensor into RGB image data of a person in pixel space through a series of deconvolution operations.

[0115] Background stripping: The generated RGB image data of the person is immediately fed into a semantic segmentation model (such as RMBG). This model classifies each pixel of the input image into "foreground" and "background". Then, the model sets the alpha channel value of the "background" category pixels to transparent, while preserving the color and opacity of the "foreground" pixels, thereby generating a new image data containing the transparency channel.

[0116] Data Output: Finally, the system encodes the image data into a standard image format file with an alpha channel (such as PNG), and transmits it through the output interface to form the final Q-version digital avatar with an alpha channel. This produces digital assets with clean backgrounds, standard compositions, and the ability to be directly embedded in any design background or used for physical printing, meeting the format requirements of industrial production for input materials.

[0117] It should be noted that, in this embodiment, the image generation model is based on the publicly available Stable Diffusion XL architecture and integrates publicly available pre-trained models, including but not limited to:

[0118] IPAdapter: Used to extract global identity features;

[0119] PuLID: Used for local semantic enhancement and geometric constraints;

[0120] ControlNet-Canny: Used for structural control;

[0121] RMBG: Used for background removal.

[0122] The models described above are all known and publicly available in the art, and their parameters can be obtained from public code libraries. Those skilled in the art can integrate and call them according to public documentation, and will not be elaborated further here. In addition, the specific models listed above (such as IP-Adapter, PuLID, etc.) are only examples. Any existing or future models, modules or algorithms that can achieve the same or similar functions can be used to implement this invention without departing from the protection scope of this invention.

[0123] Example 2

[0124] Reference Figure 2 As shown, corresponding to Embodiment 1 above, Embodiment 2 of the present invention provides a Q-version digital avatar generation system, which includes:

[0125] The data acquisition module is used to acquire pre-trained image generation models, real user portrait images and Q-version template images, as well as text prompts from users;

[0126] The structure control signal generation module is used to generate structure control signals that characterize the composition and posture of the Q-version template image based on the Q-version template image.

[0127] The feature extraction module is used to extract identity features from the user's real portrait image to obtain global identity features and local semantic enhancement features;

[0128] The collaborative generation module integrates the image generation model and multiple style fine-tuning models. It is used to receive the text prompt information, structural control signals, global identity features and local semantic enhancement features, and collaboratively control the generation process to output an RGB human image.

[0129] The post-processing module is used to remove the background from the RGB character image and generate a Q-version digital avatar with an alpha channel.

[0130] The system's execution steps are as follows:

[0131] Obtain a pre-trained diffusion model as the basis for image generation. Simultaneously, obtain two input images: one is a real portrait image of the user to be processed, and the other is a chibi-style template image defining the target composition and pose.

[0132] To achieve fine-grained control over the style of generated images, multiple independent low-rank adaptive models are obtained. Each model encodes a specific style shift (such as cartoonish, sticker-style, or expression enhancement). A weight coefficient is assigned to each model, and the low-rank weight matrices of these models are fused according to their respective coefficients using a linear superposition algorithm to generate a composite style control weight. This composite weight is then applied to the underlying diffusion model to dynamically adjust its generation behavior, achieving quantitative fusion of multiple style elements.

[0133] The acquired Q-version template image is processed by automatically extracting its contour information using an edge detection algorithm (such as the Canny algorithm). This processing generates a binary edge map that accurately preserves key spatial structural information such as the head contour, shoulder line, and pose angle in the template. This binary edge map will serve as a structural control signal to guide the subsequent image generation process.

[0134] Deep identity feature extraction is performed on the acquired real user portrait images. This process includes two parallel sub-steps:

[0135] Global identity encoding: The entire portrait image is encoded into a high-dimensional global visual embedding vector through a first identity encoder (such as an IP Adapter), which represents the overall identity features of the person.

[0136] Local semantic enhancement encoding: First, facial landmark detection algorithms are used to locate feature points such as eyes, nose, and mouth from the portrait image. Then, a second identity encoder (such as PuLID) is used to combine this landmark information to enhance the encoding of specific semantic regions such as eyes, mouth, and facial contours, generating local semantic enhancement features. Geometric constraints are introduced during the encoding process to maintain the original proportions. The feature extraction module can use a ResNet50 network to extract facial landmarks (corners of the eyes, corners of the mouth, jawline, etc.) and contour features from the base image. The Q-version feature space is a low-dimensional feature space constructed based on principal component analysis (PCA). By inputting the base image features into a pre-trained feature mapping matrix, dimensionality reduction and transformation to the Q-version feature space are completed.

[0137] Finally, the global visual embedding vector is fused with the local semantic enhancement features to form a complete identity condition.

[0138] It also receives custom text prompts (positive and negative prompts) input by the user. This prompt is then concatenated with chibi-style template words (such as "sticker style, chibi") automatically matched from a preset style template library to form a complete guidance prompt string. Subsequently, a text encoder (such as CLIP encoder) is used to convert this string into a machine-readable text semantic conditional vector.

[0139] All the data generated in the preceding steps—textual semantic condition vectors, structural control signals (edge ​​maps), and identity conditions (fusion features)—are simultaneously input into the diffusion model that has already applied composite style control weights.

[0140] During the iterative denoising and generation process of the model, structural control signals are injected through a parallel structural control network (such as ControlNet) to impose precise spatial composition constraints at each generation step. Textual and identity conditions are injected through the model's cross-attention mechanism, guiding the generated content to conform to semantic descriptions and maintain identity features, respectively. The fused style control weights influence the model's generation path throughout, shaping the final chibi-style art style. These conditions work synergistically in every computational step of the model, jointly guiding the transformation from noise to the target image, ultimately outputting an RGB character image that conforms to all constraints.

[0141] Optionally, during image generation, the feature maps in the U-Net skip connection paths can be adjusted to selectively enhance high-frequency components, thereby significantly improving the clarity and sharpness of the output image in details such as hair strands, textures, and edges. When multiple sets of input data need to be processed at once to generate avatars in batches, alignment constraints can be applied to the intermediate feature distributions of all generation tasks within the same batch to ensure that all avatars produced in the batch maintain a high degree of consistency in style rendering intensity and performance.

[0142] The RGB portrait images generated in the above steps undergo automated post-processing. A pre-trained semantic segmentation model (such as RMBG) is used to perform pixel-level classification of the images, accurately identifying and separating the subject from the background. The pixel opacity of the background area is set to full transparency, preserving the color and details of the subject. Finally, the processed image data is encoded into a standard format (such as PNG) file with an alpha channel and output. This file is a finished Q-version digital portrait with industrial application value, which can be used directly without any additional cropping or cutout.

[0143] Example 3

[0144] Please see Figure 3 , Figure 3 This is a schematic diagram of an embodiment of the electronic device of this application.

[0145] Electronic device 1000 includes a memory 200, a processor 300, and a computer program 400 stored in the memory 200 and executable on the processor 300. When the processor 300 executes the computer program 400, it implements the steps described in the above embodiments, for example... Figure 1 The steps S101~S107 of the method for generating Q-version digital avatars are shown.

[0146] Those skilled in the art will understand that the schematic diagram is merely an example of the electronic device 1000 and does not constitute a limitation on the electronic device 1000. It may include more or fewer components than shown, or combine certain components, or different components. For example, the electronic device 1000 may also include input / output devices, network access devices, buses, etc.

[0147] Processor 300 can be a Central Processing Unit (CPU), or other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. The general-purpose processor can be a microprocessor, a single-chip microcomputer, or any conventional processor.

[0148] The memory 200 can be used to store computer programs 400 and / or modules / units. The processor 300 implements various functions of the electronic device 1000 by running or executing the computer programs and / or modules / units stored in the memory 200 and by calling data stored in the memory 200. The memory 200 may mainly include a program storage area and a data storage area. The program storage area may store the operating system, application programs required for at least one function (such as sound playback function, image playback function, etc.), etc.; the data storage area may store data created according to the use of the electronic device 1000 (such as audio data), etc. In addition, the memory 200 may include high-speed random access memory, and may also include non-volatile memory, such as hard disk, RAM, plug-in hard disk, smart media card (SMC), secure digital (SD) card, flash card, at least one disk storage device, flash memory device, or other non-volatile solid-state storage device.

[0149] If the modules / units integrated in the electronic device 1000 are implemented as software functional units and sold or used as independent products, they can be stored in a storage medium. Based on this understanding, all or part of the processes in the methods of the above embodiments of this application can also be implemented by a computer program instructing related hardware, and the computer program can be stored in a storage medium. When executed by a processor, the computer program can implement the steps of the various method embodiments described above. The computer program includes computer program code, which can be in the form of source code, object code, executable files, or certain intermediate forms. The storage medium can include: any entity or device capable of carrying computer program code, recording media, USB flash drives, portable hard drives, magnetic disks, optical disks, computer memory, read-only memory (ROM), random access memory (RAM), electrical carrier signals, telecommunication signals, and software distribution media, etc. It should be noted that the content included in the storage medium can be appropriately added or subtracted according to the requirements of patent practice. For example, according to patent practice, the storage medium does not include electrical carrier signals and telecommunication signals.

[0150] The above embodiments are only used to illustrate the technical solutions of the present invention, and are not intended to limit it. Although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features. These modifications or substitutions do not cause the essence of the corresponding technical solutions to depart from the spirit and scope of the technical solutions of the embodiments of the present invention.

[0151] This invention is not limited to the optional embodiments described above, and anyone can derive other various forms of products based on the inspiration of this invention. The specific embodiments described above should not be construed as limiting the scope of protection of this invention; the scope of protection of this invention should be determined by the claims, and the specification can be used to interpret the claims.

Claims

1. A method for generating a Q-version digital avatar, characterized in that, include: The image generation model, real user portrait images, and Q-version template images are obtained. The image generation model integrates multiple style fine-tuning models, which is achieved by linearly superimposing multiple low-rank weight matrices of LoRA models corresponding to different styles with set style weight coefficients. Based on the Q-version template image, generate structural control signals that characterize its composition and posture; The system extracts identity features from the user's real portrait image and generates identity conditions to obtain global identity features and local semantic enhancement features, and obtains the text prompt information input by the user. The text prompt information, structural control signals, global identity features, and local semantic enhancement features are input into the image generation model; wherein, the image generation model integrates multiple style fine-tuning models to collaboratively control the style, identity fidelity, and composition of the generated image; The image generation model processes and outputs an RGB portrait image; the image generation model processing includes: High-frequency detail enhancement processing: High-frequency enhancement operators are applied to the skip connection feature maps in the image generation model by applying high-frequency enhancement operators to the skip connection paths of the U-Net network to enhance the high-frequency components in the feature maps; Batch style alignment processing: When generating multiple Q-version digital avatars in batches, constrain and align the distribution of intermediate features in the same batch generation process. The high-frequency enhancement operator is a multi-scale adaptive enhancement operator, and its processing includes: High-pass filtering is applied to the input skip connection feature map at at least two different scales to obtain the high-frequency components at each scale; Global statistical features of the skip connection feature map are extracted and combined with the current generation time step information and style weight coefficients, and then input into a multilayer perceptron to dynamically predict the enhancement weights corresponding to each scale. The high-frequency components at each scale are multiplied by their corresponding enhancement weights and summed to obtain the total high-frequency enhancement amount. This sum is then added to the input skip connection feature map to obtain the enhanced feature map. The background of the RGB character image is removed to generate a Q-version digital avatar with an alpha channel.

2. The method for generating Q-version digital avatars according to claim 1, characterized in that, The style fine-tuning model includes at least two of the following: cartoon style, sticker style, character facial expression, or age representation.

3. The method for generating Q-version digital avatars according to claim 1, characterized in that, Extracting identity features from the user's real portrait image and generating identity conditions includes: The user's real portrait image is encoded into a global visual embedding vector using a first identity encoder; By combining the second identity encoder with facial key point information, local feature enhancement encoding is performed on the eye, mouth or face shape regions to obtain local semantic enhancement features; Based on the global visual embedding vector and local semantic enhancement features, identity conditions are generated to guide the generation process.

4. The method for generating a Q-version digital avatar according to claim 3, characterized in that, The identity conditions used to guide the generation process include: Calculate the mutual attention score matrix between the global visual embedding vector and the local semantic enhancement feature; generate an adaptive fusion weight matrix based on the mutual attention score matrix; apply channel weighting to the local semantic enhancement feature using the adaptive fusion weight matrix; concatenate the weighted local semantic enhancement feature with the global visual embedding vector to generate the final identity condition vector.

5. The method for generating a Q-version digital avatar according to claim 1, characterized in that, The text prompt information is formed by concatenating the prompt words entered by the user with the preset Q-version style template words, and is converted into a semantic vector by a text encoder; the structure control signal generated based on the Q-version template image is achieved by extracting the edge contour of the Q-version template image through the edge detection module and generating a binarized edge map.

6. The method for generating a Q-version digital avatar according to claim 1, characterized in that, The batch style alignment process specifically includes: In at least one intermediate layer of the diffusion model denoising process, feature maps of all samples in the same batch are obtained; Calculate the mean μ and standard deviation σ of the feature map in the channel dimension within the batch; Construct a learnable mask vector M with the same number of channels as the feature map, where the value of M ranges from [0, 1]. For each sample in the batch, calculate the aligned feature map Falign based on its original feature map F, using the formula: Falign= M⊙[(F -μ) / σ] + (1 - M)⊙F Where ⊙ represents channel-by-channel multiplication, and F is the original feature map of each sample.

7. A system for generating Q-version digital avatars, characterized in that, include: The data acquisition module is used to acquire pre-trained image generation models, real user portrait images and Q-version template images, as well as text prompts from users; The image generation model integrates multiple style fine-tuning models, which is achieved by linearly superimposing multiple low-rank weight matrices of LoRA models corresponding to different styles with set style weight coefficients. The structure control signal generation module is used to generate structure control signals that characterize the composition and posture of the Q-version template image based on the Q-version template image. The feature extraction module is used to extract identity features from the user's real portrait image to obtain global identity features and local semantic enhancement features; The collaborative generation module integrates the image generation model and multiple style fine-tuning models. It receives the text prompts, structural control signals, global identity features, and local semantic enhancement features, and collaboratively controls the generation process to output an RGB portrait image. The image generation model's processing includes: High-frequency detail enhancement processing: High-frequency enhancement operators are applied to the skip connection feature maps in the image generation model by applying high-frequency enhancement operators to the skip connection paths of the U-Net network to enhance the high-frequency components in the feature maps; Batch style alignment processing: When generating multiple Q-version digital avatars in batches, constrain and align the distribution of intermediate features in the same batch generation process. The high-frequency enhancement operator is a multi-scale adaptive enhancement operator, and its processing includes: High-pass filtering is applied to the input skip connection feature map at at least two different scales to obtain the high-frequency components at each scale; Global statistical features of the skip connection feature map are extracted and combined with the current generation time step information and style weight coefficients, and then input into a multilayer perceptron to dynamically predict the enhancement weights corresponding to each scale. The high-frequency components at each scale are multiplied by their corresponding enhancement weights and summed to obtain the total high-frequency enhancement amount. This sum is then added to the input skip connection feature map to obtain the enhanced feature map. The post-processing module is used to remove the background from the RGB character image and generate a Q-version digital avatar with an alpha channel.

8. A computer storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by a processor, it performs the steps of the method for generating Q-version digital avatars as described in any one of claims 1-6.

Images