Image generation method and terminal based on joint control of single image geometry and texture

Abstract

The present invention provides an image generation method and terminal for joint control of single-image geometric texture, which receive an original image and text prompt information input by a user, and generate a face sub-image, a key point image and a frontalized face image based on the original image; extract and process features of the face sub-image to generate a face identity latent vector; generate a latent space control vector based on the key point image and the frontalized face image; generate text embedding features based on the text prompt information; inject the face identity latent vector, the text embedding features and the latent space control vector after splicing the noise vector into a diffusion transformer model by using a layered selective feature injection method, and generate an identity-preserving image in combination with a decoder; the diffusion transformer model is trained using a low-rank adaptive training strategy; and single-image driven, controllable generation of identity-preserving images in multiple posture styles is achieved.

Description

Technical Field

[0001] The present invention relates to the technical field of model-based image generation, and in particular to an image generation method and a terminal for joint control of single-image geometric texture. Background Art

[0002] Identity-preserving image generation techniques have received widespread attention in the field of computer vision, especially identity-preserving methods based on diffusion models. These methods can be divided into two main categories: fine-tuning-based methods and methods that do not require fine-tuning.

[0003] (1) Fine-tuning-based methods

[0004] There are three main approaches to fine-tuning. One is a solution similar to Textual Inversion, which requires optimizing a new word embedding for each identity (input portrait) so that the generative model can learn specific portrait features. This solution requires training a word embedding for each portrait. The second mainstream solution is a solution such as DreamBooth, which fine-tunes the entire generator to improve identity fidelity. The third solution is to fine-tune LoRa by collecting images of the same person from different angles and scenes, saving a specific LoRa for each person, and combining this LoRa during inference to generate an image with the input person's features.

[0005] (2) Methods that do not require fine-tuning

[0006] In order to reduce the resource requirements of fine-tuning-based methods, many methods that do not require fine-tuning have emerged, such as the IP-Adapter series: CLIP or face recognition models are used to extract facial features, and the features are embedded into the generative network through cross-attention, so that the generative model can generate images with specific identities. Later, in order to further improve identity consistency, PuLID introduced semantic alignment and layout alignment losses based on the principle of progressive distillation to further improve identity consistency. In addition, in order to be able to control the position of the face, InstantID also introduced an additional key point control network, so that the generated face can be generated with reference to the input face position, thereby improving the controllability of the generated face. In addition to the solution of using a single picture for one-step generation, there is also a method of inputting multiple pictures. Figure 1 The similarity of generated faces can be improved by using a step-by-step generation scheme, such as PhotoMaker, which embeds multiple facial images into the Stable Diffusion model so that the Stable Diffusion model can refer to the identity information of multiple images when generating images.

[0007] However: (1) Existing fine-tuning-based methods have high training costs and time overhead, and require multiple pictures of the same person from different angles and in different scenes, which limits their application in practical scenarios.

[0008] (2) Most of the existing methods that do not require fine-tuning are based on the SD-XL model, and the generated portraits have uncoordinated body and fingers, or even lack body parts.

[0009] (3) Existing portrait generation methods that do not require fine-tuning and have control networks require the introduction of additional control networks, which significantly increases the training and inference time.

[0010] (4) Existing portrait generation methods that do not require fine-tuning and have control networks ignore the shape and texture priors of the original input face, resulting in insufficient similarity. Summary of the Invention

[0011] The technical problem to be solved by the present invention is to provide an image generation method and terminal with joint control of single-image geometric texture, so as to realize the controllable generation of identity-preserving images driven by a single image and in multiple pose styles.

[0012] In order to solve the above technical problems, the technical solution adopted by the present invention is:

[0013] A method for generating an image by joint control of single-image geometric texture, comprising the steps of:

[0014] S1. Receive an original image and text prompt information input by a user, and generate a face sub-image, a key point image, and a frontalized face image based on the original image;

[0015] S2. Extract and process features of the face sub-image to generate a face identity latent vector; generate a latent space control vector based on the key point map and the frontalized face image; and generate a text embedding feature based on the text prompt information;

[0016] S3, injecting the face identity latent vector, the text embedding feature, and the latent space control vector after the noise vector is spliced into the diffusion transformer model using a layered selective feature injection method, and generating an identity-preserving image in combination with a decoder;

[0017] The diffusion transformer model is trained using a low-rank adaptive training strategy.

[0018] In order to solve the above technical problems, another technical solution adopted by the present invention is:

[0019] A single-image geometric texture jointly controlled image generation terminal includes a processor, a memory, and a computer program stored in the memory and executable on the processor. When the processor executes the computer program, it implements any of the steps in the above-mentioned single-image geometric texture jointly controlled image generation method.

[0020] The beneficial effects of the present invention include: a single-image, geometric-texture-jointly controlled image generation method and terminal. By extracting facial identity latent vectors (identity information), frontalization face control latent vectors (texture and shape information), and key point control latent vectors (geometric features), combined with text prompts, this method achieves precise control of "creating images from images." A layered selective injection mechanism ensures the decoupling of identity features (injected into even layers) from style / pose features (injected into all layers), while a low-rank adaptive training strategy significantly reduces computational costs, enabling high-quality generation driven by a single image. This overcomes the limitations of traditional methods that require multiple images or additional control models. BRIEF DESCRIPTION OF THE DRAWINGS

[0021] Figure 1 This is a simplified flowchart of an image generation method for joint control of single-image geometric textures according to an embodiment of the present invention;

[0022] Figure 2 This is an example diagram of a scheme overview of an image generation method for joint control of single-image geometric texture according to an embodiment of the present invention;

[0023] Figure 3 Schematic diagram of the cross-joint attention principle of an image generation method for joint control of single-image geometric texture according to an embodiment of the present invention;

[0024] Figure 4 This is a structural example diagram of a conventional MM-DiTBlock of an image generation method for joint control of single-image geometry and texture according to an embodiment of the present invention;

[0025] Figure 5 This is a structural example diagram of the MM-DiT Block with CAJ in an image generation method for joint control of single-image geometry and texture according to an embodiment of the present invention;

[0026] Figure 6 This is a structural example diagram of a Single-DiT module of an image generation method for joint control of single-image geometry and texture according to an embodiment of the present invention;

[0027] Figure 7 This is a structural example diagram of an image generation terminal for joint control of single-image geometric texture according to an embodiment of the present invention;

[0028] Description of labels:

[0029] 1. An image generation terminal with joint control of single-image geometry and texture; 2. A processor; 3. A memory. DETAILED DESCRIPTION

[0030] To illustrate the technical content, achieved objectives and effects of the present invention in detail, the following description is given in conjunction with the embodiments and accompanying drawings.

[0031] Please refer to Figure 1 , an image generation method for joint control of single image geometry and texture, comprising the steps of:

[0032] S1. Receive an original image and text prompt information input by a user, and generate a face sub-image, a key point image, and a frontalized face image based on the original image;

[0033] S2. Extract and process features of the face sub-image to generate a face identity latent vector; generate a latent space control vector based on the key point map and the frontalized face image; and generate a text embedding feature based on the text prompt information;

[0034] S3, injecting the face identity latent vector, the text embedding feature, and the latent space control vector after the noise vector is spliced into the diffusion transformer model using a layered selective feature injection method, and generating an identity-preserving image in combination with a decoder;

[0035] The diffusion transformer model is trained using a low-rank adaptive training strategy.

[0036] As can be seen from the foregoing description, the beneficial effects of the present invention lie in: a single-image, geometric-texture-jointly controlled image generation method and terminal achieve precise control of "image-to-image" by extracting facial identity latent vectors (identity information), frontalization face control latent vectors (texture and shape information), and keypoint control latent vectors (geometric features), combined with text prompts. A layered selective injection mechanism ensures the decoupling of identity features (injected into even layers) from style / pose features (injected into all layers), while a low-rank adaptive training strategy significantly reduces computational costs, enabling high-quality generation driven by a single image. This overcomes the limitations of traditional methods that require multiple images or additional control models.

[0037] Furthermore, step S1 includes the steps of:

[0038] S11, receiving the original image and text prompt information input by the user;

[0039] S12, obtaining a face region bounding box from the original image through a face recognition algorithm, and cropping a face sub-image;

[0040] S13, performing facial key point detection on the face sub-image, and generating a key point map according to the detection results;

[0041] S14: performing background removal on the face sub-image and performing posture correction processing, using a spatial transformation algorithm to align the face orientation and optimize posture parameters to generate a frontal face image.

[0042] As can be seen from the preceding description, the original image processing pipeline is refined. Through face detection (S12), keypoint extraction (S13), and pose correction (S14), a complete geometric prior system is constructed. Accurate face cropping avoids background interference, the keypoint map provides structural constraints for subsequent geometric control, and frontalization normalizes the pose baseline, laying the foundation for stable extraction of facial identity features and improving the structural rationality and identity consistency of the generated image.

[0043] Furthermore, step S14 includes the steps of:

[0044] S141, segmenting the face sub-image using the BiSeNet segmentation model and removing the background;

[0045] S142, using the implicit key point method in Live Portrait to calculate the pitch angle information of the face sub-image and the standard frontal face;

[0046] S143, inputting the pitch angle information and the face sub-image into the Stitching module in Live Portrait to obtain an intermediate face at the target position;

[0047] S144. Input the intermediate face into the Warpping module and decoder in Live Portrait to generate a frontal face image.

[0048] As can be seen from the preceding description, the frontalization process has been further optimized. Background noise is removed through BiSeNet segmentation (S141), and pose alignment is achieved using Live Portrait's implicit keypoints (S142) and Stitching / Warping modules (S143-S144). Compared to traditional 3DMM methods, this solution eliminates the need for explicit 3D modeling and efficiently generates frontalization results through 2D image transformations. This method preserves texture details while correcting pose, providing standardized input for subsequent feature extraction.

[0049] Furthermore, the generation of the face identity latent vector includes the steps of:

[0050] Input the face sub-image into the face feature extractor and extract the face features using the ElasticFace algorithm;

[0051] The extracted facial features are processed through the MLP layer and the LayerNorm layer to generate the facial identity latent vector.

[0052] As can be seen from the preceding description, the ElasticFace algorithm is used to extract facial features. Its dynamic corner boundary mechanism effectively solves the problem of overfitting simple samples and underfitting difficult samples in traditional metric learning, improving the robustness of identity features. Multi-layer perception (MLP) and layer normalization (LayerNorm) processing further enhance feature expression capabilities, ensuring that the generated images maintain identity consistency despite changes in posture and expression.

[0053] Furthermore, the generation of the latent space control vector includes the steps of:

[0054] The key point map and the frontalized face image are encoded through a VAE encoding model, and the encoded vectors are spliced to obtain the latent space control vector.

[0055] As can be seen from the above description, the VAE encoding fuses the keypoint map (geometry) with the frontalized face (texture and shape), generating a latent space control vector that decouples geometry and texture. The concatenation operation enables these two features to work synergistically in the latent space, preserving facial structural constraints (such as the location of facial features) while conveying details such as facial shape and skin texture. This provides comprehensive and controllable conditional signals for subsequent diffusion generation, enhancing the realism and diversity of the generated images.

[0056] Furthermore, the diffusion transformer model consists of 19 MM-DiT blocks and 38 Single-DiT blocks, and the MM-DiT blocks at even positions are injected with a cross-joint attention mechanism;

[0057] Step S3 includes the steps of:

[0058] The latent space control vector after the noise vector is spliced is input into the diffusion transformer model, the face identity latent vector is injected into the MM-DiT blocks at even positions in the diffusion transformer model, and the text embedding feature is injected into all MM-DiT blocks in the diffusion transformer model to obtain a model output;

[0059] The model output is input into the VAE decoder to obtain the final identity-preserving image containing the characteristics of the person in the original image.

[0060] As can be seen from the preceding description, a specific network architecture (19 MM-DiTs + 38 Single-DiTs) and a layered injection strategy were designed. The cross-joint attention mechanism (injecting identity latent vectors) in even-numbered MM-DiT blocks ensures strong identity constraints, while the full-layer MM-DiT injection of text embeddings and keypoint latent vectors enables flexible control of style and pose. This structure balances identity preservation with generative diversity.

[0061] Furthermore, the loss function of the diffusion transformer model adopts MSE Loss, which is expressed as follows:

[0062] ;

[0063] ;

[0064] in, x 1 is the latent vector after the original image is encoded by VAE, x 0 means noise, t Indicates time, c n Representation conditions include text prompt information, key point map, and frontal face image.

[0065] As can be seen from the preceding description, using MSE loss as the loss function for the diffusion model directly optimizes noise prediction, enabling the model to learn the gradient flow of the data distribution. Combined with conditional inputs (text, keypoints, and frontalized images), this loss function guides the model to precisely adhere to geometric and texture constraints during generation, improving image quality and controllability.

[0066] Furthermore, the diffusion transformer model is trained using a low-rank adaptive training strategy, including:

[0067] The main body of the diffusion transformer model is fixed, and only the cross-joint attention mechanism and the low-rank decomposition matrix of the attention weight are trained, combined with the flow matching algorithm to accelerate convergence.

[0068] As can be seen from the preceding description, the low-rank adaptation strategy freezes the main DiT parameters and only trains the low-rank decomposition matrix of the cross-joint attention and attention weights, reducing the number of parameters and memory usage. Combined with the stream matching algorithm, it improves training speed and supports deployment on consumer-grade GPUs. It also avoids catastrophic forgetting caused by full parameter fine-tuning and preserves the visual understanding capabilities of the pre-trained model.

[0069] Furthermore, the noise vector is obtained by random sampling from a Gaussian distribution and has the same shape and size as the latent space control vector.

[0070] As can be seen from the above description, the noise vector is randomly sampled from a Gaussian distribution and concatenated with the latent space control vector as input to the diffusion model, introducing randomness into the generation process. The design of identical shape and size ensures that the noise matches the dimension of the conditional signal. This enables the model to generate diverse texture details (such as expressions and lighting changes) while maintaining identity and geometric constraints, enhancing the richness and realism of the generated results.

[0071] The present invention provides an image generation method and terminal for joint control of single-image geometric texture, which are suitable for identity-preserving image generation based on single-image input.

[0072] Please refer to Figures 1 to 6 , embodiment 1 of the present invention is:

[0073] A method for generating an image by joint control of single-image geometric texture, comprising the steps of:

[0074] S1. Receive an original image and text prompt information input by a user, and generate a face sub-image, a key point image, and a frontalized face image based on the original image;

[0075] Step S1 includes the steps of:

[0076] S11, receiving the original image and text prompt information input by the user;

[0077] S12: Obtain a face region bounding box from the original image through a face recognition algorithm, and crop a face sub-image.

[0078] In this embodiment, the input original image y is input into the face detection module to obtain the face area boundary box;

[0079] Crop the face sub-image f based on the bounding box and the original image y c , sent to subsequent modules.

[0080] S13: Perform facial key point detection on the face sub-image, and generate a key point map according to the detection results.

[0081] In this embodiment, the face detection result is further subjected to face key point detection; the five points in the key point detection result (left and right eyes, nose, left and right corners of mouth) are taken as the center, and lines connecting the nose and the other four points are drawn to obtain the key point map k c .

[0082] S14: performing background removal on the face sub-image and performing posture correction processing, using a spatial transformation algorithm to align the face orientation and optimize posture parameters to generate a frontal face image.

[0083] Step S14 includes the steps of:

[0084] S141, segmenting the face sub-image using the BiSeNet segmentation model and removing the background;

[0085] S142, using the implicit key point method in Live Portrait to calculate the pitch angle information of the face sub-image and the standard frontal face;

[0086] S143, inputting the pitch angle information and the face sub-image into the Stitching module in Live Portrait to obtain an intermediate face at the target position;

[0087] S144. Input the intermediate face into the warping module and decoder in Live Portrait to generate a frontal face image.

[0088] In this embodiment, the face sub-image y c Perform face segmentation through BiSeNet, remove the background, and replace it with a white background;

[0089] Calculate the face subgraph y c Compared with the pitch, yaw, and roll angles of the standard frontal face, in particular, the implicit key point method in Live Portrait is used here;

[0090] The pitch, yaw, and roll information and the segmented face are then fed into the stitching module in Live Portrait to obtain the intermediate face at the target location.

[0091] The intermediate face is further input into the Warpping module and decoder, and finally the frontal face image f is obtained. w ;

[0092] The formula for the entire positive process is as follows:

[0093] ;

[0094] ;

[0095] ;

[0096] ;

[0097] ;

[0098] in, represents the cropped face sub-image, represents the BiSeNet segmentation model, M Represents the segmentation mask, ⊙ represents the Hadamard product, 1 represents the shape and A consistent vector of all 1s corresponds to white, Represents the segmented graph. represents the pitch angle estimation network, Represents the estimated pitch angle parameters, Stitching, Warping, They represent the Stitching module, Warping module and decoder processing of Live Portrait respectively.

[0099] S2. Extract and process features of the face sub-image to generate a face identity latent vector; generate a latent space control vector based on the key point map and the frontalized face image; and generate a text embedding feature based on the text prompt information;

[0100] The generation of the face identity latent vector comprises the following steps:

[0101] Input the face sub-image into the face feature extractor and extract the face features using the ElasticFace algorithm;

[0102] In this embodiment, the face sub-image y c Input into the face feature extractor to extract facial features, in particular, ElasticFace is used here to extract facial features e c ; It can be expressed by the formula as follows:

[0103] .

[0104] The extracted facial features are processed through the MLP layer and the LayerNorm layer to generate the facial identity latent vector.

[0105] In this embodiment, the extracted facial features e c After the MLP layer and LayerNorm layer, the face identity latent vector is obtained , the calculation process can be described by the formula as follows:

[0106] .

[0107] The generation of the latent space control vector includes the following steps:

[0108] The key point map and the frontalized face image are encoded through a VAE encoding model, and the encoded vectors are spliced to obtain the latent space control vector.

[0109] In this embodiment, the frontal face image f w and keypoint graph k c Encode them respectively through the VAE encoding model, and concatenate the encoded vectors to obtain the latent space control vector c N , the process can be described by the formula as follows:

[0110] ;

[0111] ;

[0112] .

[0113] S3, injecting the face identity latent vector, the text embedding feature, and the latent space control vector after the noise vector is spliced into the diffusion transformer model using a layered selective feature injection method, and generating an identity-preserving image in combination with a decoder;

[0114] The diffusion transformer model is trained using a low-rank adaptive training strategy.

[0115] The diffusion transformer model consists of 19 MM-DiT blocks and 38 Single-DiT blocks, and the MM-DiT blocks at even positions are injected with a cross-joint attention mechanism;

[0116] Step S3 includes the steps of:

[0117] The latent space control vector after the noise vector is spliced is input into the diffusion transformer model, the face identity latent vector is injected into the MM-DiT blocks at even positions in the diffusion transformer model, and the text embedding feature is injected into all MM-DiT blocks in the diffusion transformer model to obtain a model output;

[0118] The model output is input into the VAE decoder to obtain the final identity-preserving image containing the characteristics of the person in the original image.

[0119] In this embodiment, refer to Figure 2 , the latent space control vector c N Spliced with the noise vector to obtain the final vector z input to the DiT model N , the process can be described by the formula as follows, where the noise vector is randomly sampled from a Gaussian distribution and has the same shape and size as the control vector:

[0120] .

[0121] At the same time, in this embodiment, the face identity latent vector after embedding the projection layer is , injected into the DIT module through Cross Joint Attention (CJA), where the composition of CJA is as follows Figure 3 As shown, input three vectors to control the latent space vector z N , text latent vector e t and face identity latent vector Among them, the text latent vector is the vector obtained after the input text prompt information is passed through the text encoder of Flux. Face identity latent vector The face feature extractor extracts the face features through the face sub-graph c , and obtained through the MLP layer and LayerNorm layer, and after CJA, the image latent vector embedded with identity information is obtained .

[0122] Among them, you can refer to Figure 4 and Figure 5 , CJA only injects the identity latent vector when the multimodal DiT block (MM-DiT Block) is even (i,i+2,...i+2N), but the text latent vector is injected in all MM-DiT modules.

[0123] The output z0 of the DiT model is fed into the VAE decoder to obtain the final image containing the input person’s features. The DiT model is composed of 19 MM-DiT blocks and 38 Single-DiT blocks. The ordinary MM-DiT can be composed of Figure 4 As described, the MM-DiT module with CAJ attention mechanism consists of Figure 5 As described, the Single-DiT module consists of Figure 6 described.

[0124] The VAE uses the official open source VAE model of Flux.

[0125] The second embodiment of the present invention is:

[0126] A method for generating an image by jointly controlling geometric texture of a single image is different from the first embodiment in that the present embodiment describes the training of a diffusion transformer model.

[0127] The diffusion transformer model is trained using a low-rank adaptive training strategy including:

[0128] The main body of the diffusion transformer model is fixed, and only the cross-joint attention mechanism and the low-rank decomposition matrix of the attention weight are trained, combined with the flow matching algorithm to accelerate convergence.

[0129] In this embodiment, during the training process of the entire model, the DiT model is fixed, and only the parameters of the added LoRA layer and the parameters of the embedded projection layer and CAJ are trained. The parameters that need to be trained for the LoRA layer are as follows:

[0130]

[0131] The loss function of the diffusion transformer model adopts MSE Loss, which is expressed as follows:

[0132] ;

[0133] .

[0134] in, x 1 is the latent vector after the original image is encoded by VAE,x 0 means noise, t Indicates time, c n Representation conditions include text prompt information, key point map, and frontal face image.

[0135] Please refer to Figure 7 , the third embodiment of the present invention is:

[0136] An image generation terminal 1 for joint control of single-image geometric texture includes a processor 2, a memory 3, and a computer program stored in the memory 3 and executable on the processor 2. When the processor 2 executes the computer program, the steps of the image generation method for joint control of single-image geometric texture described in the first or second embodiment are implemented.

[0137] In summary, the present invention's single-image, geometric-texture joint control image generation method and terminal achieve precise control of "image generation from image" by extracting facial identity latent vectors (identity information), frontalization face control latent vectors (texture and shape information), and keypoint control latent vectors (geometric features), combined with textual prompts. A layered selective injection mechanism ensures the decoupling of identity features (injected into even layers) from style / pose features (injected into all layers), while a low-rank adaptive training strategy significantly reduces computational costs, enabling high-quality generation driven by a single image. This overcomes the limitations of traditional methods that require multiple images or additional control models.

[0138] The present invention can bring the following advantages:

[0139] Advantage 1: Identity-preserving image generation is achieved with only a single reference image. There is no need to collect multiple images of the same person in different scenes or angles. There is no need for additional training or storage to store each character's LoRA or DreamBooth model. Single inference time is reduced to seconds.

[0140] Advantage 2: By combining the frontalized face and key points as control conditions and inputting them into the Flux DiT model, model generation is precise and controllable. The model can also reference the texture of the frontalized face, improving the consistency of the generated image with the input image in terms of high-level semantics and low-level textures.

[0141] Advantage 3: During the Flux control network training process, no additional control network is introduced. Instead, the control conditions are directly input into DiT through LoRA injection, significantly reducing the training cost and improving training efficiency.

[0142] The solution of the present invention can solve the shortcomings of the prior art in the background technology of this application because:

[0143] 1. A universal hierarchical identity feature extraction network uses pre-training to learn common representation patterns across individuals, directly overcoming the limitations of traditional methods that rely on multi-angle data. The pre-trained network can extract the essence of identity features from a single reference image, eliminating the need to fine-tune model parameters for specific individuals and fundamentally eliminating the need for multiple data collection and model storage.

[0144] 2. The joint control condition injection strategy encodes the frontal face and key points as control signals for Flux DiT, enabling the model to simultaneously constrain both high-level semantics (such as facial feature location) and low-level textures (such as skin texture) during the generation process. This dual guidance mechanism directly affects the generation path, avoiding the semantic-texture mismatch problem caused by single-condition control in traditional methods.

[0145] 3. LoRA's lightweight control architecture directly embeds control conditions into the DiT's attention layer rather than an external control network, enabling model control capabilities through efficient parameter adaptation. Experiments have shown that this design reduces the number of training parameters by 99%, and eliminates the need to load additional network modules during inference, creating a closed-loop system with low memory consumption and high efficiency.

[0146] The above descriptions are merely embodiments of the present invention and are not intended to limit the patent scope of the present invention. Any equivalent transformations made using the contents of the present invention's description and drawings, or directly or indirectly applied in related technical fields, are also included in the patent protection scope of the present invention.

Claims

1. A method for image generation by joint control of single image geometry and texture, characterized in that: Including steps: S1. Receive an original image and text prompt information input by a user, and generate a face sub-image, a key point image, and a frontalized face image based on the original image; Step S1 includes the steps of: S11, receiving the original image and text prompt information input by the user; S12, obtaining a face region bounding box from the original image through a face recognition algorithm, and cropping a face sub-image; S13, performing facial key point detection on the face sub-image, and generating a key point map according to the detection results; S14, removing the background of the face sub-image and performing posture correction processing, using a spatial transformation algorithm to align the face orientation and optimize the posture parameters to generate a frontal face image; Step S14 includes the steps of: S141, segmenting the face sub-image using the BiSeNet segmentation model and removing the background; S142, using the implicit key point method in Live Portrait to calculate the pitch angle information of the face sub-image and the standard frontal face; S143, inputting the pitch angle information and the face sub-image into the Stitching module in Live Portrait to obtain an intermediate face at the target position; S144, inputting the intermediate face into the Warpping module and decoder in Live Portrait to generate a frontal face image; S2. Extract and process features of the face sub-image to generate a face identity latent vector; generate a latent space control vector based on the key point map and the frontalized face image; and generate a text embedding feature based on the text prompt information; The generation of the face identity latent vector comprises the following steps: Input the face sub-image into the face feature extractor and extract the face features using the ElasticFace algorithm; Processing the extracted facial features through the MLP layer and the LayerNorm layer to generate the facial identity latent vector; The generation of the latent space control vector includes the following steps: Encoding the key point map and the frontalized face image through a VAE encoding model, and concatenating the encoded vectors to obtain the latent space control vector; S3, injecting the face identity latent vector, the text embedding feature, and the latent space control vector after the noise vector is spliced into the diffusion transformer model using a layered selective feature injection method, and generating an identity-preserving image in combination with a decoder; The diffusion transformer model is trained using a low-rank adaptive training strategy.

2. The image generation method of single-image geometric texture joint control according to claim 1, characterized in that: The diffusion transformer model consists of 19 MM-DiT blocks and 38 Single-DiT blocks, and the MM-DiT blocks at even positions are injected with a cross-joint attention mechanism; Step S3 includes the steps of: The latent space control vector after the noise vector is spliced is input into the diffusion transformer model, the face identity latent vector is injected into the MM-DiT blocks at even positions in the diffusion transformer model, and the text embedding feature is injected into all MM-DiT blocks in the diffusion transformer model to obtain a model output; The model output is input into the VAE decoder to obtain the final identity-preserving image containing the characteristics of the person in the original image.

3. The image generation method of single-image geometric texture joint control according to claim 2, characterized in that: The loss function of the diffusion transformer model adopts MSE Loss, which is expressed as follows: ； ； in, x 1 is the latent vector after the original image is encoded by VAE, x 0 means noise, t Indicates time, c n Representation conditions include text prompt information, key point map, and frontal face image.

4. The image generation method of single-image geometric texture joint control according to claim 2, characterized in that: The diffusion transformer model is trained using a low-rank adaptive training strategy including: The main body of the diffusion transformer model is fixed, and only the cross-joint attention mechanism and the low-rank decomposition matrix of the attention weight are trained, combined with the flow matching algorithm to accelerate convergence.

5. The image generation method of single image geometric texture joint control according to claim 2, characterized in that: The noise vector is obtained by random sampling from a Gaussian distribution and has the same shape and size as the latent space control vector.

6. An image generation terminal for joint control of single-image geometry and texture, comprising a processor, a memory, and a computer program stored in the memory and capable of running on the processor, characterized in that: When the processor executes the computer program, the steps of the image generation method for joint control of single-image geometric texture are implemented as described in any one of claims 1-5.

Images