A method, system, device and storage medium for generating facial images

Abstract

The present invention relates to the field of image generation technology, and specifically to a facial image generation method, system, device, and medium. To solve the technical problem of low facial generation quality in the prior art, the present invention separately trains an image perception model containing an encoder and a decoder that can obtain key information and reduce data redundancy, and a diffusion model containing noise addition processing, semantic feature extraction processing, and denoising processing that can accurately extract and restore semantic features of facial images in a noisy environment. The model is then combined with a trained text-based graph model and subjected to model weight update processing to solve the overfitting problem, allowing the model to pay more attention to higher-level facial features during the face generation process rather than excessively focusing on pixel features, thereby making the model natural when generating facial images, thereby obtaining a facial image generation model. The target requirement text is processed by the facial image generation model to obtain a target facial image.

Description

Technical Field

[0001] The present invention relates to the field of image generation technology, and in particular to a method, system, device, and medium for generating a facial image. Background Art

[0002] Text-to-image models are an emerging AI technology designed to generate images based on text descriptions. They combine advanced methods from natural language processing and computer vision to enable machines to understand textual content and generate corresponding visual representations. These models typically utilize large neural networks, particularly generative adversarial networks or variational autoencoders, trained on a large number of text-image pairs. Generating facial images from text has important applications in dataset generation, avatar creation, and facial image editing.

[0003] Modern facial image models can generate impressive images, but there may still be flaws in some complex scenes or details. The generated facial images are blurry, or the generation process overemphasizes pixel information, resulting in unnatural facial images or containing visual artifacts, resulting in poor quality of facial image generation. Summary of the Invention

[0004] The purpose of the present invention is to provide a method, system, device and medium for generating natural and high-quality facial images.

[0005] The technical solutions of the present invention are as follows:

[0006] A method for generating a face image, comprising the following operations:

[0007] S1. In the training document graph model, obtain the image corresponding to the maximum cosine similarity of the feature vector corresponding to the target requirement text as the basic requirement image;

[0008] S2. The basic demand image is processed by the encoder in the trained image perception model to obtain a discretized basic image; the discretized basic image is sequentially processed by several times of noise addition and semantic feature extraction in the trained diffusion model to obtain a basic semantic feature image;

[0009] S3. After the basic semantic feature image and the target demand text are cross-fused in the training text-graph model, they are sequentially processed by denoising in the training diffusion model and decoder in the image perception model to obtain the first generated image; after the first generated image and the target demand text are cross-fused in the training text-graph model, they are sequentially processed by denoising in the training diffusion model and decoder in the image perception model to obtain the second generated image; after the second generated image and the target demand text are cross-fused in the training text-graph model, they are sequentially processed by denoising in the training diffusion model and decoder in the image perception model to obtain the third generated image; and so on, until the objective function value is less than the target threshold, the Nth generated image is output as the target face image.

[0010] The method for obtaining a trained image perception model is as follows: constructing an image perception model containing an encoder and a decoder, and obtaining the trained image perception model after training with a first image data set; the encoder is used to downsample and discretize the input image; and the decoder is used to generate an image based on a feature vector.

[0011] The method for obtaining the training diffusion model is as follows: constructing a diffusion model including noise addition processing, semantic feature extraction processing and denoising processing, and obtaining the training diffusion model after training with the second image data set.

[0012] The method for obtaining the training text-image model is as follows: the text-image model is trained on a text-image pair dataset and zero-sample trained to obtain the training text-image model.

[0013] The operations of S1, S2 and S3 are implemented based on the face image generation model. Before executing the operations of S1, S2 and S3, the face image generation model also includes updating the model weights of the face image generation model based on the third image data set.

[0014] The specific operations of the model weight update processing based on the third image data set are: use the third image data set to train the model combined by the image perception model, the training diffusion model and the training Vincent graph model to obtain the initial combined model matrix; preset the weight matrix corresponding to the image dimension range to be optimized as the weight matrix of the dimension to be optimized; take the product of the weight matrix of the dimension to be optimized and the zero-rank matrix, and the sum of the initial combined model matrix as the updated weight matrix; based on the updated weight matrix, update the model combined by the image perception model, the training diffusion model and the training Vincent graph model to obtain the face image generation model.

[0015] The operations of semantic feature extraction processing in S2 are specifically as follows: the discretized basic image is subjected to several times of denoising processing to obtain a noisy basic image; the noisy basic image is subjected to convolution, nonlinear processing and maximum pooling processing to obtain a first pooling map; the first pooling map is subjected to convolution, nonlinear processing and maximum pooling processing to obtain a second pooling map; the second pooling map is subjected to convolution, nonlinear processing and maximum pooling processing to obtain a third pooling map; the third pooling map is subjected to convolution and upsampling processing, and then is spliced ​​with the third pooling map to obtain a third fusion map; the third fusion map is subjected to convolution and upsampling processing, and then is spliced ​​with the second pooling map to obtain a second fusion map; the second fusion map is subjected to convolution and upsampling processing, and then is spliced ​​with the first pooling map to obtain a basic semantic feature image.

[0016] The specific operation of feature cross-fusion processing is as follows: the basic semantic feature image is divided into several image blocks and features are extracted respectively to obtain several image block feature vectors; several image block feature vectors are converted into sequences after being processed by multi-head attention mechanism and feedforward neural network respectively to obtain image block semantic vector sequence; the target demand text is converted into a sequence after word segmentation processing to obtain target demand vector sequence; the target demand vector sequence is processed by multi-head attention mechanism and feedforward neural network to obtain text feature vector sequence; the image block semantic vector sequence and the text feature vector sequence are respectively spliced ​​with their respective position encoding vectors to obtain image block feature vector sequence and text feature vector sequence; using image features as queries and text features as keys and values, the attention weights between each image block feature vector and each text feature vector are obtained, and each attention weight is weighted summed with the corresponding text feature vector to obtain an image-text cross-fusion vector for performing denoising processing.

[0017] The noise addition process in S2 is implemented by the following formula:

[0018]

[0019] q(x t |x t-1 ) is the noise transition probability between the tth and t-1th times, x t is the t-th noise-added basic image, x t-1 is the t-1th noise base image, β t is the noise variance, γ is the unit covariance matrix, is a normal distribution, x k is the kth denoised basic image, i.e. the last denoised basic image, x0 is the discretized basic image, Π t＝[0,t,…,k] α t , α t is the t-th noise balance coefficient, α t =1-βt .

[0020] The denoising process in S3 is achieved through the following formula:

[0021]

[0022] p(z t-1 |z t ) is the denoising noise transition probability between the tth and t-1th times, z t is the t-th denoising feature, z t-1 is the t-1th feature, μ θ (z t ,t) is the mean of Gaussian noise, ε θ (z t ,t) is the Gaussian noise variance, is a normal distribution.

[0023] The objective loss function formula for training the encoder in the image perception model is as follows:

[0024]

[0025] L s is the encoder loss value, s i is the i-th first image, s i,g For s i The corresponding discretized image, || ||2 is the second normal form processing, and I is the total number of the first images.

[0026] A facial image generation system, used to implement the above-mentioned facial image generation method, comprising:

[0027] A basic requirement image generation module is used to obtain an image corresponding to the maximum cosine similarity of the feature vector corresponding to the target requirement text in the training text-graph model of the face image generation model as the basic requirement image;

[0028] The basic semantic feature image generation module is used to process the basic demand image through the encoder in the trained image perception model to obtain a discretized basic image; the discretized basic image is then processed several times by the trained diffusion model for noise addition and semantic feature extraction to obtain the basic semantic feature image;

[0029] A target face image generation module is configured to obtain a first generated image by cross-fusing the basic semantic feature image and the target demand text in the training text graph model, and then sequentially subjecting the image to denoising in the training diffusion model and processing by a decoder in the image perception model; a second generated image is obtained by cross-fusing the first generated image and the target demand text in the training text graph model, and then sequentially subjecting the image to denoising in the training diffusion model and processing by a decoder in the image perception model; a third generated image is obtained by cross-fusing the second generated image and the target demand text in the training text graph model, and so on, until the objective function value is less than the target threshold, and outputting the Nth generated image as the target face image;

[0030] The model construction and training module is used to construct an image perception model containing an encoder and a decoder. After training with the first image dataset, a trained image perception model is obtained. The encoder is used to downsample and discretize the input image. The decoder is used to generate an image based on the feature vector. The module is used to construct a diffusion model containing noise addition, semantic feature extraction and denoising. After training with the second image dataset, a trained diffusion model is obtained. The module is used to train the text-image model with the text-image pair dataset and zero-sample training.

[0031] A facial image generation device comprises a processor and a memory, wherein the processor implements the above-mentioned facial image generation method when executing a computer program stored in the memory.

[0032] A computer-readable storage medium is used to store a computer program, wherein the computer program implements the above-mentioned facial image generation method when executed by a processor.

[0033] The beneficial effects of the present invention are:

[0034] The present invention provides a facial image generation method. The method comprises the following steps: separately training an image perception model including an encoder and a decoder capable of representing key information of an image with a small amount of data and reducing data redundancy, and a diffusion model including noise addition processing, semantic feature extraction processing and denoising processing capable of accurately extracting and restoring semantic features of facial images in a noisy environment; then combining the trained image perception model, the trained diffusion model and the trained text-generated graph model; and performing a model weight update processing based on a third image data set to solve the overfitting problem, so that the model pays more attention to higher-level facial features during the face generation process, rather than excessively focusing on pixel features, thereby making the model natural when generating facial images, thereby obtaining a facial image generation model; finally, the target requirement text is processed by the facial image generation model to obtain a target facial image. DETAILED DESCRIPTION

[0035] Exemplary embodiments of the present disclosure are described in more detail below.

[0036] In this embodiment, before executing the face image generation method, a face image generation model based on the training image perception model, the training diffusion model and the training text graph model is first constructed. The specific process is as follows.

[0037] Obtain a trained image perception model. Construct an image perception model containing an encoder and decoder. After training on the first image dataset, obtain the trained image perception model. The purpose of the image perception model is to compress images to reduce the computational complexity of subsequent work. It can represent key image information with less data, compressing high-dimensional facial image data into a low-dimensional representation space. It can effectively capture the basic facial form in this space. This provides a compact and representative data representation for subsequent model integration, reduces data redundancy, and optimizes face generation in the presence of complex text prompts.

[0038] Among them, the encoder is used to downsample and discretize the input image. Specifically, the image (including the first image) input to the encoder is first downsampled by a convolution operation, and the dimension is reduced. Then, the low-dimensional features are discretized through a pre-trained codebook to obtain a discretized image (including a discretized basic image).

[0039] The operation of the encoder can be realized by the following formula:

[0040]

[0041] s i,g For s i The corresponding discretized image, Round() is the rounding process of the vector value, proj() is the projection process, tanh() is the hyperbolic tangent function, s i is the i-th first image.

[0042] The objective loss function formula of the encoder is as follows:

[0043]

[0044] L s is the encoder loss value, s i is the i-th first image, s i,g For s i The corresponding discretized image, || ||2 is the second normal form processing, and I is the total number of the first images.

[0045] The decoder is used to generate an image based on the feature vector.

[0046] Obtain a training diffusion model. Construct a diffusion model that includes noise addition, semantic feature extraction, and denoising. After training on a second image dataset, obtain the training diffusion model. The second image dataset is a high-resolution image dataset, including but not limited to the FFHQ dataset.

[0047] Among them, the denoising process adds noise to facial images, which not only expands the training dataset but also helps the model learn how to accurately generate natural, high-quality facial images under different noise interference by exposing it to noisy facial images. It also enables the model to focus on the semantic and structural information of facial images, while minimizing the interference of noise on features. This provides a relatively pure information foundation for subsequent denoising and optimization operations, and improves the generalization ability of the diffusion model. The semantic feature extraction process enables the model to further focus on the semantic and structural information of facial images. The denoising process gradually restores facial features through denoising, making the generated facial features more natural.

[0048] Obtain a trained text-to-graph model. Build a text-to-graph model that can generate images based on text. After training on a text-to-image dataset and zero-shot training, obtain the trained text-to-graph model. Examples of text-to-graph models include, but are not limited to, the CLIP model and the DeepFloyd IF model.

[0049] The specific operation of zero-shot training of the text-image model is as follows: In the initial text-image training model obtained after training the text-image model on the text-image pair dataset, local image features of the text-image pair dataset are combined to generate word embeddings. These word embeddings are added to the initial text-image training model to enrich the dictionary of the initial text-image training model, resulting in a trained text-image model. By permuting and combining features present in the text-image pair dataset, the model learns from unseen images. For example, the text-image pair dataset does not contain images of pink bangs, but does contain images of pink clothing and various images with bangs. The features of these images (color, bangs style) can be extracted, combined, and generated into word embeddings, which are then added to the dictionary of the initial text-image training model.

[0050] The above-mentioned image perception model containing an encoder and a decoder can be trained separately to represent the key information of the image with a smaller amount of data and reduce data redundancy, and the diffusion model containing noise addition processing, semantic feature extraction processing and denoising processing can accurately extract and restore the semantic features of facial images in a noisy environment, as well as the image text-based graph model that generates an image based on text description, which are used for subsequent combination to form a facial image generation model, which is beneficial to improving the generation efficiency and quality of the facial image generation model.

[0051] Finally, the trained image perception model, the trained diffusion model and the trained text graph model are combined to obtain a face image generation model. The following S1, S2 and S3 operations are implemented based on the face image generation model. Before executing the S1, S2 and S3 operations, the face image generation model also includes updating the model weights of the face image generation model based on the third image data set to solve the overfitting problem, so that the model pays more attention to higher-level facial features during the face generation process, rather than over-focusing on pixel features, so that the model generates face images naturally.

[0052] The specific operations of the model weight update processing based on the third image data set are: using the third image data set to train the model formed by combining the image perception model, the training diffusion model and the training Vincent graph model to obtain the initial combined model matrix; presetting the weight matrix corresponding to the image dimension range to be optimized as the weight matrix of the dimension to be optimized; taking the product of the weight matrix of the dimension to be optimized and the zero-rank matrix and the sum of the initial combined model matrix as the updated weight matrix; based on the updated weight matrix, updating the model formed by combining the image perception model, the training diffusion model and the training Vincent graph model to solve the overfitting problem that may exist in the initial combined model, so that the model pays more attention to higher-level facial features during the face generation process, rather than excessively paying attention to pixel features, to avoid the occurrence of picture fragmentation, unnatural characters, etc. in the combined model, and obtain a face image generation model.

[0053] The third image dataset is a low-resolution dataset, including but not limited to the CelebA dataset.

[0054] The target face image generation process is as follows.

[0055] S1. In the training text-graph model, obtain the image corresponding to the maximum cosine similarity of the feature vector corresponding to the target requirement text as the basic requirement image.

[0056] The calculation formula of cosine similarity is as follows:

[0057]

[0058] C is the i-th image I i The cosine similarity of the feature vector T corresponding to the target requirement text, I is the total number of images, and τ is the adjustment factor.

[0059] S2. The basic demand image is processed by the encoder in the trained image perception model to obtain a discretized basic image; the discretized basic image is sequentially processed by several times of noise addition and semantic feature extraction in the trained diffusion model to obtain a basic semantic feature image.

[0060] Several times of noise addition processing is achieved through the following formula:

[0061]

[0062] q(x t |x t-1 ) is the noise transition probability between the tth and t-1th times, x t is the t-th noise-added basic image, x t-1 is the t-1th noise base image, β t is the noise variance, γ is the unit covariance matrix, is a normal distribution, x k is the kth denoised basic image, i.e. the last denoised basic image, x0 is the discretized basic image, α t is the t-th noise balance coefficient, α t =1-β t .

[0063] The specific operations of semantic feature extraction processing are as follows: the discretized basic image is subjected to several noise addition processes to obtain a noisy basic image; the noisy basic image is subjected to convolution, nonlinear processing and maximum pooling processing to obtain a first pooling map; the first pooling map is subjected to convolution, nonlinear processing and maximum pooling processing to obtain a second pooling map; the second pooling map is subjected to convolution, nonlinear processing and maximum pooling processing to obtain a third pooling map; the third pooling map is subjected to convolution and upsampling processing, and then spliced ​​with the third pooling map to obtain a third fusion map; the third fusion map is subjected to convolution and upsampling processing, and then spliced ​​with the second pooling map to obtain a second fusion map; the second fusion map is subjected to convolution and upsampling processing, and then spliced ​​with the first pooling map to obtain a basic semantic feature image.

[0064] S3. After the basic semantic feature image and the target demand text are cross-fused in the training text-graph model, they are sequentially processed by denoising in the training diffusion model and decoder in the image perception model to obtain the first generated image; after the first generated image and the target demand text are cross-fused in the training text-graph model, they are sequentially processed by denoising in the training diffusion model and decoder in the image perception model to obtain the second generated image; after the second generated image and the target demand text are cross-fused in the training text-graph model, they are sequentially processed by denoising in the training diffusion model and decoder in the image perception model to obtain the third generated image; and so on, until the objective function value is less than the target threshold, the Nth generated image is output as the target face image.

[0065] Among them, the operation of feature cross-fusion processing is specifically as follows: the basic semantic feature image is divided into several image blocks and features are extracted respectively to obtain several image block feature vectors; several image block feature vectors are converted into sequences after being processed by multi-head attention mechanism and feedforward neural network respectively to obtain image block semantic vector sequence; the target demand text is converted into a sequence after word segmentation processing to obtain target demand vector sequence; the target demand vector sequence is processed by multi-head attention mechanism and feedforward neural network to obtain text feature vector sequence; the image block semantic vector sequence and the text feature vector sequence are respectively spliced ​​with their respective position encoding vectors to obtain image block feature vector sequence and text feature vector sequence; using image features as queries and text features as keys and values, the attention weights between each image block feature vector and each text feature vector are obtained, and each attention weight is processed with the corresponding text feature vector by weighted summation. Each image feature vector integrates the relevant information in the text features, thereby realizing the cross-fusion of image features and text features, and obtaining an image-text cross-fusion vector for performing denoising processing.

[0066] For the position coding vector of the image block semantic vector, a two-dimensional position coding vector is generated based on the position of the image block in the basic semantic feature image; for the position coding vector of the text feature vector, the corresponding word is processed by the sine-cosine position coding method in the Transformer model.

[0067] The above attention weight is obtained by the following calculation formula:

[0068]

[0069] α m,n is the feature vector I of the mth image block m and the nth text feature vector T n The attention weight, n is the total number of vectors in the text feature vector sequence, score(I m ,T n ) is the image block feature vector I m With the text feature vector T n Attention score, W Q 、W K are query weight and value weight respectively, d k is the dimension of the key vector.

[0070] The above image-text cross-fusion vector is obtained by the following calculation formula:

[0071]

[0072] F is the image-text cross-fusion vector, W Vare learnable weights.

[0073] The above denoising process is achieved through the following formula:

[0074]

[0075] p(z t-1 |z t ) is the denoising noise transition probability between the tth and t-1th times, z t is the t-th denoising feature, z t-1 is the t-1th feature, μ θ (z t ,t) is the mean of Gaussian noise, ε θ (z t ,t) is the Gaussian noise variance, is a normal distribution.

[0076] In addition, in order to further improve the naturalness of the target facial image, after obtaining the target facial image, the operation also includes circling a mask area on the target facial image, using the image corresponding to the mask area and the new constraint conditions as the basic requirement image in step two and the target requirement text in step three, executing the operations of steps two and three to generate a customized facial image.

[0077] The above process can be achieved through the following formula:

[0078] X=Refine-parts~s(Q,K,V(τ θ ~color,size,shape)),

[0079] X is the customized face image, Refine-parts is the image corresponding to the mask area, s() is the operation of steps 2 and 3, τ θ is the output of the Wensheng graph model, color, size, and shape are the color, shape, and size constraints in the new constraints respectively, and Q, K, and V() are feature cross-fusion processing.

[0080] This embodiment further provides a facial image generation system for implementing the above-mentioned facial image generation method, including:

[0081] A basic requirement image generation module is used to obtain an image corresponding to the maximum cosine similarity of the feature vector corresponding to the target requirement text in the training text-graph model of the face image generation model as the basic requirement image;

[0082] The basic semantic feature image generation module is used to process the basic demand image through the encoder in the trained image perception model to obtain a discretized basic image; the discretized basic image is then processed several times by the trained diffusion model for noise addition and semantic feature extraction to obtain the basic semantic feature image;

[0083] A target face image generation module is configured to obtain a first generated image by cross-fusing the basic semantic feature image and the target demand text in the training text graph model, and then sequentially subjecting the image to denoising in the training diffusion model and processing by a decoder in the image perception model; a second generated image is obtained by cross-fusing the first generated image and the target demand text in the training text graph model, and then sequentially subjecting the image to denoising in the training diffusion model and processing by a decoder in the image perception model; a third generated image is obtained by cross-fusing the second generated image and the target demand text in the training text graph model, and so on, until the objective function value is less than the target threshold, and outputting the Nth generated image as the target face image;

[0084] The model construction and training module is used to construct an image perception model containing an encoder and a decoder. After training with the first image dataset, a trained image perception model is obtained. The encoder is used to downsample and discretize the input image. The decoder is used to generate an image based on the feature vector. The module is used to construct a diffusion model containing noise addition, semantic feature extraction and denoising. After training with the second image dataset, a trained diffusion model is obtained. The module is used to train the text-image model with the text-image pair dataset and zero-sample training.

[0085] This embodiment further provides a facial image generation device, including a processor and a memory, wherein the processor implements the above-mentioned facial image generation method when executing a computer program stored in the memory.

[0086] This embodiment further provides a computer-readable storage medium for storing a computer program, wherein the computer program implements the above-mentioned facial image generation method when executed by a processor.

[0087] This embodiment is a method for generating facial images. After separately training an image perception model containing an encoder and a decoder that can represent key information of an image with a smaller amount of data and reduce data redundancy, and a diffusion model containing noise addition processing, semantic feature extraction processing, and denoising processing that can accurately extract and restore semantic features of facial images in a noisy environment, the trained image perception model, the trained diffusion model, and the trained text-based image model are combined, and the model weights are updated based on a third image data set to solve the overfitting problem, so that the model pays more attention to higher-level facial features during the face generation process rather than excessively focusing on pixel features, thereby making the model natural when generating facial images, thereby obtaining a facial image generation model. Finally, the target requirement text is processed by the facial image generation model to obtain a target facial image.

Claims

1. A method for generating a face image, characterized in that: The following operations are included: S1. In the training document graph model, obtain the image corresponding to the maximum cosine similarity of the feature vector corresponding to the target requirement text as the basic requirement image; S2, the basic demand image is processed by the encoder in the trained image perception model to obtain a discretized basic image; The discretized basic image is processed by several times of noise addition and semantic feature extraction in the training diffusion model to obtain the basic semantic feature image; S3. After the basic semantic feature image and the target demand text are cross-fused in the training text graph model, they are sequentially processed by denoising in the training diffusion model and decoder in the image perception model to obtain a first generated image; after the first generated image and the target demand text are cross-fused in the training text graph model, they are sequentially processed by denoising in the training diffusion model and decoder in the image perception model to obtain a second generated image; after the second generated image and the target demand text are cross-fused in the training text graph model, they are sequentially processed by denoising in the training diffusion model and decoder in the image perception model to obtain a third generated image; and so on, until the objective function value is less than the target threshold, the Nth generated image is output as the target face image; The method for obtaining the training image perception model comprises: constructing an image perception model including an encoder and a decoder, and obtaining the training image perception model after training with a first image data set; the encoder is used to downsample and discretize the input image; The decoder is used to generate an image based on the feature vector; The method for obtaining the training diffusion model comprises: constructing a diffusion model including noise addition processing, semantic feature extraction processing and denoising processing, and obtaining the training diffusion model after training with the second image data set; The method for obtaining the training text-generated graph model is as follows: the text-generated graph model is trained on a text-image pair dataset and a zero-sample training to obtain a training text-generated graph model; The operations of S1, S2 and S3 are implemented based on the face image generation model. Before executing the operations of S1, S2 and S3, the face image generation model also includes updating the model weights of the face image generation model based on the third image data set.

2. The facial image generation method according to claim 1, wherein: The operations of the model weight update process based on the third image dataset are specifically as follows: Using the third image dataset to train a model that combines the image perception model, the training diffusion model, and the training Vincent graph model, an initial combined model matrix is ​​obtained; a weight matrix corresponding to the image dimension range to be optimized is preset as the weight matrix of the dimension to be optimized; and the product of the weight matrix of the dimension to be optimized and the zero-rank matrix and the sum of the initial combined model matrix are used as the updated weight matrix; Based on the updated weight matrix, the model formed by combining the image perception model, the training diffusion model and the training text graph model is updated to obtain the face image generation model.

3. The facial image generation method according to claim 1, wherein: The operations of the semantic feature extraction process in S2 are specifically as follows: The discretized basic image is subjected to several noise addition processes to obtain a noisy basic image; The noise-added base image is processed by convolution, nonlinear processing and maximum pooling to obtain a first pooling map; the first pooling map is processed by convolution, nonlinear processing and maximum pooling to obtain a second pooling map; the second pooling map is processed by convolution, nonlinear processing and maximum pooling to obtain a third pooling map; the third pooling map is processed by convolution and upsampling, and then spliced ​​with the third pooling map to obtain a third fused map; the third fused map is processed by convolution and upsampling, and then spliced ​​with the second pooling map to obtain a second fused map; After convolution and upsampling, the second fusion image is concatenated with the first pooling image to obtain the basic semantic feature image.

4. The method for generating a facial image according to claim 1, wherein: The specific operations of feature cross fusion processing are as follows: The basic semantic feature image is divided into several image blocks and features are extracted from each block to obtain several image block feature vectors; the several image block feature vectors are processed by a multi-head attention mechanism and a feedforward neural network and converted into sequences to obtain an image block semantic vector sequence; the target demand text is segmented and converted into a sequence to obtain a target demand vector sequence; the target demand vector sequence is processed by a multi-head attention mechanism and a feedforward neural network to obtain a text feature vector sequence; The image block semantic vector sequence and the text feature vector sequence are respectively spliced ​​with their respective position encoding vectors to obtain the image block feature vector sequence and the text feature vector sequence; using the image features as the query and the text features as the key and value, the attention weight between each image block feature vector and each text feature vector is obtained, and each attention weight and the corresponding text feature vector are weighted summed to obtain the image-text cross-fusion vector for performing the denoising operation.

5. The method for generating a facial image according to claim 1, wherein: The noise addition processing in S2 is implemented by the following formula: q(x t |x t-1 ) is the noise transition probability between the tth and t-1th times, x t is the t-th noise-added basic image, x t-1 is the t-1th noise base image, β t is the noise variance, γ is the unit covariance matrix, is a normal distribution, x k is the kth denoised basic image, i.e. the last denoised basic image, x0 is the discretized basic image, α t is the t-th noise balance coefficient, α t =1-β t .

6. The facial image generation method according to claim 1, wherein: The denoising process in S3 is achieved by the following formula: p(z t-1 |z t ) is the denoising noise transition probability between the tth and t-1th times, z t is the t-th denoising feature, z t-1 is the t-1th feature, μ θ (z t ,t) is the mean of Gaussian noise, ε θ (z t ,t) is the Gaussian noise variance, is a normal distribution.

7. The method for generating a facial image according to claim 1, wherein: The objective loss function formula for training the encoder in the image perception model is as follows: L s is the encoder loss value, s i is the i-th first image, s i,g For s i The corresponding discretized image, || ||2 is the second normal form processing, and I is the total number of the first images.

8. A facial image generation system, used to implement the facial image generation method according to claim 1, characterized in that: include: A basic requirement image generation module is used to obtain an image corresponding to the maximum cosine similarity of the feature vector corresponding to the target requirement text in the training text-graph model of the face image generation model as the basic requirement image; The basic semantic feature image generation module is used to process the basic requirement image through the encoder in the trained image perception model to obtain a discretized basic image; The discretized basic image is processed by several times of noise addition and semantic feature extraction in the training diffusion model to obtain the basic semantic feature image; A target face image generation module is configured to obtain a first generated image by cross-fusing the basic semantic feature image and the target demand text in the training text graph model, and then sequentially subjecting the image to denoising in the training diffusion model and processing by a decoder in the image perception model; a second generated image is obtained by cross-fusing the first generated image and the target demand text in the training text graph model, and then sequentially subjecting the image to denoising in the training diffusion model and processing by a decoder in the image perception model; a third generated image is obtained by cross-fusing the second generated image and the target demand text in the training text graph model, and so on, until the objective function value is less than the target threshold, and outputting the Nth generated image as the target face image; The model construction and training module is used to construct an image perception model including an encoder and a decoder. After training with the first image dataset, a trained image perception model is obtained. The encoder is used to downsample and discretize the input image. The decoder is used to generate an image based on the feature vector; It is used to construct a diffusion model containing noise addition processing, semantic feature extraction processing and denoising processing. After training with the second image dataset, a training diffusion model is obtained; it is used to obtain a training text-graph model after training with the text-image pair dataset and zero-sample training.

9. A facial image generation device, characterized in that: The method comprises a processor and a memory, wherein the processor implements the facial image generation method according to any one of claims 1 to 7 when executing the computer program stored in the memory.

10. A computer-readable storage medium, characterized in that Used to store a computer program, wherein when the computer program is executed by a processor, the facial image generation method according to any one of claims 1 to 7 is implemented.

Images