Face image generation method, system and device and storage medium

Abstract

The invention relates to the technical field of image generation, in particular to a face image generation method, system and device and a medium. In order to solve the technical problem of low face generation quality in the prior art, the invention provides an image sensing model which can obtain key information and reduce data redundancy and comprises an encoder and a decoder through respective training. The method comprises the following steps of: combining with a training text graph model after a diffusion model which can accurately extract and recover semantic features of a human face image and contains noise adding processing, semantic feature extraction processing and denoising processing in a noise environment, and carrying out model weight updating processing to solve an overfitting problem; the model pays more attention to face features of a higher level in a face generation process instead of excessively paying attention to pixel features, so that the model is natural when a face image is generated, and a face image generation model is obtained; and processing the target demand text through a face image generation model to obtain a target face image.

Description

Technical Field

[0001] The present invention relates to the technical field of image generation, and in particular to a method, system, device and medium for generating a face image. Background Art

[0002] The text-to-image model is an emerging artificial intelligence technology that aims to generate corresponding images based on text descriptions. The technology combines advanced methods in natural language processing and computer vision to enable machines to understand text content and generate corresponding visual expressions. Such models usually use large neural networks, especially generative adversarial networks or variational autoencoders, and are trained on a large number of text-image pairs to achieve this goal. Generating facial images from text has important application value in generating data sets, virtual images, and editing facial images.

[0003] Modern facial image models can generate impressive images, but there may still be flaws in some complex scenes or details. The generated face images are blurry, or the generation process focuses too much on pixel information, resulting in unnatural face images or containing visual artifacts, resulting in poor quality of face image generation. Summary of the invention

[0004] The purpose of the present invention is to provide a method, system, device and medium for generating natural and high-quality facial images.

[0005] The technical solution of the present invention is as follows:

[0006] A method for generating a face image comprises the following operations:

[0007] S1. In the training text graph model, an image corresponding to the maximum value of the cosine similarity of the feature vector corresponding to the target demand text is obtained as the basic demand image;

[0008] S2, the basic demand image is processed by the encoder in the training image perception model to obtain a discretized basic image; the discretized basic image is sequentially processed by several times of noise addition and semantic feature extraction in the training diffusion model to obtain a basic semantic feature image;

[0009] S3. After the basic semantic feature image and the target demand text are cross-fused in the training text graph model, they are successively processed by denoising in the training diffusion model and decoder in the image perception model to obtain the first generated image; after the first generated image and the target demand text are cross-fused in the training text graph model, they are successively processed by denoising in the training diffusion model and decoder in the image perception model to obtain the second generated image; after the second generated image and the target demand text are cross-fused in the training text graph model, they are successively processed by denoising in the training diffusion model and decoder in the image perception model to obtain the third generated image; and so on, until the objective function value is less than the target threshold, and the Nth generated image is output as the target face image.

[0010] The method for obtaining the training image perception model is as follows: constructing an image perception model containing an encoder and a decoder, and obtaining the training image perception model after training with a first image data set; the encoder is used to downsample and discretize the input image; and the decoder is used to generate an image based on a feature vector.

[0011] The method for obtaining the training diffusion model is as follows: constructing a diffusion model including noise addition processing, semantic feature extraction processing and denoising processing, and obtaining the training diffusion model after training with the second image data set.

[0012] The method for obtaining the training text-generated graph model is as follows: the text-generated graph model is trained on a text-image pair data set and trained with a zero sample to obtain the training text-generated graph model.

[0013] The operations of S1, S2 and S3 are implemented based on the face image generation model. Before executing the operations of S1, S2 and S3, the face image generation model also includes updating the model weights of the face image generation model based on the third image data set.

[0014] The operations of the model weight update processing based on the third image data set are specifically as follows: use the third image data set to train a model that is a combination of an image perception model, a training diffusion model, and a training Vincent graph model to obtain an initial combined model matrix; preset a weight matrix corresponding to the image dimension range to be optimized as the weight matrix of the dimension to be optimized; take the product of the weight matrix of the dimension to be optimized and the zero-rank matrix, and the sum of the initial combined model matrix as the updated weight matrix; based on the updated weight matrix, update the model that is a combination of an image perception model, a training diffusion model, and a training Vincent graph model to obtain a face image generation model.

[0015] The operations of semantic feature extraction processing in S2 are specifically as follows: the discretized basic image is subjected to several times of denoising to obtain a noisy basic image; the noisy basic image is subjected to convolution, nonlinear processing and maximum pooling to obtain a first pooling map; the first pooling map is subjected to convolution, nonlinear processing and maximum pooling to obtain a second pooling map; the second pooling map is subjected to convolution, nonlinear processing and maximum pooling to obtain a third pooling map; the third pooling map is subjected to convolution and upsampling processing, and then is spliced ​​with the third pooling map to obtain a third fused map; the third fused map is subjected to convolution and upsampling processing, and then is spliced ​​with the second pooling map to obtain a second fused map; the second fused map is subjected to convolution and upsampling processing, and then is spliced ​​with the first pooling map to obtain a basic semantic feature image.

[0016] The specific operation of feature cross-fusion processing is as follows: the basic semantic feature image is divided into several image blocks and features are extracted respectively to obtain several image block feature vectors; several image block feature vectors are converted into sequences after being processed by a multi-head attention mechanism and a feedforward neural network to obtain an image block semantic vector sequence; the target demand text is converted into a sequence after word segmentation processing to obtain a target demand vector sequence; the target demand vector sequence is processed by a multi-head attention mechanism and a feedforward neural network to obtain a text feature vector sequence; the image block semantic vector sequence and the text feature vector sequence are respectively spliced ​​with their respective position encoding vectors to obtain an image block feature vector sequence and a text feature vector sequence; using image features as queries and text features as keys and values, the attention weights between each image block feature vector and each text feature vector are obtained, and each attention weight is weighted and summed with the corresponding text feature vector to obtain an image-text cross-fusion vector for performing denoising processing.

[0017] The several noise addition processes in S2 are implemented by the following formula:

[0018]

[0019] q(x t |x t-1 ) is the noise transition probability between the tth and t-1th times, x t is the t-th noise-added base image, x t-1 is the t-1th noise-added basic image, β t is the noise variance, γ is the unit covariance matrix, is a normal distribution, x k is the kth denoised basic image, i.e. the last denoised basic image, x0 is the discretized basic image, Π t＝[0,t,…,k] α t , α t is the t-th noise balance coefficient, α t =1-βt .

[0020] The denoising process in S3 is achieved through the following formula:

[0021]

[0022] p(z t-1 |z t ) is the denoising noise transition probability between the tth and t-1th times, z t is the t-th denoising feature, z t-1 is the t-1th feature, μ θ (z t ,t) is the mean value of Gaussian noise, ε θ (z t ,t) is the Gaussian noise variance, is a normal distribution.

[0023] The objective loss function formula for training the encoder in the image perception model is as follows:

[0024]

[0025] L s is the encoder loss value, s i is the i-th first image, s i,g For i The corresponding discretized image, || ||2 is the second paradigm processing, and I is the total number of the first images.

[0026] A face image generation system, used to implement the above-mentioned face image generation method, comprising:

[0027] A basic demand image generation module is used to obtain an image corresponding to the maximum value of the cosine similarity of the feature vector corresponding to the target demand text in the training text graph model of the face image generation model as the basic demand image;

[0028] The basic semantic feature image generation module is used to process the basic demand image through the encoder in the trained image perception model to obtain a discretized basic image; the discretized basic image is sequentially subjected to several noise addition processes and semantic feature extraction processes in the trained diffusion model to obtain a basic semantic feature image;

[0029] A target face image generation module is used for obtaining a first generated image by performing a denoising process in a training diffusion model and a decoder process in an image perception model after the basic semantic feature image and the target demand text are cross-fused in the training text graph model; obtaining a second generated image by performing a denoising process in a training diffusion model and a decoder process in an image perception model after the first generated image and the target demand text are cross-fused in the training text graph model; obtaining a third generated image by performing a denoising process in a training diffusion model and a decoder process in an image perception model after the second generated image and the target demand text are cross-fused in the training text graph model; and so on, until the target function value is less than the target threshold, outputting the Nth generated image as the target face image;

[0030] The model building and training module is used to build an image perception model containing an encoder and a decoder. After training with the first image data set, a trained image perception model is obtained. The encoder is used to downsample and discretize the input image. The decoder is used to generate an image based on a feature vector. The module is used to build a diffusion model containing noise addition, semantic feature extraction and denoising. After training with the second image data set, a trained diffusion model is obtained. The module is used to train the text-image model with a text-image pair data set and zero-sample training to obtain a trained text-image model.

[0031] A facial image generation device comprises a processor and a memory, wherein the processor implements the above-mentioned facial image generation method when executing a computer program stored in the memory.

[0032] A computer-readable storage medium is used to store a computer program, wherein the computer program implements the above-mentioned facial image generation method when executed by a processor.

[0033] The beneficial effects of the present invention are:

[0034] The present invention discloses a method for generating a face image. The method comprises the following steps: respectively training an image perception model including an encoder and a decoder which can represent key information of an image with a small amount of data and reduce data redundancy, and a diffusion model including a noise addition process, a semantic feature extraction process and a denoising process which can accurately extract and restore semantic features of a face image in a noisy environment. The trained image perception model, the trained diffusion model and the trained text image model are combined, and the model weights are updated based on a third image data set to solve the overfitting problem, so that the model pays more attention to higher-level face features during the face generation process, rather than excessively paying attention to pixel features, so that the model is natural when generating face images, and a face image generation model is obtained. Finally, a target demand text is processed by the face image generation model to obtain a target face image. DETAILED DESCRIPTION

[0035] Exemplary embodiments of the present disclosure are described in more detail below.

[0036] In this embodiment, before executing the face image generation method operation, a face image generation model based on the training image perception model, the training diffusion model and the training Vincent graph model is first constructed. The specific process is as follows.

[0037] Obtain a training image perception model. Construct an image perception model containing an encoder and a decoder, and obtain a training image perception model after training with the first image data set. The purpose of the image perception model is to compress the image to reduce the amount of computation for subsequent work. It can represent the key information of the image with a smaller amount of data, and compress the high-dimensional face image data into a low-dimensional representation space. It can effectively capture the basic form of the face in this space. This provides a compact and representative data representation method for subsequent model combination, reduces data redundancy, and optimizes the face generation effect in the case of complex text prompts.

[0038] Among them, the encoder is used to downsample and discretize the input image. Specifically, the image (including the first image) input to the encoder is first downsampled through a convolution operation to reduce the dimension, and then the low-dimensional features are discretized through a pre-trained codebook to obtain a discretized image (including a discretized basic image).

[0039] The operation of the encoder can be realized by the following formula:

[0040]

[0041] s i,g For i The corresponding discretized image, Round() is the rounding process of the vector value, proj() is the projection process, tanh() is the hyperbolic tangent function, s i is the i-th first image.

[0042] The objective loss function formula of the encoder is as follows:

[0043]

[0044] L s is the encoder loss value, s i is the i-th first image, s i,g For i The corresponding discretized image, || ||2 is the second paradigm processing, and I is the total number of the first images.

[0045] The decoder is used to generate an image based on the feature vector.

[0046] Acquire a training diffusion model. Construct a diffusion model including noise addition processing, semantic feature extraction processing and denoising processing, and obtain a training diffusion model after training with a second image dataset. The second image dataset is a high-resolution image dataset, including but not limited to the FFHQ dataset.

[0047] Among them, the denoising process adds noise to the face image, which not only expands the training data set, but also helps the model learn how to accurately generate natural high-quality face images under different noise interference by exposing the model to noisy face images. It also allows the model to pay attention to the semantic information and structural information of the face image, while minimizing the interference of noise on the features, providing a relatively pure information basis for subsequent denoising and optimization operations, and improving the generalization ability of the diffusion model. The semantic feature extraction process enables the model to further pay attention to the semantic information and structural information of the face image. The denoising process gradually restores the face features through denoising, making the face features generated more natural.

[0048] Obtain a training text-based graph model. Build a text-based graph model that can obtain images based on text. After training with a text-image pair dataset and zero-sample training, obtain a training text-based graph model. Text-based graph models include but are not limited to CLIP models and DeepFloyd IF models.

[0049] The specific operation of zero-shot training of the text-image model is as follows: in the initial text-image training model obtained after training the text-image model with the text-image pair dataset, the local features of the images in the text-image pair dataset are combined and word embedding is generated, and the word embedding is added to the initial text-image training model to enrich the dictionary of the initial text-image training model and obtain the training text-image model. By arranging and combining the features in the text-image pair dataset, the model can learn images that it has never seen. For example, there are no images with pink bangs in the text-image pair dataset, but there are pink clothes and various images with bangs. The features of these images (color, bangs style) can be extracted and combined to generate word embeddings, which are added to the dictionary of the initial text-image training model.

[0050] The above-mentioned image perception model containing an encoder and a decoder can be trained separately to represent the key information of the image with a smaller amount of data and reduce data redundancy, and the diffusion model containing noise addition processing, semantic feature extraction processing and denoising processing can accurately extract and restore the semantic features of facial images in a noisy environment, as well as the image text graph model generated according to the text description, which are used for subsequent combination to form a facial image generation model, which is beneficial to improving the generation efficiency and quality of the facial image generation model.

[0051] Finally, the trained image perception model, the trained diffusion model and the trained text map model are combined to obtain a face image generation model. The following S1, S2 and S3 operations are implemented based on the face image generation model. Before executing S1, S2 and S3 operations, the face image generation model also includes updating the model weights of the face image generation model based on a third image data set to solve the overfitting problem, so that the model pays more attention to higher-level facial features during the face generation process, rather than over-focusing on pixel features, so that the model generates face images naturally.

[0052] The operation of the model weight update processing based on the third image data set is specifically as follows: use the third image data set to train the model that is a combination of the image perception model, the training diffusion model and the training Vincent graph model to obtain an initial combined model matrix; preset the weight matrix corresponding to the image dimension range to be optimized as the weight matrix of the dimension to be optimized; take the product of the weight matrix of the dimension to be optimized and the zero-rank matrix, and the sum of the initial combined model matrix as the updated weight matrix; based on the updated weight matrix, update the model that is a combination of the image perception model, the training diffusion model and the training Vincent graph model to solve the overfitting problem that may exist in the initial combined model, so that the model pays more attention to higher-level facial features during the face generation process, rather than excessively focusing on pixel features, to avoid the appearance of picture fragmentation, unnatural characters, etc. in the combined model, and obtain a facial image generation model.

[0053] The third image dataset is a low-resolution dataset, including but not limited to the CelebA dataset.

[0054] The target face image generation process is as follows.

[0055] S1. In the training text graph model, an image corresponding to the maximum value of the cosine similarity of the feature vector corresponding to the target requirement text is obtained as the basic requirement image.

[0056] The calculation formula of cosine similarity is as follows:

[0057]

[0058] C is the i-th image I i The cosine similarity of the feature vector T corresponding to the target requirement text, I is the total number of images, and τ is the adjustment factor.

[0059] S2. The basic demand image is processed by the encoder in the trained image perception model to obtain a discretized basic image; the discretized basic image is successively processed by several times of noise addition and semantic feature extraction in the trained diffusion model to obtain a basic semantic feature image.

[0060] Several noise addition processes are implemented by the following formula:

[0061]

[0062] q(x t |x t-1 ) is the noise transition probability between the tth and t-1th times, x t is the t-th noise-added base image, x t-1 is the t-1th noise-added basic image, β t is the noise variance, γ is the unit covariance matrix, is a normal distribution, x k is the kth denoised basic image, i.e. the last denoised basic image, x0 is the discretized basic image, α t is the t-th noise balance coefficient, α t =1-β t .

[0063] The operations of semantic feature extraction processing are specifically as follows: the discretized basic image is subjected to several times of denoising to obtain a noisy basic image; the noisy basic image is subjected to convolution, nonlinear processing and maximum pooling to obtain a first pooling map; the first pooling map is subjected to convolution, nonlinear processing and maximum pooling to obtain a second pooling map; the second pooling map is subjected to convolution, nonlinear processing and maximum pooling to obtain a third pooling map; the third pooling map is subjected to convolution and upsampling processing, and then is spliced ​​with the third pooling map to obtain a third fused map; the third fused map is subjected to convolution and upsampling processing, and then is spliced ​​with the second pooling map to obtain a second fused map; the second fused map is subjected to convolution and upsampling processing, and then is spliced ​​with the first pooling map to obtain a basic semantic feature image.

[0064] S3. After the basic semantic feature image and the target demand text are cross-fused in the training text graph model, they are successively processed by denoising in the training diffusion model and decoder in the image perception model to obtain the first generated image; after the first generated image and the target demand text are cross-fused in the training text graph model, they are successively processed by denoising in the training diffusion model and decoder in the image perception model to obtain the second generated image; after the second generated image and the target demand text are cross-fused in the training text graph model, they are successively processed by denoising in the training diffusion model and decoder in the image perception model to obtain the third generated image; and so on, until the objective function value is less than the target threshold, and the Nth generated image is output as the target face image.

[0065] Among them, the operation of feature cross-fusion processing is specifically as follows: the basic semantic feature image is divided into several image blocks and features are extracted respectively to obtain several image block feature vectors; several image block feature vectors are converted into sequences after being processed by a multi-head attention mechanism and a feedforward neural network to obtain an image block semantic vector sequence; the target demand text is converted into a sequence after word segmentation processing to obtain a target demand vector sequence; the target demand vector sequence is processed by a multi-head attention mechanism and a feedforward neural network to obtain a text feature vector sequence; the image block semantic vector sequence and the text feature vector sequence are respectively spliced ​​with their respective position encoding vectors to obtain an image block feature vector sequence and a text feature vector sequence; using image features as queries and text features as keys and values, the attention weights between each image block feature vector and each text feature vector are obtained, each attention weight is processed with the corresponding text feature vector by weighted summation, and each image feature vector fuses the relevant information in the text features, thereby realizing the cross-fusion of image features and text features, and obtaining an image-text cross-fusion vector for performing denoising processing operations.

[0066] The position coding vector of the image block semantic vector is obtained by generating a two-dimensional position coding vector based on the position of the image block in the basic semantic feature image; the position coding vector of the text feature vector is obtained by processing the corresponding word using the sine-cosine position coding method in the Transformer model.

[0067] The above attention weight is obtained by the following calculation formula:

[0068]

[0069] α m,n is the feature vector I of the mth image block m and the nth text feature vector T n The attention weight, n is the total number of vectors in the text feature vector sequence, score(I m ,T n ) is the image block feature vector I m With the text feature vector T n Attention score, W Q , W K are query weight and value weight respectively, d k is the dimension of the key vector.

[0070] The above image-text cross-fusion vector is obtained by the following calculation formula:

[0071]

[0072] F is the image-text cross-fusion vector, W Vare learnable weights.

[0073] The above denoising process is achieved through the following formula:

[0074]

[0075] p(z t-1 |z t ) is the denoising noise transition probability between the tth and t-1th times, z t is the t-th denoising feature, z t-1 is the t-1th feature, μ θ (z t ,t) is the mean value of Gaussian noise, ε θ (z t ,t) is the Gaussian noise variance, is a normal distribution.

[0076] In addition, in order to further improve the naturalness of the target facial image, after the operation of obtaining the target facial image, it also includes circling a mask area on the target facial image, using the image corresponding to the mask area and the new constraints as the basic requirement image in step two and the target requirement text in step three, executing steps two and three to generate a customized facial image.

[0077] The above process can be realized by the following formula:

[0078] X=Refine-parts~s(Q,K,V(τ θ ~color,size,shape)),

[0079] X is the customized face image, Refine-parts is the image corresponding to the mask area, s() is the operation of step 2 and step 3, τ θ is the output of the Wensheng graph model, color, size, shape are the color, shape, and size constraints in the new constraints respectively, and Q, K, V() is the feature cross fusion processing.

[0080] This embodiment also provides a face image generation system, which is used to implement the above face image generation method, including:

[0081] A basic demand image generation module is used to obtain an image corresponding to the maximum value of the cosine similarity of the feature vector corresponding to the target demand text in the training text graph model of the face image generation model as the basic demand image;

[0082] The basic semantic feature image generation module is used to process the basic demand image through the encoder in the trained image perception model to obtain a discretized basic image; the discretized basic image is sequentially subjected to several noise addition processes and semantic feature extraction processes in the trained diffusion model to obtain a basic semantic feature image;

[0083] A target face image generation module is used for obtaining a first generated image by performing a denoising process in a training diffusion model and a decoder process in an image perception model after the basic semantic feature image and the target demand text are cross-fused in the training text graph model; obtaining a second generated image by performing a denoising process in a training diffusion model and a decoder process in an image perception model after the first generated image and the target demand text are cross-fused in the training text graph model; obtaining a third generated image by performing a denoising process in a training diffusion model and a decoder process in an image perception model after the second generated image and the target demand text are cross-fused in the training text graph model; and so on, until the target function value is less than the target threshold, outputting the Nth generated image as the target face image;

[0084] The model building and training module is used to build an image perception model containing an encoder and a decoder. After training with the first image data set, a trained image perception model is obtained. The encoder is used to downsample and discretize the input image. The decoder is used to generate an image based on a feature vector. The module is used to build a diffusion model containing noise addition, semantic feature extraction and denoising. After training with the second image data set, a trained diffusion model is obtained. The module is used to train the text-image model with a text-image pair data set and zero-sample training to obtain a trained text-image model.

[0085] This embodiment also provides a facial image generation device, including a processor and a memory, wherein the processor implements the above-mentioned facial image generation method when executing a computer program stored in the memory.

[0086] This embodiment also provides a computer-readable storage medium for storing a computer program, wherein the computer program implements the above-mentioned facial image generation method when executed by a processor.

[0087] The present embodiment is a method for generating a facial image. By separately training an image perception model containing an encoder and a decoder that can represent key information of an image with a smaller amount of data and reduce data redundancy, and a diffusion model containing noise addition processing, semantic feature extraction processing and denoising processing that can accurately extract and restore semantic features of facial images in a noisy environment, the trained image perception model, the trained diffusion model and the trained text image model are combined, and the model weights are updated based on a third image data set to solve the overfitting problem, so that the model pays more attention to higher-level facial features during the face generation process instead of excessively paying attention to pixel features, so that the model is natural when generating facial images, and a facial image generation model is obtained. Finally, the target demand text is processed by the facial image generation model to obtain a target facial image.

Claims

1. A method for generating a face image, characterized in that: The following operations are included: S1. In the training text graph model, an image corresponding to the maximum value of the cosine similarity of the feature vector corresponding to the target demand text is obtained as the basic demand image; S2, the basic demand image is processed by the encoder in the trained image perception model to obtain a discretized basic image; The discretized basic image is processed by several times of noise addition and semantic feature extraction in the training diffusion model to obtain a basic semantic feature image; S3, after the basic semantic feature image and the target demand text are cross-fused in the training text graph model, they are sequentially subjected to denoising processing in the training diffusion model and decoder processing in the image perception model to obtain a first generated image; after the first generated image and the target demand text are cross-fused in the training text graph model, they are sequentially subjected to denoising processing in the training diffusion model and decoder processing in the image perception model to obtain a second generated image; after the second generated image and the target demand text are cross-fused in the training text graph model, they are sequentially subjected to denoising processing in the training diffusion model and decoder processing in the image perception model to obtain a third generated image; and so on, until the objective function value is less than the target threshold, the Nth generated image is output as the target face image; The method for obtaining the training image perception model is as follows: constructing an image perception model including an encoder and a decoder, and obtaining the training image perception model after training with a first image data set; the encoder is used to downsample and discretize the input image; The decoder is used to generate an image based on the feature vector; The method for obtaining the training diffusion model is as follows: constructing a diffusion model including noise addition processing, semantic feature extraction processing and noise removal processing, and obtaining the training diffusion model after training with the second image data set; The method for obtaining the training text graph model is as follows: the text graph model is trained with a text image pair data set and a zero-sample training to obtain a training text graph model; The operations of S1, S2 and S3 are implemented based on a facial image generation model. Before executing the operations of S1, S2 and S3, the facial image generation model also includes updating the model weights of the facial image generation model based on a third image data set.

2. The method for generating a facial image according to claim 1, characterized in that: The operation of the model weight update process based on the third image data set is specifically as follows: Using the third image data set to train a model that is a combination of an image perception model, a training diffusion model, and a training Vincent graph model, an initial combined model matrix is ​​obtained; a weight matrix corresponding to the image dimension range to be optimized is preset as the weight matrix of the dimension to be optimized; and the product of the weight matrix of the dimension to be optimized and the zero-rank matrix and the sum of the initial combined model matrix are used as the updated weight matrix; Based on the updated weight matrix, the model formed by combining the image perception model, the training diffusion model and the training viz-graph model is updated to obtain a face image generation model.

3. The method for generating a facial image according to claim 1, wherein: The operations of semantic feature extraction processing in S2 are specifically as follows: The discretized basic image is subjected to several noise addition processes to obtain a noise-added basic image; The noisy basic image is processed by convolution, nonlinear processing and maximum pooling to obtain a first pooling map; the first pooling map is processed by convolution, nonlinear processing and maximum pooling to obtain a second pooling map; the second pooling map is processed by convolution, nonlinear processing and maximum pooling to obtain a third pooling map; the third pooling map is processed by convolution and upsampling, and then spliced ​​with the third pooling map to obtain a third fused map; the third fused map is processed by convolution and upsampling, and then spliced ​​with the second pooling map to obtain a second fused map; After convolution and upsampling, the second fusion image is concatenated with the first pooling image to obtain a basic semantic feature image.

4. The method for generating a facial image according to claim 1, wherein: The specific operations of feature cross fusion processing are as follows: The basic semantic feature image is divided into several image blocks and features are extracted respectively to obtain several image block feature vectors; the several image block feature vectors are respectively processed by a multi-head attention mechanism and a feedforward neural network and converted into sequences to obtain an image block semantic vector sequence; the target demand text is processed by word segmentation and converted into a sequence to obtain a target demand vector sequence; the target demand vector sequence is processed by a multi-head attention mechanism and a feedforward neural network to obtain a text feature vector sequence; The image block semantic vector sequence and the text feature vector sequence are respectively concatenated with their respective position encoding vectors to obtain the image block feature vector sequence and the text feature vector sequence; the image features are used as queries and the text features are used as keys and values ​​to obtain the attention weights between each image block feature vector and each text feature vector; each attention weight and the corresponding text feature vector are weighted summed to obtain the image-text cross-fusion vector for performing denoising operations.

5. The method for generating a facial image according to claim 1, wherein: The several noise addition processes in S2 are implemented by the following formula: q(x t |x t-1 ) is the noise transition probability between the tth and t-1th times, x t is the t-th noise-added base image, x t-1 is the t-1th noise-added basic image, β t is the noise variance, γ is the unit covariance matrix, is a normal distribution, x k is the kth denoised basic image, i.e. the last denoised basic image, x0 is the discretized basic image, α t is the t-th noise balance coefficient, α t =1-β t .

6. The method for generating a facial image according to claim 1, characterized in that: The denoising process in S3 is implemented by the following formula: p(z t-1 |z t ) is the denoising noise transition probability between the tth and t-1th times, z t is the t-th denoising feature, z t-1 is the t-1th feature, μ θ (z t ,t) is the mean value of Gaussian noise, ε θ (z t ,t) is the Gaussian noise variance, is a normal distribution.

7. The method for generating a facial image according to claim 1, characterized in that: The objective loss function formula for training the encoder in the image perception model is as follows: L s is the encoder loss value, s i is the i-th first image, s i,g For i The corresponding discretized image, || ||2 is the second paradigm processing, and I is the total number of the first images.

8. A face image generation system, used to implement the face image generation method according to claim 1, characterized in that: include: A basic demand image generation module is used to obtain an image corresponding to the maximum value of the cosine similarity of the feature vector corresponding to the target demand text in the training text graph model of the face image generation model as the basic demand image; The basic semantic feature image generation module is used to process the basic requirement image through the encoder in the trained image perception model to obtain a discretized basic image; The discretized basic image is processed by several times of noise addition and semantic feature extraction in the training diffusion model to obtain a basic semantic feature image; A target face image generation module is used for obtaining a first generated image by performing a denoising process in a training diffusion model and a decoder process in an image perception model after the basic semantic feature image and the target demand text are cross-fused in the training text graph model; obtaining a second generated image by performing a denoising process in a training diffusion model and a decoder process in an image perception model after the first generated image and the target demand text are cross-fused in the training text graph model; obtaining a third generated image by performing a denoising process in a training diffusion model and a decoder process in an image perception model after the second generated image and the target demand text are cross-fused in the training text graph model; and so on, until the target function value is less than the target threshold, outputting the Nth generated image as the target face image; The model building and training module is used to build an image perception model including an encoder and a decoder. After training with the first image data set, a training image perception model is obtained; the encoder is used to downsample and discretize the input image; The decoder is used to generate an image based on the feature vector; It is used to construct a diffusion model containing noise addition processing, semantic feature extraction processing and denoising processing, and obtains a training diffusion model after training with the second image data set; it is used to obtain a training text graph model after training with the text image pair data set and zero sample training.

9. A facial image generation device, characterized in that: The method comprises a processor and a memory, wherein the processor implements the facial image generation method according to any one of claims 1 to 7 when executing the computer program stored in the memory.

10. A computer-readable storage medium, characterized in that: Used to store a computer program, wherein when the computer program is executed by a processor, the method for generating a facial image according to any one of claims 1 to 7 is implemented.

Images