Fast face image super-resolution method and system based on stable diffusion model

Abstract

The present invention discloses a fast human face image super-resolution method and system based on a stable diffusion model, and relates to the field of image super-resolution technology. The method comprises the following steps: constructing a human face image super-resolution model based on a stable diffusion model, including a stable diffusion model, a guide extraction module and a guide injection module, wherein the guide extraction module generates visual guidance and text guidance according to an input low-resolution human face image, and the guide injection module injects the visual guidance and text guidance into the stable diffusion model, and the stable diffusion model generates a restored image as a human face high-resolution image based on the visual guidance and text guidance; training the human face image super-resolution model; and realizing fast human face image super-resolution using the trained human face image super-resolution model. The present invention combines visual guidance and text guidance to fine-tune the stable diffusion model, which can not only greatly improve the authenticity of the human face image and maintain considerable consistency, but also speed up the restoration speed.

Description

Technical Field

[0001] The present invention relates to the technical field of image super-resolution, and in particular to a fast human face image super-resolution method and system based on a stable diffusion model. Background Art

[0002] At present, there are three main types of image super-resolution methods based on stable diffusion models: image super-resolution methods based on stable diffusion models with a de novo training strategy, image super-resolution methods based on stable diffusion models without a training strategy, and image super-resolution methods based on stable diffusion models with a fine-tuning strategy.

[0003] Image super-resolution methods based on stable diffusion models with a de novo training strategy aim to retrain a complete stable diffusion model to achieve high-quality image reconstruction; most methods based on retraining strategies require huge computational resources and a large number of sampling steps, which limits the scope of application and restoration efficiency of the model. Image super-resolution methods based on stable diffusion models without training strategies use low-resolution images as constraints and extract inherent structures and textures as a supplement to the generated images at each step to ensure data consistency; most works based on non-training strategies are dedicated to noiseless inverse problems, and the restoration results obtained are usually very inconsistent. Image super-resolution methods based on stable diffusion models with fine-tuning strategies reduce computational costs and improve image consistency, but they only use a single low-resolution image as a guide to restore the image, which results in poor authenticity of the restored image. The above methods all ignore the joint visual and textual guidance to accelerate sampling and improve the authenticity of the restored image. Summary of the invention

[0004] The purpose of the present invention is to solve the problem of low efficiency and poor effect of super-resolution restoration of facial images in the prior art.

[0005] The technical solution adopted by the present invention to solve the technical problem is: to provide a fast face image super-resolution method based on a stable diffusion model, comprising the following steps:

[0006] A face image super-resolution model based on a stable diffusion model is constructed, including a stable diffusion model, a guide extraction module based on multimodal vision and text, and a guide injection module based on cross attention; the guide extraction module generates visual guidance and text guidance according to the input low-resolution face image; the guide injection module injects the visual guidance and text guidance into the stable diffusion model, and the stable diffusion model generates a restored image as a high-resolution face image based on the visual guidance and text guidance;

[0007] Training a face image super-resolution model to obtain a trained face image super-resolution model;

[0008] Fast face image super-resolution is achieved using the trained face image super-resolution model.

[0009] Preferably, the multimodal vision and text-based guidance extraction module includes:

[0010] The visual guidance extraction module obtains a low-dimensional image representation as a visual guide by inputting the low-resolution face image into the pre-trained VAE encoder model ;

[0011] The text-guided extraction module obtains text prompt words about the low-resolution image by inputting the low-resolution face image into the pre-trained GPT-4 model. ; Then get the text prompt word Input to the pre-trained CLIP text encoder to get text guidance .

[0012] Preferably, the VAE encoder model adopts a multi-stage downsampling network, including:

[0013] The input unit is a convolutional layer, which converts the input low-resolution face image into input features. ;

[0014] 3 downsampling units, including 2 convolutional layers and 1 downsampling layer, input features After passing through three downsampling units in sequence, it is converted into downsampling features ;

[0015] The middle unit, including 2 convolutional layers and 1 spatial self-attention layer, downsamples the features Convert to intermediate features ;

[0016] The output unit includes a convolution layer with 4 convolution kernels and a normalization layer to convert the intermediate features Convert to visual guidance .

[0017] Preferably, the model of the CLIP text encoder adopts a multi-layer self-attention network, including:

[0018] Input unit, including 1 embedding layer and 1 position encoding layer; input text prompt word Generate word sequence after word segmentation ; word sequence Each word in is mapped to a word vector through the embedding layer ; The position encoding layer adds position encoding information to each word vector to obtain word embedding ;

[0019] 12 self-attention units connected in sequence, each of which includes a multi-head self-attention mechanism and a feedforward neural network; each self-attention unit embeds the word Input to the multi-head self-attention layer to get the attention output , attention output The output features are obtained through nonlinear transformation of one feedforward neural network layer. ;

[0020] The output unit includes a global average pooling layer and a linear layer, which outputs the features After being transformed by the global average pooling layer and the linear layer, the text guide is obtained .

[0021] Preferably, the guide injection module injects the visual guide and the text guide into the stable diffusion model, specifically: the visual guide obtained by the guide extraction module and text guide The cross attention is injected into the noise predictor of the stable diffusion model to obtain Gaussian noise under different denoising steps t. , represents the noise image at step t.

[0022] Preferably, the stable diffusion model generates a restored image as a high-resolution face image based on visual guidance and text guidance, specifically: combining the Gaussian noise predicted by the noise predictor at step t and the noise image at step t , get the original image predicted at step t :

[0023] ;

[0024] in, α t ∈ [ 0 , 1 ] is the known noise intensity modulation factor;

[0025] According to the noise image of step t , predicted Gaussian noise and the predicted original image , get the noise image of step t-1 :

[0026] ;

[0027] in, σ ∈ [ 0 , 1 ] is a known regulatory factor. represents noise sampled from standard Gaussian noise;

[0028] Noise image sampled from a standard normal distribution in T steps Start by iterating the above process T-1 times to get the restored image .

[0029] Preferably, the noise predictor adopts a Unet network, and each level of encoding unit in the encoding stage, each level of decoding unit in the decoding stage and the output unit include a cross attention layer, and the guide injection module converts the step number t, the visual guide and text guide Injected into the encoding stage, decoding stage and output stage of the noise predictor.

[0030] Preferably, the training of the facial image super-resolution model to obtain a trained facial image super-resolution model includes training of a noise predictor, and the training objectives are as follows:

[0031] ℒ =  x t , t , ε , ℱ V , ℱ T [ ‖ ε − ε ( x t , t , ℱ V , ℱ T ) ‖ 2 2 ] ;

[0032] in, is the noise image at time t, is a latent vector sampled from a standard Gaussian prior with different directions, t ∈ [ 1 , T ] is a random time step, Express The expected value of .

[0033] Preferably, the method of using a trained face image super-resolution model to achieve fast face image super-resolution is specifically as follows: a low-resolution face image is input into a guide extraction module, visual guidance and text guidance generated by the guide extraction module are injected into a noise predictor of a stable diffusion model through a guide injection module, the noise predictor generates predicted noise based on the visual guidance and text guidance, and the stable diffusion model generates a high-resolution face image based on the predicted noise.

[0034] The present invention also provides a fast face image super-resolution system based on a stable diffusion model, comprising:

[0035] A model building module is used to build a face image super-resolution model based on a stable diffusion model, including a stable diffusion model, a guide extraction module based on multimodal vision and text, and a guide injection module based on cross-attention; the guide extraction module based on multimodal vision and text generates visual guidance and text guidance according to the input low-resolution face image; the guide injection module based on cross-attention injects the visual guidance and text guidance into the stable diffusion model, and the stable diffusion model generates a restored image as a high-resolution face image based on the visual guidance and text guidance;

[0036] A model training module, used for training a face image super-resolution model to obtain a trained face image super-resolution model;

[0037] The model application module is used to use the trained face image super-resolution model to achieve fast face image super-resolution.

[0038] The present invention has the following beneficial effects:

[0039] (1) Visual guidance provides accurate pixel-level conditional constraints, which helps to reconstruct a restoration result that is more consistent with the real image; text guidance further corrects the semantic information on the restored image, which helps to improve the authenticity of the restoration result; by combining text guidance and visual guidance, the present invention can promote the trade-off between authenticity and consistency of the restored image and achieve better visual effects.

[0040] (2) The present invention introduces visual guidance and text guidance to modulate the stable diffusion model, which helps to obtain a more compact diffusion prior. The compact diffusion prior can reduce the requirement on the number of sampling steps, so that high-quality restoration results can be obtained with fewer sampling steps, thereby achieving fast face image super-resolution.

[0041] The present invention is further described in detail below in conjunction with the accompanying drawings and embodiments, but the present invention is not limited to the embodiments. BRIEF DESCRIPTION OF THE DRAWINGS

[0042] Figure 1 A method step diagram of an embodiment of the present invention;

[0043] Figure 2 Schematic diagram of a guide extraction module according to an embodiment of the present invention; wherein (a) is a schematic diagram of a VAE encoder, and (b) is a schematic diagram of a CLIP text encoder;

[0044] Figure 3 A schematic diagram of a noise predictor and a guide injection module according to an embodiment of the present invention;

[0045] Figure 4 This is a diagram showing the effect of an embodiment of the present invention on a ×4 blind image super-resolution task on a DIV2K test set;

[0046] Figure 5 4 is a system structure diagram of an embodiment of the present invention. DETAILED DESCRIPTION

[0047] See also Figure 1 FIG. 1 is a diagram showing steps of a method according to an embodiment of the present invention, comprising the following steps:

[0048] S101, constructing a face image super-resolution model based on a stable diffusion model;

[0049] S102, training a facial image super-resolution model to obtain a trained facial image super-resolution model;

[0050] S103, using the trained face image super-resolution model to achieve fast face image super-resolution.

[0051] Specifically, the face image super-resolution model constructed in S101 includes a stable diffusion model, a guide extraction module based on multimodal vision and text, and a guide injection module based on cross-attention; the guide extraction module generates visual guidance and text guidance according to the input low-resolution face image; the guide injection module injects the visual guidance and text guidance into the stable diffusion model, and the stable diffusion model generates a restored image as a high-resolution face image based on the visual guidance and text guidance.

[0052] Specifically, the guide extraction module includes a visual guide extraction module and a text guide extraction module. The visual guide extraction module extracts the low-resolution image Input into the pre-trained VAE encoder model to obtain a low-dimensional image representation as a visual guide ; The text-guided extraction module extracts the low-resolution image Input to the pre-trained GPT-4 model to get the low-resolution image Text prompt word , and then the obtained text prompt word Input to the pre-trained CLIP text encoder to get relevant text guidance .

[0053] Specifically, the VAE encoder model is as follows Figure 2 As shown in (a), a multi-stage downsampling network is used, which is composed of 1 input unit, 3 downsampling units, 1 intermediate unit and 1 output unit in sequence; the input unit consists of 1 convolution kernel size , a convolution layer with 128 convolution kernels, by , the number of channels is 3 for low-resolution images After the input unit, the size can be obtained , input features with 128 channels ; The three downsampling units are composed of two convolution kernels of size , a convolutional layer with 128, 256 and 512 convolution kernels and a downsampling layer with a downsampling rate of 2. The input feature After three downsampling units in sequence, we can get a size of , downsampled features with 512 channels ; The intermediate unit consists of two convolution kernels with sizes of , a convolution layer with 512 convolution kernels and a spatial self-attention layer, downsampling features Through the intermediate unit, we can get a size of , the intermediate feature with 512 channels ; The output unit consists of a normalization layer and a convolution kernel with a size of , the number of convolution kernels is 4, and the intermediate features After the output unit, the size can be obtained , visual guidance with 4 channels .

[0054] Specifically, the model of CLIP text encoder is as follows Figure 2 As shown in (b) of the figure, a multi-layer self-attention network is used. The multi-layer self-attention network consists of 1 input unit, 12 self-attention units and 1 output unit in sequence; the input unit consists of 1 embedding layer and 1 position encoding layer. First, the text prompt word is input After word segmentation, a word sequence of size (1,77) is generated. . Secondly, each word is mapped into a word vector of size (1, 77, 1024) through the embedding layer Finally, the position encoding layer adds position encoding information to each word vector to obtain a word embedding of size (1, 77, 1024) ; The 12 self-attention units are composed of a multi-head self-attention mechanism and a feedforward neural network in sequence. First, the word embedding of size (1, 77, 1024) Input to the multi-head self-attention layer to get the attention output of size (1,77,1024) , attention output After a nonlinear transformation of a feedforward neural network layer, the output features of size (1, 77, 1024) are obtained ; The output unit consists of a global average pooling layer and a linear layer. After the global average pooling layer and the linear layer are transformed, a text guide with a size of (1,77,1024) can be obtained. .

[0055] See also Figure 3As shown, it is a schematic diagram of a noise predictor and a guide injection module of an embodiment of the present invention. The noise predictor of the embodiment of the present invention adopts a Unet network, and the Unet network is composed of a trunk feature extraction branch and a jump connection guide branch; the trunk feature extraction branch can be composed of an encoding stage, a decoding stage and an output stage; the encoding stage is composed of a shallow encoding unit, an intermediate encoding unit and a deep encoding unit in sequence, and a shallow encoding feature map, an intermediate encoding feature map and a deep encoding feature map are obtained in sequence, wherein the shallow, intermediate and deep encoding units in the encoding stage are all composed of 1 The method comprises the following steps: the first step is to generate a plurality of convolutional layers, a cross-attention layer, and a downsampling layer with a downsampling rate of 2; the second step is to generate a plurality of cross-attention layer layers, and the third step is to generate a plurality of cross-attention layer layers, and the fourth step is to generate a plurality of cross-attention layer layers. The first step is to generate a plurality of cross-attention layer layers, and the fourth step is to generate a plurality of cross-attention layer layers. The fourth step is to generate a plurality of cross-attention layer layers, and the fifth step is to generate a plurality of cross-attention layer layers. The fifth step is to generate a plurality of cross-attention layer layers, and the fifth step is to generate a plurality of cross-attention layer layers. , where the output unit is composed of 1 convolutional layer, 1 cross attention layer and 1 upsampling layer with an upsampling rate of 2; the jump connection branch combines the deep encoding feature map with the deep decoding feature map, the intermediate encoding feature map with the intermediate decoding feature map, and the shallow encoding feature map with the shallow decoding feature map, and inputs them into the intermediate decoding unit, the shallow decoding unit and the output unit respectively.

[0056] The guide injection module passes the step number t, visual guide, and the cross-attention layer in all the above encoding units, all decoding units, and output units. and text guide Injected into the noise predictor of the stable diffusion model to obtain Gaussian noise under different denoising steps t ; The training objectives of the noise predictor are as follows:

[0057] ℒ =  x t , t , ε , ℱ V , ℱ T [ ‖ ε − ε ( x t , t , ℱ V , ℱ T ) ‖ 2 2 ] ,

[0058] in, yes The noise image at time instant, is a latent vector sampled from a standard Gaussian prior with different directions, t ∈ [ 1 , T ] is a random time step.

[0059] Specifically, the stable diffusion model generates a restored image as a high-resolution face image based on visual guidance and text guidance, including the following steps:

[0060] Combined with the Gaussian noise predicted at step t and the noise image at step t The original image predicted at step t can be obtained :

[0061]

[0062] in, α t ∈ [ 0 , 1 ] is the known noise intensity modulation factor;

[0063] The noise image of step t , predicted Gaussian noise and the predicted original image Input them into the iterative denoising formula together to get the noise image of step t-1 :

[0064]

[0065] in, σ ∈ [ 0 , 1 ] is a known regulatory factor. is the noise sampled from standard Gaussian noise;

[0066] Hidden code sampled from a standard normal distribution (i.e., a noisy image) taken T steps Start by iterating the above process T-1 times to get the restored image .

[0067] Specifically, the data set preparation process involved in S102 is as follows:

[0068] S1021. In order to evaluate the effectiveness of the model in reconstructing low-resolution images in the real world, relevant experiments were conducted on the face attribute dataset CelebA. In addition, 1k validation sets were reserved for model evaluation.

[0069] S1022, eliminating images in the image set that are blurry, have incorrect data formats, are too similar, etc.;

[0070] S1023. Perform real-world simulated degradation operations on the initial screening images. Use the degradation process provided by RealESRGAN to process the original CelebA dataset to simulate complex natural degradation, including blur, noise, size change, and artifacts.

[0071] S1024, the images in the CelebA dataset that have been degraded by natural simulation are cropped by center to obtain image blocks with a size of 512×512 and a number of channels of 3, which are used as the dataset, and the Experiments were conducted on the blind image super-resolution (SR) task;

[0072] S1025. In order to reasonably evaluate the reconstruction effect of SR images, multiple perceptual quality evaluation indicators are provided, including: Frechet Inception Distance score (FID↓), Learned Perceptual Image Patch Similarity (LPIPS↓), Naturalness Image Quality Evaluator (NIQE↓), Multi-Scale Image Quality (MUSIQ↑), Contrastive Language-Image Pretraining for Image Quality Assessment (CLIPIQA↑), Masked Attention Neural Image Quality Assessment (MANIQA↑).

[0073] Specifically, S103 is as follows: inputting the low-resolution face image into the guide extraction module, injecting the generated visual guidance and text guidance into the noise predictor of the stable diffusion model through the guide injection module, the noise predictor generates predicted noise according to the visual guidance and text guidance, and the stable diffusion model generates a high-resolution face image based on the predicted noise.

[0074] Specifically, in order to verify the application effect of the embodiment of the present invention, the embodiment of the present invention is used to perform a ×4 blind image super-resolution task on the DIV2K test set. The effect diagram is shown in FIG. Figure 4 As shown, it can be seen that the embodiment of the present invention can significantly improve the authenticity of the restored image.

[0075] See also Figure 5 As shown, it is a system structure diagram of an embodiment of the present invention, including:

[0076] The model construction module 501 is used to construct a face image super-resolution model based on a stable diffusion model, including a stable diffusion model, a guide extraction module based on multimodal vision and text, and a guide injection module based on cross attention; the guide extraction module based on multimodal vision and text generates visual guidance and text guidance according to the input low-resolution face image; the guide injection module based on cross attention injects the visual guidance and text guidance into the stable diffusion model, and the stable diffusion model generates a restored image as a high-resolution face image based on the visual guidance and text guidance;

[0077] A model training module 502 is used to train a facial image super-resolution model to obtain a trained facial image super-resolution model;

[0078] The model application module 503 is used to implement fast face image super-resolution using the trained face image super-resolution model.

[0079] The present invention combines vision and text guidance by injecting cross-attention into the stable diffusion model to achieve high-quality image super-resolution in an iterative manner, which can not only significantly improve the authenticity of the restored image, but also speed up the image restoration speed.

[0080] The above are only preferred embodiments of the present invention and are not intended to limit the present invention. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of the present invention should be included in the protection scope of the present invention.

Claims

1. A fast face image super-resolution method based on a stable diffusion model, characterized in that: The following steps are involved: A face image super-resolution model based on a stable diffusion model is constructed, including a stable diffusion model, a guide extraction module based on multimodal vision and text, and a guide injection module based on cross attention; the guide extraction module generates visual guidance and text guidance according to the input low-resolution face image; the guide injection module injects the visual guidance and text guidance into the stable diffusion model, and the stable diffusion model generates a restored image as a high-resolution face image based on the visual guidance and text guidance; Training a face image super-resolution model to obtain a trained face image super-resolution model; Use the trained face image super-resolution model to achieve fast face image super-resolution; The multimodal vision and text-based guidance extraction module includes: The visual guidance extraction module obtains a low-dimensional image representation as a visual guide by inputting the low-resolution face image into the pre-trained VAE encoder model The text-guided extraction module obtains text prompt words about the low-resolution image by inputting the low-resolution face image into the pre-trained GPT-4 model. Then the obtained text prompt word Input to the pre-trained CLIP text encoder to get text guidance The VAE encoder model uses a multi-stage downsampling network, including: The input unit is a convolutional layer, which converts the input low-resolution face image into input features. 3 downsampling units, including 2 convolutional layers and 1 downsampling layer, input features After passing through three downsampling units in sequence, it is converted into downsampling features The middle unit, including 2 convolutional layers and 1 spatial self-attention layer, downsamples the features Convert to intermediate features The output unit includes a convolution layer with 4 convolution kernels and a normalization layer to convert the intermediate features Convert to visual guidance 2. The fast face image super-resolution method based on the stable diffusion model according to claim 1, characterized in that: The CLIP text encoder model adopts a multi-layer self-attention network, including: Input unit, including 1 embedding layer and 1 position encoding layer; input text prompt word Generate word sequence after word segmentation Word sequence Each word in is mapped to a word vector through the embedding layer The position encoding layer adds position encoding information to each word vector to obtain word embedding 12 self-attention units connected in sequence, each of which includes a multi-head self-attention mechanism and a feedforward neural network; each self-attention unit embeds the word Input to the multi-head self-attention layer to get the attention output Attention Output The output features are obtained through nonlinear transformation of one feedforward neural network layer. The output unit includes a global average pooling layer and a linear layer, which outputs the features After being transformed by the global average pooling layer and the linear layer, the text guide is obtained 3. The fast face image super-resolution method based on a stable diffusion model according to claim 1, characterized in that: The guide injection module injects visual guides and text guides into the stable diffusion model, specifically: the visual guides obtained by the guide extraction module are injected into the stable diffusion model. and text guide The cross attention is injected into the noise predictor of the stable diffusion model to obtain Gaussian noise under different denoising steps t. x t represents the noise image at step t.

4. The fast face image super-resolution method based on the stable diffusion model according to claim 3, characterized in that: The stable diffusion model generates a restored image as a high-resolution face image based on visual guidance and text guidance, specifically: combining the Gaussian noise predicted by the noise predictor at step t and the noise image x at step t t , get the original image predicted at step t Among them, α t ∈[0,1] is the known noise intensity modulation factor; According to the noise image x at step t t , predicted Gaussian noise and the predicted original image Get the noise image x at step t-1 t-1 : Among them, σ∈[0,1] is a known control factor, and z represents the noise sampled from standard Gaussian noise; A noise image x sampled from a standard normal distribution in T steps T Starting from ~N(0,1), the above process is iterated T-1 times to obtain the restored image x0.

5. The fast face image super-resolution method based on the stable diffusion model according to claim 3, characterized in that: The noise predictor adopts the Unet network. Each level of encoding unit in the encoding stage, each level of decoding unit in the decoding stage and the output unit include a cross attention layer. The guide injection module uses the cross attention layer to convert the step number t, the visual guide and text guide Injected into the encoding stage, decoding stage and output stage of the noise predictor.

6. The fast face image super-resolution method based on the stable diffusion model according to claim 5, characterized in that: After training the face image super-resolution model, a trained face image super-resolution model is obtained, including training of the noise predictor, and the training objectives are as follows: in, is a latent vector sampled from a standard Gaussian prior with different directions, t∈[1,T] is a random time step, Express The expected value of .

7. The fast face image super-resolution method based on a stable diffusion model according to claim 1, characterized in that: The method uses a trained face image super-resolution model to achieve fast face image super-resolution, specifically: a low-resolution face image is input into a guide extraction module, visual guidance and text guidance generated by the guide extraction module are injected into a noise predictor of a stable diffusion model through a guide injection module, the noise predictor generates predicted noise according to the visual guidance and text guidance, and the stable diffusion model generates a high-resolution face image based on the predicted noise.

8. A fast face image super-resolution system based on a stable diffusion model, characterized in that: include: A model building module is used to build a face image super-resolution model based on a stable diffusion model, including a stable diffusion model, a guide extraction module based on multimodal vision and text, and a guide injection module based on cross-attention; the guide extraction module based on multimodal vision and text generates visual guidance and text guidance according to the input low-resolution face image; the guide injection module based on cross-attention injects the visual guidance and text guidance into the stable diffusion model, and the stable diffusion model generates a restored image as a high-resolution face image based on the visual guidance and text guidance; A model training module, used for training a face image super-resolution model to obtain a trained face image super-resolution model; Model application module, used to realize fast face image super-resolution using the trained face image super-resolution model; The multimodal vision and text-based guidance extraction module includes: The visual guidance extraction module obtains a low-dimensional image representation as a visual guide by inputting the low-resolution face image into the pre-trained VAE encoder model The text-guided extraction module obtains text prompt words about the low-resolution image by inputting the low-resolution face image into the pre-trained GPT-4 model. Then the obtained text prompt word Input to the pre-trained CLIP text encoder to get text guidance The VAE encoder model uses a multi-stage downsampling network, including: The input unit is a convolutional layer, which converts the input low-resolution face image into input features. 3 downsampling units, including 2 convolutional layers and 1 downsampling layer, input features After passing through three downsampling units in sequence, it is converted into downsampling features The middle unit, including 2 convolutional layers and 1 spatial self-attention layer, downsamples the features Convert to intermediate features The output unit includes a convolution layer with 4 convolution kernels and a normalization layer to convert the intermediate features Convert to visual guidance

Images