Frequency domain sensing face shadow elimination method

Abstract

The invention relates to the technical field of image processing, and discloses a frequency domain perception face shadow elimination method, which comprises the following steps of: 1, establishing a face shadow image data set X with separated portrait and background and a face shadow image data set Y with separated portrait and background by utilizing a face shadow data set; step 2, constructing a shadow removal network model; step 3, taking a shadow image x in the face shadow image data set X as input to train the shadow removal network model, outputting a skin color image x1 and a texture image x2 after x passes through a frequency domain image decomposition network, and outputting a final shadow removal result image x3 after x, x1 and x2 pass through a shadow removal network guided by color and texture; step 4, constructing a discriminator network model; and step 5, taking the final shadow elimination result image x3 and the corresponding shadow-free image y in the shadow-free image data set Y as input, and identifying the final shadow elimination result image x3 through the identifier network model.

Description

Technical Field

[0001] The invention belongs to the technical field of image processing, and in particular relates to a frequency domain perception method for removing face shadows. Background Art

[0002] Due to the changes in lighting conditions and the uniqueness of facial structure, shadows often appear in face images. The low brightness of the shadow area not only reduces the visibility and authenticity of the image, but also weakens the ability of key tasks such as face recognition, image restoration and face image reconstruction, affecting the accuracy and effectiveness of computer vision processing. Therefore, effectively removing shadows from facial images and restoring clear images is a necessary and realistic task.

[0003] Removing shadows from face images is a complex and challenging task. First, the illumination in the shadowed area varies greatly due to changes in light intensity and direction, which increases the difficulty of processing. Second, the color and illumination in the shadowed area are significantly different from those in the non-shadowed area. In addition, faces have abundant natural facial features that become blurred or missing in the shadows, such as eyes, mouth, nose, and ears. The method must maintain the accuracy and naturalness of these facial features during the shadow removal process to prevent deformation and unnatural appearance.

[0004] Natural image methods do not take into account these unique features of the human face, and they often suffer from detail distortion when processing face images. In addition, the skin color and texture of the human face are complex and delicate. These subtle changes may be difficult to accurately process in natural image methods, resulting in inconsistent skin color and blurred texture. Recently, learning-based methods have shown good performance. However, these methods have a weak correlation with facial features. They fuse the prior features more evenly and do not make full use of the relevant features of the face, which limits their effectiveness in guiding the image reconstruction process and leads to unsatisfactory results. Moreover, these methods mainly focus on processing in the spatial domain, and are prone to ignoring some important detail information, resulting in problems such as color distortion. For this reason, the present invention proposes a frequency domain-aware face shadow removal method. Summary of the invention

[0005] The present invention is made to solve the above-mentioned problems, and aims to provide a frequency domain perception face shadow removal method based on skin color and texture guidance, and proposes a network model guided by skin color and texture information to complete the image shadow removal task.

[0006] To achieve the above object, the frequency domain perception face shadow removal method based on skin color and texture guidance provided by the present invention comprises the following steps:

[0007] Step 1, using the face shadow dataset to establish a face shadow image dataset X with portrait and background separated and a face shadow-free image dataset Y with portrait and background separated;

[0008] Step 2, construct a shadow removal network model, which includes a frequency domain image decomposition network and a color texture guided shadow removal network;

[0009] Step 3: Take the shadow image x in the face shadow image dataset X as input to train the de-shadowing network model. After x passes through the frequency domain image decomposition network, the skin color image x is output. 1 and texture image x 2 , x,x 1 ,x 2 After the color texture guided shadow removal network, the final shadow removal result image x is output 3 ;

[0010] Step 4: Build a discriminator network model to identify the authenticity of the generated image.

[0011] Step 5: Eliminate the final shadow result image x 3 And the corresponding shadow-free image y in the shadow-free image dataset Y is used as input, and the final shadow removal result image x is processed by the discriminator network model. 3 To identify;

[0012] Step 6, training and optimizing the shadow removal network model and the discriminator network model through the loss function, and obtaining the optimized shadow removal network model and the discriminator network model;

[0013] Step 7, using the optimized de-shadowing network model and discriminator network model obtained in step 6 to remove shadows from the face shadow image.

[0014] Furthermore, the frequency-domain-aware face shadow removal method based on skin color and texture guidance provided by the present invention also has the following characteristics: wherein, the frequency-domain image decomposition network comprises a frequency-domain encoder, a frequency-domain decomposition module and two frequency-domain decoders; the Fourier sparse attention block of the encoder introduces the input image from the spatial domain into the frequency domain, reduces the computational complexity by introducing sparse attention, and then downsamples the input image by using convolution operations and nonlinear transformation operations to extract image features; the frequency-domain decomposition module decomposes the face shadow image into high-frequency and low-frequency information at the bottleneck layer; the decoder uses the downsampled image feature result as the network input, upsamples and reconstructs the image by deconvolution operations, and obtains the skin color image x 1 and texture image x 2 , and a skip connection is used between the encoder and decoder to connect the feature map of the encoder stage with the corresponding feature map of the decoder stage.

[0015] Furthermore, the frequency domain perception face shadow removal method based on skin color and texture guidance provided by the present invention also has the following characteristics: wherein the color texture guided shadow removal network comprises a frequency domain encoder, a color fusion module, a skin color fusion module and a frequency domain decoder, and the frequency domain encoder converts the face shadow image x and the skin color image x into a frequency domain encoder. 1 And the texture image x 2 As input, the color fusion module combines the features of the face shadow image x with the skin color image x at the bottleneck layer. 1 The skin color fusion module combines the features of the face shadow image x with the texture image x at the upsampling point. 2 The features of are specially fused; the frequency domain decoder upsamples and reconstructs the image through deconvolution operation, and outputs the final shadow removal result image x 3 .

[0016] Furthermore, in the frequency-domain-aware face shadow removal method based on skin color and texture guidance provided by the present invention, it also has the following characteristics: wherein, the discriminator network model is a PatchGAN network, which is composed of 5 convolutional layers with a convolution kernel size of 4×4 and a step size of 2. The output dimensions of the first 4 convolutional layers are 64, 128, 256, and 512, respectively. The last convolutional layer input maps a matrix with a numerical range of 0-1 as output, and the mean of the output matrix is ​​used as the output result of the discriminator network model.

[0017] Furthermore, the frequency domain perception face shadow removal method based on skin color and texture guidance provided by the present invention also has the following characteristics: wherein the formula of the loss function in step 6 is expressed as follows:

[0018] L loss =L frequency +λ 1 L apperance +λ 2 L structure +λ 3 L adv (1)

[0019] In formula (1), L loss is the total loss; L frequency is the frequency domain reconstruction loss; L apperance is the appearance consistency loss; L structure is the structural consistency loss; L adv is the generation adversarial loss; 1 , 2 , 3 are the weight values ​​of each item respectively;

[0020] Frequency domain reconstruction loss Lfrequency The calculation formula is as follows:

[0021]

[0022] In formula (2), I color For facial skin color mapping, I texture For texture mapping, For high-frequency supervision data, It is low-frequency supervision data.

[0023] Appearance consistency loss L apperance The calculation formula is as follows:

[0024] L apperance =||I free -I gt || 1 (3)

[0025] In formula (3), I free The result image of shadow removal; I gt A true shadow-free image.

[0026] Structural consistency loss L structure The calculation formula is as follows:

[0027]

[0028] In formula (4), I free The result image of shadow removal; I gt For real shadow-free images, VGG() is the feature extractor of the pre-trained VGG19 model.

[0029] Generate adversarial loss L adv The calculation formula is as follows:

[0030]

[0031] In formula (5), the generative adversarial loss is used by the discriminator to determine whether the generated result is true or false, D is the discriminator, and I is the shadow image.

[0032] Furthermore, the frequency domain perception face shadow removal method based on skin color and texture guidance provided by the present invention also has the following characteristics: wherein, the Laplace filter is used to generate the texture information of the face, and the Laplace operator is actually the sum of the second-order derivatives of the original face shadow image in the x and y directions respectively. The original face shadow image needs to be convolved with the Laplace filter in each channel to obtain the face texture map, and the formula is as follows:

[0033]

[0034]

[0035] In formulas (6) and (7), x and y represent the x-axis and y-axis of the original face shadow image, respectively.

[0036] Furthermore, the frequency domain perception face shadow removal method based on skin color and texture guidance provided by the present invention also has the following characteristics: wherein, during the network training process, the weight values ​​of each item in formula (1) are taken as λ 1 =5,λ 2 =1,λ 3 =0.01.

[0037] Furthermore, the frequency-domain-aware face shadow removal method based on skin color and texture guidance provided by the present invention also has the following characteristics: during the network training process, the back-propagation algorithm is used to update the parameters of the de-shadowing network model and the parameters of the discriminator network model. After each iteration, the parameters of the discriminator network model are updated once, and then the parameters of the de-shadowing network model are updated again, and the model test score is calculated at the same time. The de-shadowing network model with the highest final score is used as the optimized de-shadowing network model.

[0038] Compared with the prior art, the present invention has the following beneficial effects:

[0039] The frequency-domain-aware face shadow removal method based on skin color and texture guidance of the present invention constructs a network model guided by skin color and texture information. First, the image is decomposed into high-frequency and low-frequency parts through a frequency-domain image decomposition network to generate a skin color map and a texture map that are independent of shadows. In particular, we introduce a Fourier sparse attention block to convert the image from the spatial domain to the frequency domain. Through sparse feature selection, our Fourier sparse attention block can help the network reduce computational complexity while maintaining attention to key information. Then, the color-texture-guided shadow removal network uses the skin color and texture information of the face to help restore the illumination of the shadow area, designs a skin color fusion module, fuses image features with color features to obtain global color feature information, and designs a texture fusion module to obtain texture information. This module can help the network better maintain the local consistency of the image.

[0040] In addition, after training and optimization of the loss function, the final de-shadowing network model and discriminator network model can process the face shadow image more naturally and realistically, and maintain a high degree of consistency with the shadow-free image. BRIEF DESCRIPTION OF THE DRAWINGS

[0041] Figure 1 is a model training flow chart of a frequency domain perception face shadow removal method based on skin color and texture guidance in an embodiment of the present invention;

[0042] Figure 2 It is a schematic diagram of images before and after face shadow image processing in the frequency domain perception face shadow removal method guided by skin color and texture in an embodiment of the present invention, wherein (a) is the image before processing and (b) is the image after processing. DETAILED DESCRIPTION

[0043] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0044] The present invention provides a frequency domain perception method for removing face shadows, which comprises the following steps:

[0045] Step 1: Use the existing face shadow dataset, and establish a face shadow image dataset X in which the portrait is separated from the background, and a corresponding face shadow image dataset Y in which the portrait is separated from the background.

[0046] Step 2: Build a frequency domain aware de-shadowing network model guided by skin color and texture. The de-shadowing network model includes a frequency domain image decomposition network and a color texture guided shadow removal network. The frequency domain image decomposition network is used to generate a skin color image x 1 and texture image x 2 The color texture guided shadow removal network is used to guide the skin color and texture of the face shadow image x locally to obtain the final shadow removal result image x 3 .

[0047] 2.1) The frequency domain image decomposition network consists of a frequency domain encoder, a frequency domain decomposition module and two frequency domain decoders.

[0048] The Fourier sparse attention block of the encoder introduces the input image from the spatial domain to the frequency domain, reduces the computational complexity by introducing sparse attention, and then uses convolution operations and nonlinear transformation operations to downsample the input image and extract image features; the frequency domain decomposition module decomposes the face shadow image into high-frequency and low-frequency information at the bottleneck layer; the decoder uses the downsampled image feature results as the network input, upsamples and reconstructs the image through deconvolution operations, and obtains the skin color image x 1 and texture image x 2 , and a skip connection is used between the encoder and decoder to connect the feature map of the encoder stage with the corresponding feature map of the decoder stage.

[0049] 2.1.1) The frequency domain image decomposition network, the frequency domain encoder expands the number of channels through a 3×3 convolution layer, and applies three Fourier sparse attention blocks + downsampling to extract image features. In particular, the Fourier sparse attention block converts the input feature F 1 Normalize to get feature F 2 Then, a 1×1 convolution is used to adjust the number of feature channels, and a 3×3 convolution is used to extract the spatial context feature F 3 . The feature F 3 Split into small pieces and reconstruct them to get query Q, key K and value V respectively. We perform fast Fourier transform (FFT) on Q and K to get F Q and F K We are interested in F Q and F K Perform element-by-element multiplication to obtain an attention matrix F spacial . Next, the fast inverse Fourier transform (IFFT) is used to obtain the frequency-domain-based attention matrix A. In order to reduce the amount of computation, the Top-k sparse matrix is ​​used to retain the important components of attention and remove useless information. Here, k is an adjustable parameter that dynamically controls the sparsity. We can replace the indicators that are less than k with a sparse matrix. We use the scatter operation to fill the mask with the pixel values ​​in the matrix that meet the conditions. Next, we apply softmax to normalize the sparse matrix and obtain the attention weight F W We have V and F W Perform element-by-element multiplication to obtain the sparse context feature F 4 Finally, after the reshaping operation, the final sparse attention feature F is obtained using 1×1 convolution. sparse .

[0050] 2.1.2) The frequency domain decomposition module uses the LayerNorm layer to normalize the input feature D and enhance the features. Then, the Fourier sparse attention block is used to filter out unnecessary features and noise. The obtained features are transformed from the spatial domain to the frequency domain using the fast Fourier transform (FFT). Next, a low-pass filter is used to extract low-frequency features, and a high-pass filter is used to extract high-frequency features. The extracted low-frequency and high-frequency features are then converted to the spatial domain using the inverse fast Fourier transform (IFFT). Finally, a 3×3 convolution layer and a Relu activation function are used to further enhance the feature expression.

[0051] 2.1.3) The frequency domain decoder contains a Fourier sparse attention block + upsampling, two layers of Fourier sparse attention block + Fourier sparse attention block + upsampling and one layer of Fourier sparse attention block + Fourier sparse attention block + convolution block. The convolution block uses group normalization, swish activation, Dropout and convolution to restore the number of feature channels and ensure the consistency of input and output channels.

[0052] 2.2) The color texture guided shadow removal network includes a frequency domain encoder, a color fusion module and a skin color fusion module.

[0053] 2.2.1) The color fusion module accurately adjusts the color of the shadow area based on the color information of the area around the face, making the generated image appear more natural and realistic.

[0054] The color fusion module uses BatchNorm to normalize the color features F color and image features F image Then, 1×1 convolution, ReLU activation function and 3×3 depth convolution are used to obtain the structural feature representation of two normalized features, denoted as S color and S image Next, for S color Perform convolution to obtain feature S 1 and S 2 , similarly, in S image Perform convolution on it to obtain feature S 3 and S 4 , and then use the AdaIN module to transform the feature S 1 The color is transferred to S 3 , the feature S 2 The color is transferred to S 4 , we get the transferred feature S transfer1 and S transfer2 , then, in S transfer1 and S 3 Perform element-wise multiplication on the feature S 5 , in S transfer2 and S 4 Perform element-wise multiplication on the feature S 6 . S 5 and S 6 The expressive power is further enhanced through 3×3 convolution and Swish activation function. Finally, the two enhanced features are integrated using a series operation, and the original number of channels is restored using a 1×1 convolution layer to output the final color fusion feature F. color In order to further enhance the feature representation ability of the network, we merge the color feature F colorSpecifically integrated in the bottom layer of the color texture fusion decoder, by introducing color features at this layer, it can more effectively capture global features related to skin color and significantly improve its ability to recover from color changes and potential noise. This design not only improves the accuracy of the model in skin color processing, but also improves the overall quality and naturalness of the results.

[0055] 2.2.2) Texture fusion module combines image features and texture features more perfectly. Effective texture features can help the network restore texture information more accurately and remove shadows in the image.

[0056] The texture fusion module uses LayerNormalization (LayerNorm) to transform the texture feature F texture and image features F image Normalization is performed to ensure stability during data propagation. Subsequently, we use PatchEmbedding to split the two features into multiple small patches and convert them into low-dimensional vector representations, which captures local information while reducing computational costs. We input these two sets of feature vectors into a channel attention unit and obtain their channel attention weights, which are A and texture and A image The channel attention unit consists of global average pooling, 1×1 convolution, Relu activation function and 1×1 convolution. A is converted to texture and A image Convert to probability distribution P texture and P image . image and A texture Perform element-by-element multiplication to obtain the fusion feature F fuse1 . texture and A image Perform the same operation to obtain feature F fuse2 Then, in F fuse1 and F fuse2 3×3 convolution and Relu activation function are used to extract spatial features and capture local information in the image. Finally, the two spatial features are concatenated and 1×1 convolution is used to restore the original number of channels to output the final texture fusion feature. This fusion feature contains rich information from the input image and texture, emphasizing the key channel and spatial information of the shadow removal task. In order to better capture the details and structure of the image, we introduce texture features in the later layers of the decoder. These features enrich the structure and detail information of the image, enrich the feature representation of the decoder, and help the network better maintain the local consistency of the image.

[0057] Step 3: Take the shadow image x in the face shadow image dataset X (the above 3412 shadow images are used in this embodiment) as input to train the de-shadowing network model. After x passes through the frequency domain image decomposition network, the skin color image x is output. 1 and texture image x 2 , x,x 1 ,x 2 After the color texture guided shadow removal network, the final shadow removal result image x is output 3 .

[0058] Step 4: Build a discriminator network model.

[0059] In this embodiment, the discriminator network model is a PatchGAN network, which consists of 5 convolutional layers with a convolution kernel size of 4×4 and a step size of 2. The output dimensions of the first 4 convolutional layers are 64, 128, 256, and 512, respectively. The last convolutional layer inputs a mapping and outputs a matrix with a value range of 0-1, and the mean of the output matrix is ​​used as the output result of the discriminator network model.

[0060] Step 5: 3412 final shadow removal result images x 3 And the corresponding shadow-free image y in the shadow-free image dataset Y (the above 3412 shadow-free images are used in this embodiment) is used as input, and the final shadow removal result image x is processed by the discriminator network model. 3 To identify.

[0061] Step 6: Train and optimize the shadow removal network model and the discriminator network model through the loss function, and obtain the optimized shadow removal network model and the discriminator network model.

[0062] Among them, the formula of the loss function in step 6 is as follows:

[0063] L loss =L frequency +λ 1 L apperance +λ 2 L structure +λ 3 L adv (1)

[0064] In formula (1), L loss is the total loss; L frequency is the frequency domain reconstruction loss; L apperance is the appearance consistency loss; L structure is the structural consistency loss; L adv is the generation adversarial loss; 1 , 2 , 3are the weight values ​​of each item respectively;

[0065] Frequency domain reconstruction loss L frequency The calculation formula is as follows:

[0066]

[0067] In formula (2), I color For facial skin color mapping, I texture For texture mapping, For high-frequency supervision data, It is low-frequency supervision data.

[0068] Appearance consistency loss L apperance The calculation formula is as follows:

[0069] L apperance =||I free -I gt || 1 (3)

[0070] In formula (3), I free The result image of shadow removal; I gt A true shadow-free image.

[0071] Structural consistency loss L structure The calculation formula is as follows:

[0072]

[0073] In formula (4), I free The result image of shadow removal; I gt For real shadow-free images, VGG() is the feature extractor of the pre-trained VGG19 model.

[0074] Generate adversarial loss L adv The calculation formula is as follows:

[0075]

[0076] In formula (5), the generative adversarial loss is used by the discriminator to determine whether the generated result is true or false, D is the discriminator, and I is the shadow image.

[0077] The Laplace filter is used to generate the texture information of the face. The face shadow image needs to be convolved with the Laplace filter in each channel to obtain the face texture map. The formula is as follows:

[0078]

[0079] In formulas (6) and (7), x and y represent the x-axis and y-axis of the original face shadow image, respectively.

[0080] During the network training process, the weight values ​​of each item in formula (1) are taken as λ 1 =5,λ 2 =1,λ 3 =0.01. In addition, the back propagation algorithm is used to update the parameters of the de-shadowing network model and the parameters of the discriminator network model during the training process. After each iteration, the parameters of the discriminator network model are updated once, and then the parameters of the de-shadowing network model are updated again. At the same time, after each iteration, the test scores of the two models are calculated through the test set (the above-mentioned 964 sets of image data). The discriminator network model with the highest final score is used as the optimized discriminator network model, and the de-shadowing network model with the highest final score is used as the optimized de-shadowing network model.

[0081] Step 7: Use the optimized de-shadowing network model and discriminator network model obtained in step 6 to remove shadows from the face shadow image.

[0082] The above results show that: using the optimized shadow removal network model in this embodiment to process the shadow image, Figure 2 A specific example is shown below, where (a) is the image before processing and (b) is the image after processing. Figure 2 As shown, the skin color and texture guided frequency domain perception face shadow removal method of the present invention is used for image processing, and the processed image appears to be shadow-free, and compared with the image before processing, the shadow-free image appears to be real and natural, and the processing effect is better than the shadow-free image processing effect of the prior art.

[0083] It should be noted that, in this article, relational terms such as first and second, etc. are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the terms "include", "comprise" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device.

[0084] Although embodiments of the present invention have been shown and described, it will be appreciated by those skilled in the art that various changes, modifications, substitutions and variations may be made to the embodiments without departing from the principles and spirit of the present invention, and that the scope of the present invention is defined by the appended claims and their equivalents.

Claims

1. A frequency domain perception method for removing face shadows, characterized in that: The method comprises the following steps: Step 1, using the face shadow dataset to establish a face shadow image dataset X with portrait and background separated and a face shadow-free image dataset Y with portrait and background separated; Step 2, construct a shadow removal network model, which includes a frequency domain image decomposition network and a color texture guided shadow removal network; Step 3, take the shadow image x in the face shadow image dataset X as input to train the shadow removal network model, after x passes through the frequency domain image decomposition network, it outputs the skin color image x1 and the texture image x2, after x, x1, x2 pass through the color texture guided shadow removal network, it outputs the final shadow removal result image x3; Step 4: Build a discriminator network model to identify the authenticity of the generated image. Step 5, taking the final shadow removal result image x3 and the corresponding shadow-free image y in the shadow-free image dataset Y as input, and discriminating the final shadow removal result image x3 through the discriminator network model; Step 6, training and optimizing the shadow removal network model and the discriminator network model through the loss function, and obtaining the optimized shadow removal network model and the discriminator network model; Step 7, using the optimized de-shadowing network model and discriminator network model obtained in step 6 to remove shadows from the face shadow image.

2. The frequency domain perception face shadow removal method according to claim 1, characterized in that: The frequency domain image decomposition network includes a frequency domain encoder, a frequency domain decomposition module and two frequency domain decoders. The Fourier sparse attention block of the encoder introduces the input image from the spatial domain to the frequency domain, reduces the computational complexity by introducing sparse attention, and then uses convolution operations and nonlinear transformation operations to downsample the input image and extract image features. The frequency domain decomposition module decomposes the face shadow image into high-frequency and low-frequency information at the bottleneck layer; The decoder takes the downsampled image feature results as the network input, upsamples and reconstructs the image through deconvolution operations, and obtains the skin color image x1 and the texture image x2. A jump connection is used between the encoder and the decoder to connect the feature map of the encoder stage with the corresponding feature map of the decoder stage.

3. The frequency domain perception face shadow removal method according to claim 1, characterized in that: The color texture guided shadow removal network comprises a frequency domain encoder, a color fusion module, a skin color fusion module and a frequency domain decoder, wherein the frequency domain encoder takes the face shadow image x, the skin color image x1 and the texture image x2 as input to extract their feature information; The color fusion module specifically fuses the features of the face shadow image x with the features of the skin color image x1 at the bottleneck layer; the skin color fusion module specifically fuses the features of the face shadow image x with the features of the texture image x2 at the upsampling point; the frequency domain decoder upsamples and reconstructs the image through deconvolution operations, and outputs the final shadow removal result image x3.

4. The frequency domain perception face shadow removal method according to claim 1, characterized in that: The discriminator network model is a PatchGAN network, which consists of 5 convolutional layers with a convolution kernel size of 4×4 and a stride of 2. The output dimensions of the first 4 convolutional layers are 64, 128, 256, and 512, respectively. The last convolutional layer inputs a mapping and outputs a matrix with a value range of 0-1, and the mean of the output matrix is ​​used as the output result of the discriminator network model.

5. The frequency domain perception face shadow removal method according to claim 1, characterized in that: The formula of the loss function in step 6 is as follows: L loss =L frequency +λ1L apperance +λ2L structure +λ3L adv (1) In formula (1), L loss is the total loss; L frequency is the frequency domain reconstruction loss; l apperance is the appearance consistency loss; l structure is the structural consistency loss; l adv is the generated adversarial loss; λ1, λ2, λ3 are the weight values ​​of each item respectively; Frequency domain reconstruction loss frequency The calculation formula is as follows: In formula (2), I color For facial skin color mapping, I texture For texture mapping, For high-frequency supervision data, It is low-frequency supervision data. Appearance consistency loss L apperance The calculation formula is as follows: L apperance =||I free -I gt ||1 (3) In formula (3), I free The result image of shadow removal; I gt A true shadow-free image. Structural consistency loss L structure The calculation formula is as follows: In formula (4), I free The result image of shadow removal; I gt For real shadow-free images, VGG() is the feature extractor of the pre-trained VGG19 model. Generate adversarial loss L adv The calculation formula is as follows: In formula (5), the generative adversarial loss is used by the discriminator to determine whether the generated result is true or false, D is the discriminator, and I is the shadow image.

6. The frequency domain perception face shadow removal method according to claim 1, characterized in that: The Laplace filter is used to generate the texture information of the face. The Laplace operator is actually the sum of the second-order derivatives of the original face shadow image in the x and y directions. The original face shadow image needs to be convolved with the Laplace filter in each channel to obtain the face texture map. The formula is as follows: In formulas (6) and (7), x and y represent the x-axis and y-axis of the original face shadow image, respectively.

7. The frequency domain perception face shadow removal method according to claim 1, characterized in that: In the network training process, the weight values ​​of each item in formula (1) are λ1=5, λ2=1, and λ3=0.

01.

8. The frequency domain perception face shadow removal method according to claim 1, characterized in that: During the network training process, a back-propagation algorithm is used to update the parameters of the de-shadowing network model and the parameters of the discriminator network model. After each iteration, the parameters of the discriminator network model are updated once, and then the parameters of the de-shadowing network model are updated again. At the same time, the model test score is calculated, and the de-shadowing network model with the highest final score is used as the optimized de-shadowing network model.

Images