Face sketch image generation system and method based on deep bidirectional learning

Abstract

The present application provides a kind of face sketch image generation system based on depth bidirectional learning, comprising: data acquisition and preprocessing module, the data acquisition and preprocessing module gathers training data and carries out pre-processing;Generation network module, the generation network generates sketch image or photo;Adversarial discrimination module, the adversarial discrimination module discriminates whether the object is the generation network generation result;Feedback updating module, the feedback updating module defines loss function, updates the generation network module and the adversarial discrimination module parameter;Training module, the training module trains the generation network module and the adversarial discrimination module;Application module, the application module uses the model of training to realize face sketch image generation.The present application utilizes depth bidirectional learning to carry out internal and external neuron information sharing transmission, and only needs a small amount of data set to train to automatically generate sketch style image with detail performance.

Description

Technical Field

[0001] This invention relates to a method in the fields of deep learning and image processing, specifically to a deep bidirectional learning framework for reconstructing self-organizing networks based on least squares error, a face sketching image generation system and method based on the duality of neurons. Background Technology

[0002] Facial sketching image generation refers to the automatic conversion of facial photographs into corresponding sketch images. Facial sketching images can be applied to criminal investigation and law enforcement, automatically converting the identity photos of criminal suspects into sketch images, which can then be compared with sketch images drawn based on eyewitness descriptions to narrow down the suspect pool. It can also be applied in the digital entertainment field, such as automatically generating sketch-style user avatars, and has significant application value.

[0003] Early researchers synthesized facial sketches by searching for similarity between slices of facial photographs and slices of sketched images and by linear combination. For example, Tang and Wang used a feature transformation-based method to map test photos into the feature space of the training image set through principal component analysis, and then used this mapping to perform a weighted linear combination of the training sketch set to generate the final sketch image. However, the similarity search process was very time-consuming, and the synthesized samples were relatively smooth, lacking sufficient sketch texture compared to sketches drawn by real artists. These methods could not meet the requirements for generating human sketches.

[0004] With the rapid development of deep neural network technology, especially the widespread application of Convolutional Neural Networks (CNNs) and Generative Adversarial Networks (GANs) in image generation, significant progress has been made in generating face sketch images. However, models that directly map face photos to sketch images using CNNs often fail to achieve sufficient semantic and perceptual similarity to real samples, and some methods still produce images with numerous blemishes. While GANs generate relatively high-quality sketch images, they require substantial training data to achieve satisfactory performance. Some studies have introduced additional auxiliary mechanisms and modules to enhance network training, improving model generation capabilities even with limited sample sizes. However, these auxiliary modules themselves require pre-training on large datasets. Limited by the differences in data patterns and distributions between the original dataset and the auxiliary datasets, they may no longer have a positive impact on the model after the application scenario changes. Furthermore, the high cost of collecting these large auxiliary datasets severely limits their application value.

[0005] A search revealed a Chinese invention patent publication number CN111667007A, which discloses a method for generating face pencil drawing images based on generative adversarial networks (GANs). This method generates face pencil drawing images through image processing, GAN model construction, model training, and finally, face pencil drawing images. Its shortcomings are that the GAN network used in the aforementioned patent has a relatively simple network structure, resulting in poor visual quality of the generated pencil drawings, with issues such as blurriness and excessive noise. Furthermore, it requires a large-scale dataset for training. Summary of the Invention

[0006] To address the shortcomings of existing technologies, the purpose of this invention is to provide a face sketching image generation system and method based on deep bidirectional learning.

[0007] According to one aspect of the present invention, a face sketch image generation system based on deep bidirectional learning is provided, comprising: a data acquisition and preprocessing module, wherein the data acquisition and preprocessing module acquires training data and performs preprocessing;

[0008] A generative network module, wherein the generative network generates a sketch image or photograph;

[0009] An adversarial discrimination module is provided, which determines whether the object is the result generated by the generator network.

[0010] A feedback update module, which defines a loss function and updates the parameters of the generator network module and the adversarial discriminant module;

[0011] The training module trains the generative network module and the adversarial discriminant module;

[0012] The application module uses a trained model to generate facial sketch images.

[0013] Preferably, the data collected for training the network includes: the Chinese University of Hong Kong Facial Sketch Database (CUFS dataset) and the Chinese University of Hong Kong Sketch FERET Database (CUFSF dataset).

[0014] The data preprocessing includes: geometric correction based on the three points between the eyes and lips of the face in the data, data cropping, data filling, and random data cropping.

[0015] Preferably, the generating network module includes:

[0016] Photo-to-Sketch Image Generation Network Module: A deep bidirectional learning network framework based on least squares error reconstruction self-organizing network (Lmser) is established to build a photo-to-sketch image generation network;

[0017] Sketch image to photograph generation network module: A sketch image to photograph generation network is established based on the Lmser deep bidirectional learning network framework;

[0018] Feature transfer module: This includes an internal feature transfer module that performs feature transfer within the generator network and an external feature transfer module that performs feature transfer between two generator networks.

[0019] Preferably, the photo-to-sketch image generation network module includes a first neural network G-Lmser, the G-Lmser network comprising:

[0020] An encoder that maps an input face photograph to a latent space code;

[0021] A decoder that maps the latent space code to a sketch image;

[0022] The encoder includes five residual modules (ResDown blocks), each residual module including one direct mapping part of a convolution operation and two residual parts of convolution operations;

[0023] The decoder includes 5 residual modules (ResUp blocks), each residual module including one direct mapping part of a convolution operation and two residual parts of convolution operations.

[0024] Preferably, the sketch image to photo generation network module includes a second neural network F-Lmser network, the F-Lmser network comprising:

[0025] An encoder that maps an input sketch image to a latent space code;

[0026] A decoder that maps latent space codes to a face photograph;

[0027] The encoder includes five residual modules (ResDown blocks), each residual module including one direct mapping part of a convolution operation and two residual parts of convolution operations;

[0028] The decoder includes 5 residual modules (ResUp blocks), each residual module including one direct mapping part of a convolution operation and two residual parts of convolution operations.

[0029] Preferably, the feature transfer module includes:

[0030] Internal feature transfer: The internal feature transfer is implemented in the two internal Lmser networks (G-Lmser and F-Lmser) through the neuron duality (internal DPN). The internal DPN is achieved through skip connections on the shrinking path of the feature map to its corresponding expansion path. The skip connections directly transfer the feature information obtained by the encoder part during the feature extraction process of the original data to the decoder.

[0031] External feature transfer: The external feature transfer utilizes the duality of neurons (external DPN). In the G-Lmser and F-Lmser networks, the F-Lmser network provides regularization constraints for the G-Lmser network, which is achieved through feature consistency constraints between the corresponding network blocks of the two.

[0032] Preferably, the adversarial discriminator module establishes two adversarial discriminators, namely:

[0033] First adversarial discriminant (D) Y This is used to distinguish whether a sketch image is generated by a generative network or is a genuine sketch drawn by an artist. It adopts the PatchGAN structure, and the network includes 5 convolutional layers.

[0034] Second adversarial discriminant (D) X This is used to distinguish whether a face photo is generated by a generative network or a real photo. It adopts the PatchGAN structure, and the network includes 5 convolutional layers.

[0035] Preferably, the feedback update module includes:

[0036] Define a loss function, use F-Lmser as a perceptual network to extract high-level features from y and G(x), and use consistency loss to make the corresponding features of the two converge; where y is the real sketch image corresponding to the face photo x, and G(x) is the generated sketch image of the face photo x obtained by the generative network G.

[0037] The perceptual loss and the consistency loss of the output layer are fused into a single loss function;

[0038] In G-Lmser and F-Lmser, least squares adversarial loss is used.

[0039] Preferably, the training module includes:

[0040] Alternately use gradient descent to train D X and D Y ;

[0041] Train the first neural network G and the second neural network F.

[0042] The application module uses a trained generative network to encode the input face photo through an encoder, mapping it to a latent space, and then a decoder maps the latent space encoding to a sketch image, outputting a face sketch image.

[0043] According to a second aspect of the present invention, a method for generating human face sketch images based on deep bidirectional learning is provided, comprising:

[0044] Collect data and preprocess it;

[0045] A deep bidirectional learning network framework based on least squares error reconstruction self-organizing network (Lmser) is adopted to establish the first neural network G, realizing the mapping from photograph to sketch image;

[0046] A deep bidirectional learning network framework based on least squares error reconstruction self-organizing network (Lmser) is adopted to establish a second neural network F to realize the mapping from sketch image to photograph;

[0047] An internal feature transfer mechanism based on the duality of neurons is adopted, in which the feature information obtained during the feature extraction process of the original data is directly transferred to the decoder in the first neural network G and the second neural network F;

[0048] An external feature transfer mechanism based on the duality of neurons is adopted to provide a regularization constraint for the first neural network G by the second neural network F.

[0049] A discriminator with a PatchGAN structure is used to build two adversarial discriminators: one to distinguish whether a sketch image is generated by the generative network or a real sketch image drawn by an artist, and the other to distinguish whether a face photo is generated by the generative network or a real photo.

[0050] A feedback update mechanism is used to train the network;

[0051] The Adam optimizer is used to train two adversarial discriminators by alternating gradient descent, and then the first neural network G and the second neural network F are trained to complete the network training.

[0052] Compared with the prior art, the present invention has the following beneficial effects:

[0053] This invention discloses a face sketching image generation system and method based on deep bidirectional learning. It utilizes deep bidirectional learning to share and transmit information between internal and external neurons, and can automatically generate sketch-style images with detailed representations with only a small dataset for training.

[0054] In a preferred embodiment of the present invention, the photo-to-sketch image generation network module uses an Lmser network containing local skip connections as the generation network, and utilizes DPN to forward pass multi-level feature information learned from different network layers to enhance the texture details of the generated sketch.

[0055] In a preferred embodiment of the present invention, the sketch image to photo generation network module uses an Lmser network containing local skip connections. In addition, the module also provides a regularization constraint for the photo to sketch image generation network, which can improve its deep representation learning ability.

[0056] In a preferred embodiment of the present invention, a feature transfer module is provided. The feature transfer includes an internal feature transfer module and an external feature transfer module. The internal feature transfer module is implemented in the form of local skip connections, forward transferring multi-level feature information learned from different network layers. The external feature transfer module uses the F-Lmser network to send the information back to the G-Lmser network, so that the whole model has the characteristics of a recurrent neural network, which is good at capturing the sequential features of the sketch lines, and prompts the model to generate a sketch image that is close to that drawn by the artist.

[0057] The adversarial discriminator module in a preferred embodiment of the present invention is used to distinguish whether the input sample is a distribution of generated samples or a distribution of real samples. During the training process, the discriminator improves its ability to distinguish between real and fake samples to incentivize the generation ability of the generator network. The present invention adopts the PatchGan structure, which divides the input image into small blocks of 70*70 for discrimination, which helps to improve the discriminator's discrimination ability, thereby incentivizing the generator network to generate a facial sketch image that is closer to the sample. Attached Figure Description

[0058] Other features, objects, and advantages of the present invention will become more apparent from the following detailed description of non-limiting embodiments with reference to the accompanying drawings:

[0059] Figure 1 This is a diagram of the Lmser-in-Lmser network structure in a face sketching image generation system and method based on deep bidirectional learning, as described in an embodiment of the present invention. Detailed Implementation

[0060] The present invention will now be described in detail with reference to specific embodiments. These embodiments will help those skilled in the art to further understand the present invention, but do not limit the invention in any way. It should be noted that those skilled in the art can make various modifications and improvements without departing from the concept of the present invention. These all fall within the scope of protection of the present invention.

[0061] like Figure 1The diagram shown is a structural diagram of a face sketching image generation system based on deep bidirectional learning according to an embodiment of the present invention, comprising:

[0062] Data acquisition and preprocessing module: Acquires the data required for training the network and preprocesses the data;

[0063] Photo-to-Sketch Image Generation Network Module: A deep bidirectional learning network framework based on least squares error reconstruction self-organizing network (Lmser) is established to generate a photo-to-sketch image network.

[0064] Sketch Image to Photograph Generation Network Module: A network for generating sketch images to photographs is established based on the Lmser deep bidirectional learning network framework;

[0065] Feature transfer module: includes an internal feature transfer module that performs feature transfer within the generator network and an external feature transfer module that performs feature transfer between two generator networks;

[0066] Feedback update module;

[0067] Adversarial Discriminator Module: Establishes two adversarial discriminators, the first adversarial discriminator (D Y The second adversarial discriminant (D) is used to distinguish whether a sketch image is generated by a generative network or is a genuine sketch drawn by an artist. X This is used to distinguish whether a face photo is generated by a web-generated network or is a real photograph.

[0068] Training module: Trains the network;

[0069] Use a trained model to generate human face sketch images.

[0070] In a preferred embodiment of the present invention, the data acquisition and preprocessing module includes: acquiring the data required for training the network, and acquiring the Chinese University of Hong Kong Facial Sketch Database (CUFS dataset) and the Chinese University of Hong Kong Sketch FERET Database (CUFSF dataset);

[0071] The data acquisition and preprocessing module includes: geometric correction based on three points between the eyes and lips of the face in the data, data cropping, data filling, and random cropping of the data.

[0072] Furthermore, the image is first cropped to a size of 250×200, and then the image is wrapped and filled to a size of 286×286. To avoid blemishes in the generated image, the filling pixel value in this embodiment is 255 (i.e., white). During training, in order to enhance the data, this embodiment randomly crops the photo to a size of 256×256, while during generation, the central 256×256 portion is cropped.

[0073] In a preferred embodiment of the present invention, the photo-to-sketch image generation network module includes: a first neural network G, specifically an Lmser network (G-Lmser network) that implements mapping from the face photo domain to the sketch image domain. The G-Lmser network includes an encoder and a decoder. The encoder maps the input face photo to the latent space encoding, and the decoder maps the latent space encoding to the sketch image.

[0074] Furthermore, the encoder includes 5 residual modules (ResDown blocks), each of which includes a directly mapped residual portion.

[0075] The direct mapping part uses a 1×1 convolutional layer to increase the dimensionality of the feature map, followed by average pooling. The residual part first normalizes the input data using InstanceNorm, then uses LeakyReLU as the activation function, passes it through a 3×3 convolutional layer, and repeats the InstanceNorm normalization, LeakyReLU activation, and 3×3 convolutional layer operation once more before performing average pooling. Finally, it is added to the features from the direct mapping part.

[0076] The decoder consists of five residual modules (ResUp blocks). Each residual module includes a mapping part and a residual part. The direct mapping part is downscaled by an upsampling operation followed by a 1×1 convolutional layer. The residual part first normalizes the input data using InstanceNorm, then uses ReLU as the activation function, performs an upsampling operation, passes it through a 3×3 convolutional layer, and repeats the InstanceNorm normalization, ReLU activation, and 3×3 convolutional layer operation once more before adding it to the features of the direct mapping part.

[0077] In a preferred embodiment of the present invention, the sketch image to photo generation network module includes: a second neural network F, specifically an Lmser network (F-Lmser network) that implements the mapping from the sketch image domain to the face photo domain. The F-Lmser network includes an encoder and a decoder. The encoder maps the input sketch image to a latent space code, and the decoder maps the latent space code to a face photo. The encoder includes five residual modules (ResDown blocks), each residual module including a direct mapping part of one convolution operation and a residual part of two convolution operations. The decoder includes five residual modules (ResUp blocks), each residual module including a direct mapping part of one convolution operation and a residual part of two convolution operations.

[0078] In a preferred embodiment of the present invention, the feature transfer module includes: internal feature transfer, wherein the internal feature transfer operates on two internal Lmser networks (G-Lmser and F-Lmser) through neuronal duality (internal DPN). The internal DPN is implemented through skip connections on the shrinking path of the feature map to its corresponding expanding path. The skip connections directly transfer the feature information obtained by the encoder part during the feature extraction process of the original data to the decoder, providing the decoder with more information. Specifically, the skip connections adopt additive skip connections.

[0079] The feature delivery module includes: external feature delivery. External feature delivery utilizes the duality of neurons (external DPN). In the G-Lmser and F-Lmser networks, the F-Lmser network provides a regularization constraint for the G-Lmser network, implemented through feature consistency constraints between corresponding network blocks. Specifically:

[0080]

[0081] Among them, Ψ i and Φ i These represent the feature maps of the i-th layer in the G-Lmser and F-Lmser networks, respectively. Indicates hidden layer The size of the set.

[0082] In another embodiment of the invention, the feedback update module uses F-Lmser as a perceptual network to extract high-level features from y and G(x), and then uses a consistency loss to make the corresponding features of the two converge to each other.

[0083] This embodiment also fuses the perceptual loss and the consistency loss of the output layer into a single loss function to improve the model's performance, specifically:

[0084]

[0085] Where Φ5 represents the feature map extracted from the intermediate hidden layer of the network.

[0086] In the feedback update module, this embodiment uses least squares adversarial loss in both G-Lmser and F-Lmser, which helps our model to converge better.

[0087] Specifically:

[0088]

[0089] The feedback update module also employs L1 loss to incentivize the "false" outputs to fit the distribution of the "true" samples, specifically:

[0090]

[0091] The complete loss function is as follows:

[0092]

[0093] and

[0094]

[0095] In this embodiment, the weighting factors λ, β, and α in the complete loss function are set to 10, 1, and 5, respectively.

[0096] In other embodiments of the present invention, the adversarial discriminator module includes: a first adversarial discriminator (D Y This algorithm is used to distinguish between sketches generated by a generative network and genuine sketches drawn by an artist. It employs a PatchGAN architecture, with the network consisting of five convolutional layers. The input consists of 70×70 patches, with 4×4 kernels. The last two convolutional layers have a stride of 1, while the others have a stride of 2. Each of the first four convolutional layers is followed by instance regularization and a LeakyReLU of 0.2.

[0097] The adversarial discriminator module includes: a second adversarial discriminator (D X This algorithm is used to distinguish between face photos generated by a generative network and genuine photos. It employs a PatchGAN architecture, with the network consisting of five convolutional layers. Furthermore, the input consists of 70×70 patches, with 4×4 convolutional kernels. The last two convolutional layers have a stride of 1, while the others have a stride of 2. Each of the first four convolutional layers is followed by instance regularization and a Leaky ReLU of 0.2.

[0098] The discrimination result is fed back to the generator network to change the weights, making the image generated by the generator network more similar to the target (i.e., the sketch image).

[0099] In other embodiments of the present invention, the training module includes: alternatingly training D using gradient descent. X and D Y Then, train the first neural network G and the second neural network F. Training uses the Adam optimizer with a batch size of 1, a generator learning rate of 0.0001, and a discriminator learning rate of 0.0002. The training process is as follows:

[0100] Input: A set of training photos and sketch images, in binary form: {1 face photo x, 1 matching sketch image y};

[0101] The maximum number of iterations is T when the number of iterations t = 0.

[0102] Output: Generated Samples Optimal G, F, D X D Y

[0103] When t < T, perform the following operations.

[0104] 1. Randomly select a set of training samples:

[0105] {1 photo of a face x, 1 matching sketch image y};

[0106] 2. Estimate the generation of sketch images and generate human face photos:

[0107]

[0108] 3. Estimate and reconstruct facial images:

[0109]

[0110] 4. Update D X D Y

[0111]

[0112]

[0113] 5. Update G and F

[0114]

[0115]

[0116] End the loop

[0117] Among them, G, F, D X D Y These represent the first neural network G-Lmser, the second neural network F-Lmser, the first adversarial discriminator, and the second adversarial discriminator, respectively.

[0118] Based on the same inventive concept, a method for generating human face sketch images based on deep bidirectional learning is provided, including:

[0119] The data is preprocessed by using three points between the eyes and lips of the face in the data for geometric correction, cropping the data, filling the data, and then performing random cropping operations.

[0120] A deep bidirectional learning network framework based on least squares error reconstruction self-organizing network (Lmser) is adopted to establish the first neural network G, realizing the mapping from photograph to sketch image;

[0121] A deep bidirectional learning network framework based on least squares error reconstruction self-organizing network (Lmser) is adopted to establish a second neural network F to realize the mapping from sketch image to photograph;

[0122] An internal feature transfer mechanism based on the duality of neurons is adopted. In the first neural network G and the second neural network F, the feature information obtained during the feature extraction process of the original data is directly transferred to the decoder, providing the decoder with more information.

[0123] An external feature transfer mechanism based on the duality of neurons is adopted to provide a regularization constraint for the first neural network G by the second neural network F.

[0124] A feedback update mechanism is used to train the network;

[0125] A discriminator with a PatchGAN structure is used to build two adversarial discriminators: one to distinguish whether a sketch image is generated by the generative network or a real sketch image drawn by an artist, and the other to distinguish whether a face photo is generated by the generative network or a real photo.

[0126] The Adam optimizer is used to train two adversarial discriminators by alternating gradient descent, and then the first neural network G and the second neural network F are trained to complete the network training.

[0127] Specific embodiments of the present invention have been described above. It should be understood that the present invention is not limited to the specific embodiments described above, and those skilled in the art can make various modifications or variations within the scope of the claims, which do not affect the essence of the present invention. The above preferred features can be used in any combination without conflict.

[0128] Specific embodiments of the present invention have been described above. It should be understood that the present invention is not limited to the specific embodiments described above, and those skilled in the art can make various modifications or variations within the scope of the claims, which do not affect the essence of the present invention. The above preferred features can be used in any combination without conflict.

Claims

1. A face sketching image generation system based on deep bidirectional learning, comprising: A data acquisition and preprocessing module, wherein the data acquisition and preprocessing module acquires training data and performs preprocessing; A generative network module that generates sketch images or photographs; An adversarial discrimination module is used to determine whether the object being judged is the result generated by the generation network module. A feedback update module, which defines a loss function and updates the parameters of the generator network module and the adversarial discriminant module; The training module trains the generative network module and the adversarial discriminant module; Application module, which uses a trained model to generate facial sketch images; The generated network module comprises three sub-modules: Photo-to-Sketch Image Generation Network Module: A deep bidirectional learning network framework based on the least squares error reconstruction self-organizing network Lmser is established to build a photo-to-sketch image generation network; Sketch image to photograph generation network module: A sketch image to photograph generation network is established based on the Lmser deep bidirectional learning network framework; Feature transfer module: includes an internal feature transfer module that performs feature transfer within the generator network and an external feature transfer module that performs feature transfer between two generator networks; The photo-to-sketch image generation network module includes a first neural network G-Lmser, the G-Lmser network comprising: An encoder that maps an input face photograph to a latent space code; A decoder that maps the latent space code to a sketch image; The encoder includes five residual modules (ResDown blocks), each of which includes one direct mapping part of a convolution operation and two residual parts of convolution operations. The decoder includes 5 residual modules (ResUp blocks), each residual module including one direct mapping part of a convolution operation and two residual parts of convolution operations; The sketch image to photograph generation network module includes a second neural network, the F-Lmser network, which includes: An encoder that maps an input sketch image to a latent space code; A decoder that maps latent space codes to a face photograph; The encoder includes five residual modules (ResDown blocks), each of which includes one direct mapping part of a convolution operation and two residual parts of convolution operations. The decoder includes 5 residual modules (ResUp blocks), each residual module including one direct mapping part of a convolution operation and two residual parts of convolution operations; The feature transfer module includes: Internal feature transfer: The internal feature transfer is implemented through the internal DPN in two internal Lmser networks, namely G-Lmser and F-Lmser. The internal DPN is implemented through skip connections on the shrinking path of the feature map to its corresponding expansion path. The skip connections directly transfer the feature information obtained by the encoder part during the feature extraction process of the original data to the decoder. The internal DPN refers to the duality of neurons within the network; External feature transfer: The external feature transfer utilizes an external DPN. In the G-Lmser and F-Lmser networks, the F-Lmser network provides regularization constraints for the G-Lmser, which is achieved through feature consistency constraints between the corresponding network blocks of the two. The external DPN refers to the neuron duality between networks.

2. The face sketch image generation system based on deep bidirectional learning according to claim 1, wherein, The training data includes: the Chinese University of Hong Kong Facial Sketch Database (CUFS dataset) and the Chinese University of Hong Kong Sketch FERET Database (CUFSF dataset). The preprocessing includes: geometric correction based on the three points between the eyes and lips of the face in the data, data cropping, data filling, and random data cropping.

3. The face sketching image generation system based on deep bidirectional learning according to claim 2, characterized in that, The adversarial discriminator module establishes two adversarial discriminators, namely: First-class adversarial discriminant It is used to distinguish whether a sketch image is generated by a generative network or a real sketch image drawn by an artist. It adopts the PatchGAN structure, and the network includes 5 convolutional layers. Second adversarial discriminant It is used to distinguish whether a face photo is generated by a generative network or a real photo. It adopts the PatchGAN structure, and the network includes 5 convolutional layers.

4. The face sketching image generation system based on deep bidirectional learning according to claim 1, characterized in that, The feedback update module includes: Define the loss function, use F-Lmser as the receptive network, and from and High-level features are extracted from the image, and consistency loss is used to make the corresponding features of the two images converge; where y is the face photo. The corresponding real sketch image, Photographs of faces The generated sketch image obtained through the generative network G; The perceptual loss and the consistency loss of the output layer are fused into a single loss function; In G-Lmser and F-Lmser, least squares adversarial loss is used.

5. A face sketching image generation system based on deep bidirectional learning according to claim 4, characterized in that, The training module includes: Alternately use gradient descent for training and ; Train the first neural network G and the second neural network F; The application module uses a trained generative network to encode the input face photo through an encoder, mapping it to a latent space, and then a decoder maps the latent space encoding to a sketch image, outputting a face sketch image.

6. A method for generating face sketch images based on deep bidirectional learning, employing the face sketch image generation system based on deep bidirectional learning as described in claim 1, characterized in that, include: Collect data and preprocess it; A deep bidirectional learning network framework based on least squares error reconstruction self-organizing network (Lmser) is adopted to establish the first neural network G, realizing the mapping from photograph to sketch image; A deep bidirectional learning network framework based on least squares error reconstruction self-organizing network (Lmser) is adopted to build a second neural network F to realize the mapping from sketch image to photograph; An internal feature transfer mechanism based on the duality of neurons is adopted, in which the feature information obtained during the feature extraction process of the original data is directly transferred to the decoder in the first neural network G and the second neural network F; An external feature transfer mechanism based on the duality of neurons is adopted to provide a regularization constraint for the first neural network G by the second neural network F. A discriminator with a PatchGAN structure is used to build two adversarial discriminators: one to distinguish whether a sketch image is generated by a generative network or a real sketch image drawn by an artist, and the other to distinguish whether a face photo is generated by a generative network or a real photo. A feedback update mechanism is used to train the network; The two discriminators are trained by using the gradient descent method alternately, and then the first neural network G and the second neural network F are trained by using the Adam optimizer, so that the training of the network is completed.

Images