Robust automatic face fusion method

An automatic face and fusion method technology, applied in the field of image synthesis, can solve the problems of poor fusion of image features, not natural enough, and no perfect solution to the occlusion problem, achieving the effect of wide applicability and expanding boundaries

Active Publication Date: 2019-12-31
UNIV OF ELECTRONICS SCI & TECH OF CHINA
View PDF6 Cites 9 Cited by
  • Summary
  • Abstract
  • Description
  • Claims
  • Application Information

AI Technical Summary

Problems solved by technology

The earlier face fusion method is generally to cut out the face in one image and paste it on another face, and then perform color correction. As a result of this processing, the final image feature fusion degree is poor and it is not natural enough.
More recent face fusion methods are mostly implemented using deep learning generative models, which can deal with the expression and lighting inconsistencies between different faces, but there is still no perfect solution to the occlusion problem, especially based on deep learning methods with good results

Method used

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
View more

Image

Smart Image Click on the blue labels to locate them in the text.
Viewing Examples
Smart Image
  • Robust automatic face fusion method
  • Robust automatic face fusion method
  • Robust automatic face fusion method

Examples

Experimental program
Comparison scheme
Effect test

Embodiment 1

[0029] Please refer to figure 1 and figure 2 , the embodiment of the present invention provides a robust automatic face fusion method, comprising the following steps:

[0030] S1. Obtain two face images, which are face image A and face image B respectively;

[0031] S2. Perform occlusion processing on the face image A and the face image B respectively to obtain the four-channel image A and the four-channel image B. The four-channel image A contains the identity features in the composite image, and a non-occlusion mask is added relative to the image A. As a feature of a channel, the four-channel image B contains the attribute (non-identity) features in the composite image. Compared with the image B, a non-occlusion mask is added as a feature of a channel. The specific methods for occlusion processing of the face image include the following step:

[0032] S21. Use the available large batch of segmentation data to train an initial segmentation model, use the small batch of fa...

Embodiment 2

[0042] Please refer to image 3 , for step S3 of Embodiment 1, it uses the VGG network to encode the four-channel image A to obtain the encoding feature A, and uses the VAE encoder to encode the four-channel image B to obtain the encoding feature B.

[0043] In this embodiment, VGG is used for encoding identity features, which is a classic CNN network, and variational autoencoder VAE is used for encoding non-identity features. The reason for this is that identity features can be extracted under supervision. We clearly know the meaning that the extracted features should represent, which is the identity of a certain face, so we can use the feature extraction module in the pre-trained face recognition model to directly identify Extract facial features from images. For non-identity features, it may include picture background, lighting, face posture, expression, etc. We don’t even know what types of features need to be extracted, we only know that all features that have nothing to d...

Embodiment 3

[0047] For step S4 of Embodiment 1, the generative confrontation network includes a generator and a discriminator, wherein the generator is used to combine the encoding feature A and the encoding feature B to obtain a synthetic face image, and the discriminator is used to judge the face synthetic image authenticity.

[0048] In this embodiment, the discriminator is used to judge whether the image synthesized by the generator is real or not, and calculates the gap between the synthesized sample and the real sample through the loss function, which is called the current training loss of the sample. Then the network will use the optimization algorithm of gradient descent to adjust the network parameters according to the training loss, so that the training loss will be further reduced, that is, the degree of fake images will be further increased. The discriminator is only needed in the network model training phase, not in the application phase.

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to view more

PUM

No PUM Login to view more

Abstract

The invention discloses a robust automatic face fusion method. The method relates to the technical field of image synthesis, and comprises the following steps: carrying out occlusion processing on a face image A and a face image B to obtain a four-channel image A and a four-channel image B, with the four-channel image A comprising identity features in a synthetic image, and the four-channel imageB comprising attribute features in the synthetic image; encoding the four-channel image A and the four-channel image B to obtain an encoding feature A and an encoding feature B; and combining the coding feature A and the coding feature B through a generative adversarial network, and outputting a face synthesis image. According to the method, a characteristic channel of a shielding mask is added, so that the synthesized characteristic has more effective information, and the method is more robust to a complex scene in practice; occlusion information is enhanced through feature reconstruction, amore complex face fusion scene can be processed, and the applicability is wider; the image segmentation is used for generating a feature mask and fusing the feature mask into original information, andthe boundary of image segmentation is expanded.

Description

technical field [0001] The present invention relates to the technical field of image synthesis, in particular, to a robust automatic face fusion method in a complex environment, which is used to fuse the face features of two input images, and the output image will have one of the images The features related to the face identity of the image and the features irrelevant to the face identity of another image. Background technique [0002] Image synthesis technology is widely used in image and video synthesis, network security and other fields. Relevant requirements include user privacy protection in network data, intelligent avatars in film and television production, virtual try-on of glasses or decorations in online marketing, game entertainment and live broadcast It can be used to enrich product functions, improve user experience, and publicize relevant organizations, which can be used to intelligently synthesize novel promotional materials, etc. With the advent of the 5G er...

Claims

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to view more

Application Information

Patent Timeline
no application Login to view more
Patent Type & Authority Applications(China)
IPC IPC(8): G06K9/62G06K9/00G06N3/04G06N3/08
CPCG06N3/08G06V40/168G06N3/045G06F18/253
Inventor 郑瑶王文一陈建文
Owner UNIV OF ELECTRONICS SCI & TECH OF CHINA
Who we serve
  • R&D Engineer
  • R&D Manager
  • IP Professional
Why Eureka
  • Industry Leading Data Capabilities
  • Powerful AI technology
  • Patent DNA Extraction
Social media
Try Eureka
PatSnap group products

Browse by: Latest US Patents, China's latest patents, Technical Efficacy Thesaurus, Application Domain, Technology Topic.

© 2024 PatSnap. All rights reserved.Legal|Privacy policy|Modern Slavery Act Transparency Statement|Sitemap