Face replacement method based on multistage attribute encoder and attention mechanism

Abstract

The invention discloses a face replacement method based on a multi-level attribute encoder and an attention mechanism. The face replacement method mainly solves the problems that in the prior art, target attributes such as background and illumination are ignored for image replacement, and the fusion effect is poor. According to the scheme, the method comprises the following steps: 1) preprocessing a source face image by using a multi-task convolutional neural network; 2) extracting source face identity features through a feature encoder; 3) extracting target face image attributes by using a multi-level attribute encoder through multi-level cascaded convolution blocks and deconvolution blocks and interlayer connection; 4) constructing a novel generator network in combination with an attention mechanism, and designing a generator loss function; 5) making a network training set and a test set, and performing iterative training on the novel generator network; and 6) generating a face replacement image by using the trained network model. The method can comprehensively and accurately extract the attributes of the target image, better preserves the posture, expression, illumination and other information of the target face, and generates a real and natural face replacement image.

Description

technical field [0001] The invention belongs to the technical field of computer vision, and further relates to image processing technology, specifically a face replacement method based on a multi-level attribute encoder and an attention mechanism. It can be used for virtual hairstyle and clothing experience, mass entertainment, and post-production of film and television works. Background technique [0002] Face replacement refers to the target image X t The face area in is replaced by the source face image X s The corresponding part in , while preserving the target attributes such as facial expression, pose, light, etc. in the target image to the greatest extent. Blanz V proposed the earliest face replacement method in his paper "Exchanging Faces in Images" (Computer Graphics Forum journal paper, 2004), using a relatively simple 3D model scheme to roughly estimate pose and light, and The source face is replaced onto the target image. As the first attempt in the field of ...

AI Technical Summary

Problems solved by technology

However, its shortcomings are: when the facial expressions of the target face and the reference face are quite different, the generated replacement face is severely distorted and has poor realism.

This method is computationally lightweight and can run efficiently on the CPU. The disadvantage is that the face recognition effect is not good when the face angle is large, and the outline of the eyes and lips of the replaced face is unnatural.

DeepFaceLab can generate high-resolution images and generalize them to the input resolution, but it is a typical one-to-one face-changing mode, and it needs to be retrained after each face replacement, which takes a lot of time

Images