Video face replacement method based on illumination migration

Abstract

The invention relates to a video face replacement method based on illumination migration, and the method comprises the steps: 1, face detection and recognition: carrying out the positioning of a replaced target face faceA in a face replacement video, recognizing a face feature value, positioning a plurality of key points of a face, and determining the angle deviation information of the face; positioning the replacement face faceB, identifying a face feature value, and detecting a plurality of key points of the face; step 2, determining a transformation matrix, positioning and detecting face information of the face-changing video, extracting features, extracting features featureA, and finding out a matrix transferA which needs distortion transformation and is obtained by converting the featureA into the featureA; step 3, face replacement: carrying out transferA-based transformation on the face B, restoring the face B to a shooting angle of a face changing video to form new face information face B, and replacing the feature A by using the feature B; and step 4, illumination migration: applying an illumination migration algorithm to obtain a final result of video face replacement. Themethod can be widely applied and popularized to computer vision, visual design, digital entertainment, film production and the like.

Description

technical field [0001] The invention belongs to the field of visual computing and computer vision, in particular to a human face replacement method, in particular to a video human face replacement method based on light migration. Background technique [0002] Cycle GAN is an early attempt among all face transformation attempts. In the wave of confrontational generative networks (GANs), it is found that as long as the samples of the source category and the samples of the target category are given, GANs can easily learn the conversion relationship between the two categories, and it is naturally suitable for image-to-image conversion. Image conversion problems, such as winter to summer of the same landscape photo, a horse to a zebra; the core idea of ​​​​Cycle GAN is that if it can be converted from the source to the target and back from the source, it can be considered that the model is very good. The conversion relationship between the two categories is learned, and the qual...

AI Technical Summary

Problems solved by technology

However, the conversion effect of its side face angle is not good, and the skin color difference after changing the face is relatively large

[0004] In a paper published by the Samsung AI Research Center in Moscow and the Skolkovo Institute of Science and Technology in May 2019, it was proposed that given a face image, the person in the picture can make an expression according to a given action. Not only photos of real people, but even people in oil paintings can speak naturally, but it is not possible to change the expressions and facial features of multiple faces in the video face

Images