Methods and systems for image and voice processing

Abstract

Systems and methods are disclosed configured to train an autoencoder using images that include faces, wherein the autoencoder comprises an input layer, an encoder configured to output a latent image from a corresponding input image, and a decoder configured to attempt to reconstruct the input image from the latent image. An image sequence of a face exhibiting a plurality of facial expressions and transitions between facial expressions is generated and accessed. Images of the plurality of facial expressions and transitions between facial expressions are captured from a plurality of different angles and using different lighting. An autoencoder is trained using source images that include the face with different facial expressions captured at different angles with different lighting, and using destination images that include a destination face. The trained autoencoder is used to generate an output where the likeness of the face in the destination images is swapped with the likeness of the source face, while preserving expressions of the destination face.

Description

INCORPORATION BY REFERENCE TO ANY PRIORITY APPLICATIONS[0001]Any and all applications for which a foreign or domestic priority claim is identified in the Application Data Sheet as filed with the present application are hereby incorporated by reference under 37 CFR 1.57.BACKGROUND OF THE INVENTIONField of the Invention[0002]This document relates to systems and techniques for digital image and voice processing.Description of the Related Art[0003]Conventional techniques for processing computer generated videos may require large amounts of computer resources, take an inordinate amount of time. Hence, more computer resource-efficient and time-efficient techniques are needed to perform advanced forms of digital image processing, such as face-swapping.SUMMARY[0004]The following presents a simplified summary of one or more aspects in order to provide a basic understanding of such aspects. This summary is not an extensive overview of all contemplated aspects, and is intended to neither ident...

AI Technical Summary

Benefits of technology

This patent describes a way to use neural networks to classify and match faces, as well as a method to change the voice in a recording without affecting its overall structure and content. The invention uses a one-shot or few-shot architecture, which means a small number of example images are used to train the neural networks for future classification. This is in contrast to other methods that may require thousands of facial images for training. The patent also describes how a system can use an autoencoder to automatically generate a latent voice representation from a training voice sample, which can then be used to change the voice in a recording without affecting its overall structure and content. The invention is useful for applications such as voice swapping and audio re-recording.

Problems solved by technology

Conventional techniques for processing computer generated videos may require large amounts of computer resources, take an inordinate amount of time.

Images