Automated image synthesis using comb neural network architecture

Abstract

An image synthesis system includes a computing platform having a hardware processor and a system memory storing a software code including a neural encoder and multiple neural decoders each corresponding to a respective persona. The hardware processor executes the software code to receive target image data, and source data that identifies one of the personas, and to map the target image data to itslatent space representation using the neural encoder. The software code further identifies one of the neural decoders for decoding the latent space representation of the target image data based on the persona identified by the source data, uses the identified neural decoder to decode the latent space representation of the target image data as the persona identified by the source data to produce aswapped image data, and blends the swapped image data with the target image data to produce one or more synthesized images.

Description

[0001] Cross References to Related Applications [0002] This application claims a U.S. provisional patent entitled "Comb Networks for High-Resolution, Multi-Subject Face Swapping," filed May 20, 2019 The benefit and priority of Application No. 62 / 850,439, the entire contents of which are hereby incorporated by reference into this application. Background technique [0003] Transferring visual images from source domain to target domain is an important problem in visual effects. An exemplary application of this image transfer is to transfer a target performer's performance to a source actor, which is necessary when the source actor has passed away or has to play a role of a different age. [0004] On the downside, many conventional methods of performing image transfer generate low-resolution images with substantial artifacts. While techniques exist that can generate higher-resolution images, they often require time-consuming and laborious manual processing, and require careful...

AI Technical Summary

Problems solved by technology

While techniques exist that can generate higher-resolution images, they often require time-consuming and laborious manual processing, and require careful structuring of the shot scene, placement of physical feature points on target performers, and computer-generated A similar image of the target performer is manually matched to the face of the target performer

Also, despite the higher resolutions achieved with this manual and expensive method, there are often weird art effects

Images