Systems and Methods for Providing Convolutional Neural Network Based Image Synthesis Using Stable and Controllable Parametric Models, a Multiscale Synthesis Framework and Novel Network Architectures

Abstract

Systems and methods for providing convolutional neural network based image synthesis using localized loss functions is disclosed. A first image including desired content and a second image including a desired style are received. The images are analyzed to determine a local loss function. The first and second images are merged using the local loss function to generate an image that includes the desired content presented in the desired style. Similar processes can also be utilized to generate image hybrids and to perform on-model texture synthesis. In a number of embodiments, Condensed Feature Extraction Networks are also generated using a convolutional neural network previously trained to perform image classification, where the Condensed Feature Extraction Networks approximates intermediate neural activations of the convolutional neural network utilized during training.

Description

CROSS REFERENCED APPLICATION[0001]This application claims priority to U.S. Provisional Application Ser. No. 62 / 383,283, filed Sep. 2, 2016, U.S. Provisional Application Ser. No. 62 / 451,580, filed Jan. 27, 2017, and U.S. Provisional Application Ser. No. 62 / 531,778, filed Jul. 12, 2017. The contents of each of these applications are hereby incorporated by reference as if set forth herewith.FIELD OF THE INVENTION[0002]This invention generally relates to image synthesis and more specifically relates to image synthesis using convolutional neural networks based upon exemplar images.BACKGROUND[0003]With the growth and development of creative projects in a variety of digital spaces (including, but not limited to, virtual reality, digital art, as well as various industrial applications), the ability to create and design new works based on the combination of various existing sources has become an area of interest. However, the actual synthesis of such sources is a hard problem that raises a v...

AI Technical Summary

Benefits of technology

[0015]In a further embodiment again, the instructions to determine a localized loss function for a pixel in the source content image direct the one or more processors to receive a mask that identifies regions of the source content image, determine a group of pixels including the pixel that are included in one of the plurality of regions identified by the mask, determine a localized loss function for the one of the plurality of regions from the groups of pixels included in the one of the plurality of regions, and associate the localized loss function with the pixel.

[0016]In still yet another embodiment, the instructions to determine a localized loss function for a pixel in the source style image direct the one or more processors to group the pixels of the source content image into a plurali

Problems solved by technology

However, the actual synthesis of such sources is a

Images