CNN Image Synthesis Using Localized Loss Functions
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current image synthesis technologies face challenges in effectively combining content and style from source images using convolutional neural networks, particularly due to instability issues with Gram matrices and the inability to accurately transfer weathering patterns across different materials.
Innovation Solution
The use of localized loss functions, such as Gram matrices and covariance matrices, along with histogram losses, to optimize pixel values in synthesized images, and the application of CNNs for image synthesis that incorporate region-based and per-pixel style transfer techniques to achieve stable and high-quality results.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If Gram matrices are used for style transfer, then style information can be captured, but instability issues arise in the synthesis process
Solution Approach 1:
The patent changes the mathematical parameters used for style representation from Gram matrices to covariance matrices. This parameter change maintains the ability to capture style information while improving numerical stability during the synthesis process, directly resolving the contradiction between reliability and information loss.
2Manufacturing precision
If region-based style transfer is applied, then localized style control is improved, but computational complexity increases
Solution Approach 1:
The patent divides the image into multiple regions and applies style transfer independently to each region using localized loss functions. This segmentation approach enables precise localized style control while the modular nature of region-based processing allows for efficient computation through parallelization, resolving the contradiction between precision and complexity.
3Measurement precision
If per-pixel loss functions are used, then synthesis accuracy is improved, but processing time increases
Solution Approach 1:
The patent applies per-pixel loss functions selectively to specific regions or layers rather than uniformly across the entire image. This partial application maintains high synthesis accuracy where needed while reducing the overall computational burden and processing time, effectively resolving the contradiction between precision and time efficiency.
Data Source
AI summary
Systems and methods for providing convolutional neural network based image synthesis using localized loss functions is disclosed. A first image including desired content and a second image including a desired style are received. The images are analyzed to determine a local loss function. The first and second images are merged using the local loss function to generate an image that includes the desired content presented in the desired style. Similar processes can also be utilized to generate image hybrids and to perform on-model texture synthesis. In a number of embodiments, Condensed Feature Extraction Networks are also generated using a convolutional neural network previously trained to perform image classification, where the Condensed Feature Extraction Networks approximates intermediate neural activations of the convolutional neural network utilized during training.