Fourier Transform Neural Network Image Synthesis

Resolve Bottlenecks,
Find Innovative SolutionsGenerate Solutions

Solution Overview

Problem

Current methods for synthesizing larger textured images from smaller ones often result in rough edges or blurry textures, and require significant computing resources, making them unsuitable for real-time applications and limited in replicating diverse textures.

Innovation Solution

The use of a Fourier transform in conjunction with neural networks to upsample feature maps, specifically employing Fast Fourier Transform (FFT) for image expansion, which enables the generation of high-quality, larger textured images while reducing computational demands.

Engineering Contradictions & Design Principles

VSEngineering Contradiction Analysis

1Length of stationary object

If traditional tiling or scaling methods are used to synthesize larger textured images from smaller ones, then the image size is increased, but the quality deteriorates with rough edges or blurry textures

Engineering Contradiction:

Improveimage sizeVSAvoidtexture quality

Core Design Contradiction:

Length of stationary objectVSManufacturing precision

Solution Approach 1:

The patent transforms the image into the frequency domain using Fast Fourier Transform, manipulating frequency components rather than spatial pixels. By modifying parameters in the frequency domain and applying inverse transform, the system generates high-quality enlarged textures that avoid the artifacts of traditional spatial domain interpolation methods.

Inventive Principle:

Principle #35Parameter changes

2Manufacturing precision

If traditional texture synthesis techniques are used to achieve high quality, then the texture quality is improved, but the computing requirements become significant and cannot be performed in real-time

Engineering Contradiction:

Improvetexture qualityVSAvoidprocessing speed

Core Design Contradiction:

Manufacturing precisionVSProductivity

Solution Approach 1:

The patent replaces traditional mechanical interpolation operations in the spatial domain with mathematical operations in the frequency domain. The Fast Fourier Transform converts complex spatial relationships into simpler frequency domain operations, enabling real-time processing while maintaining high texture quality.

Inventive Principle:

Principle #28Mechanics substitution (Replace mechanical system)

3Adaptability or versatility

If traditional methods are used for texture replication, then the variety of textures is limited, but the computational resources required are reduced

Engineering Contradiction:

Improvetexture varietyVSAvoidcomputational resources

Core Design Contradiction:

Adaptability or versatilityVSUse of energy by moving object

Solution Approach 1:

The frequency domain approach serves multiple functions: it enables image enlargement, texture synthesis, and diverse pattern generation through a single unified framework. By manipulating frequency components, the system can create various texture types and patterns without requiring separate algorithms for each case.

Inventive Principle:

Principle #6Universality (Multi-functionality)

Data Source

PatentUS20220101494A1Fourier transform-based image synthesis using neural networks

Publication Date: 2022.03.31 NVIDIA CORP

AI summary

Apparatuses, systems, and techniques to scale textured images using a Fourier transform in conjunction with one or more neural networks. In at least one embodiment, a neural network generates an expanded image from an input image by applying a Fourier transform to one or more feature maps generated by said neural network and up-scaling one or more resulting frequency domain feature maps before generating an expanded output image based on up-scaled feature maps.