Deep Gaussian Process Image Generation Kernel
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current generative models, such as GP models, struggle to capture non-local generalization structures in data, particularly for image generation, due to reliance on rudimentary metrics and lack of incorporation of convolutional structures.
Innovation Solution
A deep Gaussian process method is introduced, utilizing a first GP for mapping latent variables to latent image vectors and a second GP for mapping these vectors to output image vectors, with a kernel that accounts for patch components and location components to incorporate prior correlations between pixels, enhancing image generation performance.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Loss of information
If GP models use rudimentary local metrics (e.g., Euclidian distances) for generalisation, then the model structure remains simple, but the model fails to capture non-local generalisation structures within data
Solution Approach 1:
The patent transforms the kernel function parameters to incorporate spatial location information. The kernel is redefined as k((i,j), (i',j')) = exp(-||x_i - x_{i'}||^2 / (2σ^2)) * exp(-γ|| (i,j) - (i',j') ||^2), where the second exponential term introduces a new parameter γ that controls the decay with spatial distance, enabling capture of non-local structures while maintaining computational tractability
Solution Approach 2:
The patent combines two kernel components: a similarity-based exponential term and a location-based exponential term. This composite kernel structure integrates both local feature similarity and spatial position information, allowing the model to capture non-local generalisation patterns without requiring complex architectural changes
2Loss of information
If convolutional structures are incorporated into GP models for non-local generalisation, then non-local structures are captured, but the method becomes inapplicable to image data generation
Solution Approach 1:
The patent creates a universal framework that works for both classification and generation tasks. The spatial-aware kernel is designed to be task-agnostic, serving dual purposes: capturing non-local structures for classification and generating realistic image patterns for synthesis. This is achieved by maintaining the probabilistic GP formulation while enhancing the kernel with location awareness
Solution Approach 2:
The patent applies different functional forms to different parts of the kernel: the similarity term handles feature space relationships while the location term handles spatial relationships. This local differentiation within the kernel structure allows the model to simultaneously capture local feature patterns and global spatial structures, making it applicable to image generation
3Reliability
If prior correlations between pixels are introduced through location-aware kernels, then image generation performance improves especially in low-data regimes, but the computational complexity of the kernel calculations increases
Solution Approach 1:
The patent segments the kernel computation into two independent components: similarity calculation and location calculation. This segmentation allows for efficient computation where the location term exp(-γ|| (i,j) - (i',j') ||^2) can be precomputed or approximated, reducing the overall computational burden while maintaining the ability to capture pixel correlations
Data Source
Figure 1
Figure 2a~2d
Figure 3
AI summary
There is provided a computer-implemented method of training a computer system to generate output images in dependence on a plurality of training images. The method includes receiving training data corresponding to the plurality of training images. The method further includes initialising a first set of parameters comprising, for each of the plurality of training images, one or more parameters for a respective variational distribution over latent variables, and initialising a second set of parameters for a deep Gaussian process (GP). The deep GP comprises: a first GP defined by a first kernel and corresponding to a distribution over first functions for mapping latent variables to latent images, each latent image comprising a plurality of elements; and a second GP defined by a second kernel and corresponding to a distribution over second functions for mapping latent images to output images. The method further includes iteratively updating the first set of parameters and the second set of parameters to increase an average probability density associated with each training image being generated when a sampled latent variable is passed through a first function sampled from the first GP and a second function sampled from second GP. The second functions map patches of latent images to pixels of output images, each patch of a latent image comprising a subset of the elements of the latent image.