Sub-pixel Convolution Kernel Initialization for Artifact-Free Upscaling
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Convolutional neural networks (CNNs) suffer from checkerboard artifacts during image upscaling, primarily due to deconvolution overlap and random initialization, which affects the quality of generated images and reduces super-resolution performance.
Innovation Solution
The method involves initializing sub-kernels of a kernel with specific parameters and copying these parameters to other sub-kernels, followed by convolution and pixel rearrangement to generate output images, effectively eliminating checkerboard artifacts while maintaining computational efficiency.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If deconvolution overlap is used in CNN upscaling, then computational efficiency is maintained, but checkerboard artifacts appear in the output images
Solution Approach 1:
The patent applies local quality by making different parts of the kernel serve different functions: the sub-kernel handles local feature extraction while the rearrangement operation handles global pixel positioning. This division allows the sub-kernel to be initialized once and reused, eliminating checkerboard artifacts while maintaining computational efficiency through localized operations.
Solution Approach 2:
The kernel is segmented into multiple sub-kernels that are applied to different channels or feature maps. By initializing only one sub-kernel and copying its parameters to others, the patent reduces redundant computations and eliminates the checkerboard artifact problem caused by traditional deconvolution overlap methods.
2Adaptability or versatility
If random initialization is used for kernel parameters, then training flexibility is improved, but checkerboard artifacts are generated
Solution Approach 1:
The patent applies preliminary action by pre-initializing the sub-kernel parameters before the forward propagation process. This pre-initialization ensures that the same parameters are used across all sub-kernels, preventing checkerboard artifacts while still allowing the parameters to be learned and adapted during training through backpropagation.
Solution Approach 2:
The patent uses copying by replicating the initialized sub-kernel parameters to multiple sub-kernels. This copying mechanism ensures consistency across different channels or feature maps, eliminating the checkerboard artifact problem caused by random independent initialization while maintaining training flexibility through subsequent parameter updates.
3Adaptability or versatility
If multiple sub-kernels are independently initialized, then modeling capabilities are enhanced, but computational complexity increases
Solution Approach 1:
The patent applies universality by making a single initialized sub-kernel serve multiple functions across different channels or feature maps. This sub-kernel is copied and reused rather than independently initialized for each channel, reducing computational complexity while maintaining enhanced modeling capabilities through the rearrangement operation that combines the outputs.
Data Source
AI summary
An example system includes a processor and a memory. The system performs sub-pixel convolution that is free of checkerboard artifacts. In one example implementation, the system may execute a method that includes initializing one or more parameters of a sub-kernel of a kernel and copying the one or more parameters of the sub-kernel to other sub-kernels of the kernel. The method may further include performing convolution of an input image with the sub-kernels of the kernel and generating a plurality of first output images. A second output image is then generated based on the plurality of first output images.