Dayscale Timelapse Generation Using Patch Merging Neural Networks
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing image-to-image translation methods require domain labels and supervised training, making it difficult to generate high-resolution diverse dayscale timelapse videos from a single image.
Innovation Solution
A method using a generative neural network and a merging neural network to create plausible dayscale timelapse sequences without explicit domain supervision, employing a generative neural network trained in swap, random, and autoencoder modes, and incorporating skip connections and adaptive instance normalization to preserve details.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Manufacturing precision
If high-capacity image-to-image translation networks are trained at high resolution, then image generation quality and detail preservation are improved, but computational complexity and training feasibility deteriorate
Solution Approach 1:
The patent divides the high-resolution image processing into multiple low-resolution patches. Each patch is processed independently by the generative network, and then the results are merged using a merging network. This segmentation approach reduces the computational burden of processing entire high-resolution images while maintaining the ability to generate high-quality output through proper merging of patch results.
Solution Approach 2:
The patent employs a nested structure where a merging network is embedded within the overall image-to-image translation system. The merging network takes the segmented patch outputs and combines them into a complete high-resolution image, creating a nested architecture that handles both local patch processing and global image reconstruction.
2Measurement precision
If domain labels and paired training images are used, then translation accuracy between specific domains is improved, but data collection difficulty and annotation effort increase
Solution Approach 1:
The patent employs self-supervised learning where the model learns to perform image-to-image translation without requiring external domain labels or manual annotations. The system uses the input images themselves and their transformations to generate training signals, eliminating the need for costly data annotation processes while maintaining translation accuracy.
Solution Approach 2:
The patent changes the training paradigm from supervised learning with fixed domain labels to self-supervised learning with dynamic style extraction. Instead of requiring pre-defined domain categories, the model learns to extract and apply styles from unpaired images, transforming the learning objective from label-based to feature-based translation.
3Manufacturing precision
If multiple images from target domain are used as guidance, then translation quality is improved, but the complexity of the few-shot setting and domain label requirements during training increase
Solution Approach 1:
The patent extracts the style information from target domain images separately from the content information. By isolating the style component and applying it to source images, the model achieves high translation quality without requiring multiple guided images or complex domain labels during training. This extraction approach simplifies the training process while maintaining translation accuracy.
Data Source
AI summary
The disclosure relates to a field of plausible timelapse image(s) generation from a single image. A method of generating one or more images of a plausible dayscale timelapse sequence based on a content image using a trained generative neural network and a trained merging neural network is provided. The method includes receiving the content image and one of one or more predefined styles respectively corresponding to times of day to be applied to the content image or style images having styles to be applied to the content image, slicing the content image into n image crops, applying the trained generative neural network with each style to n image crops to obtain n image crops re-stylized according to each style, and merging the re-stylized n image crops for each style with the trained merging neural network to obtain images of a plausible dayscale timelapse sequence for the content image.