Dynamic Image Re-timing Using Neural Style Transfer
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current image processing technologies lack the ability to dynamically apply artistic or stylistic effects to target images based on source images captured at different times, seasons, or locations, failing to provide a realistic and context-aware modification of scene objects and their surroundings.
Innovation Solution
The system identifies source images with similar scenes from an image database, extracts stylistic elements based on capture time, and applies these elements to target images, using neural networks and three-dimensional modeling to ensure a realistic transfer of style, while penalizing alterations to structural elements to maintain the original scene's integrity.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Adaptability or versatility
If stylistic elements are extracted from arbitrary source images and applied to entire target images, then artistic style transfer is achieved, but the contextual realism and scene-specific accuracy are lost
Solution Approach 1:
The patent segments the target image into multiple scene objects using object detection and segmentation algorithms. Each scene object is then independently processed with style transfer, allowing different stylistic transformations for different objects while maintaining their structural integrity. This resolves the contradiction by enabling versatile style transfer at the object level rather than applying a uniform transformation to the entire image.
Solution Approach 2:
The patent applies local quality by extracting and applying stylistic elements specifically to matching scene objects rather than the entire image. The system identifies corresponding objects between source and target images and applies style transfer only to those regions, preserving the accuracy and realism of non-matching areas. This enables high scene-specific accuracy while maintaining adaptability through selective style application.
2Adaptability or versatility
If the target image is modified to match the source image style, then artistic effect is achieved, but the original scene structure and integrity are altered
Solution Approach 1:
The patent employs a dynamic approach by using generative adversarial networks (GANs) that can adaptively adjust the style transfer intensity and characteristics. The system dynamically balances between applying artistic style and preserving scene structure through the adversarial training process, where the generator creates stylized images while the discriminator ensures structural fidelity. This resolves the contradiction by making the style transfer process flexible and controllable rather than rigid and destructive.
Solution Approach 2:
The patent uses copying by creating a stylized version of the target image that replicates the artistic style of the source image while maintaining the original scene's structural elements. The system copies stylistic features (textures, colors, patterns) from the source image and applies them to the target image's scene objects, rather than copying the entire source image structure. This enables artistic style application while preserving scene integrity.
3Productivity
If conventional filters are applied to modify pixel values, then processing speed is maintained, but contextual awareness and semantic understanding are lost
Solution Approach 1:
The patent replaces conventional mechanical pixel manipulation filters with neural network-based semantic processing. Instead of applying mathematical transformations to pixel values, the system uses deep learning models to understand scene semantics, identify objects, and apply styles based on semantic understanding. This substitution enables contextual awareness while maintaining reasonable processing speeds through optimized neural network inference and parallel processing.
Data Source
AI summary
Techniques for the modification of at least part of a target image (e.g., scene objects within the target image), e.g., to make the target image appear that it was captured at a different time (e.g., a different time of day, different time of year) are disclosed. This “dynamic re-timing” of the target image may be achieved by finding one or more source images including the same (or similar) scene depicted in the target image (but, e.g., captured at different times), extracting stylistic elements from the source image(s), and then modifying at least part of the target image in a realistic fashion (e.g., not altering the geometry of objects in the target image), based on one or more extracted stylistic elements from the source image(s). Three-dimensional modeling of scene objects may allow a more realistic-looking transfer of the extracted stylistic elements onto scene objects in the target image to be achieved.