Stylized Motion Effects via Segmented Style Transfer and Lenticular Rendering
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current digital image processing systems, particularly those using deep learning techniques, are limited to producing static 2D stylized images and fail to incorporate motion effects, which limits their ability to capture dynamic content and emotional intensity.
Innovation Solution
An image processing apparatus that segments images, applies a style transfer network to different parts of the image, and then adds a lenticular effect, such as zoom or circular motion, to create a 3D motion effect, enhancing the image with a combination of style transfer and 3D photo inpainting.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Adaptability or versatility
If deep learning techniques are used for image segmentation and style transfer, then 2D stylized images can be generated, but motion effects cannot be added
Solution Approach 1:
The patent segments the image processing pipeline into distinct modules: image segmentation module, style transfer module, and lenticular effect module. Each module handles a specific function independently, allowing motion effects to be added without redesigning the entire system. The lenticular effect module processes segmented image regions separately to apply 3D motion effects like zoom, circular motion, or swing effects.
Solution Approach 2:
The patent implements a nested architecture where the lenticular effect module is integrated within the existing deep learning pipeline. The style transfer network processes images, and then the lenticular effect module嵌套s within this structure to add motion effects. This allows the system to maintain existing functionality while adding new capabilities through hierarchical integration.
2Length of moving object
If style transfer is applied to the entire image, then consistent styling is achieved, but selective styling of different parts is lost
Solution Approach 1:
The patent applies image segmentation to divide the input image into multiple regions (e.g., foreground objects, background, sky). Each segment can then be processed independently through the style transfer network, allowing different parts of the image to receive different style treatments. This maintains styling consistency within each region while enabling flexibility across regions.
Solution Approach 2:
The patent implements local quality by applying different style transfer parameters to different image segments. Users can select which regions receive style transfer and with what intensity. For example, foreground objects can have strong style application while background regions have weaker or no style transfer, creating localized stylistic effects that enhance the overall image composition.
Data Source
AI summary
Systems and methods for image processing are described. Embodiments of the present disclosure receive a first image depicting a scene and a second image that includes a style; segment the first image to obtain a first segment and a second segment, wherein the first segment has a shape of an object in the scene; apply a style transfer network to the first segment and the second image to obtain a first image part, wherein the first image part has the shape of the object and the style from the second image; combine the first image part with a second image part corresponding to the second segment to obtain a combined image; and apply a lenticular effect to the combined image to obtain an output image.