Multi-Scale Style Transfer With User Control and Fewer Artifacts

Resolve Bottlenecks,
Find Innovative SolutionsGenerate Solutions

Solution Overview

Problem

Conventional style transfer techniques are resource-intensive, inaccurate, and lack user control over the application of visual styles, resulting in noticeable artifacts and limited artistic creativity.

Innovation Solution

A machine learning model that applies the style of a style image to a content image using a feature extraction network, merging features at multiple scales and employing residual blocks, combined with user controls to modify the style transfer process.

Engineering Contradictions & Design Principles

VSEngineering Contradiction Analysis

1Adaptability or versatility

If conventional style transfer techniques are used to handle vast number of different style and content images, then the system can process diverse images, but the manufacturing precision deteriorates resulting in noticeable visual artifacts

Engineering Contradiction:

Improvehandling diverse style and content imagesVSAvoidvisual accuracy of style-transferred images

Core Design Contradiction:

Adaptability or versatilityVSManufacturing precision

Solution Approach 1:

The patent segments the style transfer process into multiple processing stages: feature extraction from content and style images, Gram matrix computation to capture style statistics, and iterative optimization to apply style while preserving content. This segmentation allows handling diverse images while maintaining visual accuracy at each stage.

Inventive Principle:

Principle #1Segmentation

Solution Approach 2:

The patent changes key parameters including learning rate scheduling, optimization iterations, and feature extraction depths to balance adaptability across different image types with manufacturing precision. By adjusting these parameters dynamically, the system handles diverse styles while reducing visual artifacts.

Inventive Principle:

Principle #35Parameter changes

2Adaptability or versatility

If computationally-intensive models are used to handle vast number of different types of style and content images, then the adaptability improves, but the use of energy increases

Engineering Contradiction:

Improvehandling capability for diverse imagesVSAvoidcomputational energy consumption

Core Design Contradiction:

Adaptability or versatilityVSUse of energy by moving object

Solution Approach 1:

The patent extracts only the essential style features through Gram matrix computation from style images and content features from content images, rather than processing entire images through heavy models. This extraction approach maintains adaptability while significantly reducing computational energy consumption.

Inventive Principle:

Principle #2Taking out (Extraction)

Solution Approach 2:

The patent performs preliminary feature extraction and Gram matrix computation before the actual style transfer optimization. By preparing style statistics in advance, the system reduces the computational burden during the energy-intensive optimization phase while maintaining handling capability for diverse images.

Inventive Principle:

Principle #10Preliminary action

3Extent of automation

If conventional style transfer techniques are used, then style transfer can be performed, but the ease of operation deteriorates as users cannot control or adjust the influence of the style image

Engineering Contradiction:

Improveautomatic style transfer capabilityVSAvoiduser control over style application

Core Design Contradiction:

Extent of automationVSEase of operation

Solution Approach 1:

The patent introduces dynamic user controls that allow adjustment of style influence strength, content preservation level, and optimization iteration count. These dynamic parameters enable users to control the style transfer process while the system automatically optimizes the transfer, balancing automation with ease of operation.

Inventive Principle:

Principle #15Dynamics

4Extent of automation

If conventional style transfer techniques are used, then basic style transfer can be achieved, but the productivity deteriorates due to resource-intensive processing

Engineering Contradiction:

Improveautomatic style transfer functionVSAvoidprocessing speed and resource efficiency

Core Design Contradiction:

Extent of automationVSProductivity

Solution Approach 1:

The patent replaces heavy mechanical-style computational models with a more efficient approach using Gram matrix statistics and targeted optimization. This substitution maintains the automatic style transfer function while significantly improving processing speed and resource efficiency.

Inventive Principle:

Principle #28Mechanics substitution (Replace mechanical system)

Solution Approach 2:

The patent applies partial action by focusing computational resources on the most critical aspects of style transfer: capturing style statistics through Gram matrices and optimizing key feature alignments. This partial focus achieves effective style transfer with reduced resource consumption, improving productivity.

Inventive Principle:

Principle #16Partial or excessive action

Data Source

PatentUS12488428B2Universal style transfer using multi-scale feature transform and user controls

Publication Date: 2025.12.02 ADOBE INC

AI summary

Techniques for generating style-transferred images are provided. In some embodiments, a content image, a style image, and a user input indicating one or more modifications that operate on style-transferred images are received. In some embodiments, an initial style-transferred image is generated using a machine learning model. In some examples, the initial style-transferred image comprises features associated with the style image applied to content included in the content image. In some embodiments, a modified style-transferred image is generated by modifying the initial style-transferred image based at least in part on the user input indicating the one or more modifications.