Variable Style Transfer via Blended Feature Vectors
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Conventional style transfer techniques produce unwanted artefacts and are limited by relying on a single style image, leading to non-uniform style transfer and reduced realism in virtual environments, as they struggle with domain matching and annotation requirements.
Innovation Solution
A method that utilizes multiple style images for domain matching, allowing for a combination of individual style transfers to create a composite stylised content image, with user-determined blending factors to optimize the style transfer process and reduce artefacts.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Manufacturing precision
If conventional style transfer techniques use a single style image, then the process is simple, but the style transfer is non-uniform and produces unwanted artefacts
Solution Approach 1:
The patent divides the style transfer process into multiple independent style transfers, each using a different style image. Each style transfer operates on the same content image but produces a different stylized version. These individual transfers are then combined through blending to create the final result, ensuring uniformity while managing complexity through modular processing
Solution Approach 2:
The patent combines multiple stylized content images (each produced by a different style transfer) into a composite final image. By blending multiple style transfers with adjustable weights, the system creates a composite result that maintains uniformity across different regions while allowing selective emphasis of different styles in different areas
2Reliability
If conventional style transfer relies on domain matching with single style image, then the annotation requirements are manageable, but the proportion of content-style regional pairs passing similarity tests is reduced
Solution Approach 1:
The patent segments the domain matching process into multiple independent matching operations, each between the content image and a different style image. This allows each matching operation to focus on finding compatible regions without being constrained by a single style, thereby increasing the proportion of regional pairs that pass similarity tests
Solution Approach 2:
The patent makes the style transfer system multi-functional by accepting multiple style images as input. This universality allows the system to handle diverse content-style combinations, increasing the likelihood that at least some style images will have high similarity with the content image regions, thereby improving the overall pass rate of similarity tests
3Manufacturing precision
If conventional style transfer produces artefacts, then the realism is reduced, but the user experience is limited
Solution Approach 1:
The patent implements a feedback mechanism where multiple style transfers are evaluated and blended based on their quality and compatibility. The blending process allows selective reinforcement of high-quality regions and suppression of artefact-prone regions, thereby improving overall image realism while maintaining ease of operation through automated quality assessment
Solution Approach 2:
By creating a composite of multiple style transfers, the system can select and combine the best regions from each transfer, avoiding artefacts present in individual transfers. This composite approach enhances realism by presenting a refined, artefact-reduced final image that maintains operational simplicity
Data Source
AI summary
A process and apparatus for applying style images ISj to at least one content image IC containing entity classes i (i: 1, 2, . . . M), wherein attributes of a plurality j of one style images (ISj: IS1, IS2, . . . ISN), each containing entity classes i (i: 1, 2, . . . M), are transferred to the content image IC, the process comprising down-sampling the at least one content image ICi, to derive a content feature vector FCi, down-sampling the j style images ISj, to derive j style feature vectors (FSij: FSi1, FSi2, . . . , FSiN), stylising the content feature vector FCi by transferring attributes of the style feature vectors (FSij: FSi1, FSi2, . . . , FSiN) to the content feature vector FCi, to derive j stylised content feature vectors (FCSij: FCSi1, FCSi2, . . . , FCSiN), combining a blending factor (αij: αi1, αi2, . . . , αiN) of each of the respective stylised content feature vectors (FCSij: FCSi1, FCSi2, . . . , FCSiN) to derive a blended feature vector Fi* and up-sampling the blended feature vector Fi* to generate a blended stylised content image ICSij, wherein the stylising step comprises transforming the content feature vector FCi, wherein the content feature vector FCi acquires a subset of the attributes of the style feature vector (FSij: FSi1, FSi2, . . . , FSiN).