Learning Network Image Stylization With Reusable Feature Maps
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing image stylization methods require high processing resources and time, lack flexibility in applying different texture styles, and fail to separate texture style from image content, leading to inefficient and inflexible neural network operations.
Innovation Solution
A learning network architecture is employed, separating image feature extraction and stylization processing, with distinct sub-networks for feature extraction and texture style application, allowing training and operation in the feature space to enhance efficiency and flexibility.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If a unified neural network is used for both image feature extraction and stylization processing, then the network can maintain image content and apply texture styles, but the processing time and resource consumption are high
Solution Approach 1:
The unified neural network is divided into two independent sub-networks: a feature extraction network that preserves image content and a stylization network that applies texture styles. This segmentation allows each sub-network to be optimized for its specific function, reducing overall processing time and resource consumption while maintaining image content integrity through the dedicated feature extraction network.
2Productivity
If a unified neural network is used for image stylization, then the system can process images, but the processing resources and time consumption are considerable
Solution Approach 1:
The image stylization system is segmented into two independent sub-networks: feature extraction and stylization processing. The feature extraction network processes image content once, and the stylization network applies different texture styles to the extracted features. This segmentation enables reusable feature representations, reducing redundant computations and lowering processing resource consumption while maintaining full image processing capability.
3Adaptability or versatility
If a single neural network is used for stylization, then the system can apply texture styles, but the flexibility to provide multiple texture styles is limited
Solution Approach 1:
The system is segmented into a shared feature extraction network and multiple independent stylization sub-networks, each trained for a specific texture style. This architecture provides flexibility by allowing users to select from multiple pre-trained stylization networks for different texture styles without retraining the entire system. The modular structure manages complexity by organizing networks into reusable components with clear functional boundaries.
Solution Approach 2:
The feature extraction network serves as a universal component that processes image content for all different texture styles. By extracting features once and reusing them across multiple stylization sub-networks, the system achieves multi-functionality where a single feature extraction pipeline supports diverse texture style applications, enhancing adaptability without proportionally increasing overall system complexity.
4Ease of operation
If image feature extraction and stylization processing are combined, then the network can process images end-to-end, but the processing efficiency is reduced
Solution Approach 1:
The end-to-end processing capability is maintained through a segmented architecture where the feature extraction network outputs that directly feed into the stylization network. This segmentation improves processing efficiency by allowing independent optimization and caching of intermediate features. The feature extraction results can be reused across multiple stylization operations, reducing redundant computations while preserving the ease of operation through seamless integration of the two sub-networks.
Data Source
Figure 1
Figure 2
Figure 3
AI summary
In accordance with implementations of the subject matter described herein, there is provided a solution of image stylization based on a learning network. In this solution, a learning network is trained with a plurality of images and a reference image with a particular texture style. A plurality of different sub-networks of the learning network is trained, respectively. Specifically, one of the sub-networks is trained to extract one or more feature maps from the source image and transform the feature maps with the texture style applied thereon to a target image. Each of the feature maps indicates part of feature information of the source image. Another sub-network is trained to apply a specified texture style to the extracted feature maps, such that the target image generated based on the processed feature maps can embody the specified texture style.