Image Expansion Using Semantic Segmentation and Style Transfer
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing methods for expanding the outer area of an original image often introduce unnatural objects or blur, failing to naturally transition information due to lack of consideration for the properties and context of objects within the image.
Innovation Solution
An electronic device and method that identifies objects in an image, generates segmentation images, converts segmentation information to RGB, and reflects the features of the original image to create a naturally expanded outer area by using segmentation and image generating models, along with style transfer and blending processes.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Ease of manufacture
If the outer area of the original image is expanded by copying and pasting patches, then the expansion process is simple, but unnatural objects are introduced in the outer area
Solution Approach 1:
The patent segments the image into multiple regions based on object boundaries and semantic information. By dividing the image into distinct semantic segments (e.g., sky, ground, objects), the system can expand each segment appropriately while maintaining natural transitions, avoiding the patch-copying approach that introduces unnatural artifacts.
Solution Approach 2:
The patent introduces an intermediary prediction mechanism that predicts the content of the outer area based on the segmented regions and their contextual relationships. This prediction step acts as a mediator between the original image and the expanded area, ensuring natural transitions by considering semantic consistency rather than directly copying patches.
2Area of stationary object
If the outer area is expanded by analyzing pixels and predicting image content, then the expansion can cover larger areas, but prediction data becomes obscure leading to blurring
Solution Approach 1:
By segmenting the image into distinct semantic regions, the system maintains clear prediction data even when expanding to larger areas. Each segment has well-defined semantic boundaries and characteristics, which prevents the prediction data from becoming obscure and avoids blurring in the expanded outer area.
Solution Approach 2:
The patent applies local quality by considering the specific semantic characteristics of each region when generating prediction data. Instead of using generic pixel analysis, the system tailors the prediction to the local semantic context of each segment, maintaining data clarity and sharpness even in the expanded outer areas.
3Productivity
If the outer area is expanded without considering object properties and image context, then the expansion process is faster, but the expanded information does not transition naturally with the original image
Solution Approach 1:
The patent segments the image into semantic regions that preserve object properties and contextual relationships. This segmentation enables the system to maintain natural transitions in the expanded outer area by ensuring that the predicted content is consistent with the semantic structure of the original image, rather than generating arbitrary content.
Solution Approach 2:
The patent performs preliminary segmentation and semantic analysis of the original image before generating the expanded outer area. This preliminary action establishes the semantic framework and contextual relationships in advance, which then guides the expansion process to produce natural transitions without requiring complex real-time analysis.
Data Source
AI summary
An electronic device and a controlling method of the electronic device is disclosed. Specifically, the electronic device according to the disclosure may identify, based on receiving a user input for expanding an outer area of a first image, a plurality of objects included in the first image, and obtain a first segmentation image including segmentation information on areas corresponding to the respective objects, obtain a second segmentation image in which an outer area of the first segmentation image is expanded based on the segmentation information, obtain a second image in which segmentation information included in the second segmentation image is converted to RGB information, obtain a third image by reflecting a feature of the first image to the second image based on the segmentation information, and provide the obtained third image.