Image Extrapolation Using Visual Attention Model
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current image extrapolation techniques fail to effectively generate immersive images beyond the borders of a film or video without including distracting objects, are computationally intensive, and do not account for human visual system sensitivities.
Innovation Solution
An apparatus and method that uses a visual attention model to identify salient objects, modifies the extrapolated image to exclude these objects, and applies spatio-temporal filtering to ensure smooth and coherent image extension, while considering human visual system properties to avoid distracting elements.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Area of stationary object
If image extrapolation is performed using conventional techniques to extend beyond borders, then field of vision is increased, but distracting objects are included in the extrapolated regions
Solution Approach 1:
The patent extracts and removes salient objects from the extrapolated image regions while preserving the background content. The system identifies salient objects using a visual attention model and selectively removes them from the extrapolated areas, thereby eliminating distracting elements while maintaining the expanded field of vision.
Solution Approach 2:
The patent applies different processing quality to different regions of the image. The central region maintains original quality while the extrapolated peripheral regions undergo selective object removal. This local differentiation allows the system to preserve important content in the center while cleaning up distracting elements in the extended periphery.
2Reliability
If conventional image extrapolation techniques are used to generate immersive images, then viewer immersion is improved, but computational intensity increases significantly
Solution Approach 1:
The patent extracts only the essential information needed for immersion - specifically the background content and salient object locations - rather than processing entire high-resolution extrapolated images. By focusing computational resources on identifying and removing only salient objects rather than processing all image data, the system reduces computational intensity while preserving viewer immersion.
Solution Approach 2:
The patent changes the parameter of computational complexity by using a visual attention model that processes image data at a conceptual level (identifying salient objects) rather than at full pixel-level detail. This parameter change allows the system to achieve immersive results with reduced computational intensity by working with extracted features rather than complete image data.
3Reliability
If image extrapolation extends beyond original borders, then immersive experience is enhanced, but visual discomfort is caused by distracting elements
Solution Approach 1:
The patent removes salient objects from extrapolated regions that would otherwise cause visual discomfort. By extracting and eliminating these distracting elements from the peripheral extrapolated areas, the system enhances the immersive experience while preventing visual discomfort that would result from having prominent objects in the extended periphery.
Solution Approach 2:
The patent applies selective object removal specifically to the extrapolated peripheral regions where distracting elements would cause visual discomfort, while leaving the central viewing area unchanged. This local processing approach enhances immersion in the extended areas without introducing visual discomfort, as the cleaning is applied only where needed in the periphery.
Data Source
Figure 1~2
Figure 3
Figure 4~5
AI summary
A method and apparatus for generating an extrapolated image from an existing film or video content, which can be displayed beyond the borders of the existing file or video content to increase viewer immersiveness, are provided. The present principles provide to generating the extrapolated image without salient objects included therein, that is, objects that may distract the viewer from the main image. Such an extrapolated image is generated by determining salient areas and generating the extrapolated image with lesser salient objects included in its place. Alternatively, salient objects can be detected in the extrapolated image and removed. Additionally, selected salient objects may be added to the extrapolated image.