Portrait to Landscape Image Conversion with Foreground Completion
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing methods for converting portrait format videos to landscape format fail to produce compelling quality, as they either fill borders with blurred backgrounds or crop images, lacking effective aspect ratio conversion techniques.
Innovation Solution
An image processing device and method that receive image data with a portrait aspect ratio, add image areas to both sides, extend the background into these areas, and complete foreground objects by determining and arranging visual representations of missing parts, thereby generating an image with a landscape aspect ratio.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Adaptability or versatility
If portrait format video is converted to landscape format by filling black side border with blurred background, then aspect ratio conversion is achieved, but image quality and visual appeal deteriorate
Solution Approach 1:
The system performs preliminary actions by generating multiple candidate images through different extension methods (blurring, copying, synthetic generation) before final selection. This allows pre-computation of quality metrics and selection of the best candidate, resolving the contradiction between aspect ratio conversion and image quality maintenance.
Solution Approach 2:
The system changes parameters by adjusting extension methods, blur strengths, copy offsets, and synthesis parameters to generate diverse candidate images. By varying these parameters across multiple iterations, the system achieves both aspect ratio conversion and high image quality through optimal parameter selection.
2Adaptability or versatility
If portrait format video is converted to landscape format by cropping, then aspect ratio conversion is achieved, but content information is lost
Solution Approach 1:
The system transitions from 2D cropping to 2D extension by adding image areas to the sides of the portrait image. This dimensional change allows the system to maintain the original vertical content while expanding horizontally, preventing information loss and achieving aspect ratio conversion simultaneously.
Solution Approach 2:
The system performs preliminary extension of the image background and foreground elements before final composition. By pre-generating extended background regions and positioning candidate foreground elements, the system preserves original content information while creating the desired landscape aspect ratio.
3Adaptability or versatility
If image area is added to convert aspect ratio, then landscape format is achieved, but background extension quality deteriorates
Solution Approach 1:
The system performs preliminary background extension using multiple methods (blurring original background, copying adjacent regions, synthetic generation) before final composition. This pre-computation of background candidates allows selection of the highest quality extension, resolving the contradiction between aspect ratio conversion and background quality.
Solution Approach 2:
The system changes background extension parameters including blur radius, copy offset distance, synthesis probability, and mixing ratios to generate diverse background candidates. By optimizing these parameters, the system achieves both aspect ratio conversion and high-quality background extension.
4Manufacturing precision
If foreground object is completed by adding missing parts, then object completeness is improved, but processing complexity increases
Solution Approach 1:
The system performs preliminary identification and classification of foreground objects before completion processing. By pre-segmenting the image into background and foreground regions and identifying object boundaries, the system simplifies the subsequent completion task and reduces overall processing complexity while maintaining object completeness.
Solution Approach 2:
The system segments the image into background regions and foreground object regions, then processes each segment separately. This segmentation allows the system to apply appropriate completion methods to each object independently, reducing processing complexity while achieving complete and accurate object reconstruction.
Data Source
AI summary
An image processing device is disclosed, featuring interface circuitry to receive image data representing a first image with an aspect ratio smaller than one. This image could be a photograph or a still frame from a video. The device's processing circuitry generates a second image with an aspect ratio greater than one by adding image areas to the lateral sides of the first image. The processing circuitry extends the background into these added areas and identifies foreground objects. If a foreground object is incomplete, the device determines and adds a visual representation of the missing part to complete the object in the extended image areas.