Portrait Video Cropping with User-Weighted Salient Feature Selection
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing video cropping techniques for converting landscape to portrait orientation often result in suboptimal cropping, leading to unnecessary space or loss of important content, and require subsequent regeneration of the media.
Innovation Solution
A system that leverages human-assisted tagging of salient features using scene boundary detection and image recognition algorithms to intelligently crop media, assigning higher weights to user-marked features, ensuring relevant content is retained while discarding irrelevant regions.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If automatic cropping algorithms are used to convert landscape video to portrait orientation, then processing speed is improved, but cropping accuracy deteriorates leading to loss of important content
Solution Approach 1:
The patent introduces an intermediary weighting mechanism that combines automatic algorithm detection with manual user input. The system processes multiple feature types (faces, objects, text, motion) through algorithms, then applies user-defined weights to adjust the importance of each detected feature, serving as a mediator between automated processing and human judgment to achieve optimal cropping accuracy.
Solution Approach 2:
The system changes the parameter of feature importance by allowing users to assign different weights to various detected features. This parameter adjustment enables the same automatic cropping algorithm to produce different cropping results based on user preferences, improving accuracy without sacrificing processing speed.
2Productivity
If multiple features are detected and cropped automatically, then processing efficiency is improved, but relevance to viewer deteriorates due to inclusion of irrelevant features
Solution Approach 1:
The patent applies local quality by allowing different weights to be assigned to different detected features based on their relevance. Instead of treating all features equally, the system enables selective emphasis on specific features (e.g., giving higher weight to faces or text) while de-emphasizing others, ensuring the cropped video maintains local relevance to viewer expectations.
3Loss of time
If suboptimal cropping is performed initially, then processing time is reduced, but need for regeneration increases leading to time loss
Solution Approach 1:
The system performs preliminary action by allowing users to configure feature weights and cropping preferences before the actual video processing begins. This upfront configuration prevents the need for regeneration by ensuring the initial cropping is optimized according to user requirements, reducing both initial processing iterations and subsequent regeneration needs.
4Manufacturing precision
If user input is required for feature tagging, then cropping accuracy is improved, but ease of operation deteriorates
Solution Approach 1:
The system applies partial action by allowing users to provide input only for the features they care about most, rather than requiring comprehensive tagging of all detected features. Users can selectively weight specific features while leaving others to be handled automatically, improving cropping accuracy for critical elements without burdening users with excessive input requirements.
Data Source
AI summary
Described herein are systems and methods of converting media dimensions. A device may identify a set of frames from a video in a first orientation as belonging to a scene. The device may receive a selected coordinate on a frame of the set of frames for the scene. The device may identify a first region within the frame including a first feature corresponding to the selected coordinate and a second region within the frame including a second feature. The device may generate a first score for the first feature and a second score for the second feature. The first score may be greater than the second score based on the first feature corresponding to the selected coordinate. The device may crop the frame to include the first region and the second region within a predetermined display area comprising a subset of regions of the frame in a second orientation.