Intelligent Video Cropping for Portrait Mobile Displays
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Videos and media recorded in wide aspect ratios cannot fit full screen on mobile devices held in vertical orientation, often resulting in cropping that loses detail or adding black bars, as existing cropping methods assume important content is centered, which may not always be the case.
Innovation Solution
A method and system for intelligent cropping that identifies regions of interest, such as faces, text, and objects, using facial recognition, object detection, and motion analysis, to dynamically adjust the cropping and padding of media to retain important features while discarding unimportant regions, ensuring optimal display on devices with different orientations.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Area of stationary object
If center cropping is applied to fit video on portrait display, then the video fills the screen, but important content at the edges is lost
Solution Approach 1:
The patent segments the video frame into multiple regions of interest using object detection and facial recognition, identifying specific areas containing important content rather than treating the frame as a uniform whole. This allows selective retention of important regions while cropping less important areas.
Solution Approach 2:
The patent applies different quality priorities to different regions of the video frame. Regions containing detected objects, faces, or text are assigned higher quality weights, ensuring they are preserved during cropping, while regions with less important content are more readily cropped to achieve the target aspect ratio.
2Loss of information
If black bars are added to maintain aspect ratio, then content is preserved, but the display size is reduced
Solution Approach 1:
The patent dynamically adjusts the cropping regions across multiple frames of the video, allowing the displayed content area to expand and contract based on what is important in each frame. This dynamic approach eliminates the need for static black bars while maintaining content integrity through intelligent region selection.
3Loss of information
If important regions are retained in portrait orientation, then content quality is maintained, but the cropping region is constrained
Solution Approach 1:
The patent performs preliminary object detection, facial recognition, and text detection on each video frame before cropping is applied. By pre-identifying regions of interest and calculating their importance weights in advance, the system simplifies the subsequent cropping decision-making process while ensuring important content is preserved.
Solution Approach 2:
The patent uses feedback from object detection algorithms and facial recognition systems to continuously adjust cropping regions. The detected features provide real-time information about what content is important, allowing the cropping calculator to adaptively preserve these features while maintaining the target aspect ratio.
Data Source
Figure 1
Figure 2
Figure 3
AI summary
Systems and methods for cropping media for a particular orientation using a computing device are described. A method may first comprise receiving by a video preprocessor of the device, a first frame of media in a first orientation. A first region comprising a first feature within the first frame may be identified, by an image analyzer. A cropping calculator of the device may generate a score for the first region based on a characteristic of the first feature and determine that the score for the first region exceeds a threshold. An image processor of the device may then crop the first frame of the video, responsive to the determination that the score for the first region exceeds the threshold, to include the first region within a predetermined display area comprising a subset of the first frame in a second orientation.