Context-Aware Camera Preview Cropping for Non-Regular Objects
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing image cropping technologies lack the ability to dynamically adjust non-regular and non-polygonal boundaries during the preview stage, relying on manual input post-capture and failing to provide flexible boundary options based on scene context.
Innovation Solution
An AI-based system that utilizes enhanced Region Proposal Network (RPN) and Neuro-symbolic models to determine outline boundaries and relevance scores for multiple objects in a preview image, allowing users to select and crop objects with non-regular shapes, and includes early exit mechanisms to eliminate irrelevant frames.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Ease of operation
If manual input is used for cropping after image capture, then users can specify custom boundaries, but the process is time-consuming and lacks real-time interaction
Solution Approach 1:
The system performs preliminary action by detecting objects and generating crop boundary suggestions during the preview stage before the user finalizes the capture. The AI model pre-processes the scene to identify multiple objects and potential crop regions, so when the user captures the image, the cropping boundaries are already prepared and can be selected or adjusted instantly, eliminating post-capture processing time.
2Productivity
If rectangular crop boundaries are used, then the cropping process is simple and fast, but users cannot achieve non-regular shapes needed for specific objects
Solution Approach 1:
The system implements dynamics by transitioning from static rectangular crop boundaries to dynamic non-regular boundaries that adapt to object shapes. The AI model generates crop boundaries that conform to the actual contours of detected objects, allowing the crop shape to dynamically adjust based on the scene content while maintaining real-time performance during preview.
Solution Approach 2:
The system applies local quality by making different parts of the crop boundary have different properties. Instead of a uniform rectangular shape, the crop boundary locally adapts to match the contours of specific objects in different regions of the image, allowing each local area to have the appropriate shape characteristics for the object it encloses.
3Adaptability or versatility
If third-party applications provide non-rectangular cropping, then users can achieve desired shapes, but manual input is still required and cropping can only be done post-capture
Solution Approach 1:
The system implements self-service by enabling the camera application to automatically detect objects and generate crop boundaries without requiring external third-party applications. The integrated AI model autonomously analyzes the preview scene, identifies multiple objects, and proposes crop boundaries, making the system self-sufficient and eliminating the need for manual intervention or additional software.
Solution Approach 2:
The system performs preliminary action by automatically detecting objects and generating crop suggestions during the preview stage before capture. This pre-processing eliminates the need for post-capture cropping operations, as the boundaries are already determined and ready for immediate use.
4Measurement precision
If AI models process all preview frames, then accurate object detection is achieved, but computational burden and processing time increase significantly
Solution Approach 1:
The system applies partial action by processing only relevant preview frames rather than all frames. The early exit mechanism detects when sufficient object information is available and stops processing, avoiding unnecessary computation on frames that don't contribute meaningful new information, thus reducing energy consumption while maintaining detection accuracy.
Solution Approach 2:
The system extracts and processes only the essential elements from preview frames. The early exit mechanism extracts sufficient object detection information without processing the entire frame sequence, removing unnecessary computational steps while preserving the critical data needed for accurate cropping decisions.
Data Source
AI summary
A method of an electronic device, including: receiving, from a camera, a preview image including a plurality of objects; determining outline boundaries of the plurality of objects; determining a relevance score between the plurality of objects based on context of the plurality of objects and relationship between the plurality of objects; generating a plurality of boundary suggestions including the plurality of objects, based on the relevance score; receiving a user input indicating a selection of one of the plurality of boundary suggestions or a selection of at least one object from the plurality of objects present in the plurality of boundary suggestions, wherein the relevance score is updated for the at least one object based on the user input; and cropping the at least one object from the preview image based on the user input and the updated relevance score using the outline boundaries determined for the plurality of objects.