Multi-Sensor ROI Boundary Detection for Subject-Background Separation
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing image processing systems fail to provide a clear boundary distinction between the primary subject and background region, limiting the application of effects such as bokeh, dolly, and background style effects.
Innovation Solution
An image processing apparatus and method that utilizes multiple image sensors to capture primary and secondary frames, detect rough regions of interest (RROI), determine actual regions of interest (ROI) boundaries, and apply effects like bokeh outside the ROI boundaries using gradient descent and geometric transformations.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If existing systems use single image sensor and pattern-based background processing, then device complexity is reduced, but boundary distinction between primary subject and background region is poor
Solution Approach 1:
The patent divides the image processing into multiple stages: rough ROI detection, boundary refinement, and effect application. Multiple image sensors are segmented to capture different depth information, allowing precise boundary detection without requiring a single complex sensor system.
Solution Approach 2:
The patent introduces depth dimension by using multiple image sensors at different positions to capture parallax information. This transforms 2D image processing into 3D spatial processing, enabling accurate boundary distinction between subject and background by analyzing depth differences across sensors.
2Measurement precision
If multiple image sensors are used to capture primary and secondary frames, then boundary distinction precision is improved, but device complexity increases
Solution Approach 1:
The patent combines data from multiple image sensors through image fusion techniques. By merging the captured frames and analyzing parallax relationships, the system achieves precise boundary detection while distributing the complexity across multiple simpler sensor units rather than requiring one complex sensor.
Solution Approach 2:
The patent introduces an intermediary processing stage that fuses data from multiple sensors before final boundary determination. This intermediary fusion process simplifies the overall system architecture by creating a unified data representation that reduces the complexity of direct multi-sensor integration.
3Measurement precision
If rough ROI detection is performed followed by boundary refinement, then boundary precision is improved, but processing time increases
Solution Approach 1:
The patent performs preliminary rough ROI detection using simplified algorithms before applying complex boundary refinement techniques. This preliminary action identifies candidate regions of interest, allowing subsequent detailed boundary processing to focus only on relevant areas, thereby reducing overall processing time while maintaining high precision.
Solution Approach 2:
The patent segments the processing pipeline into distinct stages: rough detection, boundary refinement, and effect application. Each stage processes only the necessary data with appropriate complexity, preventing unnecessary computational overhead and reducing total processing time while achieving high boundary precision.
Data Source
Figure 1~2
Figure 3A~3B
Figure 3C~4A
AI summary
Provided is an image processing apparatus comprising at least one memory configured to store instructions, and at least one processor configured to execute the instructions to obtain a first image captured by a first image sensor, obtain a second image captured by a second image sensor located in a different position from a position of the first image sensor, determine a rough region of interest (RROI) in the first image, determine a geometric transformation that maps a position of an RROI of the second image corresponding to the RROI of the first image to a position of the RROI of the first image, and determine a boundary of a region of interest (ROI) in the first image corresponding to the RROI of the first image, based on the geometric transformation.