Occlusion Removal in Terminal Image Sharing via Depth Segmentation
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing technologies for capturing and sharing information displayed on objects like paper or whiteboards often result in occlusion due to users' hands or heads blocking the view, leading to incorrect sharing of information.
Innovation Solution
A system comprising a first terminal apparatus that captures images and determines if a blocking object is present, processing the image to identify and remove the blocked portion, and a second terminal apparatus that displays the received image without the blocked portion.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Loss of information
If a camera captures information displayed on a display object in real time during a network meeting, then the information can be shared with other participants, but users' hands or heads may block the view and cause occlusion leading to incorrect sharing
Solution Approach 1:
The patent segments the captured image into multiple regions based on depth information, identifying the display object region and blocking object region separately. This segmentation allows the system to process only the relevant display object region for information sharing, excluding blocked portions, thereby maintaining information accuracy without requiring users to change their natural operating behavior.
Solution Approach 2:
The patent introduces depth information as an intermediary element between the captured image and the final shared information. By using depth data from the imager, the system can distinguish between the display object and blocking objects, enabling accurate extraction of visible information without direct user intervention or complex camera positioning.
2Loss of information
If the captured image is transmitted and displayed as-is, then the transmission process is simple and fast, but the blocked portions cause incorrect information to be shared
Solution Approach 1:
The patent performs preliminary processing of the captured image by extracting the display object region using depth information before transmission. This preliminary action identifies and isolates the visible portions of the display object, removing blocked areas in advance, so that the transmitted image data contains only the accurate, unblocked information without requiring complex post-processing at the receiving end.
Solution Approach 2:
The patent applies local quality processing by focusing computational resources only on the display object region identified through depth information, rather than processing the entire captured image. This approach extracts and transmits only the relevant visible portions of the display object, maintaining high information accuracy while minimizing processing complexity to essential operations.
3Loss of information
If the system processes the captured image to remove blocked portions, then information sharing accuracy is improved, but the processing time and computational resources increase
Solution Approach 1:
The patent replaces complex mechanical or manual methods of ensuring unblocked views with an automated image processing system that uses depth information from the imager. This substitution enables rapid, automated identification and removal of blocked portions through software processing, which is significantly faster and more efficient than manual intervention or complex camera positioning mechanisms.
Solution Approach 2:
The patent performs preliminary extraction of the display object region using depth information before full image processing and transmission. This preliminary action quickly identifies the relevant regions that need processing, reducing the overall processing time by focusing computational resources only on the necessary areas rather than processing the entire image frame.
Data Source
AI summary
A system according to the present disclosure includes a first terminal apparatus that repeatedly transmits an image captured using an imager and a second terminal apparatus that displays the received captured image. The first terminal apparatus acquires a position of a display object capable of displaying information, determines, when existence of a blocking object between the display object and the imager is detected, whether a blocked portion, which is a portion where the display object is at least partially blocked by the blocking object, exists within the captured image, and processes, when the blocked portion exists within the captured image, the captured image for the blocked portion within the captured image to be identifiable and transmits a captured image in which the blocked portion is identifiable as the captured image. The second terminal apparatus displays the captured image without displaying the blocked portion based on the received captured image.