Video Background Removal Object Preservation via Hand Detection

Resolve Bottlenecks,
Find Innovative SolutionsGenerate Solutions

Solution Overview

Problem

Conventional videoconferencing systems often mistakenly identify objects held or shown by users as part of the background, leading to their removal during video conferences, preventing users from effectively showcasing objects to others.

Innovation Solution

A program and communication method that enable a user to scan and transmit images of objects in their environment, even when background removal is enabled, by detecting cues from the user and transmitting these images along with the processed video feed to another user's terminal, ensuring the object is displayed prominently.

Engineering Contradictions & Design Principles

VSEngineering Contradiction Analysis

1Reliability

If background removal function is enabled to improve video quality, then background can be separated from user, but objects held by user are mistakenly removed along with background

Engineering Contradiction:

Improvebackground separation accuracyVSAvoidobject visibility

Core Design Contradiction:

ReliabilityVSLoss of information

Solution Approach 1:

The patent introduces a hand detection intermediary that acts as a mediator between the background removal function and the final video output. By detecting hands and objects held by the user, the system creates an intermediate layer that protects these objects from being removed by the background removal function, thus resolving the contradiction between accurate background separation and object preservation

Inventive Principle:

Principle #24Intermediary (Mediator)

Solution Approach 2:

The system implements feedback by using hand detection results to adjust the background removal process. The detected hand and object regions are fed back to the background removal algorithm to exclude these areas from background processing, ensuring that objects held by the user are preserved while maintaining effective background separation

Inventive Principle:

Principle #23Feedback

2Loss of information

If user wants to show object to other parties, then object should be visible, but object is removed as background

Engineering Contradiction:

Improveobject visibilityVSAvoidbackground removal accuracy

Core Design Contradiction:

Loss of informationVSReliability

Solution Approach 1:

Hand detection serves as an intermediary mechanism that identifies objects of interest held by the user. This intermediary detection layer ensures that these objects are distinguished from the background and preserved in the final video stream, allowing users to effectively show objects to other parties while maintaining background removal functionality

Inventive Principle:

Principle #24Intermediary (Mediator)

Solution Approach 2:

The patent segments the video content into multiple regions: background, user, hands, and objects held by hands. This segmentation allows the system to apply different processing rules to each segment, ensuring that objects are separated from the background and preserved for display to other parties

Inventive Principle:

Principle #1Segmentation

Data Source

PatentUS20230377166A1Non-transitory computer readable medium and communication method

Publication Date: 2023.11.23 TOYOTA JIDOSHA KK

AI summary

A program is configured to cause a computer to execute operations, the computer being configured to remove a background from a first image captured in a first space in which a first user is located to generate a second image to be displayed to a second user in a second space that is separate from the first space, the operations including scanning an object in the first space upon detecting a cue from the first user, and transmitting an image of the scanned object together with the second image via a network.