Virtual Scene Interaction Using Camera-Matted Real Objects

Resolve Bottlenecks,
Find Innovative SolutionsGenerate Solutions

Solution Overview

Problem

Existing interactive methods in virtual environments are monotonous and complex, requiring players to control virtual objects for interaction, which limits diversity and efficiency.

Innovation Solution

An interactive method and apparatus that allows real objects captured by cameras to be cutout and displayed as matting objects in a virtual scene, enabling direct interaction without controlling virtual objects, reducing resource consumption for model data generation and storage.

Engineering Contradictions & Design Principles

VSEngineering Contradiction Analysis

1Adaptability or versatility

If players control virtual objects to interact with the virtual environment, then interaction can be achieved, but the interaction process becomes monotonous and complex

Engineering Contradiction:

Improveinteraction diversityVSAvoidinteraction complexity

Core Design Contradiction:

Adaptability or versatilityVSDevice complexity

Solution Approach 1:

The patent uses image copying technology to capture real-world objects and people through cameras, then copies them into the virtual environment as matting objects. This allows real objects to directly interact with virtual elements without requiring players to control virtual avatars, thereby diversifying interaction methods and simplifying the interaction process.

Inventive Principle:

Principle #26Copying

2Productivity

If virtual objects are used for interaction, then the virtual environment can be explored, but the interaction implementation process becomes complicated

Engineering Contradiction:

Improveinteraction efficiencyVSAvoidinteraction process complexity

Core Design Contradiction:

ProductivityVSDevice complexity

Solution Approach 1:

The patent replaces the mechanical control system (players controlling virtual objects through input devices) with an automated image processing system. Cameras capture real objects, image processing algorithms extract and matting the objects, then they are directly placed into the virtual environment for interaction, eliminating the need for complex control operations.

Inventive Principle:

Principle #28Mechanics substitution (Replace mechanical system)

3Adaptability or versatility

If real objects are cutout and displayed as matting objects in virtual scenes, then interaction diversity improves, but resource consumption for model data generation increases

Engineering Contradiction:

Improveinteraction diversityVSAvoidresource consumption

Core Design Contradiction:

Adaptability or versatilityVSUse of energy by moving object

Solution Approach 1:

Instead of creating new 3D models for real objects, the patent copies 2D images of real objects and uses matting technology to extract them from the background. These matting objects are then directly used in the virtual environment, avoiding the resource-intensive process of model data generation while maintaining interaction diversity.

Inventive Principle:

Principle #26Copying

Data Source

PatentUS12511827B2Interactive method and apparatus based on virtual scene, device, and medium

Publication Date: 2025.12.30 TENCENT TECHNOLOGY (SHENZHEN) CO LTD

AI summary

An interactive method and apparatus based on a virtual scene includes: receiving a virtual scene displaying operation; collecting a first scene image through a first camera coupled to a first terminal; and displaying a virtual environment picture, wherein the virtual environment picture includes a virtual scene and a matting object, the matting object copies a movement of a first object cutout from the first scene image and a movement of a second object cutout from a second scene image, where the second scene image is an image collected by a second camera coupled to a second terminal.