Virtual Object Movement Control on Tabletop Displays

Resolve Bottlenecks,
Find Innovative SolutionsGenerate Solutions

Solution Overview

Problem

Existing display systems fail to effectively control the movement of virtual objects on large screens, especially when real objects such as cups and dishes are placed on the tabletop, leading to inefficient and inconvenient interactions.

Innovation Solution

An information processing apparatus that uses a combination of cameras and microphones to detect real objects and user interactions, allowing for the control of virtual object movement based on positional relationships with real objects, enabling more appropriate and efficient display management.

Engineering Contradictions & Design Principles

VSEngineering Contradiction Analysis

1Adaptability or versatility

If virtual objects are displayed and moved on a large screen tabletop display system, then multiple users can perform simultaneous operations and view information, but real objects such as cups and dishes placed on the tabletop interfere with the display and operation of virtual objects

Engineering Contradiction:

Improvemulti-user simultaneous operation capabilityVSAvoidinterference from real objects on display

Core Design Contradiction:

Adaptability or versatilityVSObject-affected harmful factors

Solution Approach 1:

The system performs preliminary detection of real objects on the tabletop using imaging devices before virtual object operations begin. By identifying the positions and properties of real objects in advance, the system can pre-calculate alternative movement routes for virtual objects that avoid these obstacles, preventing interference before it occurs.

Inventive Principle:

Principle #10Preliminary action

Solution Approach 2:

The system transitions from two-dimensional screen coordinates to three-dimensional spatial awareness by incorporating depth information from imaging devices. This allows the system to perceive the tabletop as a three-dimensional space containing both virtual objects and real physical objects, enabling intelligent route planning that navigates around obstacles in the third dimension.

Inventive Principle:

Principle #17Another dimension (Dimensionality change)

2Speed

If virtual objects move in straight lines at constant speeds, then operations are simple and fast, but movement routes may be blocked by real objects on the tabletop

Engineering Contradiction:

Improvevirtual object movement speedVSAvoidmovement route availability

Core Design Contradiction:

SpeedVSReliability

Solution Approach 1:

The system dynamically adjusts virtual object movement parameters based on real-time conditions. When a straight-line path is blocked by real objects, the system automatically calculates and switches to alternative routes, adjusting both the path geometry and movement speed to ensure reliable delivery of virtual objects to their destinations while maintaining operational efficiency.

Inventive Principle:

Principle #15Dynamics

Solution Approach 2:

The system continuously monitors the positions of real objects and virtual objects using imaging devices. This feedback mechanism allows the system to detect when a planned movement route becomes blocked and to recalculate alternative paths in real-time, ensuring that virtual object movement remains reliable even as the physical environment changes.

Inventive Principle:

Principle #23Feedback

3Ease of operation

If the system detects and responds to real objects in real-time, then virtual object movement can avoid obstacles, but the system complexity and processing requirements increase

Engineering Contradiction:

Improveintuitive virtual object interactionVSAvoiddetection and control system complexity

Core Design Contradiction:

Ease of operationVSDevice complexity

Solution Approach 1:

The system creates a digital replica or map of the physical tabletop environment by capturing images with imaging devices and extracting information about real objects. This virtual model of the physical space allows the system to perform complex collision detection and route planning calculations in the digital domain without requiring complex physical sensors or actuators, simplifying the overall system architecture.

Inventive Principle:

Principle #26Copying

4Adaptability or versatility

If virtual objects are allowed to pass through real objects, then movement is unrestricted, but the display becomes confusing and operation becomes difficult

Engineering Contradiction:

Improvevirtual object movement freedomVSAvoiddisplay clarity and operation intuitiveness

Core Design Contradiction:

Adaptability or versatilityVSLoss of information

Solution Approach 1:

The system takes preliminary action to prevent virtual objects from overlapping with real objects by calculating alternative routes that avoid such conflicts. By proactively preventing the harmful situation of overlapping displays rather than reacting after it occurs, the system maintains display clarity and operational intuitiveness while still allowing virtual objects to reach their destinations through alternative paths.

Inventive Principle:

Principle #9Preliminary anti-action

Data Source

PatentEP3189396B1Information processing apparatus, control method, and program

Publication Date: 2021.03.31 SONY GROUP CORP

Figure 1
Figure 2~3
Figure 4

AI summary

There is provided an information processing apparatus including circuitry configured to initiate display of a virtual object, based on a gesture operation, starting from a point of origin and moving towards a target point; and continue to display the virtual object in display motion after the gesture operation, wherein a path of travel of the virtual object or a display characteristic of the virtual object is determined based on a positional relationship between the virtual object and another object that is a real object located in proximity to the path of travel of the virtual object.