AR Display Control Maintaining Spatial Position Stability

Resolve Bottlenecks,
Find Innovative Solutions
Generate Solutions

Solution Overview

Problem

Current augmented reality (AR) technologies lack effective methods for providing users with appropriate information in various situations, failing to enhance user experience by maintaining spatial positions and annotations accurately as the viewpoint changes.

Innovation Solution

A display control device and method that acquires moving images from a changing viewpoint, specifies spatial positions, and displays them in a manner to maintain these positions across frames, allowing for accurate annotation and interaction between users through a system comprising wearable displays, smartphones, and tablet terminals.

Engineering Contradictions & Design Principles

VSEngineering Contradiction Analysis

1Loss of information

If AR technology is used to present additional information superimposed on the real world, then information delivery capability is improved, but spatial position stability deteriorates when viewpoint changes

Engineering Contradiction:
Improveinformation delivery capabilityVSAvoidspatial position stability
Core Design Contradiction:
Loss of informationVSStability of the object's composition

Solution Approach 1:

The patent transitions from 2D screen coordinates to 3D spatial coordinates by introducing depth information and world coordinate systems. This allows annotations to be positioned in three-dimensional space rather than being constrained to the two-dimensional display plane, enabling stable spatial positioning even when the viewpoint changes. The conversion between screen coordinates and world coordinates maintains annotation positions across different viewing angles and distances.

Inventive Principle:
Principle #17Another dimension (Dimensionality change)

Solution Approach 2:

The system continuously tracks the camera's viewpoint changes and adjusts annotation positions in real-time to maintain their spatial relationships. By monitoring changes in the viewing angle, distance, and orientation, the system dynamically recalculates and repositions annotations to ensure they remain stable and correctly aligned with their target objects in the real world.

Inventive Principle:
Principle #23Feedback

2Adaptability or versatility

If the viewpoint changes from moment to moment, then interaction flexibility is improved, but annotation accuracy deteriorates

Engineering Contradiction:
Improveinteraction flexibilityVSAvoidannotation accuracy
Core Design Contradiction:
Adaptability or versatilityVSMeasurement precision

Solution Approach 1:

The system performs preliminary calculations to establish the relationship between screen coordinates and world coordinates before annotations are displayed. By pre-computing transformation matrices and spatial mappings based on the initial viewpoint, the system creates a reference framework that enables accurate annotation placement even as the viewpoint subsequently changes, maintaining precision without restricting flexibility.

Inventive Principle:
Principle #10Preliminary action

Solution Approach 2:

The patent implements a dynamic annotation system that automatically adjusts annotation positions and orientations in response to viewpoint changes. Rather than using static positioning, the system continuously updates annotation parameters based on real-time camera data, allowing annotations to adapt their spatial coordinates, rotation angles, and scale factors to maintain accuracy across different viewing conditions.

Inventive Principle:
Principle #15Dynamics

3Reliability

If spatial position is maintained in a predetermined state, then user experience is improved, but device complexity increases

Engineering Contradiction:
Improveuser experienceVSAvoiddevice complexity
Core Design Contradiction:
ReliabilityVSDevice complexity

Solution Approach 1:

The system introduces coordinate transformation matrices and spatial mapping algorithms as intermediary computational layers between the camera input and annotation display. These intermediaries handle the complex calculations of viewpoint transformation, coordinate system conversion, and spatial relationship maintenance, isolating the complexity from the user interface while ensuring reliable and consistent annotation positioning that enhances user experience.

Inventive Principle:
Principle #24Intermediary (Mediator)

Data Source

PatentUS10832448B2Display control device, display control method, and program
Publication Date: 2020.11.10 SONY GROUP CORP
  • US10832448B2 patent drawing
  • US10832448B2 patent drawing
  • US10832448B2 patent drawing

AI summary

There is provided a display control device including an image acquiring section configured to acquire a moving image shot from a viewpoint changing from moment to moment, a spatial position specifying section configured to specify a spatial position in a first frame of the moving image, and a display control section configured to display the moving image, in such a manner to maintain the spatial position in a predetermined state in a second frame after the first frame.