Viewpoint Image Rendering for Mobile AR Interaction Delay

Resolve Bottlenecks,
Find Innovative SolutionsGenerate Solutions

Solution Overview

Problem

The large amount of data required for displaying a 360-degree image causes delays in interaction when transferring field of view data from a mobile terminal to a display terminal, especially in augmented reality monitoring services where users are at different positions.

Innovation Solution

An information processing device that acquires a viewpoint image from a street image server based on the position of a mobile terminal and generates a rendering image from a free viewpoint specified by a second user, reducing the need for direct data transfer from the mobile terminal and minimizing data usage.

Engineering Contradictions & Design Principles

VSEngineering Contradiction Analysis

1Reliability

If 360-degree image data is transferred from mobile terminal to display terminal, then the field of view can be displayed, but the large amount of data causes delay in interaction

Engineering Contradiction:

Improvefield of view display accuracyVSAvoidinteraction delay

Core Design Contradiction:

ReliabilityVSLoss of time

Solution Approach 1:

The system pre-acquires 360-degree viewpoint images from an image server before they are needed for display. By having the image server store and provide pre-processed viewpoint images, the mobile terminal does not need to capture and transfer large amounts of raw 360-degree image data in real-time, thus reducing interaction delay while maintaining display accuracy

Inventive Principle:

Principle #10Preliminary action

Solution Approach 2:

An image server is introduced as an intermediary between the mobile terminal and display terminal. The image server stores pre-acquired 360-degree viewpoint images and provides them to the mobile terminal on demand, eliminating the need for direct real-time data transfer from the mobile terminal to display terminal and reducing interaction delay

Inventive Principle:

Principle #24Intermediary (Mediator)

2Loss of information

If mobile terminal captures 360-degree images, then field of view data is available, but security risks from unintended objects in the view increase

Engineering Contradiction:

Improveviewpoint image availabilityVSAvoidsecurity risk from unintended objects

Core Design Contradiction:

Loss of informationVSObject-affected harmful factors

Solution Approach 1:

The system pre-acquires 360-degree viewpoint images from the image server before they are needed. By using pre-stored images rather than real-time capture, the system avoids the security risks associated with capturing unintended objects while ensuring viewpoint image availability for display

Inventive Principle:

Principle #10Preliminary action

Solution Approach 2:

Instead of capturing original 360-degree images with the mobile terminal camera, the system uses copied viewpoint images from the image server. These pre-existing images provide the necessary visual information without the security risks of real-time capture, as they do not include unintended objects that may be present in the current environment

Inventive Principle:

Principle #26Copying

Data Source

PatentUS12106439B2Device and associated methodology for suppressing interaction delay of interacting with a field of view of a mobile terminal on a different display

Publication Date: 2024.10.01 SONY GROUP CORP

AI summary

An information processing device includes an acquisition unit, a generation unit, and an output unit. The acquisition unit acquires a viewpoint image according to a viewpoint of a first user from an image server, on the basis of position information indicating the position of a mobile terminal moving with the first user. The image server stores a plurality of viewpoint images obtained by capturing images from a plurality of orientations in each of a plurality of positions. The generation unit uses the acquired viewpoint image to generate a rendering image rendered from a free viewpoint specified by a second user who is located at a position different from that of the first user, as a display image. The output unit outputs the generated display image.