Augmented Reality Server System for Multi-User Virtual Process Sharing

Resolve Bottlenecks,
Find Innovative Solutions
Generate Solutions

Solution Overview

Problem

Existing augmented reality technologies do not enable multiple users to share a virtual process for a real object fixedly arranged in the real world, despite advancements in sharing virtual objects.

Innovation Solution

An information processing system that includes a server apparatus and terminal devices connected via a communication network, where the server apparatus obtains positional and image data from users, specifies a target object, and allows users to input and share drawing information to superimpose virtual content onto the real object, enabling multiple users to collaborate on a virtual process using augmented reality.

Engineering Contradictions & Design Principles

VSEngineering Contradiction Analysis

1Adaptability or versatility

If multiple users share virtual objects using augmented reality technology, then user interaction and collaboration capability are improved, but the system complexity and difficulty of coordinating multiple users' viewpoints are increased

Engineering Contradiction:
Improveuser collaboration capabilityVSAvoidsystem complexity
Core Design Contradiction:
Adaptability or versatilityVSDevice complexity

Solution Approach 1:

The patent introduces a server as an intermediary that receives image data from multiple terminal devices, performs centralized processing to generate appropriate viewpoint images, and distributes them back to users. This mediator approach resolves the complexity of direct peer-to-peer coordination by centralizing the computational burden and viewpoint transformation logic on the server side.

Inventive Principle:
Principle #24Intermediary (Mediator)

Solution Approach 2:

The system creates virtual copies of the real-world scene from different viewpoints through image processing. Instead of requiring users to physically move or exchange positions, the server generates synthesized images that replicate what each user would see from optimal viewing angles, enabling shared experience without physical coordination.

Inventive Principle:
Principle #26Copying

2Adaptability or versatility

If virtual content is superimposed onto real objects for augmented reality display, then immersive experience and user engagement are improved, but the precision of positioning and alignment is increased

Engineering Contradiction:
Improveimmersive experienceVSAvoidpositioning precision
Core Design Contradiction:
Adaptability or versatilityVSMeasurement precision

Solution Approach 1:

The patent replaces physical positioning mechanisms with image processing and computational methods. Instead of relying on precise mechanical alignment of displays with real-world objects, the system uses 2D image data and viewpoint transformation algorithms to achieve accurate virtual content placement, substituting optical-mechanical precision with computational geometry.

Inventive Principle:
Principle #28Mechanics substitution (Replace mechanical system)

Solution Approach 2:

The server acts as an intermediary that processes raw image data from terminal devices, performs viewpoint transformations, and generates corrected images with properly positioned virtual content. This centralized processing ensures consistent positioning precision across multiple users without requiring each terminal to independently achieve high measurement accuracy.

Inventive Principle:
Principle #24Intermediary (Mediator)

3Ease of operation

If augmented reality technology is used to display virtual objects in real space, then user interaction capability is improved, but the complexity of coordinating multiple users' viewing positions is increased

Engineering Contradiction:
Improveuser interaction capabilityVSAvoidviewpoint coordination complexity
Core Design Contradiction:
Ease of operationVSDevice complexity

Solution Approach 1:

The system creates virtual representations of the real-world scene from multiple viewpoints through image processing. Each user receives a customized viewpoint image that replicates their perspective, allowing natural interaction without requiring the system to physically adapt to each user's position. The virtual copy approach simplifies interaction while the server handles viewpoint coordination complexity.

Inventive Principle:
Principle #26Copying

Data Source

PatentEP3547083B1Information processing program, information processing method, and information processing system
Publication Date: 2023.05.24 SQUARE ENIX HLDG CO LTD
  • EP3547083B1 patent drawingFigure 1
  • EP3547083B1 patent drawingFigure 2
  • EP3547083B1 patent drawingFigure 3

AI summary

A server apparatus obtains positional information of a first terminal device and photographed image data on an image photographed by using the first terminal device. The server apparatus specifies a target object included in the photographed image data. The server apparatus obtains drawing information indicating content of a drawing process against the target object via the first terminal device. The server apparatus outputs superimposed information to the first terminal device or a second terminal device before a first user of the first terminal device or a second user of the second terminal device photographs the target object to which the content of the drawing process is attached, the superimposed information being used to cause the first terminal device or the second terminal device to superimpose the drawing information onto the target object included in the photographed image data and display the superimposed target object.