Symmetry-Based Viewpoint Conversion for Personalized Eye Contact

Resolve Bottlenecks,
Find Innovative Solutions
Generate Solutions

Solution Overview

Problem

Telecommunication systems face challenges in accurately performing viewpoint conversion for unspecified users due to the trade-off between personalized and versatile learning, leading to decreased accuracy in eye contact and visual alignment during remote communications.

Innovation Solution

An information processing system utilizes the symmetry of imaging device arrangements to perform personalized learning by acquiring teacher and student images, enhancing viewpoint conversion accuracy through a conversion model that integrates symmetry-based learning and machine learning techniques.

Engineering Contradictions & Design Principles

VSEngineering Contradiction Analysis

1Measurement precision

If the telecommunication system learns the viewpoint conversion specialized for a specific person, then the accuracy of viewpoint conversion for that specific person is improved, but the versatility for unspecified persons is deteriorated

Engineering Contradiction:
Improveviewpoint conversion accuracyVSAvoidversatility for unspecified persons
Core Design Contradiction:
Measurement precisionVSAdaptability or versatility

Solution Approach 1:

The learning process is segmented into two distinct phases: a first learning phase that performs viewpoint conversion for unspecified persons using general training data, and a second learning phase that performs personalized learning for specific persons using teacher images and student images. This segmentation allows the system to maintain versatility for general users while achieving high accuracy for specific individuals through targeted personalization.

Inventive Principle:
Principle #1Segmentation

2Adaptability or versatility

If the telecommunication system learns the viewpoint conversion versatilely without specifying a person, then the versatility for unspecified persons is improved, but the accuracy of viewpoint conversion for a specific person is deteriorated

Engineering Contradiction:
Improveversatility for unspecified personsVSAvoidviewpoint conversion accuracy
Core Design Contradiction:
Adaptability or versatilityVSMeasurement precision

Solution Approach 1:

The system performs preliminary learning in the first learning phase to establish a baseline viewpoint conversion capability for unspecified persons. This preliminary action creates a general-purpose conversion model that ensures versatility. Subsequently, the second learning phase performs personalized adjustments based on teacher images, thereby improving accuracy for specific persons without completely replacing the versatile baseline capability.

Inventive Principle:
Principle #10Preliminary action

3Measurement precision

If the telecommunication system performs personalized learning for specific persons, then the accuracy of viewpoint conversion is improved, but the learning data requirements and processing complexity increase

Engineering Contradiction:
Improveviewpoint conversion accuracyVSAvoidlearning processing complexity
Core Design Contradiction:
Measurement precisionVSDevice complexity

Solution Approach 1:

The system employs self-service mechanisms where teacher images are automatically captured by the imaging device when the user is in the teaching position, and the learning processor automatically performs personalized learning using these teacher images and corresponding student images. This self-service approach reduces manual intervention and simplifies the complexity of personalized learning deployment.

Inventive Principle:
Principle #25Self-service

Data Source

PatentUS20250227362A1Information processing apparatus, information processing method, and recording medium
Publication Date: 2025.07.10 SONY GROUP CORP
  • US20250227362A1 patent drawing
  • US20250227362A1 patent drawing
  • US20250227362A1 patent drawing

AI summary

An information processing apparatus (100) includes a controller (130). The controller (130) controls an output device to guide a user to oppose a first imaging device of the first imaging device and a second imaging device arranged to have symmetry with respect to a virtual camera set for a display. The controller (130) acquires a teacher image including the user opposing the first imaging device from the first imaging device. The controller (130) performs learning processing of viewpoint conversion of the second imaging device to correspond to a viewpoint of the virtual camera on the basis of the teacher image and the symmetry of the first imaging device and the second imaging device with respect to the virtual camera.