Document Reference Tracking in Video Calls

Resolve Bottlenecks,
Find Innovative Solutions
Generate Solutions

Solution Overview

Problem

During video calls, it is difficult for participants to understand where the speaker is referring to in a shared document, and they may miss content not written in the document being discussed.

Innovation Solution

An information processing device that adds supplement information to the document displayed on both speaker and listener terminals, indicating the speaker's position and emphasizing important content, using analysis of voice data and video calls to synchronize the discussion with the document.

Engineering Contradictions & Design Principles

VSEngineering Contradiction Analysis

1Ease of operation

If a document is displayed and explained during video call, then communication between participants is enabled, but it becomes difficult to understand which part of the document the speaker is referring to

Engineering Contradiction:
Improveease of understanding speaker's referenceVSAvoidloss of context information
Core Design Contradiction:
Ease of operationVSLoss of information

Solution Approach 1:

The patent applies color changes by highlighting the corresponding document portion in the color indicated by the virtual callout icon. When a participant creates a virtual callout icon with a specific color to mark a document portion, that same color is applied to the highlighted portion in the shared document display, enabling participants to easily identify and understand which part of the document is being referred to during the video call.

Inventive Principle:
Principle #32Color changes

2Reliability

If participants concentrate on understanding document content, then comprehension is improved, but they may miss content that is not written in the document being discussed

Engineering Contradiction:
Improvereliability of content understandingVSAvoidloss of supplementary information
Core Design Contradiction:
ReliabilityVSLoss of information

Solution Approach 1:

The patent implements feedback by displaying not only the highlighted portion of the document but also the text content of the virtual callout icon that contains supplementary information. When a participant creates a virtual callout icon with additional explanations or notes, this feedback information is displayed alongside the highlighted document portion, ensuring that participants receive both the primary document content and the supplementary context needed for complete understanding.

Inventive Principle:
Principle #23Feedback

3Ease of operation

If no supplement information is added to the document, then the document display remains simple, but participants cannot easily track the speaker's reference position

Engineering Contradiction:
Improveease of tracking speaker referenceVSAvoidcomplexity of document display
Core Design Contradiction:
Ease of operationVSDevice complexity

Solution Approach 1:

The patent applies segmentation by dividing the document display into distinct visual elements: the original document content, the virtual callout icons created by participants, and the highlighted portions with color-coded indicators. This segmentation allows each element to serve its specific function - the document provides base content, the callout icons provide interactive markers, and the highlighted portions with color codes provide real-time reference tracking - making the system easier to use without overwhelming complexity.

Inventive Principle:
Principle #1Segmentation

Data Source

PatentUS20240347045A1Information processing device, information processing method, and program
Publication Date: 2024.10.17 SONY GROUP CORP
  • US20240347045A1 patent drawing
  • US20240347045A1 patent drawing
  • US20240347045A1 patent drawing

AI summary

Provided are an information processing device, an information processing method, and a program that enable a person to easily grasp which part of a document another person is uttering about when the person and the other person perform conversation while referring to a common document. The information processing device includes a supplement processing unit that adds supplement information to a document displayed on an utterer terminal device used by an utterer and a listener terminal device used by a listener who performs conversation with the utterer in accordance with information regarding the conversation or the document.