AI Content Tracking for Real-Time Playback Query Recall

Resolve Bottlenecks,
Find Innovative SolutionsGenerate Solutions

Solution Overview

Problem

Individuals may struggle to remember content details in lengthy audio-only or audio-video programs due to the lack of available summaries and limited connectivity, leading to disengagement from the program.

Innovation Solution

An apparatus and method utilizing machine learning models, particularly large language models, to generate metadata based on content consumption, allowing users to query and receive responses about the content in real-time, even without internet connectivity.

Engineering Contradictions & Design Principles

VSEngineering Contradiction Analysis

1Loss of information

If the content is made lengthy and detailed, then the information completeness is improved, but the user's ability to remember and follow the content deteriorates

Engineering Contradiction:

Improvecontent detailsVSAvoiduser engagement

Core Design Contradiction:

Loss of informationVSEase of operation

Solution Approach 1:

The patent segments the lengthy content into manageable portions by training the ML model on content consumed in discrete time intervals. The system processes and stores information in segments, allowing users to query specific portions without needing to remember the entire content, thus resolving the contradiction between information completeness and user engagement.

Inventive Principle:

Principle #1Segmentation

Solution Approach 2:

The patent introduces an intermediary ML model that acts as a mediator between the lengthy content and the user. The model stores and processes content details, then retrieves and presents relevant information in response to user queries, eliminating the need for users to directly remember or follow every detail of the lengthy content.

Inventive Principle:

Principle #24Intermediary (Mediator)

2Speed

If the ML model processes content in real-time, then the response time is improved, but the computational resources required increase

Engineering Contradiction:

Improveresponse timeVSAvoidcomputational resources

Core Design Contradiction:

SpeedVSUse of energy by moving object

Solution Approach 1:

The patent applies preliminary action by training the ML model on content as it is being consumed, before the user needs to query about it. The model processes and stores information in advance during the content playback, enabling fast real-time responses without requiring intensive computational resources at the moment of querying.

Inventive Principle:

Principle #10Preliminary action

Solution Approach 2:

The patent maintains continuous useful action by processing content information continuously during playback rather than batch-processing later. The ML model continuously learns from and stores content details as they are presented, ensuring readiness for immediate user queries while distributing computational load over time.

Inventive Principle:

Principle #20Continuity of useful action

Data Source

PatentUS12520017B2Tracking content with artifical intelligence as it is consumed

Publication Date: 2026.01.06 SONY GROUP CORP

AI summary

A user starts playback of an audio or AV program, which starts up a large language model (LLM) such as a generative pre-trained transformer. The LLM follows the program with the user as the program is being watched. Audio is converted to text and video is converted to text-based description. The LLM essentially trains on the resulting “corpus” so that should the user subsequently want to access part of the program using a simple conversational query about the program, the LLM can provide the answer.