Message Output Apparatus Using ML for Real-Time Communication Simulation

Resolve Bottlenecks,
Find Innovative Solutions
Generate Solutions

Solution Overview

Problem

Users, such as players or audiences, often feel isolated during live game or sports broadcasting, lacking the sense of communication which can diminish their enjoyment.

Innovation Solution

A message output apparatus utilizing a learned machine learning model that inputs consecutive frames of images and player information to identify and output messages, simulating communication during gameplay or sports viewing.

Engineering Contradictions & Design Principles

VSEngineering Contradiction Analysis

1Measurement precision

If a machine learning model processes multiple consecutive frames of images to identify messages, then the accuracy of message identification is improved, but the processing time and computational load increase

Engineering Contradiction:
Improvemessage identification accuracyVSAvoidprocessing time
Core Design Contradiction:
Measurement precisionVSLoss of time

Solution Approach 1:

The system performs preliminary actions by pre-processing and storing key features from multiple consecutive frames before message identification is needed. The machine learning model is trained in advance with大量 labeled data, so when actual message identification occurs, the model can quickly retrieve and match features without processing all raw frames in real-time, thus improving accuracy while reducing processing time.

Inventive Principle:
Principle #10Preliminary action

Solution Approach 2:

The video stream is segmented into multiple consecutive frames that are processed individually but collectively contribute to message identification. By dividing the continuous video input into discrete frame segments, the system can analyze temporal patterns across frames without overwhelming computational resources, balancing accuracy improvement with manageable processing time.

Inventive Principle:
Principle #1Segmentation

2Adaptability or versatility

If player information including controller inputs and face images is collected and processed, then the personalization and user experience are improved, but the device complexity and data processing requirements increase

Engineering Contradiction:
Improveuser experience personalizationVSAvoiddata processing complexity
Core Design Contradiction:
Adaptability or versatilityVSDevice complexity

Solution Approach 1:

The machine learning model serves multiple functions: it processes game status images, player face images, and controller input data simultaneously to identify messages. This multi-functional approach consolidates what would otherwise require separate processing systems, improving user experience personalization while managing device complexity through a unified model architecture.

Inventive Principle:
Principle #6Universality (Multi-functionality)

Solution Approach 2:

The machine learning model acts as an intermediary that receives diverse inputs (game images, face images, controller data) and transforms them into unified message identifications. This intermediary layer simplifies the overall system architecture by handling the complexity of multi-source data integration within the model itself, rather than requiring complex external processing systems.

Inventive Principle:
Principle #24Intermediary (Mediator)

Data Source

PatentUS11711328B2Message output apparatus, learning apparatus, message output method, learning method, and program
Publication Date: 2023.07.25 SONY INTERACTIVE ENTERTAINMENT LLC
  • US11711328B2 patent drawing
  • US11711328B2 patent drawing
  • US11711328B2 patent drawing

AI summary

Provided are a message output apparatus, a learning apparatus, a message output method, a learning method, and a program that allow a user to feel as if communication is being performed. An input section inputs, to a learned machine learning model that has been learned using learning data that includes learning input data including a plurality of consecutive frames of learning images and teaching data including a message associated with the learning input data, target input data including at least a plurality of consecutive frames of target images. A message identification section identifies a message according to output produced when the target input data is input to the machine learning model. A message output section outputs the identified message.