Automated Assistant Reply Generation Using User State

Resolve Bottlenecks,
Find Innovative SolutionsGenerate Solutions

Solution Overview

Problem

Conventional automated assistants fail to effectively incorporate user state information into their responses, leading to inefficient dialog interactions that consume excessive computational resources and may not provide relevant interactive elements, resulting in user dissatisfaction.

Innovation Solution

The method involves generating reply content based on both textual input and user state information, using sensors like cameras and microphones to determine user sentiment and other states, and modifying initial textual outputs from text generation engines to create tailored responses, including interactive elements that align with the user's state.

Engineering Contradictions & Design Principles

VSEngineering Contradiction Analysis

1Adaptability or versatility

If conventional automated assistants generate responses based only on textual input, then the system complexity is low, but the relevance and effectiveness of the dialog responses deteriorates

Engineering Contradiction:

Improverelevance of dialog responsesVSAvoidsystem complexity

Core Design Contradiction:

Adaptability or versatilityVSDevice complexity

Solution Approach 1:

The system segments the response generation process into multiple independent modules: a text generation engine that creates initial responses, a user state analysis module that processes sensor data, and a response modification module that integrates both inputs. This segmentation allows each module to specialize in one function while maintaining overall system adaptability.

Inventive Principle:

Principle #1Segmentation

Solution Approach 2:

The patent introduces a response modification module as an intermediary between the text generation engine and the final output. This mediator takes the initial textual response and modifies it based on user state information from sensors, thereby improving response relevance without requiring complete system redesign.

Inventive Principle:

Principle #24Intermediary (Mediator)

2Adaptability or versatility

If automated assistants use multiple sensors to determine user state, then the adaptability to user context improves, but the computational resource consumption increases

Engineering Contradiction:

Improveuser context awarenessVSAvoidcomputational resource consumption

Core Design Contradiction:

Adaptability or versatilityVSUse of energy by moving object

Solution Approach 1:

The system performs preliminary analysis of sensor data to determine user state before generating the final response. By pre-processing sensor inputs and identifying key user state indicators in advance, the system avoids unnecessary computational operations during the main response generation phase, thereby reducing overall computational resource consumption.

Inventive Principle:

Principle #10Preliminary action

Solution Approach 2:

The system selectively processes only the necessary sensor data required for determining user state, rather than analyzing all available sensor inputs comprehensively. This partial action approach focuses computational resources on the most relevant user state indicators, achieving adequate context awareness with reduced energy consumption.

Inventive Principle:

Principle #16Partial or excessive action

3Ease of operation

If automated assistants provide generic responses without user state information, then the computational resources are conserved, but the user satisfaction and engagement deteriorates

Engineering Contradiction:

Improveuser satisfactionVSAvoiddialog efficiency

Core Design Contradiction:

Ease of operationVSProductivity

Solution Approach 1:

The system dynamically changes the parameters of the response based on detected user state. Instead of providing static generic responses, the response modification module adjusts the textual content, tone, and style parameters according to real-time user state information from sensors, thereby improving user satisfaction while maintaining dialog efficiency.

Inventive Principle:

Principle #35Parameter changes

4Adaptability or versatility

If the system modifies initial textual output based on user state, then the dialog relevance improves, but the processing time increases

Engineering Contradiction:

Improvedialog relevanceVSAvoidprocessing time

Core Design Contradiction:

Adaptability or versatilityVSLoss of time

Solution Approach 1:

The system performs preliminary determination of user state from sensor data before the text generation engine creates the initial response. By having user state information ready in advance, the response modification process can proceed efficiently without adding significant processing time, as the foundational user context is already established.

Inventive Principle:

Principle #10Preliminary action

Data Source

PatentUS11929072B2Using textual input and user state information to generate reply content to present in response to the textual input

Publication Date: 2024.03.12 GOOGLE LLC

AI summary

Methods, apparatus, and computer readable media related to receiving textual input of a user during a dialog between the user and an automated assistant (and optionally one or more additional users), and generating responsive reply content based on the textual input and based on user state information. The reply content is provided for inclusion in the dialog. In some implementations, the reply content is provided as a reply, by the automated assistant, to the user's textual input and may optionally be automatically incorporated in the dialog between the user and the automated assistant. In some implementations, the reply content is suggested by the automated assistant for inclusion in the dialog and is only included in the dialog in response to further user interface input.