Automated Assistant Failure Identification via Golden Command Monitoring

Resolve Bottlenecks,
Find Innovative SolutionsGenerate Solutions

Solution Overview

Problem

Automated assistants often experience functional issues due to server-side and client-side updates, leading to unintended changes in their functionality, which can result in futile or errant processing of user requests, requiring efficient detection and remediation of problems to minimize downtime.

Innovation Solution

Implementing an automated assistant failure identification system that determines 'golden commands' for each function, analyzes their performance in real-time, and transmits alerts to endpoints when a threshold of failures is reached, allowing for automatic or user-guided remediation, including the option to disable affected functions to prevent errant processing.

Engineering Contradictions & Design Principles

VSEngineering Contradiction Analysis

1Adaptability or versatility

If automated assistants receive frequent software updates to improve functionality, then the assistant's capabilities are enhanced, but functional issues and unintended changes increase

Engineering Contradiction:

Improveassistant functionalityVSAvoidfunction stability

Core Design Contradiction:

Adaptability or versatilityVSReliability

Solution Approach 1:

The system performs preliminary monitoring of command execution outcomes to detect potential functional issues before they affect users. By continuously analyzing whether commands trigger expected functions, the system can identify problems early and alert developers before widespread deployment causes significant harm.

Inventive Principle:

Principle #10Preliminary action

Solution Approach 2:

The system implements a feedback mechanism where command execution outcomes are continuously monitored and analyzed. When a command fails to trigger its expected function or triggers an unexpected function, the system generates alerts to developers, creating a closed-loop feedback system that enables rapid detection and correction of functional issues.

Inventive Principle:

Principle #23Feedback

2Reliability

If manual testing and monitoring of each command is performed to ensure reliability, then functional accuracy is improved, but the time and resources required increase significantly

Engineering Contradiction:

Improvecommand execution accuracyVSAvoiddetection time

Core Design Contradiction:

ReliabilityVSLoss of time

Solution Approach 1:

The system implements self-service monitoring where the automated assistant monitors its own command execution outcomes without requiring external manual testing. The system automatically tracks whether commands trigger expected functions and generates alerts when issues are detected, eliminating the need for continuous manual verification.

Inventive Principle:

Principle #25Self-service

Solution Approach 2:

The system replaces manual mechanical testing processes with automated electronic monitoring and analysis. Instead of human testers manually executing and verifying each command, the system uses automated stream processing and machine learning algorithms to continuously monitor command outcomes, significantly reducing time and resource requirements.

Inventive Principle:

Principle #28Mechanics substitution (Replace mechanical system)

3Speed

If continuous monitoring of all commands is implemented to detect issues rapidly, then problem detection speed is improved, but computational resources and processing load increase

Engineering Contradiction:

Improveproblem detection speedVSAvoidprocessing resources

Core Design Contradiction:

SpeedVSUse of energy by moving object

Solution Approach 1:

The system implements partial monitoring by focusing on specific golden commands that are most critical to assistant functionality. Rather than analyzing every single command with equal depth, the system prioritizes monitoring of high-impact commands, achieving effective problem detection with reduced processing overhead.

Inventive Principle:

Principle #16Partial or excessive action

Solution Approach 2:

The system segments the monitoring process into distinct components: command identification, execution outcome analysis, and alert generation. By dividing the monitoring task into manageable segments and processing them through a stream processing pipeline, the system achieves efficient resource utilization while maintaining rapid detection capabilities.

Inventive Principle:

Principle #1Segmentation

Data Source

PatentUS11568869B2Low latency automated identification of automated assistant function failure

Publication Date: 2023.01.31 GOOGLE LLC

AI summary

Implementations include identifying, from a database of entries reflecting past automated assistant commands submitted within a threshold amount of time relative to a current time, particular entries that each reflect corresponding features of a corresponding user submission of a particular command. Further, those implementations include determining that the particular command is a golden command, for a particular automated assistant function, responsive to determining that: at least a threshold percentage of the user submissions of the particular command triggered the particular automated assistant function, and a quantity of the user submission of the particular command satisfies a threshold quantity. Those implementations further include, responsive to determining that the particular command is currently the golden command: processing a stream of current occurrences of submissions of the particular command to determine whether the particular automated assistant function is triggered; and when the processing indicates that a threshold quantity and/or percentage of the current occurrences fails to satisfy a potential problem threshold: causing one or more electronic alerts to be transmitted.