Digital Assistant Context Enrichment With Reference-Based App Data
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Traditional chatbots rely on predefined skill or intent models, limiting their ability to engage intelligently and contextually in live conversations, and the integration of Large Language Models (LLMs) into digital assistants enhances their capabilities but poses challenges in efficiently capturing and processing user interaction context without impacting client device performance.
Innovation Solution
Passing data by reference instead of value, using identifiers like UUIDs to capture application context, and combining it with other data to generate composite context for LLMs, reducing data transfer volume and maintaining performance.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If application context data is captured and processed for LLM enrichment, then contextual accuracy and conversation quality improve, but client device performance and latency are negatively impacted
Solution Approach 1:
The patent extracts only the essential context data needed for LLM enrichment while leaving the bulk of data processing on the server side. This selective extraction reduces the amount of data that needs to be transferred and processed on the client device, thereby reducing latency while maintaining contextual accuracy.
Solution Approach 2:
The context enrichment process is segmented into distinct phases: initial context capture, selective data extraction, and server-side processing. This segmentation allows the client device to complete its part quickly and transfer control to the server for heavier processing, reducing overall latency while maintaining accuracy.
2Measurement precision
If comprehensive application context is captured for digital assistant interactions, then contextual accuracy improves, but data transfer volume increases
Solution Approach 1:
The system extracts only the most relevant context elements needed for accurate digital assistant interactions, such as key user preferences, current application state, and recent interaction history. This selective extraction maintains contextual accuracy while significantly reducing the volume of data that needs to be transferred.
Solution Approach 2:
Different types of context data are processed with different levels of detail based on their importance. Critical context elements are captured in full detail, while less critical elements are summarized or omitted, optimizing the balance between contextual accuracy and data transfer volume.
3Loss of information
If detailed user interaction context is processed on client device, then contextual richness improves, but processing overhead and performance impact increase
Solution Approach 1:
The client device performs lightweight extraction of essential context features and transfers them to the server for comprehensive processing. This division of labor maintains contextual richness while minimizing the processing overhead on the client device, as the heavy lifting is performed on the server side.
Solution Approach 2:
The client device performs preliminary context capture and filtering before data transfer, preparing only the most relevant information for server-side processing. This preliminary action reduces the amount of data that needs to be processed end-to-end, lowering overall processing overhead while maintaining contextual richness.
Data Source
AI summary
Techniques are disclosed for capturing information surrounding a user's interactions with an application for enriching application context to be used by a digital assistant for supporting the user's interactions. In one aspect, a method includes detecting an event from a client device, where the event is associated with a user's interaction with an application on the client device. In response to detecting the event, application context is obtained for the application. The obtaining includes accessing the application context from a data store based on an identifier. A first generative artificial intelligence model can then be used to generate a list having an executable action based on the event and the application context. An execution plan is then created and executed, which includes executing the executable action using an asset to obtain an output. The output or a communication derived from the output is sent to the client device.