AR Response Generation Using Context-Relevance Memory Models
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Conventional AR chatbots struggle with processing large and diverse input texts, generating coherent and relevant responses, and incorporating visual and spatial cues from the AR context, limiting their flexibility and naturalness.
Innovation Solution
Integrate a Continuous Attention Memory Model (CAMM) with existing Large Language Models (LLMs) to enhance their ability to process infinite-length inputs, utilize a dynamic memory bank, and incorporate context relevance estimation, enabling the generation of realistic and engaging AR chatbot responses.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Loss of information
If conventional AR chatbots process large and diverse input texts, then they can understand complex user queries, but they struggle to generate coherent and relevant responses
Solution Approach 1:
The patent segments the input text processing into multiple stages: initial processing by a first LLM, relevance filtering by a second LLM, and final response generation by a third LLM. This segmentation allows each model to focus on specific aspects of the input, improving overall response coherence while handling large inputs effectively.
Solution Approach 2:
The patent introduces an intermediary relevance filtering stage using a second LLM that acts as a mediator between the input processing and final response generation. This intermediary component filters and prioritizes relevant information from the input text, ensuring that only pertinent information is used in generating coherent and relevant responses.
2Adaptability or versatility
If conventional AR chatbots incorporate visual and spatial cues from the AR context, then they can provide more contextual responses, but their flexibility and naturalness are limited
Solution Approach 1:
The patent makes the LLM pipeline universal by enabling it to process multiple types of inputs (text, visual cues, spatial information) through a unified architecture. The models are trained to handle diverse input modalities and generate natural responses across different AR contexts, improving both adaptability and ease of operation simultaneously.
Solution Approach 2:
The patent changes the operational parameters of the LLMs by fine-tuning them on AR-specific data and adjusting their attention mechanisms to prioritize relevant contextual information. This parameter optimization enables the models to naturally incorporate visual and spatial cues while maintaining flexible and natural response generation.
3Reliability
If the system uses multiple LLMs in sequence, then response relevance and coherence improve, but processing time increases
Solution Approach 1:
The patent applies partial action by having the second LLM perform a focused relevance filtering function rather than generating complete responses. This partial processing approach quickly identifies and prioritizes relevant information without the time cost of generating full responses for all inputs, thus improving relevance while controlling processing time.
Solution Approach 2:
The patent ensures continuity of useful action by maintaining an active relevance filtering mechanism that continuously processes input text to identify relevant information before response generation. This continuous filtering operation ensures that only pertinent information is processed further, optimizing the balance between response quality and processing efficiency.
Data Source
AI summary
An apparatus comprises at least one processing device configured to generate, using a first machine learning model, a first data structure comprising input representations of one or more input components from an augmented reality environment. The at least one processing device is also configured to generate, using a second machine learning model that takes as input at least a portion of the first data structure, a second data structure comprising at least one vector representation characterizing relevance of one or more of the input representations in the first data structure. The at least one processing device is further configured to generate, using a third machine learning model that takes as input at least a portion of the first data structure and at least a portion of the second data structure, an output response, and to present the output response to a user in the augmented reality environment.