Live Session Response Generation With Vector Retrieval Context

Resolve Bottlenecks,
Find Innovative SolutionsGenerate Solutions

Solution Overview

Problem

Existing communication systems lack the ability to effectively utilize previous interaction content to generate tailored and accurate responses during live sessions, failing to leverage contextual attributes and historical data for enhanced customer interactions.

Innovation Solution

A host platform utilizing large language models (LLMs) to identify contextual attributes from communication sessions, convert interaction content into vectorized data, and store it in a vector database for retrieval and response generation, enabling real-time, contextually rich interactions.

Engineering Contradictions & Design Principles

VSEngineering Contradiction Analysis

1Measurement precision

If traditional communication systems are used, then system simplicity is maintained, but the ability to generate accurate and customized responses using previous interaction content is lost

Engineering Contradiction:

Improveresponse accuracyVSAvoidsystem complexity

Core Design Contradiction:

Measurement precisionVSDevice complexity

Solution Approach 1:

The system segments the complex task of response generation into distinct functional modules: a vector database stores historical interaction data, an LLM processes current queries, and a response generation component synthesizes outputs. This segmentation allows each component to specialize in specific functions, improving response accuracy while managing system complexity through modular architecture.

Inventive Principle:

Principle #1Segmentation

Solution Approach 2:

The system performs preliminary actions by storing and processing historical interaction content in advance, converting it into vector representations and saving it in a vector database. This pre-processing enables the LLM to quickly retrieve and utilize relevant context during live sessions, improving response accuracy without adding complexity to real-time operations.

Inventive Principle:

Principle #10Preliminary action

2Loss of information

If traditional communication systems are used, then processing speed is maintained, but the ability to leverage contextual attributes and historical data is lost

Engineering Contradiction:

Improvecontext retentionVSAvoidprocessing time

Core Design Contradiction:

Loss of informationVSLoss of time

Solution Approach 1:

The system creates copies of historical interaction content in vector format within a vector database, enabling efficient retrieval and comparison with current queries. This copying mechanism allows the system to leverage contextual attributes and historical data without requiring access to original large-scale data stores, thus reducing processing time while improving context retention.

Inventive Principle:

Principle #26Copying

Solution Approach 2:

The system replaces traditional mechanical search and retrieval mechanisms with LLM-based semantic processing. Instead of keyword matching, the LLM understands and processes contextual meanings, enabling efficient retrieval of relevant historical data while maintaining comprehensive context retention with reduced processing overhead.

Inventive Principle:

Principle #28Mechanics substitution (Replace mechanical system)

3Adaptability or versatility

If LLMs are executed on interaction content to identify contextual attributes, then response customization is improved, but computational energy consumption increases

Engineering Contradiction:

Improveresponse customizationVSAvoidcomputational energy

Core Design Contradiction:

Adaptability or versatilityVSUse of energy by moving object

Solution Approach 1:

The system applies partial action by selectively processing only the most relevant portions of interaction content through LLMs. Instead of analyzing entire historical datasets, the system identifies and processes key contextual attributes and segments, achieving effective response customization while significantly reducing computational energy consumption compared to exhaustive processing.

Inventive Principle:

Principle #16Partial or excessive action

4Loss of information

If vectorized data is stored in a vector database for retrieval, then information accessibility is improved, but data storage complexity increases

Engineering Contradiction:

Improveinformation accessibilityVSAvoiddata storage complexity

Core Design Contradiction:

Loss of informationVSDevice complexity

Solution Approach 1:

The system changes the parameter representation of stored data by converting textual interaction content into vector formats. This parameter transformation enables efficient similarity searching and retrieval operations in the vector database, improving information accessibility while managing storage complexity through optimized data structures and indexing methods designed for vector data.

Inventive Principle:

Principle #35Parameter changes

Data Source

PatentUS12541544B2Generating a response for a communication session based on previous conversation content using a large language model

Publication Date: 2026.02.03 THE TORONTO DOMINION BANK

AI summary

An example operation may include one or more of receiving interaction content from a communication session between a source device and a service provider device of a service provider, identifying a search criteria from the interaction content, retrieving a subset of vectors from a plurality of vectors stored in a vector database based on the search criteria of the interaction content, wherein the subset of vectors includes previous interaction content with the service provider, generating a response for the communication session based on execution of a large language model (LLM) on the subset of vectors, and outputting the response to at least one of the source device and the service provider device during the communication session.