Response Retrieval System Using Vector Similarity for Dialogue Flexibility
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing dialogue systems face challenges in data collection for labeled semantic representations and are limited by rigid ontologies, leading to artificial conversations and breakdowns when users exceed system capabilities.
Innovation Solution
A response retrieval system that encodes user queries into context vectors and compares them to pre-computed response vectors, allowing for the retrieval of relevant responses without constructing answers from basic components, using a model trained on millions of conversations to optimize similarity between queries and responses.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Device complexity
If rigid ontologies are used to model dialogue, then the system structure is simplified, but the system becomes constrained and produces artificial conversations
Solution Approach 1:
The patent changes the fundamental parameter of representation from rigid structured formats to continuous vector embeddings. This allows the system to maintain simplicity in computation while achieving flexibility in dialogue, as vectors can represent any semantic concept without predefined structural constraints.
Solution Approach 2:
Instead of constraining dialogue to fit a predefined ontology structure, the system inverts the approach by allowing the ontology to adapt to the dialogue. The vector representation system enables any user input to be mapped to appropriate responses without forcing the conversation into predetermined slots.
2Manufacturing precision
If rigid ontologies with explicit semantic representations are used, then the system can provide structured responses, but data collection becomes difficult and expensive
Solution Approach 1:
The patent uses pre-trained language models that have already copied and learned from vast amounts of unlabeled data. Instead of collecting and manually labeling data for each semantic representation, the system leverages existing pre-trained models that capture semantic meanings through their vector embeddings, eliminating the need for expensive annotation processes.
Solution Approach 2:
The system performs self-service by using the pre-trained model's inherent understanding of language and semantics. The model automatically generates appropriate vector representations and responses without requiring external data collection or human annotation, as it has already learned these patterns during its pre-training phase.
3Adaptability or versatility
If responses are constructed from basic components using language models, then the system can generate diverse answers, but processing time and computational load increase
Solution Approach 1:
The patent applies preliminary action by pre-computing and storing response vectors in a database during an offline process. During actual conversation, the system only needs to retrieve and match pre-computed vectors rather than generating responses in real-time, significantly reducing processing time while maintaining response diversity.
Solution Approach 2:
The system segments the response generation process into two phases: an offline phase where diverse responses are pre-processed and vectorized, and an online phase where only retrieval and matching occur. This segmentation allows the system to maintain response diversity without the computational burden of generating diverse responses during actual user interaction.
Data Source
AI summary
A method of obtaining a response to a query inputted by a user, the method comprising:receiving a user inputted query;encoding said query to produce a context vector;retrieving responses with associated response vectors;scoring response vectors in the database against the context vector wherein the scoring is a measure of the similarity between the context vector and a response vector; andoutputting the responses with the closest response vectors,wherein encoding said query to produce a context vector comprises using a pre-trained model, wherein said pre-trained model has been trained using corresponding queries and responses such that an encoding is used that maximises the similarity between the response vector and context vector for a corresponding query and response.