Virtual Assistant Response Harmonization via Confidence Score Normalization
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Virtual assistants provide inconsistent responses due to different confidence score scales and domain expertise, leading to user frustration and increased processing resources in selecting accurate answers.
Innovation Solution
A method that normalizes confidence scores across multiple virtual assistants and assigns domain-specific weights to rank responses accurately, leveraging a learning-based approach to determine the relevant domain and adapt weights based on user inputs.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Adaptability or versatility
If multiple virtual assistants are used to answer user queries, then the coverage of domains and expertise are improved, but the confidence scores from different assistants are on different scales making it difficult to select the best response
Solution Approach 1:
The patent applies parameter changes by transforming confidence scores from different scales into a normalized common scale. Each assistant's confidence score is adjusted using learned normalization parameters specific to their domain expertise, enabling direct comparison while preserving the original scale variations that reflect domain-specific confidence levels.
Solution Approach 2:
The patent introduces an intermediary normalization layer between the multiple virtual assistants and the final response selection. This intermediary component learns domain-specific normalization parameters that mediate the comparison process, allowing confidence scores from different assistants to be fairly evaluated without directly comparing raw scores on different scales.
2Reliability
If domain-specific weights are assigned to personal assistants, then the accuracy of response selection is improved, but the complexity of the system increases
Solution Approach 1:
The patent applies preliminary action by pre-learning domain-specific weights and normalization parameters during a training phase before actual query processing. This preliminary learning phase creates a knowledge base that simplifies the runtime operation, where the system only needs to apply pre-determined weights rather than calculating them in real-time, thus reducing operational complexity while maintaining high accuracy.
Solution Approach 2:
The system performs self-service by automatically learning and adapting domain-specific weights through the training process without requiring manual configuration. The normalization parameters and domain weights are derived automatically from training data, allowing the system to self-optimize its response selection mechanism while managing its own complexity through automated learning rather than manual setup.
3Ease of operation
If confidence scores are normalized to a common scale, then the selection process is simplified, but the domain-specific nuances of each assistant's confidence are lost
Solution Approach 1:
The patent applies local quality by implementing domain-specific normalization parameters that are tailored to each assistant's expertise area. Instead of applying a single universal normalization, each domain has its own normalization characteristics that preserve local confidence nuances while still enabling comparison. This allows the system to maintain domain-specific confidence characteristics while achieving operational simplicity through the unified normalization framework.
Data Source
AI summary
The present disclosure relates to a method of coordinating personal assistants. The method comprises: sending a request to a set of personal assistants based on a received query. Assistant responses may be received in response to the request from respective personal assistants of the set of personal assistants. Each assistant response of the assistant responses comprises a confidence score of the respective assistant response. The confidence scores may be adjusted to a common scale. The adjusted confidence scores may be weighted using domain-specific weights of the personal assistants in accordance with the domain of the query. One or more assistant responses of the received assistant responses may be selected based on the weighted scores and the selected assistant responses may be provided to the user.


