Generative AI Answer Quality Control Through Human Feedback

Resolve Bottlenecks,
Find Innovative SolutionsGenerate Solutions

Solution Overview

Problem

Existing AI-based Q&A systems lack effective quality control mechanisms and adaptability to ensure that generated answers are relevant and accurate, leading to suboptimal user interactions.

Innovation Solution

An AI-based Q&A framework with a feedback-based adaptation mechanism that utilizes human evaluators to assess and refine machine-generated answers, adjusting references and expert performance based on cumulative feedback to enhance answer quality and relevance.

Engineering Contradictions & Design Principles

VSEngineering Contradiction Analysis

1Productivity

If AI-based Q&A systems automatically generate answers without quality control mechanisms, then productivity is improved through automated responses, but reliability deteriorates due to lack of answer quality assurance

Engineering Contradiction:

Improveautomated response generation speedVSAvoidanswer quality assurance

Core Design Contradiction:

ProductivityVSReliability

Solution Approach 1:

The system implements a feedback mechanism where human evaluators assess AI-generated answers and provide ratings. This feedback is then used to continuously improve the AI model's performance, creating a closed-loop system that maintains both automation speed and answer quality over time

Inventive Principle:

Principle #23Feedback

Solution Approach 2:

Human evaluators serve as intermediaries between the AI system and the final output quality. They review and rate answers, acting as a mediating layer that ensures reliability without completely eliminating automated generation

Inventive Principle:

Principle #24Intermediary (Mediator)

2Device complexity

If AI systems use fixed reference sources and expert configurations, then device complexity is reduced through static setups, but adaptability deteriorates due to inability to respond to changing user needs

Engineering Contradiction:

Improvesystem configuration simplicityVSAvoidresponse relevance to user needs

Core Design Contradiction:

Device complexityVSAdaptability or versatility

Solution Approach 1:

The system transitions from static expert configurations to dynamic adaptation through feedback. Expert performance ratings and answer quality feedback are continuously accumulated and used to adjust system behavior, enabling the AI to adapt its responses based on learned patterns from human evaluations

Inventive Principle:

Principle #15Dynamics

Solution Approach 2:

The system modifies its operational parameters based on feedback data. By adjusting expert selection criteria, reference source weighting, and answer generation strategies based on accumulated feedback, the system maintains simplicity while improving adaptability to user needs

Inventive Principle:

Principle #35Parameter changes

Data Source

PatentUS20250322273A1Method and system for quality control of answers automatically generated via generative ai

Publication Date: 2025.10.16 VERIZON PATENT & LICENSING INC

AI summary

The present teaching relates to a Q&A framework for quality controlling of automatically generated answers via AI. Based on a question on a subject matter received from a user, at least one machine expert is selected to answer the question based on past performances of multiple machine experts for generating respective candidate answers to the question. Each selected machine expert creates a candidate answer based on a reference from a source. Quality assessment is performed with respect to each candidate answer from a respective machine expert and is relied on to determine a candidate answer as the answer to the question. Such determined answer is provided to the user as a response to the question.