Knowledge Base Scoring with Human Feedback Loops
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing methods for updating and maintaining a knowledge base, such as those used in chatbots, are inefficient and fail to effectively incorporate human feedback for improving answer quality and relevance.
Innovation Solution
A computing system that utilizes a database to store question-and-answer-pairs with metadata, allowing for user ratings to calculate scores based on keyword matching and additional metadata factors like time, user credibility, and relevance, enabling faster knowledge base updates and filtering out outdated information.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Extent of automation
If purely AI-driven optimization schemes are used to update the knowledge base, then automation is improved, but the speed of learning and updating the knowledge base deteriorates
Solution Approach 1:
The patent implements explicit feedback mechanisms where users directly rate the quality and relevance of answers through ratings and comments. This feedback is immediately incorporated into the knowledge base update process, allowing the system to learn from human evaluations rather than relying solely on slow AI-driven optimization based on indirect user behavior signals like clicks.
Solution Approach 2:
The patent introduces human experts as intermediaries who review and validate answers before they are added to the knowledge base. These experts act as a bridge between user questions and the knowledge base, ensuring quality while accelerating the learning process by providing immediate human judgment rather than waiting for AI optimization to converge.
2Reliability
If human expert answers are obtained for every question, then answer quality is improved, but computational costs and time consumption deteriorate
Solution Approach 1:
The patent applies partial human intervention by sending only selected questions to human experts rather than all questions. The system uses automated scoring to identify questions that would benefit most from human review, applying human expertise partially where needed while maintaining automation for routine queries.
Solution Approach 2:
The patent dynamically changes the parameter of human involvement based on question characteristics and knowledge base confidence levels. Questions with high confidence scores or those matching existing patterns are handled automatically, while low-confidence or novel questions are routed to human experts, optimizing the balance between automation and human review.
3Quantity of substance
If all question-and-answer-pairs are stored in the knowledge base, then knowledge base completeness is improved, but maintenance complexity and computational costs deteriorate
Solution Approach 1:
The patent implements automatic discarding of outdated, low-quality, or redundant question-and-answer-pairs from the knowledge base. The system continuously evaluates stored content and removes entries that no longer meet quality thresholds or are superseded by better answers, maintaining a compact and relevant knowledge base without manual intervention.
Solution Approach 2:
The patent changes the state of knowledge base entries by assigning dynamic quality scores, relevance weights, and confidence levels to each question-and-answer-pair. These parameter changes enable the system to prioritize high-quality content and automatically de-prioritize or remove lower-quality entries, simplifying maintenance through automated quality-based filtering.
4Ease of operation
If user behavior data alone is used for optimization, then ease of data collection is improved, but measurement precision of answer quality deteriorates
Solution Approach 1:
The patent replaces indirect feedback from user behavior data with direct explicit feedback through user ratings and comments on answer quality. Users directly evaluate whether answers solved their problems, providing precise measurement of answer quality rather than inferring it from clicks or time spent, which are indirect and imprecise metrics.
Data Source
Figure 1
Figure 2~3
Figure 4~5
AI summary
The invention refer to a method for answering user generated questions implemented in a knowledge base computing system (100), the knowledge base computing system comprising a database (104) which stores question-and-answer pairs and metadata related to the question-and-answers pairs, the method comprising the knowledge base computer system: a) receiving (11) from a user computing device a user question, the user question including a question text having one or more words and/or sentences in a natural language; b) performing (12) a search for question-and-answer pairs and calculating (13) a score for question-and-answer pairs based on keywords in the questions text and based on the metadata related to the question-and-answer pairs; c) comparing (14) the score to a predefined threshold and based on the result of the comparison perform a decision step (15) to either forward (16) at least the question-and answer pair with the highest score to the user computing device or forward (17) the question text to another computing device or output device in order to obtain an human expert answer, to be forwarded to the user computing device and storing the questions and the answer as a new question-and-answer pair d) receiving (18) from the user computing device a rating about the sent answer or question-and-answer pair and storing (19) the rating in the metadata of the question-and answer pair of the sent question-and-answer pair or sent answer.