Credibility Rating for Online Answerers Using Bayesian Smoothing
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Traditional best-answer indicators in online answer submission systems fail to accurately reflect the credibility of new answerers due to sampling errors, as they are often based on limited data, leading to unfair penalties or rewards.
Innovation Solution
A statistical technique using Bayesian smoothing estimates an answerer's credibility rating by combining their specific performance with the composite performance of a population, adjusting the influence based on the number of answers submitted, and employing a mixture of Beta distributions for accurate probability estimation.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If the traditional best-answer indicator is used for new answerers with limited submissions, then the indicator is easy to calculate, but the measurement precision is low due to sampling errors
Solution Approach 1:
The patent combines the answerer's specific performance data with the composite performance data of the entire answerer population to form a blended credibility rating. This merging allows new answerers with limited submissions to benefit from population-level statistics, reducing sampling errors while still incorporating their individual performance as it accumulates.
Solution Approach 2:
The patent dynamically adjusts the weighting parameters in the credibility rating formula based on the number of answers submitted. As the number of submissions increases, the weight shifts from population composite performance to individual specific performance. This parameter change resolves the contradiction by adapting the measurement approach to the quantity of available data.
2Reliability
If the best-answer indicator is based on a small number of submissions, then the calculation is simple, but the reliability is low due to unfair penalties on new answerers
Solution Approach 1:
The system merges individual answerer performance with population composite performance to create a more reliable credibility rating. This combination ensures fairness by preventing new answerers from being unfairly penalized due to limited sample size, while still allowing experienced answerers to be distinguished by their individual performance records.
Solution Approach 2:
The population composite performance serves as an intermediary that bridges the gap for new answerers with limited data. It acts as a statistical mediator that provides a reasonable credibility estimate when individual data is insufficient, gradually transitioning to individual performance as data accumulates.
3Measurement precision
If the credibility rating uses population composite performance, then the measurement precision improves for new answerers, but the loss of individual performance information increases
Solution Approach 1:
The credibility rating system is dynamic, with the weighting between population composite performance and individual specific performance changing over time. For new answerers, the population composite provides the majority of the rating precision. As the answerer accumulates more submissions, the system dynamically shifts weight to individual performance, minimizing information loss while maximizing precision at each stage.
Solution Approach 2:
The patent uses parameter changes to control the balance between population and individual data. The weighting parameters are adjusted based on the number of submissions, ensuring that individual performance information is progressively incorporated as it becomes available, rather than being lost or overshadowed by population statistics.
Data Source
AI summary
Techniques for statistically estimating a rating or other “figure of merit” for a user are disclosed. According to one such technique, a first quantity of submissions that were submitted by a user is determined. A second quantity of submissions that (a) were submitted by the user and (b) obtained a particular rating from a rating mechanism also is determined. A user rating for the user is determined based at least in part on the first quantity, the second quantity, and a factor that is independent of both the first quantity and the second quantity—such as the probability that an answer submitted by any answerer in a population will obtain the particular rating from the rating mechanism. The influence that the second quantity has on the user rating relative to the influence that the factor has on the user rating may depend at least in part on the first quantity.