Data Optimization Application for Subjective Quality Score Accuracy
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Subjective quality experiments face inaccuracies due to participant bias and inconsistency, leading to reduced accuracy in subjective scores, and increasing the number of participants to mitigate these issues increases resource costs without ensuring desired accuracy.
Innovation Solution
A method involving a data optimization application that generates an optimized subjective score set and participant evaluation report, compensating for participant biases and filtering out inconsistent participants to improve score accuracy without increasing participant numbers.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If the number of participants is increased to reduce random scoring inaccuracies, then the accuracy of subjective scores improves, but the resources required for recruiting and training participants increases
Solution Approach 1:
The patent replaces the mechanical approach of simply increasing participant numbers with a computational optimization system. The data optimization application uses algorithms to identify and filter inconsistent participants, substituting computational analysis for brute-force sampling expansion. This allows the system to achieve better measurement precision without proportionally increasing the quantity of participants.
Solution Approach 2:
The patent introduces an intermediary optimization layer between raw participant scores and final subjective scores. The data optimization application acts as a mediator that processes individual scores, identifies inconsistent participants through optimization operations, and generates filtered aggregated scores. This intermediary system enables accurate results with fewer participants by eliminating the need to rely on large numbers to average out random errors.
2Reliability
If the number of participants is increased to mitigate random inaccuracies, then the reliability of subjective scores improves, but the cost and complexity of the experiment increases
Solution Approach 1:
The patent substitutes manual experiment management complexity with automated computational optimization. Instead of manually coordinating and managing large numbers of participants, the system uses a data optimization application that automatically processes scores, identifies inconsistent participants through optimization algorithms, and generates reliable aggregated results. This reduces the operational complexity typically associated with large-scale participant management.
Solution Approach 2:
The optimization system performs self-service by automatically identifying inconsistent participants and adjusting the aggregation process without external intervention. The data optimization application autonomously executes optimization operations, evaluates participant consistency, and generates reliable scores, eliminating the need for complex external coordination and management overhead.
3Measurement precision
If traditional aggregation methods are used to compute subjective scores, then the process is simple, but the accuracy is reduced due to participant bias and inconsistency
Solution Approach 1:
The patent introduces an intermediary optimization layer between raw participant scores and final subjective scores. The data optimization application processes individual scores through optimization operations that identify and compensate for participant bias and inconsistency. This intermediary system enhances measurement precision while maintaining manageable process complexity through automation.
Solution Approach 2:
The patent replaces simple but inaccurate traditional aggregation with computational optimization. The data optimization application uses algorithms to analyze scoring patterns, identify inconsistent participants, and generate accurate aggregated scores. This substitution increases measurement precision while keeping the process complexity manageable through automated computation rather than manual analysis.
Data Source
AI summary
In various embodiments, a data optimization application mitigates scoring inaccuracies in subjective quality experiments. In operation, the data optimization application generates a model that includes a first set of individual scores and a first set of parameters. The first set of parameters includes a first subjective score set and a first set of subjective factor sets. The data optimization application performs one or more optimization operations on the first set of parameters to generate a second set of parameters. The second set of parameters includes a second subjective score set and a second set of subjective factor sets, wherein the second subjective score set compensates for at least a first subjective factor set included in the second set of subjective factor sets. The data optimization application also computes a participant evaluation report based on at least a second subjective factor sets included in the second set of subjective factor sets