Acoustic Quality Evaluation Apparatus Mitigating Evaluator Bias
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Evaluators accustomed to low acoustic quality environments tend to give higher evaluation values in conversational tests, leading to biased results, making it difficult to obtain appropriate acoustic quality evaluations in loudspeaker hands-free communication systems.
Innovation Solution
An acoustic quality evaluation apparatus that presents evaluation categories to evaluators and determines the lowest evaluation value among selected categories for each viewpoint, providing a subjective evaluation value for acoustic quality, thereby mitigating the bias in evaluation.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Ease of operation
If subjective evaluation is performed by evaluators in low acoustic quality environments, then the evaluation process is simple and convenient, but the evaluation values become biased toward higher values and do not reflect true quality
Solution Approach 1:
The system performs preliminary classification of evaluation viewpoints into multiple levels (e.g., excellent, good, fair, poor, very poor) before the actual evaluation. This pre-structured framework guides evaluators to make more precise judgments even in low acoustic quality environments, reducing the tendency to give uniformly high ratings.
Solution Approach 2:
The system changes the evaluation parameters by introducing multiple evaluation viewpoints (acoustic echo, far-end talker voice, near-end talker voice, ambient noise) with different weightings. This multi-dimensional approach transforms the single biased evaluation into a balanced composite score, compensating for evaluator bias in low quality environments.
2Productivity
If objective evaluation methods are used, then the evaluation can be performed by computer processing with high efficiency, but the results do not always match the quality experienced by users in actual phone calls
Solution Approach 1:
The system introduces a composite evaluation index as an intermediary that bridges objective measurements and subjective experience. By combining multiple objectively measurable parameters (acoustic echo level, noise level, voice clarity) with predetermined weightings, the system creates a composite score that better correlates with user experience while maintaining computational efficiency.
Solution Approach 2:
The evaluation system uses a composite approach by integrating multiple evaluation dimensions (acoustic echo, far-end voice, near-end voice, ambient noise) into a single composite quality index. This composite metric captures the multifaceted nature of call quality better than single-parameter objective measures, improving correlation with subjective user experience.
3Measurement precision
If traditional conversational tests are used for subjective evaluation, then the evaluation reflects actual user experience, but evaluators accustomed to low quality environments give biased high ratings
Solution Approach 1:
The system segments the evaluation into multiple independent viewpoints (acoustic echo, far-end talker voice, near-end talker voice, ambient noise) rather than relying on a single overall rating. This segmentation allows evaluators to assess different aspects separately, reducing the impact of overall quality bias and providing more reliable data for computing the composite index.
Solution Approach 2:
The system implements feedback by presenting the composite evaluation index results back to evaluators and using them to adjust evaluation criteria. The predetermined weightings and multi-level classification provide structured feedback frameworks that guide evaluators toward more consistent and reliable assessments, reducing bias in low quality environments.
Data Source
AI summary
To obtain an appropriate evaluation value in an acoustic quality evaluation by a conversational test. An acoustic quality evaluation apparatus 3 evaluates the acoustic quality of a call performed between a near-end terminal 1 and a far-end terminal 2 via a voice communication network 4. An evaluation value presenting unit 31 displays, on a display unit 13, evaluation categories obtained by classifying each of a plurality of evaluation viewpoints into a predetermined number of levels. An input unit 14 transmits the evaluation category selected by the evaluator for each of the evaluation viewpoints, to an evaluation value determination unit 32. The evaluation value determination unit 32 determines the lowest evaluation value among evaluation values assigned to the evaluation category received from the input unit 14 as a subjective evaluation value for acoustic quality.