Survey Insight Reporting System Using Probability Calibration
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing methods for summarizing survey responses on ordinal scales, such as the percent positive and integer assignment approaches, suffer from information loss and lack of calibration, leading to unreliable results due to the assumption that response options are equidistant, which is mathematically unfounded and introduces errors in scoring.
Innovation Solution
A method that determines a calibrated score for each survey response based on the probability of a respondent providing a response lower or higher on the ordinal scale, using logit scores to weight passion and outliers, and calculates composite scores for items and respondent segments, allowing for more accurate comparison and identification of noteworthy insights.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Ease of manufacture
If the percent positive approach is used to summarize ordinal scale responses, then the summary is simple to calculate, but information is lost and enthusiasm level cannot be distinguished
Solution Approach 1:
The patent transforms ordinal scale responses into continuous probability values ranging from 0 to 1, where each response option maps to a specific probability representing the likelihood of a respondent choosing that option or any lower option. This parameter transformation preserves the full distribution of responses rather than collapsing them into binary categories, thereby maintaining information while enabling straightforward calculation of summary statistics.
2Loss of information
If the integer assignment approach is used to summarize ordinal scale responses, then the summary preserves more information, but the assumption of equidistant response options introduces mathematical errors
Solution Approach 1:
The patent replaces the integer assignment method with a probability-based parameter transformation. Instead of assigning equidistant integers to ordinal scale options (which assumes equal psychological distance between options), the patent assigns probability values that reflect the actual distribution of responses and the relative likelihood of choosing each option. This eliminates the mathematically unfounded equidistance assumption while preserving the informational content of the full response distribution.
3Device complexity
If traditional summarization methods are used for ordinal scale responses, then the process is simple, but the results are unreliable due to information loss and lack of calibration
Solution Approach 1:
The patent replaces traditional mechanical summarization methods (percent positive calculation, integer averaging) with a probability-based statistical framework. This substitution introduces calibration by modeling respondent choice behavior and accounting for the unique distribution characteristics of each survey item. The approach maintains computational efficiency while significantly improving result reliability through proper statistical treatment of ordinal data.
Data Source
AI summary
A method of displaying survey results is disclosed. In one embodiment, for each response to each item by each survey respondent, a calibrated score is determined. The calibrated score is based on a probability that a person having the collection of respondent segments associated with the survey respondent would provide either the response to the item or any of the response options to the item that are lower on the ordinal scale, or the response to the item or any of the response options to the item that are higher on the ordinal scale. An organization composite score is determined based on the calibrated score for each response. A display includes images corresponding with the organization composite score and other organization composite scores. Each image is located with respect to a first axis based on the associated organization composite score.