Survey Response Prioritization Using Notability Scores
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Conventional systems for organizing textual responses from surveys are inefficient due to the presence of noise, lack of effective prioritization, and inability to quickly identify valuable insights, leading to labor-intensive and time-consuming review processes.
Innovation Solution
A system that prioritizes textual responses based on notability values calculated from character length, text entropy, and readability, filters out noise by identifying multiple parts of speech, and organizes responses to quickly present useful and interesting feedback to reviewers.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Speed
If conventional systems present all textual responses in reverse chronological order, then reviewers can view the most recent responses first, but valuable responses are moved continually deeper into the queue and may never be addressed
Solution Approach 1:
The system changes the ordering parameter from chronological to quality-based, using calculated quality scores to prioritize responses. This resolves the contradiction by ensuring valuable responses rise to the top regardless of when they were submitted, while still allowing recent responses to be viewed efficiently.
Solution Approach 2:
The system performs preliminary filtering and scoring of responses before they reach the reviewer queue. By pre-identifying and prioritizing valuable responses using quality metrics, the system ensures that important feedback is addressed promptly without requiring reviewers to manually search through all responses.
2Measurement precision
If reviewers read each response to identify valuable information, then complete analysis is achieved, but the process becomes labor-intensive, time consuming, and expensive
Solution Approach 1:
The system applies different levels of analysis to different responses based on their quality scores. High-quality responses receive full reviewer attention, while low-quality responses are filtered or given minimal processing. This resolves the contradiction by concentrating review resources on responses that actually require human analysis.
Solution Approach 2:
The system performs self-service filtering and prioritization using automated quality assessment algorithms. By having the system automatically identify and rank valuable responses, the burden of manual analysis is reduced while maintaining comprehensive coverage of important feedback.
3Quantity of substance
If conventional systems sort responses from longest to shortest, then responses with more information appear first, but response length alone does not indicate whether a response will be useful or interesting
Solution Approach 1:
The system changes the sorting parameter from length to quality score, which incorporates multiple factors including information density, relevance, and engagement metrics. This resolves the contradiction by providing a more precise measure of response usefulness that goes beyond simple word count.
4Adaptability or versatility
If systems compare textual responses to each other to organize them, then responses can be grouped by similarity, but responses must be in a particular language and a minimum number of responses must be obtained before comparison
Solution Approach 1:
The system uses quality scores as an intermediary metric that can be calculated for individual responses without requiring comparison to other responses. This resolves the contradiction by enabling organization and prioritization of responses in any language with any sample size, while still providing sophisticated filtering and grouping capabilities.
Data Source
AI summary
Embodiments of the present disclosure generally relate to organizing textual responses, such as survey responses. More specifically, one or more embodiments of the present disclosure provide a reviewer with textual responses that are prioritized according to usefulness. As an example, one or more embodiments of the present disclosure provide a reviewer with a notability value that provides an indication of the usefulness and/or interestedness of a response in relation to other responses for a particular open or textual type question.