Speech Interaction Satisfaction Evaluation Using Objective and Subjective Data
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing methods for evaluating speech interaction satisfaction are inadequate as they rely solely on the association between user intent and terminal response content, failing to provide a comprehensive and authentic user evaluation.
Innovation Solution
A method and apparatus that acquire both objective and subjective data for speech interaction, including user intent, response content, response delay, playback duration, and user feedback, to input a satisfaction evaluation model, which combines these evaluations to determine a comprehensive speech interaction satisfaction.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Device complexity
If only intent and response content are used for evaluation, then the evaluation process is simple, but the evaluation comprehensiveness and authenticity deteriorate
Solution Approach 1:
The evaluation system segments the speech interaction into multiple independent evaluation dimensions: objective data evaluation (response accuracy, response time, completion rate) and subjective data evaluation (user feedback sentiment). This segmentation allows comprehensive evaluation without overwhelming complexity, as each dimension can be processed independently and then integrated.
Solution Approach 2:
The patent introduces a satisfaction evaluation model as an intermediary that processes both objective and subjective evaluations. This model acts as a mediator that integrates multiple evaluation dimensions and produces a comprehensive satisfaction score, avoiding the need for complex manual evaluation processes while maintaining high evaluation quality.
2Measurement precision
If multiple data dimensions are collected for evaluation, then the evaluation authenticity improves, but the data processing complexity increases
Solution Approach 1:
The system segments data collection into distinct sources: objective data from system logs (response accuracy, time, completion rate) and subjective data from user feedback. This segmentation simplifies data processing by allowing each data type to be handled with appropriate specialized methods rather than requiring a single complex processing pipeline.
Solution Approach 2:
The system incorporates user feedback as subjective data that continuously feeds back into the evaluation model. This feedback mechanism allows the system to automatically adjust and improve evaluation accuracy over time without increasing processing complexity, as the feedback is integrated through the existing satisfaction evaluation model.
3Ease of manufacture
If traditional text processing techniques are used, then the implementation is straightforward, but the user evaluation authenticity deteriorates
Solution Approach 1:
The satisfaction evaluation model serves as an intermediary that enhances traditional text processing with machine learning capabilities. It processes both structured objective data and unstructured subjective feedback, transforming them into an authentic satisfaction measurement without requiring completely complex new implementation approaches.
Solution Approach 2:
The system changes the parameters of text processing by incorporating sentiment analysis and semantic understanding into the evaluation model. This allows the system to go beyond simple text matching and capture authentic user evaluation through advanced text analysis techniques while maintaining implementation feasibility through modular model integration.
Data Source
AI summary
A method and apparatus for determining a speech interaction satisfaction are provided. The method includes: acquiring a speech interaction feature, the speech interaction feature including objective data of speech interaction and subjective data of the speech interaction, and the objective data and the subjective data being data for a given topic; evaluating the objective data to obtain an objective evaluation, and evaluating the subjective data to obtain a subjective evaluation; and obtaining a speech interaction satisfaction outputted by a satisfaction evaluation model by using the objective evaluation and the subjective evaluation as an input of the satisfaction evaluation model. Some embodiments may provide an authentic and comprehensive evaluation of the speech interaction.