Digital Assistant Comment Evaluation Using Structured Interaction Signals
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing digital assistants face challenges in evaluating dialogue and task processing quality due to the varied and unstructured nature of user comments, making it difficult to effectively improve and upgrade their performance.
Innovation Solution
A method and apparatus for detecting and analyzing interactive information, such as text, voice, and video, to determine an evaluation result based on the interaction quality and configuration information of the digital assistant, using machine learning models to identify the tendency of comments, and performing operations like recommendation or alert based on the evaluation results.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If user comments are collected for digital assistant evaluation, then evaluation capability is improved, but the unstructured and varied nature of comments makes effective analysis difficult
Solution Approach 1:
The patent introduces an evaluation model as an intermediary between raw user comments and evaluation results. This model processes the unstructured comment data through standardized evaluation dimensions (dialogue quality, task completion quality, parameter quality), transforming difficult-to-analyze text into structured evaluation scores that quantify the digital assistant's performance.
Solution Approach 2:
The patent transforms qualitative comment data into quantitative evaluation parameters by defining specific evaluation dimensions and scoring criteria. Each dimension (dialogue quality, task completion, parameters) is assigned numerical scores, converting the varied and unstructured comment nature into measurable parameters that enable precise evaluation.
2Measurement precision
If comprehensive interactive information is detected and analyzed, then evaluation accuracy is improved, but processing complexity and time consumption increase
Solution Approach 1:
The patent segments the comprehensive interactive information into distinct evaluation dimensions: dialogue quality information, task completion quality information, and parameter quality information. Each dimension is evaluated separately with specific criteria, allowing parallel processing and reducing the time complexity of analyzing all information together as a single unstructured mass.
Solution Approach 2:
The patent performs preliminary classification and structuring of interactive information before detailed evaluation. By pre-organizing comments and interactions into predefined evaluation categories and extracting key features in advance, the system reduces the processing burden during the actual evaluation phase, thereby reducing overall processing time while maintaining accuracy.
Data Source
AI summary
This disclosure describes a method, an apparatus, a device, a computer-readable storage medium, and a computer program product for interactive information processing. The method includes the following steps: detecting to-be-evaluated interactive information of a digital assistant, the interactive information being used to comment on the digital assistant; and determining an evaluation result for the interactive information based on the interactive information and configuration information of the digital assistant, the evaluation result indicating a tendency of the interactive information to the comment on the digital assistant.