Concurrent Evaluation System for Agent Interaction Analysis
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current technologies for evaluating interactions, such as conversations in call centers, are imperfect as agents are often disengaged due to having to evaluate after another person or seeing their evaluation results, leading to a need for more effective and automatic analysis methods.
Innovation Solution
A system and method that allows agents and evaluators to simultaneously or concurrently evaluate interactions using a computerized evaluation form, with each participant providing independent ratings and calculating a variance between their scores, ensuring agents evaluate without knowing the other's outcome, promoting honest and engaged participation.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If agents evaluate interactions after another person or after seeing evaluation results, then evaluation process is completed, but agent engagement and honesty of evaluation decrease
Solution Approach 1:
The system performs preliminary action by sending evaluation forms to both the agent and evaluator simultaneously, allowing the agent to complete their evaluation before seeing the evaluator's results. This preliminary completion ensures the agent provides honest feedback without being influenced by the evaluator's assessment, thereby improving evaluation reliability while maintaining agent engagement through concurrent processing.
2Measurement precision
If agents evaluate interactions concurrently with evaluators without knowing other outcomes, then evaluation honesty improves, but process coordination complexity increases
Solution Approach 1:
The system introduces an intermediary mechanism - a computerized evaluation form management system that coordinates the concurrent evaluation process. This intermediary automatically manages the distribution of evaluation forms, tracks completion status, and controls the sequence of information disclosure, thereby enabling accurate concurrent evaluations without requiring complex manual coordination between agents and evaluators.
Solution Approach 2:
The system implements feedback control by monitoring the evaluation completion status of both agents and evaluators in real-time. When one party completes their evaluation, the system provides feedback by controlling when the other party's results are disclosed, ensuring that honesty is maintained while the coordination complexity is managed through automated status tracking and conditional disclosure.
3Productivity
If traditional sequential evaluation process is used, then process simplicity is maintained, but evaluation quality and engagement decrease
Solution Approach 1:
The system applies dynamics by transitioning from a static sequential evaluation process to a dynamic concurrent process. The evaluation workflow becomes adaptive, automatically adjusting the disclosure of results based on completion status. This dynamic approach enables both parties to evaluate simultaneously, improving evaluation quality and engagement while the system manages the increased complexity through automated conditional logic rather than rigid sequential steps.
Data Source
AI summary
A method and system may select an interaction involving an agent, sending the selected interaction and a computerized form to the agent and an evaluator, simultaneously or concurrently, displaying to the evaluator and agent screens defined by the form, each screen including an evaluation question, accepting from the agent, for each evaluation question, an agent answer having associated with the agent answer a rating, accepting from the evaluator a submission indicating that the evaluator has completed the computerized evaluation form, accepting from the agent a submission indicating that the agent has completed the computerized evaluation form, summing an agent rating from the ratings associated with the agent answers provided by the agent, summing an evaluator rating from the ratings associated with evaluator answers provided by the evaluator, and calculating a variance from the agent rating and evaluator rating.