Training Data Label Inference and Evaluation for Voice Recognition

Solution Overview

Problem

Existing techniques for voice recognition in contact centers require large amounts of training data and suffer from label inconsistencies due to varying experiences and policies among workers, leading to degraded estimation accuracy and inefficient evaluation of training data creators.

Innovation Solution

A support device and method that includes a label inference unit to infer labels using a learned model and an evaluation unit to generate evaluation results by comparing correct labels with inferred labels, facilitating efficient evaluation of training data creators.

Engineering Contradictions & Design Principles

VSEngineering Contradiction Analysis

1Measurement precision

If a large amount of training data is used to achieve high estimation accuracy, then the accuracy of service scene estimation is improved, but the time and cost required for creating training data increases significantly

Engineering Contradiction:

Improveestimation accuracyVSAvoidtime for creating training data

Core Design Contradiction:

Measurement precisionVSLoss of time

Solution Approach 1:

The system uses the learned model to automatically evaluate and check the quality of training data creators' work. The model serves itself by inferring labels from training data and comparing them with actual labels to automatically generate evaluation results, eliminating the need for manual evaluation processes.

Inventive Principle:

Principle #25Self-service

Solution Approach 2:

The system provides feedback to training data creators by generating evaluation results that show the quality of their label assignments. The evaluation unit compares inferred labels with actual labels and presents this information to creators, enabling them to improve their work based on automated feedback.

Inventive Principle:

Principle #23Feedback

2Quantity of substance

If multiple workers create training data to increase volume, then the quantity of training data is improved, but label consistency deteriorates due to varying experiences and policies

Engineering Contradiction:

Improvequantity of training dataVSAvoidlabel consistency

Core Design Contradiction:

Quantity of substanceVSReliability

Solution Approach 1:

The system provides feedback to training data creators by generating evaluation results that show the quality of their label assignments. The evaluation unit compares inferred labels with actual labels and presents this information to creators, enabling them to improve their work based on automated feedback.

Inventive Principle:

Principle #23Feedback

Solution Approach 2:

The system changes the parameter of evaluation from manual expert judgment to automated model-based inference. By using the learned model to generate evaluation results, the system objectively measures label consistency across different workers and identifies patterns in their labeling behavior.

Inventive Principle:

Principle #35Parameter changes

3Reliability

If manual evaluation of training data creators is performed to ensure quality, then the reliability of training data is improved, but the productivity of the evaluation process deteriorates

Engineering Contradiction:

Improvetraining data qualityVSAvoidevaluation process efficiency

Core Design Contradiction:

ReliabilityVSProductivity

Solution Approach 1:

The system uses the learned model to automatically evaluate and check the quality of training data creators' work. The model serves itself by inferring labels from training data and comparing them with actual labels to automatically generate evaluation results, eliminating the need for manual evaluation processes.

Inventive Principle:

Principle #25Self-service

Solution Approach 2:

The system replaces manual mechanical evaluation processes with automated computational inference. Instead of human experts manually reviewing each label assignment, the system uses a learned model to automatically infer labels and compare them with actual labels, generating evaluation results much faster and at scale.

Inventive Principle:

Principle #28Mechanics substitution (Replace mechanical system)

Data Source

PatentUS20240144057A1Support device, support method, and program

Publication Date: 2024.05.02 NT T INC

AI summary

A training data confirmation support device according to the present disclosure includes a label inference unit that infers inference labels that are labels corresponding to elements included in training data in which elements and correct labels corresponding to the elements are associated with each other using a model that is learned using the training data and infers labels corresponding to the elements, and an evaluation unit that generates evaluation results of the training data creators on the basis of comparison between correct labels corresponding to elements included in the training data and inference labels of the elements.