Synthetic Utterance Generation via Parallel Pipeline Segmentation
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Acquiring large and diverse training datasets for natural language processing systems is time-consuming and expensive, often resulting in sparse datasets that lead to poorly performing models, especially in applications like fraud detection where naturally occurring utterances are limited.
Innovation Solution
A system utilizing multiple synthetic utterance generation pipelines with Text2Text machine learning models, where each pipeline includes one or more synthesizers, generates and evaluates synthetic utterances based on an input utterance, storing those with sufficient similarity and using feedback loops to produce a comprehensive set of high-quality synthetic utterances efficiently.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Quantity of substance
If synthetic utterances are generated using multiple pipelines with different synthesizers, then the quantity and diversity of training data increases, but the system complexity increases
Solution Approach 1:
The system divides the synthetic utterance generation task into multiple independent pipelines, each containing different synthesizers. Each pipeline processes input utterances separately and produces synthetic utterances with different characteristics. This segmentation allows the system to generate diverse training data by combining outputs from multiple specialized components, thereby increasing data quantity and variety while managing complexity through modular architecture.
Solution Approach 2:
The system implements a universal framework that can accommodate multiple types of synthesizers within each pipeline. The same pipeline structure can work with different synthesizer models (e.g., GPT-3, GPT-4, Llama 2, Llama 3) depending on the specific generation requirements. This multi-functionality enables the system to generate diverse synthetic utterances by simply changing the synthesizer models while maintaining the same overall pipeline architecture, thus increasing data diversity without proportionally increasing system complexity.
2Reliability
If synthetic utterances are generated using multiple pipelines with different synthesizers, then the quality and diversity of training data improves, but the time required for generation increases
Solution Approach 1:
The system segments the generation task into multiple parallel pipelines that operate simultaneously. Each pipeline processes input utterances independently through its own synthesizers, and all pipelines work in parallel to generate synthetic utterances. This parallelization significantly reduces the total generation time compared to sequential processing, while maintaining high data quality through the diverse synthesizers in each pipeline.
Solution Approach 2:
The system generates synthetic utterances using multiple synthesizers that may produce redundant or overlapping results. This excessive generation approach ensures that high-quality diverse utterances are produced, as the system can filter and select the best outputs from the multiple synthesizers. The partial redundancy is acceptable because it ensures data quality and diversity, and the parallel processing mitigates the time cost.
3Adaptability or versatility
If multiple synthesizers are used in each pipeline, then the diversity of synthetic utterances increases, but the computational resources required increase
Solution Approach 1:
The system segments the computational workload into separate pipelines, each handling specific aspects of utterance generation. By dividing the task, each pipeline can use fewer, more specialized synthesizers rather than requiring all synthesizers to process all inputs simultaneously. This segmentation reduces the peak computational resource requirements while still achieving high diversity through the distributed architecture.
Solution Approach 2:
The system employs multiple synthesizers in parallel within each pipeline to generate diverse synthetic utterances. Not all synthesizers need to process all input utterances; the system can strategically activate only the necessary synthesizers based on the input characteristics and diversity requirements. This partial action approach maintains high utterance diversity while optimizing computational resource utilization by avoiding unnecessary processing.
Data Source
AI summary
This disclosure relates to generating a comprehensive set of synthetic utterances. An example system is configured to provide an input utterance to a plurality of synthetic utterance generation pipelines in parallel. Each of the plurality of synthetic utterance generation pipelines include one or more utterance synthesizers. For example, one or more pipelines may use a synthesizer chain that includes a plurality of synthesizers in parallel. The plurality of synthetic utterance generation pipelines generates synthetic utterances, which may be stored in a database after evaluating the similarity between the original input utterance and each resulting synthetic utterance. For example, a synthetic utterance may be retained if the cosine similarity between the input and synthetic utterances is less than a predetermined threshold. Additionally, the synthetic utterances may be fed back at input utterances based on the similarity evaluation and the feedback loop repeated until a desired number of utterances are generated.