Multi-Turn Dialogue Training With Adversarial Bootstrapping

Resolve Bottlenecks,
Find Innovative SolutionsGenerate Solutions

Solution Overview

Problem

Conventional systems for generating multi-turn dialogues produce irrelevant, non-useful, or generic responses due to discrepancies between training and inference stages, leading to exposure bias and limited diversity, and fail to capture long-term dependencies, resulting in repetitive and dull outputs.

Innovation Solution

Adversarial bootstrapping between a generator and a discriminator with multi-turn capabilities, using a mixture of ground truth labels, teacher-forcing outputs, and negative examples to train machine classifiers, incorporating dual sampling to improve response relevance and coherence.

Engineering Contradictions & Design Principles

VSEngineering Contradiction Analysis

1Adaptability or versatility

If conventional neural network training methods are used, then training simplicity is maintained, but response diversity and relevance deteriorate

Engineering Contradiction:

Improveresponse diversityVSAvoidtraining complexity

Core Design Contradiction:

Adaptability or versatilityVSDevice complexity

Solution Approach 1:

The patent introduces a discriminator model as an intermediary component that evaluates the quality of generated responses. This discriminator acts as a mediator between the generator and the training process, providing feedback signals that guide the generator to produce more diverse and relevant responses without requiring complete redesign of the training methodology

Inventive Principle:

Principle #24Intermediary (Mediator)

Solution Approach 2:

The training process is segmented into distinct components: a generator model that produces responses, a discriminator model that evaluates responses, and a training loop that coordinates both. This segmentation allows each component to be optimized independently while working together to achieve diverse and relevant output

Inventive Principle:

Principle #1Segmentation

2Reliability

If teacher-forcing training mode is used, then training convergence is improved, but exposure bias increases during inference

Engineering Contradiction:

Improvetraining convergenceVSAvoidinference performance

Core Design Contradiction:

ReliabilityVSAdaptability or versatility

Solution Approach 1:

The patent implements dynamic training strategies that adjust the balance between teacher-forcing and autoregressive training during the training process. The system dynamically transitions from relying on ground truth labels to relying on generated labels, allowing the model to adapt its behavior as training progresses and reducing exposure bias at inference time

Inventive Principle:

Principle #15Dynamics

Solution Approach 2:

The training process modifies parameters such as the teacher-forcing probability over time, gradually reducing the reliance on ground truth labels and increasing the use of generated labels. This parameter change strategy allows the model to learn from accurate data initially while progressively adapting to generate its own training signals

Inventive Principle:

Principle #35Parameter changes

3Stability of the object's composition

If maximum likelihood loss is used for training, then training stability is maintained, but response relevance to user input deteriorates

Engineering Contradiction:

Improvetraining stabilityVSAvoidresponse relevance

Core Design Contradiction:

Stability of the object's compositionVSLoss of information

Solution Approach 1:

The patent implements a feedback mechanism where the discriminator evaluates generated responses and provides feedback signals to the generator. This feedback loop allows the system to maintain training stability through the generator's maximum likelihood objective while simultaneously improving response relevance through the discriminator's quality assessments and gradient signals

Inventive Principle:

Principle #23Feedback

4Device complexity

If single-turn training approach is used, then model simplicity is maintained, but multi-turn dialogue coherence deteriorates

Engineering Contradiction:

Improvemodel architectureVSAvoiddialogue coherence

Core Design Contradiction:

Device complexityVSLoss of information

Solution Approach 1:

The patent applies preliminary action by training the model on multi-turn dialogue data that includes historical context and conversation flow. The generator learns to maintain coherence across multiple turns by being exposed to and training on complete dialogue sequences, preparing it to handle multi-turn interactions effectively

Inventive Principle:

Principle #10Preliminary action

Data Source

PatentUS12505305B2Adversarial bootstrapping for multi-turn dialogue model training

Publication Date: 2025.12.23 CAPITAL ONE SERVICES LLC

AI summary

Systems described herein may use machine classifiers to perform a variety of natural language understanding tasks including, but not limited to multi-turn dialogue generation. Machine classifiers in accordance with aspects of the disclosure may model multi-turn dialogue as a one-to-many prediction task. The machine classifier may be trained using adversarial bootstrapping between a generator and a discriminator with multi-turn capabilities. The machine classifiers may be trained in both auto-regressive and traditional teacher-forcing modes, with the maximum likelihood loss of the auto-regressive outputs being weighted by the score from a metric-based discriminator model. The discriminators input may include a mixture of ground truth labels, the teacher-forcing outputs of the generator, and/or negative examples from the dataset. This mixture of input may allow for richer feedback on the autoregressive outputs of the generator. Additionally, dual sampling may improve response relevance and coherence by overcoming the problem of exposure bias.