Two-Stage Spoken Dialogue System Training

Solution Overview

Problem

Current spoken dialogue systems face inefficiencies in training, as they rely solely on supervised learning, which limits their ability to adapt to real-world interactions and user variability, leading to suboptimal performance in achieving user goals.

Innovation Solution

A two-stage training technique is introduced, where a policy network is first trained using external data to produce a semi-trained policy network, and then further trained through interaction with a user simulator to produce a trained policy network, enhancing its ability to make informed decisions based on user interactions.

Engineering Contradictions & Design Principles

VSEngineering Contradiction Analysis

1Adaptability or versatility

If supervised learning is used to train the spoken dialogue system, then the training process is simple and straightforward, but the system's ability to adapt to real-world interactions and user variability is limited

Engineering Contradiction:

Improveadaptability to real-world interactionsVSAvoidtraining process complexity

Core Design Contradiction:

Adaptability or versatilityVSDevice complexity

Solution Approach 1:

The training process is divided into two distinct stages: supervised learning stage and interaction-based learning stage. This segmentation allows the system to first learn from static external data with simple supervised learning, then progressively adapt to dynamic real-world interactions through controlled interactions with a user simulator, thereby resolving the contradiction between adaptability and training complexity

Inventive Principle:

Principle #1Segmentation

Solution Approach 2:

The system performs preliminary supervised learning training using external data before engaging in interaction-based learning. This preliminary action establishes a foundational knowledge base that enables the system to subsequently adapt more effectively to real-world interactions, reducing the overall complexity of the adaptive training process

Inventive Principle:

Principle #10Preliminary action

2Reliability

If interaction-based learning is used to improve adaptability, then the system can better handle user variability, but the training time and computational resources increase

Engineering Contradiction:

Improveperformance in achieving user goalsVSAvoidtraining time

Core Design Contradiction:

ReliabilityVSLoss of time

Solution Approach 1:

The system performs preliminary supervised learning training using external data before engaging in interaction-based learning. This preliminary action establishes a foundational knowledge base that enables the system to subsequently adapt more effectively to real-world interactions, reducing the overall complexity of the adaptive training process

Inventive Principle:

Principle #10Preliminary action

Solution Approach 2:

A user simulator is introduced as an intermediary during the interaction-based learning stage. The simulator provides controlled, realistic user interactions without requiring actual user time, allowing the system to gain adaptive experience while minimizing real-time loss. This intermediary enables reliable performance improvement without proportionally increasing training time

Inventive Principle:

Principle #24Intermediary (Mediator)

3Productivity

If only external data is used for training, then the training process is efficient and fast, but the system lacks the ability to learn from actual user interactions

Engineering Contradiction:

Improvetraining efficiencyVSAvoidlearning from user interactions

Core Design Contradiction:

ProductivityVSAdaptability or versatility

Solution Approach 1:

The training process is divided into two distinct stages: supervised learning stage and interaction-based learning stage. This segmentation allows the system to first learn from static external data with simple supervised learning, then progressively adapt to dynamic real-world interactions through controlled interactions with a user simulator, thereby resolving the contradiction between adaptability and training complexity

Inventive Principle:

Principle #1Segmentation

Solution Approach 2:

The training process maintains continuity by seamlessly transitioning from supervised learning on external data to interaction-based learning with the user simulator. Both stages contribute usefully to the system's overall performance, with the first stage building foundational knowledge efficiently and the second stage continuously refining adaptability through simulated user interactions

Inventive Principle:

Principle #20Continuity of useful action

Data Source

PatentUS10395646B2Two-stage training of a spoken dialogue system

Publication Date: 2019.08.27 MICROSOFT TECHNOLOGY LICENSING LLC

AI summary

Described herein are systems and methods for two-stage training of a spoken dialog system. The first stage trains a policy network using external data to produce a semi-trained policy network. The external data includes one or more known fixed dialogs. The second stage trains the semi-trained policy network through interaction to produce a trained policy network. The interaction may be interaction with a user simulator.