Dialogue Model Generation Using Discrete Hidden Variables
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing dialogue models based on large-scale pre-training, such as BERT, generate single and insufficient response content, making it impossible to accurately respond to user inputs.
Innovation Solution
A method and apparatus for generating a dialogue model by acquiring a corpus sample set, classifying corpus samples to set discrete hidden variables, and training a conversion neural network using a one-way and two-way attention mechanism with loss functions like negative log likelihood, bag of words, and response selection to represent the relationship between input and response information.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Loss of information
If a dialogue model is based on large-scale pre-training (e.g., BERT), then the model can process input information, but the response content generated is single and lacks information
Solution Approach 1:
The patent segments the response generation process into multiple independent pathways by introducing discrete hidden variables that categorize different response types (e.g., inquiry, statement, question). Each pathway is trained separately with specific loss functions, allowing the model to generate diverse response types while maintaining information completeness. This segmentation resolves the contradiction by enabling multiple response styles without sacrificing information quality.
Solution Approach 2:
The patent adds a new dimension to the response generation by introducing discrete hidden variables as an additional layer of classification beyond the standard input-output mapping. This dimensional expansion allows the model to consider multiple response categories simultaneously, transforming a single-dimension generation process into a multi-dimensional one that produces both diverse and information-rich responses.
2Adaptability or versatility
If discrete hidden variables are introduced to model 'one-to-many' relationships, then response diversity improves, but model complexity increases
Solution Approach 1:
The patent applies local quality by assigning different loss functions to different discrete hidden variable categories rather than using a single uniform loss function for all responses. For example, inquiry responses use one loss function while statements use another. This localized approach manages complexity by treating each response type with appropriate specialized handling rather than a monolithic complex structure.
Solution Approach 2:
The patent performs preliminary classification of responses into discrete hidden variable categories before generating the actual response content. This preliminary action organizes the complexity by pre-defining response types and their corresponding generation pathways, reducing the computational burden during inference and making the overall complex model more manageable through structured preprocessing.
Data Source
AI summary
Embodiments of the present disclosure relate to a method and apparatus for generating a dialogue model. The method may include: acquiring a corpus sample set, a corpus sample including input information and target response information; classifying corpus samples in the corpus sample set, setting discrete hidden variables for the corpus samples based on a classification result to generate a training sample set, a training sample including the input information, the target response information, and a discrete hidden variable; and training a preset neural network using the training sample set to obtain the dialogue model, the dialogue model being used to represent a corresponding relationship between inputted input information and outputted target response information.