Dialog Model Response Re-ranking via Semantic Similarity
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Conventional dialog generation methods based on neural networks face issues with uneven word distribution in training sets, leading to higher-ranked responses being unsuitable and suitable responses being ranked lower, affecting the quality of generated responses.
Innovation Solution
A method that re-ranks responses by obtaining similar dialogs semantically, calculating probabilities, and statistically analyzing scores to adjust the ranking of responses generated by a dialog model, ensuring higher-ranked responses are more suitable for user inputs.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Adaptability or versatility
If a neural network dialog model generates responses based on training data, then the model has good extensibility and adaptability, but the uneven distribution of words in the training set causes higher-ranked responses to be unsuitable and suitable responses to be ranked lower
Solution Approach 1:
The patent introduces an intermediary re-ranking mechanism that sits between the neural network model's initial response generation and the final response selection. This re-ranking module uses semantic similarity calculation and probability analysis to adjust the ranking of generated responses, mediating the conflict between the model's inherent biases from uneven training data distribution and the need for high-quality suitable responses.
Solution Approach 2:
The patent implements a feedback mechanism where the system calculates semantic similarity between user inputs and retrieved dialog samples, then uses this similarity information along with probability analysis to re-rank responses. This feedback loop allows the system to correct ranking errors from the initial model generation by incorporating additional semantic information from the training data.
2Productivity
If the dialog model ranks responses by probability, then responses are generated automatically, but the uneven word distribution causes probability-based ranking to fail in identifying suitable responses
Solution Approach 1:
The patent changes the ranking parameters by introducing semantic similarity scores and probability analysis results as additional criteria beyond the model's inherent probability ranking. By transforming the single-parameter probability ranking into a multi-parameter evaluation system, the patent addresses the insufficiency of probability-based ranking in assessing response suitability.
Solution Approach 2:
The patent adds another dimension to the response evaluation by incorporating semantic similarity measurement from retrieved dialog samples. This transforms the one-dimensional probability ranking into a two-dimensional evaluation space that includes both probability and semantic similarity, allowing for more accurate assessment of response suitability.
3Manufacturing precision
If similar dialogs are retrieved and probability analysis is performed, then response quality improves, but the system complexity increases
Solution Approach 1:
The patent performs preliminary actions by retrieving similar dialog samples from the training data before the final response selection. This pre-processing step of gathering relevant semantic context in advance allows the subsequent re-ranking to be more effective without significantly increasing overall system complexity, as the retrieval operation can be efficiently cached and reused.
Data Source
AI summary
A method and an apparatus for ranking responses of a dialog model, and a non-transitory computer-readable recording medium are provided. The dialog model is trained based on a sample data set. The method includes obtaining, from the sample data set, at least one similar dialog whose content is semantically similar to content of a target dialog; obtaining a probability of at least one target response generated by the dialog model when inputting the target dialog, and obtaining a probability of a target response generated by the dialog model when inputting the similar dialog; statistically analyzing, based on the probabilities of the respective generated target responses, scores of the target responses, the scores of the target responses being positively correlated with the probabilities of the target responses; and ranking the target responses in a descending order of the scores.