Paraphrase Generation Model Using Reinforcement Learning Feedback
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current paraphrase sentence generation methods in natural language processing have low accuracy due to the limitations of existing training methods based on parallel data, which do not effectively capture the nuances and variations in language.
Innovation Solution
A paraphrase sentence generation method and apparatus using a deep neural network-based paraphrase generation model trained through reinforcement learning, integrating a paraphrase matching model to improve the accuracy of generated paraphrases by adjusting model parameters based on feedback rewards, and incorporating an attention mechanism for dynamic word adjustment.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If a paraphrase generator is trained based on parallel data, then paraphrase sentences can be directly generated, but the accuracy of the generated paraphrase sentences is not high
Solution Approach 1:
The patent implements a feedback mechanism where the paraphrase matching model evaluates the quality of generated paraphrases and provides rewards to the paraphrase generation model. This closed-loop feedback system allows the generation model to continuously improve its output quality based on explicit quality assessments, resolving the contradiction between generation efficiency and accuracy.
Solution Approach 2:
The patent introduces a paraphrase matching model as an intermediary component between the input sentence and the final paraphrase output. This intermediary model acts as a quality filter and evaluator, assessing whether generated paraphrases truly capture the intended meaning, thereby improving overall paraphrase accuracy without compromising the direct generation process.
2Measurement precision
If reinforcement learning-based training is used to improve paraphrase accuracy, then the model can be continuously improved, but the system complexity increases
Solution Approach 1:
The patent segments the paraphrase generation system into two distinct but interconnected models: a paraphrase generation model for creating paraphrases and a paraphrase matching model for evaluating them. This segmentation allows each model to specialize in its function while working together through the reinforcement learning framework, managing system complexity through functional decomposition.
Solution Approach 2:
The patent utilizes parameter changes in the reinforcement learning process, where the paraphrase generation model adjusts its internal parameters based on rewards from the matching model. This parameter optimization approach enables continuous improvement of paraphrase quality through iterative training, balancing accuracy enhancement with manageable complexity through standard machine learning techniques.
3Measurement precision
If multiple second sentences are generated and filtered through matching degrees, then more accurate paraphrases are obtained, but the processing time increases
Solution Approach 1:
The patent generates m second sentences (where m > 1) and filters them through the matching model to determine n final paraphrases (where n ≤ m). This partial action approach generates more candidates than strictly necessary, allowing the matching model to select the best n results. The system balances accuracy and time by not requiring exhaustive generation while ensuring sufficient candidate quality through the matching filter.
Data Source
AI summary
A paraphrase sentence generation method and apparatus relating to the research field of natural language processing include generating m second sentences based on a first sentence and a paraphrase generation model, determining a matching degree between each of the m second sentences and the first sentence based on a paraphrase matching model, and determining n second sentences from the m second sentences based on matching degrees among the m second sentences and the first sentence, where the paraphrase generation model is obtained through reinforcement learning-based training based on a reward of the paraphrase matching model.