Sentence Compression Using Bidirectional LSTM Networks
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Conventional methods for sentence compression rely on semantic analysis and linguistic preprocessing tools, which are complex and less accurate compared to the proposed method that uses Long Short-Term Memory (LSTM) neural networks for generating summarized sentences without requiring such tools.
Innovation Solution
The method involves tokenizing sentences and processing them using a first LSTM neural network to initialize the internal state of a second LSTM neural network, which then processes the tokens in a different order to determine whether each token should be included in the summarized version, achieving sentence compression with increased accuracy.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If conventional methods use semantic analysis and linguistic preprocessing tools for sentence compression, then the system can process sentence structure, but the device complexity increases and accuracy decreases
Solution Approach 1:
The patent replaces conventional semantic analysis and linguistic preprocessing tools with a neural network-based system. The neural network learns sentence compression patterns directly from data, substituting traditional rule-based mechanical processing with a learned model that achieves higher accuracy while simplifying the system architecture.
Solution Approach 2:
The patent changes the fundamental parameters of the sentence compression approach by transitioning from discrete linguistic rules to continuous neural network representations. The neural network processes sentences through learned embeddings and transformations, changing how linguistic information is represented and manipulated to achieve better compression results.
2Measurement precision
If the neural network processes tokens in multiple orders using bidirectional LSTM, then the measurement precision improves, but the loss of time increases
Solution Approach 1:
The patent applies preliminary action by processing tokens in both forward and backward directions simultaneously using bidirectional LSTM. This allows the network to capture contextual information from both directions before making compression decisions, improving accuracy without requiring multiple sequential passes that would increase processing time.
Data Source
AI summary
Methods, systems, apparatus, including computer programs encoded on computer storage medium, for generating a sentence summary. In one aspect, the method includes actions of tokenizing the sentence into a plurality of tokens, processing data representative of each token in a first order using an LSTM neural network to initialize an internal state of a second LSTM neural network, processing data representative of each token in a second order using the second LSTM neural network, comprising, for each token in the sentence: processing the data representative of the token using the second LSTM neural network in accordance with a current internal state of the second LSTM neural network to (i) generate an LSTM output for the token, and (ii) to update the current internal state of the second LSTM neural network, and generating the summarized version of the sentence using the outputs of the second LSTM neural network for the tokens.