LSTM Neural Networks for Automatic Syntactic Parse Tree Generation

Resolve Bottlenecks,
Find Innovative SolutionsGenerate Solutions

Solution Overview

Problem

Existing methods for generating parse trees for text segments require manual engineering of syntactic relationships and struggle with handling complex relations and varying input lengths, often necessitating binarization and special handling for unary productions or unknown words.

Innovation Solution

The use of long short-term memory (LSTM) neural networks to process input text segments, first converting them into an alternative representation and then generating a linearized parse tree representation using a second LSTM network, without the need for binarization or special handling, through a system implemented as computer programs on one or more computers.

Engineering Contradictions & Design Principles

VSEngineering Contradiction Analysis

1Adaptability or versatility

If manual engineering methods are used to generate parse trees, then syntactic relationships can be established, but the system cannot handle complex relations and varying input lengths effectively

Engineering Contradiction:

Improveability to handle complex syntactic relationshipsVSAvoidmanual engineering requirements

Core Design Contradiction:

Adaptability or versatilityVSDevice complexity

Solution Approach 1:

The patent replaces manual engineering of syntactic relationships with neural network-based automatic learning. The neural networks automatically learn complex syntactic relationships from training data, eliminating the need for manual rule engineering while handling varying input lengths and complex relations effectively.

Inventive Principle:

Principle #28Mechanics substitution (Replace mechanical system)

2Productivity

If binarization and special handling are applied to handle varying input lengths, then parse trees can be generated, but the process becomes more complex and requires special handling for unary productions or unknown words

Engineering Contradiction:

Improveparse tree generation capabilityVSAvoidspecial handling requirements

Core Design Contradiction:

ProductivityVSDevice complexity

Solution Approach 1:

The neural network architecture provides a universal solution that handles varying input lengths, unary productions, and unknown words without requiring special case handling. The networks process any input sequence directly, making the system multi-functional and eliminating the need for binarization or other special processing steps.

Inventive Principle:

Principle #6Universality (Multi-functionality)

3Extent of automation

If neural networks are used to process text segments, then automatic learning of syntactic relationships is enabled, but the computation time increases

Engineering Contradiction:

Improveautomatic learning capabilityVSAvoidcomputation time

Core Design Contradiction:

Extent of automationVSLoss of time

Solution Approach 1:

The neural networks are trained in advance on large corpora to learn syntactic relationships. This preliminary training allows the models to automatically process new text segments without requiring computation-intensive rule-based analysis during actual parsing, reducing inference time while maintaining automatic learning capabilities.

Inventive Principle:

Principle #10Preliminary action

Data Source

PatentUS10409908B2Generating parse trees of text segments using neural networks

Publication Date: 2019.09.10 GOOGLE LLC

AI summary

Methods, systems, and apparatus, including computer programs encoded on computer storage media, for generating parse trees for input text segments. One of the methods includes obtaining an input text segment, processing the input text segment using a first long short term memory (LSTM) neural network to convert the input text segment into an alternative representation for the input text segment, and processing the alternative representation for the input text segment using a second LSTM neural network to generate a linearized representation of a parse tree for the input text segment.