System for identifying paraphrases using machine translation techniques

Abstract

The present invention obtains a set of text fragments from a cluster of different documents written about a common event. The set of text fragments is then subjected to text alignment techniques to identify paraphrases from text fragments in the text. The invention can also be used to generate paraphrases.

Description

technical field

[0001] The present invention relates to identifying paraphrases in text, and more particularly to using machine translation techniques to identify and generate paraphrases. Background technique

[0002] The identification and generation of paraphrases is a key aspect of many applications of natural language processing systems. Being able to recognize that two different pieces of text are equivalent in meaning enables the system to behave more intelligently. A fundamental goal of work in this area is to generate a program that is able to restate a piece of text while preserving its semantic content when manipulating features such as vocabulary, word order, reading level, and conciseness.

[0003] One exemplary application that could benefit from paraphrase identification and generation includes question answering systems. For example, consider the question "When did John Doe resign?", where the entity "John Doe" is a well-known person. Perhaps a large langu...

Images