Extraction of facts from text

Abstract

A fact extraction tool set (“FEX”) finds and extracts targeted pieces of information from text using linguistic and pattern matching technologies, and in particular, text annotation and fact extraction. Text annotation tools break a text, such as a document, into its base tokens and annotate those tokens or patterns of tokens with orthographic, syntactic, semantic, pragmatic and other attributes. A user-defined “Annotation Configuration” controls which annotation tools are used in a given application. XML is used as the basis for representing the annotated text. A tag uncrossing tool resolves conflicting (crossed) annotation boundaries in an annotated text to produce well-formed XML from the results of the individual annotators. The fact extraction tool is a pattern matching language which is used to write scripts that find and match patterns of attributes that correspond to targeted pieces of information in the text, and extract that information.

Description

[0001] A portion of the disclosure of this patent document contains material which is subject to copyright protection. The copyright owner has no objection to the facsimile reproduction by anyone of the patent document or the patent disclosure, as it appears in the Patent and Trademark Office patent file or records, but otherwise reserves all copyright rights whatsoever. FIELD OF THE INVENTION [0002] The invention relates to the extraction of targeted pieces of information from text using linguistic pattern matching technologies, and more particularly, the extraction of targeted pieces of information using text annotation and fact extraction. BACKGROUND OF THE INVENTION [0003] Definitions and abbreviations used herein are as follows: [0004] Action—an instruction concerning what to do with some matched text. [0005] Annotation Configuration—a file that identifies and orders the set of annotators that should be applied to some text for a specific application. [0006] Annotations—attribu...

AI Technical Summary

Benefits of technology

[0134] It is therefore a primary object of the present invention to provide a fact extraction tool set that can extract targeted piece

Problems solved by technology

Regular expressions do have a problem recognizing some categories of tokens because there is little if any consistency in the structure of names in those categories, regardless of how many rules one might use.

There are also some language-specific issues that one runs into, for example: rules that recognize European language-based names in American English text often will stumble on names of Middle Eastern and Asian language origin; and rules developed to exploit capitalization patterns common in English language text may fail on languages with different capitalization patterns.

But this approach to pattern recognition soon falls apart with the addition of any linguistic complexity to the sentence, such as adding a word like only or pronoun references like her.

As sentences grow more complex, the process for annotating the text with these attributes also grows more complex—just as is seen with regular expression-based rule sets that target people names or other categories.

All of these processes can make mistakes, but because each tool feeds its results into the next one and each to

Images