Contextual phrase analyzer

Abstract

A method and a computer system for implementing a contextual phrase analyzer engine are provided. The method includes selecting at least one of a plurality of document frequencies associated with a plurality of words used in a plurality of documents and selecting a subset of the plurality of words based on the at least one selected document frequency. The method also includes selecting at least one of the words in the subset of the plurality of words based on word frequencies associated with each word in the subset of the plurality of words.

Description

BACKGROUND OF THE INVENTION [0001] 1. Field of the Invention [0002] This invention relates generally to processor-based systems, and, more particularly, to a contextual phrase analyzer. [0003] 2. Description of the Related Art [0004] The large and growing pervasiveness of electronic documents is enriching the information environment available to users. However, the abundance of information often leads to cognitive overload as users attempt to locate relevant information within an almost infinite and constantly expanding universe of potentially related documents. Computer-based text processing may therefore be used to analyze large and complex sets of documents and to filter out extraneous information. For example, computer-based text processing may be used to retrieve relevant documents from a large document set based upon a query provided by a user. Exemplary computer-based text processing tasks include information retrieval, analysis, evaluation, synthesis, summarization, and the ...

AI Technical Summary

Problems solved by technology

However, the abundance of information often leads to cognitive overload as users attempt to locate relevant information within an almost infinite and constantly expanding universe of potentially related documents.

However, not all of the instances of the word “cat” are necessarily relevant to a user who is interested in documents including information about “house cats.” Thus, context identification may be a prerequisite for many text processing tasks.

For example, the word “cat” may be considered ambiguous when taken out of context and may be of limited usefulness for identifying documents that are relevant to a user interested in information about “house cats.”

Conventional computer-based text processing tools may have difficulty identifying relevant documents due in part to the sheer size of the information universe.

For example, the words “house” and “cat” may appear with a high frequency in documents that are not relevant to the subject of “house cats,” and some instances of the words “house” and / or “cat” may be irrelevant, even if they appear in a document that is relevant to the subject of “house cats.” Adding new documents to the document set may add new words and / or combination of words to the lexicon associated with the document set, which may lead to additional ambiguity and further complicate the task of the computer-based text processing tool.

Images