Systems, methods, software, and interfaces for multilingual information retrieval

Abstract

The present inventors have a devised one or more novel methods, systems, and interfaces for facilitating multi-lingual searches. One exemplary method entails creating multiple language-specific indices for a collection of documents, with each index including stemmed and non-stemmed versions of terms from the documents. Users submit queries that are associated with a set of one or more target languages. Query processing entails translating original and stemmed versions of each term in a query into each of the target languages, using one or more techniques that each yield a set of potentially equivalent query terms. Each set of potentially equivalent query terms is then processed against the corresponding language-specific index, using a conventional monolingual search technique, such as a Boolean or natural language query, to identify documents from the collection. The resultant documents are presented to the user in language groupings or by computed relevance.

Description

technical field [0001] This application claims priority to US Provisional Application 60 / 641,669, filed January 4, 2005, which is hereby incorporated by reference. [0002] Various embodiments of the present invention relate to information retrieval, particularly multilingual or cross-lingual information retrieval systems, methods and software. Background technique [0003] The importance of search engine technology has grown considerably over the past decade or so, reflected in the expansion and use of the Internet. When a user clicks the search button, the search engine searches through tens of millions of items to find an item and corresponding document that satisfies the query. However, this superficial simplicity conceals the complexity of the underlying search technology, because a good search engine generally does not stop at simple matching of query items. [0004] To understand this complexity, consider that search engines generally fall into two categories: monol...

AI Technical Summary

Problems solved by technology

In such cases, the challenge of effective search is compounded because in non-English languages, nouns can be masculine, feminine, or neuter; verbs change forms to indicate numbers (single or multiple), display tense (present, past, future, etc.), and revealing persons (first-person "I," second-person "you," and third-person "he / she / it"); adjectives change form according to the noun they modify; and punctuation marks ( such as grave accents or other diacritics) greatly affect the meaning

While stemming addresses these complexities in monolingual search, stemming by itself does not address the added complexity of linguistic conflicts between languages, and can even interfere in some cases.

Images