Apparatus and Method for Conducting Searches with a Search Engine for Unstructured Data to Retrieve Records Enriched with Structured Data and Generate Reports Based Thereon

Abstract

Records in databases or unstructured files are enriched with metadata and are indexed for retrieval by a search engine. In response to a search request, a graphical user interface (GUI) control based on the metadata associated with the search hits is constructed and displayed with the search results in a standard view. Selection of a metadata value via the GUI control filters the previously matched records down to those matching the value selected via the GUI control. The metadata in the search results is arranged in a tabular view which is embedded in the display of search results and rendered invisible until selected by the user. Reports can be constructed from an identifier each returned record set for presenting, analyzing and modifying the data, and for generating further reports.

Description

BACKGROUND OF THE INVENTION[0001]The present invention provides a method for searching an index of structured and unstructured incoming data received from remote locations on a wide area network or global network, e.g. the Internet or an enterprise intranet. More specifically, the invention provides for capturing and enriching data records with metadata or appended data, and accessing the data through the use of a search engine designed for searching unstructured or free-form data.[0002]Such search engines are in common use. Examples presented herein have been specifically tested for use with the familiar Google® search appliance and Internet search engine. However, the teachings herein are adaptable for use with other search appliances and engines useful in searching records on the Internet and on private intranets, often configured by business enterprises to enable access to data from diverse locations, e.g., the open source Lucene search engine licensed by the Apache Software Fou...

AI Technical Summary

Benefits of technology

[0021]The present invention overcomes the aforementioned problems of prior art search engines in providing a method for zeroing in on the hits returned by a search that are most relevant to the user. The method of the invention winnows down the number of hits returned by a search request thereby enabling the user to find only the one or more relevant items from a potentially much larger list of search results obtained from an inquiry to a search engine.

[0028]When the user clicks on a value within a node, the search results, which can number 1000 or more snippets corresponding to 1000 or more respective records, are filtered by passing a further search request as a meta query, i.e., a query to search for the selected tag-value pairs in the metadata with which the records have been enriched, and only the URLs containing that value remain displayed. Hence, the user is able to narrow the search results based on data contained in the search results, without having to further refine the original search request. Using the metadata to construct a search results navigation tree allows the end user to immediately perceive the underlying structure of the search results and to leverage the knowledge of the data to refine the initial search.

[0036]Providing reports of the search results in dynamic tables allows a user to analyze the search results while not connected to the Internet or other network, and also allows a user to email the dynamic table to other users who can perform further analysis on the search results. In this way, the usefulness of the search results is extended beyond mere observation of the results to analysis of the results and retrieval of further relevant information.

Problems solved by technology

This limitation is pragmatic since the expectation is that if a user does not find the results within the top most relevant hits, it will be more efficient to refine the query than to page through all one million hits.

This process is inefficient, because:

Such snippets can also be misleading.

There is no guarantee that replacing the old results with new results will be more useful given that the user refines the search without much knowledge about the structure content of all 1000 previous results.

Even though the number of records having information of interest to a searcher might be very small, the number of hits could occupy many pages, most containing irrelevant information, making it very difficult for the searcher to find what was wanted.

If they are offline, they loose even the ability to sort by relevance or date, hence storing search results has little usefulness.

These limitations severely constrain the ability of users to efficiently analyze and manipulate search results to make faster and more informed decisions.

While this limitation may not be as obvious when searching completely unstructured data, such as word processing documents, it becomes quickly apparent when users search structured data sources.

A mere sequential listing of these records is not very useful.

Prior art search systems fail to make analysis, manipulation and storing of search results meaningful.

Images