Information Retrieval System Using Word Vector Similarity

Resolve Bottlenecks,
Find Innovative SolutionsGenerate Solutions

Solution Overview

Problem

Current information retrieval systems based on vector space and boolean models struggle to retrieve documents containing terms not present in the search space, limiting their ability to find relevant information beyond the provided documents.

Innovation Solution

A system utilizing a word vector database and a processor that calculates similarity between word vectors to select related terms, performs searches using these terms, and provides sorted results based on relevance scores, enhancing information retrieval efficiency.

Engineering Contradictions & Design Principles

VSEngineering Contradiction Analysis

1Adaptability or versatility

If traditional vector space and boolean models are used for information retrieval, then the system can find occurrences of terms and classify documents into topics, but it cannot retrieve documents by searching terms that are not present in the collection of documents

Engineering Contradiction:

Improveability to retrieve documents by searching terms not present in the collectionVSAvoideffectiveness of term matching and document classification

Core Design Contradiction:

Adaptability or versatilityVSReliability

Solution Approach 1:

The patent introduces word vectors as an intermediary representation layer between the query terms and the document collection. Word vectors capture semantic relationships and allow the system to search for concepts even when exact term matches are absent. The vector similarity calculation acts as a mediator that bridges the gap between literal term matching and semantic understanding.

Inventive Principle:

Principle #24Intermediary (Mediator)

Solution Approach 2:

The patent transforms the retrieval problem from exact term matching to vector similarity comparison. By changing the parameter from discrete term presence to continuous vector space distance, the system can retrieve documents based on semantic proximity rather than exact word matches, enabling searches for terms not present in the collection.

Inventive Principle:

Principle #35Parameter changes

2Productivity

If the system searches for exact term matches only, then the search is simple and fast, but the search results are limited to documents containing the exact terms

Engineering Contradiction:

Improvesearch speed and simplicityVSAvoidrelevant information not retrieved due to term absence

Core Design Contradiction:

ProductivityVSLoss of information

Solution Approach 1:

The patent performs preliminary action by pre-computing and storing word vectors for all terms in the document collection during an indexing phase. This preprocessing allows the system to quickly compute vector similarities during search without performing complex real-time analysis, maintaining search speed while enabling semantic retrieval for terms not explicitly present in the documents.

Inventive Principle:

Principle #10Preliminary action

Data Source

PatentUS11222055B2System, computer-implemented method and computer program product for information retrieval

Publication Date: 2022.01.11 SAP SE

AI summary

A system is provided, which may comprise: a word vector database configured to store word vectors, each of the word vectors representing a word or a phrase; a text database configured to store text data; and a processor configured to: receive a user input including a first term; obtain, for each word vector that is stored in the word vector database and that represents a word or a phrase other than the first term, a similarity between that word vector and a word vector representing the first term; select at least one second term based on the obtained similarity; perform a search in the text database using the at least one second term as a query for the search; and provide results of the search.