Vector-Based Text Representation for Search Relevance
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current search engine methodologies rely on simplistic keyword matching, failing to effectively utilize search results and subject matter indicators, leading to suboptimal content retrieval and limited relevance in search queries.
Innovation Solution
A data representation system that generates vector representations of keywords from document collections, utilizing taxonomy-based categorization and dimension-based analysis to compare query strings with content data, enhancing relevance and accuracy in content retrieval and advertising message serving.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If simple keyword matching is used to find content, then the search process is fast and simple, but the relevance and accuracy of search results deteriorates
Solution Approach 1:
The patent transforms the search matching process from simple keyword string comparison to vector space mathematical operations. Query strings and document contents are converted to vector representations, enabling similarity computation through dot products and cosine similarity calculations, which provide more nuanced relevance assessment than exact keyword matching
Solution Approach 2:
The patent replaces the mechanical keyword matching mechanism with a mathematical vector-based system. Instead of checking for exact string matches, the system uses vector embeddings and similarity metrics to determine relevance, substituting a rigid mechanical process with a more flexible mathematical approach
2Adaptability or versatility
If the scope of searched documents is expanded to include relevant documents without keyword hits, then more relevant content is found, but the complexity of the search system increases
Solution Approach 1:
The patent introduces vector representations as an intermediary layer between the query string and the document collection. This intermediary transformation enables the system to capture semantic meaning and relationships, allowing retrieval of relevant documents that would not match through direct keyword comparison alone
Solution Approach 2:
The patent moves the search problem from one dimension (keyword string matching) to another dimension (vector space similarity). By representing texts as vectors in multi-dimensional space, the system can measure relevance through geometric relationships, expanding the scope of retrievable documents beyond exact keyword matches
Data Source
AI summary
Embodiments of a data representation system for describing specific data sets, such as documents, web pages, or search engine queries, based on data tokens, such as words or n-grams, contained in a collection of documents are described. Such a system can be used in any type of information retrieval application, such as a document, web page, or online advertisement serving process, based on an information request, such as a query executed through an Internet search engine. For example, when a search is performed at a search engine, a content provider uses the system to represent the search query and compares the query representation against representations of a set of content in order to identify, retrieve and aggregate the content from the set most relevant to the search query, in the form of a web page or other data unit for display or access through the web browser.