Document Retrieval Using Embedding Vector Semantic Matching
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
The rapid increase in internet data due to smart devices makes it difficult for users to retrieve and understand relevant document data, especially when natural language queries are used, often resulting in ineffective retrieval processes.
Innovation Solution
A method utilizing a computing device with a processor to determine embedding vectors for input retrieval words, comparing them to stored vectors using network models, and providing document data mapped to the most similar vectors, which includes natural language sentence, subject word, and keyword data, to enhance relevance in retrieval.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Quantity of substance
If a large quantity of documents are retrieved using natural language queries, then the coverage of retrieved data increases, but the retrieval accuracy and relevance decrease
Solution Approach 1:
The patent introduces embedding vectors as an intermediary between natural language queries and document retrieval. The query processing unit converts natural language into embedding vectors, which then serve as a bridge to match with document embedding vectors stored in the database. This intermediary mechanism enables semantic matching rather than simple keyword matching, thereby maintaining high retrieval accuracy even when retrieving large quantities of documents.
Solution Approach 2:
The patent transforms the retrieval process by changing the parameter representation from traditional keywords to embedding vectors in a multi-dimensional space. By representing both queries and documents as vectors with semantic meanings, the system can perform similarity calculations using distance metrics (e.g., cosine similarity, Euclidean distance). This parameter transformation allows for accurate relevance assessment across large document sets.
2Device complexity
If traditional keyword-based retrieval methods are used, then the system complexity remains low, but the retrieval relevance and semantic understanding deteriorate
Solution Approach 1:
The patent replaces the mechanical keyword-matching system with a semantic vector-space system. Instead of relying on exact keyword matches or simple Boolean operations, the system uses neural network-based embedding models to convert text into semantic vectors. The retrieval process then involves computing similarity in the vector space, which captures semantic relationships between queries and documents, significantly improving retrieval relevance.
Solution Approach 2:
The patent implements preliminary action by pre-computing and storing embedding vectors for documents in the database before retrieval occurs. During the retrieval process, only the query needs to be converted to an embedding vector, and the system can quickly compare it against pre-processed document vectors. This pre-processing step reduces the computational burden during actual retrieval while maintaining high semantic matching accuracy.
3Ease of operation
If natural language sentences are used as retrieval words, then the ease of operation improves, but the retrieval precision and matching accuracy worsen
Solution Approach 1:
The embedding vector acts as an intermediary that translates natural language sentences into a form suitable for precise matching. The query processing unit accepts easy-to-input natural language sentences and converts them into embedding vectors that capture semantic meaning. This intermediary representation enables the system to maintain both the ease of natural language input and the precision of computational matching.
Data Source
AI summary
According to some exemplary embodiments of the present disclosure, disclosed is a method for retrieving document data, which is performed by a computing device including at least one processor. The method may include: determining a first embedding vector by inputting retrieval word data into a first network model; determining a second embedding vector corresponding to the first embedding vector among a plurality of embedding vectors stored in a storage unit; and providing document data mapped to the second embedding vector.