Semantic Search Framework Using Word Embeddings for Survey Feedback
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing survey data mining systems are incapable of understanding the contextual meaning of user comments and filtering relevant information beyond exact keyword matches, limiting their ability to provide accurate insights.
Innovation Solution
A semantic search framework that utilizes word embeddings and similarity thresholds to analyze and retrieve survey data based on contextual meaning, enabling precise retrieval of relevant feedback and comments.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If exact keyword matching search is used, then search speed and simplicity are improved, but search accuracy and contextual understanding deteriorate
Solution Approach 1:
The system performs preliminary actions by pre-computing word embeddings for all survey data and storing them in a lookup table before search operations. This allows the semantic search to execute quickly by comparing query embeddings against pre-computed data embeddings, resolving the contradiction between search accuracy and processing complexity.
Solution Approach 2:
The patent introduces word embeddings as an intermediary representation layer between the search query and the survey data. Instead of directly comparing keywords or full text, the system transforms both into embedding vectors and computes similarity in the embedding space, enabling contextual understanding without requiring complex natural language processing during search execution.
2Loss of information
If semantic analysis with word embeddings is implemented, then contextual understanding and search relevance are improved, but computational complexity and processing time increase
Solution Approach 1:
The system pre-computes word embeddings for all survey responses and stores them in a lookup table before search operations. This preliminary action shifts the computational burden to an offline preprocessing stage, allowing fast online search by simply comparing query embeddings against pre-computed data embeddings using cosine similarity.
Solution Approach 2:
The patent applies partial action by using only the essential embedding similarity computation (cosine similarity) without implementing full natural language processing pipelines. This selective application of semantic analysis maintains contextual understanding while keeping processing time acceptable for interactive search.
Data Source
AI summary
Disclosed are systems and methods for a semantic search framework that operates to provide a robust search feature of database hosted data. The disclosed framework (or tool) improves and expands how searches can be configured and executed. In some embodiments, the disclosed framework is configured to receive a search request for a term or phrase, contextualize it to a generally understood theme that is not hindered by language barriers, and leverage it in a manner that is able to retrieve the most relevant and accurate results. The disclosed framework can filter search requests in a manner that both expands its breadth while honing in on what is actually being requested. The disclosed framework can be embodied as computerized systems and methods that can topically search for content based on a query string (e.g., term or phrase), and output a results set that embodies the theme of a survey's feedback.