kPOOL Search System Using Latent Semantic Analysis Clustering
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Conventional search methods face challenges with imprecise and vague searches due to incorrect input queries and lack of effective methods for structuring data to retrieve relevant information efficiently and visualize search results in a coherent manner.
Innovation Solution
The system, named kPOOL, employs modified Latent Semantic Analysis techniques to create a vector space model and uses optimal agglomerative clustering to form a hierarchical structure of documents, allowing users to efficiently search and visualize semantic relationships between documents through a GUI fisheye view and self-organizing maps.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If conventional search methods use keyword matching against a corpus, then the search engine can process queries efficiently, but the search results become imprecise when users input incorrect information or lack knowledge of technical language
Solution Approach 1:
The patent introduces an intermediary layer between the user query and the document corpus. This intermediary is the latent semantic analysis model that translates user queries into conceptual vectors and matches them against conceptually indexed documents, rather than directly matching keywords. The intermediary model captures semantic relationships and retrieves documents based on conceptual similarity, thereby improving search accuracy without sacrificing processing efficiency.
Solution Approach 2:
The patent changes the fundamental parameter of search matching from keyword frequency matching to semantic concept matching. By transforming documents and queries into vector representations in a latent semantic space, the system compares documents based on their conceptual meanings rather than surface-level keyword overlaps. This parameter change enables the system to handle imprecise and vague queries effectively, retrieving relevant documents even when the user's terminology differs from the indexed terminology.
2Measurement precision
If traditional Latent Semantic Analysis techniques are used to search underlying concepts, then imprecise and vague search problems are addressed, but the system fails to provide efficient methods for users to review retrieved data and visualize search results
Solution Approach 1:
The patent adds a visual dimension to the search results presentation by implementing self-organizing maps that project high-dimensional semantic relationships into two-dimensional spaces. This dimensional transformation enables users to visually perceive semantic connections between documents and concepts, making it easier to review and navigate retrieved data. The visual representation complements the traditional list view, providing intuitive insight into the semantic structure of search results.
Solution Approach 2:
The patent segments the search results into hierarchical clusters based on semantic similarity, organizing documents into a tree-like structure where parent nodes represent broader concepts and child nodes represent more specific documents. This segmentation allows users to navigate and review results systematically, drilling down from general to specific topics. The hierarchical organization makes large result sets more manageable and easier to explore, directly addressing the ease of operation concern.
3Productivity
If the system creates a hierarchical structure of documents through optimal agglomerative clustering, then users can efficiently navigate and search through domain corpus, but the system complexity increases
Solution Approach 1:
The patent performs preliminary action by pre-computing the hierarchical clustering structure during an indexing phase before actual search queries are processed. The optimal agglomerative clustering algorithm organizes documents into a hierarchical tree structure in advance, so that during query processing, the system can efficiently navigate this pre-built structure rather than performing complex clustering operations in real-time. This preliminary organization significantly improves search efficiency while keeping the operational complexity manageable.
Solution Approach 2:
The patent creates a simplified copy of the document structure in the form of a hierarchical index that captures the essential semantic relationships without storing all original document data at every level. This compressed representation allows the system to maintain the full semantic structure for efficient searching while reducing the operational complexity during query processing. The hierarchical index acts as a navigational copy that preserves structural information without requiring complex real-time computations.
Data Source
AI summary
The disclosed embodiments provide a system and method for using modified Latent Semantic Analysis techniques to structure data for efficient search and display. The present invention creates a hierarchy of clustered documents, representing the topics of a domain corpus, through a process of optimal agglomerative clustering. The output from a search query is displayed in a fisheye view corresponding to the hierarchy of clustered documents. The fisheye view may link to a two-dimensional self-organizing map that represents semantic relationships between documents.