Hybrid Clustering Document Matching
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Pairwise comparison of all text documents is computationally intensive and infeasible in real-world scenarios, and limiting comparisons to a subset risks excluding similar documents, with existing methods providing generic results that encompass too many documents.
Innovation Solution
A method involving hybrid clustering to calculate numeric vector representations for documents, selecting a subset of nearest neighbors, and performing pairwise comparisons within this subset to generate similarity ratings and a ranked list of similar documents.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If pairwise comparison is performed over all possible pairs in a set of text documents, then measurement precision of similarity is improved, but computational resources required increase excessively
Solution Approach 1:
The patent segments the large set of text documents into multiple clusters based on topic similarity. Instead of performing pairwise comparisons across all documents, the method divides the document set into manageable clusters and performs comparisons within each cluster, significantly reducing computational resources while maintaining similarity measurement precision for relevant documents.
Solution Approach 2:
The patent performs preliminary clustering of documents into topic-based groups before conducting pairwise comparisons. This preliminary action organizes documents so that subsequent similarity comparisons only need to be performed within relevant clusters rather than across the entire document set, reducing computational burden while preserving measurement accuracy.
2Use of energy by moving object
If pairwise comparison is limited to a subset of text documents, then computational resources are reduced, but reliability of finding the desired best matching document decreases
Solution Approach 1:
The patent segments the document set into multiple topic-based clusters, ensuring that the desired matching document is not excluded by performing comparisons within relevant clusters rather than on a single arbitrary subset. This segmentation maintains reliability by organizing documents topic-wise so that similar documents are grouped together.
Solution Approach 2:
The patent changes the parameter of document selection from arbitrary subset selection to cluster-based selection based on topic similarity. By using topic similarity as the selection criterion, the method ensures that relevant documents are included in the comparison subset, maintaining reliability while reducing computational resources.
3Device complexity
If filtering is performed to select a cluster from the overall set, then computational complexity is reduced, but manufacturing precision of result quality deteriorates due to generic results encompassing too many documents
Solution Approach 1:
The patent segments the document set into multiple fine-grained topic-based clusters rather than using a single large cluster. This segmentation reduces computational complexity by limiting comparisons to smaller clusters while improving result quality precision by creating more specific, topic-relevant groups that encompass fewer but more relevant documents.
Solution Approach 2:
The patent changes the clustering parameter from generic topic grouping to fine-grained topic similarity-based grouping. By adjusting the clustering parameters to create more specific topic groups, the method reduces computational complexity while improving the precision of result quality by eliminating generic matches that encompass too many unrelated documents.
Data Source
AI summary
An approach for determining similar text documents. The approach can calculate a first set of vectors for a first cluster of text documents and a first comparison vector for a text document of interest. The approach can select a subset of text documents from the cluster of text documents based on comparing the vectors from the first set of vectors to the first comparison vector and picking a predetermined number of closest comparison text documents. The approach can calculate a second set of vectors for the subset of documents and second comparison vector for the document of interest. The approach can generate similarity ratings for the subset of documents based on pairwise comparisons of the second comparison vector and the second set of vectors. The approach can generate a ranked list of the second cluster of text documents based on the similarity ratings.