Clustering Documents Using Input Output Space Similarity
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current data clustering methods are inadequate for handling heterogeneous data types, often resulting in clusters with heterogeneous variables, which complicates data analysis and interpretation, especially in applications like medical monitoring systems where data types can be categorical, ordinal, or real.
Innovation Solution
A computer-automated method that clusters documents by considering both input and output space data, using textual similarity measures for input space and numeric similarity measures for output space, forming hierarchical trees and performing depth-first searches to aggregate documents into clusters based on similarity thresholds, and refining clusters based on combined input and output space similarity measures.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If conventional clustering methods are used, then clustering speed is improved, but cluster homogeneity deteriorates
Solution Approach 1:
The patent segments the clustering process into two distinct phases: (1) initial clustering based on input space similarity to achieve fast grouping, and (2) refinement phase based on output space similarity to achieve homogeneous clusters. This segmentation allows each phase to optimize for its specific goal without compromising the other.
Solution Approach 2:
The patent introduces a second dimension of analysis by considering both input space (query characteristics) and output space (document characteristics) similarity. This dual-dimensional approach enables the system to achieve both speed (through input space clustering) and homogeneity (through output space refinement) simultaneously.
2Quantity of substance
If heterogeneous data types are clustered together, then data volume is reduced, but data interpretability deteriorates
Solution Approach 1:
The patent applies local quality by ensuring that each cluster maintains homogeneous data types and characteristics. Instead of uniform clustering across all data, the system adapts the clustering criteria to the specific local characteristics of each data region, preserving interpretability while still achieving data consolidation.
Solution Approach 2:
The patent enforces homogeneity within clusters by using output space similarity measures that ensure documents grouped together have similar characteristics and data types. This maintains data interpretability while still achieving the benefit of consolidated data representation.
3Productivity
If input space similarity only is used, then clustering efficiency is improved, but cluster accuracy deteriorates
Solution Approach 1:
The patent performs preliminary clustering based on input space similarity to quickly organize documents into candidate groups. This preliminary action provides a good initial structure that is then refined in the second phase, combining the efficiency of input-based clustering with the accuracy of output-based refinement.
Solution Approach 2:
The patent uses output space similarity as feedback to refine and adjust the initial clusters formed by input space similarity. This feedback mechanism allows the system to correct inaccuracies in the preliminary clustering while maintaining the efficiency benefits of the initial fast grouping.
Data Source
Figure 1
Figure 2
Figure 3
AI summary
A method of clustering a plurality of documents having input and output space data is disclosed that uses both input and output space criteria. The method can include aggregating documents into clusters based on input and/or output space similarity measures, and then refining the clusters based on further input and/or output space similarity measures. Aggregating the documents into clusters can include forming a hierarchical tree based on the input and/or output space similarity measures where the hierarchical tree has a root node, branching into intermediate nodes, and branching into leaf nodes covering individual documents, where the hierarchical tree includes a leaf node for each document of the plurality of documents. The method can then include forming a forest of sub-trees of the hierarchical tree based on cluster criteria. Textual and numeric similarity measures can be used depending on the type and distribution of data in the input and output spaces.