Document Clustering Hierarchy Segmentation for E-Discovery
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current document clustering methods, such as agglomerative and partitional clustering, face challenges with large document sets, where agglomerative clustering is slow and partitional clustering may not provide accurate results, leading to inefficient document review processes in electronic discovery.
Innovation Solution
A method involving the initial clustering of documents based on one field, followed by merging and re-clustering on another field, using a combination of agglomerative and partitional clustering approaches to create a balanced and efficient cluster hierarchy, optimizing both accuracy and execution time.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If agglomerative clustering is used to cluster documents, then clustering accuracy is improved, but execution time increases significantly for large document sets
Solution Approach 1:
The patent divides the document clustering process into multiple stages: first clustering documents into initial clusters using one set of criteria, then merging clusters at different levels, and finally re-clustering merged clusters using additional criteria. This segmentation allows the system to achieve high accuracy through multiple passes while reducing overall execution time by avoiding a single comprehensive clustering operation on all documents simultaneously.
Solution Approach 2:
The patent performs preliminary clustering of documents into initial clusters before merging and re-clustering operations. This preliminary action organizes documents into manageable groups that can be efficiently merged and re-clustered, reducing the computational complexity of subsequent operations while maintaining clustering accuracy.
2Loss of time
If partitional clustering is used to cluster documents, then execution time is reduced, but clustering accuracy deteriorates
Solution Approach 1:
The patent combines multiple clustering approaches by merging initial clusters formed through one method with re-clustering operations using additional criteria. This merging process integrates the efficiency of partitional clustering with the accuracy benefits of multi-stage clustering, achieving both reduced execution time and maintained accuracy through the synthesis of different clustering strategies.
3Measurement precision
If documents are clustered into a detailed cluster hierarchy, then document review accuracy is improved, but the complexity of managing and navigating the hierarchy increases
Solution Approach 1:
The patent segments the cluster hierarchy into multiple levels with varying degrees of detail. Lower levels contain more specific clusters formed through detailed re-clustering operations, while upper levels contain broader clusters from initial clustering. This segmentation allows users to navigate the hierarchy efficiently by starting at higher levels and drilling down only when necessary, reducing overall management complexity while maintaining review accuracy.
Solution Approach 2:
The patent creates a dynamic cluster hierarchy where clusters at different levels can be merged or split based on the specific clustering operation being performed. This dynamic structure allows the hierarchy to adapt to different clustering criteria and document sets, making it easier to manage and navigate while maintaining high review accuracy through flexible organization.
Data Source
AI summary
In an automatic electronic discovery search tool, documents can be clustered into a cluster hierarchy using data contained in a first field of the documents. One of more levels of the cluster hierarchy are merged. Merged clusters are further clustered using data contained in a second field of the documents. The first field and second field may be determined by the signal-to-noise ratio of the data contained in the fields. Clusters may be filtered or used in a document review tool.