Recursive Agglomerative Clustering for Email Metadata

Resolve Bottlenecks,
Find Innovative Solutions
Generate Solutions

Solution Overview

Problem

Existing document classification methods for electronic mail messages are inefficient due to their inability to effectively utilize metadata and structure, leading to poor clustering results in large volumes of information.

Innovation Solution

The implementation of recursive agglomerative clustering methods that consider document metadata such as timestamps, sender/recipient identifiers, and message subjects, using a TF-IDF weighting scheme and a distance metric that combines time-sensitive and content-sensitive factors to improve clustering quality.

Engineering Contradictions & Design Principles

VSEngineering Contradiction Analysis

1Measurement precision

If existing document classification methods are used for electronic mail messages, then the classification process is simple, but the clustering results are poor due to inability to effectively utilize metadata and structure

Engineering Contradiction:
Improveclustering qualityVSAvoidclassification method complexity
Core Design Contradiction:
Measurement precisionVSDevice complexity

Solution Approach 1:

The patent segments the document classification process into multiple hierarchical clustering levels. Documents are first clustered into initial clusters, then these clusters are further clustered into super-clusters, and finally into mega-clusters. This multi-level segmentation allows the system to handle large volumes of documents effectively while utilizing metadata and structural information at each level, thereby improving clustering quality without overwhelming complexity.

Inventive Principle:
Principle #1Segmentation

Solution Approach 2:

The patent introduces temporal dimensions by incorporating timestamps into the clustering process. Documents are clustered not only based on content similarity but also on temporal proximity, creating time-aware clusters. This adds a temporal dimension to the traditional content-based clustering, improving the ability to capture user activity patterns and temporal aspects of document relationships.

Inventive Principle:
Principle #17Another dimension (Dimensionality change)

2Productivity

If large volumes of information are processed using traditional methods, then all documents can be classified, but navigation efficiency deteriorates due to excessive number of clusters

Engineering Contradiction:
Improvenavigation efficiencyVSAvoidnumber of clusters
Core Design Contradiction:
ProductivityVSQuantity of substance

Solution Approach 1:

The patent applies hierarchical segmentation to reduce the number of clusters presented to users. By organizing documents into multiple levels of clustering (initial clusters, super-clusters, mega-clusters), the system presents a manageable number of top-level clusters while maintaining detailed organization within each level. This allows efficient navigation through large document collections without overwhelming users with excessive cluster numbers.

Inventive Principle:
Principle #1Segmentation

Solution Approach 2:

The patent implements nested clustering where clusters are contained within larger clusters in a hierarchical structure. Initial clusters are nested within super-clusters, which are in turn nested within mega-clusters. This nested structure allows the system to represent large volumes of documents through a compact hierarchical organization, reducing the effective number of clusters users need to navigate while preserving detailed document organization.

Inventive Principle:
Principle #7Nested doll (Nesting)

3Measurement precision

If metadata and temporal information are incorporated into clustering, then clustering quality improves, but computational complexity increases

Engineering Contradiction:
Improveclustering qualityVSAvoidcomputational complexity
Core Design Contradiction:
Measurement precisionVSDevice complexity

Solution Approach 1:

The patent segments the computational process into multiple passes, each handling specific aspects of clustering. The first pass creates initial clusters based on content, the second pass creates super-clusters incorporating temporal information, and the third pass creates mega-clusters. This segmentation of computational tasks allows the system to incorporate metadata and temporal information systematically without overwhelming computational complexity in a single step.

Inventive Principle:
Principle #1Segmentation

Solution Approach 2:

The patent performs preliminary clustering based on content similarity before incorporating temporal and metadata information. By first establishing content-based clusters and then refining them with temporal and metadata considerations in subsequent passes, the system reduces computational complexity by breaking down the complex task into manageable preliminary steps rather than attempting to process all factors simultaneously.

Inventive Principle:
Principle #10Preliminary action

Data Source

PatentUS11074285B2Recursive agglomerative clustering of time-structured communications
Publication Date: 2021.07.27 VISIER SOLUTIONS
  • US11074285B2 patent drawing
  • US11074285B2 patent drawing
  • US11074285B2 patent drawing

AI summary

An example method of document clustering comprises: representing each document of a plurality of documents by a vector comprising a first plurality of real values, wherein each real value of the first plurality of real values reflects a first frequency-based metric of a term comprised by the document; partitioning the plurality of documents into a first set of document clusters based on distances between vectors representing the documents; representing each document cluster of the first set of document clusters by a vector comprising a second plurality of real values, wherein each real value of the second plurality of real values reflects a second frequency-based metric of a term comprised by the document cluster; and partitioning the first set of document clusters into a second set of document clusters based on distances between vectors representing the document clusters of the first set of document clusters.