Event Data Parsing With Two-Stage Clustering for Efficient Classification
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing supervised machine learning models for classifying structured data, such as event data, are prone to errors and computationally costly, leading to inefficient use of power and processing resources.
Innovation Solution
Implementing an unsupervised clustering model with an infinite-dimensional vector space and applying rules for labeling input event groups, combined with a supervised machine learning model for ranking classifications, to improve accuracy and conserve resources.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If supervised machine learning models are used for classifying structured data, then classification accuracy can be achieved, but computational cost and power consumption increase significantly
Solution Approach 1:
The patent segments the classification task into two distinct stages: (1) an unsupervised clustering stage that groups similar events without requiring labeled training data, and (2) a supervised classification stage that applies rules to the pre-grouped clusters. This segmentation reduces the computational burden on the supervised model by limiting its operation to cluster-level classification rather than individual event classification, thereby reducing power consumption while maintaining accuracy.
Solution Approach 2:
The patent applies preliminary action by performing unsupervised clustering before supervised classification. The clustering process pre-processes the data by grouping similar events together, creating a simplified structure that the subsequent supervised model can work with more efficiently. This preliminary organization reduces the complexity of the classification task and decreases the computational resources and power needed for the final classification step.
2Measurement precision
If supervised machine learning models are used for classifying structured data, then classification can be performed, but processing resources are consumed inefficiently
Solution Approach 1:
The patent divides the processing workflow into two specialized components: an unsupervised clustering module that handles pattern recognition and grouping, and a supervised classification module that handles rule-based labeling. This segmentation allows each component to operate at optimal efficiency for its specific function, improving overall processing throughput while maintaining classification accuracy.
Solution Approach 2:
By performing clustering as a preliminary step before classification, the system reduces the dimensionality and complexity of the data that the supervised model must process. The clustering creates a condensed representation of the data that accelerates the classification process, thereby improving processing efficiency without sacrificing accuracy.
3Measurement precision
If traditional clustering methods are used, then data grouping can be achieved, but clustering accuracy is insufficient
Solution Approach 1:
The patent merges unsupervised clustering and supervised classification into a unified two-stage framework. The unsupervised clustering component handles the complex task of pattern recognition and group formation, while the supervised component applies rules to interpret and label the clusters. This merging allows the system to achieve high clustering accuracy through the unsupervised component while using the supervised component to refine and validate the results, without requiring a single overly complex model.
Solution Approach 2:
The patent introduces an intermediary layer between raw data and final classification: the clustering output serves as an intermediate representation that bridges the gap between unstructured event data and structured classifications. This intermediary structure simplifies the overall system complexity by providing a clear separation between pattern recognition (clustering) and interpretation (classification), allowing each component to remain relatively simple while achieving high overall accuracy.
Data Source
AI summary
In some implementations, a classification system may receive credentials associated with a data source and may receive, from the data source and using the credentials, a set of structured data including input events and output events. The classification system may filter the set of structured data by applying a first set of rules to generate a filtered set of structured data and may convert the filtered set of structured data to one or more numerical vectors, where a vector space associated with the one or more numerical vectors is infinite-dimensional. The classification system may further cluster the one or more numerical vectors using a first machine learning model to generate one or more clusters. Accordingly, the classification system may determine one or more classifications based on the set of structured data, each of the one or more classifications being associated with a corresponding frequency and a corresponding category.