Multisensor Open Set Classification Using Model Agreement Graphs
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Conventional machine learning models struggle to handle real-world scenarios where data includes unknown classes, especially in multisensor environments, requiring complex sensor fusion and retraining when new sensors are deployed, and lack effective methods for identifying unknown classes without ground truth labeled data.
Innovation Solution
A heterogeneous model ensemble approach that leverages model agreement graphs to identify unknown classes by comparing current and historical agreements among multiple models, allowing for open set classification without requiring open-set models or complex sensor fusion.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Adaptability or versatility
If sensor fusion is performed to handle multisensor environments, then the system can integrate combined sensory information, but the complexity of real-time processing increases and all models must be retrained when data representation is updated
Solution Approach 1:
The patent divides the classification task into multiple independent ML models, each trained on different sensor data representations. Instead of fusing all sensor data into a single representation, each model processes its own representation independently, then their predictions are combined. This segmentation avoids the complexity of real-time sensor fusion while maintaining the ability to handle multisensor environments.
Solution Approach 2:
The patent performs sensor fusion and data representation transformation in advance during the training phase, not in real-time during inference. Multiple models are pre-trained on different sensor representations, so when new sensors are deployed, the system can add new pre-trained models without performing complex real-time fusion or retraining existing models.
2Ease of operation
If conventional ML models are used that assume MECE mapping, then classification is straightforward for known classes, but the models cannot identify unknown classes that were not in training data
Solution Approach 1:
The patent introduces an intermediary mechanism - an ensemble of multiple ML models with different training representations - that mediates between the simplicity of conventional classification and the ability to detect unknown classes. When all models in the ensemble agree on a classification, the system outputs that class; when models disagree significantly, the system identifies the input as potentially belonging to an unknown class, thus detecting novelty without requiring each individual model to be complex.
3Adaptability or versatility
If models are retrained when new sensors are deployed to update data representation, then the models adapt to new sensor types, but the retraining process is time-consuming and computationally expensive
Solution Approach 1:
The patent performs the computationally expensive sensor fusion and data representation transformation in advance during the training phase. Multiple models are pre-trained on different sensor representations before deployment. When new sensors are deployed, the system can add new pre-trained models without performing complex real-time fusion or retraining existing models, thus avoiding time loss during operational updates.
Data Source
AI summary
One example method includes identifying active device data streams, based on the active device data streams identified, selecting applicable ML models, obtaining a respective classification inference from each of the ML models, wherein each of the classification inferences applies to a respective active device data stream, identifying a current agreement between the respective classification inferences generated by the models, comparing the classification inferences associated with the current agreement with historical classification inferences associated with historical agreements, and based on the comparing, determining whether or not one or more of the data streams comprises a piece of data of an unknown class.