Data representation foundation for ai observability and explainability
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing machine learning systems face challenges in generating representative baseline reference data, leading to inconsistent anomaly detection, drift detection, and model explainability due to the mismatch between data formulation and actual data distribution, especially in high-dimensional datasets like images, and ineffective stratified sampling and greedy algorithms.
Innovation Solution
A system that performs content analysis on training datasets using mutual information (MI) and imbalance correction to generate ML-agnostic baseline reference data, applicable to image, text, and time series data, through partitioned mutual information (PMI) and data binning, ensuring accurate representation and explainability.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If random sampling or stratified sampling is used to generate baseline reference data, then the data formulation process is simple, but the representative accuracy of the baseline data mismatches the actual data distribution
Solution Approach 1:
The patent segments the baseline generation process into multiple stages: (1) partitioning the dataset into strata based on feature values, (2) calculating partitioned mutual information for each stratum, (3) selecting representative instances based on PMI thresholds, and (4) aggregating selected instances into the final baseline. This segmentation transforms a complex sampling problem into manageable sequential steps that improve representativeness while controlling complexity.
Solution Approach 2:
The patent introduces partitioned mutual information (PMI) as a new parameter to guide baseline selection, replacing traditional random or uniform stratified sampling. By calculating PMI between baseline instances and target instances across different partitions, the system dynamically adjusts selection criteria based on information content, thereby improving representative accuracy without requiring overly complex formulations.
2Reliability
If greedy algorithms are used to select representative baseline instances, then the selection process is computationally efficient, but the accuracy of anomaly detection and drift detection deteriorates
Solution Approach 1:
The patent performs preliminary partitioning of the dataset into strata based on feature values before selecting representative instances. By pre-organizing data into meaningful partitions and calculating PMI for each partition, the system establishes a structured foundation that guides subsequent instance selection. This preliminary action ensures that selected instances are representative of actual data distribution, improving anomaly detection accuracy while maintaining computational feasibility through the structured approach.
3Reliability
If baseline reference data is generated without considering data distribution, then the generation process is fast, but the consistency of model explainability and anomaly detection deteriorates
Solution Approach 1:
The patent segments the data space into multiple partitions based on feature values and calculates partitioned mutual information for each segment. This segmentation allows the system to efficiently identify representative instances from different regions of the data distribution without processing the entire dataset uniformly. The segmented approach ensures consistent model explainability across different data regions while reducing the computational time required compared to exhaustive methods.
Data Source
AI summary
Systems and methods are provided to generate improved sets of reference data that are ML model-agnostic. The system initiates an imbalance analysis on a training dataset (e.g., text, image, time series, etc.) that includes determining a set of classes in the data. Using the set of classes, the system processes mutual information (MI) across the data segments to generate a set of matrices from extracted partition-level mutual information. In some examples, the system may generate baseline reference data from the set of matrices and provide the baseline reference data for implementation with anomaly detection or model explainability in external machine learning (ML) models.