Virtual Classifier-Based Change Analysis for High-Dimensional Data
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current change detection and analysis techniques, such as two-sample tests and supervised learning methods, are inadequate for identifying detailed changes between data sets, especially in high-dimensional spaces, and are not applicable to complex analysis targets like automobile fault diagnosis and customer profile analysis.
Innovation Solution
A virtual classifier-based method that assigns virtual labels to data sets, allowing for change detection and analysis by evaluating classification accuracy and significance using a binomial distribution threshold, and utilizing an attribute selection function to rank contributing attributes.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If two-sample tests are used to detect changes between data sets, then the significance of the difference can be determined, but detailed information on which attribute produces the change cannot be obtained in high-dimensional spaces
Solution Approach 1:
The patent segments the overall change detection problem into attribute-level sub-problems by training separate binary classifiers for each attribute. This segmentation allows the system to identify which specific attributes contribute to changes between data sets, rather than only detecting that a change occurred. The segmentation transforms a holistic detection task into granular attribute-specific analysis.
Solution Approach 2:
The patent introduces binary classifiers as intermediary components between the data sets and the change analysis. These classifiers act as mediators that process the high-dimensional data and produce interpretable outputs indicating attribute-level changes. The classifiers translate complex multivariate relationships into binary decisions about attribute changes, making the analysis tractable and informative.
2Loss of information
If supervised learning methods are applied to change analysis, then detailed attribute information can be obtained, but the methods are not applicable to complex analysis targets without prior labeled information
Solution Approach 1:
The patent inverts the traditional supervised learning approach by eliminating the need for labeled training data. Instead of requiring labeled examples of changes to train classifiers, the method uses the data sets themselves to train binary classifiers that detect differences between them. This inversion makes the approach applicable to unsupervised change detection scenarios while still providing detailed attribute-level information.
Solution Approach 2:
The patent enables the system to be self-sufficient by using the input data sets to automatically train the binary classifiers without external labeled information. The data sets serve their own training purpose, allowing the system to adapt to any complex analysis target regardless of whether labeled change examples are available. This self-service capability greatly enhances versatility.
3Productivity
If correlation rules are used to detect temporal changes in databases, then simple co-occurrence patterns can be identified, but important rules may be buried in trivial rules and the method is not applicable to complex analysis targets
Solution Approach 1:
The patent changes the fundamental parameter being measured from simple co-occurrence counts to binary classification outcomes for each attribute. This parameter change transforms the analysis from detecting trivial correlation patterns to identifying meaningful attribute-level differences. The binary classification framework filters out trivial rules by focusing on statistically significant attribute changes rather than all co-occurrence patterns.
Data Source
AI summary
Different virtual labels, for example, like +1 and −1, are assigned to two data sets. A change analysis problem for the two data sets is reduced to a supervised learning problem by using the virtual labels. Specifically, a classifier such as logical regression, decision tree and SVM is prepared and is trained by use of a data set obtained by merging the two data sets assigned the virtual labels. A feature selection function of the resultant classifier is used to rank and output both every attribute contributing to classification and its contribution rate.