Min-wise Hashing for Scalable Feature Generation

Solution Overview

Problem

Conventional machine learning approaches struggle to scale effectively for large data sets with complex, non-linear relationships between input variables and target variables, leading to inefficiencies in resource usage and prediction accuracy.

Innovation Solution

The implementation of a min-wise hashing algorithm to efficiently generate approximate co-occurrence counts and correlation metrics, allowing for the selection of high-order features that are highly correlated with the target variable, thereby reducing the computational effort required to identify predictive features for linear models.

Engineering Contradictions & Design Principles

VSEngineering Contradiction Analysis

1Measurement precision

If all derivable higher-order features are expanded to capture non-linear relationships, then prediction accuracy is improved, but computation and storage requirements become unsustainable

Engineering Contradiction:

Improveprediction accuracyVSAvoidcomputation and storage requirements

Core Design Contradiction:

Measurement precisionVSQuantity of substance

Solution Approach 1:

Instead of generating all possible higher-order features (excessive action), the patent applies partial action by using min-wise hashing to approximate co-occurrence counts for a manageable subset of features. This allows capturing non-linear relationships through selective feature expansion rather than exhaustive expansion, significantly reducing computation and storage requirements while maintaining predictive accuracy.

Inventive Principle:

Principle #16Partial or excessive action

Solution Approach 2:

The patent creates a compressed representation (copy) of the data through min-wise hashing signatures. These signatures approximate the co-occurrence counts of higher-order features without actually storing or computing all such features. The signatures serve as efficient proxies that capture the essential statistical relationships between features and target variable, reducing the data representation from potentially billions of features to a manageable size.

Inventive Principle:

Principle #26Copying

2Productivity

If down-sampling data sets is performed to handle large data sizes, then computation requirements are reduced, but important information is lost

Engineering Contradiction:

Improvecomputation efficiencyVSAvoidinformation loss

Core Design Contradiction:

ProductivityVSLoss of information

Solution Approach 1:

The patent creates a compressed representation (copy) of the data through min-wise hashing signatures. These signatures approximate the co-occurrence counts of higher-order features without actually storing or computing all such features. The signatures serve as efficient proxies that capture the essential statistical relationships between features and target variable, reducing the data representation from potentially billions of features to a manageable size.

Inventive Principle:

Principle #26Copying

Solution Approach 2:

The patent transforms the data representation from raw feature combinations to hashed signature parameters. By changing the representation parameters through min-wise hashing, the system maintains the essential statistical information needed for prediction while reducing the data volume. This parameter transformation preserves information about feature co-occurrence patterns without requiring actual down-sampling of the original data.

Inventive Principle:

Principle #35Parameter changes

3Productivity

If min-wise hashing is used to approximate co-occurrence counts, then computational effort is reduced, but approximation error is introduced

Engineering Contradiction:

Improvecomputational efficiencyVSAvoidapproximation accuracy

Core Design Contradiction:

ProductivityVSMeasurement precision

Solution Approach 1:

The patent replaces the mechanical/computational system of exact co-occurrence counting with a probabilistic hashing system. Instead of mechanically counting all feature combinations (which is computationally intensive), the system uses min-wise hashing to probabilistically approximate these counts. The hashing mechanism substitutes direct computation with a more efficient probabilistic approach that achieves sufficient accuracy for machine learning purposes.

Inventive Principle:

Principle #28Mechanics substitution (Replace mechanical system)

Data Source

PatentUS11295229B1Scalable generation of multidimensional features for machine learning

Publication Date: 2022.04.05 AMAZON TECH INC

AI summary

An approximate count of a subset of records of a data set is obtained using one or more transformation functions. The subset comprises records which contain a first value of one input variable, a second value of another input variable, and a particular value of a target variable. Using the approximate count, an approximate correlation metric for a multidimensional feature and the target variable is obtained. Based on the correlation metric, the multidimensional feature is included in a candidate feature set to be used to train a machine learning model.