Cubic Spline Feature Approximation for Non-Linear Model Encoding
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current methods for representing numerical features in data analytics, such as factorization machines, fail to accurately capture non-linear relationships due to information loss and sparsity issues, particularly in binning and direct numerical encoding.
Innovation Solution
Utilizing a set of spline functions to approximate non-linear relationships by transforming numerical features into a fixed range and computing basis function values weighted by machine-learned weights, minimizing information loss and sparsity.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If binning approach is used to encode numerical features, then non-linear relationships can be represented, but information loss occurs and representation becomes coarse-grained
Solution Approach 1:
The patent segments the numerical feature range into multiple bins, but instead of treating each bin as a single categorical value, it further segments each bin into multiple sub-bins. This hierarchical segmentation allows the model to capture non-linear relationships while preserving more granular information within each segment, thereby reducing information loss compared to traditional single-level binning.
Solution Approach 2:
The patent transforms the one-dimensional numerical feature into a multi-dimensional representation by creating multiple binary features for each bin and sub-bin combination. This dimensional transformation allows the model to represent non-linear relationships in a high-dimensional space while preserving the original numerical information through the structured binary encoding scheme.
2Adaptability or versatility
If direct numerical feature encoding is used in factorization machine, then linear relationships are represented, but non-linear relationships cannot be captured accurately
Solution Approach 1:
The patent changes the parameter representation by transforming continuous numerical values into discrete binary features through binning and sub-binning. This parameter transformation enables the factorization machine to model non-linear relationships by learning interactions between binary features, thereby increasing adaptability while maintaining representation precision through the structured transformation process.
3Loss of information
If finer-grained bins are used in binning approach, then information loss is reduced, but sparsity increases making training difficult
Solution Approach 1:
The patent applies hierarchical segmentation by dividing the feature range into bins and further dividing each bin into sub-bins. This segmented structure allows information preservation at multiple granularities while maintaining a manageable number of features through the hierarchical organization, thereby reducing sparsity compared to flat fine-grained binning.
Solution Approach 2:
The patent merges adjacent sub-bins within the same bin structure and combines features across different bins that share similar characteristics. This merging strategy reduces the total number of sparse features while preserving the fine-grained information structure, making the model more tractable for training without losing important details.
Data Source
AI summary
The present teaching relates to method, system, medium, and implementations for approximating a non-linear relationship between a numerical feature and an output of a model. A value of a numerical feature is received and is transformed, via a transform function, into a transformed value within a fixed range. With respect to each of a plurality of basis functions used for approximating the non-linear relationship, a respective basis function value of the basis function is computed based on the transformed value. An approximated value of the non-linear numeric feature is generated based on a sum of the plurality of basis function values weighted respectively by each corresponding one of a set of the weights, obtained via machine learning.