Sparse SVM Training via Hemisphere Feature Screening
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Sparse Support Vector Machine (SPSVM) training is computationally intensive due to the need to solve multiple models with varying regularization parameters, especially in high-dimensional datasets, leading to significant computational iterations and processing time.
Innovation Solution
The system identifies inactive features using hemisphere constraints from an initial solution for a previous regularization parameter to exclude them in subsequent model solutions, thereby accelerating the training process by processing only active features.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If traditional SPSVM training processes all features for each regularization parameter, then model accuracy is maintained, but computational time and processing iterations increase significantly
Solution Approach 1:
The patent applies preliminary action by identifying and removing inactive features before they would be processed in subsequent regularization parameter iterations. The hemisphere constraint method predicts which features will have zero coefficients in future models based on the current solution, allowing the system to pre-filter the feature set and avoid unnecessary computational iterations on features that will ultimately be excluded.
Solution Approach 2:
The patent extracts and removes inactive features from the training process using hemisphere constraints. By calculating which features fall outside the hemisphere defined by the current weight vector and regularization parameter, the system extracts only the necessary active features for computation, significantly reducing the dimensionality of the problem in each iteration while maintaining model accuracy.
2Reliability
If SPSVM training processes high-dimensional datasets with all features, then comprehensive model evaluation is achieved, but computational complexity and resource requirements become prohibitive
Solution Approach 1:
The patent changes parameters by dynamically adjusting the effective feature dimensionality based on the regularization parameter value. As the regularization parameter changes across iterations, the set of active features identified through hemisphere constraints also changes, allowing the system to adaptively modify the problem size and computational requirements while ensuring that only relevant features are processed at each stage.
Solution Approach 2:
The patent segments the feature space into active and inactive regions using hemisphere constraints. By dividing the high-dimensional feature space into these two segments and processing only the active segment during training, the system reduces computational complexity while maintaining the reliability needed for comprehensive model evaluation. This segmentation is dynamically updated as regularization parameters change.
Data Source
AI summary
A system for machine training can comprise one or more data processors and a non-transitory computer-readable storage medium containing instructions which, when executed on the one or more data processors, cause the one or more data processors to perform operations including: accessing a dataset comprising data tracking a plurality of features; determining a series of values for a regularization parameter of a sparse support vector machine model, the series including an initial regularization value and a next regularization value; computing an initial solution to the sparse support vector machine model for the initial regularization value; identifying, using the initial solution, inactive features of the sparse support vector machine model for the next regularization value; and computing a next solution to the sparse support vector machine model for the next regularization value, wherein computing the next solution includes excluding the inactive features.


