Generalized SMO for SVM+ Training Optimization
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current methods for training support vector machines (SVMs) do not exist for the SVM+ technique, which is designed to leverage structures in training data like noise and invariants to improve generalization, despite SVMs being effective for classification and regression tasks.
Innovation Solution
A generalized Sequential Minimal Optimization (SMO) method is developed to efficiently train SVM+, which reduces the number of variables in a quadratic function using linear constraints and computes extreme points in closed form, allowing for fast computation even with large datasets.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If SVM+ technique is used to leverage structures in training data to improve generalization, then classification accuracy and generalization error are improved, but no training method exists for SVM+ making it impossible to efficiently compute optimal parameters
Solution Approach 1:
The patent segments the complex SVM+ training problem into smaller subproblems by applying sequential minimal optimization, where only two Lagrange multipliers are optimized at each iteration while others remain fixed, making the training process computationally tractable
Solution Approach 2:
The patent introduces an intermediary optimization approach by formulating a generalized SMO algorithm that acts as a bridge between the theoretical SVM+ framework and practical implementable training procedures
2Measurement precision
If standard quadratic programming methods are used to train SVM, then optimal hyperplanes can be found, but computational complexity increases significantly with large datasets
Solution Approach 1:
The patent divides the quadratic optimization problem into a sequence of simple 2-variable subproblems, optimizing two Lagrange multipliers at a time rather than solving the entire system simultaneously, which dramatically reduces computational complexity per iteration
Solution Approach 2:
The patent uses closed-form solutions for the 2-variable subproblems, performing preliminary analytical optimization before numerical iteration, which accelerates convergence compared to purely numerical methods
3Adaptability or versatility
If SVM+ uses more variables in the quadratic function to capture data structures, then model capacity and generalization improve, but the number of variables increases making computation more difficult
Solution Approach 1:
The patent segments the variable optimization space by fixing most Lagrange multipliers and only optimizing two at a time, reducing the effective dimensionality of each subproblem while maintaining the ability to handle the full variable set across multiple iterations
Solution Approach 2:
The patent transforms the high-dimensional optimization problem into a sequence of 2-dimensional problems, changing the dimensionality approach from solving all variables simultaneously to iterative 2D slices of the parameter space
Data Source
AI summary
A system and method for support vector machine plus (SVM+) computations include selecting a set of indexes for a target function to create a quadratic function depending on a number of variables, and reducing the number of variables to two in the quadratic function using linear constraints. An extreme point is computed for the quadratic function in closed form. A two-dimensional set is defined where the indexes determine whether a data point is in the two-dimensional set or not. A determination is made of whether the extreme point belongs to the two-dimensional set. If the extreme point belongs to the two-dimensional set, the extreme point defines a maximum and defines a new set of parameters for a next iteration. Otherwise, the quadratic function is restricted on at least one boundary of the two-dimensional set to create a one-dimensional quadratic function. The steps are repeated until the maximum is determined.