Transductive SVM Training via Concave-Convex Decomposition
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Conventional transductive support vector machines (TSVMs) face inefficiencies when dealing with a large number of unlabeled examples, as existing methods are intractable or practical only for a limited number of examples, and fail to scale well with high-dimensional data.
Innovation Solution
The TSVM objective function is decomposed into a convex and a concave function, allowing for iterative approximation and minimization, with a loss function for unlabeled data that duplicates examples to associate costs with classifying them, and a balancing constraint to ensure class ratios are maintained, using the Concave-Convex Procedure (CCCP) to solve the non-convex problem.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If conventional TSVM methods are used to handle unlabeled examples, then classification accuracy can be improved, but computational complexity becomes intractable for large numbers of examples
Solution Approach 1:
The patent segments the TSVM optimization problem into two separate subproblems: a primal problem that optimizes the decision boundary using only labeled data, and a dual problem that handles unlabeled data through iterative reweighting. This segmentation allows each subproblem to be solved independently and efficiently, avoiding the intractable joint optimization of conventional TSVM methods while maintaining the ability to leverage both labeled and unlabeled data for improved classification accuracy.
2Adaptability or versatility
If conventional TSVM methods are used, then the model can utilize unlabeled data, but the training time scales poorly with data size
Solution Approach 1:
The patent performs preliminary action by first solving the primal problem to obtain an initial decision boundary using only labeled data. This initial solution provides starting weights for the unlabeled data that guide the subsequent dual problem optimization. By preparing this preliminary classification framework before incorporating unlabeled data, the method avoids the computational burden of simultaneously optimizing both labeled and unlabeled examples from scratch, significantly reducing training time while still achieving effective utilization of unlabeled data.
3Manufacturing precision
If exact TSVM optimization is performed, then optimal hyperplane is obtained, but computational resources required become prohibitive for large datasets
Solution Approach 1:
The patent applies partial action by solving the primal and dual problems iteratively with a fixed number of iterations rather than performing exhaustive exact optimization. The primal problem optimizes the decision boundary with respect to labeled data, while the dual problem adjusts weights for unlabeled data. By limiting the optimization to a practical number of iterations and using efficient quadratic programming solvers for each subproblem, the method achieves sufficient hyperplane accuracy for large datasets without requiring prohibitive computational resources that would be needed for exact TSVM optimization.
Data Source
AI summary
Disclosed is a method for training a transductive support vector machine. The support vector machine is trained based on labeled training data and unlabeled test data. A non-convex objective function which optimizes a hyperplane classifier for classifying the unlabeled test data is decomposed into a convex function and a concave function. A local approximation of the concave function at a hyperplane is calculated, and the approximation of the concave function is combined with the convex function such that the result is a convex problem. The convex problem is then solved to determine an updated hyperplane. This method is performed iteratively until the solution converges.