SVM Training via GPU Parallel Processing and Dynamic Storage Formats
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current SVM training algorithms, such as LibSVM and ThunderSVM, face significant time-consuming issues due to the need for numerous iteration steps and matrix vector multiplication, especially when dealing with large datasets, which limits their application in big data scenarios, and their fixed storage formats do not efficiently utilize GPU resources.
Innovation Solution
An automatic tuning framework is employed to predict the best storage format for datasets online, allowing for format conversion and the execution of the SMO algorithm, which optimizes matrix multiplication by using a combination of dense and sparse formats, and employs GPU parallel processing to improve memory access efficiency.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Ease of operation
If LibSVM or ThunderSVM is used for SVM training, then the SVM algorithm can be implemented with standard interfaces, but the training process becomes extremely time-consuming due to numerous iteration steps and matrix vector multiplication
Solution Approach 1:
The patent replaces the traditional CPU-based sequential computation with GPU parallel computation. By using CUDA technology, the matrix multiplication operations are transferred to the GPU, which can perform these operations in parallel across multiple cores, thereby dramatically reducing the training time while maintaining the SVM algorithm's functionality
Solution Approach 2:
The patent divides the SVM training process into multiple independent tasks that can be executed in parallel on the GPU. The matrix multiplication operations are segmented into smaller sub-tasks that can be distributed across different GPU threads, allowing simultaneous computation of multiple elements and reducing overall computation time
2Device complexity
If a fixed storage format is used for datasets, then the implementation is simple, but GPU resources are not fully utilized and efficiency is reduced
Solution Approach 1:
The patent introduces a dynamic storage format selection mechanism that automatically determines the best format (dense or sparse) based on the dataset characteristics. The system evaluates the data distribution and switches between storage formats accordingly, allowing the system to adapt to different data patterns and maximize GPU resource utilization without manual intervention
Solution Approach 2:
The patent changes the storage format parameter dynamically based on dataset properties. By detecting whether the data is dense or sparse and adjusting the storage representation accordingly, the system optimizes memory access patterns and GPU compute resource utilization, thereby improving training efficiency while maintaining relatively simple implementation
3Productivity
If all kernel matrices are calculated in advance and stored in global memory, then parallel calculation capability is fully utilized, but storage space requirements increase rapidly with data size
Solution Approach 1:
Instead of calculating and storing all kernel matrices in advance (excessive action), the patent calculates only the necessary kernel matrices on-demand during the SMO algorithm iterations (partial action). This approach reduces storage requirements while still enabling efficient parallel computation by pre-calculating and caching only the essential kernel values needed for each iteration
4Speed
If dense data format is used for storage, then memory access is efficient, but redundant operations occur with sparse datasets
Solution Approach 1:
The patent dynamically changes the storage format parameter based on data sparsity characteristics. For dense datasets, it uses dense format for fast memory access; for sparse datasets, it switches to sparse format to avoid redundant operations. This adaptive parameter adjustment optimizes both memory access efficiency and computational effectiveness without requiring manual format selection
Data Source
AI summary
The embodiments of the present disclosure provide a method and apparatus for training an SVM model, a device and a computer-readable storage medium. The method comprises steps of: acquiring a dataset; loading the dataset; predicting the best storage format of the dataset online through an automatic tuning framework, and performing format conversion; and, executing an SMO algorithm to train an SVM model. In this way, the best data storage format and algorithm can be automatically selected according to the input dataset; in a global loop level, a large number of more efficient matrix multiplication operations are performed on the basis of the best data storage format to calculate a kernel matrix; and, in a local SMO solver level, finer memory optimization is performed by the register communication technology and the combined memory access method of the GPU, and the calculation resources of the hardware platform are fully utilized.