Ordinal Regression Training via Parallel Divide-and-Conquer
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Conventional methods for processing large amounts of user data to predict user behavior are inefficient, costly, and inaccurate due to biases and variances introduced by averaging or merging individual models, and existing divide-and-conquer techniques face computational challenges and inefficiencies in training ordinal regression models.
Innovation Solution
A divide-and-conquer algorithm that divides large datasets into clusters for parallel training of logistic classification models, using robust inverse variance weighted averages to combine models without synchronization, reducing computational burden and improving accuracy.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Power
If a single computer with sufficient processing power is used to store and analyze large amounts of data, then processing capability is improved, but cost increases and computational efficiency deteriorates
Solution Approach 1:
The patent divides the large dataset into multiple smaller datasets that can be processed in parallel across multiple computers. Each computer processes a subset of the data independently, then the results are combined. This segmentation allows distributed computing to achieve the processing power of a single powerful computer while improving computational efficiency through parallelization.
2Productivity
If data is divided into blocks for training individual models and then averaged or merged, then processing efficiency is improved, but model accuracy deteriorates due to introduced variances and biases
Solution Approach 1:
The patent merges individual model results using inverse variance weighting rather than simple averaging. Models trained on different data blocks contribute to the final model based on their respective variances, with lower variance models having greater influence. This merging approach maintains processing efficiency while preserving model accuracy by avoiding the introduction of variances and biases associated with simple averaging.
3Quantity of substance
If conventional divide-and-conquer techniques are used for training ordinal regression models, then processing large datasets is enabled, but computational burden and training time increase
Solution Approach 1:
The patent segments the large dataset into smaller blocks that can be processed in parallel, reducing the computational burden on each individual computer and enabling faster training. By dividing the data, each computer can train models independently without waiting for others, significantly reducing overall training time while still processing the complete large dataset.
4Device complexity
If individual models are trained on data blocks and then averaged, then processing is simplified, but harmful factors such as biases are introduced
Solution Approach 1:
The patent changes the parameter used for combining model results from simple averaging to inverse variance weighting. This parameter change allows the system to maintain processing simplicity while eliminating biases and variances introduced by conventional averaging methods. The inverse variance weighting approach gives appropriate weight to each model based on its quality, preventing harmful factors from contaminating the final model.
Data Source
AI summary
Embodiments of the present invention provide a divide-and-conquer algorithm which divides expanded data into a cluster of machines. Each portion of data is used to train logistic classification models in parallel, and then combined at the end of the training phase to create a single ordinal model. The training scheme removes the need for synchronization between the parallel learning algorithms during the training period, making training on large datasets technically feasible without the use of supercomputers or computers with specific processing capabilities. Embodiments of the present invention also provide improved estimation and prediction performance of the model learned compared to the existing techniques for training models with large datasets.