Sparse Estimation via Data Partitioning and L0 Annealing
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing methods for sparse estimation in multiple regression, such as grid search and L1 regularization, face challenges in achieving accurate and high-speed results due to computational complexity and potential data bias.
Innovation Solution
The optimization apparatus partitions learning data into subsets and performs L0 regularization with annealing to extract zero elements, calculating error ratios between multiple regressions on test and modified data to identify candidate models, thereby achieving high-accuracy sparse estimation without data bias.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If grid search is used for sparse estimation, then exhaustive search for all 0 components of β is performed, but the computational time becomes excessively long
Solution Approach 1:
The patent divides the learning data into multiple subsets and performs regularization processing on each subset separately. This segmentation allows the exhaustive search to be distributed across multiple smaller problems rather than one large problem, reducing the computational time while maintaining the accuracy of sparse estimation.
Solution Approach 2:
The patent performs preliminary regularization processing to extract elements equal to zero before conducting multiple regression analysis. By pre-identifying the sparse components through regularization on data subsets, the subsequent regression analysis only needs to consider non-zero elements, significantly reducing computational time.
2Productivity
If L1 regularization is used to relax the normalization term, then computational complexity is reduced, but the sparsity constraint is weakened
Solution Approach 1:
The patent changes the regularization parameter from L1 norm to L0 norm (counting the number of non-zero elements). This parameter change maintains the strict sparsity constraint while still allowing efficient computation through the combination of data subset partitioning and iterative optimization.
Solution Approach 2:
The patent replaces the direct L0 regularization optimization (which is computationally difficult) with an alternative approach using data subset partitioning and iterative refinement. This substitution maintains the L0 sparsity constraint while avoiding the NP-hard computational complexity through a multi-stage process.
3Measurement precision
If regularization processing is performed on the entire learning data, then accurate sparse estimation is achieved, but data bias occurs
Solution Approach 1:
The patent segments the learning data into multiple subsets and performs regularization processing on each subset independently. This segmentation prevents any single subset from dominating the regularization result, thereby reducing data bias while maintaining accurate sparse estimation through aggregation of subset results.
Data Source
AI summary
A method includes: partitioning learning data containing objective variables and explanatory variables into a plurality of subsets of data; executing regularization processing on first data in each of the partitioned subsets, and extracting a first element equal to zero; extracting, as a candidate, each model where an error ratio between first multiple regression and second multiple regression is equal to or more than a predetermined value, the first multiple regression being a result of multiple regression on second data which is test data in each of the partitioned subsets and is for use to calculate the error ratio of the learning data, the second multiple regression being a result of multiple regression on third data obtained by excluding the first element from the second data; and outputting a model where zero is substituted for an element that takes zero a predetermined or larger number of times in the candidate.