Inverse Hessian Matrix Recursive Generation for Online Learning
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing methods for updating model parameters in prediction models, especially for non-stationary pattern sequences, are computationally expensive and resource-intensive, requiring repeated training processes for each new pattern in historical data.
Innovation Solution
A computer-implemented method that recursively generates an inverse Hessian matrix using modified feature vectors to update model parameters, reducing computational costs by minimizing the absolute value of feature vector elements and incorporating a regularization term to prevent overfitting.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If the prediction model is trained using available historical data every time a new pattern is observed, then the accuracy of the prediction model is improved, but the computational cost becomes prohibitively high
Solution Approach 1:
The patent pre-calculates and stores the inverse Hessian matrix during the initial training phase. This preliminary computation allows the model to perform rapid parameter updates using only the formula θ = H^-1y without requiring full retraining, thus maintaining accuracy while drastically reducing computational cost during online learning.
Solution Approach 2:
The patent separates the training process into two distinct phases: an initial offline training phase where the inverse Hessian matrix is computed and stored, and an online learning phase where only simple matrix-vector multiplication is performed. This segmentation allows heavy computational work to be done once beforehand, enabling efficient real-time updates.
2Measurement precision
If the prediction model is trained using available historical data every time a new pattern is observed, then the accuracy of the prediction model is improved, but the time required for training increases significantly
Solution Approach 1:
The inverse Hessian matrix is pre-computed and stored during initial training, eliminating the need for time-consuming iterative optimization algorithms during online learning. This preliminary action reduces training time from potentially hours or days to milliseconds while preserving model accuracy.
Solution Approach 2:
The patent uses the stored inverse Hessian matrix as a computational shortcut that replicates the effect of full training. Instead of copying and processing entire historical datasets repeatedly, the system copies and applies the pre-computed inverse Hessian matrix to achieve the same accuracy improvement with minimal time investment.
3Productivity
If incremental learning is used to reduce computational cost, then the training efficiency is improved, but the accuracy of the model updates deteriorates
Solution Approach 1:
The patent changes the parameter representation by working with the inverse Hessian matrix rather than the raw Hessian matrix or full dataset. This parameter transformation enables accurate model updates through simple linear algebra operations (θ = H^-1y), maintaining theoretical optimality while achieving computational efficiency.
Solution Approach 2:
The patent replaces the mechanical process of repeatedly processing full historical datasets with a mathematical substitution using the inverse Hessian matrix. This substitution transforms a computationally intensive iterative optimization problem into a direct closed-form solution, preserving accuracy while dramatically improving efficiency.
Data Source
AI summary
Online learning of model parameters is performed by obtaining a first target value in a target sequence and a feature vector corresponding to the first target value. The feature vector includes a plurality of elements. The feature vector can be modified to obtain a modified feature vector by reducing an absolute value of at least one element of the feature vector. An inverse Hessian matrix can be generated recursively from a previous inverse Hessian matrix using at least the feature vector and the modified feature vector. Parameters of a model can be updated using the inverse Hessian matrix.