Scalable Decision Optimization Algorithm for Large-Scale Consumer Data
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current optimization techniques for large consumer decisions with global constraints are inefficient due to limitations in existing algorithms and memory constraints, leading to suboptimal solutions and dependency on expensive or inexperienced solvers.
Innovation Solution
An iterative algorithm that partitions the problem into smaller sub-problems, allowing the use of off-the-shelf linear programming solvers and reducing the problem size, enabling the solution of larger problems while ensuring a global solution is found.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Quantity of substance
If sampling and segmentation techniques are used to solve large consumer decisions, then the problem can be solved with existing memory constraints, but the solution quality deteriorates due to suboptimal results
Solution Approach 1:
The patent divides the large consumer decision problem into smaller sub-problems by segmenting the account data into manageable chunks that can be processed sequentially. This allows the algorithm to handle large datasets (millions of accounts) by processing them in smaller batches, avoiding memory constraints while maintaining solution quality through systematic processing of each segment.
2Device complexity
If custom solvers are written to reduce problem complexity, then specific decision types can be solved, but the adaptability deteriorates due to limited applicability
Solution Approach 1:
The patent creates a universal solver framework that can handle multiple types of consumer decisions (credit card decisions, loan decisions, insurance decisions) through a single algorithmic approach. The system uses generic linear programming formulations that can be applied to various decision contexts without requiring custom code for each decision type, thus maintaining both simplicity and adaptability.
3Quantity of substance
If 64 bit computers are used to handle large problems, then the problem size capacity increases, but the cost increases and solver experience requirements increase
Solution Approach 1:
The patent processes the complete dataset in sequential passes rather than loading all data into memory simultaneously. By using iterative algorithms that process data in chunks multiple times, the system achieves complete data utilization with limited memory resources, avoiding the need for expensive 64-bit infrastructure while still handling millions of accounts.
4Reliability
If the entire problem is loaded into memory for solving, then the complete solution can be found, but the memory requirements exceed 32 bit address space limitations
Solution Approach 1:
The patent segments the account data into smaller batches that fit within available memory constraints. The algorithm processes each batch sequentially, maintaining solution completeness by systematically working through all accounts in multiple passes, thereby achieving reliable solutions without requiring memory beyond 32-bit address space limitations.
Data Source
AI summary
An iterative approach to solving the optimization problem is provided. The invention provides an iteration of four basic operations; determining the segments, balancing the segments, expanding a segment, and solving the segment optimization. The method and apparatus can use any off-the-shelf linear programming (LP) solver, such as Dash Optimization Xpress, by Dash Optimization, during the solve operation. The size of the problem fed into the LP solver remains bounded and relatively small compared to the entire problem size. Thus, the algorithm can solve problems of several orders of magnitude larger. In one embodiment of the invention, the sampling and segmentation techniques are removed to where the problem is solved at the account-level. In the above cases, the solution is produced in a more cost-effective manner and the best possible return is achieved because the doubt of achieving a true global solution is removed.