Parallel Data Object Bundling via Pipeline Segmentation
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Real-time processing systems using relational databases face performance bottlenecks when bundling transactions due to simultaneous access to a shared master table, limiting scalability and efficiency in parallel processing.
Innovation Solution
Divide the bundling process into two steps: first, generate pipelines of data objects by pre-selecting and grouping them according to predefined criteria using one group of processors, and second, create bundles from these pipelines using another group of processors, decoupling access to the master table and preventing mutual blocking.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If multiple processors access the shared master table simultaneously to create bundles, then the system can process transactions in parallel, but the master table becomes a bottleneck that limits performance and scalability
Solution Approach 1:
The patent segments the bundle creation process into two independent phases: a pre-selection phase that groups transactions into pipelines without accessing the master table, and a bundle creation phase that processes each pipeline independently. This segmentation eliminates the bottleneck by removing simultaneous master table accesses while maintaining parallel processing capability.
Solution Approach 2:
The patent performs preliminary grouping of transactions into pipelines based on characteristics (currency, transaction code, amount ranges) before the actual bundle creation. This pre-selection phase prepares the data structure in advance, allowing subsequent bundle creation to proceed without repeated master table accesses, thus improving performance.
2Adaptability or versatility
If the system uses a shared master table for dynamic control of bundling criteria, then bundling can be flexibly controlled, but simultaneous access by multiple processors causes blocking and reduces efficiency
Solution Approach 1:
The patent divides the processor cohort into multiple cohorts, each responsible for specific pipelines. Each processor cohort operates independently on its assigned pipelines, eliminating contention for the master table while preserving dynamic bundling control through the pre-established pipeline structures.
Solution Approach 2:
The patent creates multiple independent copies of the bundling logic distributed across different processor cohorts. Each cohort has its own instance of the bundle creation process operating on its assigned pipelines, eliminating the single-point bottleneck of the shared master table while maintaining consistent bundling rules.
3Manufacturing precision
If the system bundles transactions with strict criteria (exact number or maximum sum), then transaction processing accuracy is improved, but the complexity of managing bundle constraints increases
Solution Approach 1:
The patent pre-calculates and establishes pipeline structures during the pre-selection phase, organizing transactions by characteristics and potential bundle compositions. This preliminary organization simplifies the subsequent bundle creation phase, as processors only need to follow pre-determined pipeline structures rather than managing complex constraints in real-time.
Solution Approach 2:
The patent enables pipelines to self-organize into bundles based on pre-established criteria and characteristics. The system automatically manages bundle composition by following the pre-defined pipeline structures, reducing the need for complex external constraint management while maintaining precise bundle composition accuracy.
Data Source
AI summary
Computer-implemented methods, computer systems and computer program products are provided for parallel processing a plurality of data objects with a plurality of processors. As disclosed herein, the data objects to be assembled for further processing may be in bundles, the bundles obeying first predefined criteria, which is dynamically controlled by using a bundle specific master table. The methods and systems may generate pipelines of data objects by pre-selecting and grouping the data objects according to second predefined criteria by a first group of the plurality of processors, and create the bundles from each pipeline of the pre-selected data objects by a second group of the plurality of processors.