Parallel Database Partitioning for Payment Processing
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current database management systems face performance bottlenecks due to locking mechanisms when multiple applications access the same data, limiting their ability to handle high transaction volumes without significant hardware investments.
Innovation Solution
Implementing a parallel processing technique by assigning multiple application servers to segments of a database table, allowing data to be distributed and processed independently across these segments, reducing the need for locks and enabling scalable processing of payment orders.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If locking mechanisms are used to prevent data corruption, then data integrity is maintained, but system throughput and transaction processing speed deteriorate
Solution Approach 1:
The database table is divided into multiple segments (partition 222), with each segment assigned to a specific application server. This segmentation allows multiple application servers to process different segments simultaneously without requiring locks on the entire table, thereby maintaining data integrity within each segment while enabling parallel processing across segments to improve system throughput.
2Productivity
If multiple application servers access the same database table, then processing capacity increases, but lock contention and waiting time increase
Solution Approach 1:
By segmenting the database table and assigning each segment to a dedicated application server, the system enables multiple servers to work in parallel on different segments simultaneously. This eliminates lock contention between servers for the same data, allowing processing capacity to scale with the number of servers while minimizing waiting time as each server operates independently on its assigned segment.
3Speed
If indexes are built for all partitions, then query performance improves, but storage space consumption increases
Solution Approach 1:
The system applies different indexing strategies to different partitions based on their specific characteristics and access patterns. Indexes are built only for partitions that require them for efficient querying, while partitions with different characteristics may use alternative access methods. This local optimization approach improves query performance for partitions that need indexes while conserving storage space by avoiding unnecessary indexes on other partitions.
Data Source
AI summary
Methods and apparatus, including computer program products, for parallel processing of assigned table partitions. In general, a group of data may be received and at least a portion of the data may be caused to be assigned to an application server to process the data based at least in part on the application server and the at least a portion of the data being assigned to a same segment. A segment may represent one or more partitions of data of a database, such as a multidimensional database or a flat database. The data that is processed may be payment items that are processed as part of payment order processing by a payment engine.