Transaction Segment Records for Parallel Log Processing
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
In computing systems that perform transactions in parallel, there is a challenge in processing transaction logs efficiently due to overlapping and interleaved task completions, which can lead to dependencies and reordering issues, making it difficult to ensure that transactions are completed correctly and in the right order.
Innovation Solution
The creation of non-overlapping transaction segment records with unique transaction identifiers allows for parallel processing by dispatching these records to different worker threads, while also accounting for dependencies by identifying and awaiting the completion of external dependee entities before processing.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If transactions are processed in parallel with interleaved task completions, then processing speed is improved, but transaction ordering and dependency management become problematic
Solution Approach 1:
The patent divides the log into non-overlapping transaction segments, each assigned to a different worker thread. This segmentation allows parallel processing while ensuring that each segment maintains its own transaction ordering, thus resolving the contradiction between processing speed and transaction ordering reliability.
2Productivity
If transaction segments are processed in parallel by different worker threads, then processing efficiency is improved, but dependency management between segments becomes complex
Solution Approach 1:
The patent performs preliminary actions by identifying and recording the latest dependee entity for each transaction segment before parallel processing begins. This advance preparation allows worker threads to efficiently check dependencies without complex runtime coordination, resolving the contradiction between processing efficiency and dependency management complexity.
3Productivity
If non-overlapping transaction identifier sets are used, then parallel processing is enabled, but handling transactions with external dependencies requires additional mechanisms
Solution Approach 1:
The patent introduces an intermediary mechanism (the latest dependee entity identifier stored in each transaction segment record) that mediates between parallel transaction segments. This intermediary allows each segment to independently track its dependencies without direct communication between worker threads, enabling parallel processing while managing dependency tracking complexity.
Data Source
AI summary
The creation of transaction segment records that are each associated with transaction segments within a log that includes multiple tasks, each of at least some associated with a corresponding transaction. The transaction segment records each include a transaction identifier set that define which tasks are included within the transaction segment. The transaction identifier sets are non-overlapping such that the transaction segments may be processed substantially in parallel by dispatching the transaction segment records to different worker threads. The identity of a latest external dependee entity, if any, is included within the transaction segment record. Upon being assigned a transaction segment for processing, the module assigned the transaction segment awaits completion of processing of the latest external dependee entity prior to processing the transaction segment. Accordingly, transaction segments may be processed substantially in parallel, while in those cases in which there is a dependency, the dependency is honored.