Distributed Database Plan Execution for Parallel Row Numbering
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
In distributed databases, operations that require generating unique and progressively increasing data row numbers, such as COUNT and SUM operators, hinder parallel execution due to their single-point nature, leading to reduced performance.
Innovation Solution
A method where execution threads request and receive unique row number start values from a scheduling node, allowing them to generate data row numbers locally and execute operations in parallel, thereby overcoming the single-point operation constraint.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If data row numbers are generated sequentially in a single-point operation to ensure uniqueness and progressive increase, then the correctness of row number generation is maintained, but parallel execution performance is severely affected
Solution Approach 1:
The patent segments the sequential row number generation process into multiple parallel sub-processes. Each execution thread receives a unique start value and generates row numbers independently within its own range, eliminating the need for single-point sequential operations while maintaining row number uniqueness and progressive increase across all threads
Solution Approach 2:
The scheduling node performs preliminary action by allocating unique start values to each execution thread before parallel execution begins. This pre-allocation ensures that each thread can independently generate valid row numbers without real-time coordination during execution, enabling parallel processing while maintaining correctness
2Productivity
If execution threads operate independently in parallel to improve processing speed, then parallel execution performance is enhanced, but coordination overhead for maintaining row number sequence increases
Solution Approach 1:
The patent extracts the row number generation coordination function from the execution threads and centralizes it in the scheduling node. The scheduling node allocates start values and collects results, while execution threads focus solely on independent data processing. This separation reduces coordination overhead and simplifies the execution thread logic
Solution Approach 2:
The scheduling node acts as an intermediary between execution threads, managing the allocation of start values and collection of results. This intermediary coordinate mechanism enables multiple threads to work independently in parallel while maintaining overall system coherence and row number sequence integrity
Data Source
AI summary
A plan execution method, apparatus, and system for a distributed database are provided. The plan execution method is performed by a first execution thread on a plan execution node. Feedback data returned by a child logical subplan of a current logical subplan is received in parallel with at least one second execution thread scheduled to execute the current logical subplan. Whether the current logical subplan includes an execution operator related to a row number generation operation is determined. In response to that the current logical subplan includes the execution operator related to a row number generation operation, the received data are cached, and a row number start value obtaining request is sent to the scheduling node. Data row numbers are generated for the cached data based on a received row number start value.