Intelligent task scheduling during query processing in a database system

Abstract

For a database or data store having multiple shards, shard-level data is loaded into the L2 / L3 cache or RAM (Random Access Memory) of the query processing nodes as per the best fit approach after filtering maximum possible rows or tuples from the shard by performing a predicate push down when data is being read to load into the query processing nodes. Further especially for analytical queries, the query plan is broken down into a tree-like structure depicting the fine-grained tasks which can be performed in parallel and others which are sequential. Now based on the compute capacity (or processing cores), CPU (Central Processing Unit) cycles, I / O, memory bandwidth, etc. allocated for the query processing thread serving the analytical query, the fine-grained tasks are scheduled such that the paths or sub-tree structures or task order sequences are followed but there is maximum overlap between the CPU- intensive and I / O intensive tasks so that there are minimum or preferably no tasks which are stuck on I / O resulting in better utilisation of system resources.

Description

[0001] Intelligent Task Scheduling During Query Processing in a Database System

[0002] For a database or data store having multiple shards, shard-level data is loaded into the L2 / L3 cache or RAM (Random Access Memory) of the query processing nodes as per the best fit approach after filtering maximum possible rows or tuples from the shard by performing a predicate push down when data is being read to load into the query processing nodes. Further especially for analytical queries, the query plan is broken down into a tree-like structure depicting the fine-grained tasks which can be performed in parallel and others which are sequential. Now based on the compute capacity (or processing cores), CPU (Central Processing Unit) cycles, I / O, memory bandwidth, etc. allocated for the query processing thread serving the analytical query, the fine-grained tasks are scheduled such that the paths or sub-tree structures or task order sequences are followed but there is maximum overlap between the CPUintensive and I / O intensive tasks so that there are minimum or preferably no tasks which are stuck on I / O resulting in better utilisation of system resources.

Claims

Following are the claims for this invention :-1. For a database or data store having multiple shards, shard-level data is loaded into the L2 / L3 cache or RAM (Random Access Memory) of the query processing nodes as per the best fit approach after filtering maximum possible rows or tuples from the shard by performing a predicate push down when data is being read to load into the query processing nodes.

2. Further especially for analytical queries, the query plan is broken down into a tree-like structure depicting the fine-grained tasks which can be performed in parallel and others which are sequential.

3. Now based on the compute capacity (or processing cores), CPU (Central Processing Unit) cycles, I / O, memory bandwidth, etc. allocated for the query processing thread serving the analytical query, the fine-grained tasks are scheduled such that the paths or sub-tree structures or task order sequences are followed but there is maximum overlap between the CPU-intensive and I / O intensive tasks so that there are minimum or preferably no tasks which are stuck on I / O resulting in better utilisation of system resources.