Query Planner with Parallel Sub-Query Processing Paths
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing processing systems face limitations in processing speed and power consumption due to bottlenecks in data communication and inefficient data processing methods, particularly in data analytics applications.
Innovation Solution
A hardware-based programmable data analytics processor is positioned between a data storage unit and hosts, incorporating a selector module, filter and project module, join and group module, and a communications fabric to optimize data processing and reduce data transfer.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Speed
If data is transferred between storage unit and hosts through traditional processing systems, then data communication is established, but processing speed is limited and power consumption increases due to bottlenecks
Solution Approach 1:
The processing system is divided into distinct functional modules (selector module, filter and project module, join and group module) that can operate independently and in parallel. This segmentation allows data processing to be distributed across multiple specialized units, reducing bottlenecks and improving overall processing speed while reducing power consumption through efficient resource utilization.
Solution Approach 2:
A communications fabric is introduced as an intermediary component that efficiently transfers data between the selector module, filter and project module, and join and group module. This dedicated communication infrastructure reduces data transfer bottlenecks and power consumption compared to traditional systems that rely on general-purpose communication channels.
2Productivity
If traditional data processing methods are used, then data processing is performed, but processing speed is limited due to bottlenecks in data communication
Solution Approach 1:
The selector module performs preliminary data selection and filtering operations before data is passed to subsequent processing stages. By pre-processing data and eliminating unnecessary information early in the pipeline, the system reduces the amount of data that needs to be communicated and processed in later stages, thereby reducing processing time and improving overall productivity.
Solution Approach 2:
The processing modules are designed to operate continuously and in parallel, with the communications fabric enabling seamless data flow between stages. This continuous operation eliminates idle time and bottlenecks, maintaining high productivity throughout the data processing pipeline.
3Productivity
If more data is transferred between modules, then more processing can be done, but network congestion increases and power consumption rises
Solution Approach 1:
The filter and project module extracts only the necessary data elements from the input data sets, removing unnecessary information before it is passed to subsequent processing stages. This extraction principle reduces the volume of data that needs to be transferred across the communications fabric, thereby reducing energy loss in data transfer while maintaining productive data analytics capabilities.
Data Source
AI summary
An accelerated database management system including at least one processor including circuitry and a memory. The memory includes instructions that when executed by the circuitry cause the at least one processor to: receive an initial database query: generate a main query based on the initial database query; analyze the main query, and based on the analysis of the main query, generate at least a first sub-query and a second sub-query, wherein the second sub-query differs from the first sub-query; process the first sub-query along a first processing path to provide a first input to an execution module; process the second sub-query along a second processing path, different from the first processing path, to provide a second input to the execution module; and based on the first input and the second input received by the execution module, generate a main query result.