Pruning Selector Logic for Faster Low-Power Data Analytics
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing processing systems face limitations in processing speed and power consumption due to bottlenecks in data communication and inefficient data processing methods, particularly in CPUs and GPUs, which hinder the scalability and efficiency of data analytics applications.
Innovation Solution
A hardware-based programmable data analytics processor is introduced, comprising modules such as a selector, filter and project module, join and group module, and a communications fabric, to enhance data processing between a data storage unit and hosts, with pruning logic for data pre-processing and a data filter system to reduce data subsets before query processing.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Speed
If data is transferred from data storage unit to hosts through traditional CPU/GPU processing systems, then data processing can be performed, but processing speed is limited and power consumption increases due to communication bottlenecks
Solution Approach 1:
A data analytics processor is introduced as an intermediary component between the data storage unit and hosts. This processor includes specialized modules (selector, filter and project, join and group) that handle data preprocessing tasks locally at the storage side, eliminating the need to transfer all raw data to hosts for processing. The communications fabric enables efficient data exchange between the intermediary processor and storage unit, reducing overall system power consumption while accelerating data analytics operations.
2Productivity
If all data from data storage unit is transferred to hosts for processing, then complete data analysis can be performed, but network congestion increases and processing time extends
Solution Approach 1:
The data analytics processor performs preliminary data processing actions directly at the storage unit before data is transferred to hosts. The selector module pre-filters data based on selection criteria, the filter and project module pre-processes and transforms data, and the join and group module pre-aggregates related data. This preliminary processing significantly reduces the volume of data that needs to be transferred over the network to hosts, thereby reducing network congestion and accelerating overall analytics productivity.
3Productivity
If traditional CPU/GPU systems are used for data processing, then general-purpose computing can be performed, but processing efficiency decreases due to architectural bottlenecks in data communication
Solution Approach 1:
The data analytics processor is segmented into specialized functional modules: a selector module for data selection, a filter and project module for data transformation, and a join and group module for data aggregation. Each module is optimized for its specific function and can operate independently or in parallel. This segmentation enables efficient data processing pipelines while maintaining manageable system complexity through modular design, overcoming the bottlenecks of traditional monolithic CPU/GPU architectures.
Data Source
AI summary
A data pre-processing architecture may include an interface and a pruning logic configured to receive, via the interface, at least one filter value from a query processor; use the at least one filter value to scan rows or columns of a data table stored in a memory; generate a selection indicator identifying a set of rows or columns of the data table where the at least one filter value resides; and provide to the query processor a filtered output based on the selection indicator.