Parallel Query Execution Nodes for Segmented Database Processing
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current database systems face limitations in processing speed due to hardware constraints, data storage methods, and restricted co-processing options, which hinder efficient execution of queries and data processing.
Innovation Solution
A large-scale database system architecture that employs parallelized resources, including a parallelized data input sub-system, data store and process sub-system, and query and response sub-system, which divide data into segments, restructure it based on key fields, and optimize query plans for efficient storage and processing across multiple computing devices.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Speed
If traditional hardware and storage methods are used, then device complexity is limited, but processing speed and query execution efficiency deteriorate
Solution Approach 1:
The patent segments data into multiple partitions distributed across different storage devices, and divides query processing into parallel tasks executed by multiple processing cores. This segmentation enables concurrent processing of different data segments, significantly improving processing speed without requiring a single complex hardware system
Solution Approach 2:
The patent introduces parallelism as a new dimension for processing by utilizing multiple processing cores simultaneously. Instead of improving speed through a single powerful processor, the system distributes queries across multiple cores working in parallel, effectively adding a temporal dimension to processing capacity
2Speed
If data is stored in traditional formats, then storage simplicity is maintained, but query execution speed deteriorates
Solution Approach 1:
The patent divides large datasets into multiple partitions that can be stored across different storage devices. Each partition can be independently queried and processed, enabling parallel query execution and improving query execution speed while maintaining manageable storage structures
Solution Approach 2:
The patent restructures data locally within each partition based on key fields to optimize query performance. By organizing data locally with appropriate indexing and sorting within partitions, the system achieves fast query execution without requiring complex global storage structures
3Productivity
If co-processing options are restricted, then system simplicity is maintained, but processing efficiency deteriorates
Solution Approach 1:
The patent creates a universal query processing architecture where multiple processing cores can handle various types of queries simultaneously. The system uses standardized query plans and execution mechanisms that work across different query types and data partitions, improving overall processing efficiency without requiring specialized co-processing units for each query type
Data Source
AI summary
A node includes a plurality of processing core resources. Each processing core resource of the plurality of processing core resources includes a corresponding processing module, a corresponding memory interface module, a corresponding memory device, and a corresponding cache memory. The plurality of processing core resources of the node is operable to collectively perform corresponding operations of the node. Each processing core resource of the plurality of processing core resources of the node is operable to perform operations independently from other ones of the plurality of processing core resources of the node.