Database Server Code Accelerator for Parallel Data Partitioning
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current computing systems face challenges in efficiently processing large amounts of data in parallel, particularly in distributing and executing programs across multiple nodes in a computing system, which can lead to inefficiencies and potential service interruptions when handling real-time data streams.
Innovation Solution
A computer-program product and method that enables a computing system to receive a program, determine if it contains thread and data program components, and generate computer-generated instructions in a different language for parallel execution, depending on specified key information for data partitioning and grouping, allowing for efficient parallel processing of data across multiple sets of partitioned data.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If data is processed sequentially on a single node, then system complexity is reduced, but processing speed and productivity deteriorate
Solution Approach 1:
The patent divides the computing system into multiple nodes, each capable of independently processing partitions of data. The program is automatically partitioned and distributed across these nodes, allowing parallel processing while maintaining manageable complexity at each node through clear separation of responsibilities.
Solution Approach 2:
The patent transitions from single-node sequential processing to multi-node parallel processing by adding the spatial dimension of distribution. Data and computation are distributed across multiple dimensions (nodes), enabling simultaneous execution of multiple processing tasks that would otherwise be sequential.
2Productivity
If parallel processing is implemented across multiple nodes, then productivity improves, but system reliability deteriorates due to potential service interruptions
Solution Approach 1:
The patent performs preliminary actions by pre-partitioning data and pre-distributing processing tasks across multiple nodes before actual processing begins. This preparation ensures that if one node fails, other nodes already have the necessary data and instructions to continue processing, maintaining service continuity.
Solution Approach 2:
The patent implements beforehand cushioning through redundant node deployment and distributed processing. If one node experiences service interruption, the system has预先 prepared other nodes that can absorb the failure and continue processing, providing a buffer against reliability issues.
3Ease of operation
If automatic program partitioning and code generation is implemented, then ease of operation improves, but device complexity worsens
Solution Approach 1:
The patent implements self-service by enabling the system to automatically partition programs, generate code, and distribute tasks without requiring manual intervention. The system serves itself by autonomously managing the complexity of parallel processing setup, making it easy for users to deploy complex distributed applications.
Solution Approach 2:
The patent introduces an intermediary code generation component that acts as a mediator between the high-level program specification and the low-level distributed execution details. This intermediary automatically translates user-friendly program definitions into optimized parallel execution code, shielding users from the underlying system complexity.
4Productivity
If data is partitioned and distributed across multiple nodes, then processing efficiency improves, but loss of information worsens due to potential data distribution errors
Solution Approach 1:
The patent implements feedback mechanisms that monitor data distribution and processing across nodes. The system continuously checks for data integrity, verifies correct partitioning, and provides feedback to correct any distribution errors, ensuring that no information is lost during the partitioning and distributed processing operations.
Data Source
AI summary
A computing system receives a program. The program is in a first computer language and specifies computer operations on stored data. The computing system is configured to partition the stored data into sets of partitioned data for performing parallel execution on each of the sets of partitioned data. The computing system determines whether the program comprises a thread program component. The computing system, responsive to determining that the program comprises a thread program component, generates computer-generated computer instructions. The computer-generated computer instructions are in a second computer language. The computer-generated computer instructions are dependent on whether the thread program component specifies information for partitioning and grouping the stored data; whether the program comprises a data program component; or whether the data program component specifies information for partitioning and grouping the output data of the thread program component. The computing system executes the program according to the computer-generated computer instructions.