Programmable Memory Partitioning for Data Processing Efficiency
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing data processing systems experience low data processing efficiency due to differences in input/output bandwidth between CPUs, FPGAs, and memory, leading to inefficient data flow.
Innovation Solution
A data processing method and apparatus that partitions free programmable memory according to a database execution plan, using programmable memory partitions to execute relational algebra logical operations, thereby avoiding data transmission to the processor and optimizing local memory processing.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If data flows between CPU, FPGA, and memory in existing systems, then processing can be performed, but data processing efficiency is reduced due to I/O bandwidth differences
Solution Approach 1:
The patent extracts the data processing function from the CPU-FPGA-memory data flow chain and relocates it directly to the memory itself. By implementing programmable memory that can execute relational algebra operations locally, the system eliminates the need for continuous data transmission between CPU, FPGA, and memory, thereby resolving the efficiency loss caused by I/O bandwidth differences.
Solution Approach 2:
The memory is designed to perform data processing operations autonomously without requiring constant intervention from the CPU. The programmable memory can independently execute relational algebra logical operations on stored data, making the system self-sufficient and eliminating the bottleneck of CPU-memory data transfer for processing tasks.
2Speed
If CPU processes data through FPGA and memory, then computational tasks are completed, but processing speed is limited by I/O bandwidth
Solution Approach 1:
The patent removes the data processing function from the CPU-FPGA-memory chain and embeds it directly within memory. This extraction allows memory to process data at its own intrinsic speed without being constrained by the slower I/O bandwidth between CPU and memory, thereby achieving higher processing speed while simplifying the overall data flow management.
Solution Approach 2:
The patent segments the data processing task into operations that can be executed independently within memory partitions. By dividing the processing workload and enabling parallel execution in different memory partitions, the system achieves higher processing speed without increasing the complexity of data flow management between components.
3Productivity
If memory is used for data storage and processing, then processing efficiency improves, but memory resources need to be partitioned and managed
Solution Approach 1:
The patent applies segmentation by dividing the programmable memory into multiple independent partitions, each capable of executing processing operations autonomously. This segmentation enables parallel processing and improves productivity while the partition management is handled through a systematic approach that does not significantly increase overall system complexity.
Solution Approach 2:
The memory partitions are designed with universal functionality to execute various relational algebra operations. Each partition can perform multiple types of processing tasks, making the memory system highly versatile and improving processing efficiency without requiring separate specialized hardware for different operations, thus avoiding excessive complexity.
Data Source
AI summary
A data processing method and apparatus are provided. The data processing method includes determining, according to a database execution plan, a partition quantity corresponding to a currently free programmable memory, partitioning the currently free programmable memory according to the partition quantity, to obtain a programmable memory partition corresponding to the partition quantity, and executing, using the programmable memory partition, a relational algebra logical operation corresponding to the database execution plan. Embodiments of the present disclosure can be used to improve data processing efficiency.