Single Buffer Memory for Database Backup Bottleneck Reduction
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Modern computer systems face challenges in data movement and management due to the increasing amount of data being stored, which often results in slower data transfer processes despite improved processing speeds.
Innovation Solution
The implementation of a single buffer memory system for data movement, where data is read into a buffer and processed before being sent to the final destination, optimizing processing speed and reducing bottlenecks caused by hardware disparities.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If multiple buffers are used for data movement, then continuous data retrieval is enabled, but hardware disparity between buffers and processors causes suboptimal processing speeds
Solution Approach 1:
The patent merges multiple separate buffers into a single unified buffer memory structure. This consolidation eliminates the hardware disparity between multiple buffers and processors, allowing all data operations to occur on a single buffer that can be optimally accessed by the processing hardware, thereby resolving the contradiction between maintaining continuous data flow and achieving optimal processing speed
2Duration of action of moving object
If data is stored in multiple separate buffers, then data can be retrieved continuously, but the system complexity increases due to managing multiple buffers at different locations
Solution Approach 1:
The patent combines multiple separate buffer structures into a single unified buffer memory. This merger simplifies the system by eliminating the need to manage multiple buffers at different hardware locations, reducing the complexity of buffer allocation, access coordination, and hardware interface management while still enabling continuous data retrieval operations
3Productivity
If processing functions are executed on processors different from the buffer location, then data can be processed in parallel, but hardware access speeds are reduced due to NUMA architecture
Solution Approach 1:
The patent applies local quality by ensuring that the single unified buffer memory is located on the same NUMA node as the processing functions. This local co-location optimizes hardware access speeds by eliminating remote memory access penalties inherent in NUMA architectures, while still allowing parallel processing through multiple threads or functions operating on the locally-optimized buffer
Data Source
AI summary
Embodiments of the present disclosure include techniques for backing up data. In one embodiment, a single buffer memory is allocated. Data pages are read from a datastore and loaded into a first portion of the single buffer memory. When the first portion of the single buffer memory is full, data is from the datastore is loaded into a second portion of the single buffer memory while a plurality of jobs process data pages in parallel.