Data Ingest Computing Node Memory Segmentation for Lock-Free Processing
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing database systems are limited by hardware, data storage methods, and operating system co-process options, leading to inefficiencies in processing speed and potential deadlocks.
Innovation Solution
A parallelized database system architecture with multiple sub-systems, including data input, storage and processing, query and response, administrative, and configuration, utilizing application-specific operating systems to optimize resource allocation and independent node operations, enabling efficient and lock-free processing.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If multiple operating systems share computer resources, then resource utilization is improved, but system stability deteriorates due to potential deadlocks
Solution Approach 1:
The patent segments the operating system into multiple independent layers: a standard operating system layer and a database-specific operating system layer. Each layer manages specific resources independently, allowing parallel operation without mutual interference. The database layer handles database-specific operations while the standard layer manages general system resources, eliminating deadlock potential while maintaining high resource utilization.
Solution Approach 2:
The patent introduces an intermediary database operating system layer that acts as a mediator between the standard operating system and database applications. This intermediary layer translates and manages resource requests, preventing direct conflicts between different operating system components and eliminating deadlock conditions while improving overall resource utilization.
2Speed
If hardware resources are increased, then processing speed is improved, but system complexity increases
Solution Approach 1:
The patent changes the software architecture parameter from a single monolithic operating system to a multi-layered operating system structure. This parameter change enables the system to achieve higher processing speeds through specialized database operations without requiring proportional increases in hardware resources, thereby avoiding increased system complexity.
3Productivity
If data storage methods are optimized, then execution speed is improved, but device complexity increases
Solution Approach 1:
The patent segments the storage management functionality into the database operating system layer, which handles database-specific storage optimization, and the standard operating system layer, which manages general file system operations. This segmentation enables optimized data storage methods that improve execution speed without increasing overall system complexity, as each layer manages its own storage responsibilities independently.
Data Source
AI summary
A computing node of a computing device of a computing device cluster of a data ingest sub-system of a database system includes main memory and processing core resources. The main memory is logically divided into a computing device memory section and a database memory section. The database memory section includes a first logical section for a first processing core resource, a second logical section for a second processing core resource, and a logical network communication section. A processing core resource includes processing module and a memory device. The processing core resources retrieves a first division of a sub-segment of a segment of a segment group of a partition of a dataset and processes it for short term storage and for long term storage. The processing core resource stores the short term version and outputs the long term version.