Query Redaction Logging for Parallel Database Processing
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing database systems are limited by hardware, data storage methods, and co-processing options, leading to inefficiencies in processing large volumes of data.
Innovation Solution
A database system utilizing a parallelized data input, storage, retrieval, and processing architecture that divides data into segments, optimizes query plans, and distributes processing across multiple computing devices and nodes, enabling efficient and parallel execution of queries and data storage.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If data is stored and processed in a traditional sequential manner, then hardware resources are simplified, but processing speed and efficiency deteriorate when handling large volumes of data
Solution Approach 1:
The patent divides data into multiple segments and distributes them across multiple computing devices. Each device processes a portion of the data in parallel, transforming a single sequential processing task into multiple concurrent processing tasks, thereby improving data processing speed without requiring a complete redesign of the hardware infrastructure
Solution Approach 2:
The patent introduces a distributed computing dimension by deploying processing nodes across multiple devices and locations. This transforms the traditional single-dimension sequential processing into multi-dimensional parallel processing, enabling simultaneous data handling across different computational nodes while maintaining manageable system complexity through modular architecture
2Productivity
If data is divided into segments and processed in parallel across multiple devices, then processing efficiency improves, but system complexity and coordination overhead increase
Solution Approach 1:
The patent introduces a query optimization module that acts as an intermediary between query submission and distributed execution. This module generates optimized query plans, manages data segment distribution, and coordinates processing across multiple devices, thereby reducing the coordination overhead and simplifying the management of distributed system complexity
Solution Approach 2:
The patent performs preliminary actions by pre-processing and optimizing query plans before distributed execution. The system prepares data segment mappings, optimizes query execution strategies, and establishes processing routes in advance, which reduces runtime coordination complexity and improves overall query processing throughput
3Productivity
If traditional data storage methods are used, then storage implementation is simple, but processing speed and efficiency deteriorate for large data volumes
Solution Approach 1:
The patent segments stored data into multiple distributed data segments across different storage nodes. This segmentation enables parallel data retrieval operations where multiple segments can be accessed simultaneously, significantly improving data retrieval speed while maintaining storage implementation simplicity through standardized segment formats
Data Source
AI summary
A database system operates by: receiving statements for execution; determining when to mark the corresponding one of the statements for redaction in accordance with a plurality of log file policies; when the corresponding one of the statements is determined to be marked, marking the corresponding one of the statements via a first marking indicating that the one of the corresponding statements is subject to a redaction, and a second marking indicating a portion of the corresponding one of the statements; generating a first redacted log file entry in accordance with the marking; and storing the first redacted log file entry in a first redacted log file.