Distributed Data Cube Query Processing via Virtual Process Spaces
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Distributing and querying large cubes of data across a multi-node grid of computing devices is inefficient due to the impracticality of storing multiple copies, requiring memory mapping to load the cube only once for multiple users.
Innovation Solution
A method is provided where a cube of data is distributed across multiple computing devices, with each device storing a portion, and a perturbation rule is applied to process queries efficiently by creating a process space and compiling rules for each user, allowing queries to be processed while applying perturbation rules to the data stored on each device.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Speed
If multiple copies of the cube of data are stored for multiple users, then access speed is improved, but disk space and memory requirements increase significantly
Solution Approach 1:
The patent creates a virtual copy mechanism where each user gets a process space that appears as a separate copy but actually shares the underlying physical data storage. The memory mapping technology creates virtual addresses that map to the same physical data, providing user-specific access views without duplicating the actual data bytes on disk.
Solution Approach 2:
The patent introduces a virtualization dimension by creating process spaces and memory mappings that exist in a virtual address space layer between the user and physical storage. This additional dimensional layer allows multiple users to access data simultaneously with user-specific transformations applied, without requiring physical duplication of data in multiple locations.
2Loss of time
If the cube of data is loaded into memory for all users, then access time is reduced, but memory requirements become impractical
Solution Approach 1:
The patent segments the data cube into portions distributed across multiple computing devices in a grid architecture. Each device stores a different portion of the cube, and memory mapping allows selective loading of only the required portions into memory based on user queries, rather than loading the entire cube for all users.
Solution Approach 2:
The patent introduces memory mapping technology as an intermediary layer between physical storage and user access. This intermediary mechanism allows data to be accessed efficiently by creating virtual memory references that resolve to physical storage locations on demand, eliminating the need to preload entire data cubes into memory.
3Quantity of substance
If a single copy of the cube is shared across users, then resource efficiency is improved, but user-specific data transformations become complex
Solution Approach 1:
The patent applies local quality by creating user-specific process spaces with individual perturbation rules for each user. Each user's query processing environment is customized with their own data transformations, security rules, and viewing parameters applied locally to the shared data, allowing different users to see different views of the same underlying data without complicating the core storage structure.
4Adaptability or versatility
If the data cube is distributed across multiple computing devices, then scalability is improved, but query processing coordination becomes more difficult
Solution Approach 1:
The patent implements a universal query processing approach where the same memory mapping and perturbation rule compilation mechanism handles queries across all computing devices in the distributed grid. The system uses standardized interfaces and common processing logic that works uniformly across different nodes, allowing the distributed system to scale without requiring complex device-specific coordination protocols.
Data Source
AI summary
A method of performing a query on a cube of data is provided. A cube of data is distributed onto a plurality of computing devices with each computing device of the plurality of computing devices storing a different portion of the cube of data. A perturbation rule configured for application to the cube of data and associated with a user is received. A process space associated with the user is created. The received perturbation rule is compiled in association with the created process space. A query on the portion of the cube of data stored at the computing device is received. The received query is associated with the created process space. The query is processed while applying the compiled perturbation rule to data extracted from the portion of the cube of data stored at the computing device. A result of the processed query is sent to a requesting computing device.