Multi-dimensional Range Index Search Using Box Approximation and Splitting
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing methods for multi-dimensional range-index searching are inefficient in distributed data stores, particularly in big data environments, as they struggle with high dimensionality and result in significant excursions outside the desired search box, leading to false alarms and missed solutions.
Innovation Solution
The method employs search box approximation and splitting along the longest bit boundaries, using space-filling curves like Morton indexing, to reduce false alarms and improve performance by iteratively splitting search boxes until only relevant data is encompassed, optimizing the trade-off between missed solutions and false positives.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If hierarchical trees are used for multi-dimensional range-index searching, then search capability is provided, but performance degrades in distributed data stores and big data environments
Solution Approach 1:
The patent segments the multi-dimensional search space into multiple one-dimensional projection ranges along different axes. Instead of using a single hierarchical tree structure for multi-dimensional data, the invention projects the multi-dimensional range query into multiple one-dimensional ranges and performs separate searches along each dimension, then combines results. This segmentation approach eliminates the need for complex multi-dimensional tree structures while maintaining search capability in distributed environments.
2Reliability
If search box size is increased to reduce missed solutions, then detection probability improves, but false positives increase significantly
Solution Approach 1:
The patent segments the search box into multiple smaller sub-boxes along the longest bit boundaries. By dividing the search space into smaller segments, the system can perform multiple targeted searches instead of one large search, thereby reducing the number of false positives while maintaining detection probability. Each sub-search operates on a smaller region, limiting the spread of false alarms.
Solution Approach 2:
The patent introduces a new dimension of searching by projecting multi-dimensional range queries into multiple one-dimensional spaces. This dimensional transformation allows the system to search along each axis independently and combine results, effectively reducing false positives that occur when searching in the original multi-dimensional space with large search boxes.
3Object-generated harmful factors
If search box is split into smaller regions, then false alarms are reduced, but the number of searches increases
Solution Approach 1:
The patent transforms the search problem from multi-dimensional to multiple one-dimensional searches. By projecting the search query onto different axes and performing separate one-dimensional range searches, the system reduces false alarms inherent in multi-dimensional searches while avoiding the exponential increase in search operations that would result from exhaustive splitting. The dimensional transformation provides an efficient middle ground.
Data Source
AI summary
A method involves receiving, from a user, a user query for data contained within a multi-dimensional dataset comprising a plurality of data objects, generating a multi-dimensional search box for use in searching the multi-dimensional dataset according to the user query, wherein the size of the multi-dimensional search box is approximated based upon a percentage of the scale of the dataset and upon the standard deviation of the data object distribution of the dataset, searching the multi-dimensional dataset using the multi-dimensional search box, and returning, to the user, data from the multi-dimensional dataset contained within the multi-dimensional search box. The multi-dimensional search box may be generated in part by modeling the range of values as an arbitrary start point within the multi-dimensional data set with extents that are Gaussian distributed with a standard deviation less than the scale of the data set.