Inverted Indexing via Segmented Memory Blocks
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
The resizing of indexes in databases is resource-intensive and causes processing delays due to the need to lock, copy, and unlock existing indexes, which becomes costly as the number of documents increases.
Innovation Solution
Inverted indexing system using memory blocks instead of arrays to store document identifiers, allowing for the addition of new identifiers without locking the index, and allocating new memory blocks as needed, reducing the need for resource-intensive copying processes.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Adaptability or versatility
If traditional indexing resizing process is used (locking, copying, unlocking), then the index can be resized to accommodate new values, but system resources are consumed and processing delays occur
Solution Approach 1:
The patent divides the index into multiple segments or partitions, allowing new values to be added by creating new segments rather than resizing the entire index. This segmentation enables concurrent access where some segments can be modified while others remain locked and accessible, thus maintaining productivity during index evolution.
Solution Approach 2:
The patent pre-allocates space or pre-creates index structures in advance, so when new values need to be added, the infrastructure is already in place and no locking/copying operations are required. This preliminary preparation eliminates the resource-intensive resize process while maintaining adaptability.
2Adaptability or versatility
If index resizing is performed, then the index can accommodate new values, but the existing index must be locked during the process
Solution Approach 1:
By segmenting the index into multiple independent parts, the patent allows different segments to be accessed concurrently. New values can be added to unlocked segments while other segments remain in use, eliminating the need to lock the entire index during capacity expansion.
Solution Approach 2:
The patent introduces intermediary structures such as hash tables or routing mechanisms that direct queries to appropriate index segments. This intermediary layer allows the index to grow without requiring locks on existing segments, as new segments can be added and routed to independently.
3Quantity of substance
If the number of documents increases, then the database can store more data, but the resizing process becomes more costly in terms of time and resources
Solution Approach 1:
The patent divides the document index into multiple segments that can grow independently. As document quantity increases, new segments are created rather than copying entire index structures, dramatically reducing the time and resources required for capacity expansion regardless of the total document count.
Solution Approach 2:
The patent implements a dynamic index structure that can adapt its capacity without停机. New segments or partitions are created on-demand as documents are added, allowing the system to scale with document quantity while maintaining constant-time or near-constant-time operations regardless of index size.
4Device complexity
If traditional array-based indexing is used, then simple storage is achieved, but copying existing indexes during resizing consumes valuable computing resources
Solution Approach 1:
The patent segments the index into multiple independent storage units or partitions. When resizing is needed, only new segments are created and populated, rather than copying entire array structures. This segmentation reduces computing resource consumption from O(n) copying operations to O(1) or O(log n) segment creation operations.
Solution Approach 2:
The patent eliminates the need for expensive array copying operations by using reference-based or pointer-based segment structures. Instead of physically copying index entries during resizing, the system creates new segments with new references, reducing computing resource consumption while maintaining the simplicity of array-based storage for each segment.
Data Source
AI summary
Disclosed herein are system, method, and computer program product embodiments for inverted indexing. An embodiment operates by receiving a request to add a new document identifier (doc ID) to an inverted index. A memory block corresponding to where the doc ID is to be added is determined or located, and the available space remaining in the memory block is determined. If the memory block has available space, the doc ID is added to the memory block.