Multi-source BFS Memory Management via Dynamic Matrix Expansion
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing graph traversal techniques, such as multi-source breadth-first search (MS-BFS), face significant memory challenges due to the large size of real-world graph datasets, leading to inefficient memory allocation and potential system crashes during intensive graph analysis.
Innovation Solution
The approach involves dynamically updating matrices with progressively larger primitive integer types based on the expanding search radius, allowing for efficient encoding of distance values and minimizing memory usage by deferring allocation until necessary, thereby avoiding disk thrashing and accelerating computation.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Loss of time
If excessive memory is pre-allocated for frontier history, then preparation time is minimized, but memory space is wasted and virtual memory thrashing occurs during search execution
Solution Approach 1:
The patent implements dynamic memory allocation for frontier history by using a hash table that expands automatically as the search radius increases. Instead of pre-allocating fixed memory, the system dynamically adjusts memory usage based on the actual graph diameter discovered during traversal, eliminating both excessive pre-allocation and preparation time calculations
Solution Approach 2:
The system changes the memory allocation parameter from a fixed pre-allocated size to a dynamic size that adapts to the graph's actual properties. The hash table capacity is adjusted based on the search radius and graph diameter, allowing optimal memory usage without wasting space or requiring cubic-time diameter calculations
2Quantity of substance
If a modest amount of memory is pre-allocated for frontier history, then memory space is conserved, but the system may crash or abort execution on large graphs
Solution Approach 1:
The hash table implementation provides dynamic memory expansion capability, starting with modest initial allocation and automatically growing as needed. This ensures execution stability on large graphs while maintaining efficient memory usage on smaller graphs, eliminating the risk of crashes associated with fixed modest allocation
Solution Approach 2:
The system performs preliminary action by implementing a hash table with automatic expansion capability before encountering memory demands. This preliminary structure allows the system to handle any graph size without risking execution failure, while still starting with minimal memory footprint
3Quantity of substance
If cubic complexity algorithm is used to calculate graph diameter, then pre-allocated memory is minimized, but immense preparation time is required
Solution Approach 1:
The patent extracts the diameter calculation from the preliminary preparation phase and integrates it into the traversal process itself. The hash table expansion during BFS naturally discovers the graph diameter without requiring a separate cubic complexity calculation, eliminating the time-costly preprocessing step while maintaining optimal memory allocation
Solution Approach 2:
The system performs the diameter discovery as a preliminary action during the natural course of traversal rather than as a separate preprocessing step. The hash table automatically expands to accommodate the graph's actual diameter, achieving optimal memory allocation without the cubic-time calculation overhead
Data Source
AI summary
Techniques herein minimize memory needed to store distances between vertices of a graph for use during a multi-source breadth-first search (MS-BFS). In an embodiment, during each iteration of a first sequence of iterations of a MS-BFS, a computer updates a first matrix that contains elements that use a first primitive integer type having a first width to record a distance from a source vertex of a graph to another vertex. The computer detects that a count of iterations of the first sequence of iterations exceeds a threshold. Responsively, the computer creates a second matrix that contains elements that use a second primitive integer type having a second width that is larger than the first width to record a distance from a source vertex of the graph to another vertex. During each iteration of a second sequence of iterations of the MS-BFS, the computer updates the second matrix.