Parallel BFS Graph Processing via Matrix Segmentation
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing Breadth-First Search (BFS) methods for graph data processing are inefficient in large data sets, requiring significant time and space, and previous parallel processing approaches incur high costs and communication overhead.
Innovation Solution
A computer-implemented method using parallel processors with an inter-processor communication network and a master controller to perform matrix multiplication and dimensionality reduction, allowing for efficient construction of a logical pathway and subgraph in undirected graphs by reducing redundant calculations and leveraging parallel processing.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Speed
If traditional BFS methods are used for graph data processing, then the search can be performed sequentially with simple memory access, but the time complexity is O(n^3) and processing speed is slow
Solution Approach 1:
The patent divides the graph processing into multiple independent segments that can be processed in parallel. The graph is partitioned across multiple processors, each handling a portion of the adjacency matrix and corresponding graph nodes. This segmentation enables simultaneous processing of different graph portions, reducing overall execution time from O(n^3) to O(n^2) or linear time.
Solution Approach 2:
The patent transitions from sequential single-processor BFS to parallel multi-processor BFS, adding the dimension of parallel processing. By distributing the computation across multiple processors working simultaneously, the algorithm achieves dimensional expansion in the processing space, transforming the time complexity from cubic to quadratic or linear.
2Speed
If larger random-access memories are provided to store graph data, then BFS can be performed faster with direct memory access, but the cost increases significantly
Solution Approach 1:
The patent segments the large graph data and adjacency matrix across multiple processors, each holding a portion in local memory. This distributed memory approach eliminates the need for a single large random-access memory, as each processor only needs to store its local portion of the graph data, reducing overall memory cost while maintaining processing speed through parallel access.
Solution Approach 2:
The patent introduces an intermediary communication mechanism where processors exchange boundary information and aggregated results through a coordinated protocol. This intermediary approach allows distributed memory systems to function as a unified memory space without requiring physically connected large-capacity memory, reducing hardware costs while enabling fast parallel access.
3Productivity
If parallel processing techniques are used to increase processing speed, then BFS can be performed faster, but communication overhead between processors increases
Solution Approach 1:
The patent extracts and eliminates redundant communication operations by carefully designing the parallel algorithm to minimize data exchange. Only essential boundary information and aggregated results are communicated between processors, removing unnecessary communication overhead while maintaining high processing throughput through efficient parallel computation.
Solution Approach 2:
The patent performs preliminary local computations at each processor before communication is needed, preparing data in advance to reduce the frequency and volume of inter-processor messages. This preliminary action minimizes communication overhead by ensuring that processors have ready-to-use data locally, reducing the need for frequent synchronization and data exchange.
4Speed
If the complete graph is stored in memory for BFS processing, then random access to any node is fast, but the space requirements become prohibitive for large graphs
Solution Approach 1:
The patent segments the complete graph into distributed portions across multiple processors, each storing only its local portion in memory. This segmentation reduces the memory space requirement from storing the entire graph to storing only local portions, while random access speed is maintained for local nodes through direct memory access at each processor.
Solution Approach 2:
The patent implements local quality by optimizing memory storage and access patterns at each processor to handle its specific portion of the graph efficiently. Each processor maintains fast random access to its local data while the distributed architecture reduces overall space requirements, as not all processors need access to all graph data simultaneously.
Data Source
AI summary
A method includes receiving, at a master controller, a matrix representing a graph and a first vector, and initializing a counter variable and an array to track dimensionality reduction for the matrix. The method also includes multiplying a subset of the matrix based on the counter variable, by a subset of the first binary vector based on the counter variable. Multiplying includes providing, the vector and a matrix portion to a first processor, and the vector and another portion of the matrix to a second processor. The method also includes, at the processors, multiplying the vectors by the portions of the matrix and returning the results. The method also includes combining the results at the master controller. The method also includes incrementing the counter variable and updating the tracking array for larger dimensionality reduction of the matrix. The method also includes constructing the logical pathway based on the tracking array.