Hierarchical Adjacency Matrix for DNN Graph Processing
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Deep neural networks (DNNs) face significant computational and memory burdens due to high demands in training and inference processes, particularly when handling large graphs, leading to inefficient data transmission, storage, and processing, as existing solutions like mini-batching and bipartite graphs either increase memory usage or result in unnecessary computation.
Innovation Solution
The introduction of hierarchical adjacency matrices, which are generated by selecting target nodes and their neighbors in a hierarchical manner, allowing for optimized processing by reducing memory footprint and computation through the use of mini-batching and sampling, while avoiding the drawbacks of existing methods.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Power
If mini-batching is used to process large graphs, then computation cost is reduced, but memory footprint increases
Solution Approach 1:
The patent segments the graph processing into multiple mini-batches, where each batch processes a subset of nodes and their neighborhoods. This segmentation allows computation to be performed in smaller steps, reducing the memory required to hold the entire graph adjacency matrix at once, while still achieving the overall processing goal through iterative refinement.
Solution Approach 2:
The patent introduces a temporal dimension to graph processing by processing nodes in sequential batches rather than all at once. This transforms the problem from a static memory-intensive operation to a dynamic iterative process, where intermediate results are stored and reused across batches, reducing peak memory requirements.
2Productivity
If sampling is used to reduce computation, then processing speed improves, but measurement precision deteriorates
Solution Approach 1:
The patent applies local quality by processing each node's neighborhood with full precision while using sampling for broader graph structures. The adjacency matrix is constructed to include exact local connections (1-hop, 2-hop neighbors) while using sampled representations for larger neighborhoods, ensuring local accuracy is maintained while achieving global processing efficiency.
Solution Approach 2:
The patent uses partial action by selectively processing only the necessary neighborhood depth for each node based on the specific task requirements. Rather than uniformly processing the entire graph at maximum depth, it computes adjacency matrices only up to the required hop distance, reducing unnecessary computation while maintaining precision where needed.
3Power
If bipartite graphs are used to reduce computation, then computation cost decreases, but device complexity increases
Solution Approach 1:
The patent inverts the traditional approach by constructing the adjacency matrix to include self-loops and symmetric connections, allowing the graph neural network to process undirected graphs more efficiently. This inversion simplifies the computational logic by eliminating the need for separate forward and backward edge processing that would be required in asymmetric bipartite representations.
Data Source
AI summary
Hierarchical adjacency matrices of a graph may be generated for DNN training or inference. The graph includes nodes connected by edges. One or more target nodes may be selected from the graph. A hierarchical sequence of node groups may be formed. A node group may be a neighborhood in the graph. A first node group (e.g., 0-hop neighborhood) may include the target node(s). A subsequent node group (e.g., 1-hop neighborhood, 2-hop neighborhood, etc.) may include one or more nodes directly connected to any node of the previous node group in the hierarchical sequence. A hierarchical adjacency matrix may be generated based on the hierarchical sequence. The hierarchical adjacency matrix may include rows, each of which rows represents a respective node in the graph. The rows may be arranged in accordance with the hierarchical sequence. The hierarchical adjacency matrix may include elements encoding edges between the nodes in the node groups.