Network And System Graph Workload Deduplication Using Hash Values
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Modern network and system graph processing frameworks often result in redundant processing due to parallel processing of subgraphs with common workloads, leading to inefficiency and resource wastage.
Innovation Solution
The method involves partitioning and segmenting graphs into subgraphs, identifying and eliminating redundancy by comparing hash values, and scheduling a single instance of workloads for processing to eliminate duplicate tasks and results.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If parallel processing of subgraphs is used, then processing efficiency is improved, but redundant processing occurs leading to resource wastage
Solution Approach 1:
The patent applies preliminary action by computing hash values of subgraphs before parallel processing to identify duplicates. This pre-check mechanism detects redundant workloads in advance, allowing the system to eliminate duplicate processing tasks before they consume computing resources, thereby resolving the contradiction between maintaining high processing efficiency and avoiding resource wastage
2Productivity
If independent processing resources are used for each subgraph, then parallel processing capability is improved, but redundancy detection becomes difficult
Solution Approach 1:
The patent uses copying by creating hash value representations of subgraphs. Instead of directly comparing entire subgraphs which would be computationally expensive, the system copies essential features into hash values that serve as fingerprints. This allows independent processing resources to efficiently detect redundancy through simple hash comparison while maintaining parallel processing capability
Data Source
AI summary
Aspects of the subject disclosure may include, for example, obtaining a graph, partitioning the graph to generate a plurality of subgraphs, segmenting the plurality of subgraphs to obtain a plurality of segmented subgraphs, determining that at least a first segmented subgraph of the plurality of segmented subgraphs and a second segmented subgraph of the plurality of segmented subgraphs are redundant to one another, resulting in a first redundancy, and eliminating the first redundancy. Other embodiments are disclosed.