Fingerprint Status Detection in Distributed Processing Systems
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing methods for detecting failures in distributed processing systems are inefficient, generating excessive network traffic and consuming significant memory and CPU resources, especially in large systems with thousands of nodes, where end-to-end response time is critical.
Innovation Solution
A fingerprint-based status detection method that uses XOR operations to generate and send fingerprints at root, middle, and leaf nodes, reducing tracking overhead by only generating traffic at leaf nodes and ensuring reliable data processing without traffic at each node.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If traditional status detection methods are used in distributed processing systems, then reliability of data processing is maintained, but network traffic and resource consumption increase significantly
Solution Approach 1:
The patent extracts the essential status information from all nodes by using fingerprints that aggregate processing status. Instead of collecting raw status data from every node, the system computes a fingerprint that represents the aggregate state, thereby reducing data volume while maintaining reliability information.
Solution Approach 2:
The patent merges status information from multiple nodes into a single fingerprint value. By combining the processing status of all nodes into one aggregated fingerprint, the system reduces the total amount of data transmitted while still enabling reliable status detection across the distributed system.
2Measurement precision
If status tracking is performed at all nodes, then complete status monitoring is achieved, but tracking overhead and latency increase
Solution Approach 1:
The patent extracts only the essential status information needed for monitoring by computing fingerprints at intermediate nodes rather than transmitting complete status data from all nodes. This extraction approach maintains monitoring completeness while significantly reducing tracking overhead.
Solution Approach 2:
The patent performs preliminary fingerprint computation at intermediate nodes before data reaches the final tracking point. By preparing and aggregating status information in advance at intermediate nodes, the system reduces the amount of data that needs to be processed and transmitted, thereby lowering latency.
3Difficulty of detecting and measuring
If detailed status data is collected from all nodes, then comprehensive failure detection is achieved, but CPU and memory resources are consumed excessively
Solution Approach 1:
The patent extracts only the essential status information required for failure detection by computing compact fingerprints that aggregate node states. This extraction maintains comprehensive failure detection capability while dramatically reducing CPU and memory resource consumption compared to collecting detailed status data from all nodes.
Solution Approach 2:
The patent transforms detailed status data into a compressed fingerprint representation by changing the data parameters. Instead of storing and processing raw status information from all nodes, the system uses transformed fingerprint values that preserve failure detection capability with minimal resource consumption.
Data Source
AI summary
A method, device, and computer program product for fingerprint based status detection in a distributed processing system is provided. The method comprises: generating and sending, at a root node, an initial fingerprint based on the output message ID to be sent; at a middle node, generating and sending an updated fingerprint based on a received input message ID, an output message ID to be sent and a received fingerprint; sending, at the leaf node, a final updated fingerprint to a tracking task point; and obtaining, at the tracking task point, status information based on the final updated fingerprint. As the method does not generate traffic at each node but only generates tracking traffic at leaf nodes, the approach according to the present disclosure reduces tracking overhead significantly while guaranteeing the reliability of data processing.