Distributed Data Processing Nodes Using Range-Based Segmentation
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
In distributed computer systems, there is a need to efficiently identify and process content units across multiple nodes to handle data protection, redundancy, and deletion tasks, as existing methods lack a systematic approach to distribute processing effectively across nodes.
Innovation Solution
A method where each node generates a list of content units, divides them into sub-lists by identifier ranges, and shares these with other nodes to create combined lists, identifying which units warrant processing and distributing the processing load across nodes.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If a single node processes all content units in the distributed system, then processing can be completed, but the processing load becomes unbalanced and creates a bottleneck
Solution Approach 1:
The patent divides the distributed system into multiple nodes, each responsible for processing specific content units. The system segments the overall processing task across N nodes, where each node maintains and processes content units within its assigned identifier range, eliminating the bottleneck of centralized processing
Solution Approach 2:
The patent introduces a hierarchical dimension to processing by implementing range-based division. Content units are organized by identifier ranges (e.g., 0-999, 1000-1999), and nodes process units in their assigned ranges. This dimensional organization allows parallel processing across nodes while maintaining systematic control
2Loss of information
If content unit lists are shared among all nodes, then complete information is available, but network communication overhead increases
Solution Approach 1:
Each node maintains local knowledge of content units within its assigned identifier range. Nodes only share and process information relevant to their specific range, rather than all nodes having complete system-wide information. This local quality approach reduces network overhead while ensuring each node has the information it needs
Solution Approach 2:
The patent segments the content unit information space into N partitions, where node i stores and shares information about content units in range (i-1)×R to i×R-1. This segmentation allows selective information sharing between nodes, reducing the total communication overhead compared to complete list distribution
3Quantity of substance
If content units are distributed across multiple nodes, then system capacity increases, but identifying and processing specific content units becomes more complex
Solution Approach 1:
The patent uses content unit identifier ranges as a key parameter for location determination. Each node is assigned a specific range (e.g., node 1: 0-999, node 2: 1000-1999), and the system determines which node stores a content unit by evaluating its identifier against these ranges. This parameter-based approach simplifies detection compared to searching across all nodes
Data Source
Figure 1
Figure 2
Figure 3
AI summary
Some embodiments are directed to processing content units stored on a distributed computer system that comprises a plurality of independent nodes. The content units may be processed by determining which content units are stored on each node and identifying which content units warrant processing. Nodes may be selected to process the content units that warrant processing and instructions may be sent to these nodes to instruct them to process these content units.