Distributed Processing System for Characteristic Data
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Conventional methods for processing characteristic data in massive data systems face issues of long processing durations and limited performance due to the reliance on a single machine, which restricts the speed of characteristic data updates.
Innovation Solution
A distributed processing system is implemented, where a control node determines the appropriate computing node for processing characteristic data based on its type and identifier, allowing for efficient scheduling and processing without constructing a characteristic tree, thereby reducing processing duration and alleviating single-machine performance limitations.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Device complexity
If characteristic data processing is performed on a single machine, then the processing logic is simple, but the processing duration is long and the performance is limited
Solution Approach 1:
The patent segments the characteristic data processing task by dividing the characteristic tree into multiple sub-trees and distributing them across multiple computing nodes. Each node processes a portion of the data independently, transforming a single bottleneck into parallel processing units, thereby increasing overall processing speed without significantly complicating individual node logic.
Solution Approach 2:
The patent introduces a new dimension of parallelism by distributing processing across multiple computing nodes simultaneously. Instead of sequentially processing the entire characteristic tree on one machine, the system processes multiple sub-trees in parallel across different nodes, effectively adding a spatial dimension to the processing architecture to overcome single-machine performance limits.
2Ease of operation
If characteristic data is processed on a single machine, then data management is simple, but the characteristic update speed is low
Solution Approach 1:
The system segments the characteristic data into multiple partitions distributed across different computing nodes. Each node manages its own subset of data independently, allowing parallel updates without requiring centralized coordination for every operation. This segmentation enables faster characteristic updates while maintaining manageable data organization through clear partition boundaries.
Solution Approach 2:
The patent introduces a data management intermediary layer that coordinates data distribution and updates across multiple computing nodes. This intermediary handles the complexity of distributed data management, allowing individual nodes to process data independently while maintaining overall data consistency, thus achieving fast update speeds without sacrificing operational simplicity at the node level.
Data Source
Figure 1~2
Figure 3~4
Figure 5~6
AI summary
Disclosed are a distributed processing system, a data processing method and a control node apparatus. The distributed processing system comprises a control node, a first computing node and a second computing node. The control node acquires a type and a characteristic identifier of characteristic data to be processed. The type is used to identify a home computing node of the characteristic data to be processed. The home computing node is the first computing node or the second computing node. The control node determines, according to the characteristic identifier, process index information. The control node schedules, for process to be processed of the home computing node, and according to the process index information, data index information of the characteristic data to be processed or the characteristic data to be processed. The data index information is used to instruct the home computing node to acquire the characteristic data to be processed, such that the home computing node processes the characteristic data to be processed.