Storage Node Compression Scheduling Based on Node Workload
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing storage systems face low compression efficiency due to inadequate assignment of data compression tasks based on the current working status of each node, leading to slow compression rates and overall performance issues when nodes are busy.
Innovation Solution
A data compression method where a metadata controller periodically collects the working status of computing and storage nodes, identifying active nodes that are not busy, and assigns data compression tasks to these nodes using distributed software headers or memory managers for efficient compression.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If data compression tasks are assigned to control components in nodes regardless of their working status, then data compression can be performed, but compression efficiency decreases and system performance is affected when nodes are busy
Solution Approach 1:
The patent implements dynamic task assignment by continuously monitoring the working status of nodes and adjusting compression task allocation in real-time. The metadata controller collects status information and dynamically determines which nodes are suitable for compression tasks based on current system state, transforming the static task assignment into a dynamic adaptive process that resolves the contradiction between maintaining compression functionality and preserving system performance.
Solution Approach 2:
The system establishes a feedback mechanism where the metadata controller periodically collects working status information from nodes, evaluates their suitability for compression tasks, and adjusts task assignment accordingly. This feedback loop ensures that compression tasks are only assigned to nodes in appropriate states, preventing performance degradation while maintaining compression efficiency.
2Speed
If compression tasks are assigned without considering node status, then task assignment is simple, but compression rate becomes slow
Solution Approach 1:
The metadata controller serves as an intermediary between compression task generation and node execution. It collects status information from nodes, evaluates their suitability, and makes intelligent assignment decisions. This intermediary layer adds complexity to the task assignment process but enables faster compression rates by ensuring tasks are assigned to appropriate nodes rather than using simple round-robin or random assignment.
Solution Approach 2:
Nodes periodically report their own working status to the metadata controller, enabling self-service monitoring. The metadata controller uses this self-reported information to make informed assignment decisions, balancing the complexity of status monitoring with the benefit of improved compression rate through intelligent task placement.
Data Source
Figure 1
Figure 2
Figure 3
AI summary
The present invention discloses a data compression method, including: periodically collecting, by a metadata controller MDC in a storage system, first information indicating a current working status of each computing node in the storage system, and second information indicating a current working status of each storage node in the storage system; determining, by the MDC based on the first information, a computing node that is one of the computing nodes and whose current working status is not a busy state as a current active computing node; determining, by the MDC based on the second information, a storage node that is one of the storage nodes and whose current working status is not a busy state as a current active storage node; and compressing, by a distributed software header DSH in the current active computing node or a memory manager MM in the current active storage node, a piece of data when the storage system needs to compress the data. Meanwhile, the present invention further discloses a storage system.