Storage Resource Allocation for Video Cloud Clusters
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing video cloud storage systems face challenges in load balancing across storage nodes, leading to uneven stress distribution and potential packet loss, especially when managing large storage requests, as current strategies focus on individual node load conditions without considering global system views or varying storage capacities.
Innovation Solution
A method for allocating storage resources in a video cloud storage system that involves real-time node synchronization information management, stress estimation algorithms, and strategic selection of storage nodes based on current writes and capacities to balance load stress across the cluster, ensuring no single node is overwhelmed.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If data is allocated to storage nodes based on current load conditions (reads/writes, CPU, memory, disk IO), then individual node stress is minimized at the moment of allocation, but storage nodes may receive too much data writing in a short time causing instant stress increase and packet loss
Solution Approach 1:
The patent introduces a preliminary evaluation mechanism that assesses storage node capacity and load conditions before allocating data writing tasks. The management server pre-calculates the stress impact of potential allocations and selects nodes that can handle the additional load without exceeding capacity thresholds, preventing instant stress increases and packet loss while maintaining throughput.
Solution Approach 2:
The patent implements a feedback mechanism where storage nodes continuously report their load conditions (CPU usage, memory usage, disk IO, current number of reads/writes) to the management server. The server uses this real-time feedback to dynamically adjust data allocation decisions, ensuring that nodes are not overloaded while maintaining optimal system throughput.
2Reliability
If load balancing is performed only at individual node level, then individual node stress is optimized, but global storage capacity utilization is uneven with some nodes full and others having remaining space
Solution Approach 1:
The patent extends the load balancing approach from a single-node dimension to a cluster-wide dimension. The management server evaluates storage nodes not only based on their current load conditions but also on their remaining storage capacity and overall cluster utilization. This multi-dimensional evaluation enables the system to distribute data writing across the entire cluster, balancing both individual node stress and global capacity utilization, preventing situations where some nodes are full while others remain underutilized.
3Adaptability or versatility
If storage nodes have varying storage capacities, then system flexibility is improved, but uneven stress distribution occurs with nodes having different remaining spaces suffering from increased writing stress
Solution Approach 1:
The patent applies local quality by tailoring the data allocation strategy to the specific characteristics of each storage node. Nodes with larger remaining capacity receive proportionally more data writing allocations, while nodes with smaller remaining capacity receive fewer allocations. This node-specific approach ensures that stress distribution is uniform across the cluster despite varying storage capacities, as each node operates within its own capacity constraints while contributing to overall system balance.
Data Source
Figure 1
Figure 2~3
Figure 4
AI summary
Embodiments of the present disclosure provide a method and apparatus for allocating storage resource of video cloud storage, applicable to a management node in a storage cluster. The method comprising: receiving a video data storage request from a client; obtaining a queue of allocable storage nodes according to an encoder identifier contained in the request; determining if the request is sent for the first time according to the encoder identifier, if so, selecting from the queue of allocable storage nodes a target storage node allocable as a write node according to a stress estimation algorithm; or otherwise, selecting from the queue a target storage node allocable as a write node according to the number of current writes of each of the storage nodes in pre-recorded node synchronization information and the stress estimation algorithm; and feeding back address information of the target storage node to the client. Embodiments of the present disclosure allow storage loads of a cloud storage system to be balanced.