Storage Path Allocation for Real-Time Rate-Matched Data Writing
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
The disparity in transmission rates between frontend and backend components in storage devices causes delays in data access, and existing storage solutions fail to efficiently manage diverse data service scenarios, leading to inefficiencies and reliability issues.
Innovation Solution
A method and apparatus that optimize data storage by determining storage paths and adjusting transmission amounts based on real-time rates and service scenarios, ensuring balanced resource allocation across server-to-storage, server-to-switch-to-storage, and server-to-direct just a bunch of disks cascades.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Device complexity
If uniform data transmission and processing is performed in various service scenarios, then hardware resources are allocated simply, but data storage efficiency and service reliability deteriorate due to indiscriminate IO resource occupation
Solution Approach 1:
The patent applies local quality by differentiating data storage handling based on service scenarios. Different service scenarios (e.g., backup, archiving, hot data access) are assigned different storage paths and resource allocation strategies. Critical data receives priority treatment while less critical data uses standard pathways, enabling tailored resource allocation that improves overall storage efficiency without requiring complete system redesign.
Solution Approach 2:
The patent segments the storage system into multiple independent storage paths (direct storage, switch-mediated storage, cascade storage) that can be selectively activated based on service scenario requirements. This segmentation allows the system to route different data types through appropriate pathways, avoiding uniform processing bottlenecks and optimizing resource utilization across diverse workloads.
2Adaptability or versatility
If QOS resource restriction and automatic tiering algorithms are applied at management software level, then different services are provided according to data importance, but system complexity increases and does not fully address hardware-level resource contention
Solution Approach 1:
The patent introduces storage paths as intermediary structures between data and hardware resources. These paths act as mediators that automatically route data to appropriate storage locations based on service scenario metadata, eliminating the need for complex software-based QOS management. The intermediary paths handle resource allocation at the hardware level, reducing software complexity while maintaining service differentiation.
3Speed
If high-speed transmission links (PCIE 5.0, FC, 10G network) are used, then data transmission rate increases, but transmission bandwidth mismatch with backend storage causes delays in data writing and access
Solution Approach 1:
The patent implements dynamic storage path selection that adapts to real-time transmission conditions. The system monitors backend storage availability and transmission status, dynamically switching between different storage paths (direct, switch-mediated, cascade) to optimize data flow. This dynamic adaptation ensures that high-speed transmission links are fully utilized when backend storage can keep up, while automatically routing through alternative paths when bottlenecks are detected, thereby minimizing data access delays.
Data Source
AI summary
The present disclosure provides a method for processing stored data. The method includes: obtaining service data, where the service data includes a service scenario and a total transmission amount; determining a storage path according to the service scenario, where the storage path includes a first storage sub-address corresponding to the server-to-storage cascade, a second storage sub-address corresponding to the server-to-switch-to-storage cascade and a third storage sub-address corresponding to the server-to-direct just a bunch of disks cascade; detecting a real-time transmission rate of the storage topology; dividing the total transmission amount into a first transmission amount, a second transmission amount and a third transmission amount based on the real-time transmission rate; and storing the first transmission amount based on the first storage sub-address, storing the second transmission amount based on the second storage sub-address, and storing the third transmission amount based on the third storage sub-address.