A method and system for improving resource utilization efficiency of a storage system

CN116880752BActive Publication Date: 2026-11-03HUARUI INDEX CLOUD TECH (SHENZHEN) CO LTD
View PDF 2 Cites 0 Cited by

Patent Information

Application Number
CN202310840847.X
Authority / Receiving Office
CN · China
Patent Type
Patents(China)
Current Assignee / Owner
Filing Date
2023-07-07
Publication Date
2026-11-03
Estimated Expiration
2043-07-07

AI Technical Summary

Technical Problem

Traditional storage system software has low resource utilization, especially in software-defined and hyper-converged scenarios where it is difficult to achieve dynamic planning and intelligent scheduling of system resources. This makes it difficult to quantify the relationship between resource consumption and performance, affecting the overall performance and cost of the storage system.

Method used

By binding system resources to lock-free data channels and combining dynamic performance demand feedback from storage system users, the deployment of lock-free data channels can be adjusted in real time to achieve more precise resource utilization optimization.

Benefits of technology

It achieves more efficient utilization of storage system resources, reduces system resource consumption, improves performance and capacity, adapts to dynamic business needs, and supports hyperconverged deployments and cloud-native environments.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN116880752B_ABST
    Figure CN116880752B_ABST
Patent Text Reader

Abstract

The application belongs to the technical field of data storage, and particularly relates to a method and system for improving resource utilization efficiency of a storage system. The method calculates deployment information of a lock-free data channel required to be deployed according to a preset performance requirement of the storage system, a correspondence between the deployment information of the lock-free data channel and a performance index reached by the deployment information, deploys specifications and quantities of the lock-free data channel according to the deployment information of the lock-free data channel required to be deployed, and binds corresponding system resources to each lock-free data channel. The method calculates how many system resources are required to be occupied in order to reach a specified performance index, deploys the lock-free data channel based on how many system resources are required to be occupied, optimizes occupation of system resources by the storage system itself, and improves resource utilization efficiency of the storage system, that is, better performance and larger capacity of data storage services are provided to the outside with less occupation of system resources.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] This invention belongs to the field of data storage technology, specifically relating to a method and system for improving the resource utilization efficiency of a storage system. Background Technology

[0002] A storage system can be broadly divided into two parts: system hardware and storage system software. The storage system software manages the system hardware, rationally scheduling, allocating, and utilizing its resources, responding to read and write access requests from storage clients, and efficiently processing storage I / O. In this process, the storage system software consumes critical system resources, including CPU, memory, network bandwidth, and disk media bandwidth. The efficiency with which the storage system software utilizes these critical resources determines the overall performance of the storage system (generally measured by metrics such as IOPS, latency, throughput & bandwidth, and storage space utilization), determines the investment in system hardware, and ultimately, the overall cost of the storage system.

[0003] Traditional storage system software typically runs on a computer operating system, relying on the operating system to manage and schedule system hardware resources. It depends on the operating system's CPU scheduling, multi-threading scheduling, memory management, interrupt management, kernel I / O processing, and peripheral management mechanisms such as network cards and disks to complete I / O processing, network transmission, and data storage. In this process, the system resources consumed by the storage system, relying on the various resource scheduling mechanisms of the general-purpose computer operating system, are difficult to accurately estimate and measure, let alone achieve intelligent allocation and elastic scheduling of system resources. This is especially true when the storage system software needs to share hardware resources with other system software and application software (such as in a typical hyperconverged system). The general-purpose computer operating system needs to adopt a more universal and equitable method to uniformly coordinate, manage, and schedule system resources. Furthermore, it becomes even more difficult to establish a quantifiable relationship between the storage system's resource consumption and its performance metrics. Therefore, traditional storage system software adopts a closed, hardware-software integrated approach, ensuring that all system hardware resources are available for the storage system software. Software-defined storage system software, on the other hand, uses pre-allocated sufficient system resources to ensure the storage system's resource consumption. Both approaches lack the ability to dynamically plan and intelligently schedule system resources, resulting in low overall resource utilization efficiency of the storage system. In emerging data storage and processing scenarios such as software-defined, hyper-converged, cloud storage, and compute offloading to storage, it is even more difficult to balance the relationship between the storage system's resource consumption and that of other systems, making it difficult to provide the highest and most stable storage system performance with the least possible system resource consumption. Summary of the Invention

[0004] The purpose of this invention is to provide a method and system for improving the resource utilization efficiency of storage systems, so as to solve the problem of low resource utilization in existing storage systems.

[0005] To address the aforementioned technical problems, this invention provides a method for improving the resource utilization efficiency of a storage system, comprising the following steps:

[0006] 1) Based on the preset storage system performance requirements and the correspondence between the deployment information of lock-free data channels and the performance indicators achieved by the deployment information, calculate the deployment information of lock-free data channels required to meet the preset storage system performance requirements; the deployment information includes specifications and quantity.

[0007] 2) Deploy the specifications and quantity of the lockless data channels according to the deployment information of the lockless data channels to be deployed, and bind the corresponding system resources to each lockless data channel.

[0008] Its beneficial effects are as follows: Since each lock-free data channel is bound to a specified system resource, the lock-free data channel does not have problems such as resource contention, resource round-robin waiting, context switching, and mutual interference. It can accurately estimate how much system resource is required to achieve specified performance indicators (IOPS, latency, and bandwidth, etc.). Therefore, deploying the lock-free data channel based on the required system resource usage can achieve more precise and intelligent optimization of the storage system's own system resource usage, so as to provide better performance and larger capacity data storage services with less system resource usage (referring to the storage system's own consumption of key system resources such as CPU, memory, network bandwidth, and disk media).

[0009] Furthermore, in step 2), when the storage system application corresponding to the lockless data channel bound to the corresponding system resource is applied, the preset storage system performance requirements are corrected in real time based on the dynamic performance demand feedback of the corresponding storage system application user and the actual performance of the storage system. Based on the corrected storage system performance requirements, the lockless data channels after binding the corresponding system resources are dynamically adjusted so that the actual performance indicators of the dynamically adjusted lockless data channels are closer to the preset storage system performance requirements than the actual performance indicators of the lockless data channels without dynamic adjustment.

[0010] In this invention, when the storage system is used, the preset storage system performance requirements are also corrected in real time based on the user's dynamic performance demand feedback and the actual performance of the storage system. The lockless data channel of the storage system is dynamically adjusted based on the real-time acquired data, so that the actual performance indicators of the adjusted lockless data channel are closer to the preset storage system performance requirements. This optimizes the storage system's own occupation of system resources, thereby improving the resource utilization efficiency of the storage system with less system resource consumption.

[0011] Furthermore, the dynamic adjustment includes reducing and increasing lock-free data channel adjustments. Reducing lock-free data channel adjustments includes: identifying lock-free data channels to be reduced, ceasing new task allocation for these channels, and releasing them after the tasks already performed on them are completed. Increasing lock-free data channel adjustments includes: determining the number of lock-free data channels to be increased, deploying the corresponding number of lock-free data channels, and binding the increased lock-free data channels to corresponding system resources.

[0012] Furthermore, by calculating the difference between the current system performance requirements and the actual system performance, the number of lock-free data channels that need to be added or removed at each node of the system is calculated. If the calculation result is to increase the number of lock-free data channels, then the adjustment of adding lock-free data channels is performed; if the calculation result is to reduce the number of lock-free data channels, then the adjustment of reducing lock-free data channels is performed.

[0013] Furthermore, in step 1), the correspondence between the deployment information of the lockless data channel and the performance indicators achieved by the deployment information is obtained by testing the hardware configured in the storage system.

[0014] To address the aforementioned technical problems, this invention also provides a system for improving the resource utilization efficiency of a storage system, comprising a storage management unit and a management agent unit. The storage management unit includes a system controller, which calculates the deployment information of lock-free data channels required to meet the preset storage system performance requirements, based on preset storage system performance requirements and the correspondence between the deployment information of lock-free data channels and the performance indicators achieved by the deployment information. The system controller then sends deployment instruction information, including the deployment information, to the management agent unit. The deployment information includes specifications and quantity.

[0015] The management agent unit is used to deploy the specifications and quantity of the lockless data channels according to the deployment information of the lockless data channels to be deployed when it receives the deployment instruction information, and to bind the corresponding system resources to each lockless data channel.

[0016] Its beneficial effects are as follows: Since each lock-free data channel is bound to a specified system resource, the lock-free data channel does not have problems such as resource contention, resource round-robin waiting, context switching, and mutual interference. It can accurately estimate how much system resource is required to achieve specified performance indicators (IOPS, latency, and bandwidth, etc.). Therefore, deploying the lock-free data channel based on the required system resource usage can achieve more precise and intelligent optimization of the storage system's own system resource usage, so as to provide better performance and larger capacity data storage services with less system resource usage (referring to the storage system's own consumption of key system resources such as CPU, memory, network bandwidth, and disk media).

[0017] Furthermore, the system controller is also configured to, when applying the storage system corresponding to the lockless data channel bound to the corresponding system resource, adjust the preset storage system performance requirements in real time based on the dynamic performance demand feedback of the corresponding storage system user and the actual performance of the storage system, and dynamically adjust the lockless data channels after binding the corresponding system resource according to the adjusted storage system performance requirements, so that the actual performance index of the dynamically adjusted lockless data channel is closer to the preset storage system performance requirements than the actual performance index of the lockless data channel when it is not dynamically adjusted.

[0018] In this invention, when the storage system is used, the preset storage system performance requirements are also corrected in real time based on the user's dynamic performance demand feedback and the actual performance of the storage system. The lockless data channel of the storage system is dynamically adjusted based on the real-time acquired data, so that the actual performance indicators of the adjusted lockless data channel are closer to the preset storage system performance requirements. This optimizes the storage system's own occupation of system resources, thereby improving the resource utilization efficiency of the storage system with less system resource consumption.

[0019] Furthermore, the dynamic adjustment includes reducing and increasing lock-free data channel adjustments. Reducing lock-free data channel adjustments includes: identifying lock-free data channels to be reduced, ceasing new task allocation for these channels, and releasing them after the tasks already performed on them are completed. Increasing lock-free data channel adjustments includes: determining the number of lock-free data channels to be increased, deploying the corresponding number of lock-free data channels, and binding the increased lock-free data channels to corresponding system resources.

[0020] Furthermore, by calculating the difference between the current system performance requirements and the actual system performance, the number of lock-free data channels that need to be added or removed at each node of the system is calculated. If the calculation result is to increase the number of lock-free data channels, then the adjustment of adding lock-free data channels is performed; if the calculation result is to reduce the number of lock-free data channels, then the adjustment of reducing lock-free data channels is performed.

[0021] Furthermore, the correspondence between the deployment information of the lockless data channel and the performance indicators achieved by the deployment information is obtained by testing the hardware configured in the storage system. Attached Figure Description

[0022] Figure 1 This is a system block diagram of the present invention for improving the resource utilization efficiency of a storage system;

[0023] Figure 2 This is a flowchart of the initial installation and deployment method in the method for improving the resource utilization efficiency of a storage system according to the present invention;

[0024] Figure 3 This is a flowchart of the method for continuous performance feedback and triggering dynamic control adjustment in the method for improving the resource utilization efficiency of a storage system according to the present invention;

[0025] Figure 4 This is a flowchart of the method for dynamically controlling and adjusting system resource consumption in the method for improving the resource utilization efficiency of the storage system of the present invention. Detailed Implementation

[0026] To make the objectives, technical solutions, and advantages of the present invention clearer, the present invention will be further described in detail below with reference to the accompanying drawings and embodiments.

[0027] System implementation examples for improving storage system resource utilization efficiency:

[0028] This embodiment aims to achieve more precise and intelligent control over the storage system's resource consumption. Specifically, it dynamically determines the CPU, memory, network bandwidth, and other system resources required by the storage system software based on the storage system's rated performance indicators or the dynamic performance requirements of the business systems accessing the storage system. It also features a feedback system based on the storage system's performance requirements to continuously and dynamically optimize the storage system software's resource consumption, thereby providing better-performing and more stable data storage services with less and more optimized system resource consumption (referring to the storage system's own consumption of critical system resources such as CPU, memory, network bandwidth, and disk media).

[0029] like Figure 1 The diagram shown is a system architecture diagram of this embodiment, which includes a storage management unit (i.e., Figure 1 The system comprises a storage management platform and a management agent unit. The storage management unit includes a system controller, a system performance monitoring unit, a lock-free data channel performance baseline testing unit, and other management and maintenance components. Specifically, each component performs the following functions: The storage management platform provides unified storage management functions for the entire storage system or storage cluster; the management agent runs on each storage node, responding to instructions from the storage management platform and performing corresponding operations on the storage node; the lock-free data channel executes a new storage I / O model based on lock-free data channels, achieving end-to-end processing of storage I / O after sharding; the lock-free data channel performance baseline test is used to test the performance metrics (defined as RTC_1(N1,N2,…,Nn)) achievable by a lock-free data channel with a certain configuration in the current actual hardware environment; system performance monitoring continuously monitors the system's performance metrics (defined as System(P1,P2,…,Pn)) in the current actual storage system environment; and the system controller is the core control component of the system, including the following core functions:

[0030] 1. The storage system administrator can define system performance requirements on the system controller (defined as System(P1,P2,…,Pn)).

[0031] 2. Conduct baseline performance tests on the unlocked data channel to determine the performance provided by unlocked data channels of different specifications.

[0032] 3. Provide an API interface to enable the system performance monitoring component to continuously report the actual performance that the system can currently provide.

[0033] 4. Provide API interfaces to allow external systems to continuously provide feedback on their current performance requirements.

[0034] 5. Complete the initial installation and deployment of lock-free data channels on each node.

[0035] 6. Based on the actual performance that the current system can provide and the performance required by the external system, dynamically control the number of IO fragments entering each lock-free data channel, and dynamically adjust the number and specifications of lock-free data channels on each node.

[0036] The system establishes several lock-free data channels as needed. Storage I / O is fragmented into different lock-free data channels according to IDs. End-to-end data processing is performed within each data channel until the data is persisted to the storage medium and confirmation information is sent to the client. Within a lock-free data channel, a polling mode is used to respond to I / O fragments entering this channel and process them. The entire processing uses a single-threaded model, with no process / thread context switching, no interrupt handling, no lock waiting, and no CPU time slice rotation, thus achieving extremely high I / O processing speeds.

[0037] Each lock-free data channel is bound to one or more CPU cores and a certain amount of memory, responsible for handling a specified number of read / write operations per second (IOPS) and bandwidth of disk media, or for handling a specified network bandwidth. This network can be a network spanning multiple hardware machines, including but not limited to TCP / IP networks, RDMA networks, or InfiniBand networks, or an interconnection network of multiple nodes within the same chassis, including but not limited to various internal interconnect buses, SAS buses, etc. Because each lock-free data channel is bound to specific system resources, there are no issues such as resource contention, resource round-robin waiting, context switching, or mutual interference. It is possible to accurately estimate how many system resources are needed to achieve specified performance metrics (IOPS, latency, and bandwidth, etc.).

[0038] Each lock-free data channel, after consuming (N1, N2, ..., Nn) system resources (e.g., N1 = number of CPU cores, N2 = CPU frequency and level, N3 = memory size, ...), can achieve performance indicators (P1, P2, ..., Pn) (e.g., P1 = IOPS, P2 = single-path latency, P3 = latency when reaching the IOPS specified by P1, ...). This correspondence is called the lock-free data channel performance baseline. Based on some common general-purpose hardware combinations, a set of lock-free data channel performance baselines can be obtained through laboratory testing, serving as a benchmark for formal deployment and use of storage system software.

[0039] The specific methods for improving the utilization efficiency of storage system resources based on this embodiment include an initial installation and deployment method, a continuous performance feedback and dynamic control adjustment method, and a dynamic control adjustment method for system resource consumption.

[0040] like Figure 2 The following is a flowchart illustrating the initial installation and deployment method of this embodiment:

[0041] 1) Define the storage system performance requirements in the system controller: System(P1, P2, ..., Pn);

[0042] 2) Determine if the hardware currently configured in the system has laboratory baseline data, i.e., determine if the hardware currently configured in the system has a lock-free data channel performance baseline. If not, perform a lock-free data channel performance baseline test on the current hardware and record the performance of the lock-free data channel on the current hardware as the laboratory baseline data for the next installation and deployment. At this time, the performance baselines of lock-free data channels with different configurations are obtained (in the following performance baseline formula, n in Nn is the same as n in Pn, while n in RTC_n is not necessarily the same as n in Pn): RTC_1(N1,N2,…,Nn)=RTC_1(P1,P2,…,Pn), RTC_2(N1,N2,…,Nn)=RTC_2(P1,P2,…,Pn), …, RTC_n(N1,N2,…,Nn)=RTC_n(P1,P2,…,Pn); If the hardware currently configured in the system has a lock-free data channel performance baseline, then directly obtain the performance baselines of lock-free data channels with different configurations.

[0043] 3) Based on the system performance requirements System(P1, P2, ..., Pn) and the performance baselines of lockless data channels of different specifications RTC_1(N1,N2,...,Nn)=RTC_1(P1,P2,...,Pn), RTC_2(N1,N2,...,Nn)=RTC_2(P1,P2,...,Pn), ..., RTC_n(N1,N2,...,Nn)=RTC_n(P1,P2,...,Pn), the system controller calculates the specifications and quantity of lockless data channels that need to be deployed on different nodes of the system.

[0044] 4) The system controller sends instructions to the management agent on each node to execute the actual installation and deployment of the lockless data channel and bind the lockless data channel to the corresponding system resources (in this embodiment, this is done by using core binding, resource isolation and other means).

[0045] like Figure 3 The diagram shows the method flow for continuous performance feedback and triggering dynamic control adjustments in this embodiment:

[0046] 1) The storage system has completed initial installation and deployment and is now operational;

[0047] 2) The business management system (the user of the storage system) provides real-time feedback on dynamic performance requirements through the API provided by the system controller: System_Dynamic(P1,P2,…,Pn);

[0048] 3) The system controller monitors the actual performance pressure of the system in real time through the management agent, and adjusts the system performance requirements in real time based on the dynamic performance requirements of the business system obtained in step 2) and the actual performance pressure monitored by the system: System(P1,P2,…,Pn);

[0049] 4) Based on the baseline performance that the currently deployed lockless data channel can provide and the real-time corrected system performance requirements obtained in step 3), determine whether the difference between the current system performance requirements and the baseline performance that the system can provide is greater than a certain threshold (the certain threshold is a set value).

[0050] 5) If the actual resource consumption exceeds a certain threshold, the system controller dynamically controls and adjusts the system's actual resource consumption; if it does not exceed the threshold, return to step 3).

[0051] like Figure 4 The following is a flowchart illustrating the method for dynamically controlling and adjusting system resource consumption in this embodiment:

[0052] 1) Obtain the difference d between the current system performance requirements and the baseline performance that the system can provide; that is, the difference d = current system performance requirements - baseline performance that the system can provide;

[0053] 2) Based on the difference d obtained in step 1), the system controller calculates the specifications or quantity of lockless data channels that need to be added or removed at each node of the system.

[0054] In this embodiment, the sign of the difference d determines whether the number of unlocked data channels needs to be increased or decreased. Then, based on the specific value of the difference d, the specific specifications or quantity of the unlocked data channels to be adjusted (i.e., determining the specific increase or decrease in specifications or quantity) is determined. Specifically, if the difference d is positive, it indicates that the number or specifications of unlocked data channels need to be increased; if the difference d is negative, it indicates that the number or specifications of unlocked data channels need to be decreased. Furthermore, based on the difference d, the number u of unlocked data channels to be increased or decreased can be calculated using the formula: Quantity u = (Rounded up) / (Absolute value (d) / Baseline performance of the unlocked data channel). When the value of the difference d is relatively small and does not reach the minimum threshold for adjusting the number of unlocked data channels (i.e., when the absolute value of the difference d < the minimum threshold value), adjustments can also be made by replacing the unlocked data channels with different specifications. The calculation formula is: Absolute value (difference d) < Absolute value (Baseline performance of the new unlocked data channel – Baseline performance of the old unlocked data channel).

[0055] 3) If the calculated result indicates that the number of lock-free data channels needs to be reduced, the sharding rules of storage I / O are adjusted. The lock-free data channels to be released will no longer bear new I / O. After the existing I / O on the lock-free data channels to be released is completed, the system controller sends instructions to the management agents on each node to complete the release of the lock-free data channels, thereby releasing system resources.

[0056] If the calculation result indicates that an additional lock-free data channel is needed, the system controller sends instructions to the management agents on each node to deploy the new lock-free data channel, binds the corresponding system resources to the new data channel, adjusts the sharding rules of storage I / O, and distributes storage I / O into the newly added lock-free data channel to improve system performance.

[0057] The optimization of storage system resource consumption by this invention has the following effects: it provides a stable performance experience while optimizing resource consumption; the system resource consumption saved by optimization can better support hyperconverged deployments; it reduces hardware specifications and investment while meeting business requirements; under the software-defined, hardware-hardware decoupled deployment method, it can better adapt to and support different specifications and types of hardware; it better adapts to dynamic and elastic cloud-native environments; and it allows more computing work to be done on the storage system, realizing the offloading of various computing power and the sinking of operators.

[0058] Examples of methods to improve the resource utilization efficiency of storage systems:

[0059] The method in this embodiment calculates the deployment information of lock-free data channels required to meet the preset storage system performance requirements, based on the correspondence between the deployment information of lock-free data channels and the performance indicators achieved by the deployment information. It then deploys the specifications and quantity of lock-free data channels according to the required deployment information and binds each lock-free data channel to corresponding system resources. The specific steps for improving storage system resource utilization efficiency have been detailed in the system embodiments for improving storage system resource utilization efficiency and will not be repeated here.

[0060] The above description is merely a preferred embodiment of the present invention and is not intended to limit the present invention. The scope of patent protection of the present invention shall be determined by the claims. Similarly, any equivalent structural changes made based on the description and drawings of the present invention shall also be included within the scope of protection of the present invention.

Claims

1. A method for improving the resource utilization efficiency of a storage system, characterized in that, Includes the following steps: 1) Based on the preset storage system performance requirements and the correspondence between the deployment information of the lock-free data channel and the performance indicators achieved by the deployment information, calculate the deployment information of the lock-free data channel required to meet the preset storage system performance requirements. Deployment information includes specifications and quantity; storage I / O will be sharded to different lock-free data channels according to ID, and end-to-end data processing will be implemented in the lock-free data channels. The lock-free data channels use a polling mode to respond to the I / O shards entering this channel and use a single-threaded model for processing. Each lock-free data channel is bound to a specified system resource. 2) Deploy the lockless data channels according to the required deployment information, specifying the specifications and quantity of the lockless data channels, and bind the corresponding system resources to each lockless data channel; When applying storage systems, the preset storage system performance requirements are modified in real time based on the dynamic performance needs feedback of the storage system users corresponding to the lockless data channels and the actual performance of the storage system. Based on the modified storage system performance requirements, the lockless data channels after binding the corresponding system resources are dynamically adjusted. Dynamic adjustments include reducing and adding lock-free data channels. Reducing lock-free data channels involves: identifying the lock-free data channels to be reduced, ceasing new task assignments to those channels, and releasing them only after all tasks already performed on those channels are completed. Adding lock-free data channels involves: determining the number of lock-free data channels to be added, deploying the corresponding number of lock-free data channels, and binding the added channels to corresponding system resources.

2. The method for improving the resource utilization efficiency of a storage system according to claim 1, characterized in that, The dynamic performance requirements feedback of the storage system users corresponding to the lockless data channel are obtained from the API provided by the system controller.

3. The method for improving the resource utilization efficiency of a storage system according to claim 1, characterized in that, By calculating the difference between the current system performance requirements and the actual system performance, the number of lock-free data channels that need to be added or removed at each node of the system is calculated. If the calculation result is to increase the number of lock-free data channels, then the adjustment of adding lock-free data channels is performed; if the calculation result is to reduce the number of lock-free data channels, then the adjustment of reducing lock-free data channels is performed.

4. The method for improving the resource utilization efficiency of a storage system according to claim 3, characterized in that, Divide the absolute value of the difference between the current system performance requirement and the actual system performance by the actual system performance, and round the result up to the nearest integer. This value represents the number of lock-free data channels that need to be added or removed at each node of the system.

5. The method for improving the resource utilization efficiency of a storage system according to claim 1, characterized in that, In step 1), the correspondence between the deployment information of the lockless data channel and the performance indicators achieved by the deployment information is obtained by testing the hardware configured in the storage system.

6. A system for improving the resource utilization efficiency of a storage system, characterized in that, The system includes a storage management unit and a management agent unit. The storage management unit includes a system controller, which calculates the deployment information of the lock-free data channels required to meet the preset storage system performance requirements, based on the correspondence between the deployment information of the lock-free data channels and the performance indicators achieved by the deployment information. The system controller then sends deployment instructions, including the deployment information, to the management agent unit. The deployment information includes specifications and quantity. Storage I / O is sharded to different lock-free data channels according to ID. End-to-end data processing is implemented in the lock-free data channels. The lock-free data channels use a polling mode to respond to I / O shards entering this channel and use a single-threaded model for processing. Each lock-free data channel is bound to a specified system resource. The management agent unit is used to deploy the lockless data channels according to the required deployment information when it receives deployment instruction information, and to bind the corresponding system resources to each lockless data channel. When applying storage systems, the preset storage system performance requirements are modified in real time based on the dynamic performance needs feedback of the storage system users corresponding to the lockless data channels and the actual performance of the storage system. Based on the modified storage system performance requirements, the lockless data channels after binding the corresponding system resources are dynamically adjusted. Dynamic adjustments include reducing and adding lock-free data channels. Reducing lock-free data channels involves: identifying the lock-free data channels to be reduced, ceasing new task assignments to those channels, and releasing them only after all tasks already performed on those channels are completed. Adding lock-free data channels involves: determining the number of lock-free data channels to be added, deploying the corresponding number of lock-free data channels, and binding the added channels to corresponding system resources.

7. The system for improving the resource utilization efficiency of a storage system according to claim 6, characterized in that, The dynamic performance requirements feedback of the storage system users corresponding to the lockless data channel are obtained from the API provided by the system controller.

8. The system for improving the resource utilization efficiency of a storage system according to claim 6, characterized in that, By calculating the difference between the current system performance requirements and the actual system performance, the number of lock-free data channels that need to be added or removed at each node of the system is calculated. If the calculation result is to increase the number of lock-free data channels, then the adjustment of adding lock-free data channels is performed; if the calculation result is to reduce the number of lock-free data channels, then the adjustment of reducing lock-free data channels is performed.

9. The system for improving the resource utilization efficiency of a storage system according to claim 8, characterized in that, Divide the absolute value of the difference between the current system performance requirement and the actual system performance by the actual system performance, and round the result up to the nearest integer. This value represents the number of lock-free data channels that need to be added or removed at each node of the system.

10. The system for improving the resource utilization efficiency of a storage system according to claim 6, characterized in that, The correspondence between the deployment information of the lockless data channel and the performance indicators achieved by the deployment information is obtained by testing the hardware configured in the storage system.

Citation Information

Patent Citations

  • Controller hardware architecture of memory system

    CN102609379A

  • Resource uniform management method and system

    CN107566184A