Resource scheduling method, product, equipment and medium

Abstract

The invention relates to the technical field of storage, in particular to a resource scheduling method, a product, equipment and a medium, and the method comprises the following steps: when a redundant array of independent disks is reconstructed, obtaining a global reconstruction resource ratio corresponding to a reconstruction resource pool, and obtaining an actually used foreground resource ratio corresponding to a service resource pool; determining an available reconstruction resource ratio corresponding to the reconstruction resource pool according to the global reconstruction resource ratio and the foreground resource ratio; obtaining a resource allocation weight of each logical volume of the redundant array of independent disks; determining a resource allocation ratio of each logical volume according to the resource allocation weight and the available reconstruction resource ratio of each logical volume; and performing resource allocation according to the resource allocation ratio of each logical volume. According to the method, resources can be reasonably allocated, the reconstruction process is excellent in effect, and the flexibility of the system is improved.

Description

Technical Field

[0001] The present invention relates to the field of storage technology, and in particular to a resource scheduling method, product, device and medium. Background Art

[0002] RAID (Redundant Array of Independent Disks) is a technology that combines multiple disks to improve data storage performance, reliability, and fault tolerance. RAID technology is widely used in enterprise storage systems, data centers, and high-availability computing environments. RAID1, RAID3, RAID5, and RAID6 provide data redundancy protection. When a disk in a RAID array fails and is replaced, the system restores data from other functioning disks to the new disk, ensuring normal system operation. This is called a disk rebuild. For example, in a RAID5 array consisting of four disks, if a disk fails, the data on the failed disk can be restored using the data on the other three functioning disks using an XOR algorithm.

[0003] The reconstruction process consumes significant system resources (such as CPU, memory, and I / O bandwidth) and may significantly impact ongoing business I / O operations, resulting in reduced system performance. To balance resource competition between the reconstruction process and business I / O, RAID systems typically support configuring a reconstruction rate. This configuration option controls resource usage during the reconstruction process, thereby balancing reconstruction speed with foreground business performance. However, this approach may result in poor reconstruction performance.

[0004] It can be seen that how to reasonably allocate resources to achieve excellent results in the reconstruction process and provide system flexibility is a problem that those skilled in the art need to solve. Summary of the Invention

[0005] The present invention aims to provide a resource scheduling method, product, device and medium that can reasonably allocate resources, make the reconstruction process excellent, and provide system flexibility.

[0006] In a first aspect, a resource scheduling method is provided, comprising: obtaining a global reconstruction resource ratio corresponding to a reconstruction resource pool and an actually used foreground resource ratio corresponding to a business resource pool when a redundant array of independent disks is reconstructed; determining an available reconstruction resource ratio corresponding to the reconstruction resource pool based on the global reconstruction resource ratio and the foreground resource ratio; obtaining resource allocation weights of each logical volume of the redundant array of independent disks; determining a resource allocation ratio of each logical volume based on the resource allocation weights of each logical volume and the available reconstruction resource ratio; and allocating resources based on the resource allocation ratios of each logical volume.

[0007] In a preferred example, the present invention can be further configured as follows: determining the available reconstruction resource ratio corresponding to the reconstruction resource pool based on the global reconstruction resource ratio and the foreground resource ratio, including: determining the elasticity coefficient and the elastic resource ratio corresponding to the elastic resource pool based on the global reconstruction resource ratio and the foreground resource ratio; determining the available reconstruction resource ratio corresponding to the reconstruction resource pool based on the global reconstruction resource ratio, the elastic resource ratio and the elasticity coefficient.

[0008] In a preferred example, the present invention can be further configured as follows: determining the elasticity coefficient and the elastic resource ratio corresponding to the elastic resource pool based on the global reconstruction resource ratio and the foreground resource ratio, including: determining the elastic resource ratio corresponding to the elastic resource pool based on the global reconstruction resource ratio and the foreground resource ratio; determining the maximum resource upper limit ratio corresponding to the business resource pool based on the global reconstruction resource ratio; and determining the elasticity coefficient based on the maximum resource upper limit ratio and the foreground resource ratio.

[0009] In a preferred example, the present invention may be further configured as follows: further comprising: limiting the maximum value of the resource ratio of the foreground resource pool corresponding to the foreground service to a maximum resource upper limit ratio.

[0010] In a preferred example, the present invention can be further configured as: obtaining the actual used foreground resource ratio corresponding to the business resource pool, including: according to the first period, obtaining the bandwidth utilization, cache occupancy, and computing power utilization of the foreground business; performing weighted calculation based on the bandwidth utilization, cache occupancy, and computing power utilization of the foreground business to obtain the actual used foreground resource ratio corresponding to the business resource pool.

[0011] In a preferred example, the present invention can be further configured as follows: obtaining the resource allocation weights of each logical volume of the independent redundant array of disks, including: obtaining the priorities of each logical volume of the independent redundant array of disks; calculating the total priority based on the priorities of each logical volume; and determining the resource allocation weight of each logical volume based on the priority corresponding to the logical volume and the total priority.

[0012] In a preferred example, the present invention can be further configured as follows: obtaining the priority of each logical volume of the independent redundant disk array, including: obtaining volume information corresponding to each logical volume, the volume information including: data importance, delay sensitivity, and data access popularity; for each logical volume, determining the priority score of the logical volume based on the data importance, delay sensitivity, and data access popularity of the logical volume; and determining the priority of the logical volume based on a preset score range within which the priority score lies.

[0013] In a preferred example, the present invention can be further configured as follows: a first custom attribute is set in the configuration record of the logical volume to store the priority of the logical volume; a second custom attribute is set in the configuration record of the redundant array of independent disks controller to store the global reconstruction resource ratio.

[0014] In a preferred example, the present invention can be further configured to include: obtaining user demand information; determining the global reconstruction resource ratio corresponding to the user demand information based on the user demand information and the mapping relationship; the mapping relationship is a mapping relationship between the demand information and the global reconstruction resource ratio.

[0015] In a preferred example, the present invention can be further configured as follows: it also includes: during the reconstruction process of the independent disk redundant array, obtaining the current business load and the load change rate according to the second cycle; if the current business load is greater than the preset load threshold and the load change rate is greater than the preset load change rate, then reducing the available reconstruction resource ratio and adjusting the resource allocation weights of each logical volume.

[0016] In a preferred example, the present invention may be further configured as follows: the current business load is the current actual business load, or a business load predicted based on historical business loads.

[0017] In a preferred example, the present invention can be further configured as: adjusting the resource allocation weights of each logical volume, including: increasing the resource allocation weight of the first logical volume according to a first ratio to obtain an initial first resource allocation weight; the first logical volume is a logical volume whose resource allocation weight is greater than the first allocation weight threshold; increasing the resource allocation weight of the second logical volume according to a second ratio to obtain an initial second resource allocation weight; the second logical volume is a logical volume whose resource allocation weight is greater than the second allocation weight threshold and less than the first allocation weight threshold; the first allocation weight threshold is greater than the second allocation weight threshold; the second ratio is less than the first ratio; using the resource allocation weight of the third logical volume as the corresponding initial third resource allocation weight; the third logical volume is a logical volume whose resource allocation weight is less than the second allocation weight threshold; and determining the adjusted resource allocation weights of each logical volume based on the initial first resource allocation weight, the initial second resource allocation weight, and the initial third resource allocation weight.

[0018] In a second aspect, a resource scheduling device is provided, including: a first acquisition module, used to obtain the global reconstruction resource ratio corresponding to the reconstruction resource pool and the actually used foreground resource ratio corresponding to the business resource pool when the independent disk redundant array is reconstructed; a first determination module, used to determine the available reconstruction resource ratio corresponding to the reconstruction resource pool based on the global reconstruction resource ratio and the foreground resource ratio; a second acquisition module, used to obtain the resource allocation weights of each logical volume of the independent disk redundant array; a second determination module, used to determine the resource allocation ratio of each logical volume based on the resource allocation weight of each logical volume and the available reconstruction resource ratio; and an allocation module, used to allocate resources according to the resource allocation ratio of each logical volume.

[0019] According to a third aspect, an electronic device is provided, comprising a memory and a processor, wherein a computer program is stored in the memory, and the processor executes any one of the methods described in the first aspect when running the computer program.

[0020] In a fourth aspect, a computer-readable storage medium is provided, wherein at least one program code is stored in the computer-readable storage medium, and the program code is loaded and executed by a processor to implement any method as described in the first aspect.

[0021] In a fifth aspect, a computer program product is provided, comprising a computer program or instructions, which, when executed by a processor, implements any method described in the first aspect.

[0022] In summary, the method provided by the present invention includes the following beneficial technical effects: When a redundant array of independent disks is reconstructed, the global reconstruction resource ratio corresponding to the reconstruction resource pool and the actual used foreground resource ratio corresponding to the business resource pool are first obtained. The global reconstruction resource ratio reflects the minimum resources that can be invested in the reconstruction process, and the foreground resource ratio reflects the resources currently actually occupied by the business. Based on these two ratios, the available reconstruction resource ratio corresponding to the reconstruction resource pool is determined, so that during the reconstruction process, reconstruction resources can be reasonably allocated according to the actual business resource usage, avoiding resource idleness, thereby ensuring the balance between the reconstruction process and business operation. In addition, the available reconstruction resource ratio is allocated based on the resource allocation weight of each logical volume, so that during the reconstruction process, critical logical volumes are reconstructed first than non-critical logical volumes. Resources can be reasonably allocated according to the actual business situation and the importance of the logical volumes, making the reconstruction process excellent and providing system flexibility. The present invention also provides a resource scheduling device, equipment and medium, all of which have the above technical effects. BRIEF DESCRIPTION OF THE DRAWINGS

[0023] In order to more clearly illustrate the embodiments of the present invention, the following is a brief introduction to the drawings required for use in the embodiments. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without any creative work.

[0024] Figure 1 This is a flow chart of a resource scheduling method provided by an embodiment of the present invention.

[0025] Figure 2 This is a schematic diagram of a system resource partition provided by an embodiment of the present invention.

[0026] Figure 3 This is a schematic diagram of a hard disk group provided by an embodiment of the present invention.

[0027] Figure 4 The present invention provides a flowchart of a RAID reconstruction resource scheduling method based on differentiated logical volume priorities.

[0028] Figure 5 It is a structural diagram of a resource scheduling device provided by an embodiment of the present invention.

[0029] Figure 6 It is a structural diagram of an electronic device provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0030] The following will clearly and completely describe the technical solutions in the embodiments of the present invention in conjunction with the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of them. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making any creative efforts shall fall within the scope of protection of the present invention.

[0031] The terms "including" and "having," as used in the present description and accompanying drawings, and any variations thereof, are intended to cover non-exclusive inclusions. For example, a process, method, system, product, or apparatus comprising a series of steps or elements is not limited to the listed steps or elements and may include steps or elements that are not listed.

[0032] In order to enable those skilled in the art to better understand the solutions of the present invention, the professional terms are explained below.

[0033] RAID (Redundant Array of Independent Disks) is a technology that combines multiple physical disks into a single logical unit to improve data storage reliability, performance, or capacity. It achieves data protection and performance improvements through redundancy, striping, mirroring, or parity. Common levels include RAID 0 / 1 / 5 / 6 / 10.

[0034] A RAID controller card is a dedicated hardware device used to manage RAID (Redundant Array of Independent Disks). It acts as a bridge between the operating system and physical disks through hardware or firmware, enabling unified management of multiple disks. A RAID controller card manages disk array creation, data distribution, redundancy checking, performance optimization, and fault recovery, making it a crucial component of enterprise-class storage solutions.

[0035] Rebuild: In a RAID array, after a disk fails and is replaced, the process of restoring the data on the failed disk to the new disk using existing redundant data (such as mirror or parity information).

[0036] Drive Group (DG): Multiple hard drives form a drive group, and different RAID levels can be applied to the drive group.

[0037] Logical volume (Virtual Drive, VD), a partition of DG, VD can be part of DG, the entire DG, or span multiple DGs.

[0038] A hot spare refers to a pre-configured but inactive backup drive in a RAID system. If one or more hard drives in the array fail, the hot spare drive automatically takes over the failed drive's functions, ensuring data availability and system operation.

[0039] Common RAID DDF (Disk Data Format): This specification defines a standard data structure that describes how to format data between disks in a RAID group, supporting in-place data migration between systems from different vendors.

[0040] Generally, the reconstruction process consumes a large amount of system resources (such as CPU, memory, and I / O bandwidth) and may have a significant impact on ongoing business I / O operations, resulting in a decrease in system performance. In order to balance the resource competition between the reconstruction process and business I / O, RAID systems usually support setting the Rebuild Rate configuration option. By adjusting the reconstruction priority, users can control the amount of resources used during the reconstruction process, thereby making a trade-off between reconstruction speed and foreground business performance. For example, setting a higher reconstruction priority can speed up the reconstruction speed, but may significantly affect business I / O performance; setting a lower reconstruction priority can reduce the impact on business I / O, but will extend the reconstruction time.

[0041] Existing RAID systems typically allow users to set a global reconstruction ratio for the entire RAID array. This means that when a drive in the array fails, the reconstruction process for all logical volumes follows the same setting. This global control fails to address the varying reconstruction priorities of different logical volumes, limiting the flexibility and efficiency of RAID systems when addressing complex business needs.

[0042] The present invention proposes a RAID reconstruction resource scheduling method based on differentiated logical volume priorities, which allows users to assign different reconstruction priorities to logical volumes in the same RAID array, so that critical logical volumes can be reconstructed first, while non-critical logical volumes can be reconstructed later or with lower resource usage, thereby improving the flexibility and adaptability of the system.

[0043] In order to enable those skilled in the art to better understand the present invention, the present invention will be further described in detail below with reference to the accompanying drawings and specific implementation methods.

[0044] Next, an embodiment of the present invention provides a resource scheduling method, such as Figure 1 As shown, the method provided in an embodiment of the present invention can be executed by an electronic device, which is a server. The server can be an independent physical server, a server cluster or distributed system composed of multiple physical servers, or a cloud server providing cloud computing services. The resource scheduling method includes: S101, when a redundant array of independent disks is reconfigured, obtaining a global reconstruction resource ratio corresponding to a reconstruction resource pool, and obtaining an actually used foreground resource ratio corresponding to a service resource pool.

[0045] The present invention divides the system resources into three areas based on the overall system resources: reconstruction resource pool, elastic resource pool, and business resource pool for front-end business. Figure 2The global reconstruction resource ratio refers to the ratio of the resources used for reconstruction operations to the total available resources in the entire redundant array of independent disks system. It is the global reconstruction ratio set by the user ( ), is the minimum resource used by the system for reconstruction (static guarantee): The business resource pool refers to the collection of resources allocated and managed to meet the needs of business I / O operations. The actual used front-end resource ratio refers to the proportion of resources currently used by front-end businesses in the business resource pool to the total resources in the business resource pool. It is used to indicate the actual resource usage of front-end businesses.

[0046] Global reconstruction ratio (rebuild_rate, denoted as , ranging from 0 to 100%, is the minimum reconstruction resource ratio. This determines the minimum amount of system resources allocated for reconstruction tasks, ensuring that regardless of business load, the system allocates at least a certain percentage of resources for reconstruction. A larger value indicates a higher overall reconstruction resource utilization ratio. In particular, when foreground business is busy, a global reconstruction ratio of 0 indicates that foreground business I / O is prioritized, and reconstruction tasks do not use system resources.

[0047] In one achievable method, obtaining the actual used foreground resource ratio corresponding to the business resource pool includes: obtaining the bandwidth utilization, cache occupancy, and computing power utilization of the foreground business according to a first cycle; and performing a weighted calculation based on the bandwidth utilization, cache occupancy, and computing power utilization of the foreground business to obtain the actual used foreground resource ratio corresponding to the business resource pool. Obtaining the bandwidth utilization, cache occupancy, and computing power utilization of the foreground business according to the first cycle and performing a weighted calculation to obtain the actual used foreground resource ratio corresponding to the business resource pool comprehensively considers the impact of multiple factors on resource occupancy, making the obtained foreground resource ratio more accurate.

[0048] The first period is a pre-set fixed time interval used to periodically acquire data, such as every 5 minutes or every 10 minutes. Bandwidth utilization refers to the ratio of the network bandwidth actually used during the operation of the foreground service to the total network bandwidth available in the system. It indicates the degree to which network bandwidth resources are utilized by the foreground service. For example, if the total system bandwidth is 100 Mbps and the foreground service actually uses 50 Mbps, the bandwidth utilization is 50%. Cache occupancy refers to the ratio of the cache space occupied by the foreground service during operation to the total cache space allocated for that service by the system. It indicates the proportion of cache resources used by the foreground service. For example, if the total cache is 1 GB and 500 MB is occupied, the cache occupancy is 50%. Computing power utilization refers to the ratio of the computing resources consumed by the foreground service during data processing to the total computing resources available in the system. It indicates the degree to which computing resources are utilized by the foreground service. For example, if the total system computing power is 1000 GFLOPS and the foreground service uses 300 GFLOPS, the computing power utilization is 30%. The actual front-end resource ratio refers to the ratio value obtained by combining the bandwidth utilization rate, cache occupancy rate, and computing power utilization rate indicators to fully reflect the actual utilization of resources in the business resource pool by the front-end business.

[0049] By obtaining data on multiple indicators such as bandwidth utilization, cache occupancy, computing power utilization, and performing weighted calculations, we can comprehensively consider the impact of different resources on front-end business and obtain a ratio value that can truly reflect the utilization status of front-end business resources.

[0050] The ratio of front-end resources actually used by the business Calculate by real-time monitoring of front-end business resource usage: , where BW is IO bandwidth, Cache is cache, and CPU is computing power. 、 、 They are expressed as the percentage of currently occupied resources to total resources, representing bandwidth utilization, cache utilization, and computing power utilization respectively. It is a configurable parameter, typically 0.4, 0.4, or 0.2.

[0051] S102: Determine the available reconstruction resource ratio corresponding to the reconstruction resource pool according to the global reconstruction resource ratio and the foreground resource ratio.

[0052] Among them, the available reconstruction resource ratio corresponding to the reconstruction resource pool It is the actual available ratio during the reconstruction process, which is not less than the global reconstruction resource ratio.

[0053] S103: Obtain resource allocation weights of each logical volume of the redundant array of independent disks.

[0054] In a RAID card, multiple physical disks can form a hard disk group, and the hard disk group uses a certain RAID level. In order to meet the user's more flexible usage requirements, such as Figure 3 As shown, a disk group can be divided into multiple or a single logical volume. Users can use different logical volumes for different purposes and store different data. Different logical volumes have different resource allocation weights W. These weights can be user-specified or calculated through an algorithm, and are not limited in this embodiment.

[0055] S104: Determine the resource allocation ratio of each logical volume according to the resource allocation weight of each logical volume and the available reconstruction resource ratio.

[0056] S105: Allocate resources according to the resource allocation ratio of each logical volume.

[0057] Logical volume resource allocation basis , where the resource allocation weight of logical volume i is , the resource allocation ratio of logical volume i is ; and then based on Allocate resources to each logical volume.

[0058] It can be seen that in the embodiment of the present invention, when the independent disk redundant array is reconstructed, the global reconstruction resource ratio corresponding to the reconstruction resource pool and the actually used foreground resource ratio corresponding to the business resource pool are first obtained. The global reconstruction resource ratio reflects the minimum resources that can be invested in the reconstruction process, and the foreground resource ratio reflects the resources currently actually occupied by the business; based on these two ratios, the available reconstruction resource ratio corresponding to the reconstruction resource pool is determined, so that during the reconstruction process, the reconstruction resources can be reasonably allocated according to the actual business resource usage to avoid idle resources, thereby ensuring the balance between the reconstruction process and business operation, and the available reconstruction resource ratio is allocated based on the resource allocation weight of each logical volume, so that during the reconstruction process, the critical logical volumes are reconstructed first than the non-critical logical volumes, and resources can be reasonably allocated according to the actual business situation and the importance of the logical volumes, so that the reconstruction process is effective and the flexibility of the system is provided.

[0059] In one possible implementation of an embodiment of the present invention, S102 determines the available reconstruction resource ratio corresponding to the reconstruction resource pool based on the global reconstruction resource ratio and the foreground resource ratio, including: determining an elasticity coefficient and an elastic resource ratio corresponding to the elastic resource pool based on the global reconstruction resource ratio and the foreground resource ratio; and determining the available reconstruction resource ratio corresponding to the reconstruction resource pool based on the global reconstruction resource ratio, the elastic resource ratio, and the elasticity coefficient. The elastic resource ratio can be dynamically adjusted based on the global reconstruction resource ratio, the elastic resource ratio, and the elasticity coefficient, making the determination of the available reconstruction resource ratio more flexible and accurate.

[0060] Among them, the elasticity coefficient and the elastic resource ratio corresponding to the elastic resource pool are determined according to the global reconstruction resource ratio and the foreground resource ratio, including: determining the elastic resource ratio corresponding to the elastic resource pool according to the global reconstruction resource ratio and the foreground resource ratio; determining the maximum resource upper limit ratio corresponding to the business resource pool according to the global reconstruction resource ratio; and determining the elasticity coefficient according to the maximum resource upper limit ratio and the foreground resource ratio.

[0061] Specifically, the front-end business resource pool is used to process business data. The maximum resource limit available for foreground services, namely: ; The elastic resource pool is the unused part of the business resource pool and other idle resources, which can be dynamically allocated to the reconstruction task. The elastic resource ratio The calculation method is: Update the elasticity coefficient based on the front-end resource ratio of the real-time load , thus affecting the logical volume reconstruction resource allocation: , is the elasticity coefficient. Available reconstruction resource ratio ( ), which indicates the total resources that the RAID card can allocate to the reconstruction task at a specific moment, equal to the reconstruction resource pool and the preemptible elastic resources, where .

[0062] One possible implementation of the present invention limits the maximum resource ratio of the foreground resource pool corresponding to the foreground service to a maximum resource upper limit ratio. This clarifies the upper limit of service resource usage, prevents the service from occupying system resources without limit during operation, and thus reserves the necessary resource space for reconfiguring the resource pool.

[0063] A possible implementation of an embodiment of the present invention is to obtain resource allocation weights for each logical volume of a redundant array of independent disks, including: obtaining priorities of each logical volume of the redundant array of independent disks; calculating a total priority based on the priorities of each logical volume; and determining the resource allocation weight of each logical volume based on the priority corresponding to the logical volume and the total priority.

[0064] Among them, the priority of logical volume reconstruction (vd_rebuild_priority, recorded as , which can be a positive integer between 0 and 100, represents the priority of each logical volume. It supports manual configuration or dynamic calculation. The larger the value, the higher the priority.

[0065] The amount of resources allocated to each logical volume during the reconstruction process is dynamically adjusted based on the priority of the logical volume. Logical volumes with higher priorities will receive more resources to speed up the reconstruction process; logical volumes with lower priorities will receive fewer resources to reduce the impact on business I / O. The resource allocation weights of logical volumes are: .

[0066] By determining resource allocation weights based on priority and total priority, it is possible to ensure that logical volumes with high importance and urgent resource needs obtain more resources, thus achieving differentiated resource allocation.

[0067] A possible implementation of an embodiment of the present invention obtains the priority of each logical volume of an independent disk redundant array, including: obtaining volume information corresponding to each logical volume, the volume information including: data importance, delay sensitivity, and data access popularity; for each logical volume, determining a priority score of the logical volume based on the data importance, delay sensitivity, and data access popularity of the logical volume; and determining the priority of the logical volume based on a preset score range within which the priority score falls.

[0068] Data importance refers to the importance of data to business operations, system stability, and data security. Users can customize the data importance of each logical volume, with a value ranging from 0 to 100. Higher values indicate more important data in the logical volume. Latency sensitivity refers to the sensitivity of access to logical volume data to response time. For example, transaction data requires fast responses and is very sensitive to latency. Excessive access latency may impact the normal operation of transactions. Meanwhile, offline data used for data analysis has relatively low latency requirements. Users can customize the latency sensitivity of each logical volume, with a value ranging from 0 to 100. Higher values indicate greater latency sensitivity. Data access popularity refers to how frequently data is accessed over a period of time. Frequently accessed data has higher access popularity. Data access popularity ranges from 0 to 100, with higher values indicating higher data popularity. A numerical value is calculated based on a comprehensive consideration of the three factors of a logical volume's data importance, latency sensitivity, and data access popularity. This value quantifies the priority of the logical volume, with higher scores indicating higher priority. The preset score range is a series of pre-defined score intervals, each corresponding to a priority level. For example, 0-60 points is set as the third priority range, 61-150 points is set as the second priority range, and 151-300 points is set as the first priority range. Priority refers to the order in which logical volumes are ranked in terms of resource allocation and data processing. Logical volumes with higher priorities receive more resources.

[0069] As can be seen, in the embodiment of the present invention, differentiated resource allocation and processing can be performed on logical volumes according to their priorities. By comprehensively considering the three key factors of logical volume data importance, delay sensitivity, and data access popularity, the importance of logical volumes in the system can be more accurately reflected.

[0070] In one possible implementation of an embodiment of the present invention, a first custom attribute is provided in the configuration record of a logical volume to store the logical volume priority; and a second custom attribute is provided in the configuration record of a redundant array of independent disks controller to store the global reconstruction resource ratio. In this embodiment of the present invention, the reconstruction priority attribute of the logical volume needs to be saved. The DDF protocol specifies a format for saving RAID card attributes, and the reconstruction priority attribute can be added to the logical volume attributes of the DDF protocol.

[0071] In the Virtual Disk Configuration Record structure specified by the DDF protocol, a vendor-specific scratch space is reserved for setting the first custom attribute, as shown in Table 1.

[0072] Table 1 DDF protocol table for logical volumes

[0073]

[0074] Use the manufacturer-defined space and add the logical volume priority attribute, which is the first customized attribute, to it, as shown in Table 2.

[0075] Table 2: Contents of the vendor-specific space for the DDF protocol for a logical volume

[0076]

[0077] The controller attributes in the DDF protocol are shown in Table 3.

[0078] Table 3 Controller DDF protocol table

[0079]

[0080] The controller's global reconstruction ratio attribute, i.e., the second custom attribute, is added to Vendor_Unique_Controller_Data, as shown in Table 4.

[0081] Table 4 Table of contents of the vendor-specific space of the controller's DDF protocol

[0082]

[0083] A possible implementation method of an embodiment of the present invention also includes: obtaining user demand information; determining the global reconstruction resource ratio corresponding to the user demand information based on the user demand information and the mapping relationship; the mapping relationship is a mapping relationship between the demand information and the global reconstruction resource ratio.

[0084] User demand information represents the specific requirements a user places on system resources. The mapping relationship may include: if business priority is given, the global reconstruction resource ratio is 0; if reconstruction priority is given, the global reconstruction resource ratio is 100% or a first preset value; if conventional settings are given, the global reconstruction resource ratio is a second preset value, which is less than the first preset value.

[0085] It can be seen that in the embodiment of the present invention, by establishing a mapping relationship between user demand information and the global reconstruction resource ratio, the system can respond to the needs of different users more intelligently, thereby improving the adaptability and flexibility of the system.

[0086] A possible implementation of an embodiment of the present invention further includes: during the reconstruction of the redundant array of independent disks, obtaining the current business load and the load change rate according to the second cycle; if the current business load is greater than a preset load threshold and the load change rate is greater than the preset load change rate, reducing the available reconstruction resource ratio and adjusting the resource allocation weights of each logical volume.

[0087] The second period, the preset load threshold, and the preset load change rate can be customized and are not limited in the present embodiment. The current service load includes, but is not limited to, one or more of the following: I / O throughput (e.g., MB / s), IOPS, queue depth, CPU utilization, and memory usage; the load change rate is the percentage of load increase per unit time.

[0088] When the current business load is greater than the preset load threshold and the load change rate is greater than the preset load change rate, the global reconstruction resource ratio can be reduced. The reduction method can be that the available reconstruction resource ratio is reduced by 10% for every 10% that exceeds the preset load threshold. At the same time, it is necessary to ensure that the available reconstruction resource ratio is at least the global reconstruction resource ratio.

[0089] During the redundant array of independent disks (RAID) reconstruction process, the current service load and load change rate are obtained in the second cycle. When the current service load exceeds the preset load threshold and the load change rate exceeds the preset load change rate, the available reconstruction resource ratio is reduced and the resource allocation weights of each logical volume are adjusted. This allows for timely resource allocation adjustments when the service load is excessively high and fluctuates drastically, ensuring stable service operation.

[0090] In one achievable manner, the current service load is the current actual service load, or a service load predicted based on historical service loads. The current service load is estimated by training a prediction model with historical load data and estimating future load trends.

[0091] A possible implementation of an embodiment of the present invention is to adjust the resource allocation weights of each logical volume, including: increasing the resource allocation weight of the first logical volume according to a first ratio to obtain an initial first resource allocation weight; the first logical volume is a logical volume whose resource allocation weight is greater than the first allocation weight threshold; increasing the resource allocation weight of the second logical volume according to a second ratio to obtain an initial second resource allocation weight; the second logical volume is a logical volume whose resource allocation weight is greater than the second allocation weight threshold and less than the first allocation weight threshold; the first allocation weight threshold is greater than the second allocation weight threshold; the second ratio is less than the first ratio; using the resource allocation weight of the third logical volume as the corresponding initial third resource allocation weight; the third logical volume is a logical volume whose resource allocation weight is less than the second allocation weight threshold; and determining the adjusted resource allocation weights of each logical volume based on the initial first resource allocation weight, the initial second resource allocation weight, and the initial third resource allocation weight.

[0092] The adjustment rules are as follows: First logical volume (high priority): weight > first allocation weight threshold ( ), according to the first ratio ( ) Increase the weight. Second logical volume (medium priority): Second allocation weight threshold ( )<Weight≤ , according to the second ratio ( ) Increase weight ( ). The third logical volume (low priority): weight ≤ , keeping the weight unchanged. The ultimate goal is to generate adjusted resource allocation weights to ensure that high-priority volumes get more resources while avoiding low-priority volumes from being overly squeezed.

[0093] First logical volume: Initial first resource allocation weight = current weight × ( ); Example: Current weight = 90%, R1 = 20% → Adjusted = 90 × 1.2 = 108%. Second logical volume: Initial second resource allocation weight = current weight × ( ). The third logical volume: the initial third resource allocation weight = the current weight (remains unchanged).

[0094] Since increasing weights may cause the total to exceed the system resource limit (such as 100%), all weights need to be normalized. Calculate the total weight: , .

[0095] If the resource allocation weight of the first logical volume is greater than the first allocation weight threshold, the weight is increased according to the larger first ratio to ensure that important logical volumes obtain more resources; if the resource allocation weight of the second logical volume is between the two thresholds, the weight is increased according to the smaller second ratio to appropriately take care of logical volumes of medium importance; if the resource allocation weight of the third logical volume is less than the second allocation weight threshold, the weight remains unchanged; the differentiated adjustment method can reasonably allocate resources according to the importance of the logical volumes, thereby improving the rationality of resource allocation.

[0096] A possible implementation method of an embodiment of the present invention also includes: triggering a resource release process when a logical volume reconstruction is completed; determining the logical volumes whose reconstruction completion rate reaches a preset completion rate, keeping their allocated resources unchanged, and reducing the impact of resource fluctuations; redetermining the available reconstruction resources, and for the remaining logical volumes, redetermining the resource allocation and reconstruction resource weights based on priority, and reallocating resources. Specifically, when the system detects that the reconstruction operation of any logical volume is completed (through a status flag or event notification), the resource release process is immediately triggered; the current completion rate of all logical volumes is verified before triggering; the released resources are aggregated into the global resource pool; for the logical volumes whose completion rate reaches a preset threshold, their original resource allocation is kept unchanged to avoid performance fluctuations due to reallocation. After redetermining the resources that need to be increased for the remaining logical volumes, resources are gradually increased to avoid instantaneous performance fluctuations.

[0097] Based on any of the above embodiments, the present invention proposes a RAID reconstruction resource scheduling method based on differentiated logical volume priorities, allowing users to assign different reconstruction priorities to logical volumes in the same RAID array, so that critical logical volumes can be reconstructed first, while non-critical logical volumes can be reconstructed later or with lower resource usage, thereby improving the flexibility and adaptability of the system. Figure 4 , its specific implementation is as follows: Step 1, when the system is initialized, use the custom configuration command to set the reconstruction priority attributes, including the global reconstruction ratio and the priority attributes based on the logical volume. It can also be loaded by reading the DDF configuration information; Step 2, store the reconstruction priority attributes set in step 1 in a custom DDF structure, and save it to a persistent medium such as a disk; Step 3, continuously monitor the triggering of reconstruction events; Step 4, after the reconstruction event is triggered, initialize the resource pool and divide the system resources into: reconstruction resource pool, elastic resource pool, and foreground business resource pool. Among them, the reconstruction resource pool is the global reconstruction ratio set by the user, the foreground business resource pool is all other resources, and the elastic resource pool is 0; Step 5, real-time monitoring of the foreground business load, each time after Time calculation of bandwidth, cache, CPU resource utilization, weighted calculation of front-end business resource utilization The weights are typical values. Then update the elasticity coefficient; Step 6, calculate the total available reconstruction resources according to the above formula ; Step seven, calculate the logical volume reconstruction resources according to the logical volume priority weight; Step eight, update the resource allocation table, dynamically adjust resource allocation, according to Adjust the I / O scheduling weight and resource quota of each logical volume.

[0098] It can be seen that the present invention relates to a reconstruction method based on RAID card logical volumes. By independently setting reconstruction priority parameters for multiple logical volumes, the I / O bandwidth, cache, computing and other hardware resources occupied by the reconstruction task are dynamically allocated, thereby performing differentiated reconstruction operations and allowing high-priority logical volumes to complete data recovery first.

[0099] Through priority reconstruction based on the granularity of logical volumes, the differentiated needs of users are met and the flexibility and adaptability of the system are improved. When faced with complex business needs, users can more flexibly set priorities according to the business criticality of the logical volumes to ensure the rapid recovery of critical business data. In the RAID card, independent reconstruction priority parameters are configured for multiple logical volumes in the same RAID group, including parameter setting custom commands and metadata storage structures. Based on the reconstruction priority parameters of the logical volumes, the resources occupied by the reconstruction tasks are dynamically allocated, and the total reconstruction resources are allocated to each logical volume. Resource utilization is optimized to balance performance and efficiency. Flexible scheduling of system resources avoids the increased risk of data loss caused by reconstruction stalls during business peaks, and improves reconstruction resource utilization during business off-peak periods to avoid resource waste. The elastic resource pool design can cope with sudden business traffic and prevent reconstruction tasks from affecting core business.

[0100] The following is an introduction to a device provided by an embodiment of the present invention. The device described below and the method described above can be referenced to each other. The device of this embodiment is set in an electronic device. Figure 5 , Figure 5 This is a structural block diagram of an apparatus according to one embodiment of the present invention, comprising: a first acquisition module 210, for acquiring a global reconstruction resource ratio corresponding to a reconstruction resource pool and an actually used foreground resource ratio corresponding to a business resource pool when an independent redundant array of disks is reconstructed; a first determination module 220, for determining an available reconstruction resource ratio corresponding to the reconstruction resource pool based on the global reconstruction resource ratio and the foreground resource ratio; a second acquisition module 230, for acquiring a resource allocation weight of each logical volume of the independent redundant array of disks; a second determination module 240, for determining a resource allocation ratio of each logical volume based on the resource allocation weight and the available reconstruction resource ratio of each logical volume; and an allocation module 250, for allocating resources based on the resource allocation ratio of each logical volume.

[0101] In one implementable manner, the first determination module 220 is used to: determine the elasticity coefficient and the elastic resource ratio corresponding to the elastic resource pool based on the global reconstruction resource ratio and the foreground resource ratio; determine the available reconstruction resource ratio corresponding to the reconstruction resource pool based on the global reconstruction resource ratio, the elastic resource ratio and the elasticity coefficient.

[0102] In one feasible manner, the first determination module 220 is used to: determine the elastic resource ratio corresponding to the elastic resource pool based on the global reconstruction resource ratio and the foreground resource ratio; determine the maximum resource upper limit ratio corresponding to the business resource pool based on the global reconstruction resource ratio; and determine the elastic coefficient based on the maximum resource upper limit ratio and the foreground resource ratio.

[0103] In an implementable manner, the first determining module 220 is further configured to limit a maximum resource ratio of the foreground resource pool corresponding to the foreground service to a maximum resource upper limit ratio.

[0104] In one feasible manner, the first acquisition module 210 is used to: obtain the bandwidth utilization, cache occupancy, and computing power utilization of the foreground business according to the first period; perform weighted calculation based on the bandwidth utilization, cache occupancy, and computing power utilization of the foreground business to obtain the actual used foreground resource ratio corresponding to the business resource pool.

[0105] In one achievable manner, the second acquisition module 230 is used to: obtain the priority of each logical volume of the independent disk redundant array; calculate the total priority based on the priority of each logical volume; and determine the resource allocation weight of each logical volume based on the priority corresponding to the logical volume and the total priority.

[0106] In one achievable method, the second acquisition module 230 is used to: obtain volume information corresponding to each logical volume, the volume information including: data importance, delay sensitivity, and data access popularity; for each logical volume, determine the priority score of the logical volume based on the data importance, delay sensitivity, and data access popularity of the logical volume; and determine the priority of the logical volume based on a preset score range within which the priority score lies.

[0107] In one achievable manner, a first custom attribute is set in the configuration record of the logical volume to store the priority of the logical volume; a second custom attribute is set in the configuration record of the redundant array of independent disks controller to store the global reconstruction resource ratio.

[0108] In one implementable manner, the first acquisition module 210 is configured to: acquire user demand information; determine a global reconstruction resource ratio corresponding to the user demand information based on the user demand information and a mapping relationship; the mapping relationship is a mapping relationship between the demand information and the global reconstruction resource ratio.

[0109] In one feasible manner, it further includes: an adjustment module, which is used to obtain the current business load and load change rate according to the second cycle during the reconstruction process of the independent disk redundant array; if the current business load is greater than the preset load threshold and the load change rate is greater than the preset load change rate, then reduce the available reconstruction resource ratio and adjust the resource allocation weight of each logical volume.

[0110] In one achievable manner, the current service load is the current actual service load, or a service load predicted based on historical service loads.

[0111] In one implementable manner, the adjustment module is used to: increase the resource allocation weight of the first logical volume according to a first ratio to obtain an initial first resource allocation weight; the first logical volume is a logical volume whose resource allocation weight is greater than the first allocation weight threshold; increase the resource allocation weight of the second logical volume according to a second ratio to obtain an initial second resource allocation weight; the second logical volume is a logical volume whose resource allocation weight is greater than the second allocation weight threshold and less than the first allocation weight threshold; the first allocation weight threshold is greater than the second allocation weight threshold; the second ratio is less than the first ratio; use the resource allocation weight of the third logical volume as the corresponding initial third resource allocation weight; the third logical volume is a logical volume whose resource allocation weight is less than the second allocation weight threshold; and determine the adjusted resource allocation weights of each logical volume based on the initial first resource allocation weight, the initial second resource allocation weight, and the initial third resource allocation weight.

[0112] Figure 6 A structural diagram of an electronic device provided by an embodiment of the present invention, such as Figure 6 As shown, the electronic device includes: a memory 60 for storing computer programs; The processor 61 is configured to implement the steps of the method in the above embodiment when executing a computer program.

[0113] The processor 61 may include one or more processing cores, such as a quad-core processor or an octa-core processor. The processor 61 may be implemented using at least one of the following hardware forms: a digital signal processing (DSP), a field-programmable gate array (FPGA), or a programmable logic array (PLA). The processor 61 may also include a main processor and a coprocessor. The main processor is a processor for processing data in the awake state, also known as a central processing unit (CPU); the coprocessor is a low-power processor for processing data in the standby state. In some embodiments, the processor 61 may be integrated with a graphics processing unit (GPU), which is responsible for rendering and drawing content required to be displayed on the display screen. In some embodiments, the processor 61 may also include an artificial intelligence (AI) processor for handling computational operations related to machine learning.

[0114] The memory 60 may include one or more computer-readable storage media, which may be non-transitory. The memory 60 may also include a high-speed random access memory, and a non-volatile memory, such as one or more disk storage devices, flash memory storage devices. In this embodiment, the memory 60 is at least used to store the following computer program 601, wherein, after the computer program is loaded and executed by the processor 61, it can implement the relevant steps of the method disclosed in any of the aforementioned embodiments. In addition, the resources stored in the memory 60 may also include an operating system 602 and data 603, etc., and the storage method may be temporary storage or permanent storage. Among them, the operating system 602 may include Windows, Unix, Linux, etc.

[0115] In some embodiments, the electronic device may further include a display screen 62 , an input / output interface 63 , a communication interface 64 , a power supply 65 , and a communication bus 66 .

[0116] Those skilled in the art will appreciate that the structures shown in the figures do not limit the electronic device and may include more or fewer components than shown in the figures.

[0117] It is understood that if the methods in the above embodiments are implemented in the form of software functional units and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, or the portion that contributes to the current technology, or all or part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and executes all or part of the steps of the methods in each embodiment of the present invention. The aforementioned storage medium includes: a USB flash drive, a mobile hard drive, a read-only memory (ROM), a random access memory (RAM), an electrically erasable programmable ROM, a register, a hard drive, a removable disk, a CD-ROM, a magnetic disk, or an optical disk, and other media that can store program code.

[0118] Based on this, an embodiment of the present invention further provides a computer-readable storage medium, on which a computer program is stored. When the computer program is executed by a processor, the steps of the above method are implemented.

[0119] Based on this, an embodiment of the present invention further provides a computer program product, including a computer program / instruction, which implements the steps of the above method when executed by a processor.

[0120] The above describes in detail the resource scheduling method, product, device, and medium provided by the embodiments of the present invention. The various embodiments are described in a progressive manner, with each embodiment focusing on its differences from other embodiments. For similar or identical parts between the embodiments, reference can be made to the other embodiments. The devices disclosed in the embodiments are described briefly because they correspond to the methods disclosed in the embodiments. For relevant details, refer to the method description.

[0121] Professionals may further appreciate that the units and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, computer software, or a combination of the two. In order to clearly illustrate the interchangeability of hardware and software, the above description has generally described the components and steps of each example according to their functions. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professionals and technicians may use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of the present invention.

[0122] The above is a detailed introduction to the resource scheduling method, product, device and medium provided by the present invention. Specific examples are used herein to illustrate the principles and implementation methods of the present invention. The description of the above embodiments is only used to help understand the method of the present invention and its core ideas. It should be pointed out that for ordinary technicians in this technical field, without departing from the principles of the present invention, the present invention can also be improved and modified in a number of ways, and these improvements and modifications also fall within the scope of protection of the present invention.

Claims

1. A resource scheduling method, characterized in that: include: When a redundant array of independent disks is reconstructed, a global reconstruction resource ratio corresponding to the reconstruction resource pool is obtained, and an actually used foreground resource ratio corresponding to the business resource pool is obtained; Determining an available reconstruction resource ratio corresponding to a reconstruction resource pool according to the global reconstruction resource ratio and the foreground resource ratio; Obtaining resource allocation weights of each logical volume of the redundant array of independent disks; determining a resource allocation ratio of each logical volume according to the resource allocation weight of each logical volume and the available reconstruction resource ratio; Resources are allocated based on the resource allocation ratio of each logical volume.

2. The resource scheduling method according to claim 1, characterized in that: Determining an available reconstruction resource ratio corresponding to a reconstruction resource pool according to the global reconstruction resource ratio and the foreground resource ratio includes: Determining an elastic coefficient and an elastic resource ratio corresponding to an elastic resource pool according to the global reconstruction resource ratio and the foreground resource ratio; An available reconstruction resource ratio corresponding to a reconstruction resource pool is determined according to the global reconstruction resource ratio, the elastic resource ratio, and the elastic coefficient.

3. The resource scheduling method according to claim 2, characterized in that: Determining an elastic coefficient and an elastic resource ratio corresponding to an elastic resource pool according to the global reconstruction resource ratio and the foreground resource ratio includes: Determining an elastic resource ratio corresponding to an elastic resource pool according to the global reconstruction resource ratio and the foreground resource ratio; According to the global reconstruction resource ratio, a maximum resource upper limit ratio corresponding to the business resource pool is determined; and an elasticity coefficient is determined according to the maximum resource upper limit ratio and the foreground resource ratio.

4. The resource scheduling method according to claim 3, characterized in that: Also includes: The maximum resource ratio of the foreground resource pool corresponding to the foreground business is limited to the maximum resource upper limit ratio.

5. The resource scheduling method according to claim 1, characterized in that: Get the actual usage ratio of the front-end resources in the business resource pool, including: According to the first cycle, obtain the bandwidth utilization, cache occupancy, and computing power utilization of the front-end business; A weighted calculation is performed based on the bandwidth utilization, cache occupancy, and computing power utilization of the front-end business to obtain the actual used front-end resource ratio corresponding to the business resource pool.

6. The resource scheduling method according to claim 1, characterized in that: Obtaining resource allocation weights of each logical volume of the redundant array of independent disks includes: Obtaining the priority of each logical volume of the redundant array of independent disks; Calculate the total priority based on the priority of each logical volume; For each logical volume, a resource allocation weight of the logical volume is determined according to the priority corresponding to the logical volume and the total priority.

7. The resource scheduling method according to claim 6, characterized in that: Obtaining the priority of each logical volume of the redundant array of independent disks includes: Obtaining volume information corresponding to each logical volume, the volume information including: data importance, delay sensitivity, and data access popularity; For each logical volume, a priority score is determined based on the data importance, latency sensitivity, and data access popularity of the logical volume; and the priority of the logical volume is determined based on the preset score range in which the priority score falls.

8. The resource scheduling method according to claim 1, wherein: The configuration record of the logical volume is provided with a first custom attribute for storing the priority of the logical volume; the configuration record of the redundant array of independent disks controller is provided with a second custom attribute for storing the global reconstruction resource ratio.

9. The resource scheduling method according to claim 1, characterized in that: Also includes: Obtain user demand information; The global reconstruction resource ratio corresponding to the user demand information is determined according to the user demand information and the mapping relationship; the mapping relationship is a mapping relationship between the demand information and the global reconstruction resource ratio.

10. The resource scheduling method according to claim 9, characterized in that: Also includes: During the reconstruction of the redundant array of independent disks, the current service load and the load change rate are obtained according to the second cycle; If the current service load is greater than a preset load threshold and the load change rate is greater than a preset load change rate, the available reconstruction resource ratio is reduced, and the resource allocation weight of each logical volume is adjusted.

11. The resource scheduling method according to claim 10, characterized in that: The current business load is the current actual business load, or a business load predicted based on historical business loads.

12. The resource scheduling method according to claim 10, characterized in that: Adjust the resource allocation weight of each logical volume, including: Increasing the resource allocation weight of the first logical volume according to a first ratio to obtain an initial first resource allocation weight; the first logical volume is a logical volume whose resource allocation weight is greater than a first allocation weight threshold; Increasing the resource allocation weight of the second logical volume according to the second ratio to obtain an initial second resource allocation weight; the second logical volume is a logical volume whose resource allocation weight is greater than the second allocation weight threshold and less than the first allocation weight threshold; the first allocation weight threshold is greater than the second allocation weight threshold; and the second ratio is less than the first ratio; using the resource allocation weight of the third logical volume as the corresponding initial third resource allocation weight; the third logical volume being a logical volume whose resource allocation weight is less than the second allocation weight threshold; The adjusted resource allocation weights of the respective logical volumes are determined according to the initial first resource allocation weight, the initial second resource allocation weight, and the initial third resource allocation weight.

13. A computer program product comprising a computer program / instructions, characterized in that When the computer program / instruction is executed by a processor, the steps of the resource scheduling method according to any one of claims 1 to 12 are implemented.

14. An electronic device, characterized in that: include: Memory for storing computer programs; A processor, configured to execute the computer program to implement the steps of the resource scheduling method according to any one of claims 1 to 12.

15. A computer-readable storage medium, characterized in that The computer-readable storage medium stores a computer program, which, when executed by a processor, implements the steps of the resource scheduling method according to any one of claims 1 to 12.

Images