Data processing methods, terminals, and storage media for combined disk arrays

Abstract

This invention relates to the field of storage technology, specifically providing a data processing method, terminal, and storage medium for a combined disk array. The method includes: decomposing the combined disk array into a parent array and child arrays, wherein each child array is a basic disk array with corresponding hardware acceleration units; splitting the parent volume corresponding to the parent array into a corresponding number of child volumes based on the number of child arrays, and establishing a correspondence between child volumes and child arrays; and based on the correspondence, decomposing write commands received by the parent volume into sub-commands, and writing the sub-commands into the child arrays using the corresponding child volumes. This invention fully utilizes the hardware acceleration units of the basic disk array, ensuring the data processing efficiency of the combined disk array.

Description

Technical Field

[0001] This invention belongs to the field of storage technology, specifically relating to a data processing method, terminal, and storage medium for a combined disk array. Background Technology

[0002] Combined RAID arrays create large RAID arrays based on smaller base RAID arrays. For example, RAID50 combines RAID5 and RAID0. First, RAID5 is implemented on smaller groups of disks, each using distributed parity to provide data redundancy and protection in the event of a single disk failure. These individual RAID5 arrays are then combined to form a larger RAID0 stripe set, enabling data access across multiple disks.

[0003] Different physical disks form a RAID array. Logical volumes, also known as namespaces, are created on top of the RAID array. These logical volumes are storage space units mapped to the host. The host issues I / O in units of namespaces, which are first mapped to the RAID group through the namespace, then through the RAID group, and finally to the physical disk.

[0004] To ensure efficient transmission of host I / O data, existing technologies introduce a computationally accelerated RAID controller between the host and the disks, implemented by a hardware acceleration unit integrating multiple components. However, for disk arrays with combined RAID configurations such as RAID 10 / 50 / 60, implementing I / O processing entirely in hardware requires a complex hardware computing circuit, resulting in high costs. If implemented entirely in software, the processing involves numerous repetitive sub-units with low execution efficiency, making it difficult to guarantee I / O performance.

[0005] Therefore, designing a data processing mechanism for combined disk arrays that adapts to existing hardware resources is a technical problem that urgently needs to be solved. Summary of the Invention

[0006] In view of the above-mentioned shortcomings of the prior art, the present invention provides a data processing method, terminal and storage medium for combined disk arrays to solve the above-mentioned technical problems.

[0007] In a first aspect, the present invention provides a data processing method for a combined disk array, comprising:

[0008] The combined disk array is broken down into a parent array and a child array, wherein the child array is a basic disk array with a corresponding hardware acceleration unit;

[0009] Based on the number of subarrays, the parent volume corresponding to the parent array is split into the corresponding number of subvolumes, and the correspondence between subvolumes and subarrays is established.

[0010] Based on the aforementioned correspondence, the write command received by the parent volume is decomposed into sub-commands, and the sub-commands are written into the sub-array using the corresponding sub-volume.

[0011] In an optional implementation, the combined disk array is decomposed into a parent array and a child array, wherein the child array is a base disk array with an independent hardware acceleration unit, including:

[0012] The management command for creating a disk array is intercepted, and the basic information of the disk array to be created is parsed from the parameters of the management command. The basic information includes the disk array level, the number of member disks, and the information of the underlying disk array.

[0013] If the disk array level is a composite level, then the disk array to be created is marked as the parent array;

[0014] Based on the information of the underlying disk array contained in the parent array, the corresponding underlying disk array is marked as a child array of the parent array;

[0015] Execute the management command to create the disk array.

[0016] In an optional implementation, executing the management command to create the disk array includes:

[0017] Based on the information of the basic disk array contained in the parent array, the member disks that make up the sub-arrays are determined, and the list of member disks in the parameters of the management command is evenly partitioned according to the number of sub-arrays, and used as the member disks that make up each sub-array.

[0018] Configure the member disk information of the subarray to the hardware acceleration unit;

[0019] Create a global array for the array. The global array maintains the disk array information to be created and records the connection relationship between the parent array and the child array. The global array includes parent array information and child array information. The parent array information includes the parent array ID, level, member disk information, number of child arrays, and an array of pointers to the child arrays. The child array information includes the child array ID, level, member disk information, and pointers to the parent array.

[0020] The system will reply with a message indicating that the disk array was successfully created on the host.

[0021] In an optional implementation, the parent volume corresponding to the parent array is split into a corresponding number of sub-volumes based on the number of sub-arrays, and a correspondence between sub-volumes and sub-arrays is established, including:

[0022] Upon receiving a command to create a volume, the volume parameters are parsed from the command, including the parent array ID, volume size, and starting logical address;

[0023] Access the array global array containing the parent array using the parent array ID, and extract the number of child arrays from the array global array;

[0024] Set the number of subarrays to the number of subvolumes;

[0025] The size of each sub-volume and its starting logical address in the sub-array are calculated based on the volume size and starting logical address in the volume parameters and the number of sub-volumes.

[0026] Configure subvolume information to the hardware acceleration unit;

[0027] Create a volume global array to maintain parent and child volume information, and record the relationship between parent and child volumes through the volume global array; the volume global array includes parent volume information and child volume information, the parent volume information includes the parent volume ID and the ID of the disk array it belongs to, the detailed information of the disk array, the volume size, the starting logical address, the number of child volumes, and an array of pointers to child volumes, the child volume information includes the child volume ID and the ID of the subarray it belongs to, the detailed information of the subarray, the volume size, the starting logical address, and a pointer to the parent volume;

[0028] Reply with a message indicating that the host volume was created successfully, and report an asynchronous event indicating that the host volume attributes have been changed.

[0029] In an optional implementation, the size of each sub-volume and its starting logical address in the sub-array are calculated based on the volume size and starting logical address in the volume parameters and the number of sub-volumes, including:

[0030] The formula for calculating the size of a subvolume is:

[0031] child_nlb = nlb / child_ns_num;

[0032] Where nlb is the volume size and child_ns_num is the number of child volumes;

[0033] The formula for calculating the starting logical address of a subvolume is:

[0034] child_raid_slba=raid_slba / child_ns_num;

[0035] Where raid_slba is the starting logical address in the volume parameters, and child_ns_num is the number of child volumes.

[0036] In an optional implementation, based on the correspondence, the write command received by the parent volume is decomposed into sub-commands, and the sub-commands are written to the sub-array using the corresponding sub-volume, including:

[0037] Parse the parent volume ID, starting position, and length of the write command;

[0038] Retrieve the corresponding target volume global array based on the parent volume ID;

[0039] Based on the target volume global array, the starting position is converted into an actual address, and the write command is broken down into multiple sub-commands according to the length and the size of the subarray corresponding to each subvolume in the target volume global array;

[0040] Execute the subcommands one by one until all subcommands have been executed.

[0041] In an optional implementation, the starting position is converted into an actual address based on the target volume global array, and the write command is decomposed into multiple sub-commands according to the length and the size of the subarrays corresponding to each subvolume in the target volume global array, including:

[0042] The starting position of the write command relative to the actual address of the parent array is calculated based on the size of the parent volume, the starting logical address, and the actual address in the parent array in the target volume global array.

[0043] Access the parent array and obtain the stripe size of the child array based on the connection relationship between the parent and child arrays;

[0044] Based on the start position and length of the write command, as well as the stripe size of the subarray, it is determined whether the write command spans multiple subarrays. If the length is less than the stripe size of the subarray, the write command is executed directly. If the length is greater than the stripe size of the subarray, the write command is broken up.

[0045] The method for disassembling the write command includes:

[0046] The formula for calculating the subarray number is:

[0047] child_raid_index=slba / child_stripe_size;

[0048] Wherein, child_raid_index is the subarray number, slba is the starting position of the write command, and child_stripe_size is the stripe size of the subarray [3];

[0049] The subarray ID is obtained from the subarray number and the subarray information recorded in the parent array. The subvolume ID is obtained from the subarray ID and the parent volume ID.

[0050] Calculate the number of logical blocks in the subarray for each segmented subcommand:

[0051] child_nlb=child_stripe_size–slba%child_stripe_size[4];

[0052] Where child_stripe_size is the size of the subarray's stripes, and slba is the starting position;

[0053] Calculate the starting logical address of the split sub-command in the subarray:

[0054] child_slba=stripe_id*child_stripe_size+slba%child_stripe_size;

[0055] Where stripe_id represents the stripe number of the host I / O hit, and its calculation formula is:

[0056] stripe_id=slba / father_stripe_size;

[0057] Where father_stripe_size is the stripe size of the parent array;

[0058] Update the number of remaining logical blocks for the write command, confirm that the number of remaining logical blocks exceeds the number of logical blocks in the subarray, and use the smaller value between the number of remaining logical blocks and the stripe size of the subarray as the number of logical blocks for the subcommand split into the next subarray.

[0059] Update the starting position of the latest subcommand. The starting position of the latest subcommand is updated to the starting position of the write command plus the number of logical blocks of all subarrays that have been allocated the corresponding subcommand.

[0060] Update the subarray number of the subarray corresponding to the new subcommand to the previous subarray number plus 1;

[0061] Apply for the corresponding sub-command processing node and fill in the sub-volume ID, sub-array ID, starting position of the split sub-command, and number of logical blocks of the split sub-command;

[0062] Increment the number of commands in the parent command processing node by 1;

[0063] Repeat the splitting of the write command until the number of remaining logical blocks of the write command is 0.

[0064] In one optional implementation, subcommands are executed one by one until all subcommands have been executed, including:

[0065] Upon receiving an event indicating that a subcommand has been executed, extract the subcommand information and execution result from the event.

[0066] Obtain the processing node of the superior write command based on the sub-command information, and send the sub-command information and execution result to the processing node of the superior write command;

[0067] The processing node that writes the command summarizes the received sub-command information and execution results, updates the number of sub-commands to be executed, and reclaims the processing node corresponding to the completed sub-command.

[0068] If all subcommands execute successfully, the parent write command is considered to have executed successfully. If any subcommands fail to execute, the parent write command is considered to have failed.

[0069] In a second aspect, the present invention provides a data processing system for a combined disk array, comprising:

[0070] The array analysis module is used to decompose the combined disk array into a parent array and a child array, wherein the child array is a basic disk array with a corresponding hardware acceleration unit;

[0071] The volume analysis module is used to split the parent volume corresponding to the parent array into a corresponding number of sub-volumes based on the number of sub-arrays, and to establish the correspondence between sub-volumes and sub-arrays;

[0072] The command decomposition module is used to decompose the write command received by the parent volume into sub-commands based on the correspondence, and to write the sub-commands into the sub-array using the corresponding sub-volume.

[0073] Thirdly, a terminal is provided, including:

[0074] Processor, memory, among which,

[0075] This memory is used to store computer programs.

[0076] The processor is used to retrieve and run the computer program from memory, causing the terminal to perform the terminal method described above.

[0077] Fourthly, a computer storage medium is provided, wherein instructions are stored therein, which, when executed on a computer, cause the computer to perform the methods described in the above aspects.

[0078] The beneficial effects of the present invention are that the data processing method, terminal and storage medium of the combined disk array provided by the present invention, by decomposing the combined disk array into a structure of parent array and child array, and splitting the write command based on the decomposed architecture, and then executing the split write command step by step, make full use of the hardware acceleration unit of the basic disk array and ensure the data processing efficiency of the combined disk array.

[0079] Furthermore, the design principle of this invention is reliable, the structure is simple, and it has a very wide range of application prospects. Attached Figure Description

[0080] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, for those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0081] Figure 1 This is a schematic flowchart of a method according to an embodiment of the present invention.

[0082] Figure 2 This is a schematic diagram illustrating the architecture of a method according to an embodiment of the present invention.

[0083] Figure 3 This is a schematic diagram of the structure of a combined disk array according to an embodiment of the present invention.

[0084] Figure 4 This is a schematic diagram of the structure of a volume according to an embodiment of the present invention.

[0085] Figure 5 This is a schematic flowchart illustrating the hierarchical management of a method according to an embodiment of the present invention.

[0086] Figure 6 This is a schematic diagram of the data structure of a method according to an embodiment of the present invention.

[0087] Figure 7 This is a schematic flowchart illustrating the breakdown of the write command in an embodiment of the present invention.

[0088] Figure 8 This is a schematic flowchart illustrating the execution of subcommands in a method according to an embodiment of the present invention.

[0089] Figure 9 This is a schematic block diagram of a system according to an embodiment of the present invention.

[0090] Figure 10 This is a schematic diagram of the structure of a terminal provided in an embodiment of the present invention. Detailed Implementation

[0091] To enable those skilled in the art to better understand the technical solutions of this invention, the technical solutions of the embodiments of this invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this invention, and not all embodiments. Based on the embodiments of this invention, all other embodiments obtained by those skilled in the art without creative effort should fall within the scope of protection of this invention.

[0092] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this invention pertains. The terminology used herein in the description of the invention is for the purpose of describing particular embodiments only and is not intended to be limiting of the invention.

[0093] The key terms used in this invention will be explained below.

[0094] NVMe: Short for Non-Volatile Memory Express, it is a communication interface and driver standard designed specifically for solid-state drives (SSDs). It utilizes the PCI Express (PCIe) bus to provide high-speed data transfer and features high bandwidth and low latency.

[0095] RAID stands for Redundant Array of Independent Disks, a storage virtualization technology that combines multiple individual physical disk drives in different ways to form a single logical disk drive, thereby improving storage capacity, read / write performance, and data security.

[0096] Combined RAID: Also known as hybrid RAID or nested RAID, it is a method of combining two or more different RAID levels. Common combined RAID levels include RAID10, RAID50, and RAID60.

[0097] ns: namespace, an independent storage space, each namespace has its own logical block address range. From the host's perspective, a namespace is an independent storage device, corresponding to the concept of a logical volume in a disk array.

[0098] SLBA stands for Start Logic Block Address. It is the starting logical block address on the device side included in the I / O command. The device divides its address space into logical blocks. When the host submits an I / O command, it specifies the logical block address, which indicates which address in the device space to read or write data from.

[0099] nlb: The full name is num of logic blocks. It refers to the number of device-side logic blocks included in an I / O command. When submitting an I / O command, the host specifies the number of logic blocks in the command, indicating how many logic blocks of data to read or write from the slba address in the device space.

[0100] The data processing method for the combined disk array provided in this embodiment of the invention is executed by a computer device, and correspondingly, the data processing system for the combined disk array runs in the computer device.

[0101] Figure 1 This is a schematic flowchart illustrating a method according to an embodiment of the present invention. Wherein, Figure 1 The executing entity can be a combined disk array data processing system. Depending on different requirements, the order of the steps in this flowchart can be changed, and some can be omitted.

[0102] like Figure 1 As shown, the method includes:

[0103] Step 110: Decompose the combined disk array into a parent array and a child array, wherein the child array is a basic disk array with a corresponding hardware acceleration unit;

[0104] Step 120: Based on the number of subarrays, split the parent volume corresponding to the parent array into the corresponding number of subarrays, and establish the correspondence between subarrays and subarrays;

[0105] Step 130: Based on the correspondence, the write command received by the parent volume is decomposed into sub-commands, and the sub-commands are written into the sub-array using the corresponding sub-volume.

[0106] This invention addresses combined RAID by proposing the concepts of parent array, child array, parent volume, and child volume, and provides management methods for child arrays and child volumes.

[0107] First, software breaks down host write commands for volumes on a combined disk array into sub-commands for volumes on individual disk arrays, and then these sub-commands are processed by hardware, reusing existing hardware acceleration units for individual disk arrays to save costs. This invention provides a method for splitting write commands and a method for integrating the results of the split sub-commands.

[0108] To facilitate understanding of the present invention, the following description further illustrates the data processing method for combined disk arrays provided by the present invention, based on the principle of the data processing method for combined disk arrays and the data processing process of the combined disk arrays in the embodiments.

[0109] Specifically, the data processing methods for combined disk arrays include:

[0110] S1, Architecture Management.

[0111] A subarray is a single-disk array level, while a parent array is a combined disk array level. For example, RAID 50 consists of three RAID 5 arrays; the RAID 50 is called the parent array, and the three RAID 5 arrays are called the subarrays. Figure 2As shown, two volumes are created on the RAID50 array: parent ns_1 and parent ns_2. Parent ns_1 consists of three child volumes (child ns_1_1, child ns_1_2, and child ns_1_3), and parent ns_2 consists of three child volumes (child ns_2_1, child ns_2_2, and child ns_2_3), located in child RAID1, child RAID2, and child RAID3, respectively. The number of child arrays and the number of child volumes are exactly the same.

[0112] In one embodiment of the present invention, the following architecture management method is adopted:

[0113] The management command for creating a disk array is intercepted, and the basic information of the disk array to be created is parsed from the parameters of the management command. The basic information includes the disk array level, the number of member disks, and the information of the underlying disk arrays. If the disk array level is a composite level, the disk array to be created is marked as the parent array. Based on the information of the underlying disk arrays contained in the parent array, the corresponding underlying disk arrays are marked as child arrays of the parent array. The management command for creating the disk array is then executed.

[0114] Specifically, it includes the following steps:

[0115] First, intercept the management commands for creating the disk array from user input or system configuration.

[0116] Perform format validation on the command to ensure it meets the expected syntax and parameter requirements.

[0117] If the command is formatted incorrectly, an error message will be returned and you will be asked to re-enter or correct it.

[0118] Use a parser (such as a regular expression or command-line parsing library) to extract all the necessary parameters from the management commands.

[0119] Verify the validity of each parameter, such as checking whether the disk array level is a system-supported level (e.g., RAID0, RAID 1, RAID 5, RAID 10, JBOD, etc.), whether the number of member disks is reasonable (non-zero and does not exceed the system limit), and whether the base disk array information has been correctly specified (if the command includes such information).

[0120] Extract the disk array level, number of member disks, and base disk array information (if applicable) from the parsed parameters.

[0121] Further verification of this basic information is required, such as checking whether the specified base disk array exists and is in good condition, and whether the number of member disks meets the requirements of the selected array level.

[0122] If the disk array level is composite (such as some advanced RAID configurations, which may involve a combination of multiple subarrays), then perform the following steps:

[0123] The disk array to be created is marked as the parent array, and its unique identifier and related information are recorded in the system database or configuration file.

[0124] Check and confirm that all the underlying disk arrays required by the parent array have been correctly specified and that there are no conflicts or dependencies between them.

[0125] Based on the configuration information of the parent array, traverse all disk arrays that are specified as basic components.

[0126] These basic disk arrays are marked as child arrays of the parent array, and the parent-child relationship between them is established internally within the system.

[0127] Update the system database or configuration files to reflect these changes.

[0128] The disk array is actually created by calling the underlying disk management API or executing system commands.

[0129] Monitor the creation process to ensure all steps are completed successfully.

[0130] If any errors or exceptions are encountered during the creation process, stop the operation immediately, roll back all completed changes, and log the error for later analysis.

[0131] In one embodiment of the present invention, the disk array architecture is as follows: Figure 3 As shown, the process of executing the management command to create a disk array includes:

[0132] Based on the information of the base disk array contained in the parent array, the member disks that make up the sub-arrays are determined. The list of member disks in the parameters of the management command is evenly partitioned according to the number of sub-arrays, and these partitions are used as the member disks for each sub-array. The member disk information of the sub-arrays is configured to the hardware acceleration unit. An array global array is created, which maintains the information of the disk arrays to be created and records the connection relationship between the parent array and the sub-arrays. The array global array includes parent array information and sub-array information. The parent array information includes the parent array ID, level, member disk information, number of sub-arrays, and an array of pointers to the sub-arrays. The sub-array information includes the sub-array ID, level, member disk information, and pointers to the parent array. A message indicating successful disk array creation is sent to the host.

[0133] In another embodiment of the invention, please refer to Figure 4 The methods for managing volumes include:

[0134] Upon receiving a command to create a volume, the system parses volume parameters from the command, including the parent array ID, volume size, and starting logical address. It then accesses the array global array containing the parent array via the parent array ID and extracts the number of subarrays from the array global array. The number of subarrays is set as the number of subvolumes. Based on the volume size and starting logical address in the volume parameters, and the number of subvolumes, the system calculates the size of each subvolume and its starting logical address within the subarray. The subvolume information is configured to the hardware acceleration unit. A volume global array is created to maintain parent and subvolume information, recording the relationship between parent and subvolumes. The volume global array includes parent and subvolume information. The parent volume information includes the parent volume ID, the ID of the disk array it belongs to, detailed information about the disk array, volume size, starting logical address, number of subvolumes, and an array of pointers to subvolumes. The subvolume information includes the subvolume ID, the ID of the subarray it belongs to, detailed information about the subarray, volume size, starting logical address, and a pointer to the parent volume. Finally, the system replies to the host with a message indicating successful volume creation and reports an asynchronous event indicating a change in host volume attributes.

[0135] The calculation of the size of each sub-volume and its starting logical address in the sub-array based on the volume size and starting logical address in the volume parameters, and the number of sub-volumes, includes:

[0136] The formula for calculating the size of a subvolume is:

[0137] child_nlb = nlb / child_ns_num;

[0138] Where nlb is the volume size and child_ns_num is the number of child volumes;

[0139] The formula for calculating the starting logical address of a subvolume is:

[0140] child_raid_slba=raid_slba / child_ns_num;

[0141] Where raid_slba is the starting logical address in the volume parameters, and child_ns_num is the number of child volumes.

[0142] Please refer to Figure 5 In one embodiment of the architecture management of the present invention, the specific steps include:

[0143] Upon receiving the host management command "Create RAID", the system parses the information of the RAID disk array to be created from the command parameters, including the RAID level, the number of member disks, and the number of single RAID arrays.

[0144] Determine the sub-RAID level. If you are creating RAID 10, the sub-RAID level is RAID 1. If you are creating RAID 50, the sub-RAID level is RAID 5. If you are creating RAID 60, the sub-RAID level is RAID 6.

[0145] Determine the member disks that make up the sub-RAID. The list of member disks in the creation command parameters is evenly partitioned according to the number of single RAIDs, and used as the member disks to make up each sub-RAID.

[0146] Configure the sub-RAID information to the hardware acceleration unit;

[0147] Populate the global array that maintains RAID information in the software module, and record the connection relationship between parent and child RAIDs;

[0148] The host computer responded that the "Create RAID" command was executed successfully.

[0149] Parse configuration parameters from the command "Create ns on combined RAID", including RAID group number, ns size nlb, RAID group number, offset in RAID group, raid_slba, etc.

[0150] By accessing the global array that records RAID information through the RAID number, the number of child RAIDs in the RAID disk array corresponding to that number can be obtained, which is the number of child ns, child_ns_num;

[0151] Calculate the information for the child ns. The size of the parent ns and the starting offset of the parent ns in the RAID are nlb and raid_slba in the configuration command parameters (limited to stripe alignment). The formula for calculating the size of each child ns, child_nlb, is: child_nlb = nlb / child_ns_num;

[0152] The formula for calculating the starting offset of the child ns in the child RAID, child_raid_slba, is as follows:

[0153] child_raid_slba=raid_slba / child_ns_num;

[0154] Configure the subns information into the hardware acceleration unit;

[0155] Fill the global array that maintains ns information in the software module to record the connection relationship between parent and child ns;

[0156] The host responded that the "Create ns" command was executed successfully and reported an asynchronous event indicating that the host's ns attribute had changed.

[0157] After the above steps, from the host's perspective, the storage space obtained through the `identify` command is the parent ns located on the combined RAID, and the range of I / O that can be distributed depends on the size of the parent ns. From a hardware perspective, the form factor difference between combined RAID and a single RAID is transparent, which lays the foundation for combined RAID to reuse the hardware acceleration unit for processing I / O in a single RAID.

[0158] S2, Command Processing.

[0159] Please refer to Figure 6 The data structure lists fields related to IO splitting. In the parent IO node, slba and nlb represent the starting position and length of the original host IO, respectively, and io_cnt represents the number of IOs after splitting according to the child ns. The child IO node records information about the sliced ​​IOs, where slba, nlb, and io_result represent the starting position, length, and result of the split IO, respectively.

[0160] S201, Write command decomposition, please refer to Figure 7 The specific process is as follows:

[0161] a. Parse the host IO commands to obtain slba, nlb, and nsid information, request a parent IO node, and populate it;

[0162] b. Access the global array that records ns information through nsid, and calculate the offset of slba relative to the actual address of the RAID group in the original host command based on the size, logical address, and actual address of the created ns in the RAID group.

[0163] c. The NSID in the host's original command is the parent NSID, from which the parent RAID_ID is obtained. The parent RAID node is accessed, and the child RAID stripe size child_stripe_size is obtained according to the connection relationship between the parent RAID and the child RAID.

[0164] d. Determine whether this IO command spans multiple sub-RAIDs based on slba, nlb, and sub-RAID stripe size in the original command. If so, proceed to the next step. Otherwise, no IO splitting is needed, and the process ends.

[0165] e. Calculate the child RAID number, which is the initial value of child_raid_index. The formula is: child_raid_index = slba / child_stripe_size;

[0166] f. Obtain the child RAID_id from the child RAID number and the information recorded in the parent RAID node, and then obtain the child ns_id based on the child RAID_id and the parent ns_id;

[0167] g. Calculate nlb, which is split into the child RAID. The formula is: child_nlb = child_stripe_size – slba % child_stripe_size;

[0168] h. Calculate the stripe number (slba) of the split I / O in the child RAID. The formula is: child_slba = stripe_id * child_stripe_size + slba % child_stripe_size; where stripe_id represents the stripe number hit by the host I / O, and its calculation formula is: stripe_id = slba / father_stripe_size;

[0169] i. Update the remaining nlb by subtracting the nlb of all sub-RAIDs from the original nlb of the host RAID, and check if the value is 0. If it is 0, the process ends. Otherwise, compare the remaining nlb with the sub-RAID stripe size, and use the smaller value as the nlb in the next sub-RAID.

[0170] j. Update SLBA, adding the initial value to the nlb of all sub-RAIDs;

[0171] k. Update the child RAID number, increment child_raid_index by 1;

[0172] l. Request a child IO node and populate it with the child nsid, child raid_id, split slba, and split nlb;

[0173] m. Increment 1 in the parent IO node's io_cnt;

[0174] n. Repeat the above fm step until the remaining nlb is 0, indicating that the split is complete and the process ends.

[0175] S202, the I / O on the combined RAID is split into individual sub-RAIDs, and each split I / O is handled by the existing hardware acceleration unit of a single RAID. The specific sub-command execution flow is as follows: Figure 8 As shown, it includes:

[0176] a. After receiving an event indicating that a slice of I / O has been completed, the software module parses out the sub-I / O and obtains the execution result of the slice of I / O.

[0177] b. Obtain the parent IO node based on the parent-child connection relationship, and integrate io_result into the parent IO node, using the following formula:

[0178] father_result=child1_result|child2_result|…|childN_result;

[0179] c. Decrement the io_cnt register of the number of slice IOs by 1 each time and reclaim the child IO nodes;

[0180] d. Repeat steps ac above until io_cnt in the parent IO node is reduced to 0, which means that all slice IOs have been executed.

[0181] e. Reply to the host IO command execution result. If io_result is 0 in all child IO nodes, it means that all split child IO requests have been completed, and the host IO command execution was successful. If io_result is not 0 in any child IO node, the host IO command execution failed.

[0182] f. Reclaim the resources of the parent IO node.

[0183] This method leverages the high performance of hardware processing while maintaining the flexibility of software, balancing the system performance and cost-effectiveness of the RAID controller. For the complex I / O of RAID 10 / 50 / 60, the hardware does not need to implement additional digital circuitry for combining RAID, allowing the reuse of existing single RAID controllers, thus improving hardware resource utilization, reducing hardware implementation complexity, and saving hardware design costs.

[0184] In some embodiments, the combined disk array data processing system may include multiple functional modules composed of computer program segments. The computer programs for each program segment in the combined disk array data processing system may be stored in the memory of a computer device and executed by at least one processor to perform (see details). Figure 1 (Description) The data processing functions of the combined disk array.

[0185] In this embodiment, the data processing system of the combined disk array can be divided into multiple functional modules according to the functions it performs, such as... Figure 9 As shown. The functional modules of system 900 may include: array analysis module 910, volume analysis module 920, and command decomposition module 930. The module referred to in this invention is a series of computer program segments that can be executed by at least one processor and perform a fixed function, and which are stored in memory. In this embodiment, the functions of each module will be described in detail in subsequent embodiments.

[0186] The array analysis module is used to decompose the combined disk array into a parent array and a child array, wherein the child array is a basic disk array with a corresponding hardware acceleration unit;

[0187] The volume analysis module is used to split the parent volume corresponding to the parent array into a corresponding number of sub-volumes based on the number of sub-arrays, and to establish the correspondence between sub-volumes and sub-arrays;

[0188] The command decomposition module is used to decompose the write command received by the parent volume into sub-commands based on the correspondence, and to write the sub-commands into the sub-array using the corresponding sub-volume.

[0189] Optionally, as an embodiment of the present invention, the combined disk array is decomposed into a parent array and a child array, wherein the child array is a basic disk array with an independent hardware acceleration unit, including:

[0190] The management command for creating a disk array is intercepted, and the basic information of the disk array to be created is parsed from the parameters of the management command. The basic information includes the disk array level, the number of member disks, and the information of the underlying disk array.

[0191] If the disk array level is a composite level, then the disk array to be created is marked as the parent array;

[0192] Based on the information of the underlying disk array contained in the parent array, the corresponding underlying disk array is marked as a child array of the parent array;

[0193] Execute the management command to create the disk array.

[0194] Optionally, as an embodiment of the present invention, executing the management command for creating the disk array includes:

[0195] Based on the information of the basic disk array contained in the parent array, the member disks that make up the sub-arrays are determined, and the list of member disks in the parameters of the management command is evenly partitioned according to the number of sub-arrays, and used as the member disks that make up each sub-array.

[0196] Configure the member disk information of the subarray to the hardware acceleration unit;

[0197] Create a global array for the array. The global array maintains the disk array information to be created and records the connection relationship between the parent array and the child array. The global array includes parent array information and child array information. The parent array information includes the parent array ID, level, member disk information, number of child arrays, and an array of pointers to the child arrays. The child array information includes the child array ID, level, member disk information, and pointers to the parent array.

[0198] The system will reply with a message indicating that the disk array was successfully created on the host.

[0199] Optionally, as an embodiment of the present invention, the parent volume corresponding to the parent array is split into a corresponding number of sub-volumes based on the number of sub-arrays, and a correspondence between sub-volumes and sub-arrays is established, including:

[0200] Upon receiving a command to create a volume, the volume parameters are parsed from the command, including the parent array ID, volume size, and starting logical address;

[0201] Access the array global array containing the parent array using the parent array ID, and extract the number of child arrays from the array global array;

[0202] Set the number of subarrays to the number of subvolumes;

[0203] The size of each sub-volume and its starting logical address in the sub-array are calculated based on the volume size and starting logical address in the volume parameters and the number of sub-volumes.

[0204] Configure subvolume information to the hardware acceleration unit;

[0205] Create a volume global array to maintain parent and child volume information, and record the relationship between parent and child volumes through the volume global array; the volume global array includes parent volume information and child volume information, the parent volume information includes the parent volume ID and the ID of the disk array it belongs to, the detailed information of the disk array, the volume size, the starting logical address, the number of child volumes, and an array of pointers to child volumes, the child volume information includes the child volume ID and the ID of the subarray it belongs to, the detailed information of the subarray, the volume size, the starting logical address, and a pointer to the parent volume;

[0206] Reply with a message indicating that the host volume was created successfully, and report an asynchronous event indicating that the host volume attributes have been changed.

[0207] Optionally, as an embodiment of the present invention, calculating the size of each sub-volume and its starting logical address in the sub-array based on the volume size and starting logical address in the volume parameters and the number of sub-volumes includes:

[0208] The formula for calculating the size of a subvolume is:

[0209] child_nlb = nlb / child_ns_num;

[0210] Where nlb is the volume size and child_ns_num is the number of child volumes;

[0211] The formula for calculating the starting logical address of a subvolume is:

[0212] child_raid_slba=raid_slba / child_ns_num;

[0213] Where raid_slba is the starting logical address in the volume parameters, and child_ns_num is the number of child volumes.

[0214] Optionally, as an embodiment of the present invention, based on the correspondence, the write command received by the parent volume is decomposed into sub-commands, and the sub-commands are written to the sub-array using the corresponding sub-volume, including:

[0215] Parse the parent volume ID, starting position, and length of the write command;

[0216] Retrieve the corresponding target volume global array based on the parent volume ID;

[0217] Based on the target volume global array, the starting position is converted into an actual address, and the write command is broken down into multiple sub-commands according to the length and the size of the subarray corresponding to each subvolume in the target volume global array;

[0218] Execute the subcommands one by one until all subcommands have been executed.

[0219] Optionally, as an embodiment of the present invention, the starting position is converted into an actual address based on the target volume global array, and the write command is decomposed into multiple sub-commands according to the length and the size of the subarrays corresponding to each sub-volume in the target volume global array, including:

[0220] The starting position of the write command relative to the actual address of the parent array is calculated based on the size of the parent volume, the starting logical address, and the actual address in the parent array in the target volume global array.

[0221] Access the parent array and obtain the stripe size of the child array based on the connection relationship between the parent and child arrays;

[0222] Based on the start position and length of the write command, as well as the stripe size of the subarray, it is determined whether the write command spans multiple subarrays. If the length is less than the stripe size of the subarray, the write command is executed directly. If the length is greater than the stripe size of the subarray, the write command is broken up.

[0223] The method for disassembling the write command includes:

[0224] The formula for calculating the subarray number is:

[0225] child_raid_index=slba / child_stripe_size;

[0226] Wherein, child_raid_index is the subarray number, slba is the starting position of the write command, and child_stripe_size is the stripe size of the subarray [5];

[0227] The subarray ID is obtained from the subarray number and the subarray information recorded in the parent array. The subvolume ID is obtained from the subarray ID and the parent volume ID.

[0228] Calculate the number of logical blocks in the subarray for each segmented subcommand:

[0229] child_nlb=child_stripe_size–slba%child_stripe_size[6];

[0230] Where child_stripe_size is the size of the subarray's stripes, and slba is the starting position;

[0231] Calculate the starting logical address of the split sub-command in the subarray:

[0232] child_slba=stripe_id*child_stripe_size+slba%child_stripe_size;

[0233] Where stripe_id represents the stripe number of the host I / O hit, and its calculation formula is:

[0234] stripe_id=slba / father_stripe_size;

[0235] Where father_stripe_size is the stripe size of the parent array;

[0236] Update the number of remaining logical blocks for the write command, confirm that the number of remaining logical blocks exceeds the number of logical blocks in the subarray, and use the smaller value between the number of remaining logical blocks and the stripe size of the subarray as the number of logical blocks for the subcommand split into the next subarray.

[0237] Update the starting position of the latest subcommand. The starting position of the latest subcommand is updated to the starting position of the write command plus the number of logical blocks of all subarrays that have been allocated the corresponding subcommand.

[0238] Update the subarray number of the subarray corresponding to the new subcommand to the previous subarray number plus 1;

[0239] Apply for the corresponding sub-command processing node and fill in the sub-volume ID, sub-array ID, starting position of the split sub-command, and number of logical blocks of the split sub-command;

[0240] Increment the number of commands in the parent command processing node by 1;

[0241] Repeat the splitting of the write command until the number of remaining logical blocks of the write command is 0.

[0242] Optionally, as an embodiment of the present invention, sub-commands are executed one by one until all sub-commands are executed, including:

[0243] Upon receiving an event indicating that a subcommand has been executed, extract the subcommand information and execution result from the event.

[0244] Obtain the processing node of the superior write command based on the sub-command information, and send the sub-command information and execution result to the processing node of the superior write command;

[0245] The processing node that writes the command summarizes the received sub-command information and execution results, updates the number of sub-commands to be executed, and reclaims the processing node corresponding to the completed sub-command.

[0246] If all subcommands execute successfully, the parent write command is considered to have executed successfully. If any subcommands fail to execute, the parent write command is considered to have failed.

[0247] Figure 10 This is a schematic diagram of the structure of a terminal 1000 provided in an embodiment of the present invention. The terminal 1000 can be used to execute the data processing method of the combined disk array provided in the embodiment of the present invention.

[0248] The terminal 1000 may include a processor 1010, a memory 1020, and a communication unit 1030. These components communicate via one or more buses. Those skilled in the art will understand that the server structure shown in the figure does not constitute a limitation of the present invention. It may be a bus topology or a star topology, and may include more or fewer components than shown, or combine certain components, or have different component arrangements.

[0249] The memory 1020 can be used to store the execution instructions of the processor 1010. The memory 1020 can be implemented by any type of volatile or non-volatile memory terminal or a combination thereof, such as static random access memory (SRAM), electrically erasable programmable read-only memory (EEPROM), erasable programmable read-only memory (EPROM), programmable read-only memory (PROM), read-only memory (ROM), magnetic storage, flash memory, magnetic disk, or optical disk. When the execution instructions in the memory 1020 are executed by the processor 1010, the terminal 1000 is able to perform some or all of the steps in the above method embodiments.

[0250] The processor 1010 serves as the control center of the storage terminal, connecting various parts of the electronic terminal via various interfaces and lines. It executes software programs and / or modules stored in the memory 1020, and calls data stored in the memory to perform various functions of the electronic terminal and / or process data. The processor can be composed of integrated circuits (ICs), such as a single packaged IC or multiple packaged ICs with the same or different functions connected together. For example, the processor 1010 may only include a central processing unit (CPU). In this embodiment of the invention, the CPU may have a single processing core or include multiple processing cores.

[0251] The communication unit 1030 is used to establish a communication channel, enabling the storage terminal to communicate with other terminals. It can receive user data sent by other terminals or send user data to other terminals.

[0252] The present invention also provides a computer storage medium, wherein the computer storage medium may store a program, which, when executed, may include some or all of the steps provided in the embodiments of the present invention. The storage medium may be a magnetic disk, an optical disk, read-only memory (ROM), or random access memory (RAM), etc.

[0253] Therefore, this invention fully utilizes the hardware acceleration unit of the basic disk array by decomposing the combined disk array into a parent array and a child array structure, and splitting the write commands based on the decomposed architecture, and then executing the split write commands step by step. This ensures the data processing efficiency of the combined disk array. The technical effects achieved by this embodiment can be found in the description above, and will not be repeated here.

[0254] Those skilled in the art will clearly understand that the techniques in the embodiments of the present invention can be implemented using software plus necessary general-purpose hardware platforms. Based on this understanding, the technical solutions in the embodiments of the present invention, or the parts that contribute to the prior art, can be embodied in the form of a software product. This computer software product is stored in a storage medium such as a USB flash drive, mobile hard drive, read-only memory (ROM), random access memory (RAM), magnetic disk, or optical disk, or other media capable of storing program code. It includes several instructions to cause a computer terminal (which may be a personal computer, server, or a second terminal, network terminal, etc.) to execute all or part of the steps of the methods described in the various embodiments of the present invention.

[0255] The same or similar parts between the various embodiments in this specification can be referred to mutually. In particular, the terminal embodiments are basically similar to the method embodiments, so the description is relatively simple, and the relevant parts can be referred to the description in the method embodiments.

[0256] In the embodiments provided by this invention, it should be understood that the disclosed systems and methods can be implemented in other ways. For example, the system embodiments described above are merely illustrative. For instance, the division of modules is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple modules or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be through some interfaces; the indirect coupling or communication connection between systems or modules may be electrical, mechanical, or other forms.

[0257] The modules described as separate components may or may not be physically separate. The components shown as modules may or may not be physical modules; that is, they may be located in one place or distributed across multiple network modules. Some or all of the modules can be selected to achieve the purpose of this embodiment according to actual needs.

[0258] In addition, the functional modules in the various embodiments of the present invention can be integrated into one processing module, or each module can exist physically separately, or two or more modules can be integrated into one module.

[0259] Although the present invention has been described in detail with reference to the accompanying drawings and preferred embodiments, the present invention is not limited thereto. Various equivalent modifications or substitutions can be made to the embodiments of the present invention by those skilled in the art without departing from the spirit and essence of the invention, and such modifications or substitutions should all be within the scope of the present invention. Any variations or substitutions that can be easily conceived by those skilled in the art within the technical scope disclosed in the present invention should also be covered within the protection scope of the present invention.

Claims

1. A data processing method for a combined disk array, characterized in that, include: The combined disk array is broken down into a parent array and a child array, wherein the child array is a basic disk array with a corresponding hardware acceleration unit; Based on the number of subarrays, the parent volume corresponding to the parent array is split into the corresponding number of subvolumes, and the correspondence between subvolumes and subarrays is established. Based on the aforementioned correspondence, the write command received by the parent volume is decomposed into sub-commands, and the sub-commands are written into the sub-array using the corresponding sub-volume; The combined disk array is broken down into a parent array and a child array, wherein the child array is a basic disk array with an independent hardware acceleration unit, including: The management command for creating a disk array is intercepted, and the basic information of the disk array to be created is parsed from the parameters of the management command. The basic information includes the disk array level, the number of member disks, and the information of the underlying disk array. If the disk array level is a composite level, then the disk array to be created is marked as the parent array; Based on the information of the underlying disk array contained in the parent array, the corresponding underlying disk array is marked as a child array of the parent array; Execute the management command to create the disk array; Based on the aforementioned correspondence, the write command received by the parent volume is decomposed into sub-commands, and the sub-commands are written to the sub-array using the corresponding sub-volume, including: Parse the parent volume ID, starting position, and length of the write command; Retrieve the corresponding target volume global array based on the parent volume ID; Based on the target volume global array, the starting position is converted into an actual address, and the write command is broken down into multiple sub-commands according to the length and the size of the subarray corresponding to each subvolume in the target volume global array; Execute the subcommands one by one until all subcommands have been executed.

2. The method according to claim 1, characterized in that, Executing the management command for creating the disk array includes: Based on the information of the basic disk array contained in the parent array, the member disks that make up the sub-arrays are determined, and the list of member disks in the parameters of the management command is evenly partitioned according to the number of sub-arrays, and used as the member disks that make up each sub-array. Configure the member disk information of the subarray to the hardware acceleration unit; Create a global array for the array. The global array maintains the disk array information to be created and records the connection relationship between the parent array and the child array. The global array includes parent array information and child array information. The parent array information includes the parent array ID, level, member disk information, number of child arrays, and an array of pointers to the child arrays. The child array information includes the child array ID, level, member disk information, and pointers to the parent array. The system will reply with a message indicating that the disk array was successfully created on the host.

3. The method according to claim 1, characterized in that, Based on the number of subarrays, the parent volume corresponding to the parent array is split into a corresponding number of subvolumes, and a correspondence between subvolumes and subarrays is established, including: Upon receiving a command to create a volume, the volume parameters are parsed from the command, including the parent array ID, volume size, and starting logical address; Access the array global array containing the parent array using the parent array ID, and extract the number of child arrays from the array global array; Set the number of subarrays to the number of subvolumes; The size of each sub-volume and its starting logical address in the sub-array are calculated based on the volume size and starting logical address in the volume parameters and the number of sub-volumes. Configure subvolume information to the hardware acceleration unit; Create a volume global array to maintain parent and child volume information, and record the relationship between parent and child volumes through the volume global array; the volume global array includes parent volume information and child volume information, the parent volume information includes the parent volume ID and the ID of the disk array it belongs to, detailed information of the disk array it belongs to, volume size, starting logical address, number of child volumes, and an array of pointers to child volumes, the child volume information includes the child volume ID and the ID of the subarray it belongs to, detailed information of the subarray it belongs to, volume size, starting logical address, and a pointer to the parent volume; Reply with a message indicating that the host volume was created successfully, and report an asynchronous event indicating that the host volume attributes have been changed.

4. The method according to claim 3, characterized in that, Based on the volume size and starting logical address in the volume parameters, and the number of sub-volumes, calculate the size of each sub-volume and its starting logical address in the sub-array, including: The formula for calculating the size of a subvolume is: child_nlb = nlb / child_ns_num; Where nlb is the volume size and child_ns_num is the number of child volumes; The formula for calculating the starting logical address of a subvolume is: child_raid_slba=raid_slba / child_ns_num; Where raid_slba is the starting logical address in the volume parameters, and child_ns_num is the number of child volumes.

5. The method according to claim 1, characterized in that, Based on the target volume global array, the starting position is converted into an actual address, and according to the length and the size of the subarray corresponding to each subvolume in the target volume global array, the write command is broken down into multiple subcommands, including: The starting position of the write command relative to the actual address of the parent array is calculated based on the size of the parent volume, the starting logical address, and the actual address in the parent array in the target volume global array. Access the parent array and obtain the stripe size of the child array based on the connection relationship between the parent and child arrays; Based on the start position and length of the write command, as well as the stripe size of the subarray, it is determined whether the write command spans multiple subarrays. If the length is less than the stripe size of the subarray, the write command is executed directly. If the length is greater than the stripe size of the subarray, the write command is broken up. The method for disassembling the write command includes: The formula for calculating the subarray number is: child_raid_index=slba / child_stripe_size; Wherein, child_raid_index is the sub-array number, slba is the starting position of the write command, and child_stripe_size is the stripe size of the sub-array; The subarray ID is obtained from the subarray number and the subarray information recorded in the parent array. The subvolume ID is obtained from the subarray ID and the parent volume ID. Calculate the number of logical blocks in the subarray for each segmented subcommand: child_nlb=child_stripe_size–slba%child_stripe_size; Where child_stripe_size is the size of the subarray's stripes, and slba is the starting position; Calculate the starting logical address of the split sub-command in the subarray: child_slba=stripe_id*child_stripe_size+slba%child_stripe_size; Where stripe_id represents the stripe number of the host I / O hit, and its calculation formula is: stripe_id=slba / father_stripe_size; Where father_stripe_size is the stripe size of the parent array; Update the number of remaining logical blocks for the write command, confirm that the number of remaining logical blocks exceeds the number of logical blocks in the subarray, and use the smaller value between the number of remaining logical blocks and the stripe size of the subarray as the number of logical blocks for the subcommand split into the next subarray. Update the starting position of the latest subcommand. The starting position of the latest subcommand is updated to the starting position of the write command plus the number of logical blocks of all subarrays that have been allocated the corresponding subcommand. Update the subarray number of the subarray corresponding to the new subcommand to the previous subarray number plus 1; Apply for the corresponding sub-command processing node and fill in the sub-volume ID, sub-array ID, starting position of the split sub-command, and number of logical blocks of the split sub-command; Increment the number of commands in the parent command processing node by 1; Repeat the splitting of the write command until the number of remaining logical blocks of the write command is 0.

6. The method according to claim 1, characterized in that, Execute subcommands one by one until all subcommands have been executed, including: Upon receiving an event indicating that a subcommand has been executed, extract the subcommand information and execution result from the event. Obtain the processing node of the superior write command based on the sub-command information, and send the sub-command information and execution result to the processing node of the superior write command; The processing node that writes the command summarizes the received sub-command information and execution results, updates the number of sub-commands to be executed, and reclaims the processing node corresponding to the completed sub-command. If all subcommands execute successfully, the parent write command is considered to have executed successfully. If any subcommands fail to execute, the parent write command is considered to have failed.

7. A terminal, characterized in that, include: Memory, used to store data processing programs for the combined disk array; A processor, configured to implement the steps of the data processing method for the combined disk array as described in any one of claims 1-6 when executing the data processing program of the combined disk array.

8. A computer-readable storage medium storing a computer program, characterized in that, The readable storage medium stores a data processing program for the combined disk array, which, when executed by a processor, implements the steps of the data processing method for the combined disk array as described in any one of claims 1-6.

Images