Data processing method and system

Abstract

The invention provides a data processing method and system which can be applied to the technical field of storage. The method is applied to the data processing system, the system comprises a plurality of stripes, a data block and a verification block in a first stripe in the plurality of stripes are provided by a plurality of first hard disks, and a first control disk in the plurality of first hard disks is used for controlling a data read-write process in the first stripe. The method comprises the steps that a first control panel receives a first IO instruction from a host and sends a second IO instruction to a first data panel; wherein the first IO instruction is used for indicating to process the first stripe, and feeding back a processing result to the host; the second IO instruction is generated based on the first IO instruction, the second IO instruction is used for indicating to process the first data block in the first strip, and the first processing result is fed back to the first control panel. Therefore, the method can unload the data read-write task from the host to the hard disk, thereby reducing the workload of the host and releasing the resources of the host.

Description

Technical Field

[0001] This application relates to the field of storage technology, and in particular to a data processing method and system. Background Technology

[0002] Servers consist of processors and solid-state drives (SSDs). SSDs store data, while processors handle the data on the SSDs, performing tasks such as management, reading, writing, and calculations. Typically, servers contain between 1 and 8 processors, and dozens to thousands of SSDs.

[0003] When a server receives a large number of data read / write tasks on SSDs, these tasks are shared among all the processors in the server. Each processor, however, can only execute the assigned tasks one by one. Since a single data read / write task often involves data from multiple SSDs, the processor needs to control the data read / write process for each SSD involved in the task. Specifically, the processor issues input / output (I / O) instructions to each SSD and receives data or write completion messages from each SSD. Therefore, the execution of data read / write tasks consumes a significant amount of processor resources, reducing the processor's available resources and preventing it from executing more tasks. Summary of the Invention

[0004] To address the aforementioned technical problems, this application provides a data processing method and system that can offload data read and write tasks from the host to the hard disk, thereby reducing the workload of the host and freeing up host resources.

[0005] Firstly, a data processing system is provided, including multiple hard disks. The multiple hard disks are used to provide multiple stripes. Data blocks and parity blocks in each stripe originate from different hard disks, where data blocks are physical blocks used to store data, and parity blocks are physical blocks used to store parity data. The parity data is used to verify the data. The multiple stripes include a first stripe. The data blocks and parity blocks in the first stripe are provided by multiple first hard disks. The multiple first hard disks belong to the multiple hard disks. A first control disk is used to control the data read / write process in the first stripe, and the first control disk is any one of the multiple first hard disks.

[0006] In the above scheme, after striping the data stored on the hard drive in the data processing system, a control disk (i.e., the first control disk) is configured for the first stripe. The first control disk controls the data read and write process in the first stripe. This offloads all data read and write tasks involving the first stripe that should be performed by the host to the first control disk. Specifically, it offloads the host's control of the data read and write process in the first stripe to the first control disk, thereby reducing the host's workload and freeing up host resources.

[0007] In some possible implementations, the aforementioned data processing system includes a host computer. The first control disk is used to receive first input / output (I / O) instructions from the host computer and send second I / O instructions to the first data disk. The first I / O instruction instructs processing of a first stripe and feeding back the processing result to the host computer. The second I / O instruction is generated based on the first I / O instruction. Furthermore, the second I / O instruction instructs processing of a first data block within the first stripe and feeding back the first processing result to the first control disk. The first data disk belongs to the plurality of first hard disks, and is different from the first control disk.

[0008] This technical solution provides a method for the first control disk to control the data read / write process in the first stripe. Specifically, the first control disk controls the data read / write process in the first stripe by having the first control disk perform data read / write tasks on behalf of the host, and by controlling the data read / write operations of other first hard disks (i.e., the first data disk). The first control disk performing data read / write tasks on behalf of the host can be achieved by the host sending the data read / write tasks for the first stripe to the first control disk in the form of I / O instructions (specifically, first I / O instructions). The way the first control disk completes the data read / write tasks is by completing the first I / O instructions. The first control disk controls the data read / write operations of the first data disk based on the data read / write tasks for the first stripe by generating second I / O instructions based on the first I / O instructions, sending the second I / O instructions to the first data disk, and waiting for the first processing result returned by the first data disk. It can be understood that each second I / O instruction is used to instruct the processing of a data block in the first stripe. When the first I / O instruction is used to instruct the processing of multiple data blocks in the first stripe, the first control disk needs to control the data read / write operations of multiple first hard disks during the process of completing the first I / O instruction. Specifically, it sends a second I / O instruction to each of these first hard disks. It can be seen that by having the first control disk control the data read / write process in the first stripe, the host only needs to wait for the processing result returned by the first control disk, thus reducing the host's workload and freeing up host resources. Furthermore, since the multiple stripes are independent of each other, configuring a control disk for each stripe allows these control disks to control the data read / write processes in multiple stripes in parallel, thereby improving the execution efficiency of multiple data read / write tasks.

[0009] In some possible implementations, the first control board and the first data board are deployed on the same device, or the first control board and the first data board are deployed on different devices. Specifically, the first control board is used to send second I / O instructions to the first data board via a high-speed interconnect bus.

[0010] In the above scheme, multiple hard drives connected via a high-speed interconnect bus have equal status, thus enabling direct data exchange and communication between them. This technical solution supports direct communication between multiple hard drives in a single-machine system via a high-speed interconnect bus, as well as direct communication between multiple hard drives across devices in a multi-machine system via a high-speed interconnect bus. Therefore, in a multi-machine system, when the first control disk controls data read / write operations on the first hard drive (i.e., the first data disk) in another device, it can directly send the second IO instruction to the first hard drive in the other device via the high-speed interconnect bus, without needing forwarding from the host in the device where the first control disk resides, nor needing to interact with the host in the other device. This avoids consuming host resources in the device where the first control disk resides or in the host resources of the other device.

[0011] In some possible implementations, the first data disk is used to read first data from the host's memory after receiving the second IO instruction, write the first data into the first data block on the first data disk, and then send an instruction completion message to the first control disk.

[0012] This technical solution provides a method for storing first data on a first data disk under the control of a first control disk.

[0013] In some possible implementations, the first data disk is used to read first data from the host's memory after receiving a second I / O instruction; obtain data from the second data block in the first stripe; and perform calculations on the first and second data blocks based on a RAID algorithm to obtain the parity data of the first parity block in the first stripe. The data block used to store the first data in the first stripe is the first data block. The second data block is different from the first data block.

[0014] This technical solution provides an implementation method in an application scenario based on RAID algorithm data storage, where a first control disk controls other first hard drives to store the first data. When the first data needs to be stored, since the data processing system is based on RAID algorithm data storage, when the first control disk receives the task of storing the first data, it instructs the hard drive containing the first data block to store the first data, and simultaneously instructs the hard drive responsible for calculating the parity data of the first stripe to update the parity data of the first stripe using the RAID algorithm. This ensures that if the data in the first stripe is corrupted, the corrupted data can be recovered in a timely manner based on the parity data. As can be seen, in this technical solution, the first data disk has data calculation and processing capabilities, and can perform calculation and processing tasks other than error detection and garbage collection.

[0015] In one specific implementation, the first data disk is used to send a third I / O instruction to the second data disk and receive data from the second data block sent by the second data disk after completing the third I / O instruction. The second data disk belongs to the plurality of first hard disks and is different from the first data disks.

[0016] In this technical solution, instructions and data can be directly transmitted between different hard drives without going through the host, thus avoiding the consumption of host resources.

[0017] In some possible implementations, the first data disk is also used to send the verification data of the first verification block to the third data disk. The third data disk is different from the first data disk.

[0018] When the hard drive used to calculate the verification data is different from the hard drive used to store the verification data, this technical solution supports the hard drive used to calculate the verification data to store the verification data on another hard drive.

[0019] In some possible implementations, the first data block comes from the first data disk. The first data disk is used to read the second data from the first data block after receiving the second IO instruction, write the second data into the host's memory, and then send an instruction completion message to the first control disk.

[0020] This technical solution provides a method for reading second data from a first data disk under the control of a first control disk.

[0021] In some possible implementations, the first data block is not part of the first data disk. In this case, the first data disk is used, after receiving the second I / O instruction, to retrieve the data of the second data block in the first stripe and the parity data of the first parity block in the first stripe; to calculate the second data based on the RAID algorithm using the data of the second data block and the parity data of the first parity block; to write the second data into the host's memory; and then to send a completion message to the first control disk. The second data is stored on the first data block. The second data block is different from the first data block.

[0022] This technical solution provides an implementation method in which the first data disk recovers the second data and provides it to the host in the event of a second data failure. In application scenarios where data processing systems store data based on RAID algorithms, when the first control disk receives a task to retrieve the second data, even if the second data is corrupted, the first control disk can instruct the first data disk to calculate the second data based on the RAID algorithm and then provide the recovered second data to the host. As can be seen, in this technical solution, the first data disk has data calculation and processing capabilities, and can perform calculation and processing tasks other than error detection and garbage collection.

[0023] In summary, this technical solution supports the first control disk in controlling the first data disk to perform various tasks, including reading data from the first data disk, storing data on the first data disk, and performing calculations on the data.

[0024] This technical solution provides a data processing flow with separate control and numerical control in the architecture of the data processing system. The first control panel sends a second I / O instruction to the first data panel, instructing it to transfer either first or second data between itself and the host's memory. This achieves separation of the data plane and the control plane; control flow occurs between the first control panel and the first data panel, while data flow occurs between the first data panel and the host's memory. This enables centralized global control of the control plane and high-performance data transmission. In this scenario of separate control and numerical control, the host is unaware of the specific execution process of data read / write tasks. The host only needs to delegate the data read / write tasks to the first control panel; the host does not need to participate in how the first control panel completes the task. The host considers itself to have completed the task upon receiving a notification from the first control panel that the task has been completed.

[0025] In some possible implementations, the first control disk is further configured to receive a fourth IO instruction from the host before sending the second IO instruction to the first data disk; if the processing method and stripe indicated by the fourth IO instruction and the first IO instruction are the same, the first IO instruction and the fourth IO instruction are merged, and a second IO instruction is generated based on the merging result. The fourth IO instruction is used to instruct the processing of the second stripe and to feed back the processing result to the host.

[0026] Compared to executing each I / O instruction individually, this technical solution combines multiple I / O instructions for processing, reducing the number of second I / O instructions generated and the number of communications between the first control disk, the host, and the first data disk, thus reducing the communication overhead of the first control disk. Furthermore, since one second I / O instruction represents one I / O operation, the reduced number of second I / O instructions correspondingly reduces the number of I / O operations performed on the data disk used to execute them.

[0027] In some possible implementations, the aforementioned first control panel is used to cache I / O instructions from the host via an I / O queue, including first I / O instructions and fourth I / O instructions.

[0028] In the above scheme, the first control panel uses an I / O queue to manage I / O instructions from the host, which can process multiple I / O instructions in a specified order and priority. This can better manage and schedule I / O operations, improve the efficiency and performance of I / O operations, and avoid competition and conflicts between I / O operations.

[0029] Secondly, a data processing method is provided, applied to a data processing system as described in any of the first aspects. The method includes: a first control disk receiving a first input / output (I / O) instruction from a host in the data processing system, and sending a second I / O instruction to a first data disk. The first I / O instruction instructs processing of a first stripe and feeding back the processing result to the host. The second I / O instruction is generated based on the first I / O instruction. The second I / O instruction instructs processing of a first data block in the first stripe and feeding back a first processing result to the first control disk. The first data disk belongs to a plurality of first hard disks in the data processing system, and the first data disk is different from the first control disk.

[0030] In some possible implementations, the first control panel and the first data panel are deployed on the same device, or the first control panel and the first data panel are deployed on different devices. The first control panel sending a second I / O instruction to the first data panel includes: the first control panel sending the second I / O instruction to the first data panel via a high-speed interconnect bus.

[0031] In some possible implementations, the above method further includes: after receiving the second IO instruction, the first data disk reads the first data from the host's memory, writes the first data into the first data block on the first data disk, and then sends an instruction completion message to the first control disk.

[0032] In some possible implementations, the method further includes: after receiving a second I / O instruction, the first data disk reads first data from the host's memory; obtains data from the second data block in the first stripe; and calculates the parity data of the first data and the second data block based on a disk array RAID algorithm to obtain the parity data of the first parity block in the first stripe. The data block used to store the first data in the first stripe is the first data block. The second data block is different from the first data block.

[0033] In some possible implementations, obtaining the data of the second data block in the first stripe includes: sending a third I / O instruction to the second data disk, and receiving the data of the second data block sent by the second data disk after completing the third I / O instruction. The second data disk belongs to the plurality of first hard disks mentioned above, and the second data disk is different from the first data disks.

[0034] In some possible implementations, the method further includes: the first data disk sending the verification data of the first check block to the third data disk. The third data disk belongs to the plurality of first hard disks and is different from the first data disks.

[0035] In some possible implementations, the first data block originates from the first data disk. The method further includes: after receiving the second I / O instruction, the first data disk reads second data from the first data block, writes the second data into the host's memory, and then sends a completion message to the first control disk.

[0036] In some possible implementations, the aforementioned first data block does not belong to the first data disk. The method further includes: after receiving the second I / O instruction, the first data disk acquires the data of the second data block in the first stripe and the parity data of the first parity block in the first stripe; calculates the second data based on the RAID algorithm using the data of the second data block and the parity data of the first parity block; writes the second data into the host's memory; and then sends a completion message to the first control disk. The second data is stored on the first data block. The second data block is different from the first data block.

[0037] In some possible implementations, before the first control disk sends the second I / O instruction to the first data disk, the method further includes: the first control disk receiving a fourth I / O instruction from the host; if the processing method and stripe indicated by the fourth I / O instruction are the same as those indicated by the first I / O instruction, the first I / O instruction and the fourth I / O instruction are merged, and a second I / O instruction is generated based on the merging result. The fourth I / O instruction is used to instruct the processing of the second stripe and to feed back the processing result to the host.

[0038] In some possible implementations, the aforementioned first control panel caches I / O instructions from the host via an I / O queue, including the first I / O instruction and the fourth I / O instruction.

[0039] Thirdly, a data processing method is provided, applied to a first control disk. The first control disk is a hard disk in a data processing system as described in any of the first aspects. The method includes: the first control disk receiving a first input / output (I / O) instruction from a host in the data processing system, and sending a second I / O instruction to a first data disk. The first I / O instruction instructs processing of a first stripe and feeding back the processing result to the host. The second I / O instruction is generated based on the first I / O instruction. The second I / O instruction instructs processing of a first data block in the first stripe and feeding back a first processing result to the first control disk. The first data disk belongs to a plurality of first hard disks in the data processing system, and the first data disk is different from the first control disk.

[0040] In some possible implementations, the first control panel and the first data panel are deployed on the same device, or the first control panel and the first data panel are deployed on different devices. The first control panel sending a second I / O instruction to the first data panel includes: the first control panel sending the second I / O instruction to the first data panel via a high-speed interconnect bus.

[0041] In some possible implementations, before the first control disk sends the second I / O instruction to the first data disk, the method further includes: the first control disk receiving a fourth I / O instruction from the host; if the processing method and stripe indicated by the fourth I / O instruction are the same as those indicated by the first I / O instruction, the first I / O instruction and the fourth I / O instruction are merged, and a second I / O instruction is generated based on the merging result. The fourth I / O instruction is used to instruct the processing of the second stripe and to feed back the processing result to the host.

[0042] In some possible implementations, the aforementioned first control panel caches I / O instructions from the host via an I / O queue, including the first I / O instruction and the fourth I / O instruction.

[0043] Fourthly, a data processing method is provided, applied to a first data disk. The first data disk is a hard disk in a data processing system as described in any of the first aspects. The data processing system includes a plurality of first hard disks, including a first control disk and a first data disk. The first control disk differs from the first data disk. The first control disk is used to receive a first input / output (I / O) instruction from a host in the data processing system. The first I / O instruction is used to instruct processing of a first stripe and to feed back the processing result to the host. The method includes: the first data disk receiving a second I / O instruction from the first control disk; and sending an instruction completion message to the first control disk after completing the second I / O instruction. The second I / O instruction is generated based on the first I / O instruction. The second I / O instruction is used to instruct processing of a first data block in the first stripe.

[0044] In some possible implementations, the first control disk and the first data disk are deployed on the same device, or the first control disk and the first data disk are deployed on different devices. The first data disk receives second I / O instructions from the first control disk, including: the first data disk receiving second I / O instructions from the first control disk via a high-speed interconnect bus.

[0045] In some possible implementations, after the first data disk receives the second IO instruction from the first control disk and before the first data disk sends an instruction completion message to the first control disk after completing the second IO instruction, the method further includes: the first data disk reading first data from the host's memory; and writing the first data into a first data block on the first data disk.

[0046] In some possible implementations, after the first data disk receives the second I / O instruction from the first control disk and before the first data disk sends an instruction completion message to the first control disk after completing the second I / O instruction, the method further includes: the first data disk reading first data from the host's memory; obtaining data from the second data block in the first stripe; and calculating the first data and the second data block based on a disk array RAID algorithm to obtain the parity data of the first parity block in the first stripe. Here, the data block used to store the first data in the first stripe is the first data block. The second data block is different from the first data block.

[0047] In some possible implementations, the first data disk acquiring data from the second data block in the first stripe includes: the first data disk sending a third I / O instruction to the second data disk; and receiving data from the second data block sent by the second data disk after completing the third I / O instruction. The second data disk belongs to one of the aforementioned plurality of first hard disks, and is different from the first data disk.

[0048] In some possible implementations, the method further includes: the first data disk sending the verification data of the first check block to the third data disk. The third data disk belongs to the plurality of first hard disks and is different from the first data disks.

[0049] In some possible implementations, the first data block originates from the first data disk. After the first data disk receives the second I / O instruction from the first control disk, and before the first data disk sends an instruction completion message to the first control disk after completing the second I / O instruction, the method further includes: the first data disk reading second data from the first data block and writing the second data into the host's memory.

[0050] In some possible implementations, the aforementioned first data block does not belong to the first data disk. After the first data disk receives the second I / O instruction from the first control disk, and before the first data disk sends an instruction completion message to the first control disk after completing the second I / O instruction, the method further includes: the first data disk acquiring the data of the second data block in the first stripe and the parity data of the first parity block in the first stripe; calculating the data of the second data block and the parity data of the first parity block based on the RAID algorithm to obtain second data; and writing the second data into the host's memory. The second data is stored on the first data block. The second data block is different from the first data block.

[0051] Fifthly, a chip is provided, including a controller and a power supply circuit for supplying power to the controller, which is used to perform the method of any of the third aspects, or to perform the method of any of the fourth aspects.

[0052] In a sixth aspect, a storage device is provided, including a controller and a storage medium for storing instructions, the controller for executing the instructions, and when the controller executes the instructions, implementing the method of any of the third aspects, or implementing the method of any of the fourth aspects.

[0053] In a seventh aspect, a computer program product comprising instructions is provided, which, when executed by a storage device, causes the storage device to perform the method of any of the third aspects, or the method of any of the fourth aspects.

[0054] Eighthly, a computer-readable storage medium is provided, including computer program instructions that, when executed by a storage device, perform the method of any of the third aspects or the method of any of the fourth aspects. Attached Figure Description

[0055] Figure 1 This is a structural diagram of a data processing system provided in an embodiment of this application;

[0056] Figure 2 This is a schematic diagram illustrating the result of RAID5-based data storage provided in an embodiment of this application;

[0057] Figure 3 This is a structural diagram of another data processing system provided in an embodiment of this application;

[0058] Figure 4 This is a schematic diagram illustrating another result of RAID5-based data storage provided in an embodiment of this application;

[0059] Figure 5 This is a structural diagram of another data processing system provided in an embodiment of this application;

[0060] Figure 6 This is a schematic diagram illustrating another result of RAID5-based data storage provided in an embodiment of this application;

[0061] Figure 7 This is a flowchart illustrating a data processing method provided in an embodiment of this application;

[0062] Figure 8 This is a schematic diagram of the distribution of LBAs in a data processing system provided in an embodiment of this application;

[0063] Figure 9A This is a schematic diagram illustrating the process of a first data disk executing a second I / O instruction, as provided in an embodiment of this application.

[0064] Figure 9B This is a schematic diagram illustrating another process by which a first data disk executes a second I / O instruction, as provided in an embodiment of this application.

[0065] Figure 9C This is a schematic diagram illustrating another process by which a first data disk executes a second I / O instruction, as provided in an embodiment of this application.

[0066] Figure 9D This is a schematic diagram illustrating another process by which a first data disk executes a second I / O instruction, as provided in an embodiment of this application.

[0067] Figure 10 This is a schematic diagram of the structure of a storage device provided in an embodiment of this application. Detailed Implementation

[0068] The technical solutions of the embodiments of this application will be described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those of ordinary skill in the art without creative effort are within the scope of protection of this application.

[0069] This application provides three data processing systems, all of which can offload data read / write tasks to the hard drives within the system. The hard drives control the data read / write processes of other hard drives involved in the tasks. This reduces the processor's workload and frees up processor resources, while leveraging the abundant resources (including computing power and read / write bandwidth) provided by numerous hard drives to improve the parallel processing efficiency of multiple data read / write tasks. The first data processing system can be found in [reference needed]. Figure 1 And an introduction to related content; the second type of data processing system can be found in [reference needed]. Figure 3 And an introduction to related content; the third type of data processing system can be found in [reference needed]. Figure 5 And an introduction to related content.

[0070] (I) The first type of data processing system

[0071] See Figure 1 , Figure 1 This is a structural diagram of a data processing system provided in an embodiment of this application. For example... Figure 1 As shown, the data processing system 10 includes a host 11, a hard disk 12, and a high-speed interconnect bus 13. The host 11 communicates with the hard disk 12, and multiple hard disks 12 communicate with each other via the high-speed interconnect bus 13.

[0072] The data processing system 10 is a computing device, with the host 11, hard disk 12, and high-speed interconnect bus 13 deployed on the same device.

[0073] The host 11 can be, for example, a central processing unit (CPU), a data processing unit (DPU), a graphics processing unit (GPU), a neural network processing unit (NPU), etc., or an application-specific integrated circuit (ASIC), or one or more integrated circuits.

[0074] Hard drive 12 can be either a physical hard drive or a virtual hard drive. A physical hard drive refers to an actual hard drive device, such as a hard disk drive (HDD) or a solid-state drive (SSD). A virtual hard drive refers to a hard drive device simulated by software. A virtual hard drive can be created on a physical hard drive or on a network storage device (such as direct-attached storage, network-attached storage, storage area networks, etc.), such as virtual hard drives in virtual machines (e.g., VMDK, VHD, VDI formats), virtual hard drives in cloud storage services, etc.

[0075] Since there are various product types of hard drive 12, for ease of explanation, the following text will use hard drive 12 as a solid-state drive for example.

[0076] When hard drive 12 is an SSD (see...) Figure 1 The hard disk 12 (SDK 12) is a storage device built from flash memory chips, including an SSD controller and a storage medium. The SSD controller executes input / output (I / O) instructions. The SSD controller can be a chip, such as a field-programmable gate array (FPGA) or an ASIC. The storage medium consists of several flash memory chips. Each flash memory chip can be divided into several physical chunks of a fixed size. Therefore, each physical chunk has a standard capacity, for example, 2 to the power of N (M), where N is a positive integer. Optionally, the SSD controller and the flash memory chips in the storage medium can be mounted on the same printed circuit board (PCB), presented as a disk or card, and communicate with the host 11 or other hard disks 12 via a high-speed interconnect bus 13 through the input / output (I / O) interface on the PCB. In the case where the hard disk 12 includes an SSD controller and storage media, if the IO instruction received by the hard disk 12 is a read instruction, the SSD controller concurrently reads data from one or more physical blocks of the storage media and then sends the data to the outside of the hard disk 12. If the IO instruction received by the hard disk 12 is a write instruction, the SSD controller concurrently writes data from the outside of the hard disk 12 to one or more physical blocks of the storage media.

[0077] High-speed interconnect bus 13 refers to the communication bus connecting host 11, hard disk 12, and other devices. Host 11, hard disk 12, and other devices connected via high-speed interconnect bus 13 have equal status, thus allowing direct data exchange and communication between these devices. High-speed interconnect bus 13 can be, for example, a unified bus (UB), an NV-LINK bus, a peripheral component interconnect express (PCIe) bus, etc.

[0078] In the data processing system 10, physical blocks in the hard disk 12 can be addressed based on their logical unit numbers (LUNs). The specific implementation mainly includes the following configuration and application phases:

[0079] Configuration Phase: A LUN is created using physical blocks from multiple hard disks 12. The size of the LUN depends on business requirements. After determining the size of the LUN, the physical blocks included in the LUN can be determined. In specific implementations, either all physical blocks from a single hard disk 12 can be used to form the LUN, or only a portion of the physical blocks from a single hard disk 12 can be used to form the LUN; this application does not impose a specific limitation.

[0080] For example, the specific process of using a portion of the physical blocks from all hard disks 12 to form a LUN is as follows: A portion of the physical blocks from all hard disks 12 are mapped into multiple logical blocks; logical block addresses (LBAs) are assigned to all logical blocks; and a mapping relationship is established between LBAs and physical block addresses (PBAs). When the mapping between physical blocks and logical blocks is one-to-one, one PBA corresponds to one LBA; when the mapping is one-to-many, one PBA corresponds to multiple LBAs; and when the mapping is many-to-one, multiple PBAs correspond to one LBA. The mapping relationship between physical blocks and logical blocks is determined by the user.

[0081] Application Phase: LBAs are used to indicate the physical blocks that make up the LUN. Specifically, the PBA corresponding to a given LBA is determined by looking up the correspondence between LBAs and PBAs. For ease of explanation, the following descriptions of LBAs will consistently use the example of a one-to-one correspondence between LBAs and PBAs.

[0082] To provide reliable data services, the data processing system 10 typically employs a redundant array of independent disks (RAID) algorithm to write data to the LUN. Data reliability refers to the stability and reliability of data during storage, transmission, and processing. Data reliability requires that data can be correctly accessed, used, and recovered when needed. RAID is a high-performance, highly reliable storage technology that combines multiple independent hard drives in different ways to create a large-capacity hard drive group. RAID algorithms primarily utilize data striping, mirroring, and data verification technologies to achieve high performance, reliability, fault tolerance, and scalability in data processing. RAID algorithms include RAID5, RAID6, RAID50, and RAID60, among others. Different RAID algorithms differ in disk utilization, reliability, read / write performance, and redundancy calculation requirements.

[0083] Writing data using RAID algorithms primarily refers to encoding the data using RAID to obtain parity data, and then distributing the data and its parity data across multiple hard drives. Writing data to a LUN using RAID algorithms typically operates on a logical block basis, dividing all logical blocks in the LUN into multiple stripes for management. Each stripe includes data blocks and parity blocks. Data blocks are physical blocks used to store data, and parity blocks are physical blocks used to store parity data. The data blocks and parity blocks in each stripe originate from different hard drives. The parity data in a stripe is used to verify the data within that stripe, ensuring the reliability of the data in that stripe. Because different stripes contain different logical blocks, the data in different stripes is independent of each other. Therefore, multiple stripes are independent of each other; specifically, reading or writing data in one stripe does not affect the data in another stripe.

[0084] In summary, implementing striped management in LUNs facilitates parallel data reading and writing and redundancy verification, thereby providing a certain degree of redundancy and fault tolerance. Even if one hard drive fails, the data on that hard drive can still be recovered using data from the other hard drives.

[0085] The following will use RAID5 as an example of the RAID algorithm, combined with... Figure 1 The data processing system 10 describes the process of storing data in a LUN using RAID 5.

[0086] RAID 5 requires a 1:1 ratio of data blocks to parity blocks within a stripe, meaning n data blocks correspond to one parity block. Furthermore, RAID 5 requires that data blocks and parity blocks be distributed across all the hard drives that make up the LUN. This way, if one hard drive fails, causing data blocks stored on that drive to be corrupted, the corrupted data blocks can be recovered using data blocks and parity blocks from the remaining hard drives that belong to the same stripe as the corrupted data blocks.

[0087] because Figure 1 Only four hard disks 12 used for grouping LUNs are shown in the data processing system 10, therefore... Figure 1 In the data processing system 10, when storing data based on RAID5, the ratio of data blocks to parity blocks in each stripe is 3:1.

[0088] See Figure 2 , Figure 2 This is a schematic diagram of a result of storing data based on RAID5 provided in an embodiment of this application. Figure 2 The four solid-state drives (SSDs) 11, 12, 13, and 14 shown correspond to different hard drives 12 in the data processing system 10. The process of storing data on SSDs 11, 12, 13, and 14 using RAID 5 is as follows:

[0089] The parity block PA1 is calculated using erasure coding (EC) algorithm with data blocks A11, A12, and A13. Then, data blocks A11, A12, A13 and parity block PA1 are stored in SSD11, SSD12, SSD13 and SSD14 respectively. The stripe S11 is composed of data blocks A11, A12, A13 and parity block PA1.

[0090] The check block PB1 is calculated using the EC encoding algorithm using data blocks B11, B12, and B13. Then, data blocks B11, B12, B13, and PB1 are stored on SSD 11, SSD 12, SSD 14, and SSD 13 respectively. The stripe S12 is composed of data blocks B11, B12, B13, and PB1.

[0091] The check block PC1 is calculated using the EC encoding algorithm using data blocks C11, C12, and C13. Then, data blocks C11, C12, C13, and check block PC1 are stored in SSD 11, SSD 13, SSD 14, and SSD 12 respectively. The stripe S13 is composed of data blocks C11, C12, C13, and check block PC1.

[0092] The check block PD1 is calculated using the EC encoding algorithm using data blocks D11, D12, and D13. Then, data blocks D11, D12, D13, and check block PD1 are stored in SSD 12, SSD 13, SSD 14, and SSD 11, respectively. The stripe S14 is composed of data blocks D11, D12, D13, and check block PD1.

[0093] It should be understood that for other data blocks, a check block is still calculated from three data blocks, and the different check blocks are distributed and stored on different hard disks 12.

[0094] As described above, when using RAID 5 to store data in a LUN, all logical blocks in the LUN are divided into multiple stripes. The data blocks and parity blocks in each stripe come from different hard drives.

[0095] In each stripe, the data blocks and parity blocks come from different hard drives, which can be understood as one or more of the following:

[0096] (1) Each stripe consists of a data block and a parity block, and the data block and the parity block in the stripe are not on the same hard drive;

[0097] (2) Each stripe includes multiple data blocks and a parity block. The parity block in the stripe is not on the same hard drive as any of the data blocks in the stripe.

[0098] (3) Each stripe includes multiple data blocks and a parity block. The parity block in the stripe is not on the same hard drive as any of the data blocks in the stripe, and any two data blocks in the stripe are not on the same hard drive.

[0099] (4) Each stripe includes multiple data blocks and multiple parity blocks. Any parity block in the stripe is not on the same hard drive as any of the data blocks in the stripe.

[0100] (5) Each stripe includes multiple data blocks and multiple parity blocks. Any one of the parity blocks in the stripe is not on the same hard disk as any one of the data blocks in the stripe, and any two of the parity blocks in the stripe are not on the same hard disk.

[0101] (6) Each stripe includes multiple data blocks and multiple parity blocks. Any parity block in the stripe is not on the same hard disk as any data block in the stripe, and any two data blocks in the stripe are not on the same hard disk.

[0102] (7) Each stripe includes multiple data blocks and multiple parity blocks. Any parity block in the stripe is not on the same hard disk as any data block in the stripe, and any two parity blocks in the stripe are not on the same hard disk, and any two data blocks in the stripe are not on the same hard disk.

[0103] As can be seen, when other RAID algorithms are used to store data in a LUN, such as RAID6, RAID50, RAID60, etc., all logical blocks in the LUN will also be divided into multiple stripes, and the data blocks and parity blocks in each stripe come from different hard drives.

[0104] In a scenario where data processing system 10 combines LUN and RAID algorithms to store data, a control disk is configured for each stripe to control the data read and write process within that stripe. Taking the first stripe among multiple stripes in data processing system 10 as an example, a first hard drive is selected from multiple first hard drives providing data blocks and parity blocks for the first stripe as the first control disk, which controls the data read and write process within the first stripe. These multiple first hard drives belong to multiple hard drives 12 within data processing system 10. Figure 2 To further explain, when Figure 2 When stripe S11 is used as the first stripe, data blocks A11, A12, A13, and parity block PA1 in stripe S11 are provided by SSD 11, SSD 12, SSD 13, and SSD 14, respectively. Therefore, SSD 11, SSD 12, SSD 13, and SSD 14 are all first hard drives. Thus, any one of SSD 11, SSD 12, SSD 13, and SSD 14 can be selected as the first control drive. For example, SSD 11 can be selected as the first control drive, and SSD 11 can control the data read and write process in stripe S11.

[0105] Since the data in the first stripe is distributed across multiple first hard disks, the first control disk, in order to control the data read / write process in the first stripe, must aggregate all data read / write tasks involving the first stripe onto the first control disk. The first control disk then performs the data read / write tasks on the first stripe on behalf of the host 11, and also controls the data read / write operations of the other first hard disks. In this application, to distinguish the first control disk from the other first hard disks, the other first hard disks can all be referred to as data disks; that is, the multiple first hard disks include the first control disk and at least one data disk.

[0106] In some potential application scenarios, data read / write tasks are generated by the client and sent to host 11. Therefore, host 11 distributes all data read / write tasks involving the first stripe to the first control disk, enabling the first control disk to aggregate all data read / write tasks involving the first stripe. When the first control disk receives a data read / write task involving the first stripe, it executes the task. When the first control disk completes the task, it can be considered that host 11 has completed it. As can be seen, when completing data read / write tasks in this way, host 11 does not need to directly read or write data in the first stripe.

[0107] In some possible implementations, the host 11 sends data read / write tasks involving the first stripe to the first control disk by sending a first I / O instruction to the first control disk. The first control disk completes the data read / write task by executing the first I / O instruction. The first I / O instruction instructs the host 11 to process the first stripe and return the processing result to the host 11. In a specific implementation, the host 11 directly sends the first I / O instruction to the first control disk via the high-speed interconnect bus 13. This process can be seen in the following... Figure 7 The execution process of the host sending the first IO instruction to the first control disk in step S101 of the data processing method.

[0108] In a specific application scenario, if a data read / write task involves multiple stripes (including the first stripe), the host 11 can generate multiple I / O instructions (including the first I / O instruction) based on the data read / write task. Each I / O instruction instructs processing one stripe and sends the processing result back to the host 11. Furthermore, different I / O instructions instruct different stripes. Subsequently, the host 11 sends these I / O instructions to the control disks of different stripes. When all the I / O instructions have been executed by the control disks that received the I / O instructions, the data read / write task can be considered complete.

[0109] In some possible implementations, the first control disk controls the data read / write operations of the data disk by sending I / O instructions to the data disk. Taking the first data disk in at least one data disk as an example, the first control disk controls the data read / write operations of the first data disk by generating a second I / O instruction based on the first I / O instruction, and then sending the second I / O instruction to the first data disk. The second I / O instruction is used to instruct the processing of the first data block in the first stripe and to feed back the first processing result to the first control disk. The process by which the first control disk generates the second I / O instruction based on the first I / O instruction can be seen in the following... Figure 7 In step S102 of the data processing method, the first control panel generates a second IO instruction based on the first IO instruction.

[0110] The first data disk is then used to receive and execute the second I / O instructions from the first control disk. This process can be seen below. Figure 7 The process of the first data disk executing the second IO instruction in step S104 of the data processing method.

[0111] It should be understood that the above-described control flow implementation in the form of instructions during the execution of data read and write tasks is merely an example, and this application does not specifically limit the implementation method of control flow during the execution of data read and write tasks.

[0112] It should be understood that the number of host computers 11 and hard disks 12 in data processing system 10 is based on... Figure 1 The data processing system 10 shown is illustrated using one host 11 and four hard disks 12 as an example. In practical applications, the number of host 11 and hard disks 12 in the data processing system 10 can be greater, and this application does not impose a specific limitation. Furthermore, the multiple hard disks 12 in the data processing system 10 can be composed of hard disks of different specifications and types.

[0113] In summary, in the data processing system 10, after striping the data stored on the hard disk 12, a control disk is configured for each stripe. The control disk controls the data read / write process within that stripe. This offloads data read / write tasks that would otherwise be performed by the host to the control disk. Specifically, it offloads the host's control over the data read / write process within the stripe to the stripe's control disk, which then controls the data read / write operations of other hard disks, thereby reducing the host's workload and freeing up its resources. Furthermore, since multiple stripes are independent of each other, the control disks for multiple stripes can control the data read / write processes in multiple stripes in parallel, thus improving the execution efficiency of multiple data read / write tasks.

[0114] (II) Second type of data processing system

[0115] The second type of data processing system is as described above. Figure 1 Based on the structure of the data processing system 10, multiple data processing systems 10 are used as multiple nodes in the second type of data processing system. For details, please refer to... Figure 3 And an introduction to related content.

[0116] See Figure 3 , Figure 3 This is a structural diagram of another data processing system provided in an embodiment of this application. For example... Figure 3 As shown, the data processing system 20 includes nodes 21 and a high-speed interconnect bus 22. Multiple nodes 21 communicate with each other via the high-speed interconnect bus 22. Each node 21 includes a host 23 and a hard disk 24. Within a node 21, the host 23 and the hard disk 24, as well as multiple hard disks 24, communicate with each other via the high-speed interconnect bus 22.

[0117] If the data processing system 20 can be a cluster of computing devices, then node 21 is a computing device. When node 21 is a computing device, node 21 can be one of the aforementioned... Figure 1 The data processing system 10, the host 23 in node 21 can be the one described above. Figure 1 The host 11 in the data processing system 10 and the hard disk 24 in the node 21 can be the aforementioned Figure 1 The hard disk 12 in the data processing system 10. For the sake of brevity, the structure of node 21 will not be described in detail here.

[0118] The high-speed interconnect bus 22 is used to transmit I / O instructions or data between multiple nodes 21. Specifically, the high-speed interconnect bus 22 is used to directly transmit I / O instructions or data between a host 23 in one node 21 and a host 23 in another node 21, to directly transmit I / O instructions or data between a host 23 in one node 21 and a hard disk 24 in another node 21, and to directly transmit I / O instructions or data between hard disks 24 in one node 21 and hard disks 24 in another node 21.

[0119] The high-speed interconnect bus 22 is also used to transfer I / O instructions or data between the host 23 and the hard disk 24 in the same node 21, and to transfer I / O instructions or data between multiple hard disks 24 in the same node 21.

[0120] Among them, the high-speed interconnect bus 22 can be the above-mentioned Figure 1 The high-speed interconnect bus 13 in the data processing system 10.

[0121] In the data processing system 20, physical blocks in all hard disks 24 of the data processing system 20 can be addressed based on LUNs. The specific implementation process can be referred to the process of addressing physical blocks in hard disk 12 based on LUNs in the data processing system 10 described above. For the sake of brevity, it will not be elaborated here.

[0122] In data processing system 20, by using a RAID algorithm to store data in a LUN, data processing system 20 can implement striped management of the data stored on hard disk 12. The following will use RAID5 as an example of the RAID algorithm, combined with... Figure 3 The data processing system 20 describes the process of storing data in a LUN using RAID 5.

[0123] because Figure 3 Only the number of nodes 21 used for group LUNs in data processing system 20 is shown as 2, and the number of hard disks 24 in each node 21 is 4, therefore in Figure 3In the data processing system 20, when storing data based on RAID5, the ratio of data blocks to parity blocks in each stripe is 7:1.

[0124] See Figure 4 , Figure 4 This is a schematic diagram of another result of storing data based on RAID5 provided in an embodiment of this application. Figure 4 The eight solid-state drives (SSDs) 21, 22, 23, 24, 25, 26, 27, and 28 shown correspond to different hard drives 24 in different nodes 21 of the data processing system 20. The process of storing data on SSDs 21, 22, 23, 24, 25, 26, 27, and 28 using RAID 5 is as follows:

[0125] The parity block PA2 is calculated using the EC encoding algorithm using data blocks A21, A22, A23, A24, A25, A26, and A27. Then, data blocks A21, A22, A23, A24, A25, A26, A27, and parity block PA2 are stored in SSDs 21, 22, 23, 24, 25, 26, 27, and 28 respectively. This results in stripe S21 composed of data blocks A21, A22, A23, A24, A25, A26, A27, and parity block PA2.

[0126] The parity block PB2 is calculated using the EC encoding algorithm using data blocks B21, B22, B23, B24, B25, B26, and B27. Then, data blocks B21, B22, B23, B24, B25, B26, B27, and parity block PB2 are stored on SSDs 21, 22, 23, 24, 25, 26, 28, and 27 respectively. This results in stripe S22 composed of data blocks B21, B22, B23, B24, B25, B26, B27, and parity block PB2.

[0127] The parity block PC2 is calculated using the EC encoding algorithm using data blocks C21, C22, C23, C24, C25, C26, and C27. Then, data blocks C21, C22, C23, C24, C25, C26, C27, and parity block PC2 are stored on SSDs 21, 22, 23, 24, 25, 27, 28, and 26 respectively. This results in stripe S23 composed of data blocks C21, C22, C23, C24, C25, C26, C27, and parity block PC2.

[0128] The parity block PD2 is calculated using the EC encoding algorithm using data blocks D21, D22, D23, D24, D25, D26, and D27. Then, data blocks D21, D22, D23, D24, D25, D26, D27, and parity block PD2 are stored on SSDs 21, 22, 23, 24, 26, 27, 28, and 25 respectively. This results in stripe S24 composed of data blocks D21, D22, D23, D24, D25, D26, D27, and parity block PD2.

[0129] It should be understood that for other data blocks, a check block is still calculated from seven data blocks, and the different check blocks are distributed and stored on different hard disks 24.

[0130] From the above, we can see that in a LUN composed of multiple nodes, using RAID 5 to store data, all logical blocks in the LUN are divided into multiple stripes. The data blocks and parity blocks in each stripe come from different hard drives, and each stripe is arranged across nodes. Similarly, when using other RAID algorithms to store data in a LUN composed of multiple nodes, such as RAID 6, RAID 50, or RAID 60, all logical blocks in the LUN will also be divided into multiple stripes, with the data blocks and parity blocks in each stripe coming from different hard drives, and each stripe being arranged across nodes.

[0131] In a scenario where data processing system 20 combines LUN and RAID algorithms to store data, a control disk is configured for each stripe to control the data read and write process within that stripe. Taking the first stripe among multiple stripes in data processing system 20 as an example, a first hard drive is selected from multiple first hard drives providing data blocks and parity blocks for the first stripe as the first control disk, which controls the data read and write process within the first stripe. These multiple first hard drives belong to multiple hard drives 24 within data processing system 20. Figure 4 To further explain, when Figure 4 When stripe S21 is used as the first stripe, data blocks A21, A22, A23, A24, A25, A26, A27, and parity block PA2 in stripe S21 are provided by SSD 21, SSD 22, SSD 23, SSD 24, SSD 25, SSD 26, SSD 27, and SSD 28, respectively. Therefore, SSD 21 to SSD 28 are all first hard drives. Thus, any one of SSD 21 to SSD 28 can be selected as the first control drive. For example, SSD 21 can be selected as the first control drive, and SSD 21 can control the data read and write process in stripe S21.

[0132] The first control panel controls the data read / write process in the first stripe by aggregating all data read / write tasks involving the first stripe and controlling the data read / write operations of the control data disk.

[0133] In some possible implementations, the first control disk aggregates all data read / write tasks involving the first stripe, including: the first control disk receiving multiple first I / O instructions from the host 23. Each first I / O instruction corresponds to a data read / write task involving the first stripe. The first I / O instruction is used to instruct processing of the first stripe and to return the processing result to the host 23. Therefore, the host 23 completes the data read / write task involving the first stripe by sending the relevant first I / O instruction to the first control disk and waiting for the processing result returned by the first control disk, without the host 23 directly performing read / write operations on the data in the first stripe. This process can be seen in the following... Figure 7 The execution process of the host sending the first IO instruction to the first control disk in step S101 of the data processing method.

[0134] In some possible implementations, the first control disk controls the data read / write operations of the data disk, including: after receiving a first I / O instruction, the first control disk generates a second I / O instruction based on the first I / O instruction, and then sends the second I / O instruction to the first data disk. The second I / O instruction is used to instruct the processing of a first data block in the first stripe, and to feed back the first processing result to the first control disk. The first data disk belongs to multiple first hard disks, and the first data disk is different from the first control disk. The process by which the first control disk generates the second I / O instruction based on the first I / O instruction can be seen in the following... Figure 7 In step S102 of the data processing method, the first control panel generates a second IO instruction based on the first IO instruction.

[0135] The first data disk is then used to receive and execute the second I / O instructions from the first control disk. This process can be seen below. Figure 7 The process of the first data disk executing the second IO instruction in step S104 of the data processing method.

[0136] It should be noted that the host 23 sending the first I / O command can be either a host 23 located on the same node 21 as the first control panel or a host 23 located on a different node 21 than the first control panel. Regardless of the location of the host 23 sending the first I / O command, the host 23 can directly send the first I / O command to the first control panel via the high-speed interconnect bus 13.

[0137] It should be noted that the first data disk receiving the second I / O command can be either a hard disk 24 located on the same node 21 as the first control disk, or a hard disk 24 located on a different node 21 than the first control disk. Regardless of the location of the first data disk receiving the second I / O command, the first data disk can receive the second I / O command from the first control disk through the high-speed interconnect bus 13.

[0138] It should be understood that the above-described control flow implementation in the form of instructions during the execution of data read and write tasks is merely an example, and this application does not specifically limit the implementation method of control flow during the execution of data read and write tasks.

[0139] It should be understood that the number of nodes 21 in the data processing system 20, and the number of hosts 23 and hard disks 24 in a single node 21, are based on... Figure 3The data processing system 20 shown includes two nodes 21, each node 21 including one host 23 and four hard disks 24 for illustrative purposes. In practical applications, the number of nodes 21, hosts 23, and hard disks 24 in the data processing system 20 can be greater, and this application does not impose a specific limitation. When the data processing system 20 includes more nodes 21, and each node 21 includes more hosts 23, these hosts 23 can distribute the data read / write tasks received from different clients within the same node or across nodes, and the control disk receiving the task controls the data read / write process in the corresponding stripe.

[0140] In summary, in the data processing system 20, hard drives in different nodes are connected via a high-speed interconnect bus, giving these hard drives equal status and enabling direct data exchange and communication. The data processing system 20 supports direct communication between multiple hard drives in a single node via the high-speed interconnect bus, as well as direct communication between hard drives in different nodes via the high-speed interconnect bus. This technical solution manages the data stored on hard drives from different nodes using striping, and then configures a control disk for each stripe. The control disk controls the data read / write process within that stripe. This offloads data read / write tasks that would otherwise be performed by the host to the control disk. Specifically, it offloads the host's control over the data read / write process within the stripe to the stripe's control disk, which then controls the data read / write operations of the hard drives in different nodes. This reduces the host's workload and frees up its resources. Furthermore, when the control disk controls data read / write operations on the hard drive in another node, it can directly send the second I / O command to the hard drive in that node via the high-speed interconnect bus, without needing forwarding from the host in the device where the control disk resides, nor interacting with the host in the other device. This avoids consuming host resources in either the device where the control disk resides or the host resources in the other device. Moreover, since multiple stripes are independent, control disks with multiple stripes can control data read / write processes in multiple stripes in parallel, thus improving the execution efficiency of multiple data read / write tasks.

[0141] (III) The third type of data processing system

[0142] See Figure 5 , Figure 5 This is a structural diagram of another data processing system provided in an embodiment of this application. For example... Figure 5 As shown, the data processing system 30 includes a first device cluster 31, a second device cluster 32, and a high-speed interconnect bus 33. The first device cluster 31 and the second device cluster 32 are different device clusters.

[0143] The first device cluster 31 includes multiple hosts 34. Figure 5The illustration shows a scenario where the first device cluster 31 includes two hosts 34. In a specific implementation, the first device cluster 31 can be a computing device cluster, in which case the first device cluster 31 includes multiple computing devices, and each computing device includes one or more hosts 34. The hosts 34 can be as described above. Figure 1 The host 11 in the data processing system 10.

[0144] The second device cluster 32 includes multiple hard drives 35. Figure 5 The illustration shows a scenario where the second device cluster 32 includes eight hard disks 35. In a specific implementation, the second device cluster 32 can be a storage device cluster, in which case the second device cluster 32 includes multiple storage devices, each storage device including one or more hard disks 35. Here, storage devices refer to hardware devices used for storing and preserving data, including storage servers, storage arrays, etc. The hard disks 35 can be as described above. Figure 1 The hard disk 12 in the data processing system 10.

[0145] The high-speed interconnect bus 33 is used to transmit I / O instructions or data between the first device cluster 31 and the second device cluster 32, and also to transmit I / O instructions or data between multiple hosts 34 within the first device cluster 31, and between multiple hard disks 35 within the second device cluster 32. The high-speed interconnect bus 33 can be as described above. Figure 1 The high-speed interconnect bus 13 in the data processing system 10.

[0146] In the data processing system 30, physical blocks in all hard disks 35 of the second device cluster 32 can be addressed based on LUNs. The specific implementation process can be referred to the process of addressing physical blocks in hard disks 12 based on LUNs in the data processing system 10 described above. For the sake of brevity, it will not be elaborated here.

[0147] In data processing system 30, by using a RAID algorithm to store data in a LUN, data processing system 30 can implement striped management of the data stored on hard disk 35. The following will use RAID5 as an example of the RAID algorithm, combined with... Figure 5 The data processing system 30 describes the process of storing data in a LUN using RAID 5.

[0148] because Figure 5 Only the number of hard disks 35 used for group LUNs in the second device cluster 32 is shown to be 8, therefore in Figure 5 In the data processing system 30, when storing data based on RAID5, the ratio of data blocks to parity blocks in each stripe is 7:1.

[0149] See Figure 6 , Figure 6This is a schematic diagram of another result of storing data based on RAID5 provided in an embodiment of this application. Figure 6 The eight solid-state drives (SSDs) shown, SSD 31, SSD 32, SSD 33, SSD 34, SSD 35, SSD 36, SSD 37, and SSD 38, correspond to different drives 35 in the second device cluster 32. The process of storing data on SSDs 31, SSD 32, SSD 33, SSD 34, SSD 35, SSD 36, SSD 37, and SSD 38 using RAID 5 is as follows:

[0150] The parity block PA3 is calculated using the EC encoding algorithm using data blocks A31, A32, A33, A34, A35, A36, and A37. Then, data blocks A31, A32, A33, A34, A35, A36, A37, and parity block PA3 are stored in SSDs 31, 32, 33, 34, 35, 36, 37, and 38 respectively. The data blocks A31, A32, A33, A34, A35, A36, A37, and parity block PA3 together form stripe S31.

[0151] The parity block PB3 is calculated using the EC encoding algorithm using data blocks B31, B32, B33, B34, B35, B36, and B37. Then, data blocks B31, B32, B33, B34, B35, B36, B37, and parity block PB3 are stored in SSDs 31, 32, 33, 34, 35, 36, 28, and 37 respectively. This results in stripe S32 composed of data blocks B31, B32, B33, B34, B35, B36, B37, and parity block PB3.

[0152] The check block PC3 is calculated using the EC encoding algorithm using data blocks C31, C32, C33, C34, C35, C36, and C37. Then, data blocks C31, C32, C33, C34, C35, C36, C37, and check block PC3 are stored on SSDs 31, 32, 33, 34, 35, 37, 28, and 36 respectively. The data blocks C31, C32, C33, C34, C35, C36, C37, and check block PC3 together form stripe S33.

[0153] The parity block PD3 is calculated using the EC encoding algorithm using data blocks D31, D32, D33, D34, D35, D36, and D37. Then, data blocks D31, D32, D33, D34, D35, D36, D37, and parity block PD3 are stored on SSDs 31, 32, 33, 34, 36, 37, 28, and 35 respectively. The data blocks D31, D32, D33, D34, D35, D36, D37, and parity block PD3 together form stripe S34.

[0154] It should be understood that for other data blocks, a check block is still calculated from seven data blocks, and the different check blocks are distributed and stored on different hard disks 35.

[0155] As described above, when a LUN is formed in the second device cluster 32 and RAID 5 is used to store data, all logical blocks in the LUN are divided into multiple stripes. The data blocks and parity blocks in each stripe come from different hard drives. Optionally, when the second device cluster 32 is a storage device cluster, the multiple hard drives 35 providing the data blocks and parity blocks in the same stripe may be deployed in different storage devices; that is, the stripes are arranged across devices. It can be seen that when other RAID algorithms are used to store data in the LUN formed by the second device cluster 32, such as RAID 6, RAID 50, RAID 60, etc., all logical blocks in the LUN will also be divided into multiple stripes, and the data blocks and parity blocks in each stripe will come from different hard drives.

[0156] In a scenario where data processing system 30 combines LUN and RAID algorithms to store data, a control disk is configured for each stripe to control the data read and write process within that stripe. Taking the first stripe among multiple stripes in data processing system 30 as an example, a first hard drive is selected from multiple first hard drives providing data blocks and parity blocks for the first stripe as the first control disk, which controls the data read and write process within the first stripe. These multiple first hard drives belong to multiple hard drives 35 within data processing system 30. Figure 5 To further explain, when Figure 5 When stripe S31 is used as the first stripe, data blocks A31, A32, A33, A34, A35, A36, A37, and parity block PA3 in stripe S31 are provided by SSD 31, SSD 32, SSD 33, SSD 34, SSD 35, SSD 36, SSD 37, and SSD 38, respectively. Therefore, SSD 31 to SSD 38 are all first hard drives. Thus, any one of SSD 31 to SSD 38 can be selected as the first control drive. For example, SSD 31 can be selected as the first control drive, and SSD 31 can control the data read and write process in stripe S31.

[0157] The first control disk controls the data read / write process in the first stripe by aggregating all data read / write tasks involving the first stripe and controlling data read / write operations on other first hard disks.

[0158] In some possible implementations, the first control disk aggregates all data read / write tasks involving the first stripe, including: the first control disk receiving multiple first I / O instructions from the host 34. Each first I / O instruction corresponds to a data read / write task involving the first stripe. The first I / O instruction instructs the processing of the first stripe and returns the processing result to the host 34. Therefore, the host 34 completes the data read / write task involving the first stripe by sending the relevant first I / O instruction to the first control disk and waiting for the processing result, without the host 34 directly performing read / write operations on the data in the first stripe. This process can be seen in the following... Figure 7 The execution process of the host sending the first IO instruction to the first control disk in step S101 of the data processing method.

[0159] In some possible implementations, the first control disk controls data read / write operations on other first hard disks, including: after receiving a first I / O instruction, the first control disk generates a second I / O instruction based on the first I / O instruction, and then sends the second I / O instruction to the first data disk. The second I / O instruction is used to instruct the processing of a first data block in the first stripe, and to feed back the first processing result to the first control disk. The first data disk belongs to multiple first hard disks, and the first data disk is different from the first control disk. The process by which the first control disk generates the second I / O instruction based on the first I / O instruction can be seen below. Figure 7 In step S102 of the data processing method, the first control panel generates a second IO instruction based on the first IO instruction.

[0160] The first data disk is then used to receive and execute the second I / O instructions from the first control disk. This process can be seen below. Figure 7 The process of the first data disk executing the second IO instruction in step S104 of the data processing method.

[0161] It should be noted that the host 34 sending the first I / O command can be any host 34 in the first device cluster 31. Any host 34 in the first device cluster 31 can directly send the first I / O command to the first control panel via the high-speed interconnect bus 33.

[0162] It should be noted that the first data disk receiving the second I / O command can be any hard disk 35 in the second device cluster 32 other than the first control disk. In the second device cluster 32, the first control disk can directly send the second I / O command to the first data disk via the high-speed interconnect bus 33.

[0163] It should be understood that the above-described control flow implementation in the form of instructions during the execution of data read and write tasks is merely an example, and this application does not specifically limit the implementation method of control flow during the execution of data read and write tasks.

[0164] It should be understood that in the data processing system 30, the number of hosts 34 in the first device cluster 31 and the number of hard disks 35 in the second device cluster 32 are based on... Figure 5 The data processing system 30 shown is illustrated with an example where the first device cluster 31 includes two hosts 34 and the second device cluster 32 includes eight hard disks 35. In practical applications, the number of hosts 34 and hard disks 35 in the data processing system 30 can be greater, and this application does not impose a specific limitation. When the data processing system 30 includes more hosts 34, each of these hosts 34 can distribute the data read / write tasks received from different clients, and the control disk receiving the task controls the data read / write process in the corresponding stripe.

[0165] In summary, in the data processing system 30, after striping the data stored on multiple hard drives 35 in the second device cluster 32, a control disk is configured for each stripe. The control disk controls the data read / write process within that stripe. This offloads the data read / write tasks that would otherwise be performed by the host 34 in the first device cluster 31 to the control disk. Specifically, it offloads the host's control over the data read / write process within the stripe to the stripe's control disk, which then controls the data read / write operations of other hard drives, thereby reducing the host's workload and freeing up its resources. Furthermore, since the multiple stripes are independent, the control disks for multiple stripes can control the data read / write processes in multiple stripes in parallel, thus improving the execution efficiency of multiple data read / write tasks.

[0166] The above text combined Figures 1-6 The data processing system provided in the embodiments of this application has been introduced. Next, a data processing method provided in the embodiments of this application will be described. It should be noted that the application scenarios of the data processing method provided in the embodiments of this application are not limited to those described above. Figure 1 The first type of data processing system Figure 3 The second type of data processing system or Figure 5 The third type of data processing system, and all scenarios in which the data processing methods provided in the embodiments of this application can be applied, are within the protection scope of this application.

[0167] See Figure 7 , Figure 7 This is a flowchart illustrating a data processing method provided in an embodiment of this application. Figure 7 As shown, the data processing method provided in this application includes:

[0168] S101: The host sends the first I / O command to the first control panel.

[0169] Accordingly, the first control panel receives the first I / O command from the host.

[0170] In one specific implementation, the host sends a first I / O instruction to the first control panel via a high-speed interconnect bus, and the first control panel receives the first I / O instruction from the host via the high-speed interconnect bus.

[0171] In scenarios where data processing systems provide LUN-based addressing, logical blocks within a LUN are divided into multiple stripes, and LBAs are assigned to the logical blocks within each stripe. The host machine of the data processing system stores the mapping between all stripes and all LBAs, while each control disk stores the mapping between the stripes it controls and their corresponding LBAs. Logical blocks within a stripe correspond only to data blocks within that stripe; parity blocks do not have corresponding logical blocks. That is, parity blocks typically do not have LBAs; only data blocks have LBAs. This is because data blocks store actual data information, and therefore each data block has a corresponding LBA to uniquely identify its location in the first storage space. Parity blocks, on the other hand, exist to provide data redundancy verification; therefore, parity blocks typically do not have independent LBAs but are associated with their corresponding data blocks.

[0172] In one specific implementation, consecutive LBAs can be allocated to logic blocks within the same stripe, and the LBAs in adjacent stripes can be consecutive. Specifically, the smallest LBA in one stripe is adjacent to the largest LBA in one of its adjacent stripes, and the largest LBA in one stripe is adjacent to the smallest LBA in another of its adjacent stripes. Figure 2 Taking stripes S11-S14 in the data storage system 10 as an example, consecutive LBAs are assigned to all data blocks in each stripe, and the LBAs of data blocks in adjacent stripes are also consecutive, such as... Figure 8 As shown. See also Figure 8 , Figure 8 This is a schematic diagram illustrating the distribution of LBAs in a data processing system provided in an embodiment of this application. Figure 8 In the diagram, from stripe S11 to stripe S14, the LBA value gradually increases, and the LBAs of data blocks within each stripe are consecutive. For example, in stripe S11, the LBAs of data blocks A11, A12, and A13 are LBA 0, LBA 1, and LBA 2, respectively, while in stripe S12, the LBAs of data blocks B13, B12, and B11 are LBA 3, LBA 4, and LBA 5, respectively. Furthermore, the LBAs of any two adjacent stripes are also consecutive. For instance, the smallest LBA in stripe S12—LBA 3—is adjacent to the largest LBA in the adjacent stripe S11—LBA 2, and the largest LBA in stripe S12—LBA 5—is adjacent to the smallest LBA in the adjacent stripe S13—LBA 6.

[0173] It should be understood that the above-described assignment of consecutive LBAs to logical blocks within the same strip is merely an example. In practical applications, discrete LBAs can also be assigned to logical blocks within the same strip, and this application does not impose any specific limitations. However, for ease of explanation, the following description of LBA distribution in data processing systems will uniformly use the example of consecutive LBAs for logical blocks within the same strip and consecutive LBAs in adjacent strips.

[0174] After allocating LBAs to logical blocks in a stripe, the specific process for users to read and write data to all physical blocks that make up the LUN by accessing the LUN is as follows: If a user wants to read data, the user generates a data read / write task carrying the first LBA through the client and sends the data read / write task to the host in the data processing system; if a user wants to store data, the user generates a data read / write task carrying the first LBA and the target data through the client and sends the data read / write task to the host in the data processing system.

[0175] In some possible implementations, after receiving a data read / write task, the host obtains the first LBA from the task and generates a first I / O instruction based on the first LBA. Furthermore, the process of generating the first I / O instruction varies slightly depending on the number of first LBAs, mainly including the following two cases:

[0176] Case 1: The first LBA has one instance. For example, the first LBA is LBA 2.

[0177] Case 2: There are multiple first LBAs. For example, the first LBAs are LBA 0-LBA 8, or the first LBAs are LBA 0-LBA 2.

[0178] In case 1 of step S101, the host determines the stripe corresponding to the first LBA from the correspondence between stripes and LBAs as the first stripe. Therefore, the number of first stripes is one. The host then generates a first I / O instruction carrying the first LBA. This first I / O instruction instructs the processing of the data block with the first LBA in the first stripe, and feeds back the processing result (e.g., successful or unsuccessful execution) to the host. In this case, the number of data blocks with the first LBA is one. In a specific implementation, the correspondence between stripes and LBAs is recorded in a table, and the host determines the first stripe by looking up the table.

[0179] Continuing with the example of LBA 2 as the first LBA, combined with Figure 8From the distribution of LBAs, we can see that LBA2 corresponds to stripe S11, and LBA2 specifically corresponds to data block A13 in stripe S11. Therefore, stripe S11 is the first stripe, and data block A13 is the data block with the first LBA. The first IO instruction carries LBA2 and is used to instruct the processing of data block A13 in stripe S11, and to feed back the processing result to the host.

[0180] In case 2 of step S101, the host determines the stripe corresponding to each first LBA as the first stripe from the correspondence between stripes and LBAs, and the number of first stripes is one or more.

[0181] When all the first LBAs correspond to the same stripe, the number of first stripes is one. The host then generates a first I / O instruction carrying all the first LBAs. This first I / O instruction instructs the processing of data blocks with first LBAs within the first stripe, and the processing result is fed back to the host. In this case, the number of data blocks with first LBAs is multiple.

[0182] Continuing with the example of the first LBA (LBA 0-LBA 2), and combining... Figure 8 From the distribution of LBAs, we can see that LBA 0, LBA 1, and LBA 2 all correspond to stripe S11. Specifically, LBA 0 corresponds to data block A11 in stripe S11, LBA 1 corresponds to data block A12 in stripe S11, and LBA 2 corresponds to data block A13 in stripe S11. Therefore, stripe S11 is the first stripe, and data blocks A11, A12, and A13 are all considered data blocks with the first LBA. The first I / O instruction carries LBA 0-LBA 2 and is used to instruct the processing of data blocks A11, A12, and A13 in stripe S11, and to feed the processing result back to the host.

[0183] When different first LBAs correspond to different stripes, and there are multiple first stripes, the host generates a first I / O instruction for each first stripe. Each first I / O instruction carries a portion of the first LBA, and each first I / O instruction instructs the processing of a data block with a first LBA within a first stripe, and the processing result is fed back to the host. In this case, the number of data blocks with first LBAs in each first stripe can be one or more.

[0184] Continuing with the example of the first LBA (LBA 0-LBA8), and combining... Figure 8From the distribution of LBAs, we can see that LBAs 0-LBA2 correspond to stripe S11, LBAs 3-LBA 5 correspond to stripe S12, and LBAs 6-LBA 8 correspond to stripe S13. Furthermore, LBAs 0-LBA 2 correspond to data blocks A11-A13 in stripe S11, LBAs 3-LBA 5 correspond to data blocks B13-B11 in stripe S12, and LBAs 6-LBA 8 correspond to data blocks C11-C13 in stripe S13. Therefore, stripes S11, S12, and S13 are all considered the first stripe. Data blocks A11-A13 are considered data blocks with the first LBA in stripe S11, data blocks B11-B13 are considered data blocks with the first LBA in stripe S12, and data blocks C11-C13 are considered data blocks with the first LBA in stripe S13. In this scenario, the host generates a first I / O instruction for stripe S11, carrying LBAs 0-LBA2. This instruction instructs the processing of data blocks A11-A13 within stripe S11, and the processing result is then returned to the host. The host generates a first I / O instruction for stripe S12, carrying LBAs 3-LBA5. This instruction instructs the processing of data blocks B13-B11 within stripe S12, and the processing result is then returned to the host. The host generates a first I / O instruction for stripe S13, carrying LBAs 6-LBA8. This instruction instructs the processing of data blocks C11-C13 within stripe S13, and the processing result is then returned to the host.

[0185] In summary, the number of first I / O instructions generated by the host varies in different scenarios.

[0186] Optionally, the first I / O instruction carries the data length. The data length is used to identify the size of the data. For example, if the first I / O instruction carries the information "8K", it means that the data is 8 kilobytes (KB).

[0187] Optionally, the first I / O instruction carries an instruction type. The instruction type is used to identify whether the first I / O instruction is a read instruction or a write instruction. If the first I / O instruction is a read instruction, it carries a read flag; if the first I / O instruction is a write instruction, it carries a write flag.

[0188] Optionally, the first I / O instruction carries a RAID algorithm. In this case, the first I / O instruction is used to instruct data read / write operations to be performed according to the RAID algorithm. For example, if the first I / O instruction carries "RAID5", then RAID5 should be used for data read / write operations.

[0189] After generating the first I / O instruction, for each first I / O instruction, the host determines the control disk corresponding to the first stripe indicated by the first I / O instruction as the first control disk from the mapping relationship between stripes and control disks, and sends the first I / O instruction to the first control disk. The mapping relationship between stripes and control disks can be stored on the host after configuring a control disk for each stripe. For example, if the first stripe is stripe S11, and the control disk for stripe S11 is SSD 11, then SSD 11 serves as the first control disk, and the host sends the first I / O instruction to SSD 11.

[0190] S102: The first control panel generates a second IO instruction based on the first IO instruction.

[0191] In some possible implementations, after receiving the first I / O instruction, the first control disk retrieves the first LBA from the first I / O instruction and generates a second I / O instruction based on the first LBA. The process by which the first control disk generates the second I / O instruction varies slightly depending on the number of first LBAs and the information stored on the multiple first hard disks used to provide the data blocks and parity blocks for the first stripe, as detailed below:

[0192] In a scenario where the hard drive containing the data block with the first LBA is the first control disk, the first control disk does not generate the second I / O instruction.

[0193] Taking LBA 0 as the first LBA and SSD 11 as the first control disk as an example, combined with Figure 8 Based on the LBA distribution, the data block with LBA 0 is data block A11, and data block A11 is on SSD 11. Therefore, the first control disk does not generate the second I / O instruction.

[0194] In a scenario where the hard drive containing the data block with the first LBA is not the first control drive, and the number of first LBAs is only one, if each first hard drive stores the correspondence between LBAs and PBAs of data blocks in the first stripe, the first control drive generates a second I / O instruction carrying the first LBA. This second I / O instruction is used to instruct the processing of the data block containing the first LBA and to feed back the first processing result (e.g., successful or unsuccessful execution) to the first control drive. If each first hard drive only stores the PBAs of data blocks in the first stripe, the first control drive can determine the PBA corresponding to the first LBA as the first PBA from its stored correspondence between LBAs and PBAs, and then generate a second I / O instruction carrying the first PBA. This second I / O instruction is used to instruct the processing of the data block containing the first PBA and to feed back the first processing result to the first control drive.

[0195] Continuing with the example of LBA 2 as the first LBA and SSD 11 as the first control disk, it is assumed that SSD 11-SSD 14 all store the correspondence between LBA and PBA of data blocks A11-A13 in stripe S11.

[0196] Combination Figure 8 Based on the LBA distribution, the data block with LBA 2 is data block A13, which resides on SSD 13. Therefore, the second I / O instruction generated by the first control disk carries LBA 2. This second I / O instruction is used to instruct the processing of data block A13 and to feed back the processing result to the first control disk.

[0197] In a scenario where the hard drive containing the data block with the first LBA is not the first control disk, and there are multiple first LBAs, the first control disk generates a second I / O instruction for each first LBA. The process of the first control disk generating a second I / O instruction for each first LBA can be referred to in step S102 above, which describes the process of the first control disk generating a second I / O instruction in a scenario where there is only one first LBA; it will not be repeated here.

[0198] Continuing with the example of LBA 1 and LBA 2 as the first LBA and SSD 11 as the first control disk, assume that SSDs 11-SSD 14 all store the correspondence between LBAs and PBAs for data blocks A11-A13 in stripe S11. Combined with... Figure 8 From the LBA distribution, we know that data blocks with LBA 1 and LBA 2 are data blocks A12 and A13, respectively, and data blocks A12 and A13 are located on SSD 12 and SSD 13, respectively. Therefore, the first control disk generates a second I / O instruction for LBA 1, which carries LBA 1 and is used to instruct the processing of data block A12, and feeds back the processing result to the first control disk. The first control disk generates a second I / O instruction for LBA 2, which carries LBA 2 and is used to instruct the processing of data block A13, and feeds back the processing result to the first control disk.

[0199] In summary, the number of second I / O instructions generated by the first control panel varies in different scenarios.

[0200] Optionally, if the first I / O instruction carries the data length, the second I / O instruction carries the data length.

[0201] Optionally, if the first I / O instruction carries an instruction type, the second I / O instruction carries an instruction type, and the second I / O instruction carries the same instruction type as the first I / O instruction.

[0202] Optionally, if the first I / O instruction carries a RAID algorithm, the second I / O instruction carries a RAID algorithm, and the second I / O instruction carries the same RAID algorithm as the first I / O instruction.

[0203] In this application, for ease of explanation, the data block with the first LBA or the data block with the first PBA in the above step S102 can be simply referred to as the first data block. The above second IO instruction is uniformly described as follows: the second IO instruction is used to instruct the processing of the first data block and to feed back the first processing result to the first control panel.

[0204] S103: The first control disk sends a second IO command to the first data disk.

[0205] After generating the second I / O instruction, for each second I / O instruction, the first control disk determines the data disk corresponding to the first data block indicated by the second I / O instruction from the correspondence between data blocks, parity blocks, and data disks stored in the first stripe, and sends the second I / O instruction to the first data disk. Correspondingly, the first data disk receives the second I / O instruction from the first control disk.

[0206] In one specific implementation, the first control board sends a second I / O instruction to the first data board via a high-speed interconnect bus, and the first data board receives the second I / O instruction from the first control board via the high-speed interconnect bus.

[0207] S104: The first data disk executes the second I / O instruction.

[0208] The first data disk sends the first processing result of the second I / O instruction to the first control disk.

[0209] In some possible implementations, the process of the first data disk executing the second I / O instruction is related to the instruction type of the second I / O instruction (such as read instruction or write instruction) and the data status in the first data block (such as data normal or data fault).

[0210] The following will describe the specific process of the first data disk executing the second IO instruction in different scenarios.

[0211] Scenario 1: The second IO instruction carries a write instruction (such as carrying a write flag). The second IO instruction is used to instruct the first data in the host's memory to be stored in the first data block.

[0212] In a data processing system, host memory refers to the devices or components used to store programs and data. It temporarily stores the data and instructions needed by the host so that the processor can access and operate them quickly. Host memory can be directly accessed using direct memory access (DMA) technology. Examples of host memory include dynamic random access memory (DRAM), static random access memory (SRAM), flash memory, virtual memory, and so on.

[0213] In a data processing system using RAID algorithms to store data, writing the first data from the host's memory to the first data block in the first stripe involves two processes: writing the first data to the first data block, and updating the first parity block in the first stripe. To implement these two processes, the first control disk needs to send second I / O commands to the first hard drive containing the first data block and the first hard drive used to update the first parity block, respectively. The process of these two hard drives executing the second I / O commands can be seen in scenarios 11 and 12 below.

[0214] Scenario 11: The first hard drive containing the first data block is designated as the first data disk.

[0215] See Figure 9A , Figure 9A This is a schematic diagram illustrating the process of a first data disk executing a second I / O instruction, as provided in an embodiment of this application.

[0216] like Figure 9A As shown, the process of the first data disk executing the second I / O instruction is as follows: the first data disk reads the first data from the host's memory, and writes the first data into the first data block on the first data disk according to the first LBA (or first PBA) in the second I / O instruction. After writing the first data into the first data block, it means that the first data disk has completed the second I / O instruction. Therefore, the first data disk sends an instruction completion message (i.e., the first processing result of the second I / O instruction) to the first control disk through the high-level interconnect bus to indicate that the execution was successful.

[0217] The following are respectively based on Figure 2 Data processing system 10 Figure 4 Data processing system 20 Figure 6 Using the data processing system 30 as an example, the process of executing the second IO instruction on the first data disk is described in detail.

[0218] exist Figure 2In the data processing system 10, taking SSD 11 as the first control disk and data block A13 as the first data block as an example, the first data disk is SSD 13. The process of SSD 13 executing the second IO instruction is as follows: SSD 13 reads the first data from the host's memory and writes the first data into data block A13. After writing the first data into data block A13, SSD 13 sends an instruction completion message to SSD 11.

[0219] exist Figure 4 In the data processing system 20, taking SSD 21 as the first control disk and data block A27 as the first data block as an example, the first data disk is SSD 27, which is located on a different node than SSD 21. The process of SSD 27 executing the second IO instruction is as follows: SSD 27 reads the first data from the host's memory and writes the first data to data block A27. After writing the first data to data block A27, SSD 27 sends an instruction completion message to SSD 21.

[0220] exist Figure 6 In the data processing system 30, taking SSD 31 as the first control disk and data block A37 as the first data block as an example, the first data disk is SSD 37. The process of SSD 37 executing the second IO instruction is as follows: SSD 37 reads the first data from the host's memory and writes the first data to data block A37. After writing the first data to data block A37, SSD 37 sends an instruction completion message to SSD 31.

[0221] Scenario 12: The first hard drive used to update the first check block is used as the first data disk.

[0222] See Figure 9B , Figure 9B This is a schematic diagram illustrating another process by which a first data disk executes a second I / O instruction, as provided in an embodiment of this application.

[0223] like Figure 9BAs shown, the process of the first data disk executing the second I / O instruction is as follows: The first data disk reads the first data from the host's memory. The first data disk sends a third I / O instruction to the first hard disk (hereinafter referred to as the second data disk) where the second data block is located. The third I / O instruction is used to instruct the second data disk to read the data of the second data block and send the data of the second data block to the first data disk. After receiving the third I / O instruction, the second data disk reads the data from the second data block on the second data disk and sends a read completion message carrying the data to the first data disk through the high-speed interconnect bus. After receiving the read completion message, the first data disk obtains the data of the second data block from the read completion message, calculates the data of the first data block and the data of the second data block based on the RAID algorithm in the second I / O instruction, and obtains the parity data of the first parity block in the first stripe. Among them, the second data block is a data block in the first stripe that is different from the first data block.

[0224] When the first data disk is a first hard disk used to provide the first parity block, after the first data disk calculates the parity data of the first parity block, it directly stores the parity data in the first parity block on the first data disk, thereby updating the parity block in the first stripe. Next, the first data disk sends a completion message indicating successful execution to the first control disk via the high-level interconnect bus.

[0225] If the first data disk is not the first hard disk used to provide the first parity block, after calculating the parity data for the first parity block, the first data disk sends the parity data to the first hard disk used to provide the first parity block (hereinafter referred to as the third data disk). Upon receiving the parity data, the third data disk directly stores the parity data in the first parity block on its own disk, thereby updating the parity block in the first stripe. Next, the third data disk sends a write completion message indicating successful writing to the first data disk via the high-level interconnect bus. Upon receiving the write completion message, the first data disk sends an instruction completion message indicating successful execution to the first control disk via the high-level interconnect bus. The address of the third data disk can be carried by a second I / O instruction, stored on the first data disk, or requested from the first control disk after calculating the parity data for the first parity block (e.g., by sending an address retrieval instruction).

[0226] The following are respectively based on Figure 2 Data processing system 10 Figure 4 Data processing system 20 Figure 6 Using the data processing system 30 as an example, the process of executing the second IO instruction on the first data disk is described in detail.

[0227] exist Figure 2In the data processing system 10, taking SSD 11 as the first control disk, data block A13 as the first data block, and SSD 14 as the first data disk as an example, the second data block includes data block A11 and data block A12, and the second data disk includes SSD 11 and SSD 12. The process of SSD 14 executing the second IO instruction is as follows: SSD 14 reads the first data from the host memory. SSD 14 sends a third IO instruction to SSD 11 and SSD 12 respectively. After receiving the third IO instruction, SSD 11 reads data from data block A11 and sends a message indicating that the data in data block A11 has been read to SSD 14. After receiving the third IO instruction, SSD 12 reads data from data block A12 and sends a message indicating that the data in data block A12 has been read to SSD 14. After receiving read completion messages from SSD 11 and SSD 12, SSD 14 retrieves the data of data block A11 and data block A12 from these messages, respectively. Then, SSD 14 performs calculations based on RAID 5 on the first data, the data of data block A11, and the data of data block A12 to obtain the parity data for parity block PA1 in stripe S11. Since SSD 14 is the first hard drive used to provide parity block PA1, after calculating the parity data, SSD 14 directly stores the parity data in parity block PA1 and then sends a completion instruction message to SSD 11.

[0228] exist Figure 4 In the data processing system 20, taking SSD 21 as the first control disk, data block A27 as the first data block, and SSD 28 as the first data disk as an example, the second data block includes data blocks A21-A26, and the second data disk includes SSD 21-SSD 26. The process of SSD 28 executing the second IO instruction is as follows: SSD 28 reads the first data from the host memory. SSD 28 sends the third IO instruction to SSD 21-SSD 26 respectively. After receiving the third IO instruction, SSD 21-SSD 26 read data from data blocks A21-A26 respectively and send a read completion message carrying the data to SSD 28. After receiving the read completion message from SSD 21-SSD 26, SSD 28 retrieves the data from data blocks A21-A26 respectively from the read completion message of SSD 21-SSD 26. Then, SSD 28 calculates the data of the first data block and data blocks A21-A26 based on RAID5 to obtain the parity data of parity block PA2 in stripe S21, stores the parity data in parity block PA2, and then sends a command completion message to SSD 21.

[0229] exist Figure 6In the data processing system 30, taking SSD 31 as the first control disk, data block A37 as the first data block, and SSD 38 as the first data disk as an example, the second data block includes data blocks A31-A36, and the second data disk includes SSD 31-SSD 36. The process of SSD 38 executing the second IO instruction is as follows: SSD 38 reads the first data from the host memory. SSD 38 sends the third IO instruction to SSD 31-SSD 36 respectively. After receiving the third IO instruction, SSD 31-SSD 36 read data from data blocks A31-A36 respectively and sends a read completion message carrying the data to SSD 14. After receiving the read completion messages from SSD 31-SSD 36, SSD 38 retrieves the data from data blocks A31-A36 respectively from the read completion messages of SSD 31-SSD 36. Then, SSD 38 calculates the data of the first data block and data blocks A31-A36 based on RAID5 to obtain the parity data of parity block PA3 in stripe S31, stores the parity data in parity block PA3, and then sends a command completion message to SSD 31.

[0230] Scenario 2: The second IO instruction carries a read instruction (such as carrying a read flag). The second IO instruction is used to instruct the second data in the first data block to be stored in the host's memory.

[0231] In a data processing system that uses RAID algorithms to store data, retrieving the second data from the first data block in the first stripe can be achieved in at least two ways: directly reading the second data from the first data block, and calculating the second data using the first parity block in the first stripe. These two methods are triggered by the first control disk sending second I / O commands to the first hard drive containing the first data block and the first hard drive containing the first parity block, respectively. The process of these two hard drives executing the second I / O commands can be seen in scenarios 21 and 22 below.

[0232] Scenario 21: The first hard drive containing the first data block is designated as the first data disk. Scenario 21 typically corresponds to the case where the second data block is normal.

[0233] See Figure 9C , Figure 9C This is a schematic diagram illustrating another process by which a first data disk executes a second I / O instruction, as provided in an embodiment of this application.

[0234] like Figure 9CAs shown, the process of the first data disk executing the second I / O instruction is as follows: the first data disk reads the second data from the first data block on the first data disk according to the first LBA (or first PBA) in the second I / O instruction, and writes the second data into the host's memory. After writing the second data into the host's memory, it means that the first data disk has completed the second I / O instruction. Therefore, the first data disk sends an instruction completion message indicating successful execution to the first control disk through the high-level interconnect bus.

[0235] The following are respectively based on Figure 2 Data processing system 10 Figure 4 Data processing system 20 Figure 6 Using the data processing system 30 as an example, the process of executing the second IO instruction on the first data disk is described in detail.

[0236] exist Figure 2 In the data processing system 10, continuing with the example of the first control disk being SSD 11 and the first data block being data block A13, the first data disk is SSD 13. The process of SSD 13 executing the second IO instruction is as follows: SSD 13 reads the second data from data block A13 and writes the second data into the host's memory. After writing the second data into the host's memory, SSD 13 sends an instruction completion message to SSD 11 through the high-level interconnect bus.

[0237] exist Figure 4 In the data processing system 20, continuing with the example of SSD 21 as the first control disk and data block A27 as the first data block, the first data disk is SSD 27, which is located on a different node than SSD 21. The process of SSD 27 executing the second IO instruction is as follows: SSD 27 reads the second data from data block A27 and writes the second data into the host's memory. After writing the second data into the host's memory, SSD 27 sends an instruction completion message to SSD 21 through the high-level interconnect bus.

[0238] exist Figure 6 In the data processing system 30, continuing with the example of the first control disk being SSD 31 and the first data block being data block A37, the first data disk is SSD 37. The process of SSD 37 executing the second IO instruction is as follows: SSD 37 reads the second data from data block A37 and writes the second data into the host's memory. After writing the second data into the host's memory, SSD 37 sends an instruction completion message to SSD 31 through the high-level interconnect bus.

[0239] Scenario 22: The first hard drive containing the first check block is used as the first data disk. Scenario 22 typically corresponds to a second data failure.

[0240] See Figure 9D , Figure 9D This is a schematic diagram illustrating another process by which a first data disk executes a second I / O instruction, as provided in an embodiment of this application.

[0241] like Figure 9D As shown, the process of the first data disk executing the second I / O instruction is as follows: The first data disk reads parity data from the first parity block on the first data disk. The first data disk sends a third I / O instruction to the first hard disk (hereinafter referred to as the second data disk) where the second data block is located. The third I / O instruction is used to instruct the second data disk to read the data of the second data block and send the data of the second data block to the first data disk. After receiving the third I / O instruction, the second data disk reads data from the second data block on the second data disk and sends a read completion message carrying the data to the first data disk through the high-speed interconnect bus. After receiving the read completion message, the first data disk obtains the data of the second data block from the read completion message, calculates the parity data and the data of the second data block based on the RAID algorithm in the second I / O instruction, and obtains the second data. The first data disk writes the second data into the host memory. After writing the second data into the host memory, it means that the first data disk has completed the second I / O instruction. Therefore, the first data disk sends an instruction completion message indicating successful execution to the first control disk through the high-speed interconnect bus. Here, the second data block is a data block in the first stripe that is different from the first data block.

[0242] The following are respectively based on Figure 2 Data processing system 10 Figure 4 Data processing system 20 Figure 6 Using the data processing system 30 as an example, the process of executing the second IO instruction on the first data disk is described in detail.

[0243] exist Figure 2In the data processing system 10, continuing with the example of SSD 11 as the first control disk, data block A13 as the first data block, and SSD 14 as the first data disk, the first parity block is parity block PA1, the second data block includes data block A11 and data block A12, and the second data disk includes SSD 11 and SSD 12. The process of SSD 14 executing the second IO instruction is as follows: SSD 14 reads parity data from parity block PA1. SSD 14 sends a third IO instruction to SSD 11 and SSD 12 respectively. After receiving the third IO instruction, SSD 11 and SSD 12 read data from data block A11 and data block A12 respectively, and send a read completion message carrying the data to SSD 14. After receiving the read completion messages from SSD 11 and SSD 12, SSD 14 obtains the data from data block A11 and data block A12 respectively from the read completion messages of SSD 11 and SSD 12. SSD 14 uses RAID 5 to calculate the parity data, data from data block A11, and data from data block A12 to recover the second data in data block A13. Then, SSD 14 writes the second data to the host's memory and sends a completion message to SSD 11.

[0244] exist Figure 4 In the data processing system 20, continuing with the example of the first control disk being SSD 21, the first data block being data block A27, and the first data disk being SSD 28, the first parity block is parity block PA2, the second data block includes data blocks A21-A26, and the second data disk includes SSD 21-SSD 26. The process of SSD 28 executing the second IO instruction is as follows: SSD 28 reads parity data from parity block PA2. SSD 28 sends a third IO instruction to SSD 21-SSD 26 respectively. After receiving the third IO instruction, SSD 21-SSD 26 read data from data blocks A21-A26 respectively and sends a read completion message carrying the data to SSD 28. After receiving the read completion messages from SSD 21-SSD 26, SSD 28 retrieves the data from data blocks A21-A26 respectively from the read completion messages of SSD 21-SSD 26. SSD 28 uses RAID 5 to calculate the parity data and data blocks A21-A26 to recover the second data in data block A27. Then, SSD 28 writes the second data to the host's memory and sends a completion message to SSD 21.

[0245] exist Figure 6In the data processing system 30, continuing with the example of the first control disk being SSD 31, the first data block being data block A37, and the first data disk being SSD 38, the first parity block is parity block PA3, the second data block includes data blocks A31-A36, and the second data disk includes SSD 31-SSD 36. The process of SSD 38 executing the second IO instruction is as follows: SSD 38 reads parity data from parity block PA3. SSD 38 sends a third IO instruction to SSD 31-SSD 36 respectively. After receiving the third IO instruction, SSD 31-SSD 36 read data from data blocks A31-A36 respectively and sends a read completion message carrying the data to SSD 38. After receiving the read completion messages from SSD 31-SSD 36, SSD 38 retrieves the data from data blocks A31-A36 respectively from the read completion messages of SSD 31-SSD 36. SSD 38 uses RAID 5 to calculate the parity data and data blocks A31-A36 to recover the second data in data block A37. Then, SSD 38 writes the second data to the host's memory and sends a completion message to SSD 31.

[0246] It should be understood that in the event of a second data failure, besides using the first hard drive containing the first parity block as the first data drive, the first hard drive containing the second data block can also be used as the first data drive. When the first hard drive containing the second data block is used as the first data drive, the process of the first data drive executing the second I / O instruction is as follows: the first data drive acquires the data of the second data block in the first stripe and the parity data of the first parity block in the first stripe; it calculates the data of the second data block and the parity data of the first parity block based on the RAID algorithm to obtain the second data; it writes the second data into the host's memory; and then sends an instruction completion message to the first control drive. The process of the first data drive acquiring data from a second data block is related to whether the first hard drive containing the second data block is the first data drive. If the first hard drive containing the second data block is the first data drive, the first data drive directly reads the data from the second data block on the first data drive. If the first hard drive containing the second data block is not the first data drive, the first data drive sends a third I / O instruction to the first hard drive containing the second data block to obtain the data in the second data block. The first data drive also sends a third I / O instruction to the first hard drive containing the first parity block to obtain the data in the second data block.

[0247] S105: The first control panel generates the processing result of the first IO instruction based on the first processing result of the second IO instruction.

[0248] The first control panel sends the processing result of the first I / O instruction to the host.

[0249] When the first control board generates one second I / O instruction, and the first data board receiving the second I / O instruction also has one instruction completion message (i.e., the first processing result of the second I / O instruction) from the first data board via the high-level interconnect bus, the first control board generates an instruction completion message (i.e., the processing result of the first I / O instruction) indicating successful execution. Subsequently, the first control board sends its generated instruction completion message to the host via the high-level interconnect bus.

[0250] When the number of second I / O instructions generated by the first control disk is multiple, the number of first data disks receiving the second I / O instructions is also multiple. Only after the first control disk receives the instruction completion messages from all the first data disks will it generate an instruction completion message to indicate successful execution (i.e., the processing result of the first I / O instruction) and send the generated instruction completion message to the host.

[0251] If the first I / O instruction is a write instruction, then after the host receives the instruction completion message from the first control disk, it indicates that the processor has also completed the data read / write task corresponding to the first I / O instruction. If the first I / O instruction is a read instruction, then after the host receives the instruction completion message from the first control disk, it reads the second data from the host's memory. In this way, the processor has also completed the data read / write task corresponding to the first I / O instruction.

[0252] In some possible implementations, the first control board caches I / O instructions from the host via an I / O queue, including the first I / O instruction from step S101 above, as well as other I / O instructions (such as the fourth I / O instruction). The I / O queue is a data structure used to manage I / O instructions, typically for storing and scheduling I / O operations. When the first control board receives I / O instructions from the host, it places these instructions into the I / O queue and then processes them according to a scheduling algorithm (such as first-in-first-out, shortest seek time first, shortest service time first, highest response ratio first, etc.).

[0253] In one specific implementation, the first control panel manages I / O instructions from the host through an I / O queue, as follows:

[0254] The first control disk creates a submission queue (SQ) and a completion queue (CQ). The submission queue stores I / O instructions (including the first and fourth I / O instructions) sent by the host to the first control disk, while the completion queue stores I / O instructions that the first control disk successfully executed. Further, a write queue (WQ) and a read queue (RQ) are created within the submission queue according to instruction type. The write queue stores I / O instructions of type write, and the read queue stores I / O instructions of type read.

[0255] When the first control disk has a write queue, a read queue, and a completion queue, if the first control disk receives an I / O instruction of type write from the host, it places the I / O instruction into the write queue; if the first control disk receives an I / O instruction of type read from the host, it places the I / O instruction into the read queue. Furthermore, the first control disk can process I / O instructions in the write queue and the read queue separately using the same or different scheduling algorithms. Upon completion of each I / O instruction, regardless of whether the I / O instruction is in the write queue or the read queue, the first control disk places the processing result of the I / O instruction into the completion queue to notify the host that the I / O instruction has been completed.

[0256] The first control panel uses an I / O queue to manage I / O instructions from the host, allowing it to process multiple I / O instructions in a specified order and priority. This improves I / O operation management and scheduling, enhancing efficiency and performance, and avoiding contention and conflicts between I / O operations. Furthermore, by managing read and write instructions separately, the first control panel can handle read and write operations simultaneously, avoiding contention and conflicts between read and write instructions and improving the parallelism of I / O instruction execution.

[0257] In some possible implementations, after receiving multiple I / O instructions from the host, the first control disk can merge the different I / O instructions according to the processing method and striping indicated by the I / O instructions, and generate a second I / O instruction based on the merged result. Compared to executing each I / O instruction individually, this can reduce the communication overhead of the first control disk and the number of I / O operations on the data disk.

[0258] The following explanation uses the first and fourth I / O instructions as examples of multiple I / O instructions. The first I / O instruction instructs the processing of the first stripe and the return of the processing result to the host; the fourth I / O instruction instructs the processing of the second stripe and the return of the processing result to the host.

[0259] The following will describe the processing of the first I / O command and the fourth I / O command by the first control panel in different scenarios.

[0260] Case 1: Both the first and fourth I / O instructions are write instructions, and both the first and fourth I / O instructions carry the same RAID algorithm or neither carries a RAID algorithm.

[0261] Scenario 2: Both the first and fourth I / O instructions are read instructions, and both the first and fourth I / O instructions carry the same RAID algorithm or neither carries a RAID algorithm.

[0262] Cases 1 and 2 indicate that the first I / O instruction and the fourth I / O instruction indicate the same processing method. Otherwise, the first I / O instruction and the fourth I / O instruction indicate different processing methods.

[0263] Case 3: The LBAs carried by the first IO instruction and the fourth IO instruction correspond to the same stripe.

[0264] Specifically, this includes the following situations: the LBA carried by the first IO instruction and the fourth IO instruction are the same; or, the LBA carried by the first IO instruction and the fourth IO instruction are not completely the same, but the stripe corresponding to the LBA carried by the first IO instruction and the fourth IO instruction is the same stripe; or, the LBA carried by the first IO instruction and the fourth IO instruction are different, but the stripe corresponding to the LBA carried by the first IO instruction and the fourth IO instruction is the same stripe.

[0265] For example, in Figure 8 In the data processing system 10, the LBAs carried by the first IO instruction are LBA 0 and LBA 1, and the LBAs carried by the fourth IO instruction are LBA 1 and LBA 2. Therefore, some of the LBAs carried by the first IO instruction and the fourth IO instruction are the same, while the other part of the LBAs are different. However, LBA 0, LBA 1, and LBA 2 all correspond to stripe S11, so the stripes corresponding to the LBAs carried by the first IO instruction and the fourth IO instruction are the same stripe.

[0266] Case 4: The LBAs carried by the first IO instruction and the fourth IO instruction correspond to different stripes, but at least one of the data disks where the data blocks indicated by the LBAs carried by the first IO instruction and the fourth IO instruction are located is the same.

[0267] For example, in Figure 8In the data processing system 10, the LBA carried by the first IO instruction is LBA 0-LBA 2, and the LBA carried by the fourth IO instruction is LBA 3-LBA 5. Then the stripes corresponding to the LBAs carried by the first IO instruction and the fourth IO instruction are stripe S11 and stripe S12, respectively. However, LBA 0-LBA 2 correspond to data blocks A11-A13 in stripe S11, and data blocks A11-A13 are located on SSD 11, SSD 12, and SSD 13, respectively. Therefore, the data disks where the data blocks indicated by the LBA carried by the first IO instruction are located include SSD 11, SSD 12, and SSD 13. LBA 3-LBA 5 correspond to data blocks B13-B11 in stripe S12, and data blocks B13-B11 are located on SSD 14, SSD 12, and SSD 11, respectively. Therefore, the data disks where the data blocks indicated by the LBA carried by the fourth IO instruction are located include SSD 11, SSD 12, and SSD 14. Thus, the data disks where the data blocks indicated by the LBA carried by the first and fourth IO instructions are located have two identical locations: SSD 11 and SSD 12.

[0268] If the first IO instruction and the fourth IO instruction satisfy any of the following: (1) Case 1 and Case 3; (2) Case 1 and Case 4; (3) Case 2 and Case 3; (4) Case 2 and Case 4; the first control panel merges the first IO instruction and the fourth IO instruction, and then generates the second IO instruction based on the merging result.

[0269] The following explanation uses cases 2 and 3 (3) as examples of the first and fourth I / O instructions satisfying these conditions.

[0270] Continuing with the first I / O instruction carrying LBAs LBA 0 and LBA 1, and the fourth I / O instruction carrying LBAs LBA 1 and LBA 2, the first I / O instruction instructs the data in the data block with LBA 0 and LBA 1 to be stored in the host's memory, and returns the processing result (e.g., successful or unsuccessful execution) to the host; the fourth I / O instruction instructs the data in the data block with LBA 1 and LBA 2 to be stored in the host's memory, and returns the processing result (e.g., successful or unsuccessful execution) to the host. Therefore, the processing method and striping indicated by the first and fourth I / O instructions are the same.

[0271] Therefore, the first control disk merges the first IO instruction and the fourth IO instruction to obtain a merged IO instruction as the merging result. This merged IO instruction instructs the data in the data blocks with LBA 0, LBA 1, and LBA 2 to be stored in the host's memory, and feeds back the processing result (e.g., successful or unsuccessful execution) to the host. The first control disk generates three second IO instructions based on the merged IO instruction. These three second IO instructions carry LBA 0, LBA 1, and LBA 2 respectively, and each instruction instructs the data in the data blocks indicated by its LBA to be stored in the host's memory, feeding back the first processing result (e.g., successful or unsuccessful execution) to the first control disk. Subsequently, the first control disk sends the corresponding second IO instructions to the data disks containing the data blocks indicated by LBA 0, LBA 1, and LBA 2, specifically SSD 11, SSD 12, and SSD 13, and waits to receive instruction completion messages returned by SSD 11, SSD 12, and SSD 13 after executing their respective second IO instructions. After receiving instruction completion messages from SSD 11, SSD 12 and SSD 13, the first control disk generates an instruction completion message indicating that both the first and fourth I / O instructions were executed successfully, and then sends the generated instruction completion message to the host.

[0272] In summary, after receiving the first and fourth I / O instructions from the host, assuming the processing methods and striping of the first and fourth I / O instructions are identical, if the first control disk first merges the first and fourth I / O instructions and then controls the read / write process of the data disk based on the merged result, the first control disk only needs to generate three second I / O instructions, receive three instruction completion messages, generate one instruction completion message, and send it to the host. If the first control disk does not merge the first and fourth I / O instructions but executes them separately, then for the first I / O instruction, the first control disk needs to generate two second I / O instructions, receive two instruction completion messages, generate one instruction completion message, and send it to the host; for the fourth I / O instruction, the first control disk needs to generate two second I / O instructions, receive two instruction completion messages, generate one instruction completion message, and send it to the host. Therefore, by merging the first and fourth I / O instructions before processing, the first control disk can generate one fewer second I / O instruction, receive one fewer instruction completion message, generate one fewer instruction completion message, and send one fewer instruction completion message to the host, thus reducing the communication overhead of the first control disk. In addition, a second IO instruction means one IO operation. Therefore, the number of second IO instructions is reduced, and the number of IO operations of the data disk used to execute the second IO instructions is reduced accordingly. For example, in the process of completing the first IO instruction and the fourth IO instruction, the SSD12 only needs to read the data in the data block indicated by LBA 1 once, and does not need to read it repeatedly.

[0273] It should be noted that when the above Figure 7 The data processing methods applied to the aforementioned Figure 1 When using the data processing system 10, the host in steps S101 and S102 above can be... Figure 1 In the data processing system 10, the host 11, the first control disk, the first data disk, the second data disk, and the third data disk can be... Figure 1 The first hard drive in the data processing system 10, the high-speed interconnect bus can be Figure 1 The high-speed interconnect bus 13 in the data processing system 10.

[0274] When the above Figure 7 The data processing methods applied to the aforementioned Figure 3 In the data processing system 20, the processor in steps S101 and S102 above can be... Figure 3 In the data processing system 20, the host 23, the first control disk, the first data disk, the second data disk, and the third data disk can be... Figure 3 The hard disk 24 in the data processing system 20, the high-speed interconnect bus can be Figure 3 The high-speed interconnect bus 22 in the data processing system 20.

[0275] When the above Figure 7 The data processing methods applied to the aforementioned Figure 5 In the data processing system 30, the processor in steps S101 and S102 above can be... Figure 5 In the data processing system 30, the host 34, the first control disk, the first data disk, the second data disk, and the third data disk can be... Figure 5 The hard disk 35 in the data processing system 30, the high-speed interconnect bus can be Figure 5 The high-speed interconnect bus 33 in the data processing system 30.

[0276] In summary, in a data processing system, after striping the data stored on the hard drive, a control disk (i.e., the first control disk) is configured for the first stripe. The first control disk controls the data read / write process within the first stripe. This technical solution supports offloading all data read / write tasks involving the first stripe that would otherwise be performed by the host to the first control disk. Specifically, it offloads the host's control over the data read / write process within the first stripe to the first control disk. This reduces the interaction between the host and the hard drive, thereby reducing the host's workload and freeing up host resources.

[0277] In addition, this technical solution is described above. Figure 1 , Figure 3 or Figure 5 The data processing system architecture provides a data processing method for CNC separation suitable for different scenarios. The first control panel sends a second I / O instruction to the first data panel, instructing it to transfer first or second data between itself and the host's memory. This achieves separation of the data plane and the control plane. Control flow occurs between the first control panel and the first data panel, while data flow occurs between the first data panel and the host's memory. This enables centralized global control of the control plane and high-performance data transmission. In this CNC separation scenario, the host is unaware of the specific execution process of data read / write tasks. The host only needs to delegate the data read / write tasks to the first control panel. The host does not need to participate in how the first control panel completes the task; it considers itself to have completed the task upon receiving a notification from the first control panel that the task has been completed.

[0278] Furthermore, this technical solution supports the first control disk controlling the first data disk to perform diverse tasks, including reading data from the first data disk, storing data on the first data disk, and performing calculations on the data. Specifically, in an application scenario where the data processing system stores data based on the RAID algorithm, when the first control disk receives a task to store the first data, it instructs the hard drive containing the first data block to store the first data, and simultaneously instructs the hard drive responsible for calculating the parity data of the first stripe to update the parity data of the first stripe using the RAID algorithm. This ensures that if the data in the first stripe is corrupted, the corrupted data can be recovered promptly based on the parity data. When the first control disk receives a task to read the second data, if the second data is normal, the first control disk instructs the hard drive containing the first data block to read the second data and provide it to the host; if the second data is faulty, the first control disk can also instruct one of the first hard drives to calculate the second data based on the RAID algorithm and then provide the recovered second data to the host. As can be seen, in this technical solution, the first data disk has data calculation and processing capabilities and can perform calculation and processing tasks other than error detection and garbage collection.

[0279] This application also provides a chip system including a controller and a power supply circuit. The power supply circuit supplies power to the controller, and the controller performs the aforementioned operations. Figure 7 The data processing method involves operation steps executed by the first control panel, or the controller is used to execute the aforementioned steps. Figure 7 The data processing method involves operations performed by the first data disk. For simplicity, these steps will not be elaborated upon here. The controller can be implemented using computing devices or AI chips such as CPUs, DPUs, GPUs, NPUs, XPUs, system-on-chips (SoCs), offloading cards, and accelerator cards.

[0280] See Figure 10 , Figure 10 This is a schematic diagram of the structure of a storage device provided in an embodiment of this application. Figure 10 As shown, the storage device 1000 provided in this embodiment includes: a bus 1001, a processor 1002, a memory 1003, and a communication interface 1004. The processor 1002, the memory 1003, and the communication interface 1004 communicate with each other via the bus 1001. The storage device 1000 can be a server or a terminal device. It should be understood that this application does not limit the number of processors and memories in the storage device 1000.

[0281] Bus 1001 can be a Peripheral Component Interconnect Express (PCIe) bus, an Extended Industry Standard Architecture (EISA) bus, a Unified Bus (Ubus or UB), a Compute Express Link (CXL) bus, a Cache Coherent Interconnect for Accelerators (CCIX) bus, etc. The Unified Bus is also known as the Lingqu Bus. Buses can be divided into address buses, data buses, control buses, etc. For ease of representation, Figure 10 The bus 1001 may be represented by a single line, but this does not mean that there is only one bus or one type of bus. The bus 1001 may include a path for transmitting information between various components of the storage device 1000 (e.g., memory 1003, processor 1002, communication interface 1004).

[0282] The processor 1002 may include any one or more of the following computing devices: central processing unit (CPU), graphics processing unit (GPU), microprocessor (MP) or digital signal processor (DSP), ASIC, FPGA, CPLD, NPU, SoC, offload card, accelerator card, etc.

[0283] Memory 1003 may include volatile memory, such as random access memory (RAM). Processor 1002 may also include non-volatile memory, such as read-only memory (ROM), flash memory, hard disk drive (HDD), or solid state drive (SSD). Furthermore, memory 1003 may also be implemented using storage class memory (SCM), phase change memory (PCM), or other types of storage media.

[0284] It is worth noting that the same type of storage medium can be configured in the same computing device to realize the function of memory 1003, or two or more types of storage media can be configured to realize the function of memory 1003. This application does not limit this.

[0285] The memory 1003 stores executable program code, and the processor 1002 executes the executable program code to implement the aforementioned. Figure 7 The data processing method involves operation steps executed by the first control panel. That is, the memory 1003 stores instructions for executing the data processing method.

[0286] Alternatively, the memory 1003 stores executable program code, and the processor 1002 executes the executable program code to implement the aforementioned functionality. Figure 7 The operation steps in the data processing method are executed by the first data disk. That is, the memory 1003 stores the instructions for executing the data processing method.

[0287] The communication interface 1004 uses transceiver modules such as, but not limited to, network interface cards and transceivers to enable communication between the storage device 1000 and other storage devices or communication networks.

[0288] This application also provides a computer program product containing instructions. This computer program product may be a software or program product containing instructions, capable of running on a storage device or stored on any usable medium. When the computer program product runs on the storage device, it causes the storage device to perform the aforementioned actions. Figure 7 The steps executed by the first control panel in the data processing method, or the aforementioned steps, are performed. Figure 7 The steps performed by the first data disk in the data processing method.

[0289] This application also provides a computer-readable storage medium. The computer-readable storage medium can be any usable medium that a storage device can store, or a data storage device such as a data center containing one or more usable media. The usable medium can be a magnetic medium (e.g., floppy disk, hard disk, magnetic tape), an optical medium (e.g., DVD), or a semiconductor medium (e.g., solid-state drive). The computer-readable storage medium includes instructions that direct the storage device to perform the aforementioned actions. Figure 7 The steps executed by the first control panel in the data processing method, or the aforementioned steps, are performed. Figure 7 The steps performed by the first data disk in the data processing method.

[0290] It should be understood that in the embodiments of this application, "when," "...when," and "if" all refer to the device making corresponding processing under certain objective circumstances, and are not time-limited, nor do they require the device to make a judgment action, nor do they imply any other limitations.

[0291] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of this application, and are not intended to limit them. Although this application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the protection scope of the technical solutions of the embodiments of this application.

Claims

1. A data processing system, characterized in that, It includes multiple hard drives, which are used to provide multiple stripes. The data blocks and parity blocks in each stripe come from different hard drives. The data blocks are physical blocks used to store data, and the parity blocks are physical blocks used to store parity data. The parity data is used to verify the data. The plurality of stripes includes a first stripe, wherein the data blocks and parity blocks in the first stripe are provided by a plurality of first hard disks, and the plurality of first hard disks belong to the plurality of hard disks; The first control disk is used to control the data read and write process in the first stripe, and the first control disk is any one of the plurality of first hard disks.

2. The system according to claim 1, characterized in that, The data processing system includes a host computer. The first control panel is used to receive a first input / output (IO) instruction from the host computer and send a second IO instruction to the first data panel; Wherein, the first IO instruction is used to instruct the processing of the first stripe and to feed back the processing result to the host; the second IO instruction is generated based on the first IO instruction and is used to instruct the processing of the first data block in the first stripe and to feed back the first processing result to the first control disk; the first data disk belongs to the plurality of first hard disks and is different from the first control disk.

3. The system according to claim 2, characterized in that, The first control board and the first data board are deployed on the same device, or the first control board and the first data board are deployed on different devices. Specifically, the first control board is used to send the second IO instruction to the first data board through a high-speed interconnect bus.

4. The system according to claim 2 or 3, characterized in that, The first data disk is used to read first data from the host's memory after receiving the second IO instruction, write the first data into the first data block on the first data disk, and then send an instruction completion message to the first control disk.

5. The system according to claim 2 or 3, characterized in that, The first data disk is used to read first data from the host's memory after receiving the second IO instruction; obtain data of the second data block in the first stripe; and calculate the first data and the data of the second data block based on the disk array RAID algorithm to obtain the parity data of the first parity block in the first stripe. Wherein, the data block used to store the first data in the first strip is the first data block, and the second data block is different from the first data block.

6. The system according to claim 5, characterized in that, The first data disk is specifically used to send a third IO instruction to the second data disk and receive the data of the second data block sent by the second data disk after completing the third IO instruction; The second data disk belongs to the plurality of first hard disks, and the second data disk is different from the first data disk.

7. The system according to claim 5 or 6, characterized in that, The first data disk is also used to send the verification data of the first verification block to the third data disk; The third data disk is different from the first data disk.

8. The system according to claim 2 or 3, characterized in that, The first data block comes from the first data disk. The first data disk is used to read second data from the first data block after receiving the second IO instruction, write the second data into the memory of the host, and then send an instruction completion message to the first control disk.

9. The system according to claim 2 or 3, characterized in that, The first data block does not belong to the first data disk. The first data disk is used to, after receiving the second IO instruction, acquire the data of the second data block in the first stripe and the parity data of the first parity block in the first stripe; calculate the data of the second data block and the parity data of the first parity block based on the RAID algorithm to obtain the second data; write the second data into the memory of the host; and then send an instruction completion message to the first control disk. The second data is stored on the first data block, and the second data block is different from the first data block.

10. The system according to claim 2, characterized in that, The first control disk is also used to receive a fourth IO instruction from the host before sending the second IO instruction to the first data disk; when the processing method and striping indicated by the fourth IO instruction and the first IO instruction are the same, the first IO instruction and the fourth IO instruction are merged, and the second IO instruction is generated according to the merging result. The fourth IO instruction is used to instruct the processing of the second stripe and to feed back the processing result to the host.

11. The system according to claim 10, characterized in that, The first control panel is used to cache I / O instructions from the host through an I / O queue, including the first I / O instruction and the fourth I / O instruction.

12. A data processing method, characterized in that, The method, applied to the data processing system as described in any one of claims 1-11, comprises: The first control panel receives a first input / output (IO) instruction from the host in the data processing system, wherein the first IO instruction is used to instruct the processing of the first stripe and to feed back the processing result to the host. The first control disk sends a second I / O instruction to the first data disk, wherein the second I / O instruction is generated based on the first I / O instruction, and the second I / O instruction is used to instruct the processing of the first data block in the first stripe and to feed back the first processing result to the first control disk; the first data disk belongs to the plurality of first hard disks, and the first data disk is different from the first control disk.

13. The method according to claim 12, characterized in that, The first control panel and the first data panel are deployed on the same device, or the first control panel and the first data panel are deployed on different devices. The first control disk sends a second I / O instruction to the first data disk, including: The first control board sends the second I / O command to the first data board via a high-speed interconnect bus.

14. The method according to claim 12 or 13, characterized in that, The method further includes: After receiving the second IO instruction, the first data disk reads the first data from the host's memory, writes the first data into the first data block on the first data disk, and then sends an instruction completion message to the first control disk.

15. The method according to claim 12 or 13, characterized in that, The method further includes: After receiving the second IO instruction, the first data disk reads the first data from the host's memory; obtains the data of the second data block in the first stripe; and calculates the first data and the data of the second data block based on the disk array RAID algorithm to obtain the parity data of the first parity block in the first stripe. Wherein, the data block used to store the first data in the first strip is the first data block, and the second data block is different from the first data block.

16. The method according to claim 15, characterized in that, The step of obtaining data from the second data block in the first strip includes: Send a third I / O instruction to the second data disk, and receive the data of the second data block sent by the second data disk after completing the third I / O instruction; The second data disk belongs to the plurality of first hard disks, and the second data disk is different from the first data disk.

17. The method according to claim 15 or 16, characterized in that, The method further includes: The first data disk sends the verification data of the first verification block to the third data disk; The third data disk belongs to the plurality of first hard disks, and the third data disk is different from the first data disk.

18. The method according to claim 12 or 13, characterized in that, The first data block comes from the first data disk. The method further includes: After receiving the second IO instruction, the first data disk reads the second data from the first data block, writes the second data into the host's memory, and then sends an instruction completion message to the first control disk.

19. The method according to claim 12 or 13, characterized in that, The first data block does not belong to the first data disk. The method further includes: After receiving the second IO instruction, the first data disk acquires the data of the second data block in the first stripe and the parity data of the first parity block in the first stripe; it calculates the data of the second data block and the parity data of the first parity block based on the RAID algorithm to obtain the second data; it writes the second data into the memory of the host; and then sends an instruction completion message to the first control disk. The second data is stored on the first data block, and the second data block is different from the first data block.

20. The method according to claim 12, characterized in that, Before the first control disk sends the second I / O command to the first data disk, the method further includes: The first control panel receives a fourth IO instruction from the host, wherein the fourth IO instruction is used to instruct the processing of the second stripe and to feed back the processing result to the host. When the processing method and stripe indicated by the fourth IO instruction and the first IO instruction are the same, the first control panel merges the first IO instruction and the fourth IO instruction, and generates the second IO instruction based on the merging result.

21. The method according to claim 20, characterized in that, The first control panel caches I / O instructions from the host through an I / O queue, including the first I / O instruction and the fourth I / O instruction.

22. A data processing method, characterized in that, Applied to a first control disk, wherein the first control disk is a hard disk in the data processing system as described in any one of claims 1-11, the method comprises: The first control panel receives a first input / output (IO) instruction from the host in the data processing system, wherein the first IO instruction is used to instruct the processing of the first stripe and to feed back the processing result to the host. The first control disk sends a second I / O instruction to the first data disk, wherein the second I / O instruction is generated based on the first I / O instruction, and the second I / O instruction is used to instruct the processing of the first data block in the first stripe and to feed back the first processing result to the first control disk; the first data disk belongs to the plurality of first hard disks, and the first data disk is different from the first control disk.

23. The method according to claim 22, characterized in that, The first control panel and the first data panel are deployed on the same device, or the first control panel and the first data panel are deployed on different devices. The first control disk sends a second I / O instruction to the first data disk, including: The first control board sends the second I / O command to the first data board via a high-speed interconnect bus.

24. The method according to claim 22, characterized in that, Before the first control disk sends the second I / O command to the first data disk, the method further includes: The first control panel receives a fourth IO instruction from the host, wherein the fourth IO instruction is used to instruct the processing of the second stripe and to feed back the processing result to the host. When the processing method and stripe indicated by the fourth IO instruction and the first IO instruction are the same, the first control panel merges the first IO instruction and the fourth IO instruction, and generates the second IO instruction based on the merging result.

25. The method according to claim 24, characterized in that, The first control panel caches I / O instructions from the host through an I / O queue, including the first I / O instruction and the fourth I / O instruction.

26. A data processing method, characterized in that, The first data disk is a hard disk in the data processing system as described in any one of claims 1-11. The plurality of first hard disks include a first control disk and a first data disk. The first control disk is different from the first data disk. The first control disk is used to receive a first input / output (IO) instruction from the host in the data processing system. The first IO instruction is used to instruct the processing of the first stripe and to feed back the processing result to the host. The method includes: The first data disk receives a second I / O command from the first control disk; After completing the second I / O instruction, the first data disk sends an instruction completion message to the first control disk; The second IO instruction is generated based on the first IO instruction and is used to instruct the processing of the first data block in the first stripe.

27. The method according to claim 26, characterized in that, The first control panel and the first data panel are deployed on the same device, or the first control panel and the first data panel are deployed on different devices. The first data disk receives a second I / O command from the first control disk, including: The first data disk receives the second IO command from the first control disk via a high-speed interconnect bus.

28. The method according to claim 26 or 27, characterized in that, After the first data disk receives the second I / O instruction from the first control disk, and before the first data disk sends an instruction completion message to the first control disk after completing the second I / O instruction, the method further includes: The first data disk reads first data from the host's memory; The first data disk writes the first data into the first data block on the first data disk.

29. The method according to claim 26 or 27, characterized in that, After the first data disk receives the second I / O instruction from the first control disk, and before the first data disk sends an instruction completion message to the first control disk after completing the second I / O instruction, the method further includes: The first data disk reads first data from the host's memory; The first data disk acquires data from the second data block in the first stripe; The first data disk calculates the data of the first data and the second data block based on the disk array RAID algorithm to obtain the parity data of the first parity block in the first stripe; Wherein, the data block used to store the first data in the first strip is the first data block, and the second data block is different from the first data block.

30. The method according to claim 29, characterized in that, The first data disk acquires data from the second data block in the first stripe, including: The first data disk sends a third I / O command to the second data disk; The first data disk receives the data of the second data block sent by the second data disk after completing the third IO instruction; The second data disk belongs to the plurality of first hard disks, and the second data disk is different from the first data disk.

31. The method according to claim 29 or 30, characterized in that, The method further includes: The first data disk sends the verification data of the first verification block to the third data disk; The third data disk belongs to the plurality of first hard disks, and the third data disk is different from the first data disk.

32. The method according to claim 26 or 27, characterized in that, The first data block comes from the first data disk. After the first data disk receives the second I / O instruction from the first control disk, and before the first data disk sends an instruction completion message to the first control disk after completing the second I / O instruction, the method further includes: The first data disk reads the second data from the first data block and writes the second data into the host's memory.

33. The method according to claim 26 or 27, characterized in that, The first data block does not belong to the first data disk. After the first data disk receives the second I / O instruction from the first control disk, and before the first data disk sends an instruction completion message to the first control disk after completing the second I / O instruction, the method further includes: The first data disk acquires data from the second data block in the first stripe and verification data from the first verification block in the first stripe; The first data disk calculates the second data based on the data of the second data block and the parity data of the first parity block using a RAID algorithm; The first data disk writes the second data into the host's memory; The second data is stored on the first data block, and the second data block is different from the first data block.

34. A chip, characterized in that, The device includes a controller and a power supply circuit, the power supply circuit being used to supply power to the controller, the controller being used to perform the method as described in any one of claims 22-25, or to perform the method as described in any one of claims 26-33.

35. A storage device, characterized in that, The storage device includes a controller and a storage medium; The controller is configured to execute instructions stored in the storage medium to cause the storage device to perform the method as described in any one of claims 22-25, or to perform the method as described in any one of claims 26-33.

Images