RAID acceleration method based on SIMD instruction

Abstract

The invention belongs to the technical field of storage systems, and particularly relates to an RAID acceleration method based on an SIMD instruction. Comprising the steps of data writing, data reading and execution of data recovery through a CPU service thread, the advantages of an SIMD instruction set in the aspect of data parallel processing are fully utilized, and data read-write operation is remarkably accelerated by optimizing the data processing flow of the RAID. The innovation not only improves the overall performance of the storage system, but also provides powerful technical support for the security and high efficiency of data storage, and has important practical application value and wide development prospect.

Description

Technical Field

[0001] The invention belongs to the technical field of storage systems, and in particular relates to a RAID acceleration method based on SIMD instructions. Background Art

[0002] RAID (Redundant Array of Independent Disks) is a data storage technology that combines multiple physical hard disk drives to provide higher data transfer rates, larger storage capacity, data redundancy, and fault tolerance.

[0003] Data parity is a key technology of RAID. It provides data security by verifying data, can detect data errors, and reconstruct data under the premise of capability. Since a large number of verification operations are required for each data read and write, data verification requires a high computing speed of the computer. At present, the main method is to use DMA hardware or RAID controller to perform verification operations. In order to speed up the verification operation, Lu Jianhai and others invented a Raid acceleration method based on GPU operation (CN118312099A), which uses the parallel processing capability of GPU to improve the calculation speed of data verification and reduce the calculation cost.

[0004] However, with the rapid development of CPU technology, the number of its cores has continued to increase. At the same time, advanced SIMD (Single Instruction Multiple Data) instruction sets such as AVX in the X86 architecture and Neon in the ARM architecture have also been widely introduced. SIMD (Single Instruction Multiple Data) is a parallel technology. NEON is a SIMD instruction set extension proprietary to the ARM architecture, mainly used in applications such as multimedia processing, signal processing, and numerical calculations, and aims to improve the data parallel processing capabilities of the processor. The NEON instruction set allows the processor to process multiple data points at a time, significantly improving computing efficiency. The application of these technologies has greatly improved the computing power and parallel processing capabilities of the CPU. Summary of the invention

[0005] The purpose of the present invention is to utilize SIMD instructions to optimize existing data verification operations and to provide a RAID acceleration method based on SIMD instructions.

[0006] In order to achieve the above object, the technical solution of the present invention is as follows:

[0007] A RAID acceleration method based on SIMD instructions includes data writing, data reading, and data recovery using a CPU service thread. The specific steps are as follows:

[0008] (1) Data writing:

[0009] (1.1): The client splits the data D0 to be written into i data blocks D of fixed size 01 , D 02 ,……,D 0i ;

[0010] (1.2): The client checks whether there are any unwritten data blocks; if there are any unwritten data blocks, execute step (1.3); otherwise, execute step (1.9);

[0011] (1.3): For the data block to be written, the client performs the data block write operation;

[0012] (1.4): The client submits the log; each CPU maintains a log queue, and the CPU on which the data block falls will perform data verification calculation;

[0013] (1.5): After receiving each queue log submission request in step (1.4), the CPU service thread obtains log data from the log queue;

[0014] (1.6): The CPU service thread uses SIMD instructions to perform data verification calculations and generate the corresponding verification block P0;

[0015] (1.7): The CPU service thread writes both the data block and the check block to the disk;

[0016] (1.8): After the write operation is completed, the CPU service thread will send a message to the client to inform it that the log is completed;

[0017] (1.9): After all data blocks are written, the client will end the data writing process;

[0018] (2) Data reading:

[0019] (2.1): The client splits the data to be read into multiple data blocks to be read;

[0020] (2.2): The client initiates a data block read request and reads each data block separately;

[0021] (2.3): The client checks whether the data block to be read exists in the cache; if the cache does not hit, execute step (2.4); otherwise, execute step (2.8);

[0022] (2.4): The client notifies the CPU service thread, and the CPU service thread reads the corresponding data block;

[0023] (2.5): After receiving the data block read request from the client, the CPU service thread will check whether the data block is intact; if the data block is damaged, execute step (2.6); otherwise, execute step (2.7);

[0024] (2.6): For damaged data blocks, the CPU service thread will read the undamaged data blocks and check blocks in the stripe and use SIMD instructions to restore the data blocks;

[0025] (2.7): The CPU service thread transmits the requested data block to the client, notifying that the data block reading is completed;

[0026] (2.8): The client indicates that the data block is ready;

[0027] (2.9): The client checks whether there are any unread data blocks; if there are any unread data blocks, repeat steps (2.2)-(2.8); otherwise, the data reading process ends;

[0028] (3) After a disk fails and a replacement disk is inserted, the CPU service thread will perform data recovery:

[0029] (3.1) The client notifies the CPU service thread to perform data recovery;

[0030] (3.2): The CPU service thread checks whether the data recovery work is completed; if the data in the failed disk has not been completely recovered, execute step (3.3); otherwise, execute step (3.6);

[0031] (3.3): The CPU service thread reads the data blocks and check blocks in multiple stripes;

[0032] (3.4): The CPU service thread executes SIMD instructions to recover the lost data blocks;

[0033] (3.5): The CPU service thread writes the recovered data blocks to the new disk, and then executes step (3.2);

[0034] (3.6): After all data is restored, the CPU service thread will send a message to the client to inform it that the data recovery has been completed.

[0035] The beneficial effect of the present invention is that: the present invention proposes an innovative solution, namely, using the SIMD instruction set to accelerate the data verification process of RAID (Redundant Array of Independent Disks). By cleverly using the significant advantages of the SIMD instruction set in parallel processing, the data processing flow of RAID is deeply optimized without relying on external GPU or RAID acceleration hardware. This optimization strategy not only significantly accelerates the reading and writing operations of data, but also greatly improves the overall performance of the storage system. It has the advantages of high performance and low cost. The method of the present invention makes full use of the advantages of the SIMD instruction set in data parallel processing, and achieves significant acceleration of data reading and writing operations by optimizing the data processing flow of RAID. This innovation not only improves the overall performance of the storage system, but also provides strong technical support for the security and efficiency of data storage, and has important practical application value and broad development prospects. BRIEF DESCRIPTION OF THE DRAWINGS

[0036] Figure 1 A schematic diagram of a data writing process of the present invention;

[0037] Figure 2 It is a schematic diagram of the data reading process of the present invention;

[0038] Figure 3 It is a schematic diagram of the data recovery process of the present invention;

[0039] Figure 4 It is a schematic diagram of the implementation of the present invention under the ARM platform SPDK. DETAILED DESCRIPTION

[0040] The present invention is further described below by way of embodiments in conjunction with the accompanying drawings.

[0041] Embodiment 1: Hardware platform: The hardware implementation of the present invention is based on the ARMv8 multi-core (8-core) platform. The ARMv8 architecture supports efficient parallel processing and advanced instruction sets, such as Neon, which provides a solid foundation for utilizing SIMD (single instruction multiple data) technology. The multi-core characteristics of the ARMv8 multi-core platform further enhance the parallel processing capability, enabling computationally intensive tasks such as data verification to be performed more efficiently.

[0042] Software environment: In terms of software, the present invention runs on the SPDK (Storage Performance Development Kit) storage system on the Ubuntu 22.04 operating system. SPDK is an open source storage performance development toolkit that provides high-performance storage solutions and supports a variety of storage protocols and hardware. As an operating system, Ubuntu 22.04 provides a stable operating environment and rich software resources, providing good support for the implementation of SPDK and the present invention.

[0043] The present invention uses this method to implement the Neon data verification acceleration module in the framework of SPDK, such as Figure 4 By cleverly using Neon instructions, the present invention deeply optimizes the data processing flow of RAID, significantly accelerates the data reading and writing operations, and improves the overall performance of nvmef-tgt (nvme over fabric storage service) based on RAID.

[0044] like Figure 1 The data writing process shown is as follows:

[0045] Step 1: The client splits the data D0 to be written into 64KB data blocks D 01 , D 02 ,……,D 0i ;

[0046] Step 2: The client checks whether there are any unwritten data blocks; if there are any unwritten data blocks, execute step 3; otherwise, execute step 9;

[0047] Step 3: For the data block to be written, the client performs a data block write operation;

[0048] Step 4: The client submits the log; each CPU maintains a log queue, and the data block enters a log queue according to the hash of the stripe number to which it belongs. The corresponding CPU uses the Neon instruction to perform data verification calculation;

[0049] Step 5: After receiving the log submission request, the CPU service thread obtains the log data from the log queue;

[0050] Step 6: The CPU service thread uses the Neon instruction to perform data verification calculation and generate the corresponding verification block P0;

[0051] Step 7: The CPU service thread writes the data block and the check block to the disk;

[0052] Step 8: After the write operation is completed, the CPU service thread will send a message to the client to inform it that the log is completed;

[0053] Step 9: After all data blocks are written, the client ends the data writing process.

[0054] like Figure 2 The data reading process shown is as follows:

[0055] Step 1: The client splits the data to be read into multiple data blocks to be read;

[0056] Step 2: The client initiates a data block read request and reads each data block separately;

[0057] Step 3: The client checks whether the data block to be read is in the cache; if the cache does not hit, execute step 4; otherwise, execute step 8;

[0058] Step 4: The client notifies the CPU service thread to read the corresponding data block;

[0059] Step 5: After receiving the data block read request from the client, the CPU service thread will check whether the data block is intact; if the data block is damaged, execute step 6; otherwise, execute step 7;

[0060] Step 6: For the damaged data block, the CPU service thread reads the undamaged data block and check block in the stripe, and uses the Neon instruction to perform check calculation to restore the data block;

[0061] Step 7: The CPU service thread transmits the requested data block to the client, notifying that the data block reading is completed;

[0062] Step 8: The client indicates that the data block is ready;

[0063] Step 9: The client checks whether there are any unread data blocks; if there are any unread data blocks, steps 2 to 8 are repeated; otherwise, the data reading process ends.

[0064] like Figure 3 As shown in the figure, after a disk fails and a replacement disk is inserted, the CPU service thread will perform data recovery. The specific steps are as follows:

[0065] Step 1: The client notifies all CPU service threads to perform data recovery;

[0066] Step 2: The CPU service thread checks whether its data recovery work is completed; if the data in the failed disk has not been completely recovered, execute step 3; otherwise, execute step 6;

[0067] Step 3: The CPU service thread reads the data blocks and check blocks in multiple stripes;

[0068] Step 4: The CPU service thread executes the Neon instruction to perform verification and recovery calculations to recover the lost data blocks;

[0069] Step 5: The CPU service thread writes the recovered data blocks to the new disk, and then executes step 2;

[0070] Step 6: After all data is restored, the CPU service thread will send a message to the client to inform it that the data recovery has been completed.

[0071] Compared with the previous method of using CPU integer instructions for verification, the performance of data verification using Neon instructions in the present invention is improved by more than 30%. This significant performance improvement is due to the SIMD characteristics and parallel processing capabilities of Neon instructions, which enable data verification tasks to be completed more quickly, thereby improving the throughput and response speed of the storage system.

[0072] Other possible implementation schemes: Although the embodiments of the present invention are based on the Neon instructions of the ARM platform, in other implementation schemes, advanced SIMD instruction sets such as the AVX instructions of the x86 platform can also be used. These instruction sets also have powerful parallel processing capabilities and can be used to optimize the data verification process of RAID. Therefore, the implementation scheme of the present invention is not limited to a specific hardware platform or instruction set, but can be flexibly selected and adjusted according to actual needs and technical background.

Claims

1. A RAID acceleration method based on SIMD instructions, characterized in that: Including data writing, data reading, and data recovery using CPU service threads. The specific steps are as follows: (1) Data writing: (1.1): The client splits the data D0 to be written into i data blocks D of fixed size 01 , D 02 ,……,D 0i ; (1.2): The client checks whether there are any unwritten data blocks; if there are any unwritten data blocks, execute step (1.3); otherwise, execute step (1.9); (1.3): For the data block to be written, the client performs the data block write operation; (1.4): The client submits the log; each CPU maintains a log queue, and the CPU on which the data block falls will perform data verification calculation; (1.5): After receiving each queue log submission request in step (1.4), the CPU service thread obtains log data from the log queue; (1.6): The CPU service thread uses SIMD instructions to perform data verification calculations and generate the corresponding verification block P0; (1.7): The CPU service thread writes both the data block and the check block to the disk; (1.8): After the write operation is completed, the CPU service thread will send a message to the client to inform it that the log is completed; (1.9): After all data blocks are written, the client will end the data writing process; (2) Data reading: (2.1): The client splits the data to be read into multiple data blocks to be read; (2.2): The client initiates a data block read request and reads each data block separately; (2.3): The client checks whether the data block to be read exists in the cache; If the cache misses, execute step (2.4); otherwise, execute step (2.8); (2.4): The client notifies the CPU service thread, and the CPU service thread reads the corresponding data block; (2.5): After receiving the data block read request from the client, the CPU service thread will check whether the data block is intact; if the data block is damaged, execute step (2.6); otherwise, execute step (2.7); (2.6): For damaged data blocks, the CPU service thread will read the undamaged data blocks and check blocks in the stripe and use SIMD instructions to restore the data blocks; (2.7): The CPU service thread transmits the requested data block to the client, notifying that the data block reading is completed; (2.8): The client indicates that the data block is ready; (2.9): The client checks whether there are any unread data blocks; if there are any unread data blocks, repeat steps (2.2)-(2.8); otherwise, the data reading process ends; (3) After a disk fails and a replacement disk is inserted, the CPU service thread will perform data recovery: (3.1) The client notifies the CPU service thread to perform data recovery; (3.2): The CPU service thread checks whether the data recovery work is completed; if the data in the failed disk has not been completely recovered, execute step (3.3); otherwise, execute step (3.6); (3.3): The CPU service thread reads the data blocks and check blocks in multiple stripes; (3.4): The CPU service thread executes SIMD instructions to recover the lost data blocks; (3.5): The CPU service thread writes the recovered data blocks to the new disk, and then executes step (3.2); (3.6): After all data is restored, the CPU service thread will send a message to the client to inform it that the data recovery has been completed.

Images