Data processing system and method

Abstract

The present application provides a data processing system and method, which can be applied to the technical field of storage. The data processing system comprises a processor and a plurality of solid state drives. Physical blocks provided by the plurality of solid state drives constitute a first storage space. The processor stores a first set of addresses. Each address in the first set of addresses is used for indicating a physical block in the first storage space. Each solid state drive stores a second set of addresses. The first set of addresses comprises the second set of addresses. A first solid state drive is used to, after receiving an IO instruction from the processor, determine a target solid state drive from the plurality of solid state drives on the basis of a target address in the IO instruction and the second set of addresses stored on the first solid state drive. The IO instruction is used for instructing to process a storage space indicated by the target address. The data processing system supports the processor to distribute a large number of tasks to the solid state drives for execution, thereby reducing the workload of the processor and improving parallel execution efficiency of a plurality of tasks.

Description

A data processing system and method

[0001] This application claims priority to Chinese Patent Application No. 202411181262.2, filed with the China National Intellectual Property Administration on August 26, 2024, entitled "A Data Processing System and Method", the entire contents of which are incorporated herein by reference. Technical Field

[0002] This application relates to the field of storage technology, and in particular to a data processing system and method. Background Technology

[0003] Servers consist of processors and solid-state drives (SSDs). SSDs store data, while processors handle the data on the SSDs, performing tasks such as management, reading, writing, and computation. Typically, a server has 1-8 processors and dozens to thousands of SSDs. When a server receives a large number of processing tasks to process data on different SSDs, these tasks are shared among all the processors in the server. Each processor can only execute the tasks assigned to it one by one. Therefore, the performance bottleneck in executing processing tasks lies with the processor; both execution efficiency and the number of tasks that can be executed are limited by the processor's computing power.

[0004] To enhance a server's ability to process data from SSDs, an expansion card—a Redundant Array of Independent Disks (RAID) card—can be installed. RAID cards are specifically designed to manage disk arrays composed of multiple SSDs, providing more efficient data transfer and processing capabilities, thereby accelerating SSD operations. Typically, depending on the server's storage needs and the complexity of the RAID card configuration, the number of RAID cards in a server ranges from 1 to 9. When a server has RAID cards installed, and it receives a large number of processing tasks to handle data from different SSDs, the processor distributes these tasks to the RAID cards, which then share the workload. Each RAID card can only execute the tasks assigned to it one by one. Therefore, for servers with RAID cards, the performance bottleneck for processing tasks lies with the RAID cards themselves; both execution efficiency and the number of tasks that can be executed are limited by the capabilities of the RAID cards.

[0005] In conclusion, in a server, the number of processors and RAID cards is far less than the number of SSDs. Therefore, using a small number of processors or RAID cards to handle a large number of SSDs will inevitably become a bottleneck for processing performance. Summary of the Invention

[0006] To address the aforementioned technical problems, this application provides a data processing system and method that can distribute a large number of tasks to solid-state drives for execution. Due to the large number of solid-state drives, parallel execution of a large number of tasks can be achieved, thereby improving the execution efficiency of multiple tasks.

[0007] In a first aspect, a data processing system is provided, including a processor and a plurality of solid-state drives (SSDs). The plurality of SSDs are used to provide at least one physical block. The at least one physical block constitutes a first storage space. The processor stores a first address set. The first address set includes multiple addresses. Each address is used to indicate one or more physical blocks in the first storage space. Each SSD stores a second address set. The first address set includes the second address set. A first SSD, after receiving an input / output (I / O) instruction from the processor, determines at least one target SSD from the plurality of SSDs based on a target address in the I / O instruction and the second address set stored on the first SSD. The I / O instruction instructs processing of the storage space indicated by the target address. The storage space indicated by the target address is located on at least one target SSD.

[0008] In the above scheme, within the data processing system, the processor stores the addresses included in the second address set of each solid-state drive (SSD). Therefore, the processor can determine the first SSD to execute the task and send an I / O instruction carrying the target address corresponding to the task to the first SSD. The first SSD stores the second address set related to the target address, thus enabling it to parse the target address to determine the target SSD and process it to complete the task. Therefore, in the data processing system, the processor can distribute a large number of tasks to the SSDs for execution, reducing the processor's workload. Furthermore, because there are numerous SSDs in the data processing system, parallel execution of a large number of tasks can be achieved, improving the execution efficiency of multiple tasks.

[0009] In some possible implementations, the above address is based on the logical unit number (LUN) semantics to indicate the physical block in the first storage space.

[0010] In the above scheme, LUN semantics are used to address physical blocks in the first storage space. This abstracts the addressing of physical blocks, allowing applications or the operating system to access data in the data processing system without needing to know the specific physical block address, but instead accessing the data through the logical unit number. This addressing method provides greater flexibility, simplifies the complexity of storage management, and reduces the direct management and maintenance of physical block addresses.

[0011] In some possible implementations, the aforementioned address includes a Logical Unit Number (LUN) ID, a Logical Block Address (LBA), and a Physical Block Address (PBA). The LUN ID identifies the LUN within the first storage space. Each LUN comprises at least one logical block. A mapping relationship exists between logical blocks and physical blocks within the first storage space. The LBA identifies the logical block, and the PBA identifies the physical block.

[0012] In some possible implementations, the data processing system described above includes a peer-to-peer interconnect bus. The processor described above is used to send I / O instructions to each solid-state drive via the peer-to-peer interconnect bus.

[0013] In the above scheme, the processor and solid-state drive connected via the peer-to-peer interconnect bus have the same status, so these devices can directly exchange data and communicate with each other.

[0014] In some possible implementations, the aforementioned processor and multiple solid-state drives are deployed on the same device.

[0015] In some possible implementations, the aforementioned processor and the first solid-state drive are deployed on different devices.

[0016] In some possible implementations, the data processing system described above includes multiple processors. These processors are deployed in a first device cluster. Multiple solid-state drives (SSDs) are deployed in a second device cluster. The first and second device clusters are different device clusters.

[0017] The above solution supports both distributing multiple tasks to multiple SSDs for execution in a single-machine system and distributing tasks across devices in a multi-machine system. In a multi-machine system, the number of SSDs capable of executing tasks can be added or removed as needed, thus supporting application scenarios with varying task scales. Furthermore, when distributing tasks across devices, the processor can directly issue I / O instructions to the SSDs in another device via the peer-to-peer interconnect bus without needing to interact with the processor in that other device. Therefore, the processor's cross-device task distribution does not require interaction with the processor in another device and does not consume the processor resources of that other device.

[0018] In some possible implementations, the first solid-state drive (SSD) is used to process the storage space indicated by the target address according to I / O instructions. Specifically, the I / O instructions instruct the storage space at the target address on the target SSD to store the first data in the dynamic random access memory (DRAM). Alternatively, the I / O instructions instruct the calculation of the first data to obtain a first calculation result, and store the first calculation result in the DRAM. Alternatively, the I / O instructions instruct the storage of the second data in the DRAM to be stored in the storage space at the target address on the target SSD. Alternatively, the I / O instructions instruct the calculation of the second data to obtain a second calculation result, and store the second calculation result in the storage space at the target address on the target SSD.

[0019] In the above solution, based on the fact that the first solid-state drive can correctly parse the target address carried by the IO instruction, this technical solution supports the first solid-state drive to perform various tasks on the target solid-state drive, including reading data on the target solid-state drive, storing data on the target solid-state drive, performing calculations on the data, etc.

[0020] In some possible implementations, the first solid-state drive (SSD) manages the storage space indicated by the target address on the target SSD, and the target SSD and the first SSD are different SSDs. Specifically, the first SSD sends a read command to the target SSD and receives a read completion message from the target SSD after completing the read command. The read command instructs the target SSD to read first data and write it to DRAM. Alternatively, the first SSD sends a write command to the target SSD and receives a write completion message from the target SSD after completing the write command. The write command instructs the target SSD to read second data from DRAM and write the second data to the storage space indicated by the target address on the target SSD.

[0021] This technical solution provides a data processing flow with separate data plane and control plane in the architecture of the data processing system. In scenarios where the target SSD and the first SSD are different SSDs, and the first SSD manages the storage space indicated by the target address on the target SSD, the first SSD sends control commands (such as read commands and write commands) to the target SSD, instructing the target SSD to transfer first or second data between itself and the DRAM. This achieves separation of the data plane and control plane; the control flow occurs between the first SSD and the target SSD, while the data flow occurs between the target SSD and the DRAM. This enables globally centralized control of the control plane and high-performance data transmission.

[0022] In some possible implementations, the aforementioned plurality of solid-state drives (SSDs) includes a second SSD. The second SSD manages the storage space indicated by the target address on the target SSD. Furthermore, the first SSD, the second SSD, and the target SSD are all different SSDs. Specifically, the first SSD sends an acquisition command to the second SSD and receives an acquisition completion message sent by the second SSD after completing the acquisition command. The acquisition command instructs the second SSD to notify the target SSD to read the first data and write the first data into DRAM. Alternatively, the first SSD specifically sends a storage command to the second SSD and receives a storage completion message sent by the second SSD after completing the storage command. The storage command instructs the second SSD to notify the target SSD to read the second data from DRAM and write the second data into the storage space indicated by the target address on the target SSD.

[0023] This technical solution provides a data processing flow with separation of data storage and management in complex scenarios. For scenarios where data storage and management are separated, the first solid-state drive (SSD) sends control commands (such as fetch commands or store commands) to the SSD managing the storage space indicated by the target address (i.e., the second SSD), instructing the second SSD to complete the first or second data transfer task between the target SSD and the DRAM. This means that in this scenario of separated data storage and management, the first SSD is unaware of the specific execution process of the task; it only needs to instruct the second SSD to complete the task. The first SSD does not need to participate in how the second SSD completes the task; it is considered to have completed the task upon receiving a notification from the second SSD that it has finished. As can be seen, this achieves separation of the data plane and the control plane. The control flow occurs between the first SSD, the second SSD, and the target SSD, while the data flow occurs only between the target SSD and the DRAM. This enables globally centralized control of the control plane and high-performance data transfer.

[0024] In some possible implementations, the aforementioned I / O instructions are used to instruct the storage of the first data in DRAM according to a RAID (Range Automation) algorithm. Specifically, the first solid-state drive (SSD) is used to retrieve the data blocks and parity blocks from the first stripe, obtain the first data based on the data blocks and parity blocks, and write the first data into DRAM. Here, the first stripe is the stripe to which the first data belongs.

[0025] This technical solution provides an implementation method in an application scenario where data is stored based on a RAID algorithm, and the first solid-state drive (SSD) recovers the first data and provides it to the processor. When the first data is corrupted, since the data processing system stores data based on the RAID algorithm, when the first SSD receives a task to retrieve the first data, it can calculate the first data based on the RAID algorithm and then provide the recovered first data to the processor. As can be seen, in this technical solution, the first SSD has data calculation and processing capabilities and can perform other calculation and processing tasks issued by the processor, except for error detection and garbage collection.

[0026] In some possible implementations, the aforementioned I / O instructions are used to instruct the second data to be stored in the storage space indicated by the target address on the target solid-state drive according to the RAID algorithm. Specifically, the first solid-state drive is used to read the second data from DRAM, obtain a data block from the second stripe, obtain a parity block based on the data block and the second data, store the second data block in the physical block corresponding to the target address on the target solid-state drive, and store the parity block in the second stripe. Here, the second stripe is the stripe to which the second data belongs.

[0027] This technical solution provides an implementation method for storing second data using a first solid-state drive (SSD) in an application scenario based on RAID algorithm data storage. When the second data needs to be stored, since the data processing system is based on RAID algorithm data storage, when the first SSD receives the task of storing the second data, it needs to first use the RAID algorithm to calculate the parity block in the second stripe to which the second data belongs, and then store the second data. This ensures that if the data in the second stripe is damaged, the damaged data can be recovered in time based on the parity block. As can be seen, in this technical solution, the first SSD has data calculation and processing capabilities and can execute other calculation and processing tasks issued by the processor, except for error detection and garbage collection.

[0028] Secondly, a data processing method is provided, applied to a data processing system. The data processing system includes a processor and multiple solid-state drives (SSDs). The multiple SSDs provide at least one physical block. The at least one physical block constitutes a first storage space. The processor stores a first address set. The first address set includes multiple addresses. Each address indicates one or more physical blocks in the first storage space. The multiple SSDs include the first SSD. The first SSD stores a second address set. The first address set includes the second address set. The method includes: the processor sending an input / output (I / O) instruction to the first SSD, and receiving a completion message sent by the first SSD after completing the I / O instruction. The I / O instruction carries a target address. The I / O instruction instructs the first SSD to determine at least one target SSD from the multiple SSDs based on the target address and the second address set. The I / O instruction instructs processing of the storage space indicated by the target address. The storage space indicated by the target address is on at least one target SSD.

[0029] In the above scheme, when the processor needs to process the storage space indicated by the target address, the processor dispatches the task to the first solid-state drive (SSD) by sending I / O instructions. The processor only needs to receive notification from the first SSD that the task has been completed. When the first SSD completes the task, it is equivalent to the processor completing the task. Furthermore, because the first SSD stores a second set of addresses, it can resolve the target address based on the second set to determine the target SSD and then process it. It can be understood that the number of SSDs in a data processing system far exceeds the number of processors. Therefore, having the SSD perform tasks that would otherwise be performed by the processor reduces the processor's workload and enables the parallel execution of a large number of tasks, improving the execution efficiency of multiple tasks.

[0030] In some possible implementations, after the processor sends an I / O instruction to the first solid-state drive (SSD) and before the processor receives a completion message sent by the first SSD after completing the I / O instruction, the method further includes: the first SSD processing the storage space indicated by the target address according to the I / O instruction. The I / O instruction is used to instruct the storage space at the target address on the target SSD to store the first data in the dynamic random access memory (DRAM). Alternatively, the I / O instruction is used to instruct the first data to be calculated to obtain a first calculation result, and the first calculation result is stored in the DRAM. Alternatively, the I / O instruction is used to instruct the storage space at the target address on the target SSD to store the second data in the DRAM. Alternatively, the I / O instruction is used to instruct the second data to be calculated to obtain a second calculation result, and the second calculation result is stored in the storage space at the target address on the target SSD.

[0031] In some possible implementations, the first solid-state drive (SSD) is used to manage the storage space indicated by the target address on the target SSD, and the target SSD and the first SSD are different SSDs. The first SSD processes the storage space indicated by the target address according to I / O instructions, including: the first SSD sending a read instruction to the target SSD and receiving a read completion message sent by the target SSD after completing the read instruction. The read instruction instructs the target SSD to read first data and write the first data into DRAM.

[0032] In some possible implementations, the aforementioned plurality of solid-state drives (SSDs) includes a second SSD. The second SSD manages the storage space indicated by the target address on the target SSD. The first SSD, the second SSD, and the target SSD are all different SSDs. The first SSD processes the storage space indicated by the target address according to I / O instructions, including: the first SSD sending a fetch instruction to the second SSD, and receiving a fetch completion message sent by the second SSD after completing the fetch instruction. The fetch instruction instructs the second SSD to notify the target SSD to read first data and write the first data to DRAM.

[0033] In some possible implementations, the aforementioned I / O instructions are used to instruct the storage of the first data in DRAM according to a RAID algorithm. The first solid-state drive (SSD) processes the storage space indicated by the target address according to the I / O instructions, including: the first SSD acquiring data blocks and parity blocks from the first stripe, obtaining the first data based on the data blocks and parity blocks, and writing the first data into DRAM. Here, the first stripe is the stripe to which the first data belongs.

[0034] In some possible implementations, the first solid-state drive (SSD) is used to manage the storage space indicated by the target address on the target SSD, and the target SSD and the first SSD are different SSDs. The first SSD processes the storage space indicated by the target address according to I / O instructions, including: the first SSD sending a write instruction to the target SSD and receiving a write completion message sent by the target SSD after completing the write instruction. The write instruction instructs the target SSD to read second data from DRAM and write the second data to the storage space indicated by the target address on the target SSD.

[0035] In some possible implementations, the aforementioned plurality of solid-state drives (SSDs) includes a second SSD. The second SSD manages the storage space indicated by the target address on the target SSD. The first SSD, the second SSD, and the target SSD are all different SSDs. The first SSD processes the storage space indicated by the target address according to I / O instructions, including: the first SSD sending a storage instruction to the second SSD and receiving a storage completion message sent by the second SSD after completing the storage instruction. The storage instruction instructs the second SSD to notify the target SSD to read second data from DRAM and write the second data to the storage space indicated by the target address on the target SSD.

[0036] In some possible implementations, the aforementioned I / O instructions are used to instruct the second data to be stored in the storage space indicated by the target address on the target solid-state drive according to the RAID algorithm. The first solid-state drive processes the storage space indicated by the target address according to the I / O instructions, including: the first solid-state drive reads the second data from DRAM, obtains a data block from the second stripe, obtains a parity block based on the data block and the second data, stores the second data block in the physical block corresponding to the target address on the target solid-state drive, and stores the parity block in the second stripe. Here, the second stripe is the stripe to which the second data belongs.

[0037] In some possible implementations, the aforementioned address includes a Logical Unit Number (LUN) ID, a Logical Block Address (LBA), and a Physical Block Address (PBA). The aforementioned target address includes a target LUN ID and a target LBA. The LUN ID identifies the LUN in the first storage space. Each LUN includes at least one logical block. There is a mapping relationship between logical blocks and physical blocks in the first storage space. The LBA identifies the logical block. The PBA identifies the physical block.

[0038] In some possible implementations, the aforementioned plurality of solid-state drives (SSDs) includes a second SSD. The second set of addresses stored on the second SSD belongs to the first set of addresses. The method further includes: the processor sending an I / O instruction carrying a first address to the second SSD, and receiving a first completion message sent by the second SSD after completing the I / O instruction carrying the first address. The I / O instruction sent by the processor to the first SSD is generated based on a first task, and the I / O instruction carrying the first address is generated based on a second task. The first task and the second task are independent of each other. The I / O instruction carrying the first address instructs the second SSD to determine the target SSD corresponding to the first address from among the plurality of SSDs based on the first address and the second set of addresses stored on the second SSD. The I / O instruction carrying the first address instructs processing of the storage space indicated by the first address. The storage space indicated by the first address is located on the target SSD corresponding to the first address.

[0039] In the above scheme, the processor can distribute multiple tasks to different solid-state drives, enabling multiple solid-state drives to execute tasks in parallel, thereby improving the execution efficiency of multiple tasks.

[0040] Thirdly, a computer program product containing instructions is provided that, when executed by a computing device, causes the computing device to perform the method as described in any of the second aspects.

[0041] Fourthly, a computer-readable storage medium is provided, characterized in that it includes computer program instructions, which, when executed by a computing device, perform the method as described in any of the second aspects. Attached Figure Description

[0042] Figure 1A is a structural diagram of a data processing system according to an embodiment of this application;

[0043] Figure 1B is a structural diagram of another data processing system according to an embodiment of this application;

[0044] Figure 2A is a structural diagram of a data processing system provided in an embodiment of this application;

[0045] Figure 2B is a structural diagram of another data processing system provided in an embodiment of this application;

[0046] Figure 2C is a structural diagram of another data processing system provided in an embodiment of this application;

[0047] Figure 3 is a flowchart illustrating a data processing method provided in an embodiment of this application;

[0048] Figure 4A is a flowchart illustrating the execution of I / O instructions by a first solid-state drive according to an embodiment of this application;

[0049] Figure 4B is a flowchart illustrating another first solid-state drive executing I / O instructions according to an embodiment of this application;

[0050] Figure 4C is a flowchart illustrating another first solid-state drive executing I / O instructions according to an embodiment of this application;

[0051] Figure 5 is a schematic diagram of the result of a RAID 5 storage data block provided in an embodiment of this application;

[0052] Figure 6A is a flowchart illustrating another first solid-state drive executing I / O instructions according to an embodiment of this application;

[0053] Figure 6B is a flowchart illustrating another first solid-state drive executing I / O instructions according to an embodiment of this application;

[0054] Figure 6C is a flowchart illustrating another first solid-state drive executing I / O instructions according to an embodiment of this application;

[0055] Figure 7 is a schematic diagram of another result of storing data blocks based on RAID5 provided in an embodiment of this application. Detailed Implementation

[0056] The technical solutions of the embodiments of this application will be described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those of ordinary skill in the art without creative effort are within the scope of protection of this application.

[0057] Before introducing the embodiments of this application, let's briefly introduce the existing data processing system and its task processing process.

[0058] Based on the different processes by which data processing systems handle tasks, existing data processing systems are mainly divided into two types. The first type of data processing system and its task handling process can be seen in Figure 1A and related descriptions. The second type of data processing system and its task handling process can be seen in Figure 1B and related descriptions.

[0059] (I) The first type of data processing system

[0060] Referring to Figure 1A, Figure 1A is a structural diagram of a data processing system according to an embodiment of this application. As shown in Figure 1A, the data processing system 10 includes a processor 11, a solid-state drive 12, and a data bus 13. The processor 11 and the solid-state drive 12 communicate via the data bus 13.

[0061] The data processing system 10 can be a computing device. A computing device is an electronic device used for calculating, processing, and storing data, including servers, supercomputers, personal computers, workstations, industrial control computers, etc. In practical applications, the data processing system 10 includes several processors 11 and dozens to thousands of solid-state drives 12. That is, the number difference between processors 11 and solid-state drives 12 in the data processing system 10 is significant. For ease of understanding, this application only shows one processor 11 and six solid-state drives 12 in Figure 1A.

[0062] Processor 11 is used to execute tasks in a task set. The task set describes tasks waiting to be executed by processor 11; therefore, the task set includes one or more tasks. Processor 11 is also used to send input / output (I / O) instructions to solid-state drive 12 according to the tasks.

[0063] Optionally, the processor 11 may be, for example, a central processing unit (CPU), a data processing unit (DPU), a graphics processing unit (GPU), etc., or an application-specific integrated circuit (ASIC), or one or more integrated circuits.

[0064] The tasks in the task set can be tasks received by the data processing system 10 from the client, or tasks generated by the processor 11 during the execution of other tasks (such as computational tasks). The client and the data processing system 10 can be deployed on different computing devices, or on the same computing device.

[0065] The process by which the processor 11 sends I / O instructions to the solid-state drive 12 according to the task is as follows: if the task type is a read operation, the I / O instruction sent by the processor 11 to the solid-state drive 12 through the data bus 13 is a read instruction; if the task type is a write operation, the I / O instruction sent by the processor 11 to the solid-state drive 12 through the data bus 13 is a write instruction.

[0066] Solid-state drive 12 is used to operate according to I / O instructions sent by processor 11. If the I / O instruction received by solid-state drive 12 from processor 11 is a read instruction, that is, solid-state drive 12 needs to send data to processor 11, then solid-state drive 12 reads the data from its own disk and then sends the data to processor 11 via data bus 13. If the I / O instruction received by solid-state drive 12 from processor 11 is a write instruction, that is, solid-state drive 12 needs to store data from processor 11, then solid-state drive 12 retrieves the data from processor 11 and then writes the data to its own disk.

[0067] Optionally, if the solid-state drive 12 (SSD) is a storage device built from flash memory chips, then the SSD 12 includes an SSD controller 121 and a storage medium 122. The SSD controller 121 executes read or write commands sent by the processor 11 to the SSD 12 via the data bus 13. The SSD controller 121 can be a chip, such as a field-programmable gate array (FPGA) or an ASIC. The storage medium 122 consists of several flash memory chips. Each flash memory chip can be divided into several physical chunks of a fixed size. Therefore, each physical chunk has a standard capacity, for example, 2 to the power of N (M), where N is a positive integer.

[0068] Optionally, the flash memory chips in the SSD controller 121 and the storage medium 122 can be placed on the same printed circuit board (PCB) and presented in the form of a disk or card. They can then communicate with the processor 11 via the data bus 13 through the input / output (I / O) interface on the PCB.

[0069] In the case where the solid-state drive 12 includes an SSD controller 121 and a storage medium 122, if the solid-state drive 12 receives a read command, the SSD controller 121 concurrently reads data from one or more physical blocks of the storage medium 122, and then sends the data to the processor 11 via the data bus 13. If the solid-state drive 12 receives a write command, the SSD controller 121 obtains data from the processor 11, and then concurrently writes the data to multiple physical blocks of the storage medium 122.

[0070] The data bus 13 is used to transfer I / O instructions and data between the processor 11 and the solid-state drive 12.

[0071] Alternatively, the data bus 13 may be, for example, a peripheral component interconnect express (PCIe) bus. The data bus 13 may include a path for transmitting information between various components (e.g., processor 11, solid-state drive 12) in the data processing system 10.

[0072] It should be understood that the number of processors 11 and solid-state drives 12 in the data processing system 10 is illustrated using the example shown in Figure 1A, which includes one processor 11 and six solid-state drives 12. In practical applications, the number of processors 11 and solid-state drives 12 in the data processing system 10 can be more, and this application does not make a specific limitation. When the data processing system 10 includes multiple processors 11, these processors 11 execute a portion of the tasks in the task set respectively, so as to jointly complete all the tasks in the task set.

[0073] In the data processing system 10 shown in Figure 1A, the processor 11 executes tasks from the task set. When the task set includes a large number of tasks, a single processor 11 can only execute the tasks assigned to it one by one, that is, a single processor 11 can only process tasks serially. Therefore, the performance bottleneck of the data processing system 10 in executing tasks lies in the processor 11, and the execution efficiency and the number of tasks executed are both limited by the computing power and number of processors 11.

[0074] Optionally, to enhance the processing capabilities of the current data processing system 10 for the solid-state drive 12, an expansion card—a redundant array of independent disks (RAID) card—can be installed on the current data processing system 10. RAID cards are used to specifically manage disk arrays composed of multiple SSDs, providing more efficient data transfer and processing capabilities, thereby accelerating the operation of the solid-state drive 12. Typically, depending on the storage requirements of the data processing system 10 and the complexity of the RAID card configuration, the number of RAID cards in the data processing system 10 ranges from 1 to 9. When the data processing system 10 includes RAID cards, the processor 11 distributes all tasks in the task set to all RAID cards, which then execute a portion of the tasks in the task set. Each RAID card can only execute the tasks assigned to it one by one. Therefore, for the data processing system 10 with RAID cards, the performance bottleneck in executing processing tasks lies in the RAID cards; both execution efficiency and the number of tasks executed are limited by the capabilities and quantity of the RAID cards.

[0075] (II) Second type of data processing system

[0076] The second type of data processing system is based on the structure of the data processing system 10 in Figure 1A, with multiple data processing systems 10 serving as multiple nodes in the second type of data processing system. For details, please refer to Figure 1B and related content.

[0077] Referring to Figure 1B, which is a structural diagram of another data processing system according to an embodiment of this application, the data processing system 20 includes nodes 21 and a communication bus 22. Multiple nodes 21 communicate with each other via the communication bus 22. Each node 21 includes a processor 23 and a solid-state drive 24. In one node 21, the processor 23 and the solid-state drive 24 communicate via the communication bus 22.

[0078] The data processing system 20 can be a cluster of computing devices. This cluster can include multiple computing devices, with nodes 21 being the computing devices themselves. In practical applications, nodes 21 typically include several processors 23 and dozens to thousands of solid-state drives (SSDs) 24. That is, the number difference between processors 23 and SSDs 24 in nodes 21 is significant. Therefore, when the data processing system 20 includes multiple nodes 21, the difference in the number of processors 23 and SSDs 24 in the data processing system 20 will further increase as the number of nodes 21 increases. For ease of understanding, this application only shows two nodes 21 in Figure 1B, and each node 21 includes one processor 23 and six SSDs 24.

[0079] When node 21 is a computing device, node 21 can be the data processing system 10 of Figure 1A above, processor 23 in node 21 can be processor 11 in the data processing system 10 of Figure 1A above, and solid-state drive 24 in node 21 can be solid-state drive 12 in the data processing system 10 of Figure 1A above. For the sake of brevity, the structure of node 21 will not be described in detail here.

[0080] The communication bus 22 is used to transmit processing requests or data between multiple nodes 21, and also to transmit read instructions, storage instructions or data between the processor 23 and the solid-state drive 24 in the same node 21.

[0081] Optionally, the communication bus 22 can be a PCIe bus, network bus, extended industry standard architecture (EISA) bus, unified bus (UB) bus, compute express link (CXL) bus, cache coherent interconnect for accelerators (CCIX) bus, etc. The unified bus is also known as the Lingqu bus. Buses can be divided into address buses, data buses, control buses, etc. For ease of representation, only one line is used in Figure 1B, but this does not mean that there is only one bus or one type of bus. The communication bus 22 includes both the communication channel for transmitting processing requests or data between the nodes 21 in the data processing system 20, and the path for transmitting read instructions, store instructions, or data between the components of node 21 (e.g., processor 23, solid-state drive 24).

[0082] It should be understood that the number of nodes 21 in the data processing system 20, and the number of processors 23 and solid-state drives 24 in a single node 21, are illustrated using the example shown in Figure 1B, where the data processing system 20 includes two nodes 21, each node 21 including one processor 23 and six solid-state drives 24. In practical applications, the number of nodes 21, processors 23, and solid-state drives 24 in the data processing system 20 can be greater, and this application does not impose specific limitations.

[0083] In the current data processing system 20, when the task set on a node 21 (hereinafter referred to as the first node) includes a large number of tasks, if the first node has only one processor 23, that processor 23 will execute all the tasks in the task set; if the first node has multiple processors 23, these processors 23 will execute a portion of the tasks in the task set respectively, in order to jointly complete all the tasks in the task set. For a single processor 23, the processor 23 can only execute the tasks assigned to it one by one. Therefore, the performance bottleneck of the first node in executing tasks lies in the computing power and number of processors 23.

[0084] If the task executed by the processor 23 of the first node includes reading or writing to the solid-state drive 24 of another node 21 (hereinafter referred to as the second node), the processor 23 of the first node converts the read or write operation into a processing request and sends the processing request to the processor 23 of the second node via the communication bus 22. Upon receiving the processing request, the processor 23 of the second node performs the read or write operation on the solid-state drive 24 of the second node according to the processing request, and then sends the operation result back to the processor 23 of the first node via the communication bus 22. Therefore, when the task executed by the processor of one node involves the solid-state drive of another node, the execution efficiency of the task also depends on the interaction between the processors of the two nodes. This not only consumes the processor resources of both nodes, but also makes the execution time of the task depend on the individual processing time and the interaction time of the processors in both nodes. Therefore, the performance bottleneck of the data processing system 20 in executing tasks still lies with the processor 23.

[0085] To improve the execution efficiency of data processing systems handling multiple tasks on different SSDs, this application provides three data processing systems. These systems can distribute a large number of tasks to the solid-state drives for execution, thus reducing the processor's workload and enabling multiple tasks to be executed in parallel, thereby improving the execution efficiency of multiple tasks. The first data processing system is illustrated in Figure 2A and related descriptions; the second data processing system is illustrated in Figure 2B and related descriptions; and the third data processing system is illustrated in Figure 2C and related descriptions.

[0086] (I) The first type of data processing system

[0087] Referring to Figure 2A, which is a structural diagram of a data processing system provided in an embodiment of this application, the data processing system 30 includes a processor 31, a solid-state drive 32, and a peer-to-peer interconnect bus 33. The processor 31 and the solid-state drive 32 communicate in the data processing system 30 through the peer-to-peer interconnect bus 33.

[0088] The data processing system 30 is a computing device, in which the processor 31, solid-state drive 32, and peer-to-peer interconnect bus 33 are deployed on the same device. The processor 31 can be, for example, a CPU, DPU, GPU, etc., or an ASIC, or one or more integrated circuits. The solid-state drive 32 is a storage device built from Flash chips. The peer-to-peer interconnect bus 33 is the communication bus connecting the processor 31, solid-state drive 32, and other devices. The processor 31, solid-state drive 32, and other devices connected via the peer-to-peer interconnect bus 33 have equal status, therefore these devices can directly exchange data and communicate. The peer-to-peer interconnect bus 33 can be, for example, a UB bus, an NV-LINK bus, a PCIe bus, etc.

[0089] In the data processing system 30, at least one physical block can be provided by all solid-state drives (SSDs) 32, or at least one physical block can be provided by only some of the SSDs 32, while others do not provide any physical blocks. The at least one physical block provided by the SSDs 32 constitutes a first storage space. This first storage space may contain only one physical block or multiple physical blocks. When the first storage space contains multiple physical blocks, these physical blocks may originate from the same SSD 32 or from different SSDs 32.

[0090] The processor 31 stores a first address set 310, and each solid-state drive 32 stores a second address set 320. The first address set 310 includes multiple addresses 311. Each address 311 is used to indicate one or more physical blocks in the first storage space. The way the address 311 indicates a physical block and the specific content of the address 311 can be found in the relevant description of the processor addressing physical blocks in the first storage space based on logical unit number (LUN) semantics in step S101 of a data processing method in Figure 3 below. The second address set 320 belongs to the first address set 310, and the second address set 320 also includes one or more addresses 311.

[0091] Processor 31 is used to execute tasks in the task set, but the way processor 31 executes tasks is by sending tasks to solid-state drive 32, which then executes the tasks. When solid-state drive 32 completes the task, it can also be considered that processor 31 has completed the task.

[0092] In some possible implementations, the processor 31 assigns tasks to the solid-state drive 32 by sending I / O instructions to the solid-state drive 32, and the solid-state drive 32 completes the task by completing the I / O instructions. Unlike the I / O instructions sent by the processor 11 to the solid-state drive 12 in the aforementioned data processing system 10, the I / O instructions sent by the processor 31 to the solid-state drive 32 in the data processing system 30 carry a target address, and this I / O instruction is used to instruct the solid-state drive 32 to process the storage space indicated by the target address. The specific content of the target address can be found in the description of the target address in step S101 of a data processing method in Figure 3 below. The storage space indicated by the target address is on at least one target solid-state drive. When there is only one target solid-state drive, the target solid-state drive corresponds to one of the solid-state drives 32 providing physical blocks. When there are multiple target solid-state drives, the target solid-state drives correspond to multiple solid-state drives 32 providing physical blocks.

[0093] In a specific application scenario, if a task involves multiple target addresses, the processor 31 generates multiple I / O instructions, each carrying a target address. The processor 31 can then send these I / O instructions to the same solid-state drive 32 or to different solid-state drives 32. Once all these I / O instructions have been executed, the task can be considered complete.

[0094] Solid-state drive 32, after receiving an I / O instruction from processor 31, determines the target solid-state drive from among the solid-state drives 32 providing physical blocks based on the target address in the I / O instruction and the second address set 320 stored on solid-state drive 32. This process can be seen in step S102 of the data processing method in Figure 3 below, where the first solid-state drive determines the target solid-state drive corresponding to the target address from among multiple solid-state drives based on the target address in the I / O instruction and the second address set stored on the first solid-state drive.

[0095] It is understandable that SSD 32 can only correctly resolve the target address and thus determine the correct target SSD if the second address set 320 stored on SSD 32 involves the target address. When the second address set 320 stored on SSD 32 does not involve the target address, SSD 32 cannot determine the correct target SSD based on the target address. Therefore, processor 31 also stores the addresses 311 included in the second address set 320 of each SSD 32. In this way, processor 31 can determine the SSD to perform the task and send I / O instructions to that SSD.

[0096] It should be noted that the objects to which the processor 31 can send I / O instructions depend on the addresses included in the second address set 320 stored in the solid-state drive 32. Specifically:

[0097] If each solid-state drive 32 stores the same second address set 320, and the second address set 320 relates to a target address, then the processor 31 can send an I / O instruction carrying the target address to any one of the solid-state drives 32. If different solid-state drives 32 store different second address sets 320, then the processor 31 can only send an I / O instruction carrying the target address to the solid-state drive 32 that stores the second address set 320 related to the target address.

[0098] Optionally, if each solid-state drive 32 stores the same second address set 320, and the second address set 320 is the first address set 310, then the processor 31 can distribute all tasks in the task set to different solid-state drives 32, so that different solid-state drives 32 can execute multiple tasks in parallel.

[0099] Optionally, if each solid-state drive 32 stores the same second address set 320, and the second address set 320 consists only of a portion of the addresses in the first address set 310, then the processor 31 can only assign tasks to the solid-state drives 32 to instruct processing of storage spaces indicated by addresses belonging to the second address set 320. Tasks instructing processing of storage spaces indicated by addresses not belonging to the second address set 320 still need to be executed by the processor 31 itself.

[0100] In this technical solution, the addresses 311 included in the second address set 320 stored by different solid-state drives 32 can be set as needed according to the actual application scenario, and this application does not make specific limitations.

[0101] It should be understood that the number of processors 31 and solid-state drives 32 in the data processing system 30 is illustrated using the example shown in Figure 2A, which includes one processor 31 and four solid-state drives 32. In practical applications, the number of processors 31 and solid-state drives 32 in the data processing system 30 can be more, and this application does not make a specific limitation. When the data processing system 30 includes multiple processors 31, each processor 31 can distribute its respective task to the solid-state drive 32, which then executes the task. Therefore, a solid-state drive 32 may receive I / O instructions from one processor 31 or multiple processors 31. When a solid-state drive 32 receives multiple I / O instructions, it executes the I / O instructions one by one.

[0102] In summary, in the data processing system 30, the processor 31 stores the addresses included in the second address set 320 of each solid-state drive 32. Therefore, the processor 31 can determine the solid-state drive to which the task will be executed and send an I / O instruction carrying the target address corresponding to the task to that solid-state drive. The solid-state drive 32 stores the second address set 320 involving the target address, thus the solid-state drive 32 can resolve the target address to determine the target solid-state drive and then process the target solid-state drive to complete the task. Therefore, in the data processing system 30, the processor 31 can distribute a large number of tasks to the solid-state drives 32 for execution, thereby reducing the workload of the processor 31. Furthermore, because there are a large number of solid-state drives 32 in the data processing system 30, a large number of tasks can be executed in parallel, thereby improving the execution efficiency of multiple tasks.

[0103] (II) Second type of data processing system

[0104] The second type of data processing system is based on the structure of the data processing system 30 in Figure 2A, with multiple data processing systems 30 serving as multiple nodes in the second type of data processing system. For details, please refer to Figure 2B and related content.

[0105] Referring to Figure 2B, which is a structural diagram of another data processing system provided in an embodiment of this application, the data processing system 40 includes nodes 41 and a peer-to-peer interconnect bus 42. Multiple nodes 41 communicate with each other via the peer-to-peer interconnect bus 42. Each node 41 includes a processor 43 and a solid-state drive 44. Within one node 41, the processor 43 and the solid-state drive 44 communicate with each other via the peer-to-peer interconnect bus 42.

[0106] If the data processing system 40 can be a cluster of computing devices, then node 41 is a computing device. When node 41 is a computing device, node 41 can be the data processing system 30 shown in Figure 2A, processor 43 in node 41 can be processor 31 in the data processing system 30 shown in Figure 2A, and solid-state drive 44 in node 41 can be solid-state drive 32 in the data processing system 30 shown in Figure 2A. For the sake of brevity, the structure of node 41 will not be described in detail here.

[0107] The peer-to-peer interconnect bus 42 is used to transmit I / O instructions or data between multiple nodes 41, and also to transmit I / O instructions or data between the processor 43 and the solid-state drive 44 in the same node 41. The peer-to-peer interconnect bus 42 may be the peer-to-peer interconnect bus 33 in the data processing system 30 of Figure 2A above.

[0108] In the data processing system 40, at least one physical block can be provided by multiple solid-state drives 44 in the same node 41, or by multiple solid-state drives 44 in multiple nodes 41. The at least one physical block provided by the solid-state drives 44 constitutes a first storage space. The first storage space may include only one physical block or multiple physical blocks. When the first storage space includes multiple physical blocks, these physical blocks may come from the same solid-state drive 44 or from different solid-state drives 44.

[0109] In the data processing system 40, at least one processor 43 stores a first address set 430, and each solid-state drive 44 stores a second address set 440. The first address set 430 includes multiple addresses 431. Each address 431 is used to indicate one or more physical blocks in the first storage space. The way the address 431 indicates a physical block and the specific content of the address 431 can be found in the description of step S101 of a data processing method in Figure 3 below, where the processor addresses physical blocks in the first storage space based on LUN semantics. The second address set 440 belongs to the first address set 430, and the second address set 440 also includes one or more addresses 431. Figure 2B shows a scenario where each processor 43 stores the first address set 430.

[0110] The processor 43, which stores the first address set 430, is used to execute tasks in the task set. However, the processor 43 executes tasks by sending the tasks to the solid-state drive 44, which then executes the tasks. When the solid-state drive 44 completes the task, it can also be considered that the processor 43 has completed the task.

[0111] In some possible implementations, the processor 43 sends the task to the solid-state drive 44 by sending I / O instructions to the solid-state drive 44, and the solid-state drive 44 completes the task by completing the I / O instructions.

[0112] Similar to the data processing system 30 described above, the I / O instruction sent by the processor 43 to the solid-state drive 44 can be the same I / O instruction sent by the processor 31 to the solid-state drive 32 in the aforementioned data processing system 30. That is, the I / O instruction sent by the processor 43 to the solid-state drive 44 carries a target address, and this I / O instruction is used to instruct the solid-state drive 44 to process the storage space indicated by the target address. The specific content of the target address can be found in the relevant description of the target address in step S101 of a data processing method in Figure 3 below. The storage space indicated by the target address is on at least one target solid-state drive. When there is only one target solid-state drive, the target solid-state drive corresponds to one of the solid-state drives 44 providing physical blocks. When there are multiple target solid-state drives, the target solid-state drives correspond to multiple solid-state drives 44 providing physical blocks.

[0113] Unlike the data processing system 30 described above, the processor 43 can send I / O instructions not only to the solid-state drive 44 located on the same node 41 as the processor 43, but also to solid-state drives 44 located on different nodes 41. Taking the data processing system 40 shown in Figure 2B as an example, the node 41 on the left side of Figure 2B is called the first node, and the processor 43 in the first node is called the first processor. The node 41 on the right side is called the second node, and the processor 43 in the second node is called the second processor. When the first processor sends an I / O instruction carrying a target address, any solid-state drive 44 that stores the second address set 440 involving the target address can be used as the target of the I / O instruction sent by the first processor. The solid-state drive 44 that receives the I / O instruction is called the first solid-state drive. The first processor and the first solid-state drive can be located on the same node 41 or on different nodes 41. When the first processor and the first solid-state drive are located on the same node 41, the first solid-state drive is the solid-state drive 44 in the first node; when the first processor and the first solid-state drive are located on different nodes 41, the first solid-state drive is the solid-state drive 44 in the second node. Regardless of whether the first processor and the first solid-state drive are located on the same node 41, the first node can directly send IO instructions to the first solid-state drive through the peer-to-peer interconnect bus 42.

[0114] After receiving an I / O instruction from the processor 43, the first solid-state drive (SSD) determines the target SSD from the SSDs 44 that provide the physical blocks, based on the target address in the I / O instruction and the second address set 440 stored on the first SSD. This process can be seen in step S102 of the data processing method in Figure 3 below, where the first SSD determines the target SSD corresponding to the target address from multiple SSDs based on the target address in the I / O instruction and the second address set stored on the first SSD.

[0115] It should be noted that the processor 43 also stores the addresses 431 included in the second address set 440 of multiple solid-state drives 44. In this way, the processor 43 can determine the solid-state drive to which the task is to be performed and send I / O instructions to that solid-state drive.

[0116] It should be noted that the first address set 430 stored by different processors 43 can be the same or different. When different processors 43 store different first address sets 430, each processor 43 needs to check whether the second address set 440 of the solid-state drive 44 belongs to its stored first address set 430 to determine the scope of the receiving task. Taking the first processor as an example, the first processor needs to check whether the second address set 440 of each solid-state drive 44 in the first node belongs to the first address set 430 stored by the first processor. Also, the first processor needs to check whether the second address set 440 of each solid-state drive 44 in the second node belongs to the first address set 430 stored by the first processor. Only when the second address set 440 of only one solid-state drive 44 belongs to the first address set 430 stored by the first processor will the first processor store the address 431 included in the second address set 440 of that solid-state drive 44, and then use that solid-state drive 44 as the object of the receiving task, sending the corresponding IO instructions to it.

[0117] In this technical solution, the addresses 431 included in the first address set 430 stored by different processors 43 and the addresses 431 included in the second address set 440 stored by different solid-state drives 44 can be set as needed according to the actual application scenario, and this application does not make specific limitations.

[0118] It should be understood that the number of nodes 41 in the data processing system 40, and the number of processors 43 and solid-state drives 44 in a single node 41, are illustrated using the example shown in Figure 2B, where the data processing system 40 includes two nodes 41, each node 41 including one processor 43 and three solid-state drives 44. In practical applications, the number of nodes 41, processors 43, and solid-state drives 44 in the data processing system 40 can be greater, and this application does not impose a specific limitation. When the data processing system 40 includes more nodes 41, and each node 41 includes more processors 43 storing the first address set 430, these processors 43 can all distribute their respective tasks to be executed within the same node or across nodes, and the solid-state drives 44 receiving the tasks will execute the tasks.

[0119] In summary, in the data processing system 40, the processor 43 stores the addresses included in the second address set 440 of the solid-state drive 44. Therefore, the processor 43 can determine the solid-state drive to which the task will be executed and send an I / O instruction carrying the target address corresponding to the task to that solid-state drive. The solid-state drive 44 stores the second address set 440 involving the target address, thus the solid-state drive 44 can parse the target address to determine the target solid-state drive, and then process the target solid-state drive to complete the task. By assigning the task to the solid-state drive 44 for execution, the processor 43 reduces its workload.

[0120] In addition, processor 43 can distribute tasks across nodes. This increases the number of solid-state drives that can execute tasks, further improving the parallel execution efficiency of multiple tasks and supporting application scenarios with different task scales.

[0121] Furthermore, when distributing tasks across nodes, processor 43 can directly send I / O instructions to the solid-state drive in another node via the peer-to-peer interconnect bus 42 without going through the processor in the other node. Therefore, the task distribution by processor 43 does not require interaction with the processor in the other node and does not require consuming the processor resources in the other node.

[0122] (III) The third type of data processing system

[0123] Referring to Figure 2C, which is a structural diagram of another data processing system provided in an embodiment of this application, the data processing system 50 includes a first device cluster 51, a second device cluster 52, and a peer-to-peer interconnect bus 53. The first device cluster 51 and the second device cluster 52 are different device clusters.

[0124] The first device cluster 51 includes multiple processors 54. Figure 2C illustrates a scenario where the first device cluster 51 includes two processors 54. In a specific implementation, the first device cluster 51 may be a computing device cluster, which includes multiple computing devices, each of which includes one or more processors 54. The processors 54 may be the processor 31 in the data processing system 30 of Figure 2A.

[0125] The second device cluster 52 includes multiple solid-state drives (SSDs) 55. Figure 2C illustrates a scenario where the second device cluster 52 includes four SSDs 55. In a specific implementation, the second device cluster 52 can be a storage device cluster, comprising multiple storage devices, each including one or more SSDs 55. Here, a storage device refers to a hardware device used for storing and preserving data, including storage servers, storage arrays, etc. The SSDs 55 can be the SSDs 32 in the data processing system 30 of Figure 2A.

[0126] The peer-to-peer interconnect bus 53 is used to transmit I / O instructions or data between the first device cluster 51 and the second device cluster 52, and also to transmit I / O instructions or data between multiple processors 54 within the first device cluster 51, and between multiple solid-state drives 55 within the second device cluster 52. The peer-to-peer interconnect bus 53 may be the peer-to-peer interconnect bus 33 in the data processing system 30 of Figure 2A above.

[0127] In the data processing system 50, at least one physical block can be provided by all solid-state drives 55 in the second device cluster 52, or at least one physical block can be provided by only some of the solid-state drives 55 in the second device cluster 52, while other solid-state drives 55 do not provide any physical blocks. The at least one physical block provided by the solid-state drives 55 constitutes a first storage space. The first storage space may include only one physical block or multiple physical blocks. When the first storage space includes multiple physical blocks, these physical blocks may come from the same solid-state drive 55 or from different solid-state drives 55.

[0128] In the data processing system 50, at least one processor 54 in the first device cluster 51 stores a first address set 540, and each solid-state drive 55 in the second device cluster 52 stores a second address set 550. The first address set 540 includes multiple addresses 541. Each address 541 is used to indicate one or more physical blocks in the first storage space. The way the address 541 indicates the physical block and the specific content of the address 541 can be found in the relevant description of the processor addressing the physical blocks in the first storage space based on LUN semantics in step S101 of a data processing method in Figure 3 below. The second address set 550 belongs to the first address set 540, and the second address set 550 also includes one or more addresses 541. Figure 2C shows a scenario where each processor 54 stores the first address set 540.

[0129] Processor 54 is used to execute tasks in the task set, but processor 54 executes tasks by sending tasks to solid-state drive 55, which then executes the tasks. When solid-state drive 55 completes the task, it can also be considered that processor 54 has completed the task.

[0130] In some possible implementations, the processor 54 sends a task to the solid-state drive 55 by sending I / O instructions to the solid-state drive 55, and the solid-state drive 55 completes the task by completing the I / O instructions. Specifically, the processor 54 stores addresses 541 included in the second address set 550 of multiple solid-state drives 55, so that the processor 43 can determine the solid-state drive to execute the task and send I / O instructions to that solid-state drive. After the processor 54 determines the solid-state drive 55 to execute the task, the processor 54 directly sends the I / O instructions to the solid-state drive 55 through the peer-to-peer interconnect bus 53. The I / O instructions sent by the processor 54 carry a target address, and the I / O instructions are used to instruct the solid-state drive 55 to process the storage space indicated by the target address. The specific content of the target address can be found in the relevant description of the target address in step S101 of a data processing method in Figure 3 below. The storage space indicated by the target address is on at least one target solid-state drive. When there is only one target solid-state drive, the target solid-state drive corresponds to one of the solid-state drives 55 that provides physical blocks. When there are multiple target solid-state drives, each target solid-state drive corresponds to multiple solid-state drives 55 that provide physical blocks.

[0131] Solid-state drive 55, after receiving an I / O instruction from processor 54, determines the target solid-state drive from among the solid-state drives 55 providing physical blocks based on the target address in the I / O instruction and the second address set 550 stored on solid-state drive 55. This process can be seen in step S102 of the data processing method in Figure 3 below, where the first solid-state drive determines the target solid-state drive corresponding to the target address from among multiple solid-state drives based on the target address in the I / O instruction and the second address set stored on the first solid-state drive.

[0132] It should be noted that the first address sets 540 stored by different processors 54 in the first device cluster 51 can be the same or different. When different processors 54 store different first address sets 540, each processor 54 needs to determine whether the second address set 550 of each solid-state drive 55 in the second device cluster 52 belongs to its stored first address set 540, thereby determining the scope of the receiving task. Only when the second address set 550 of only one solid-state drive 55 belongs to the first address set 540 stored by one processor 54 will that processor 54 store the addresses 541 included in the second address set 550 of that solid-state drive 55, and then use that solid-state drive 55 as the object of the receiving task, sending the corresponding IO instructions to it.

[0133] In this technical solution, the addresses 541 included in the first address set 540 stored by different processors 54, and the addresses 541 included in the second address set 550 stored by different solid-state drives 55 can be set as needed according to the actual application scenario, and this application does not make specific limitations.

[0134] It should be understood that the number of processors 54 in the first device cluster 51 and the number of solid-state drives 55 in the second device cluster 52 of the data processing system 50 are illustrated using the example shown in Figure 2C, where the first device cluster 51 includes two processors 54 and the second device cluster 52 includes four solid-state drives 55. In practical applications, the number of processors 54 and solid-state drives 55 in the data processing system 50 can be greater, and this application does not impose a specific limitation. When the number of processors 54 in the data processing system 50 is greater, each processor 54 can send its respective task to the solid-state drive 44 in the second device cluster 52 corresponding to the task, and the solid-state drive 44 receiving the task will execute the task. When the number of solid-state drives 55 in the data processing system 50 is greater, the number of solid-state drives that can execute tasks increases, enabling the parallel execution of a large number of tasks and improving the execution efficiency of multiple tasks.

[0135] In summary, in the data processing system 50, the processor 54 in the first device cluster 51 stores addresses 541 included in the second address set 550 of the solid-state drives 55. Therefore, the processor 54 can determine the solid-state drive to execute the task and send an I / O instruction carrying the target address corresponding to the task to that solid-state drive. The solid-state drive 55 stores the second address set 550 involving the target address, thus the solid-state drive 55 can parse the target address to determine the target solid-state drive, and then process the target solid-state drive to complete the task. The processor 54 reduces its workload by sending the task to the solid-state drive 55 for execution. Furthermore, the second device cluster 52 has a large number of solid-state drives 55, thus enabling the parallel execution of a large number of tasks, thereby improving the execution efficiency of multiple tasks.

[0136] The data processing system provided by the embodiments of this application has been described above with reference to Figures 2A, 2B, and 2C. Next, a data processing method provided by an embodiment of this application will be described. It should be noted that the application scenarios of the data processing method provided by the embodiments of this application are not limited to the first data processing system of Figure 2A, the second data processing system of Figure 2B, or the third data processing system of Figure 2C. All scenarios in which the data processing method provided by the embodiments of this application can be applied are within the protection scope of this application.

[0137] Referring to Figure 3, Figure 3 is a schematic flowchart of a data processing method provided in an embodiment of this application. As shown in Figure 3, the data processing method provided in this application includes:

[0138] S101: The processor sends input / output (I / O) instructions to the first solid-state drive.

[0139] Accordingly, the first solid-state drive receives I / O instructions from the processor.

[0140] In one specific implementation, the processor sends I / O instructions to the first solid-state drive via a peer-to-peer interconnect bus, and the first solid-state drive receives the I / O instructions from the processor via the peer-to-peer interconnect bus.

[0141] In some possible implementations, the processor stores a first set of addresses. This first set of addresses includes multiple addresses. Each address indicates one or more physical blocks in a first memory space. The specific content of each address depends on the way it indicates a physical block; this way of indicating a physical block is the processor's addressing method for physical blocks in the first memory space. The one or more physical blocks in the first memory space are provided by multiple solid-state drives (SSDs).

[0142] In some possible implementations, the processor addresses physical blocks in the first memory space based on logical unit number (LUN) semantics.

[0143] First, all physical blocks in the first storage space are mapped into multiple logical blocks. The mapping between physical and logical blocks can be one-to-one, one-to-many, or many-to-one, depending on the user.

[0144] Subsequently, these logical blocks are grouped into one or more chunk groups, and the storage space provided by the chunk groups is virtualized into logical units (LUs) for the processor to use. Each logical unit has a unique LUN. Typically, LUN is used to refer directly to a logical unit, and each LUN is considered to have a logical unit number identifier (LUN ID), which is used to identify this LUN. If all logical blocks are in the same chunk group, the processor stores only one LUN ID for the first storage space, let's call it LUN 0; if all logical blocks are divided into two chunk groups, the processor will store two LUN IDs for the first storage space, let's call them LUN 0 and LUN 1; if all logical blocks are divided into three chunk groups, the processor will store three LUN IDs for the first storage space, let's call them LUN 0, LUN 1, and LUN 2; and so on. The more chunk groups there are, the more LUN IDs the processor stores for the first storage space.

[0145] Next, logical block addresses (LBAs) are assigned to all logical blocks within the same logical block group. If LUN 0 contains one logical block, its logical block address is LBA 0; if LUN 0 contains two logical blocks, their logical block addresses are LBA 0 and LBA 1, respectively; if LUN 0 contains three logical blocks, their logical block addresses are LBA 0, LBA 1, and LBA 2, respectively; and so on. The more logical blocks in a logical block group, the more LBAs are assigned to the LUN of that logical block group.

[0146] Finally, each logical block has an LBA, and a mapping between LBAs and physical block addresses (PBAs) is established. When there is a one-to-one mapping between physical blocks and logical blocks, one PBA corresponds to one LBA; when there is a one-to-many mapping between physical blocks and logical blocks, one PBA corresponds to multiple LBAs; when there is a many-to-one mapping between physical blocks and logical blocks, multiple PBAs correspond to one LBA.

[0147] In this implementation, each address in the first address set includes a LUN ID, an LBA, and a PBA. The LUN ID identifies the LUN in the first memory space; the LBA identifies the logical block; and the PBA identifies the physical block. This ensures that each address indicates a physical block in the first memory space. Table 1 below shows the eight addresses in the first address set.

[0148] Table 1

[0149] As shown in Table 1, the first storage space includes LUN0 and LUN 1. LUN 0 and LUN 1 each include four LBAs, and each LBA corresponds to one PBA. Therefore, in Table 1, each address corresponds to one physical block.

[0150] It should be understood that Table 1 above is merely an example, and this application does not specifically limit the content of the address.

[0151] In summary, using LUN semantics to address physical blocks in the first storage space abstracts the addressing of physical blocks. This allows applications or the operating system to access data in the data processing system without needing to know the specific physical block address, but instead accessing the data through the logical unit number. This addressing method provides greater flexibility, simplifies the complexity of storage management, and reduces the direct management and maintenance of physical block addresses.

[0152] In some possible implementations, a second set of addresses is stored on the first solid-state drive. The second set of addresses includes one or more addresses. The second set of addresses belongs to the first set of addresses. Continuing with the example of the eight addresses in the first set of addresses in Table 1 above, the second set of addresses stored on the first solid-state drive can include only one address, such as address 11; it can include only addresses corresponding to the same LUN, such as addresses 11-14; or it can include all addresses in the first set of addresses, such as addresses 11-18.

[0153] In some possible implementations, input / output (IO) instructions carry a destination address. IO instructions are used to instruct processing of the memory space indicated by the destination address. When each address in the first address set includes a LUN ID, LBA, and PBA, the destination address includes the destination LUN ID and the destination LBA. Continuing with the example of the eight addresses in the first address set in Table 1 above, the destination LUN ID in the destination address is either LUN 0 or LUN 1, and the destination LBA in the destination address is either LBA 0, LBA 1, LBA 2, or LBA 3.

[0154] Optionally, the I / O instruction carries a data length. The data length is used to identify the length of the data to be processed.

[0155] Optionally, the I / O instruction carries an instruction type. The instruction type is used to identify whether the I / O instruction is a read instruction or a write instruction. If the I / O instruction is a read instruction, it carries a read flag; if the I / O instruction is a write instruction, it carries a write flag.

[0156] Optionally, the I / O instruction carries a processing algorithm. If the I / O instruction is a read instruction, it instructs the first data in the memory space indicated by the target address to be calculated according to the processing algorithm to obtain a first calculation result; if the I / O instruction is a write instruction, it instructs the second data indicated by the I / O instruction to be calculated according to the processing algorithm to obtain a second calculation result, and stores the second calculation result in the memory space indicated by the target address.

[0157] For example, if the processing algorithm is a RAID algorithm, then the I / O command carries a RAID algorithm identifier. The I / O command is used to instruct the use of the RAID algorithm for data encoding and decoding. RAID is a high-performance, high-reliability storage technology that combines multiple independent solid-state drives (SSDs) in different ways to create a large-capacity SSD group. RAID algorithms primarily utilize three technologies—data striping, mirroring, and data verification—to achieve high performance, reliability, fault tolerance, and scalability in data processing. RAID algorithms include RAID5, RAID6, and RAIDTP, among others. Encoding data using RAID algorithms is mainly applied to scenarios such as storing data on SSDs, while decoding data using RAID algorithms is mainly applied to scenarios such as reading and recovering data from SSDs.

[0158] Taking RAID5 as an example of a RAID algorithm, the I / O commands carry a RAID5 identifier, indicating that RAID5 is used for data encoding and decoding. Specifically, RAID5 distributes data across various solid-state drives (SSDs) in units of data blocks. In RAID5, the relationship between data blocks and parity blocks is n:1, meaning n data blocks correspond to one parity block. Furthermore, RAID5 distributes both data blocks and parity blocks across all SSDs. Thus, if one SSD fails, causing the data blocks stored on that SSD to be corrupted, the corrupted data blocks can be recovered using the data blocks and parity blocks stored on the remaining SSDs. The correspondence between data blocks, parity blocks, and physical blocks on the SSDs depends on the size of the data blocks and physical blocks. If the size of the data block is the same as the size of the physical block, there is a one-to-one correspondence; if the data block is larger than the physical block, there is a one-to-many correspondence; and if the data block is smaller than the physical block, there is a many-to-one correspondence. For ease of explanation, the following text will use the example of a one-to-one correspondence between data blocks and physical blocks.

[0159] It should be understood that the above-described RAID algorithm is merely an example. In practical applications, the processing algorithm can also be an encryption or decryption algorithm. When the processing algorithm is an encryption algorithm (such as DES, RSA, MD5, etc.), the IO instruction is used to instruct the data to be encrypted using the encryption algorithm. When the processing algorithm is a decryption algorithm, the IO instruction is used to instruct the data to be decrypted using the encryption algorithm.

[0160] In some possible implementations, I / O instructions are generated based on the first task. The process of generating I / O instructions based on the first task can be compared with the process by which a processor on a computing device receives a read / write request and generates read / write instructions based on the read / write request. For the sake of brevity, it will not be elaborated here.

[0161] The acquisition methods for the first task mentioned above can be referred to as acquisition methods 1-3 below:

[0162] Acquisition Method 1: The processor receives the first task from the client.

[0163] For example, the first task can be a data reading task or a data storage task. When the first task is a data reading task, it carries information about the data to be read. When the first task is a data storage task, it carries data to be stored. It should be understood that the first task being a data reading task or a data storage task is merely an example, and this application does not specifically limit the content of the first task.

[0164] Method 2: After receiving the zeroth task from the client, the processor splits the zeroth task into multiple independent subtasks, and then takes one or more of these subtasks as the first task, so that the first task is only a part of the zeroth task.

[0165] For example, the zeroth task can be a RAID algorithm task. After receiving the RAID algorithm task, the processor can split the RAID algorithm task into multiple independent RAID algorithm subtasks based on the data information in the RAID algorithm task, and then use one or more of these RAID algorithm subtasks as the first task. For example, when the RAID algorithm task specifically uses RAID5 to encode data, if the RAID algorithm task carries 12KB of data to be stored, the processor first splits the 12KB of data to be stored into twelve 1KB data parts, making these 1KB data parts independent of each other, and then encodes each 1KB data part using RAID5 as an encoding subtask, resulting in twelve encoding subtasks. Subsequently, the processor can use one of the twelve encoding subtasks as the first task, or it can use multiple encoding subtasks as the first task. It should be understood that the zeroth task being a RAID algorithm task and the first task being one or more encoding subtasks are merely examples, and this application does not specifically limit the specific content of the zeroth task and the first task.

[0166] Method 3: The processor generates the first task while running the application.

[0167] For example, the first task could be to store the data generated by the application.

[0168] It should be noted that the aforementioned clients and processors can be deployed on different computing devices, or on the same computing device.

[0169] It should be understood that the acquisition methods 1 to 3 in step S101 above are merely examples and are not specifically limited here.

[0170] S102: The first solid-state drive processes the storage space indicated by the target address in the IO instruction according to the IO instruction, and sends a completion message to the processor after completing the IO instruction.

[0171] In some possible implementations, the first solid-state drive processes the storage space indicated by the target address in the I / O instruction according to the I / O instruction, which mainly includes the following two stages:

[0172] Phase 1: The first solid-state drive (SSD) determines the target SSD corresponding to the target address from among multiple SSDs based on the target address in the IO instruction and the second set of addresses stored on the first SSD.

[0173] Specifically, after receiving the I / O instruction from the processor, the first solid-state drive obtains the target address from the I / O instruction, then searches for the physical block address corresponding to the target address from the second address set stored on the first solid-state drive, and determines the target solid-state drive corresponding to the target address from multiple solid-state drives based on the physical block address.

[0174] For example, continuing with the example of the 8 addresses in Table 1 of step S101 above as addresses in the second address set, assuming that the target LUN ID and target LBA in the target address are LUN 0 and LBA 2 respectively, the first solid-state drive uses LUN 0 and LBA 2 to look up Table 1 and finds that the PBA corresponding to the combination of LUN 0 and LBA 2 is PBA 2. Then the first solid-state drive will use the solid-state drive that provides the physical block identified by PBA 2 as the target solid-state drive corresponding to the target address.

[0175] It should be understood that since there is a one-to-one mapping between LBAs and PBAs in Table 1, the number of PBAs corresponding to the combination of LUN 0 and LBA 2 is one. When there is a one-to-many mapping between LBAs and PBAs, the number of PBAs corresponding to the combination of LUN 0 and LBA 2 is multiple. When there are multiple PBAs, these PBAs identify multiple physical blocks, which may all be provided by the same SSD or by different SSDs. When multiple PBAs identify physical blocks that are all provided by the same SSD, that SSD is the target SSD corresponding to the target address; when multiple PBAs identify physical blocks that are provided by different SSDs, the multiple SSDs providing these physical blocks are all target SSDs corresponding to the target address. Therefore, the number of target SSDs corresponding to the target address may be one or more, depending on the mapping relationship between LBAs and PBAs set in the data processing system.

[0176] It is understandable that the first SSD can only find the physical block address corresponding to the target address in the second address set stored on the first SSD if the address information in the second address set contains the target address information. In the example above, the target LUNID and target LBA in the target address are LUN 0 and LBA 2 respectively. Therefore, the addresses in the second address set stored on the first SSD must include address 13. Furthermore, the processor also stores the addresses included in the second address set of the first SSD. This allows the processor to determine that the first SSD can correctly resolve the target address and find the target SSD corresponding to the target address.

[0177] The second stage: The first solid-state drive processes the storage space indicated by the target address on the target solid-state drive according to the I / O instructions, and sends a completion message to the processor after completing the I / O instructions. Correspondingly, the processor receives the completion message sent by the first solid-state drive after completing the I / O instructions.

[0178] After identifying the target SSD, the first SSD determines the specific processing content for the storage space indicated by the target address on the target SSD based on the information carried by the I / O instructions. This mainly includes the following scenarios:

[0179] Scenario 1: If the I / O instruction carries a read instruction (e.g., with a read flag), then the first solid-state drive (SSD) needs to write the first data from the storage space indicated by the target address on the target SSD into the dynamic random access memory (DRAM) of the data processing system. DRAM is used for temporary storage of data and instructions that the processor needs to access quickly.

[0180] Scenario 2: If the IO instruction carries a read instruction and carries a RAID algorithm (such as a RAID5 identifier), then the first solid-state drive needs to obtain the first data in the storage space indicated by the target address on the target solid-state drive according to the RAID algorithm (such as RAID5) and write the first data into DRAM.

[0181] Scenario 3: If the IO instruction carries a read instruction and a processing algorithm (such as a decryption algorithm identifier), then the first solid-state drive needs to obtain the first data in the storage space indicated by the target address on the target solid-state drive, use the processing algorithm (such as a decryption algorithm) to calculate the first data to obtain the first calculation result, and write the first calculation result into DRAM.

[0182] Optionally, in scenario 1 or scenario 2 of step S102 above, if the IO instruction carries a data length (such as the data length being the size of a data block), then the first data is a data block (referred to as the first data block).

[0183] Scenario 4: If the IO instruction carries a write instruction (such as carrying a write flag), then the first solid-state drive needs to store the second data in DRAM in the storage space indicated by the target address on the target solid-state drive.

[0184] Scenario 5: If the IO instruction carries a write instruction and carries a RAID algorithm (such as a RAID5 identifier), then the first solid-state drive needs to obtain the second data from DRAM and store the second data in the storage space indicated by the target address on the target solid-state drive according to the RAID algorithm (such as RAID5).

[0185] Scenario 6: If the IO instruction carries a write instruction and a processing algorithm (such as an encryption algorithm identifier), then the first solid-state drive needs to obtain the second data from the DRAM, use the processing algorithm (such as an encryption algorithm) to calculate the second data to obtain the second calculation result, and store the second calculation result in the storage space indicated by the target address on the target solid-state drive.

[0186] Optionally, in scenario 3 or scenario 4 of step S102 above, if the IO instruction carries a data length (such as the data length being the size of a data block), then the second data is a data block (referred to as the second data block).

[0187] For ease of explanation, the following text will uniformly refer to the first data as the first data block and the second data as the second data block, and will illustrate the one-to-one correspondence between data blocks and physical blocks.

[0188] In scenario 1 of step S102, the process of the first solid-state drive executing I / O instructions includes the following three scenarios:

[0189] Scenario 11: The target solid-state drive is the first solid-state drive.

[0190] Referring to Figure 4A, Figure 4A is a flowchart illustrating the execution of I / O instructions by a first solid-state drive according to an embodiment of this application.

[0191] As shown in Figure 4A, in scenario 11, the process of the first solid-state drive (SSD) executing the I / O instruction is as follows: the first SSD reads the first data block from the physical block corresponding to the target address on the first SSD and writes the first data block into the DRAM. After writing the first data block into the DRAM, it means that the first SSD has completed the I / O instruction. Therefore, the first SSD sends a completion message to the processor through the peer-to-peer interconnect bus. After receiving the completion message, the processor initiates a read operation to the DRAM and reads the first data block from the DRAM. In this way, the processor also completes the first task corresponding to the I / O instruction.

[0192] Scenario 12: The first solid-state drive (SSD) is used to manage the storage space indicated by the target address on the target SSD. The target SSD and the first SSD are different SSDs.

[0193] Referring to Figure 4B, Figure 4B is a flowchart illustrating another first solid-state drive executing I / O instructions according to an embodiment of this application.

[0194] As shown in Figure 4B, in scenario 12, the process of the first solid-state drive (SSD) executing the I / O instruction is as follows: The first SSD sends a read instruction to the target SSD via the peer-to-peer interconnect bus. This read instruction instructs the target SSD to read the first data block and write it into DRAM. Upon receiving the read instruction, the target SSD reads the first data block from the physical block corresponding to the target address on the target SSD and writes it into DRAM. After writing the first data block into DRAM, the target SSD sends a read completion message to the first SSD via the peer-to-peer interconnect bus. Upon receiving the read completion message, the first SSD has completed the I / O instruction and therefore sends a completion message to the processor via the peer-to-peer interconnect bus. Upon receiving the completion message, the processor initiates a read operation on the DRAM and reads the first data block from the DRAM. In this way, the processor also completes the first task corresponding to the I / O instruction.

[0195] Scenario 13: The second SSD is used to manage the storage space indicated by the target address on the target SSD. The first SSD, the second SSD, and the target SSD are all different SSDs.

[0196] Referring to Figure 4C, Figure 4C is a flowchart illustrating another first solid-state drive executing I / O instructions according to an embodiment of this application.

[0197] As shown in Figure 4C, in scenario 13, the process of the first solid-state drive (SSD) executing the I / O instruction is as follows: The first SSD sends a fetch instruction to the second SSD via the peer-to-peer interconnect bus. This fetch instruction instructs the second SSD to fetch a first data block from the target SSD. Upon receiving the fetch instruction, the second SSD sends a read instruction to the target SSD via the peer-to-peer interconnect bus. Upon receiving the read instruction, the target SSD reads the first data block from the physical block corresponding to the target address on the target SSD and writes the first data block into DRAM. After writing the first data block into DRAM, the target SSD sends a read completion message to the second SSD via the peer-to-peer interconnect bus. Upon receiving the read completion message, the second SSD sends a fetch completion message to the first SSD via the peer-to-peer interconnect bus. Upon receiving the fetch completion message, the first SSD has completed the I / O instruction and therefore sends a completion message to the processor via the peer-to-peer interconnect bus. Upon receiving the completion message, the processor initiates a read operation on the DRAM and reads the first data block from the DRAM. In this way, the processor also completes the first task corresponding to the I / O instruction.

[0198] In scenario 2 of step S102, the solid-state drive (SSD) uses a RAID algorithm to store data blocks. The RAID algorithm requires dividing the data blocks on the SSD into multiple stripes for management. Each stripe includes one or more data blocks. Multiple stripes are distributed across multiple SSDs to achieve parallel data read / write and redundancy verification.

[0199] Continuing with RAID5 as an example, RAID5 provides a certain degree of redundancy and fault tolerance through technologies such as data striping and data verification. Even if one of the solid-state drives fails, the data blocks on that solid-state drive can still be recovered.

[0200] The process of storing data blocks based on RAID 5 will be explained below with reference to Figure 5.

[0201] Referring to Figure 5, which is a schematic diagram of the result of storing data blocks based on RAID5 according to an embodiment of this application, as shown in Figure 5, each of the four solid-state drives (including SSD 1, SSD 2, SSD 3, and SSD 4) can store multiple data blocks. Specifically, when the solid-state drive used for storing data blocks using RAID5 is the solid-state drive 32 in the data processing system 30 of Figure 2A, SSD 1, SSD 2, SSD 3, and SSD 4 correspond to the four solid-state drives 32 in Figure 2A. When the solid-state drive used for storing data blocks using RAID5 is the solid-state drive 44 in the data processing system 40 of Figure 2B, SSD 1, SSD 2, SSD 3, and SSD 4 can be from different nodes 41 in Figure 2B. For example, SSD 1 and SSD 2 are the solid-state drives 44 in the first node of Figure 2B, and SSD 3 and SSD 4 are the solid-state drives 44 in the second node of Figure 2B. When the solid-state drive used to store data blocks in RAID5 is the solid-state drive 55 in the data processing system 50 of Figure 2C, SSD 1, SSD 2, SSD 3 and SSD 4 are all solid-state drives 55 in the second device cluster 52 in Figure 2B.

[0202] The process of implementing RAID 5 storage for data blocks on four solid-state drives is as follows:

[0203] The parity block PA is calculated using erasure coding (EC) algorithm with data blocks A1, A2, and A3. Then, data blocks A1, A2, A3 and parity block PA are stored in SSD 1, SSD 2, SSD 3 and SSD 4 respectively. The stripe S1 is composed of data blocks A1, A2, A3 and parity block PA.

[0204] The check block PB is calculated using the EC encoding algorithm using data blocks B1, B2, and B3. Then, data blocks B1, B2, B3, and PB are stored in SSD 1, SSD 2, SSD 4, and SSD 3 respectively. The stripe S2 is composed of data blocks B1, B2, B3, and PB.

[0205] The check block PC is calculated using the EC encoding algorithm using data blocks C1, C2, and C3. Then, data blocks C1, C2, C3, and check block PC are stored in SSD 1, SSD 3, SSD 4, and SSD 2 respectively. The stripe S3 is composed of data blocks C1, C2, C3, and check block PC.

[0206] The check block PD is calculated using the EC encoding algorithm using data blocks D1, D2, and D3. Then, data blocks D1, D2, D3, and check block PD are stored in SSD 2, SSD 3, SSD 4, and SSD 1 respectively. The stripe S4 is composed of data blocks D1, D2, D3, and check block PD.

[0207] It should be understood that for other data blocks, a check block is still calculated from three data blocks, and the different check blocks are distributed and stored on different solid-state drives.

[0208] As discussed above, when a solid-state drive (SSD) uses a RAID algorithm to store data, all data in the first storage space is typically divided into multiple stripes. Each stripe contains multiple data blocks. Furthermore, different data blocks within the same stripe are usually distributed across physical blocks on different SSDs.

[0209] When the target SSD fails, causing corruption of the first data block in the physical block corresponding to the target address on the target SSD, the first SSD cannot directly access the first data block. In this case, the first SSD executes the I / O instruction as follows: The first SSD determines the first stripe to which the first data block belongs, retrieves the data block and parity block from the first stripe, uses the data block and parity block from the first stripe to calculate the first data block using the EC decoding algorithm, and writes the first data block to DRAM. After writing the first data block to DRAM, it means the first SSD has completed the I / O instruction. Therefore, the first SSD sends a completion message to the processor via the peer-to-peer interconnect bus. Upon receiving the completion message, the processor initiates a read operation from the DRAM to read the first data block. In this way, the processor also completes the first task corresponding to the I / O instruction.

[0210] The following section continues to illustrate the process of executing I / O instructions on the first solid-state drive, using the RAID5 storage data block shown in Figure 5 as an example, with SSD 1 as the target solid-state drive and data block A1 as the first data block.

[0211] When SSD 1 fails, causing data block A1 on SSD 1 to be corrupted, the process of the first solid-state drive executing I / O instructions is as follows:

[0212] First, the first solid-state drive (SSD) determines that the first stripe to which data block A1 belongs is stripe S1. Therefore, the first SSD retrieves data block A2, data block A3, and parity block PA from stripe S1. The process of retrieving data block A2 depends on the relationship between the first SSD, the SSD managing stripe S1, and the SSD containing data block A2, mainly including the following three scenarios:

[0213] Scenario 1: The first SSD, the SSD managing stripe S1, and the SSD containing data block A2 are the same SSD, corresponding to scenario 11 in step S102 above. Therefore, the first SSD reads data block A2 from SSD 2.

[0214] Scenario 2: The first SSD and the SSD managing stripe S1 are the same SSD, but the first SSD and the SSD containing data block A2 are different SSDs. This corresponds to scenario 12 in step S102 above, specifically: the SSD containing data block A2 is the target SSD. Therefore, the first SSD sends a read command to SSD 2 via the peer-to-peer interconnect bus. This read command instructs SSD 2 to read data block A2 and send it to the first SSD. After receiving the read command, SSD 2 reads data block A2 from SSD 2 and sends it to the first SSD via the peer-to-peer interconnect bus.

[0215] Scenario 3: The first SSD, the SSD managing stripe S1, and the SSD containing data block A2 are all different SSDs, corresponding to scenario 13 in step S102 above. Specifically, the SSD containing data block A2 is the target SSD, and the SSD managing stripe S1 is the second SSD. Therefore, the first SSD sends a retrieval command to the SSD managing stripe S1 via the peer-to-peer interconnect bus. This retrieval command instructs the SSD managing stripe S1 to retrieve data block A2 from SSD 2. After receiving the retrieval command, the SSD managing stripe S1 sends a read command to SSD 2 via the peer-to-peer interconnect bus. After receiving the read command, SSD 2 reads data block A2 from SSD 2 and sends data block A2 to the SSD managing stripe S1 via the peer-to-peer interconnect bus. After receiving data block A2, the SSD managing stripe S1 sends data block A2 to the first SSD via the peer-to-peer interconnect bus.

[0216] It is understandable that the process of obtaining data block A3 and parity block PA is similar to the process of obtaining data block A2 described above. For the sake of brevity in the manual, it will not be elaborated here.

[0217] Subsequently, the first solid-state drive uses data block A2, data block A3, and parity block PA to calculate data block A1 using the EC decoding algorithm, thereby realizing the recovery of data block A1.

[0218] In one specific implementation, the SSD controller in the first solid-state drive (SSD) is not only responsible for managing and controlling the overall operation of the first SSD, but also for performing various calculation and processing tasks. In this case, the SSD controller in the first SSD obtains data block A2, data block A3, and parity block PA, then uses the EC decoding algorithm to calculate data block A2, data block A3, and parity block PA, and uses the calculation result as data block A1.

[0219] In another specific implementation, the SSD controller in the first solid-state drive (SSD) manages and controls the overall operation of the SSD, while the SSD processor in the first SSD executes the instructions issued by the SSD controller and performs various calculations and processing tasks. The SSD controller and SSD processor can be independent of each other or integrated on the same PCB. The SSD processor can be, for example, an Advanced Reduced Instruction Set Computing (RISC) processor (ARM). In this case, the SSD controller obtains data block A2, data block A3, and parity block PA, and then sends them to the SSD processor. The SSD processor uses the EC decoding algorithm to calculate the values ​​of data block A2, data block A3, and parity block PA, and sends the calculation result as data block A1 to the SSD controller.

[0220] Next, the first solid-state drive (SSD) writes data block A1 to the DRAM. After writing data block A1 to the DRAM, it means the first SSD has completed the I / O instruction. Therefore, the first SSD sends a completion message to the processor via the peer-to-peer interconnect bus. Upon receiving the completion message, the processor initiates a read operation from the DRAM to read data block A1. In this way, the processor has also completed the first task corresponding to the I / O instruction.

[0221] It should be understood that the above example only uses RAID5 as the RAID algorithm, with four solid-state drives (SSDs) used to store data blocks, and one target SSD and one first data block. In practical applications, the RAID algorithm can also be RAID6, RAIDTP, etc., and the number of SSDs used to store data blocks can be more or less, and the number of target SSDs and the first data block can also be more. This application does not make any specific limitations.

[0222] In scenario 3 of step S102, the process of the first solid-state drive (SSD) executing the I / O instruction is as follows: the first SSD obtains the first data block from the physical block corresponding to the target address on the target SSD, performs calculations (such as decryption) on the first data block to obtain a first calculation result (such as a decryption result), and then writes the first calculation result into DRAM. After writing the first calculation result into DRAM, it means that the first SSD has completed the I / O instruction. Therefore, the first SSD sends a completion message to the processor through the peer-to-peer interconnect bus. After receiving the completion message, the processor initiates a read operation on the DRAM to read the first calculation result from the DRAM. In this way, the processor also completes the first task corresponding to the I / O instruction.

[0223] The process by which the first solid-state drive (SSD) acquires the first data block depends on the relationship between the first SSD, the target SSD, and the SSD managing the target SSD. If the first SSD, the target SSD, and the SSD managing the target SSD are the same SSD, the process of the first SSD acquiring the first data block can refer to the process of the first SSD acquiring data block A2 in scenario 2, case 1 of step S102. If the first SSD and the SSD managing the target SSD are the same SSD, but the first SSD and the target SSD are different SSDs, the process of the first SSD acquiring the first data block can refer to the process of the first SSD acquiring data block A2 in scenario 2, case 2 of step S102. If the first SSD, the target SSD, and the SSD managing the target SSD are all different SSDs, the process of the first SSD acquiring the first data block can refer to the process of the first SSD acquiring data block A2 in scenario 2, case 3 of step S102.

[0224] Specifically, the action of acquiring the first data block is performed by the SSD controller in the first solid-state drive, the action of calculating the first data block to obtain the first calculation result is performed by the SSD processor in the first solid-state drive, and the first data block is sent from the SSD controller to the SSD processor.

[0225] In scenario 4 of step S102, the process of the first solid-state drive executing I / O instructions includes the following three scenarios:

[0226] Scenario 41: The target solid-state drive corresponding to the target address is the first solid-state drive.

[0227] Referring to Figure 6A, Figure 6A is a flowchart illustrating another first solid-state drive executing I / O instructions according to an embodiment of this application.

[0228] As shown in Figure 6A, in scenario 41, the process of the first solid-state drive (SSD) executing the I / O instruction is as follows: The SSD controller in the first SSD initiates a read operation to the DRAM, reads the second data block from the DRAM, and writes the second data block to the physical block corresponding to the target address on the first SSD. After writing the second data block to the physical block corresponding to the target address on the first SSD, it means that the first SSD has completed the I / O instruction. Therefore, the first SSD sends a completion message to the processor through the peer-to-peer interconnect bus. When the processor receives the completion message, it is equivalent to the processor also completing the first task corresponding to the I / O instruction.

[0229] Scenario 42: The first solid-state drive (SSD) is used to manage the storage space indicated by the target address on the target SSD. The target SSD and the first SSD are different SSDs.

[0230] Referring to Figure 6B, Figure 6B is a flowchart illustrating another first solid-state drive executing I / O instructions according to an embodiment of this application.

[0231] As shown in Figure 6B, in scenario 42, the process of the first solid-state drive (SSD) executing the I / O instruction is as follows: The first SSD sends a write instruction to the target SSD via the peer-to-peer interconnect bus. This write instruction instructs the target SSD to read the second data block from the DRAM and write it to the physical block corresponding to the target address on the target SSD. After receiving the write instruction, the SSD controller in the target SSD initiates a read operation to the DRAM, reads the second data block from the DRAM, and writes it to the physical block corresponding to the target address on the target SSD. After writing the second data block to the physical block corresponding to the target address on the target SSD, the target SSD sends a write completion message to the first SSD via the peer-to-peer interconnect bus. Upon receiving the write completion message, the first SSD has completed the I / O instruction. Therefore, the first SSD sends a completion message to the processor via the peer-to-peer interconnect bus. The processor receiving the completion message signifies that the processor has also completed the first task corresponding to the I / O instruction.

[0232] Scenario 43: The second SSD is used to manage the storage space indicated by the target address on the target SSD. The first SSD, the second SSD, and the target SSD are all different SSDs.

[0233] Referring to Figure 6C, Figure 6C is a flowchart illustrating another first solid-state drive executing I / O instructions according to an embodiment of this application.

[0234] As shown in Figure 6C, in scenario 43, the process of the first solid-state drive (SSD) executing the I / O instruction is as follows: The first SSD sends a storage instruction to the second SSD via the peer-to-peer interconnect bus. This storage instruction instructs the second SSD to notify the target SSD to write the second data block. After receiving the storage instruction, the second SSD sends a write instruction to the target SSD via the peer-to-peer interconnect bus. After receiving the write instruction, the SSD controller in the target SSD initiates a read operation on the DRAM, reads the second data block from the DRAM, and writes the second data block to the physical block corresponding to the target address on the target SSD. After writing the second data block to the physical block corresponding to the target address on the target SSD, the target SSD sends a write completion message to the second SSD via the peer-to-peer interconnect bus. After receiving the write completion message, the second SSD sends a storage completion message to the first SSD via the peer-to-peer interconnect bus. After receiving the storage completion message, the first SSD has completed the I / O instruction. Therefore, the first SSD sends a completion message to the processor via the peer-to-peer interconnect bus. The processor receiving the completion message means that the processor has also completed the first task corresponding to the I / O instruction.

[0235] In scenario 5 of step S102, the solid-state drive (SSD) uses a RAID algorithm to store data blocks. The process of the first SSD executing the I / O instruction is as follows: The SSD controller in the first SSD initiates a read operation to the DRAM, reads the second data block from the DRAM, determines the second stripe to which the second data block belongs, obtains the data block in the second stripe, calculates the parity block using the data block in the second stripe and the second data block through the EC encoding algorithm, stores the second data block on the target SSD at the physical block corresponding to the target address, and stores the parity block in the second stripe. After writing the second data block to the physical block corresponding to the target address on the first SSD, it means that the first SSD has completed the I / O instruction. Therefore, the first SSD sends a completion message to the processor through the peer-to-peer interconnect bus. The processor receiving the completion message is equivalent to the processor also completing the first task corresponding to the I / O instruction.

[0236] The following section continues to illustrate the process of executing I / O instructions on the first solid-state drive, using the RAID5 storage data block shown in Figure 5 above as an example, with SSD 1 as the target solid-state drive and data block E1 as the second data block.

[0237] Suppose that data block E1 is to be stored on stripe S5, and there are already data blocks E2 and E3 on stripe S5. Data block E2 is stored on SSD 2, and data block E3 is stored on SSD 3.

[0238] First, the first solid-state drive (SSD) determines that the second stripe to which data block E1 belongs is stripe S5. Therefore, the first SSD acquires data blocks E2 and E3 in stripe S5. The process of the first SSD acquiring data blocks E2 and E3 can be referred to the process of the first SSD acquiring data block A2 in scenario 2 of step S102 above. For the sake of brevity, it will not be elaborated here.

[0239] Subsequently, the first solid-state drive uses data blocks E1, E2, and E3 to calculate the test block PE using the EC encoding algorithm.

[0240] Next, the first SSD stores data block E1 on the physical block corresponding to the target address on SSD 1, and stores the verification block PE on SSD 4. The process of the first SSD storing data block E1 depends on the relationship between the first SSD, the SSD managing stripe S5, and the SSD containing data block E1, mainly including the following three cases:

[0241] Scenario 1: The first SSD, the SSD managing stripe S5, and the SSD containing data block E1 are the same SSD, corresponding to scenario 41 in step S102 above. Therefore, the first SSD writes data block E1 to the physical block corresponding to the target address on SSD 1.

[0242] Scenario 2: The first SSD and the SSD managing stripe S5 are the same SSD, but the first SSD and the SSD containing data block E1 are different SSDs. This corresponds to scenario 42 in step S102 above, specifically: the SSD containing data block E1 is the target SSD. Therefore, the first SSD sends a write command to SSD 1 via the peer-to-peer interconnect bus. This write command instructs SSD 1 to read data block E1 from DRAM and write data block E1 to the physical block corresponding to the target address on SSD 1. After receiving the write command, the SSD controller in SSD 1 initiates a read operation to DRAM, reads data block E1 from DRAM, and writes data block E1 to the physical block corresponding to the target address on SSD 1.

[0243] Scenario 3: The first SSD, the SSD managing stripe S5, and the SSD containing data block E1 are all different SSDs, corresponding to scenario 43 in step S102 above. Specifically, the SSD containing data block E1 is the target SSD, and the SSD managing stripe S5 is the second SSD. Therefore, the first SSD sends a storage command to the SSD managing stripe S5 via the peer-to-peer interconnect bus. This storage command instructs the SSD managing stripe S5 to notify SSD 1 to write data block E1. After receiving the storage command, the SSD managing stripe S5 sends a write command to SSD 1 via the peer-to-peer interconnect bus. After receiving the write command, the SSD controller in SSD 1 initiates a read operation on the DRAM, reads data block E1 from the DRAM, and writes data block E1 to the physical block corresponding to the target address on SSD 1.

[0244] The process of storing the first solid-state drive (SSD) parity block (PE) depends on the relationship between the first SSD, the SSD managing the S5 stripe, and the SSD where the PE parity block resides. This mainly includes the following three scenarios:

[0245] Scenario 1: The first SSD, the SSD managing stripe S5, and the SSD containing the parity block PE are the same SSD, corresponding to scenario 41 in step S102 above. Therefore, the first SSD writes the parity block PE onto SSD 4.

[0246] Scenario 2: The first SSD and the SSD managing stripe S5 are the same SSD, but the first SSD and the SSD containing the parity block PE are different SSDs. This corresponds to scenario 42 in step S102 above, specifically: the SSD containing the parity block PE is the second SSD. Therefore, the first SSD sends a write command and the parity block PE to SSD 4 via the peer-to-peer interconnect bus. This write command instructs SSD 4 to write the parity block PE onto SSD 4. After receiving the write command and the parity block PE, SSD 1 writes the parity block PE onto SSD 4.

[0247] Scenario 3: The first SSD, the SSD managing stripe S5, and the SSD containing the parity block PE are all different SSDs, corresponding to scenario 43 in step S102 above. Specifically, the SSD containing the parity block PE is the target SSD, and the SSD managing stripe S5 is the second SSD. Therefore, the first SSD sends a storage command and the parity block PE to the SSD managing stripe S5 via the peer-to-peer interconnect bus. This storage command instructs the SSD managing stripe S5 to notify SSD 4 to write the parity block PE. After receiving the storage command and the parity block PE, the SSD managing stripe S5 sends a write command and the parity block PE to SSD 4 via the peer-to-peer interconnect bus. After receiving the write command and the parity block PE, SSD 4 writes the parity block PE to itself.

[0248] In summary, the result of using RAID5 to store data block E1 on a solid-state drive is shown in Figure 7. Referring to Figure 7, Figure 7 is a schematic diagram of another result based on RAID5 data block storage provided in an embodiment of this application.

[0249] In scenario 6 of step S102, the process of the first solid-state drive (SSD) executing the I / O instruction is as follows: the SSD controller in the first SSD initiates a read operation to the DRAM, reads the second data block from the DRAM, performs calculations (such as encryption) on the second data block to obtain a second calculation result (such as an encryption result), and then stores the second calculation result in the physical block corresponding to the target address on the target SSD. After storing the second calculation result in the physical block corresponding to the target address on the target SSD, it means that the first SSD has completed the I / O instruction. Therefore, the first SSD sends a completion message to the processor through the peer-to-peer interconnect bus. When the processor receives the completion message, it is equivalent to the processor also completing the first task corresponding to the I / O instruction.

[0250] The process of the first solid-state drive (SSD) storing the second calculation result depends on the relationship between the first SSD, the target SSD, and the SSD that manages the target SSD. If the first SSD, the target SSD, and the SSD that manages the target SSD are the same SSD, then the process of the first SSD storing the second calculation result can refer to scenario 1 of step S102, where the first SSD stores the verification block PE. If the first SSD and the SSD that manages the target SSD are the same SSD, but the first SSD and the target SSD are different SSDs, then the process of the first SSD obtaining the first data block can refer to scenario 2 of step S102, where the first SSD stores the verification block PE. If the first SSD, the target SSD, and the SSD that manages the target SSD are all different SSDs, then the process of the first SSD obtaining the first data block can refer to scenario 3 of step S102, where the first SSD stores the verification block PE.

[0251] Specifically, the action of calculating the second data block to obtain the second calculation result is performed by the SSD processor in the first solid-state drive, the action of storing the second calculation result is performed by the SSD controller in the first solid-state drive, and the second calculation result is sent by the SSD processor to the SSD controller.

[0252] It should be understood that Figure 3 above only illustrates the process of the processor issuing tasks to the first solid-state drive (SSD) in the form of I / O instructions. In this technical solution, the processor can also send I / O instructions carrying a first address to the second SSD. Correspondingly, the second SSD receives the I / O instructions carrying the first address from the processor. The second SSD processes the storage space indicated by the first address according to the I / O instructions carrying the first address, and sends a first completion message to the processor after completing the I / O instructions carrying the first address. Correspondingly, the processor receives the first completion message sent by the second SSD after completing the I / O instructions carrying the first address. The I / O instructions carrying the first address are generated based on a second task. The second task is independent of the first task. The I / O instructions carrying the first address instruct the second SSD to determine the target SSD corresponding to the first address from multiple SSDs based on the first address and the second address set stored on the second SSD. The first address includes a first LUN ID and a first LBA. The storage space indicated by the first address is on the target SSD corresponding to the first address. The I / O instructions carrying the first address instruct processing of the storage space indicated by the first address. The process of the processor sending an I / O instruction carrying the first address to the second solid-state drive can be referred to the execution process of step S101 above. The process of the second solid-state drive processing the storage space indicated by the first address according to the I / O instruction carrying the first address, and sending a first completion message to the processor after completing the I / O instruction carrying the first address, can be referred to the execution process of step S102 above. In this way, the processor can distribute multiple tasks to different solid-state drives, enabling multiple solid-state drives to execute tasks in parallel, thereby improving the execution efficiency of multiple tasks.

[0253] It should be noted that when the data processing method of Figure 3 is applied to the data processing system 30 of Figure 2A, the processor in steps S101 and S102 can be the processor 31 in the data processing system 30 of Figure 2A, the first address set is the first address set 310 on the processor 31 in the data processing system 30 of Figure 2A, the address in the first address set is the address 311 in the first address set 310 on the processor 31 in the data processing system 30 of Figure 2A, the first solid-state drive, the second solid-state drive, and the target solid-state drive can be the solid-state drive 32 in the data processing system 30 of Figure 2A, the second address set is the second address set 320 on the solid-state drive 32 in the data processing system 30 of Figure 2A, and the peer-to-peer interconnect bus can be the peer-to-peer interconnect bus 33 in the data processing system 30 of Figure 2A.

[0254] When the data processing method of Figure 3 is applied to the data processing system 40 of Figure 2B, the processor in steps S101 and S102 can be the processor 43 in the data processing system 40 of Figure 2B, the first address set is the first address set 430 on the processor 43 in the data processing system 40 of Figure 2B, the address in the first address set is the address 431 in the first address set 430 on the processor 43 in the data processing system 40 of Figure 2B, the first solid-state drive, the second solid-state drive, and the target solid-state drive can be the solid-state drive 44 in the data processing system 40 of Figure 2B, the second address set is the second address set 440 on the solid-state drive 44 in the data processing system 40 of Figure 2B, and the peer-to-peer interconnect bus can be the peer-to-peer interconnect bus 42 in the data processing system 40 of Figure 2B.

[0255] When the data processing method of Figure 3 is applied to the data processing system 50 of Figure 2C, the processor in steps S101 and S102 can be the processor 54 in the data processing system 50 of Figure 2C, the first address set is the first address set 540 on the processor 54 in the data processing system 50 of Figure 2C, the address in the first address set is the address 541 in the first address set 540 on the processor 54 in the data processing system 50 of Figure 2C, the first solid-state drive, the second solid-state drive, and the target solid-state drive can be the solid-state drive 55 in the data processing system 50 of Figure 2C, the second address set is the second address set 550 on the solid-state drive 55 in the data processing system 50 of Figure 2C, and the peer-to-peer interconnect bus can be the peer-to-peer interconnect bus 53 in the data processing system 50 of Figure 2C.

[0256] In summary, based on the first solid-state drive's ability to correctly parse the target address carried by I / O instructions, this technical solution supports the first solid-state drive in performing diverse tasks on the target solid-state drive, including reading data from the target solid-state drive, storing data on the target solid-state drive, and performing calculations on the data. Furthermore, when the processor needs to process the storage space indicated by the target address, the processor delegates the task to the first solid-state drive by sending I / O instructions. The processor only needs to receive notification that the first solid-state drive has completed the task. When the first solid-state drive completes the task, it is equivalent to the processor completing the task. It is understandable that the number of solid-state drives in a data processing system far exceeds the number of processors. Therefore, having the solid-state drive perform tasks that would otherwise be performed by the processor reduces the processor's workload and enables the parallel execution of a large number of tasks, improving the execution efficiency of multiple tasks.

[0257] On the other hand, this technical solution provides a data processing method for CNC separation suitable for different scenarios based on the architecture of the data processing system in Figure 2A, Figure 2B or Figure 2C.

[0258] In a scenario where the target SSD and the first SSD are different SSDs, and the first SSD manages the storage space indicated by the target address on the target SSD, the first SSD sends control commands (such as read commands and write commands) to the target SSD to instruct the target SSD to transfer first or second data between itself and the DRAM. This achieves separation of the data plane and the control plane; control flow occurs between the first SSD and the target SSD, while data flow occurs between the target SSD and the DRAM. This enables centralized global control of the control plane and high-performance data transmission.

[0259] In scenarios where data storage and management are separated, the first solid-state drive (SSD) sends control commands (such as fetch or store commands) to the SSD managing the storage space indicated by the target address (i.e., the second SSD), instructing the second SSD to complete a first or second data transfer task between the target SSD and the DRAM. This means that in such a scenario, the first SSD is unaware of the specific execution process of the task; it only needs to instruct the second SSD to complete the task. The first SSD does not need to participate in how the second SSD completes the task; it is considered to have completed the task upon receiving a notification from the second SSD that it has finished. This achieves separation of the data plane and control plane. Control flow occurs between the first SSD, the second SSD, and the target SSD, while data flow occurs only between the target SSD and the DRAM. This enables globally centralized control of the control plane and high-performance data transfer.

[0260] In applications where data is stored using RAID algorithms, a first solid-state drive (SSD) is used to recover first data and provide it to the processor, and also to store second data. Specifically, when the first data is corrupted, since the data processing system uses RAID algorithms for data storage, the first SSD, upon receiving a task to retrieve the first data, can calculate the first data using the RAID algorithm and then provide the recovered data to the processor. When the second data needs to be stored, the first SSD, upon receiving a task to store the second data, needs to first use the RAID algorithm to calculate the parity block in the second stripe to which the second data belongs, and then store the second data. This ensures that if data in the second stripe is corrupted, the corrupted data can be recovered promptly based on the parity block. As can be seen, in this technical solution, the first SSD has data calculation and processing capabilities, and can execute other calculation and processing tasks issued by the processor, except for error detection and garbage collection.

[0261] This application also provides a computer program product containing instructions. This computer program product may be a software or program product containing instructions, capable of running on a computing device or stored on any usable medium. When the computer program product runs on a computing device, it causes the computing device to perform the data processing method described in FIG3.

[0262] This application also provides a computer-readable storage medium. The computer-readable storage medium can be any available medium that a computing device can store, or a data storage device such as a data center containing one or more available media. The available medium can be a magnetic medium (e.g., floppy disk, hard disk, magnetic tape), an optical medium (e.g., DVD), or a semiconductor medium (e.g., solid-state drive). The computer-readable storage medium includes instructions that instruct the computing device to perform the data processing method of FIG3 described above.

[0263] It should be understood that in the embodiments of this application, "when," "...when," and "if" all refer to the device making corresponding processing under certain objective circumstances, and are not time-limited, nor do they require the device to make a judgment action, nor do they imply any other limitations.

[0264] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of this application, and are not intended to limit them. Although this application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the protection scope of the technical solutions of the embodiments of this application.

Claims

1. A data processing system, characterized in that, Includes a processor and multiple solid-state drives, The plurality of solid-state drives are used to provide at least one physical block, the at least one physical block constituting a first storage space. The processor stores a first address set, which includes multiple addresses, each address indicating one or more physical blocks in the first storage space. Each solid-state drive stores a second set of addresses, and the first set of addresses includes the second set of addresses. The first solid-state drive (SSD) of the plurality of SSDs is used to determine at least one target SSD from the plurality of SSDs after receiving an input / output (I / O) instruction from the processor, based on the target address in the I / O instruction and a second set of addresses stored on the first SSD. The I / O instruction is used to instruct processing of the storage space indicated by the target address, and the storage space indicated by the target address is on the at least one target SSD.

2. The system according to claim 1, characterized in that, The address is based on Logical Unit Number (LUN) semantics to indicate the physical block in the first storage space.

3. The system according to claim 2, characterized in that, The address includes a Logical Unit Number (LUN) ID, a Logical Block Address (LBA), and a Physical Block Address (PBA). The LUN ID is used to identify a LUN in the first storage space. The LUN includes at least one logical block, and there is a mapping relationship between the logical block and the physical block. The LBA is used to identify the logical block, and the PBA is used to identify the physical block.

4. The system according to any one of claims 1-3, characterized in that, The data processing system includes a peer-to-peer interconnect bus. The processor is used to send I / O instructions to each solid-state drive via the peer-to-peer interconnect bus.

5. The system according to any one of claims 1-4, characterized in that, The processor and the plurality of solid-state drives are deployed in the same device.

6. The system according to any one of claims 1-4, characterized in that, The processor and the first solid-state drive are deployed in different devices.

7. The system according to claim 6, characterized in that, The data processing system includes multiple processors, which are deployed in a first device cluster, and multiple solid-state drives are deployed in a second device cluster. The first device cluster and the second device cluster are different device clusters.

8. The system according to any one of claims 1-7, characterized in that, The first solid-state drive is used to process the storage space indicated by the target address according to the IO instructions, wherein, The I / O instruction is used to instruct the storage space indicated by the target address on the target solid-state drive to be stored in the dynamic random access memory (DRAM); or, The I / O instruction is used to instruct the calculation of the first data to obtain a first calculation result, and to store the first calculation result in the DRAM; or... The I / O instruction is used to instruct the second data in the DRAM to be stored in the storage space indicated by the target address on the target solid-state drive; or, The IO instruction is used to instruct the second data to be calculated to obtain a second calculation result, and to store the second calculation result in the storage space indicated by the target address on the target solid-state drive.

9. The system according to claim 8, characterized in that, The first solid-state drive (SSD) is used to manage the storage space indicated by the target address on the target SSD, wherein the target SSD and the first SSD are different SSDs. The first solid-state drive is specifically used to send a read command to the target solid-state drive and receive a read completion message sent by the target solid-state drive after completing the read command; The read instruction is used to instruct the target solid-state drive to read the first data and write the first data into the DRAM.

10. The system according to claim 8, characterized in that, The plurality of solid-state drives (SSDs) includes a second SSD, which manages the storage space indicated by the target address on the target SSD. The first SSD, the second SSD, and the target SSD are all different SSDs. The first solid-state drive is specifically used to send an acquisition command to the second solid-state drive and receive an acquisition completion message sent by the second solid-state drive after completing the acquisition command; The acquisition instruction is used to instruct the second solid-state drive to notify the target solid-state drive to read the first data and write the first data into the DRAM.

11. The system according to claim 8, characterized in that, The I / O instruction is used to instruct the first data to be stored in the DRAM according to the disk array RAID algorithm. The first solid-state drive is specifically used to obtain data blocks and check blocks in the first stripe, obtain the first data based on the data blocks and the check blocks, and write the first data into the DRAM; The first strip is the strip to which the first data belongs.

12. The system according to claim 8, characterized in that, The first solid-state drive (SSD) is used to manage the storage space indicated by the target address on the target SSD, wherein the target SSD and the first SSD are different SSDs. The first solid-state drive is specifically used to send a write command to the target solid-state drive and receive a write completion message sent by the target solid-state drive after completing the write command; The write instruction is used to instruct the target solid-state drive to read the second data from the DRAM and write the second data into the storage space indicated by the target address on the target solid-state drive.

13. The system according to claim 8, characterized in that, The plurality of solid-state drives (SSDs) includes a second SSD, which manages the storage space indicated by the target address on the target SSD. The first SSD, the second SSD, and the target SSD are all different SSDs. The first solid-state drive is specifically used to send a storage instruction to the second solid-state drive and receive a storage completion message sent by the second solid-state drive after completing the storage instruction; The storage instruction is used to instruct the second solid-state drive to notify the target solid-state drive to read the second data from the DRAM and write the second data to the storage space indicated by the target address on the target solid-state drive.

14. The system according to claim 8, characterized in that, The IO instruction is used to instruct the second data to be stored on the target solid-state drive in the storage space indicated by the target address, according to the RAID algorithm. The first solid-state drive is specifically used to read the second data from the DRAM, obtain a data block in the second stripe, obtain a check block based on the data block and the second data, store the second data block in the physical block corresponding to the target address on the target solid-state drive, and store the check block in the second stripe; The second strip is the strip to which the second data belongs.

15. A data processing method, characterized in that, This invention relates to a data processing system, comprising a processor and multiple solid-state drives (SSDs). The SSDs provide at least one physical block, which constitutes a first storage space. The processor stores a first address set, which includes multiple addresses, each indicating one or more physical blocks within the first storage space. The SSDs include a first SSD, which stores a second address set, which includes the second address set. The method includes: The processor sends input / output I / O instructions to the first solid-state drive; The processor receives a completion message sent by the first solid-state drive after completing the IO instruction; The IO instruction carries a target address; the IO instruction is used to instruct the first solid-state drive to determine at least one target solid-state drive from the plurality of solid-state drives based on the target address and the second address set; the IO instruction is used to instruct processing of the storage space indicated by the target address; the storage space indicated by the target address is on the at least one target solid-state drive.

16. The method according to claim 15, characterized in that, After the processor sends an I / O instruction to the first solid-state drive and before the processor receives a completion message sent by the first solid-state drive after completing the I / O instruction, the method further includes: The first solid-state drive processes the storage space indicated by the target address according to the I / O instructions, wherein, The I / O instruction is used to instruct the storage space indicated by the target address on the target solid-state drive to be stored in the dynamic random access memory (DRAM); or, The I / O instruction is used to instruct the calculation of the first data to obtain a first calculation result, and to store the first calculation result in the DRAM; or... The I / O instruction is used to instruct the second data in the DRAM to be stored in the storage space indicated by the target address on the target solid-state drive; or, The IO instruction is used to instruct the second data to be calculated to obtain a second calculation result, and to store the second calculation result in the storage space indicated by the target address on the target solid-state drive.

17. The method according to claim 16, characterized in that, The first solid-state drive (SSD) is used to manage the storage space indicated by the target address on the target SSD, wherein the target SSD and the first SSD are different SSDs. The first solid-state drive processes the storage space indicated by the target address according to the IO instructions, including: The first solid-state drive sends a read command to the target solid-state drive, the read command being used to instruct the target solid-state drive to read the first data and write the first data into the DRAM; The first solid-state drive receives a read completion message sent by the target solid-state drive after completing the read instruction.

18. The method according to claim 16, characterized in that, The plurality of solid-state drives (SSDs) includes a second SSD, which manages the storage space indicated by the target address on the target SSD. The first SSD, the second SSD, and the target SSD are all different SSDs. The first solid-state drive processes the storage space indicated by the target address according to the IO instructions, including: The first solid-state drive sends an acquisition command to the second solid-state drive, the acquisition command being used to instruct the second solid-state drive to notify the target solid-state drive to read the first data and write the first data into the DRAM; The first solid-state drive receives an acquisition completion message sent by the second solid-state drive after completing the acquisition instruction.

19. The method according to claim 16, characterized in that, The I / O instruction is used to instruct the first data to be stored in the DRAM according to the disk array RAID algorithm. The first solid-state drive processes the storage space indicated by the target address according to the IO instructions, including: The first solid-state drive acquires the data block and check block in the first stripe, where the first stripe is the stripe to which the first data belongs; The first solid-state drive obtains the first data based on the data block and the check block; The first solid-state drive writes the first data into the DRAM.

20. The method according to claim 16, characterized in that, The first solid-state drive (SSD) is used to manage the storage space indicated by the target address on the target SSD, wherein the target SSD and the first SSD are different SSDs. The first solid-state drive processes the storage space indicated by the target address according to the IO instructions, including: The first solid-state drive sends a write command to the target solid-state drive, the write command being used to instruct the target solid-state drive to read the second data from the DRAM and write the second data into the storage space indicated by the target address on the target solid-state drive; The first solid-state drive receives a write completion message sent by the target solid-state drive after completing the write instruction.

21. The method according to claim 16, characterized in that, The plurality of solid-state drives (SSDs) includes a second SSD, which manages the storage space indicated by the target address on the target SSD. The first SSD, the second SSD, and the target SSD are all different SSDs. The first solid-state drive processes the storage space indicated by the target address according to the IO instructions, including: The first solid-state drive sends a storage instruction to the second solid-state drive, the storage instruction being used to instruct the second solid-state drive to notify the target solid-state drive to read the second data from the DRAM and write the second data into the storage space indicated by the target address on the target solid-state drive; The first solid-state drive receives a storage completion message sent by the second solid-state drive after completing the storage instruction.

22. The method according to claim 16, characterized in that, The IO instruction is used to instruct the second data to be stored on the target solid-state drive in the storage space indicated by the target address, according to the RAID algorithm. The first solid-state drive processes the storage space indicated by the target address according to the IO instructions, including: The first solid-state drive reads the second data from the DRAM; The first solid-state drive acquires data blocks from the second stripe, where the second stripe is the stripe to which the second data belongs; The first solid-state drive obtains a verification block based on the data block and the second data; The first solid-state drive stores the second data block in the physical block corresponding to the target address on the target solid-state drive, and stores the check block in the second stripe.

23. The method according to any one of claims 15-22, characterized in that, The address includes a Logical Unit Number (LUN) ID, a Logical Block Address (LBA), and a Physical Block Address (PBA). The target address includes a target LUN ID and a target LBA. The LUN ID is used to identify a LUN in the first storage space. The LUN includes at least one logical block. There is a mapping relationship between the logical block and the physical block. The LBA is used to identify the logical block, and the PBA is used to identify the physical block.

24. The method according to any one of claims 15-23, characterized in that, The plurality of solid-state drives (SSDs) includes a second SSD, and the second set of addresses stored on the second SSD belongs to the first set of addresses. The method further includes: The processor sends an I / O instruction carrying a first address to the second solid-state drive; The processor receives a first completion message sent by the second solid-state drive after completing the IO instruction carrying the first address; Wherein, the I / O instructions sent by the processor to the first solid-state drive are generated based on the first task, and the I / O instructions carrying the first address are generated based on the second task. The first task and the second task are independent of each other. The I / O instructions carrying the first address are used to instruct the second solid-state drive to determine the target solid-state drive corresponding to the first address from the plurality of solid-state drives according to the first address and the second address set stored on the second solid-state drive. The I / O instructions carrying the first address are used to instruct the processing of the storage space indicated by the first address. The storage space indicated by the first address is on the target solid-state drive corresponding to the first address.

25. A computer program product containing instructions, characterized in that, When the instructions are executed by the computing device, the computing device performs the method as described in any one of claims 15-24.

26. A computer-readable storage medium, characterized in that, It includes computer program instructions that, when run on a computing device, implement the method as described in any one of claims 15-24.

Images