Data processing system and method

Abstract

The invention provides a data processing system and method which can be applied to the technical field of storage. A data processing system includes a processor and a plurality of solid state disks. Wherein the physical blocks provided by the plurality of solid state disks form a first storage space. The processor stores a first set of addresses. Each address in the first address set is used for indicating a physical block in the first storage space. And each solid state disk stores a second address set. The first address set includes a second address set. And the first solid state disk is used for determining a target solid state disk from the plurality of solid state disks according to a target address in the IO instruction and a second address set stored on the first solid state disk after receiving the IO instruction from the processor. The IO instruction is used for indicating to process the storage space indicated by the target address. The data processing system supports the processor to distribute a large number of tasks to the solid state disk for execution, so that the workload of the processor is reduced, and the parallel execution efficiency of multiple tasks is improved.

Description

Technical Field

[0001] This application relates to the field of storage technology, and in particular to a data processing system and method. Background Technology

[0002] Servers consist of processors and solid-state drives (SSDs). SSDs store data, while processors handle the data on the SSDs, performing tasks such as management, reading, writing, and computation. Typically, a server has 1-8 processors and dozens to thousands of SSDs. When a server receives a large number of processing tasks to process data on different SSDs, these tasks are shared among all the processors in the server. Each processor can only execute the tasks assigned to it one by one. Therefore, the performance bottleneck in executing processing tasks lies with the processor; both execution efficiency and the number of tasks that can be executed are limited by the processor's computing power.

[0003] To enhance a server's ability to process data from SSDs, an expansion card—a Redundant Array of Independent Disks (RAID) card—can be installed. RAID cards are specifically designed to manage disk arrays composed of multiple SSDs, providing more efficient data transfer and processing capabilities, thereby accelerating SSD operations. Typically, depending on the server's storage needs and the complexity of the RAID card configuration, the number of RAID cards in a server ranges from 1 to 9. When a server has RAID cards installed, and it receives a large number of processing tasks to handle data from different SSDs, the processor distributes these tasks to the RAID cards, which then share the workload. Each RAID card can only execute the tasks assigned to it one by one. Therefore, for servers with RAID cards, the performance bottleneck for processing tasks lies with the RAID cards themselves; both execution efficiency and the number of tasks that can be executed are limited by the capabilities of the RAID cards.

[0004] In conclusion, in a server, the number of processors and RAID cards is far less than the number of SSDs. Therefore, using a small number of processors or RAID cards to handle a large number of SSDs will inevitably become a bottleneck for processing performance. Summary of the Invention

[0005] To address the aforementioned technical problems, this application provides a data processing system and method that can distribute a large number of tasks to solid-state drives for execution. Due to the large number of solid-state drives, parallel execution of a large number of tasks can be achieved, thereby improving the execution efficiency of multiple tasks.

[0006] In a first aspect, a data processing system is provided, including a processor and a plurality of solid-state drives (SSDs). The plurality of SSDs are used to provide at least one physical block. The at least one physical block constitutes a first storage space. The processor stores a first address set. The first address set includes multiple addresses. Each address is used to indicate one or more physical blocks in the first storage space. Each SSD stores a second address set. The first address set includes the second address set. A first SSD, after receiving an input / output (I / O) instruction from the processor, determines at least one target SSD from the plurality of SSDs based on a target address in the I / O instruction and the second address set stored on the first SSD. The I / O instruction instructs processing of the storage space indicated by the target address. The storage space indicated by the target address is located on at least one target SSD.

[0007] In the above scheme, within the data processing system, the processor stores the addresses included in the second address set of each solid-state drive (SSD). Therefore, the processor can determine the first SSD to execute the task and send an I / O instruction carrying the target address corresponding to the task to the first SSD. The first SSD stores the second address set related to the target address, thus enabling it to parse the target address to determine the target SSD and process it to complete the task. Therefore, in the data processing system, the processor can distribute a large number of tasks to the SSDs for execution, reducing the processor's workload. Furthermore, because there are numerous SSDs in the data processing system, parallel execution of a large number of tasks can be achieved, improving the execution efficiency of multiple tasks.

[0008] In some possible implementations, the above address is based on the logical unit number (LUN) semantics to indicate the physical block in the first storage space.

[0009] In the above scheme, LUN semantics are used to address physical blocks in the first storage space. This abstracts the addressing of physical blocks, allowing applications or the operating system to access data in the data processing system without needing to know the specific physical block address, but instead accessing the data through the logical unit number. This addressing method provides greater flexibility, simplifies the complexity of storage management, and reduces the direct management and maintenance of physical block addresses.

[0010] In some possible implementations, the aforementioned address includes a Logical Unit Number (LUN) ID, a Logical Block Address (LBA), and a Physical Block Address (PBA). The LUN ID identifies the LUN within the first storage space. Each LUN comprises at least one logical block. A mapping relationship exists between logical blocks and physical blocks within the first storage space. The LBA identifies the logical block, and the PBA identifies the physical block.

[0011] In some possible implementations, the data processing system described above includes a peer-to-peer interconnect bus. The processor is used to send I / O instructions to each solid-state drive via the peer-to-peer interconnect bus.

[0012] In the above scheme, the processor and solid-state drive connected via the peer-to-peer interconnect bus have the same status, so these devices can directly exchange data and communicate with each other.

[0013] In some possible implementations, the aforementioned processor and multiple solid-state drives are deployed on the same device.

[0014] In some possible implementations, the aforementioned processor and the first solid-state drive are deployed on different devices.

[0015] In some possible implementations, the data processing system described above includes multiple processors. These processors are deployed in a first device cluster. Multiple solid-state drives (SSDs) are deployed in a second device cluster. The first and second device clusters are different device clusters.

[0016] The above solution supports both distributing multiple tasks to multiple SSDs for execution in a single-machine system and distributing tasks across devices in a multi-machine system. In a multi-machine system, the number of SSDs capable of executing tasks can be added or removed as needed, thus supporting application scenarios with varying task scales. Furthermore, when distributing tasks across devices, the processor can directly issue I / O instructions to the SSDs in another device via the peer-to-peer interconnect bus without needing to interact with the processor in that other device. Therefore, the processor's cross-device task distribution does not require interaction with the processor in another device and does not consume the processor resources of that other device.

[0017] In some possible implementations, the first solid-state drive (SSD) is used to process the storage space indicated by the target address according to I / O instructions. Specifically, the I / O instructions instruct the storage space at the target address on the target SSD to store the first data in the dynamic random access memory (DRAM). Alternatively, the I / O instructions instruct the calculation of the first data to obtain a first calculation result, and store the first calculation result in the DRAM. Alternatively, the I / O instructions instruct the storage of the second data in the DRAM to be stored in the storage space at the target address on the target SSD. Alternatively, the I / O instructions instruct the calculation of the second data to obtain a second calculation result, and store the second calculation result in the storage space at the target address on the target SSD.

[0018] In the above solution, based on the fact that the first solid-state drive can correctly parse the target address carried by the IO instruction, this technical solution supports the first solid-state drive to perform various tasks on the target solid-state drive, including reading data on the target solid-state drive, storing data on the target solid-state drive, performing calculations on the data, etc.

[0019] In some possible implementations, the first solid-state drive (SSD) manages the storage space indicated by the target address on the target SSD, and the target SSD and the first SSD are different SSDs. Specifically, the first SSD sends a read command to the target SSD and receives a read completion message from the target SSD after completing the read command. The read command instructs the target SSD to read first data and write it to DRAM. Alternatively, the first SSD sends a write command to the target SSD and receives a write completion message from the target SSD after completing the write command. The write command instructs the target SSD to read second data from DRAM and write the second data to the storage space indicated by the target address on the target SSD.

[0020] This technical solution provides a data processing flow with separate data plane and control plane in the architecture of the data processing system. In scenarios where the target SSD and the first SSD are different SSDs, and the first SSD manages the storage space indicated by the target address on the target SSD, the first SSD sends control commands (such as read commands and write commands) to the target SSD, instructing the target SSD to transfer first or second data between itself and the DRAM. This achieves separation of the data plane and control plane; the control flow occurs between the first SSD and the target SSD, while the data flow occurs between the target SSD and the DRAM. This enables globally centralized control of the control plane and high-performance data transmission.

[0021] In some possible implementations, the aforementioned plurality of solid-state drives (SSDs) includes a second SSD. The second SSD manages the storage space indicated by the target address on the target SSD. Furthermore, the first SSD, the second SSD, and the target SSD are all different SSDs. Specifically, the first SSD sends an acquisition command to the second SSD and receives an acquisition completion message sent by the second SSD after completing the acquisition command. The acquisition command instructs the second SSD to notify the target SSD to read the first data and write the first data into DRAM. Alternatively, the first SSD specifically sends a storage command to the second SSD and receives a storage completion message sent by the second SSD after completing the storage command. The storage command instructs the second SSD to notify the target SSD to read the second data from DRAM and write the second data into the storage space indicated by the target address on the target SSD.

[0022] This technical solution provides a data processing flow with separation of data storage and management in complex scenarios. For scenarios where data storage and management are separated, the first solid-state drive (SSD) sends control commands (such as fetch commands or store commands) to the SSD managing the storage space indicated by the target address (i.e., the second SSD), instructing the second SSD to complete the first or second data transfer task between the target SSD and the DRAM. This means that in this scenario of separated data storage and management, the first SSD is unaware of the specific execution process of the task; it only needs to instruct the second SSD to complete the task. The first SSD does not need to participate in how the second SSD completes the task; it is considered to have completed the task upon receiving a notification from the second SSD that it has finished. As can be seen, this achieves separation of the data plane and the control plane. The control flow occurs between the first SSD, the second SSD, and the target SSD, while the data flow occurs only between the target SSD and the DRAM. This enables globally centralized control of the control plane and high-performance data transfer.

[0023] In some possible implementations, the aforementioned I / O instructions are used to instruct the storage of the first data in DRAM according to a RAID (Range Automation) algorithm. Specifically, the first solid-state drive (SSD) is used to retrieve the data blocks and parity blocks from the first stripe, obtain the first data based on the data blocks and parity blocks, and write the first data into DRAM. Here, the first stripe is the stripe to which the first data belongs.

[0024] This technical solution provides an implementation method in an application scenario where data is stored based on a RAID algorithm, and the first solid-state drive (SSD) recovers the first data and provides it to the processor. When the first data is corrupted, since the data processing system stores data based on the RAID algorithm, when the first SSD receives a task to retrieve the first data, it can calculate the first data based on the RAID algorithm and then provide the recovered first data to the processor. As can be seen, in this technical solution, the first SSD has data calculation and processing capabilities and can perform other calculation and processing tasks issued by the processor, except for error detection and garbage collection.

[0025] In some possible implementations, the aforementioned I / O instructions are used to instruct the second data to be stored in the storage space indicated by the target address on the target solid-state drive according to the RAID algorithm. Specifically, the first solid-state drive is used to read the second data from DRAM, obtain a data block from the second stripe, obtain a parity block based on the data block and the second data, store the second data block in the physical block corresponding to the target address on the target solid-state drive, and store the parity block in the second stripe. Here, the second stripe is the stripe to which the second data belongs.

[0026] This technical solution provides an implementation method for storing second data using a first solid-state drive (SSD) in an application scenario based on RAID algorithm data storage. When the second data needs to be stored, since the data processing system is based on RAID algorithm data storage, when the first SSD receives the task of storing the second data, it needs to first use the RAID algorithm to calculate the parity block in the second stripe to which the second data belongs, and then store the second data. This ensures that if the data in the second stripe is damaged, the damaged data can be recovered in time based on the parity block. As can be seen, in this technical solution, the first SSD has data calculation and processing capabilities and can execute other calculation and processing tasks issued by the processor, except for error detection and garbage collection.

[0027] Secondly, a data processing method is provided, applied to a data processing system. The data processing system includes a processor and multiple solid-state drives (SSDs). The multiple SSDs provide at least one physical block. The at least one physical block constitutes a first storage space. The processor stores a first address set. The first address set includes multiple addresses. Each address indicates one or more physical blocks in the first storage space. The multiple SSDs include the first SSD. The first SSD stores a second address set. The first address set includes the second address set. The method includes: the processor sending an input / output (I / O) instruction to the first SSD, and receiving a completion message sent by the first SSD after completing the I / O instruction. The I / O instruction carries a target address. The I / O instruction instructs the first SSD to determine at least one target SSD from the multiple SSDs based on the target address and the second address set. The I / O instruction instructs processing of the storage space indicated by the target address. The storage space indicated by the target address is on at least one target SSD.

[0028] In the above scheme, when the processor needs to process the storage space indicated by the target address, the processor dispatches the task to the first solid-state drive (SSD) by sending I / O instructions. The processor only needs to receive notification from the first SSD that the task has been completed. When the first SSD completes the task, it is equivalent to the processor completing the task. Furthermore, because the first SSD stores a second set of addresses, it can resolve the target address based on the second set to determine the target SSD and then process it. It can be understood that the number of SSDs in a data processing system far exceeds the number of processors. Therefore, having the SSD perform tasks that would otherwise be performed by the processor reduces the processor's workload and enables the parallel execution of a large number of tasks, improving the execution efficiency of multiple tasks.

[0029] In some possible implementations, after the processor sends an I / O instruction to the first solid-state drive (SSD) and before the processor receives a completion message sent by the first SSD after completing the I / O instruction, the method further includes: the first SSD processing the storage space indicated by the target address according to the I / O instruction. The I / O instruction is used to instruct the storage space at the target address on the target SSD to store the first data in the dynamic random access memory (DRAM). Alternatively, the I / O instruction is used to instruct the first data to be calculated to obtain a first calculation result, and the first calculation result is stored in the DRAM. Alternatively, the I / O instruction is used to instruct the storage space at the target address on the target SSD to store the second data in the DRAM. Alternatively, the I / O instruction is used to instruct the second data to be calculated to obtain a second calculation result, and the second calculation result is stored in the storage space at the target address on the target SSD.

[0030] In some possible implementations, the first solid-state drive (SSD) is used to manage the storage space indicated by the target address on the target SSD, and the target SSD and the first SSD are different SSDs. The first SSD processes the storage space indicated by the target address according to I / O instructions, including: the first SSD sending a read instruction to the target SSD and receiving a read completion message sent by the target SSD after completing the read instruction. The read instruction instructs the target SSD to read first data and write the first data into DRAM.

[0031] In some possible implementations, the aforementioned plurality of solid-state drives (SSDs) includes a second SSD. The second SSD manages the storage space indicated by the target address on the target SSD. The first SSD, the second SSD, and the target SSD are all different SSDs. The first SSD processes the storage space indicated by the target address according to I / O instructions, including: the first SSD sending a fetch instruction to the second SSD, and receiving a fetch completion message sent by the second SSD after completing the fetch instruction. The fetch instruction instructs the second SSD to notify the target SSD to read first data and write the first data to DRAM.

[0032] In some possible implementations, the aforementioned I / O instructions are used to instruct the storage of the first data in DRAM according to a RAID algorithm. The first solid-state drive (SSD) processes the storage space indicated by the target address according to the I / O instructions, including: the first SSD acquiring data blocks and parity blocks from the first stripe, obtaining the first data based on the data blocks and parity blocks, and writing the first data into DRAM. Here, the first stripe is the stripe to which the first data belongs.

[0033] In some possible implementations, the first solid-state drive (SSD) is used to manage the storage space indicated by the target address on the target SSD, and the target SSD and the first SSD are different SSDs. The first SSD processes the storage space indicated by the target address according to I / O instructions, including: the first SSD sending a write instruction to the target SSD and receiving a write completion message sent by the target SSD after completing the write instruction. The write instruction instructs the target SSD to read second data from DRAM and write the second data to the storage space indicated by the target address on the target SSD.

[0034] In some possible implementations, the aforementioned plurality of solid-state drives (SSDs) includes a second SSD. The second SSD manages the storage space indicated by the target address on the target SSD. The first SSD, the second SSD, and the target SSD are all different SSDs. The first SSD processes the storage space indicated by the target address according to I / O instructions, including: the first SSD sending a storage instruction to the second SSD and receiving a storage completion message sent by the second SSD after completing the storage instruction. The storage instruction instructs the second SSD to notify the target SSD to read second data from DRAM and write the second data to the storage space indicated by the target address on the target SSD.

[0035] In some possible implementations, the aforementioned I / O instructions are used to instruct the second data to be stored in the storage space indicated by the target address on the target solid-state drive according to the RAID algorithm. The first solid-state drive processes the storage space indicated by the target address according to the I / O instructions, including: the first solid-state drive reads the second data from DRAM, obtains a data block from the second stripe, obtains a parity block based on the data block and the second data, stores the second data block in the physical block corresponding to the target address on the target solid-state drive, and stores the parity block in the second stripe. Here, the second stripe is the stripe to which the second data belongs.

[0036] In some possible implementations, the aforementioned address includes a Logical Unit Number (LUN) ID, a Logical Block Address (LBA), and a Physical Block Address (PBA). The aforementioned target address includes a target LUN ID and a target LBA. The LUN ID identifies the LUN in the first storage space. Each LUN includes at least one logical block. There is a mapping relationship between logical blocks and physical blocks in the first storage space. The LBA identifies the logical block. The PBA identifies the physical block.

[0037] In some possible implementations, the aforementioned plurality of solid-state drives (SSDs) includes a second SSD. The second set of addresses stored on the second SSD belongs to the first set of addresses. The method further includes: the processor sending an I / O instruction carrying a first address to the second SSD, and receiving a first completion message sent by the second SSD after completing the I / O instruction carrying the first address. The I / O instruction sent by the processor to the first SSD is generated based on a first task, and the I / O instruction carrying the first address is generated based on a second task. The first task and the second task are independent of each other. The I / O instruction carrying the first address instructs the second SSD to determine the target SSD corresponding to the first address from among the plurality of SSDs based on the first address and the second set of addresses stored on the second SSD. The I / O instruction carrying the first address instructs processing of the storage space indicated by the first address. The storage space indicated by the first address is located on the target SSD corresponding to the first address.

[0038] In the above scheme, the processor can distribute multiple tasks to different solid-state drives, enabling multiple solid-state drives to execute tasks in parallel, thereby improving the execution efficiency of multiple tasks.

[0039] Thirdly, a computer program product containing instructions is provided that, when executed by a computing device, causes the computing device to perform the method as described in any of the second aspects.

[0040] Fourthly, a computer-readable storage medium is provided, characterized in that it includes computer program instructions, which, when executed by a computing device, perform the method as described in any of the second aspects. Attached Figure Description

[0041] Figure 1A This is a structural diagram of a data processing system according to an embodiment of this application;

[0042] Figure 1B This is a structural diagram of another data processing system involved in the embodiments of this application;

[0043] Figure 2A This is a structural diagram of a data processing system provided in an embodiment of this application;

[0044] Figure 2B This is a structural diagram of another data processing system provided in an embodiment of this application;

[0045] Figure 2C This is a structural diagram of another data processing system provided in an embodiment of this application;

[0046] Figure 3 This is a flowchart illustrating a data processing method provided in an embodiment of this application;

[0047] Figure 4A This is a flowchart illustrating the execution of I / O instructions by a first solid-state drive according to an embodiment of this application;

[0048] Figure 4B This is a flowchart illustrating another first solid-state drive executing I / O instructions provided in an embodiment of this application;

[0049] Figure 4C This is a flowchart illustrating another first solid-state drive executing I / O instructions according to an embodiment of this application;

[0050] Figure 5 This is a schematic diagram illustrating the result of storing data blocks based on RAID 5, as provided in an embodiment of this application.

[0051] Figure 6A This is a flowchart illustrating another first solid-state drive executing I / O instructions according to an embodiment of this application;

[0052] Figure 6B This is a flowchart illustrating another first solid-state drive executing I / O instructions provided in an embodiment of this application;

[0053] Figure 6C This is a flowchart illustrating another first solid-state drive executing I / O instructions according to an embodiment of this application;

[0054] Figure 7 This is a schematic diagram of another result of storing data blocks based on RAID5 provided in an embodiment of this application. Detailed Implementation

[0055] The technical solutions of the embodiments of this application will be described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those of ordinary skill in the art without creative effort are within the scope of protection of this application.

[0056] Before introducing the embodiments of this application, let's briefly introduce the existing data processing system and its task processing process.

[0057] Based on the different processes by which data processing systems handle tasks, existing data processing systems are mainly divided into two types. The first type of data processing system and its task handling process can be found in [link to relevant documentation]. Figure 1A And related information. For the second type of data processing system and its task processing procedure, please refer to... Figure 1B And an introduction to related content.

[0058] (I) The first type of data processing system

[0059] See Figure 1A , Figure 1A This is a structural diagram of a data processing system according to an embodiment of this application. Figure 1A As shown, the data processing system 10 includes a processor 11, a solid-state drive 12, and a data bus 13. The processor 11 and the solid-state drive 12 communicate via the data bus 13.

[0060] The data processing system 10 can be a computing device. A computing device is an electronic device used for computing, processing, and storing data, including servers, supercomputers, personal computers, workstations, industrial control computers, etc. In practical applications, the data processing system 10 includes several processors 11 and dozens to thousands of solid-state drives 12. That is, the number difference between the processors 11 and solid-state drives 12 in the data processing system 10 is significant. For ease of understanding, this application only... Figure 1A The image shows one processor 11 and six solid-state drives 12.

[0061] Processor 11 is used to execute tasks in a task set. The task set describes tasks waiting to be executed by processor 11; therefore, the task set includes one or more tasks. Processor 11 is also used to send input / output (I / O) instructions to solid-state drive 12 according to the tasks.

[0062] Optionally, the processor 11 may be, for example, a central processing unit (CPU), a data processing unit (DPU), a graphics processing unit (GPU), etc., or an application-specific integrated circuit (ASIC), or one or more integrated circuits.

[0063] The tasks in the task set can be tasks received by the data processing system 10 from the client, or tasks generated by the processor 11 during the execution of other tasks (such as computational tasks). The client and the data processing system 10 can be deployed on different computing devices, or on the same computing device.

[0064] The process by which the processor 11 sends I / O instructions to the solid-state drive 12 according to the task is as follows: if the task type is a read operation, the I / O instruction sent by the processor 11 to the solid-state drive 12 through the data bus 13 is a read instruction; if the task type is a write operation, the I / O instruction sent by the processor 11 to the solid-state drive 12 through the data bus 13 is a write instruction.

[0065] Solid-state drive 12 is used to operate according to I / O instructions sent by processor 11. If the I / O instruction received by solid-state drive 12 from processor 11 is a read instruction, that is, solid-state drive 12 needs to send data to processor 11, then solid-state drive 12 reads the data from its own disk and then sends the data to processor 11 via data bus 13. If the I / O instruction received by solid-state drive 12 from processor 11 is a write instruction, that is, solid-state drive 12 needs to store data from processor 11, then solid-state drive 12 retrieves the data from processor 11 and then writes the data to its own disk.

[0066] Optionally, if the solid-state drive 12 (SSD) is a storage device built from flash memory chips, then the SSD 12 includes an SSD controller 121 and a storage medium 122. The SSD controller 121 executes read or write commands sent by the processor 11 to the SSD 12 via the data bus 13. The SSD controller 121 can be a chip, such as a field-programmable gate array (FPGA) or an ASIC. The storage medium 122 consists of several flash memory chips. Each flash memory chip can be divided into several physical chunks of a fixed size. Therefore, each physical chunk has a standard capacity, for example, 2 to the power of N (M), where N is a positive integer.

[0067] Optionally, the flash memory chips in the SSD controller 121 and the storage medium 122 can be placed on the same printed circuit board (PCB) and presented in the form of a disk or card. They can then communicate with the processor 11 via the data bus 13 through the input / output (I / O) interface on the PCB.

[0068] In the case where the solid-state drive 12 includes an SSD controller 121 and a storage medium 122, if the solid-state drive 12 receives a read command, the SSD controller 121 concurrently reads data from one or more physical blocks of the storage medium 122, and then sends the data to the processor 11 via the data bus 13. If the solid-state drive 12 receives a write command, the SSD controller 121 obtains data from the processor 11, and then concurrently writes the data to multiple physical blocks of the storage medium 122.

[0069] The data bus 13 is used to transfer I / O instructions and data between the processor 11 and the solid-state drive 12.

[0070] Alternatively, the data bus 13 may be, for example, a peripheral component interconnect express (PCIe) bus. The data bus 13 may include a path for transmitting information between various components (e.g., processor 11, solid-state drive 12) in the data processing system 10.

[0071] It should be understood that the number of processors 11 and solid-state drives 12 in the data processing system 10 is based on... Figure 1A The data processing system 10 shown is illustrated using one processor 11 and six solid-state drives 12 as an example. In practical applications, the number of processors 11 and solid-state drives 12 in the data processing system 10 can be greater, and this application does not impose a specific limitation. When the data processing system 10 includes multiple processors 11, these processors 11 execute a portion of the tasks in the task set respectively, so as to jointly complete all the tasks in the task set.

[0072] exist Figure 1A In the data processing system 10 shown, the processor 11 executes the tasks in the task set. When the task set includes a large number of tasks, a single processor 11 can only execute the tasks assigned to it one by one, that is, a single processor 11 can only process tasks serially. Therefore, the performance bottleneck of the data processing system 10 in executing tasks lies in the processor 11, and the execution efficiency and the number of tasks executed are both limited by the computing power and number of processors 11.

[0073] Optionally, to enhance the processing capabilities of the current data processing system 10 for the solid-state drive 12, an expansion card—a redundant array of independent disks (RAID) card—can be installed on the current data processing system 10. RAID cards are used to specifically manage disk arrays composed of multiple SSDs, providing more efficient data transfer and processing capabilities, thereby accelerating the operation of the solid-state drive 12. Typically, depending on the storage requirements of the data processing system 10 and the complexity of the RAID card configuration, the number of RAID cards in the data processing system 10 ranges from 1 to 9. When the data processing system 10 includes RAID cards, the processor 11 distributes all tasks in the task set to all RAID cards, which then execute a portion of the tasks in the task set. Each RAID card can only execute the tasks assigned to it one by one. Therefore, for the data processing system 10 with RAID cards, the performance bottleneck in executing processing tasks lies in the RAID cards; both execution efficiency and the number of tasks executed are limited by the capabilities and quantity of the RAID cards.

[0074] (II) Second type of data processing system

[0075] The second type of data processing system is as described above. Figure 1A Based on the structure of the data processing system 10, multiple data processing systems 10 are used as multiple nodes in the second type of data processing system. For details, please refer to... Figure 1B And an introduction to related content.

[0076] See Figure 1B , Figure 1B This is a structural diagram of another data processing system involved in an embodiment of this application. For example... Figure 1B As shown, the data processing system 20 includes nodes 21 and a communication bus 22. Multiple nodes 21 communicate with each other via the communication bus 22. Each node 21 includes a processor 23 and a solid-state drive 24. Within a node 21, the processor 23 and the solid-state drive 24 communicate via the communication bus 22.

[0077] The data processing system 20 can be a cluster of computing devices. This cluster can include multiple computing devices, with nodes 21 being the computing devices themselves. In practical applications, nodes 21 include several processors 23 and dozens to thousands of solid-state drives 24. That is, the number difference between processors 23 and solid-state drives 24 in nodes 21 is significant. Therefore, when the data processing system 20 includes multiple nodes 21, the difference in the number of processors 23 and solid-state drives 24 in the data processing system 20 will further increase as the number of nodes 21 increases. For ease of understanding, this application only... Figure 1B The diagram shows two nodes 21, each of which includes one processor 23 and six solid-state drives 24.

[0078] When node 21 is a computing device, node 21 can be as described above. Figure 1A The processor 23 in the data processing system 10, node 21 can be the one described above. Figure 1A The processor 11 in the data processing system 10 and the solid-state drive 24 in the node 21 can be the aforementioned Figure 1A Solid-state drive 12 in data processing system 10. For the sake of brevity, the structure of node 21 will not be described in detail here.

[0079] The communication bus 22 is used to transmit processing requests or data between multiple nodes 21, and also to transmit read instructions, storage instructions or data between the processor 23 and the solid-state drive 24 in the same node 21.

[0080] Optionally, the communication bus 22 can be a PCIe bus, network bus, Extended Industry Standard Architecture (EISA) bus, Unified Bus (UB) bus, Compute Express Link (CXL) bus, Cache Coherent Interconnect for Accelerators (CCIX) bus, etc. The Unified Bus is also known as the Lingqu Bus. Buses can be divided into address buses, data buses, control buses, etc. For ease of representation, Figure 1B The bus 22 is represented by only one line, but this does not mean that there is only one bus or one type of bus. The communication bus 22 includes both the communication channel for transmitting processing requests or data between the nodes 21 in the data processing system 20, and the path for transmitting read instructions, store instructions or data between the components of the node 21 (e.g., processor 23, solid-state drive 24).

[0081] It should be understood that the number of nodes 21 in the data processing system 20, and the number of processors 23 and solid-state drives 24 in a single node 21, are based on... Figure 1B The data processing system 20 shown includes two nodes 21, each node 21 including one processor 23 and six solid-state drives 24. This is used as an example for illustration. In actual applications, the number of nodes 21, processors 23 and solid-state drives 24 in the data processing system 20 can be more. This application does not make a specific limitation.

[0082] In the current data processing system 20, when the task set on a node 21 (hereinafter referred to as the first node) includes a large number of tasks, if the first node has only one processor 23, that processor 23 will execute all the tasks in the task set; if the first node has multiple processors 23, these processors 23 will execute a portion of the tasks in the task set respectively, in order to jointly complete all the tasks in the task set. For a single processor 23, the processor 23 can only execute the tasks assigned to it one by one. Therefore, the performance bottleneck of the first node in executing tasks lies in the computing power and number of processors 23.

[0083] If the task executed by the processor 23 of the first node includes reading or writing to the solid-state drive 24 of another node 21 (hereinafter referred to as the second node), the processor 23 of the first node converts the read or write operation into a processing request and sends the processing request to the processor 23 of the second node via the communication bus 22. Upon receiving the processing request, the processor 23 of the second node performs the read or write operation on the solid-state drive 24 of the second node according to the processing request, and then sends the operation result back to the processor 23 of the first node via the communication bus 22. Therefore, when the task executed by the processor of one node involves the solid-state drive of another node, the execution efficiency of the task also depends on the interaction between the processors of the two nodes. This not only consumes the processor resources of both nodes, but also makes the execution time of the task depend on the individual processing time and the interaction time of the processors in both nodes. Therefore, the performance bottleneck of the data processing system 20 in executing tasks still lies with the processor 23.

[0084] To improve the execution efficiency of data processing systems handling multiple tasks on different SSDs, this application provides three data processing systems. These systems can distribute a large number of tasks to the solid-state drives for execution, thus reducing the processor's workload and enabling multiple tasks to be executed in parallel, thereby improving the execution efficiency of multiple tasks. The first data processing system can be found in [reference needed]. Figure 2A And an introduction to related content; the second type of data processing system can be found in [reference needed]. Figure 2B And an introduction to related content; the third type of data processing system can be found in [reference needed]. Figure 2C And an introduction to related content.

[0085] (I) The first type of data processing system

[0086] See Figure 2A , Figure 2A This is a structural diagram of a data processing system provided in an embodiment of this application. For example... Figure 2A As shown, the data processing system 30 includes a processor 31, a solid-state drive 32, and a peer-to-peer interconnect bus 33. The processor 31 and the solid-state drive 32 communicate within the data processing system 30 via the peer-to-peer interconnect bus 33.

[0087] The data processing system 30 is a computing device, in which the processor 31, solid-state drive 32, and peer-to-peer interconnect bus 33 are deployed on the same device. The processor 31 can be, for example, a CPU, DPU, GPU, etc., or an ASIC, or one or more integrated circuits. The solid-state drive 32 is a storage device built from Flash chips. The peer-to-peer interconnect bus 33 is the communication bus connecting the processor 31, solid-state drive 32, and other devices. The processor 31, solid-state drive 32, and other devices connected via the peer-to-peer interconnect bus 33 have equal status, therefore these devices can directly exchange data and communicate. The peer-to-peer interconnect bus 33 can be, for example, a UB bus, an NV-LINK bus, a PCIe bus, etc.

[0088] In the data processing system 30, at least one physical block can be provided by all solid-state drives (SSDs) 32, or at least one physical block can be provided by only some of the SSDs 32, while others do not provide any physical blocks. The at least one physical block provided by the SSDs 32 constitutes a first storage space. This first storage space may contain only one physical block or multiple physical blocks. When the first storage space contains multiple physical blocks, these physical blocks may originate from the same SSD 32 or from different SSDs 32.

[0089] The processor 31 stores a first address set 310, and each solid-state drive 32 stores a second address set 320. The first address set 310 includes multiple addresses 311. Each address 311 is used to indicate one or more physical blocks in the first storage space. The method of indicating physical blocks by address 311 and the specific content of address 311 can be found below. Figure 3 In step S101 of a data processing method, the processor performs a description of the addressing of physical blocks in the first storage space based on logical unit number (LUN) semantics. The second address set 320 belongs to the first address set 310, and the second address set 320 also includes one or more addresses 311.

[0090] Processor 31 is used to execute tasks in the task set, but the way processor 31 executes tasks is by sending tasks to solid-state drive 32, which then executes the tasks. When solid-state drive 32 completes the task, it can also be considered that processor 31 has completed the task.

[0091] In some possible implementations, processor 31 assigns tasks to solid-state drive 32 by sending I / O instructions to solid-state drive 32, and the solid-state drive 32 completes the task by completing the I / O instructions. Unlike the I / O instructions sent by processor 11 to solid-state drive 12 in data processing system 10, the I / O instructions sent by processor 31 to solid-state drive 32 in data processing system 30 carry a target address, and these I / O instructions instruct solid-state drive 32 to process the storage space indicated by the target address. The specific content of the target address can be found below. Figure 3 The description of the target address in step S101 of a data processing method. The storage space indicated by the target address is on at least one target solid-state drive (SSD). When there is one target SSD, the target SSD corresponds to one of the SSDs 32 that provides the physical blocks. When there are multiple target SSDs, the target SSD corresponds to multiple SSDs 32 that provide the physical blocks.

[0092] In a specific application scenario, if a task involves multiple target addresses, the processor 31 generates multiple I / O instructions, each carrying a target address. The processor 31 can then send these I / O instructions to the same solid-state drive 32 or to different solid-state drives 32. Once all these I / O instructions have been executed, the task can be considered complete.

[0093] Solid-state drive 32, after receiving an I / O instruction from processor 31, determines the target solid-state drive from the solid-state drives 32 that provide physical blocks, based on the target address in the I / O instruction and the second address set 320 stored on solid-state drive 32. This process can be described in the following... Figure 3 In step S102 of the data processing method, the first solid-state drive determines the target solid-state drive corresponding to the target address from multiple solid-state drives based on the target address in the IO instruction and the second address set stored on the first solid-state drive.

[0094] It is understandable that SSD 32 can only correctly resolve the target address and thus determine the correct target SSD if the second address set 320 stored on SSD 32 involves the target address. When the second address set 320 stored on SSD 32 does not involve the target address, SSD 32 cannot determine the correct target SSD based on the target address. Therefore, processor 31 also stores the addresses 311 included in the second address set 320 of each SSD 32. In this way, processor 31 can determine the SSD to perform the task and send I / O instructions to that SSD.

[0095] It should be noted that the objects to which the processor 31 can send I / O instructions depend on the addresses included in the second address set 320 stored in the solid-state drive 32. Specifically:

[0096] If each solid-state drive 32 stores the same second address set 320, and the second address set 320 relates to a target address, then the processor 31 can send an I / O instruction carrying the target address to any one of the solid-state drives 32. If different solid-state drives 32 store different second address sets 320, then the processor 31 can only send an I / O instruction carrying the target address to the solid-state drive 32 that stores the second address set 320 related to the target address.

[0097] Optionally, if each solid-state drive 32 stores the same second address set 320, and the second address set 320 is the first address set 310, then the processor 31 can distribute all tasks in the task set to different solid-state drives 32, so that different solid-state drives 32 can execute multiple tasks in parallel.

[0098] Optionally, if each solid-state drive 32 stores the same second address set 320, and the second address set 320 consists only of a portion of the addresses in the first address set 310, then the processor 31 can only assign tasks to the solid-state drives 32 to instruct processing of storage spaces indicated by addresses belonging to the second address set 320. Tasks instructing processing of storage spaces indicated by addresses not belonging to the second address set 320 still need to be executed by the processor 31 itself.

[0099] In this technical solution, the addresses 311 included in the second address set 320 stored by different solid-state drives 32 can be set as needed according to the actual application scenario, and this application does not make specific limitations.

[0100] It should be understood that the number of processors 31 and solid-state drives 32 in the data processing system 30 is based on... Figure 2A The data processing system 30 shown is illustrated using one processor 31 and four solid-state drives 32 as an example. In practical applications, the number of processors 31 and solid-state drives 32 in the data processing system 30 can be greater, and this application does not impose a specific limitation. When the data processing system 30 includes multiple processors 31, each processor 31 can distribute its respective task to the solid-state drive 32, which then executes the task. Therefore, a solid-state drive 32 may receive I / O instructions from one processor 31 or multiple processors 31. When a solid-state drive 32 receives multiple I / O instructions, it executes the I / O instructions one by one.

[0101] In summary, in the data processing system 30, the processor 31 stores the addresses included in the second address set 320 of each solid-state drive 32. Therefore, the processor 31 can determine the solid-state drive to which the task will be executed and send an I / O instruction carrying the target address corresponding to the task to that solid-state drive. The solid-state drive 32 stores the second address set 320 involving the target address, thus the solid-state drive 32 can resolve the target address to determine the target solid-state drive and then process the target solid-state drive to complete the task. Therefore, in the data processing system 30, the processor 31 can distribute a large number of tasks to the solid-state drives 32 for execution, thereby reducing the workload of the processor 31. Furthermore, because there are a large number of solid-state drives 32 in the data processing system 30, a large number of tasks can be executed in parallel, thereby improving the execution efficiency of multiple tasks.

[0102] (II) Second type of data processing system

[0103] The second type of data processing system is as described above. Figure 2A Based on the structure of the data processing system 30, multiple data processing systems 30 are used as multiple nodes in the second type of data processing system. For details, please refer to... Figure 2B And an introduction to related content.

[0104] See Figure 2B , Figure 2B This is a structural diagram of another data processing system provided in an embodiment of this application. For example... Figure 2B As shown, the data processing system 40 includes nodes 41 and a peer-to-peer interconnect bus 42. Multiple nodes 41 communicate with each other via the peer-to-peer interconnect bus 42. Each node 41 includes a processor 43 and a solid-state drive 44. Within a node 41, the processor 43 and the solid-state drive 44 communicate with each other via the peer-to-peer interconnect bus 42.

[0105] If the data processing system 40 can be a cluster of computing devices, then node 41 is a computing device. When node 41 is a computing device, node 41 can be one of the aforementioned... Figure 2A The processor 43 in the data processing system 30 and node 41 can be the aforementioned Figure 2A The processor 31 in the data processing system 30 and the solid-state drive 44 in the node 41 can be the aforementioned Figure 2A The solid-state drive 32 is located in the data processing system 30. For the sake of brevity, the structure of node 41 will not be described in detail here.

[0106] The peer-to-peer interconnect bus 42 is used to transfer I / O instructions or data between multiple nodes 41, and also for transferring I / O instructions or data between the processor 43 and the solid-state drive 44 within the same node 41. The peer-to-peer interconnect bus 42 can be as described above. Figure 2AThe peer-to-peer interconnect bus 33 in the data processing system 30.

[0107] In the data processing system 40, at least one physical block can be provided by multiple solid-state drives 44 in the same node 41, or by multiple solid-state drives 44 in multiple nodes 41. The at least one physical block provided by the solid-state drives 44 constitutes a first storage space. The first storage space may include only one physical block or multiple physical blocks. When the first storage space includes multiple physical blocks, these physical blocks may come from the same solid-state drive 44 or from different solid-state drives 44.

[0108] In the data processing system 40, at least one processor 43 stores a first address set 430, and each solid-state drive 44 stores a second address set 440. The first address set 430 includes multiple addresses 431. Each address 431 is used to indicate one or more physical blocks in the first storage space. The way the address 431 indicates the physical block and the specific content of the address 431 can be found in the following. Figure 3 In step S101 of a data processing method, the processor performs a description of the physical blocks in the first storage space based on LUN semantics. The second address set 440 belongs to the first address set 430, and the second address set 440 also includes one or more addresses 431. Figure 2B This illustrates a scenario where each processor 43 stores a first address set 430.

[0109] The processor 43, which stores the first address set 430, is used to execute tasks in the task set. However, the processor 43 executes tasks by sending the tasks to the solid-state drive 44, which then executes the tasks. When the solid-state drive 44 completes the task, it can also be considered that the processor 43 has completed the task.

[0110] In some possible implementations, the processor 43 sends the task to the solid-state drive 44 by sending I / O instructions to the solid-state drive 44, and the solid-state drive 44 completes the task by completing the I / O instructions.

[0111] Similar to the data processing system 30 described above, the I / O instructions sent by the processor 43 to the solid-state drive 44 can be the same I / O instructions sent by the processor 31 to the solid-state drive 32 in the aforementioned data processing system 30. That is, the I / O instructions sent by the processor 43 to the solid-state drive 44 carry a target address, and these I / O instructions instruct the solid-state drive 44 to process the storage space indicated by the target address. The specific content of the target address can be found below. Figure 3The description of the target address in step S101 of a data processing method. The storage space indicated by the target address is on at least one target solid-state drive (SSD). When there is one target SSD, the target SSD corresponds to one of the SSDs 44 that provides the physical blocks. When there are multiple target SSDs, the target SSD corresponds to multiple SSDs 44 that provide the physical blocks.

[0112] Unlike the data processing system 30 described above, the processor 43 can send I / O instructions not only to the solid-state drive 44 located on the same node 41 as the processor 43, but also to the solid-state drive 44 located on a different node 41 than the processor 43. Figure 2B Taking the data processing system 40 shown as an example, Figure 2B The node 41 on the left is called the first node, and the processor 43 in the first node is called the first processor. The node 41 on the right is called the second node, and the processor 43 in the second node is called the second processor. When the first processor sends an I / O instruction carrying a target address, any solid-state drive 44 that stores the second address set 440 involving the target address can be used as the target of the I / O instruction sent by the first processor. The solid-state drive 44 that receives the I / O instruction is called the first solid-state drive. The first processor and the first solid-state drive can be located in the same node 41 or in different nodes 41. When the first processor and the first solid-state drive are located in the same node 41, the first solid-state drive is the solid-state drive 44 in the first node; when the first processor and the first solid-state drive are located in different nodes 41, the first solid-state drive is the solid-state drive 44 in the second node. Regardless of whether the first processor and the first solid-state drive are located in the same node 41, the first node can directly send I / O instructions to the first solid-state drive through the peer-to-peer interconnect bus 42.

[0113] The first solid-state drive (SSD) is used to determine the target SSD from the SSDs 44 that provide physical blocks, based on the target address in the I / O instruction and the second address set 440 stored on the first SSD, after receiving an I / O instruction from the processor 43. This process can be described in the following... Figure 3 In step S102 of the data processing method, the first solid-state drive determines the target solid-state drive corresponding to the target address from multiple solid-state drives based on the target address in the IO instruction and the second address set stored on the first solid-state drive.

[0114] It should be noted that the processor 43 also stores the addresses 431 included in the second address set 440 of multiple solid-state drives 44. In this way, the processor 43 can determine the solid-state drive to which the task is to be performed and send I / O instructions to that solid-state drive.

[0115] It should be noted that the first address set 430 stored by different processors 43 can be the same or different. When different processors 43 store different first address sets 430, each processor 43 needs to check whether the second address set 440 of the solid-state drive 44 belongs to its stored first address set 430 to determine the scope of the receiving task. Taking the first processor as an example, the first processor needs to check whether the second address set 440 of each solid-state drive 44 in the first node belongs to the first address set 430 stored by the first processor. Also, the first processor needs to check whether the second address set 440 of each solid-state drive 44 in the second node belongs to the first address set 430 stored by the first processor. Only when the second address set 440 of only one solid-state drive 44 belongs to the first address set 430 stored by the first processor will the first processor store the address 431 included in the second address set 440 of that solid-state drive 44, and then use that solid-state drive 44 as the object of the receiving task, sending the corresponding IO instructions to it.

[0116] In this technical solution, the addresses 431 included in the first address set 430 stored by different processors 43 and the addresses 431 included in the second address set 440 stored by different solid-state drives 44 can be set as needed according to the actual application scenario, and this application does not make specific limitations.

[0117] It should be understood that the number of nodes 41 in the data processing system 40, and the number of processors 43 and solid-state drives 44 in a single node 41, are based on... Figure 2B The data processing system 40 shown includes two nodes 41, each node 41 including one processor 43 and three solid-state drives 44. This is used as an example for illustration. In practical applications, the number of nodes 41, processors 43, and solid-state drives 44 in the data processing system 40 can be greater, and this application does not make a specific limitation. When the data processing system 40 includes more nodes 41, and each node 41 includes more processors 43 storing the first address set 430, these processors 43 can all distribute the tasks to be executed within the same node or across nodes, and the solid-state drives 44 that receive the tasks will execute the tasks.

[0118] In summary, in the data processing system 40, the processor 43 stores the addresses included in the second address set 440 of the solid-state drive 44. Therefore, the processor 43 can determine the solid-state drive to which the task will be executed and send an I / O instruction carrying the target address corresponding to the task to that solid-state drive. The solid-state drive 44 stores the second address set 440 involving the target address, thus the solid-state drive 44 can parse the target address to determine the target solid-state drive, and then process the target solid-state drive to complete the task. By assigning the task to the solid-state drive 44 for execution, the processor 43 reduces its workload.

[0119] In addition, processor 43 can distribute tasks across nodes. This increases the number of solid-state drives that can execute tasks, further improving the parallel execution efficiency of multiple tasks and supporting application scenarios with different task scales.

[0120] Furthermore, when distributing tasks across nodes, processor 43 can directly send I / O instructions to the solid-state drive in another node via the peer-to-peer interconnect bus 42 without going through the processor in the other node. Therefore, the task distribution by processor 43 does not require interaction with the processor in the other node and does not require consuming the processor resources in the other node.

[0121] (III) The third type of data processing system

[0122] See Figure 2C , Figure 2C This is a structural diagram of another data processing system provided in an embodiment of this application. For example... Figure 2C As shown, the data processing system 50 includes a first device cluster 51, a second device cluster 52, and a peer-to-peer interconnection bus 53. The first device cluster 51 and the second device cluster 52 are different device clusters.

[0123] The first device cluster 51 includes multiple processors 54. Figure 2C The illustration shows a scenario where the first device cluster 51 includes two processors 54. In a specific implementation, the first device cluster 51 may be a computing device cluster, comprising multiple computing devices, each computing device including one or more processors 54. The processors 54 may be as described above. Figure 2A The processor 31 in the data processing system 30.

[0124] The second device cluster 52 includes multiple solid-state drives 55. Figure 2C The illustration shows a scenario where the second device cluster 52 includes four solid-state drives 55. In a specific implementation, the second device cluster 52 can be a storage device cluster, comprising multiple storage devices, each including one or more solid-state drives 55. Here, a storage device refers to a hardware device used for storing and preserving data, including storage servers, storage arrays, etc. The solid-state drives 55 can be as described above. Figure 2A Solid-state drive 32 in data processing system 30.

[0125] The peer-to-peer interconnect bus 53 is used to transmit I / O instructions or data between the first device cluster 51 and the second device cluster 52, and also to transmit I / O instructions or data between multiple processors 54 within the first device cluster 51, and between multiple solid-state drives 55 within the second device cluster 52. Figure 2A The peer-to-peer interconnect bus 33 in the data processing system 30.

[0126] In the data processing system 50, at least one physical block can be provided by all solid-state drives 55 in the second device cluster 52, or at least one physical block can be provided by only some of the solid-state drives 55 in the second device cluster 52, while other solid-state drives 55 do not provide any physical blocks. The at least one physical block provided by the solid-state drives 55 constitutes a first storage space. The first storage space may include only one physical block or multiple physical blocks. When the first storage space includes multiple physical blocks, these physical blocks may come from the same solid-state drive 55 or from different solid-state drives 55.

[0127] In the data processing system 50, at least one processor 54 in the first device cluster 51 stores a first address set 540, and each solid-state drive 55 in the second device cluster 52 stores a second address set 550. The first address set 540 includes multiple addresses 541. Each address 541 is used to indicate one or more physical blocks in the first storage space. The way the address 541 indicates the physical block and the specific content of the address 541 can be found below. Figure 3 In step S101 of a data processing method, the processor performs a description of the addressing of physical blocks in the first storage space based on LUN semantics. The second address set 550 belongs to the first address set 540, and the second address set 550 also includes one or more addresses 541. Figure 2C This illustrates a scenario where each processor 54 stores a first address set 540.

[0128] Processor 54 is used to execute tasks in the task set, but processor 54 executes tasks by sending tasks to solid-state drive 55, which then executes the tasks. When solid-state drive 55 completes the task, it can also be considered that processor 54 has completed the task.

[0129] In some possible implementations, the processor 54 sends a task to the solid-state drive 55 by sending I / O instructions to the solid-state drive 55. The solid-state drive 55 completes the task by executing I / O instructions. Specifically, the processor 54 stores addresses 541 included in a second address set 550 of multiple solid-state drives 55. This allows the processor 43 to determine the solid-state drive to execute the task and send I / O instructions to that solid-state drive. After determining the solid-state drive 55 to execute the task, the processor 54 directly sends the I / O instructions to that solid-state drive 55 via the peer-to-peer interconnect bus 53. The I / O instructions sent by the processor 54 carry a target address and are used to instruct the solid-state drive 55 to process the storage space indicated by the target address. The specific content of the target address can be found below. Figure 3 The description of the target address in step S101 of a data processing method. The storage space indicated by the target address is on at least one target solid-state drive (SSD). When there is one target SSD, the target SSD corresponds to one of the SSDs 55 that provides the physical block. When there are multiple target SSDs, the target SSD corresponds to multiple SSDs 55 that provide the physical block.

[0130] Solid-state drive 55, after receiving an I / O instruction from processor 54, determines the target solid-state drive from the solid-state drives 55 that provide physical blocks, based on the target address in the I / O instruction and the second address set 550 stored on solid-state drive 55. This process can be described below. Figure 3 In step S102 of the data processing method, the first solid-state drive determines the target solid-state drive corresponding to the target address from multiple solid-state drives based on the target address in the IO instruction and the second address set stored on the first solid-state drive.

[0131] It should be noted that the first address sets 540 stored by different processors 54 in the first device cluster 51 can be the same or different. When different processors 54 store different first address sets 540, each processor 54 needs to determine whether the second address set 550 of each solid-state drive 55 in the second device cluster 52 belongs to its stored first address set 540, thereby determining the scope of the receiving task. Only when the second address set 550 of only one solid-state drive 55 belongs to the first address set 540 stored by one processor 54 will that processor 54 store the addresses 541 included in the second address set 550 of that solid-state drive 55, and then use that solid-state drive 55 as the object of the receiving task, sending the corresponding IO instructions to it.

[0132] In this technical solution, the addresses 541 included in the first address set 540 stored by different processors 54, and the addresses 541 included in the second address set 550 stored by different solid-state drives 55 can be set as needed according to the actual application scenario, and this application does not make specific limitations.

[0133] It should be understood that in the data processing system 50, the number of processors 54 in the first device cluster 51 and the number of solid-state drives 55 in the second device cluster 52 are based on... Figure 2C The data processing system 50 shown is illustrated using the example of a first device cluster 51 comprising two processors 54 and a second device cluster 52 comprising four solid-state drives 55. In practical applications, the number of processors 54 and solid-state drives 55 in the data processing system 50 can be greater; this application does not impose a specific limitation. When the number of processors 54 in the data processing system 50 is greater, each processor 54 can send its respective task to the corresponding solid-state drive 55 in the second device cluster 52, where the receiving solid-state drive 54 executes the task. When the number of solid-state drives 55 in the data processing system 50 is greater, the number of solid-state drives capable of executing tasks increases, enabling the parallel execution of a large number of tasks and improving the execution efficiency of multiple tasks.

[0134] In summary, in the data processing system 50, the processor 54 in the first device cluster 51 stores addresses 541 included in the second address set 550 of the solid-state drives 55. Therefore, the processor 54 can determine the solid-state drive to execute the task and send an I / O instruction carrying the target address corresponding to the task to that solid-state drive. The solid-state drive 55 stores the second address set 550 involving the target address, thus the solid-state drive 55 can parse the target address to determine the target solid-state drive, and then process the target solid-state drive to complete the task. The processor 54 reduces its workload by sending the task to the solid-state drive 55 for execution. Furthermore, the second device cluster 52 has a large number of solid-state drives 55, thus enabling the parallel execution of a large number of tasks, thereby improving the execution efficiency of multiple tasks.

[0135] The above text combined Figure 2A , Figure 2B as well as Figure 2C The data processing system provided in the embodiments of this application has been introduced. Next, a data processing method provided in the embodiments of this application will be described. It should be noted that the application scenarios of the data processing method provided in the embodiments of this application are not limited to those described above. Figure 2A The first type of data processing system Figure 2B The second type of data processing system or Figure 2C The third type of data processing system, and all scenarios in which the data processing methods provided in the embodiments of this application can be applied, are within the protection scope of this application.

[0136] See Figure 3 , Figure 3 This is a flowchart illustrating a data processing method provided in an embodiment of this application. Figure 3 As shown, the data processing method provided in this application includes:

[0137] S101: The processor sends input / output (I / O) instructions to the first solid-state drive.

[0138] Accordingly, the first solid-state drive receives I / O instructions from the processor.

[0139] In one specific implementation, the processor sends I / O instructions to the first solid-state drive via a peer-to-peer interconnect bus, and the first solid-state drive receives the I / O instructions from the processor via the peer-to-peer interconnect bus.

[0140] In some possible implementations, the processor stores a first set of addresses. This first set of addresses includes multiple addresses. Each address indicates one or more physical blocks in a first memory space. The specific content of each address depends on the way it indicates a physical block; this way of indicating a physical block is the processor's addressing method for physical blocks in the first memory space. The one or more physical blocks in the first memory space are provided by multiple solid-state drives (SSDs).

[0141] In some possible implementations, the processor addresses physical blocks in the first memory space based on logical unit number (LUN) semantics.

[0142] First, all physical blocks in the first storage space are mapped into multiple logical blocks. The mapping between physical and logical blocks can be one-to-one, one-to-many, or many-to-one, depending on the user.

[0143] Subsequently, these logical blocks are grouped into one or more chunk groups, and the storage space provided by the chunk groups is virtualized into logical units (LUs) for the processor to use. Each logical unit has a unique LUN. Typically, LUN is used to refer directly to a logical unit, and each LUN is considered to have a logical unit number identifier (LUN ID), which is used to identify this LUN. If all logical blocks are in the same chunk group, the processor stores only one LUN ID for the first storage space, let's call it LUN 0; if all logical blocks are divided into two chunk groups, the processor will store two LUN IDs for the first storage space, let's call them LUN 0 and LUN 1; if all logical blocks are divided into three chunk groups, the processor will store three LUN IDs for the first storage space, let's call them LUN 0, LUN 1, and LUN 2; and so on. The more chunk groups there are, the more LUN IDs the processor stores for the first storage space.

[0144] Next, logical block addresses (LBAs) are assigned to all logical blocks within the same logical block group. If LUN 0 contains one logical block, its logical block address is LBA0; if LUN 0 contains two logical blocks, their logical block addresses are LBA0 and LBA1, respectively; if LUN 0 contains three logical blocks, their logical block addresses are LBA0, LBA1, and LBA2, respectively; and so on. The more logical blocks in a logical block group, the more LBAs are assigned to the LUN of that logical block group.

[0145] Finally, each logical block has an LBA, and a mapping between LBAs and physical block addresses (PBAs) is established. When there is a one-to-one mapping between physical blocks and logical blocks, one PBA corresponds to one LBA; when there is a one-to-many mapping between physical blocks and logical blocks, one PBA corresponds to multiple LBAs; when there is a many-to-one mapping between physical blocks and logical blocks, multiple PBAs correspond to one LBA.

[0146] In this implementation, each address in the first address set includes a LUN ID, an LBA, and a PBA. The LUN ID identifies the LUN in the first memory space; the LBA identifies the logical block; and the PBA identifies the physical block. This ensures that each address indicates a physical block in the first memory space. Table 1 below shows the eight addresses in the first address set.

[0147] Table 1

[0148] address LUN ID LBA PBA Address 11 LUN 0 LBA 0 PBA 0 Address 12 LUN 0 LBA 1 PBA 1 Address 13 LUN 0 LBA 2 PBA 2 Address 14 LUN 0 LBA 3 PBA 3 Address 15 LUN 1 LBA 0 PBA 4 Address 16 LUN 1 LBA 1 PBA 5 Address 17 LUN 1 LBA 2 PBA 6 Address 18 LUN 1 LBA 3 PBA 7

[0149] As shown in Table 1, the first storage space includes LUN0 and LUN 1. LUN 0 and LUN 1 each include four LBAs, and each LBA corresponds to one PBA. Therefore, in Table 1, each address corresponds to one physical block.

[0150] It should be understood that Table 1 above is merely an example, and this application does not specifically limit the content of the address.

[0151] In summary, using LUN semantics to address physical blocks in the first storage space abstracts the addressing of physical blocks. This allows applications or the operating system to access data in the data processing system without needing to know the specific physical block address, but instead accessing the data through the logical unit number. This addressing method provides greater flexibility, simplifies the complexity of storage management, and reduces the direct management and maintenance of physical block addresses.

[0152] In some possible implementations, a second set of addresses is stored on the first solid-state drive. The second set of addresses includes one or more addresses. The second set of addresses belongs to the first set of addresses. Continuing with the example of the eight addresses in the first set of addresses in Table 1 above, the second set of addresses stored on the first solid-state drive can include only one address, such as address 11; it can include only addresses corresponding to the same LUN, such as addresses 11-14; or it can include all addresses in the first set of addresses, such as addresses 11-18.

[0153] In some possible implementations, input / output (IO) instructions carry a destination address. IO instructions are used to instruct processing of the memory space indicated by the destination address. When each address in the first address set includes a LUN ID, LBA, and PBA, the destination address includes a destination LUN ID and a destination LBA. Continuing with the example of the eight addresses in the first address set in Table 1 above, the destination LUN ID in the destination address is either LUN 0 or LUN 1, and the destination LBA in the destination address is either LBA 0, LBA 1, LBA 2, or LBA 3.

[0154] Optionally, the I / O instruction carries a data length. The data length is used to identify the length of the data to be processed.

[0155] Optionally, the I / O instruction carries an instruction type. The instruction type is used to identify whether the I / O instruction is a read instruction or a write instruction. If the I / O instruction is a read instruction, it carries a read flag; if the I / O instruction is a write instruction, it carries a write flag.

[0156] Optionally, the I / O instruction carries a processing algorithm. If the I / O instruction is a read instruction, it instructs the first data in the memory space indicated by the target address to be calculated according to the processing algorithm to obtain a first calculation result; if the I / O instruction is a write instruction, it instructs the second data indicated by the I / O instruction to be calculated according to the processing algorithm to obtain a second calculation result, and stores the second calculation result in the memory space indicated by the target address.

[0157] For example, if the processing algorithm is a RAID algorithm, then the I / O command carries a RAID algorithm identifier. The I / O command is used to instruct the use of the RAID algorithm for data encoding and decoding. RAID is a high-performance, high-reliability storage technology that combines multiple independent solid-state drives (SSDs) in different ways to create a large-capacity SSD group. RAID algorithms primarily utilize three technologies—data striping, mirroring, and data verification—to achieve high performance, reliability, fault tolerance, and scalability in data processing. RAID algorithms include RAID5, RAID6, and RAIDTP, among others. Encoding data using RAID algorithms is mainly applied to scenarios such as storing data on SSDs, while decoding data using RAID algorithms is mainly applied to scenarios such as reading and recovering data from SSDs.

[0158] Taking RAID5 as an example of a RAID algorithm, the I / O commands carry a RAID5 identifier, indicating that RAID5 is used for data encoding and decoding. Specifically, RAID5 distributes data across various solid-state drives (SSDs) in units of data blocks. In RAID5, the relationship between data blocks and parity blocks is n:1, meaning n data blocks correspond to one parity block. Furthermore, RAID5 distributes both data blocks and parity blocks across all SSDs. Thus, if one SSD fails, causing the data blocks stored on that SSD to be corrupted, the corrupted data blocks can be recovered using the data blocks and parity blocks stored on the remaining SSDs. The correspondence between data blocks, parity blocks, and physical blocks on the SSDs depends on the size of the data blocks and physical blocks. If the size of the data block is the same as the size of the physical block, there is a one-to-one correspondence; if the data block is larger than the physical block, there is a one-to-many correspondence; and if the data block is smaller than the physical block, there is a many-to-one correspondence. For ease of explanation, the following text will use the example of a one-to-one correspondence between data blocks and physical blocks.

[0159] It should be understood that the above-described RAID algorithm is merely an example. In practical applications, the processing algorithm can also be an encryption or decryption algorithm. When the processing algorithm is an encryption algorithm (such as DES, RSA, MD5, etc.), the IO instruction is used to instruct the data to be encrypted using the encryption algorithm. When the processing algorithm is a decryption algorithm, the IO instruction is used to instruct the data to be decrypted using the encryption algorithm.

[0160] In some possible implementations, I / O instructions are generated based on the first task. The process of generating I / O instructions based on the first task can be compared with the process by which a processor on a computing device receives a read / write request and generates read / write instructions based on the read / write request. For the sake of brevity, it will not be elaborated here.

[0161] The acquisition methods for the first task mentioned above can be referred to as acquisition methods 1-3 below:

[0162] Acquisition Method 1: The processor receives the first task from the client.

[0163] For example, the first task can be a data reading task or a data storage task. When the first task is a data reading task, it carries information about the data to be read. When the first task is a data storage task, it carries data to be stored. It should be understood that the first task being a data reading task or a data storage task is merely an example, and this application does not specifically limit the content of the first task.

[0164] Method 2: After receiving the zeroth task from the client, the processor splits the zeroth task into multiple independent subtasks, and then takes one or more of these subtasks as the first task, so that the first task is only a part of the zeroth task.

[0165] For example, the zeroth task can be a RAID algorithm task. After receiving the RAID algorithm task, the processor can split the RAID algorithm task into multiple independent RAID algorithm subtasks based on the data information in the RAID algorithm task, and then use one or more of these RAID algorithm subtasks as the first task. For example, when the RAID algorithm task specifically uses RAID5 to encode data, if the RAID algorithm task carries 12KB of data to be stored, the processor first splits the 12KB of data to be stored into twelve 1KB data parts, making these 1KB data parts independent of each other, and then encodes each 1KB data part using RAID5 as an encoding subtask, resulting in twelve encoding subtasks. Subsequently, the processor can use one of the twelve encoding subtasks as the first task, or it can use multiple encoding subtasks as the first task. It should be understood that the zeroth task being a RAID algorithm task and the first task being one or more encoding subtasks are merely examples, and this application does not specifically limit the specific content of the zeroth task and the first task.

[0166] Method 3: The processor generates the first task while running the application.

[0167] For example, the first task could be to store the data generated by the application.

[0168] It should be noted that the aforementioned clients and processors can be deployed on different computing devices, or on the same computing device.

[0169] It should be understood that the acquisition methods 1 to 3 in step S101 above are merely examples and are not specifically limited here.

[0170] S102: The first solid-state drive processes the storage space indicated by the target address in the IO instruction according to the IO instruction, and sends a completion message to the processor after completing the IO instruction.

[0171] In some possible implementations, the first solid-state drive processes the storage space indicated by the target address in the I / O instruction according to the I / O instruction, which mainly includes the following two stages:

[0172] Phase 1: The first solid-state drive (SSD) determines the target SSD corresponding to the target address from among multiple SSDs based on the target address in the IO instruction and the second set of addresses stored on the first SSD.

[0173] Specifically, after receiving the I / O instruction from the processor, the first solid-state drive obtains the target address from the I / O instruction, then searches for the physical block address corresponding to the target address from the second address set stored on the first solid-state drive, and determines the target solid-state drive corresponding to the target address from multiple solid-state drives based on the physical block address.

[0174] For example, continuing with the example of the 8 addresses in Table 1 of step S101 above as addresses in the second address set, assuming that the target LUN ID and target LBA in the target address are LUN 0 and LBA 2 respectively, the first solid-state drive uses LUN0 and LBA 2 to look up Table 1 and finds that the PBA corresponding to the combination of LUN 0 and LBA 2 is PBA 2. Then the first solid-state drive will use the solid-state drive that provides the physical block identified by PBA 2 as the target solid-state drive corresponding to the target address.

[0175] It should be understood that since there is a one-to-one mapping between LBAs and PBAs in Table 1, the number of PBAs corresponding to the combination of LUN 0 and LBA 2 is one. When there is a one-to-many mapping between LBAs and PBAs, the number of PBAs corresponding to the combination of LUN 0 and LBA 2 is multiple. When there are multiple PBAs, these PBAs identify multiple physical blocks, which may all be provided by the same SSD or by different SSDs. When multiple PBAs identify physical blocks that are all provided by the same SSD, that SSD is the target SSD corresponding to the target address; when multiple PBAs identify physical blocks that are provided by different SSDs, the multiple SSDs providing these physical blocks are all target SSDs corresponding to the target address. Therefore, the number of target SSDs corresponding to the target address may be one or more, depending on the mapping relationship between LBAs and PBAs set in the data processing system.

[0176] It is understandable that the first SSD can only find the physical block address corresponding to the target address in the second address set stored on the first SSD if the address information in the second address set contains the target address information. In the example above, the target LUN ID and target LBA in the target address are LUN 0 and LBA 2 respectively. Therefore, the addresses in the second address set stored on the first SSD must include address 13. Furthermore, the processor also stores the addresses included in the second address set of the first SSD. This allows the processor to determine that the first SSD can correctly resolve the target address and find the target SSD corresponding to the target address.

[0177] The second stage: The first solid-state drive processes the storage space indicated by the target address on the target solid-state drive according to the I / O instructions, and sends a completion message to the processor after completing the I / O instructions. Correspondingly, the processor receives the completion message sent by the first solid-state drive after completing the I / O instructions.

[0178] After identifying the target SSD, the first SSD determines the specific processing content for the storage space indicated by the target address on the target SSD based on the information carried by the I / O instructions. This mainly includes the following scenarios:

[0179] Scenario 1: If the I / O instruction carries a read instruction (e.g., with a read flag), then the first solid-state drive (SSD) needs to write the first data from the storage space indicated by the target address on the target SSD into the dynamic random access memory (DRAM) of the data processing system. DRAM is used for temporary storage of data and instructions that the processor needs to access quickly.

[0180] Scenario 2: If the IO instruction carries a read instruction and carries a RAID algorithm (such as a RAID5 identifier), then the first solid-state drive needs to obtain the first data in the storage space indicated by the target address on the target solid-state drive according to the RAID algorithm (such as RAID5) and write the first data into DRAM.

[0181] Scenario 3: If the IO instruction carries a read instruction and a processing algorithm (such as a decryption algorithm identifier), then the first solid-state drive needs to obtain the first data in the storage space indicated by the target address on the target solid-state drive, use the processing algorithm (such as a decryption algorithm) to calculate the first data to obtain the first calculation result, and write the first calculation result into DRAM.

[0182] Optionally, in scenario 1 or scenario 2 of step S102 above, if the IO instruction carries a data length (such as the data length being the size of a data block), then the first data is a data block (referred to as the first data block).

[0183] Scenario 4: If the IO instruction carries a write instruction (such as carrying a write flag), then the first solid-state drive needs to store the second data in DRAM in the storage space indicated by the target address on the target solid-state drive.

[0184] Scenario 5: If the IO instruction carries a write instruction and carries a RAID algorithm (such as a RAID5 identifier), then the first solid-state drive needs to obtain the second data from DRAM and store the second data in the storage space indicated by the target address on the target solid-state drive according to the RAID algorithm (such as RAID5).

[0185] Scenario 6: If the IO instruction carries a write instruction and a processing algorithm (such as an encryption algorithm identifier), then the first solid-state drive needs to obtain the second data from the DRAM, use the processing algorithm (such as an encryption algorithm) to calculate the second data to obtain the second calculation result, and store the second calculation result in the storage space indicated by the target address on the target solid-state drive.

[0186] Optionally, in scenario 3 or scenario 4 of step S102 above, if the IO instruction carries a data length (such as the data length being the size of a data block), then the second data is a data block (referred to as the second data block).

[0187] For ease of explanation, the following text will uniformly refer to the first data as the first data block and the second data as the second data block, and will illustrate the one-to-one correspondence between data blocks and physical blocks.

[0188] In scenario 1 of step S102, the process of the first solid-state drive executing I / O instructions includes the following three scenarios:

[0189] Scenario 11: The target solid-state drive is the first solid-state drive.

[0190] See Figure 4A , Figure 4A This is a flowchart illustrating the execution of I / O instructions by a first solid-state drive according to an embodiment of this application.

[0191] like Figure 4A As shown, in scenario 11, the process of the first solid-state drive (SSD) executing the I / O instruction is as follows: the first SSD reads the first data block from the physical block corresponding to the target address on the first SSD and writes the first data block into the DRAM. After writing the first data block into the DRAM, it means that the first SSD has completed the I / O instruction. Therefore, the first SSD sends a completion message to the processor through the peer-to-peer interconnect bus. After receiving the completion message, the processor initiates a read operation on the DRAM and reads the first data block from the DRAM. In this way, the processor also completes the first task corresponding to the I / O instruction.

[0192] Scenario 12: The first solid-state drive (SSD) is used to manage the storage space indicated by the target address on the target SSD. The target SSD and the first SSD are different SSDs.

[0193] See Figure 4B , Figure 4B This is a flowchart illustrating another first solid-state drive executing I / O instructions provided in an embodiment of this application.

[0194] like Figure 4BAs shown in Scenario 12, the process of the first solid-state drive (SSD) executing the I / O instruction is as follows: The first SSD sends a read instruction to the target SSD via the peer-to-peer interconnect bus. This read instruction instructs the target SSD to read the first data block and write it into DRAM. Upon receiving the read instruction, the target SSD reads the first data block from the physical block corresponding to the target address on the target SSD and writes it into DRAM. After writing the first data block into DRAM, the target SSD sends a read completion message to the first SSD via the peer-to-peer interconnect bus. Upon receiving the read completion message, the first SSD has completed the I / O instruction and therefore sends a completion message to the processor via the peer-to-peer interconnect bus. Upon receiving the completion message, the processor initiates a read operation on the DRAM and reads the first data block from the DRAM. In this way, the processor also completes the first task corresponding to the I / O instruction.

[0195] Scenario 13: The second SSD is used to manage the storage space indicated by the target address on the target SSD. The first SSD, the second SSD, and the target SSD are all different SSDs.

[0196] See Figure 4C , Figure 4C This is a flowchart illustrating another first solid-state drive executing I / O instructions provided in an embodiment of this application.

[0197] like Figure 4C As shown, in scenario 13, the process of the first solid-state drive (SSD) executing the I / O instruction is as follows: The first SSD sends a fetch instruction to the second SSD via the peer-to-peer interconnect bus. This fetch instruction instructs the second SSD to fetch a first data block from the target SSD. After receiving the fetch instruction, the second SSD sends a read instruction to the target SSD via the peer-to-peer interconnect bus. After receiving the read instruction, the target SSD reads the first data block from the physical block corresponding to the target address on the target SSD and writes the first data block into DRAM. After writing the first data block into DRAM, the target SSD sends a read completion message to the second SSD via the peer-to-peer interconnect bus. After receiving the read completion message, the second SSD sends a fetch completion message to the first SSD via the peer-to-peer interconnect bus. After receiving the fetch completion message, the first SSD has completed the I / O instruction. Therefore, the first SSD sends a completion message to the processor via the peer-to-peer interconnect bus. After receiving the completion message, the processor initiates a read operation on the DRAM and reads the first data block from the DRAM. In this way, the processor also completes the first task corresponding to the I / O instruction.

[0198] In scenario 2 of step S102, the solid-state drive (SSD) uses a RAID algorithm to store data blocks. The RAID algorithm requires dividing the data blocks on the SSD into multiple stripes for management. Each stripe includes one or more data blocks. Multiple stripes are distributed across multiple SSDs to achieve parallel data read / write and redundancy verification.

[0199] Continuing with RAID5 as an example, RAID5 provides a certain degree of redundancy and fault tolerance through technologies such as data striping and data verification. Even if one of the solid-state drives fails, the data blocks on that solid-state drive can still be recovered.

[0200] The following will combine Figure 5 The process of storing data blocks based on RAID 5 is explained.

[0201] See Figure 5 , Figure 5 This is a schematic diagram illustrating the result of storing data blocks based on RAID5, as provided in an embodiment of this application. Figure 5 As shown, each of the four solid-state drives (SSD 1, SSD 2, SSD 3, and SSD 4) can store multiple data blocks. Specifically, when using RAID 5 to store data blocks, the SSDs mentioned above... Figure 2A When the solid-state drive 32 is used in the data processing system 30, SSD1, SSD2, SSD3, and SSD4 respectively correspond to Figure 2A The four solid-state drives (SSDs) in the configuration are 32. When using RAID 5 to store data blocks, the SSDs are as described above. Figure 2B When the solid-state drives 44 in the data processing system 40 are used, SSD 1, SSD 2, SSD 3, and SSD 4 can be from... Figure 2B Different nodes 41 in the array, for example, SSD 1 and SSD 2 are Figure 2B The first node in the process contains SSD 44, SSD3, and SSD 4. Figure 2B The second node contains solid-state drive 44. When using RAID 5 to store data blocks, the solid-state drive is as described above. Figure 2C In the data processing system 50, the solid-state drives 55 include SSD 1, SSD 2, SSD 3, and SSD 4. Figure 2B Solid-state drives 55 in the second device cluster 52.

[0202] The process of implementing RAID 5 storage for data blocks on four solid-state drives is as follows:

[0203] The parity block PA is calculated using erasure coding (EC) algorithm with data blocks A1, A2, and A3. Then, data blocks A1, A2, A3 and parity block PA are stored in SSD 1, SSD 2, SSD 3 and SSD 4 respectively. The stripe S1 is composed of data blocks A1, A2, A3 and parity block PA.

[0204] The check block PB is calculated using the EC encoding algorithm using data blocks B1, B2, and B3. Then, data blocks B1, B2, B3, and PB are stored in SSD 1, SSD 2, SSD 4, and SSD 3 respectively. The stripe S2 is composed of data blocks B1, B2, B3, and PB.

[0205] The check block PC is calculated using the EC encoding algorithm using data blocks C1, C2, and C3. Then, data blocks C1, C2, C3, and check block PC are stored in SSD 1, SSD 3, SSD 4, and SSD 2 respectively. The stripe S3 is composed of data blocks C1, C2, C3, and check block PC.

[0206] The check block PD is calculated using the EC encoding algorithm using data blocks D1, D2, and D3. Then, data blocks D1, D2, D3, and check block PD are stored in SSD 2, SSD 3, SSD 4, and SSD 1 respectively. The stripe S4 is composed of data blocks D1, D2, D3, and check block PD.

[0207] It should be understood that for other data blocks, a check block is still calculated from three data blocks, and the different check blocks are distributed and stored on different solid-state drives.

[0208] As discussed above, when a solid-state drive (SSD) uses a RAID algorithm to store data, all data in the first storage space is typically divided into multiple stripes. Each stripe contains multiple data blocks. Furthermore, different data blocks within the same stripe are usually distributed across physical blocks on different SSDs.

[0209] When the target SSD fails, causing corruption of the first data block in the physical block corresponding to the target address on the target SSD, the first SSD cannot directly access the first data block. In this case, the first SSD executes the I / O instruction as follows: The first SSD determines the first stripe to which the first data block belongs, retrieves the data block and parity block from the first stripe, uses the data block and parity block from the first stripe to calculate the first data block using the EC decoding algorithm, and writes the first data block to DRAM. After writing the first data block to DRAM, it means the first SSD has completed the I / O instruction. Therefore, the first SSD sends a completion message to the processor via the peer-to-peer interconnect bus. Upon receiving the completion message, the processor initiates a read operation from the DRAM to read the first data block. In this way, the processor also completes the first task corresponding to the I / O instruction.

[0210] The following will continue to combine Figure 5 The data block shown is based on RAID5 storage, with SSD 1 as the target solid-state drive and data block A1 as an example of the first data block. The process of executing IO instructions on the first solid-state drive is explained.

[0211] When SSD 1 fails, causing data block A1 on SSD 1 to be corrupted, the process of the first solid-state drive executing I / O instructions is as follows:

[0212] First, the first solid-state drive (SSD) determines that the first stripe to which data block A1 belongs is stripe S1. Therefore, the first SSD retrieves data block A2, data block A3, and parity block PA from stripe S1. The process of retrieving data block A2 depends on the relationship between the first SSD, the SSD managing stripe S1, and the SSD containing data block A2, mainly including the following three scenarios:

[0213] Scenario 1: The first SSD, the SSD managing stripe S1, and the SSD containing data block A2 are the same SSD, corresponding to scenario 11 in step S102 above. Therefore, the first SSD reads data block A2 from SSD 2.

[0214] Scenario 2: The first SSD and the SSD managing stripe S1 are the same SSD, but the first SSD and the SSD containing data block A2 are different SSDs. This corresponds to scenario 12 in step S102 above, specifically: the SSD containing data block A2 is the target SSD. Therefore, the first SSD sends a read command to SSD 2 via the peer-to-peer interconnect bus. This read command instructs SSD 2 to read data block A2 and send it to the first SSD. After receiving the read command, SSD 2 reads data block A2 from SSD 2 and sends it to the first SSD via the peer-to-peer interconnect bus.

[0215] Scenario 3: The first SSD, the SSD managing stripe S1, and the SSD containing data block A2 are all different SSDs, corresponding to scenario 13 in step S102 above. Specifically, the SSD containing data block A2 is the target SSD, and the SSD managing stripe S1 is the second SSD. Therefore, the first SSD sends a retrieval command to the SSD managing stripe S1 via the peer-to-peer interconnect bus. This retrieval command instructs the SSD managing stripe S1 to retrieve data block A2 from SSD 2. After receiving the retrieval command, the SSD managing stripe S1 sends a read command to SSD 2 via the peer-to-peer interconnect bus. After receiving the read command, SSD 2 reads data block A2 from SSD 2 and sends data block A2 to the SSD managing stripe S1 via the peer-to-peer interconnect bus. After receiving data block A2, the SSD managing stripe S1 sends data block A2 to the first SSD via the peer-to-peer interconnect bus.

[0216] It is understandable that the process of obtaining data block A3 and parity block PA is similar to the process of obtaining data block A2 described above. For the sake of brevity in the manual, it will not be elaborated here.

[0217] Subsequently, the first solid-state drive uses data block A2, data block A3, and parity block PA to calculate data block A1 using the EC decoding algorithm, thereby realizing the recovery of data block A1.

[0218] In one specific implementation, the SSD controller in the first solid-state drive (SSD) is not only responsible for managing and controlling the overall operation of the first SSD, but also for performing various calculation and processing tasks. In this case, the SSD controller in the first SSD obtains data block A2, data block A3, and parity block PA, then uses the EC decoding algorithm to calculate data block A2, data block A3, and parity block PA, and uses the calculation result as data block A1.

[0219] In another specific implementation, the SSD controller in the first solid-state drive (SSD) manages and controls the overall operation of the SSD, while the SSD processor in the first SSD executes the instructions issued by the SSD controller and performs various calculations and processing tasks. The SSD controller and SSD processor can be independent of each other or integrated on the same PCB. The SSD processor can be, for example, an Advanced Reduced Instruction Set Computing (RISC) processor (ARM). In this case, the SSD controller obtains data block A2, data block A3, and parity block PA, and then sends them to the SSD processor. The SSD processor uses the EC decoding algorithm to calculate the values ​​of data block A2, data block A3, and parity block PA, and sends the calculation result as data block A1 to the SSD controller.

[0220] Next, the first solid-state drive (SSD) writes data block A1 to the DRAM. After writing data block A1 to the DRAM, it means the first SSD has completed the I / O instruction. Therefore, the first SSD sends a completion message to the processor via the peer-to-peer interconnect bus. Upon receiving the completion message, the processor initiates a read operation from the DRAM to read data block A1. In this way, the processor has also completed the first task corresponding to the I / O instruction.

[0221] It should be understood that the above example only uses RAID5 as the RAID algorithm, with four solid-state drives (SSDs) used to store data blocks, and one target SSD and one first data block. In practical applications, the RAID algorithm can also be RAID6, RAIDTP, etc., and the number of SSDs used to store data blocks can be more or less, and the number of target SSDs and the first data block can also be more. This application does not make any specific limitations.

[0222] In scenario 3 of step S102, the process of the first solid-state drive (SSD) executing the I / O instruction is as follows: the first SSD obtains the first data block from the physical block corresponding to the target address on the target SSD, performs calculations (such as decryption) on the first data block to obtain a first calculation result (such as a decryption result), and then writes the first calculation result into DRAM. After writing the first calculation result into DRAM, it means that the first SSD has completed the I / O instruction. Therefore, the first SSD sends a completion message to the processor through the peer-to-peer interconnect bus. After receiving the completion message, the processor initiates a read operation on the DRAM to read the first calculation result from the DRAM. In this way, the processor also completes the first task corresponding to the I / O instruction.

[0223] The process by which the first solid-state drive (SSD) acquires the first data block depends on the relationship between the first SSD, the target SSD, and the SSD managing the target SSD. If the first SSD, the target SSD, and the SSD managing the target SSD are the same SSD, the process of the first SSD acquiring the first data block can refer to the process of the first SSD acquiring data block A2 in scenario 2, case 1 of step S102. If the first SSD and the SSD managing the target SSD are the same SSD, but the first SSD and the target SSD are different SSDs, the process of the first SSD acquiring the first data block can refer to the process of the first SSD acquiring data block A2 in scenario 2, case 2 of step S102. If the first SSD, the target SSD, and the SSD managing the target SSD are all different SSDs, the process of the first SSD acquiring the first data block can refer to the process of the first SSD acquiring data block A2 in scenario 2, case 3 of step S102.

[0224] Specifically, the action of acquiring the first data block is performed by the SSD controller in the first solid-state drive, the action of calculating the first data block to obtain the first calculation result is performed by the SSD processor in the first solid-state drive, and the first data block is sent from the SSD controller to the SSD processor.

[0225] In scenario 4 of step S102, the process of the first solid-state drive executing I / O instructions includes the following three scenarios:

[0226] Scenario 41: The target solid-state drive corresponding to the target address is the first solid-state drive.

[0227] See Figure 6A , Figure 6A This is a flowchart illustrating another first solid-state drive executing I / O instructions provided in an embodiment of this application.

[0228] like Figure 6A As shown, in scenario 31, the process of the first solid-state drive (SSD) executing the I / O instruction is as follows: the SSD controller in the first SSD initiates a read operation to the DRAM, reads the second data block from the DRAM, and writes the second data block to the physical block corresponding to the target address on the first SSD. After writing the second data block to the physical block corresponding to the target address on the first SSD, it means that the first SSD has completed the I / O instruction. Therefore, the first SSD sends a completion message to the processor through the peer-to-peer interconnect bus. When the processor receives the completion message, it is equivalent to the processor also completing the first task corresponding to the I / O instruction.

[0229] Scenario 42: The first solid-state drive (SSD) is used to manage the storage space indicated by the target address on the target SSD. The target SSD and the first SSD are different SSDs.

[0230] See Figure 6B , Figure 6B This is a flowchart illustrating another first solid-state drive executing I / O instructions provided in an embodiment of this application.

[0231] like Figure 6B As shown in scenario 32, the process of the first solid-state drive (SSD) executing the I / O instruction is as follows: The first SSD sends a write instruction to the target SSD via the peer-to-peer interconnect bus. This write instruction instructs the target SSD to read a second data block from DRAM and write it to the physical block corresponding to the target address on the target SSD. After receiving the write instruction, the SSD's SSD controller initiates a read operation on the DRAM, reads the second data block from the DRAM, and writes it to the physical block corresponding to the target address on the target SSD. After writing the second data block to the physical block corresponding to the target address on the target SSD, the target SSD sends a write completion message to the first SSD via the peer-to-peer interconnect bus. Upon receiving the write completion message, the first SSD has completed the I / O instruction and therefore sends a completion message to the processor via the peer-to-peer interconnect bus. The processor receiving the completion message signifies that the processor has also completed the first task corresponding to the I / O instruction.

[0232] Scenario 43: The second SSD is used to manage the storage space indicated by the target address on the target SSD. The first SSD, the second SSD, and the target SSD are all different SSDs.

[0233] See Figure 6C , Figure 6C This is a flowchart illustrating another first solid-state drive executing I / O instructions provided in an embodiment of this application.

[0234] like Figure 6CAs shown in scenario 33, the process of the first SSD executing the I / O instruction is as follows: The first SSD sends a storage instruction to the second SSD via the peer-to-peer interconnect bus. This storage instruction instructs the second SSD to notify the target SSD to write the second data block. After receiving the storage instruction, the second SSD sends a write instruction to the target SSD via the peer-to-peer interconnect bus. After receiving the write instruction, the SSD controller in the target SSD initiates a read operation to the DRAM, reads the second data block from the DRAM, and writes the second data block to the physical block corresponding to the target address on the target SSD. After writing the second data block to the physical block corresponding to the target address on the target SSD, the target SSD sends a write completion message to the second SSD via the peer-to-peer interconnect bus. After receiving the write completion message, the second SSD sends a storage completion message to the first SSD via the peer-to-peer interconnect bus. After receiving the storage completion message, the first SSD has completed the I / O instruction. Therefore, the first SSD sends a completion message to the processor via the peer-to-peer interconnect bus. The processor receiving the completion message means that the processor has also completed the first task corresponding to the I / O instruction.

[0235] In scenario 5 of step S102, the solid-state drive (SSD) uses a RAID algorithm to store data blocks. The process of the first SSD executing the I / O instruction is as follows: The SSD controller in the first SSD initiates a read operation to the DRAM, reads the second data block from the DRAM, determines the second stripe to which the second data block belongs, obtains the data block in the second stripe, calculates the parity block using the data block in the second stripe and the second data block through the EC encoding algorithm, stores the second data block on the target SSD at the physical block corresponding to the target address, and stores the parity block in the second stripe. After writing the second data block to the physical block corresponding to the target address on the first SSD, it means that the first SSD has completed the I / O instruction. Therefore, the first SSD sends a completion message to the processor through the peer-to-peer interconnect bus. The processor receiving the completion message is equivalent to the processor also completing the first task corresponding to the I / O instruction.

[0236] The following continues, combining the aforementioned points. Figure 5 The data block shown is based on RAID5 storage, with SSD 1 as the target solid-state drive and data block E1 as the second data block. This example illustrates the process of executing I / O instructions on the first solid-state drive.

[0237] Suppose that data block E1 is to be stored on stripe S5, and there are already data blocks E2 and E3 on stripe S5. Data block E2 is stored on SSD 2, and data block E3 is stored on SSD 3.

[0238] First, the first solid-state drive (SSD) determines that the second stripe to which data block E1 belongs is stripe S5. Therefore, the first SSD acquires data blocks E2 and E3 in stripe S5. The process of the first SSD acquiring data blocks E2 and E3 can be referred to the process of the first SSD acquiring data block A2 in scenario 2 of step S102 above. For the sake of brevity, it will not be elaborated here.

[0239] Subsequently, the first solid-state drive uses data blocks E1, E2, and E3 to calculate the test block PE using the EC encoding algorithm.

[0240] Next, the first SSD stores data block E1 on the physical block corresponding to the target address on SSD 1, and stores the verification block PE on SSD 4. The process of the first SSD storing data block E1 depends on the relationship between the first SSD, the SSD managing stripe S5, and the SSD containing data block E1, mainly including the following three cases:

[0241] Scenario 1: The first SSD, the SSD managing stripe S5, and the SSD containing data block E1 are the same SSD, corresponding to scenario 41 in step S102 above. Therefore, the first SSD writes data block E1 to the physical block corresponding to the target address on SSD 1.

[0242] Scenario 2: The first SSD and the SSD managing stripe S5 are the same SSD, but the first SSD and the SSD containing data block E1 are different SSDs. This corresponds to scenario 42 in step S102 above, specifically: the SSD containing data block E1 is the target SSD. Therefore, the first SSD sends a write command to SSD 1 via the peer-to-peer interconnect bus. This write command instructs SSD 1 to read data block E1 from DRAM and write data block E1 to the physical block corresponding to the target address on SSD 1. After receiving the write command, the SSD controller in SSD 1 initiates a read operation to DRAM, reads data block E1 from DRAM, and writes data block E1 to the physical block corresponding to the target address on SSD 1.

[0243] Scenario 3: The first SSD, the SSD managing stripe S5, and the SSD containing data block E1 are all different SSDs, corresponding to scenario 43 in step S102 above. Specifically, the SSD containing data block E1 is the target SSD, and the SSD managing stripe S5 is the second SSD. Therefore, the first SSD sends a storage command to the SSD managing stripe S5 via the peer-to-peer interconnect bus. This storage command instructs the SSD managing stripe S5 to notify SSD 1 to write data block E1. After receiving the storage command, the SSD managing stripe S5 sends a write command to SSD 1 via the peer-to-peer interconnect bus. After receiving the write command, the SSD controller in SSD 1 initiates a read operation on the DRAM, reads data block E1 from the DRAM, and writes data block E1 to the physical block corresponding to the target address on SSD 1.

[0244] The process of storing the first solid-state drive (SSD) parity block (PE) depends on the relationship between the first SSD, the SSD managing the S5 stripe, and the SSD where the PE parity block resides. This mainly includes the following three scenarios:

[0245] Scenario 1: The first SSD, the SSD managing stripe S5, and the SSD containing the parity block PE are the same SSD, corresponding to scenario 41 in step S102 above. Therefore, the first SSD writes the parity block PE onto SSD 4.

[0246] Scenario 2: The first SSD and the SSD managing stripe S5 are the same SSD, but the first SSD and the SSD containing the parity block PE are different SSDs. This corresponds to scenario 42 in step S102 above, specifically: the SSD containing the parity block PE is the second SSD. Therefore, the first SSD sends a write command and the parity block PE to SSD 4 via the peer-to-peer interconnect bus. This write command instructs SSD 4 to write the parity block PE onto SSD 4. After receiving the write command and the parity block PE, SSD 1 writes the parity block PE onto SSD 4.

[0247] Scenario 3: The first SSD, the SSD managing stripe S5, and the SSD containing the parity block PE are all different SSDs, corresponding to scenario 43 in step S102 above. Specifically, the SSD containing the parity block PE is the target SSD, and the SSD managing stripe S5 is the second SSD. Therefore, the first SSD sends a storage command and the parity block PE to the SSD managing stripe S5 via the peer-to-peer interconnect bus. This storage command instructs the SSD managing stripe S5 to notify SSD 4 to write the parity block PE. After receiving the storage command and the parity block PE, the SSD managing stripe S5 sends a write command and the parity block PE to SSD 4 via the peer-to-peer interconnect bus. After receiving the write command and the parity block PE, SSD 4 writes the parity block PE to itself.

[0248] In summary, the results of using RAID 5 to store data blocks E1 on a solid-state drive are as follows: Figure 7 As shown. See also Figure 7 , Figure 7 This is a schematic diagram of another result of storing data blocks based on RAID5 provided in an embodiment of this application.

[0249] In scenario 6 of step S102, the process of the first solid-state drive (SSD) executing the I / O instruction is as follows: the SSD controller in the first SSD initiates a read operation to the DRAM, reads the second data block from the DRAM, performs calculations (such as encryption) on the second data block to obtain a second calculation result (such as an encryption result), and then stores the second calculation result in the physical block corresponding to the target address on the target SSD. After storing the second calculation result in the physical block corresponding to the target address on the target SSD, it means that the first SSD has completed the I / O instruction. Therefore, the first SSD sends a completion message to the processor through the peer-to-peer interconnect bus. When the processor receives the completion message, it is equivalent to the processor also completing the first task corresponding to the I / O instruction.

[0250] The process of the first solid-state drive (SSD) storing the second calculation result depends on the relationship between the first SSD, the target SSD, and the SSD that manages the target SSD. If the first SSD, the target SSD, and the SSD that manages the target SSD are the same SSD, then the process of the first SSD storing the second calculation result can refer to scenario 1 of step S102, where the first SSD stores the verification block PE. If the first SSD and the SSD that manages the target SSD are the same SSD, but the first SSD and the target SSD are different SSDs, then the process of the first SSD obtaining the first data block can refer to scenario 2 of step S102, where the first SSD stores the verification block PE. If the first SSD, the target SSD, and the SSD that manages the target SSD are all different SSDs, then the process of the first SSD obtaining the first data block can refer to scenario 3 of step S102, where the first SSD stores the verification block PE.

[0251] Specifically, the action of calculating the second data block to obtain the second calculation result is performed by the SSD processor in the first solid-state drive, the action of storing the second calculation result is performed by the SSD controller in the first solid-state drive, and the second calculation result is sent by the SSD processor to the SSD controller.

[0252] It should be understood that the above Figure 3This paper only illustrates the process of the processor issuing tasks to the first solid-state drive (SSD) in the form of I / O instructions. In this technical solution, the processor can also send I / O instructions carrying a first address to the second SSD. Correspondingly, the second SSD receives the I / O instructions carrying the first address from the processor. The second SSD processes the storage space indicated by the first address according to the I / O instructions carrying the first address, and sends a first completion message to the processor after completing the I / O instructions carrying the first address. Correspondingly, the processor receives the first completion message sent by the second SSD after completing the I / O instructions carrying the first address. The I / O instructions carrying the first address are generated based on a second task. The second task is independent of the first task. The I / O instructions carrying the first address instruct the second SSD to determine the target SSD corresponding to the first address from multiple SSDs based on the first address and a second set of addresses stored on the second SSD. The first address includes a first LUN ID and a first LBA. The storage space indicated by the first address is on the target SSD corresponding to the first address. The I / O instructions carrying the first address instruct processing of the storage space indicated by the first address. The process of the processor sending an I / O instruction carrying the first address to the second solid-state drive can be referred to the execution process of step S101 above. The process of the second solid-state drive processing the storage space indicated by the first address according to the I / O instruction carrying the first address, and sending a first completion message to the processor after completing the I / O instruction carrying the first address, can be referred to the execution process of step S102 above. In this way, the processor can distribute multiple tasks to different solid-state drives, enabling multiple solid-state drives to execute tasks in parallel, thereby improving the execution efficiency of multiple tasks.

[0253] It should be noted that when the above Figure 3 The data processing methods applied to the aforementioned Figure 2A In the data processing system 30, the processor in steps S101 and S102 above can be... Figure 2A The processor 31 in the data processing system 30 has a first address set that is Figure 2A The first address set 310 on the processor 31 in the data processing system 30, the addresses in the first address set are Figure 2A Address 311 in the first address set 310 on the processor 31 in the data processing system 30, the first solid-state drive, the second solid-state drive, and the target solid-state drive can be Figure 2A The solid-state drive 32 in the data processing system 30 has a second address set. Figure 2A The second address set 320 on the solid-state drive 32 in the data processing system 30, the peer-to-peer interconnect bus can be Figure 2A The peer-to-peer interconnect bus 33 in the data processing system 30.

[0254] When the above Figure 3 The data processing methods applied to the aforementioned Figure 2B In the data processing system 40, the processor in steps S101 and S102 above can be... Figure 2B The processor 43 in the data processing system 40 has a first address set that is Figure 2B The first address set 430 on the processor 43 in the data processing system 40, the addresses in the first address set are Figure 2B Address 431 in the first address set 430 on the processor 43 in the data processing system 40, the first solid-state drive, the second solid-state drive, and the target solid-state drive can be Figure 2B The solid-state drive 44 in the data processing system 40, the second address set is Figure 2B The second address set 440 on the solid-state drive 44 in the data processing system 40, the peer-to-peer interconnect bus can be Figure 2B The peer-to-peer interconnect bus 42 in the data processing system 40.

[0255] When the above Figure 3 The data processing methods applied to the aforementioned Figure 2C In the data processing system 50, the processor in steps S101 and S102 above can be... Figure 2C The processor 54 in the data processing system 50 has a first address set that is Figure 2C The first address set 540 on the processor 54 in the data processing system 50, the addresses in the first address set are Figure 2C In the data processing system 50, the processor 54 has an address 541 in the first address set 540, and the first solid-state drive, the second solid-state drive, and the target solid-state drive can be... Figure 2C The solid-state drive 55 in the data processing system 50 has a second address set. Figure 2C The second address set 550 on the solid-state drive 55 in the data processing system 50, the peer-to-peer interconnect bus can be Figure 2C The peer-to-peer interconnect bus 53 in the data processing system 50.

[0256] In summary, based on the first solid-state drive's ability to correctly parse the target address carried by I / O instructions, this technical solution supports the first solid-state drive in performing diverse tasks on the target solid-state drive, including reading data from the target solid-state drive, storing data on the target solid-state drive, and performing calculations on the data. Furthermore, when the processor needs to process the storage space indicated by the target address, the processor delegates the task to the first solid-state drive by sending I / O instructions. The processor only needs to receive notification that the first solid-state drive has completed the task. When the first solid-state drive completes the task, it is equivalent to the processor completing the task. It is understandable that the number of solid-state drives in a data processing system far exceeds the number of processors. Therefore, having the solid-state drive perform tasks that would otherwise be performed by the processor reduces the processor's workload and enables the parallel execution of a large number of tasks, improving the execution efficiency of multiple tasks.

[0257] On the other hand, this technical solution is based on the aforementioned Figure 2A , Figure 2B or Figure 2C The data processing system architecture provides a data processing method for CNC separation suitable for different scenarios.

[0258] In a scenario where the target SSD and the first SSD are different SSDs, and the first SSD manages the storage space indicated by the target address on the target SSD, the first SSD sends control commands (such as read commands and write commands) to the target SSD to instruct the target SSD to transfer first or second data between itself and the DRAM. This achieves separation of the data plane and the control plane; control flow occurs between the first SSD and the target SSD, while data flow occurs between the target SSD and the DRAM. This enables centralized global control of the control plane and high-performance data transmission.

[0259] In scenarios where data storage and management are separated, the first solid-state drive (SSD) sends control commands (such as fetch or store commands) to the SSD managing the storage space indicated by the target address (i.e., the second SSD), instructing the second SSD to complete a first or second data transfer task between the target SSD and the DRAM. This means that in such a scenario, the first SSD is unaware of the specific execution process of the task; it only needs to instruct the second SSD to complete the task. The first SSD does not need to participate in how the second SSD completes the task; it is considered to have completed the task upon receiving a notification from the second SSD that it has finished. This achieves separation of the data plane and control plane. Control flow occurs between the first SSD, the second SSD, and the target SSD, while data flow occurs only between the target SSD and the DRAM. This enables globally centralized control of the control plane and high-performance data transfer.

[0260] In applications where data is stored using RAID algorithms, a first solid-state drive (SSD) is used to recover first data and provide it to the processor, and also to store second data. Specifically, when the first data is corrupted, since the data processing system uses RAID algorithms for data storage, the first SSD, upon receiving a task to retrieve the first data, can calculate the first data using the RAID algorithm and then provide the recovered data to the processor. When the second data needs to be stored, the first SSD, upon receiving a task to store the second data, needs to first use the RAID algorithm to calculate the parity block in the second stripe to which the second data belongs, and then store the second data. This ensures that if data in the second stripe is corrupted, the corrupted data can be recovered promptly based on the parity block. As can be seen, in this technical solution, the first SSD has data calculation and processing capabilities, and can execute other calculation and processing tasks issued by the processor, except for error detection and garbage collection.

[0261] This application also provides a computer program product containing instructions. This computer program product may be a software or program product containing instructions, capable of running on a computing device or stored on any usable medium. When the computer program product runs on a computing device, it causes the computing device to perform the aforementioned... Figure 3 Data processing methods.

[0262] This application also provides a computer-readable storage medium. The computer-readable storage medium can be any usable medium that a computing device can store, or a data storage device such as a data center containing one or more usable media. The usable medium can be a magnetic medium (e.g., floppy disk, hard disk, magnetic tape), an optical medium (e.g., DVD), or a semiconductor medium (e.g., solid-state drive). The computer-readable storage medium includes instructions that direct a computing device to perform the aforementioned actions. Figure 3 Data processing methods.

[0263] It should be understood that in the embodiments of this application, "when," "...when," and "if" all refer to the device making corresponding processing under certain objective circumstances, and are not time-limited, nor do they require the device to make a judgment action, nor do they imply any other limitations.

[0264] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of this application, and are not intended to limit them. Although this application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the protection scope of the technical solutions of the embodiments of this application.

Claims

1. A data processing system, characterized in that, Includes a processor and multiple solid-state drives, The plurality of solid-state drives are used to provide at least one physical block, the at least one physical block constituting a first storage space. The processor stores a first address set, which includes multiple addresses, each address indicating one or more physical blocks in the first storage space. Each solid-state drive stores a second set of addresses, and the first set of addresses includes the second set of addresses. The first solid-state drive (SSD) of the plurality of SSDs is used to determine at least one target SSD from the plurality of SSDs after receiving an input / output (I / O) instruction from the processor, based on the target address in the I / O instruction and a second set of addresses stored on the first SSD. The I / O instruction is used to instruct processing of the storage space indicated by the target address, and the storage space indicated by the target address is on the at least one target SSD.

2. The system according to claim 1, characterized in that, The address is based on Logical Unit Number (LUN) semantics to indicate the physical block in the first storage space.

3. The system according to claim 2, characterized in that, The address includes a Logical Unit Number (LUN) ID, a Logical Block Address (LBA), and a Physical Block Address (PBA). The LUN ID is used to identify a LUN in the first storage space. The LUN includes at least one logical block, and there is a mapping relationship between the logical block and the physical block. The LBA is used to identify the logical block, and the PBA is used to identify the physical block.

4. The system according to any one of claims 1-3, characterized in that, The data processing system includes a peer-to-peer interconnect bus. The processor is used to send I / O instructions to each solid-state drive via the peer-to-peer interconnect bus.

5. The system according to any one of claims 1-4, characterized in that, The processor and the plurality of solid-state drives are deployed in the same device.

6. The system according to any one of claims 1-4, characterized in that, The processor and the first solid-state drive are deployed in different devices.

7. The system according to claim 6, characterized in that, The data processing system includes multiple processors, which are deployed in a first device cluster, and multiple solid-state drives are deployed in a second device cluster. The first device cluster and the second device cluster are different device clusters.

8. The system according to any one of claims 1-7, characterized in that, The first solid-state drive is used to process the storage space indicated by the target address according to the IO instructions, wherein, The I / O instruction is used to instruct the storage space indicated by the target address on the target solid-state drive to be stored in the dynamic random access memory (DRAM); or, The I / O instruction is used to instruct the calculation of the first data to obtain a first calculation result, and to store the first calculation result in the DRAM; or... The I / O instruction is used to instruct the second data in the DRAM to be stored in the storage space indicated by the target address on the target solid-state drive; or, The IO instruction is used to instruct the second data to be calculated to obtain a second calculation result, and to store the second calculation result in the storage space indicated by the target address on the target solid-state drive.

9. The system according to claim 8, characterized in that, The first solid-state drive (SSD) is used to manage the storage space indicated by the target address on the target SSD, wherein the target SSD and the first SSD are different SSDs. The first solid-state drive is specifically used to send a read command to the target solid-state drive and receive a read completion message sent by the target solid-state drive after completing the read command; The read instruction is used to instruct the target solid-state drive to read the first data and write the first data into the DRAM.

10. The system according to claim 8, characterized in that, The plurality of solid-state drives (SSDs) includes a second SSD, which manages the storage space indicated by the target address on the target SSD. The first SSD, the second SSD, and the target SSD are all different SSDs. The first solid-state drive is specifically used to send an acquisition command to the second solid-state drive and receive an acquisition completion message sent by the second solid-state drive after completing the acquisition command; The acquisition instruction is used to instruct the second solid-state drive to notify the target solid-state drive to read the first data and write the first data into the DRAM.

11. The system according to claim 8, characterized in that, The I / O instruction is used to instruct the first data to be stored in the DRAM according to the disk array RAID algorithm. The first solid-state drive is specifically used to obtain data blocks and check blocks in the first stripe, obtain the first data based on the data blocks and the check blocks, and write the first data into the DRAM; The first strip is the strip to which the first data belongs.

12. The system according to claim 8, characterized in that, The first solid-state drive (SSD) is used to manage the storage space indicated by the target address on the target SSD, wherein the target SSD and the first SSD are different SSDs. The first solid-state drive is specifically used to send a write command to the target solid-state drive and receive a write completion message sent by the target solid-state drive after completing the write command; The write instruction is used to instruct the target solid-state drive to read the second data from the DRAM and write the second data into the storage space indicated by the target address on the target solid-state drive.

13. The system according to claim 8, characterized in that, The plurality of solid-state drives (SSDs) includes a second SSD, which manages the storage space indicated by the target address on the target SSD. The first SSD, the second SSD, and the target SSD are all different SSDs. The first solid-state drive is specifically used to send a storage instruction to the second solid-state drive and receive a storage completion message sent by the second solid-state drive after completing the storage instruction; The storage instruction is used to instruct the second solid-state drive to notify the target solid-state drive to read the second data from the DRAM and write the second data to the storage space indicated by the target address on the target solid-state drive.

14. The system according to claim 8, characterized in that, The IO instruction is used to instruct the second data to be stored on the target solid-state drive in the storage space indicated by the target address, according to the RAID algorithm. The first solid-state drive is specifically used to read the second data from the DRAM, obtain a data block in the second stripe, obtain a check block based on the data block and the second data, store the second data block in the physical block corresponding to the target address on the target solid-state drive, and store the check block in the second stripe; The second strip is the strip to which the second data belongs.

15. A data processing method, characterized in that, This invention relates to a data processing system, comprising a processor and multiple solid-state drives (SSDs). The SSDs provide at least one physical block, which constitutes a first storage space. The processor stores a first address set, which includes multiple addresses, each indicating one or more physical blocks within the first storage space. The SSDs include a first SSD, which stores a second address set, which includes the second address set. The method includes: The processor sends input / output I / O instructions to the first solid-state drive; The processor receives a completion message sent by the first solid-state drive after completing the IO instruction; The IO instruction carries a target address; the IO instruction is used to instruct the first solid-state drive to determine at least one target solid-state drive from the plurality of solid-state drives based on the target address and the second address set; the IO instruction is used to instruct processing of the storage space indicated by the target address; the storage space indicated by the target address is on the at least one target solid-state drive.

16. The method according to claim 15, characterized in that, After the processor sends an I / O instruction to the first solid-state drive and before the processor receives a completion message sent by the first solid-state drive after completing the I / O instruction, the method further includes: The first solid-state drive processes the storage space indicated by the target address according to the I / O instructions, wherein, The I / O instruction is used to instruct the storage space indicated by the target address on the target solid-state drive to be stored in the dynamic random access memory (DRAM); or, The I / O instruction is used to instruct the calculation of the first data to obtain a first calculation result, and to store the first calculation result in the DRAM; or... The I / O instruction is used to instruct the second data in the DRAM to be stored in the storage space indicated by the target address on the target solid-state drive; or, The IO instruction is used to instruct the second data to be calculated to obtain a second calculation result, and to store the second calculation result in the storage space indicated by the target address on the target solid-state drive.

17. The method according to claim 16, characterized in that, The first solid-state drive (SSD) is used to manage the storage space indicated by the target address on the target SSD, wherein the target SSD and the first SSD are different SSDs. The first solid-state drive processes the storage space indicated by the target address according to the IO instructions, including: The first solid-state drive sends a read command to the target solid-state drive, the read command being used to instruct the target solid-state drive to read the first data and write the first data into the DRAM; The first solid-state drive receives a read completion message sent by the target solid-state drive after completing the read instruction.

18. The method according to claim 16, characterized in that, The plurality of solid-state drives (SSDs) includes a second SSD, which manages the storage space indicated by the target address on the target SSD. The first SSD, the second SSD, and the target SSD are all different SSDs. The first solid-state drive processes the storage space indicated by the target address according to the IO instructions, including: The first solid-state drive sends an acquisition command to the second solid-state drive, the acquisition command being used to instruct the second solid-state drive to notify the target solid-state drive to read the first data and write the first data into the DRAM; The first solid-state drive receives an acquisition completion message sent by the second solid-state drive after completing the acquisition instruction.

19. The method according to claim 16, characterized in that, The I / O instruction is used to instruct the first data to be stored in the DRAM according to the disk array RAID algorithm. The first solid-state drive processes the storage space indicated by the target address according to the IO instructions, including: The first solid-state drive acquires the data block and check block in the first stripe, where the first stripe is the stripe to which the first data belongs; The first solid-state drive obtains the first data based on the data block and the check block; The first solid-state drive writes the first data into the DRAM.

20. The method according to claim 16, characterized in that, The first solid-state drive (SSD) is used to manage the storage space indicated by the target address on the target SSD, wherein the target SSD and the first SSD are different SSDs. The first solid-state drive processes the storage space indicated by the target address according to the IO instructions, including: The first solid-state drive sends a write command to the target solid-state drive, the write command being used to instruct the target solid-state drive to read the second data from the DRAM and write the second data into the storage space indicated by the target address on the target solid-state drive; The first solid-state drive receives a write completion message sent by the target solid-state drive after completing the write instruction.

21. The method according to claim 16, characterized in that, The plurality of solid-state drives (SSDs) includes a second SSD, which manages the storage space indicated by the target address on the target SSD. The first SSD, the second SSD, and the target SSD are all different SSDs. The first solid-state drive processes the storage space indicated by the target address according to the IO instructions, including: The first solid-state drive sends a storage instruction to the second solid-state drive, the storage instruction being used to instruct the second solid-state drive to notify the target solid-state drive to read the second data from the DRAM and write the second data into the storage space indicated by the target address on the target solid-state drive; The first solid-state drive receives a storage completion message sent by the second solid-state drive after completing the storage instruction.

22. The method according to claim 16, characterized in that, The IO instruction is used to instruct the second data to be stored on the target solid-state drive in the storage space indicated by the target address, according to the RAID algorithm. The first solid-state drive processes the storage space indicated by the target address according to the IO instructions, including: The first solid-state drive reads the second data from the DRAM; The first solid-state drive acquires data blocks from the second stripe, where the second stripe is the stripe to which the second data belongs; The first solid-state drive obtains a verification block based on the data block and the second data; The first solid-state drive stores the second data block in the physical block corresponding to the target address on the target solid-state drive, and stores the check block in the second stripe.

23. The method according to any one of claims 15-22, characterized in that, The address includes a Logical Unit Number (LUN) ID, a Logical Block Address (LBA), and a Physical Block Address (PBA). The target address includes a target LUN ID and a target LBA. The LUN ID is used to identify a LUN in the first storage space. The LUN includes at least one logical block. There is a mapping relationship between the logical block and the physical block. The LBA is used to identify the logical block, and the PBA is used to identify the physical block.

24. The method according to any one of claims 15-23, characterized in that, The plurality of solid-state drives (SSDs) includes a second SSD, and the second set of addresses stored on the second SSD belongs to the first set of addresses. The method further includes: The processor sends an I / O instruction carrying a first address to the second solid-state drive; The processor receives a first completion message sent by the second solid-state drive after completing the IO instruction carrying the first address; Wherein, the I / O instructions sent by the processor to the first solid-state drive are generated based on the first task, and the I / O instructions carrying the first address are generated based on the second task. The first task and the second task are independent of each other. The I / O instructions carrying the first address are used to instruct the second solid-state drive to determine the target solid-state drive corresponding to the first address from the plurality of solid-state drives according to the first address and the second address set stored on the second solid-state drive. The I / O instructions carrying the first address are used to instruct the processing of the storage space indicated by the first address. The storage space indicated by the first address is on the target solid-state drive corresponding to the first address.

25. A computer program product containing instructions, characterized in that, When the instructions are executed by the computing device, the computing device performs the method as described in any one of claims 15-24.

26. A computer-readable storage medium, characterized in that, It includes computer program instructions that, when run on a computing device, implement the method as described in any one of claims 15-24.

Images