Storage system and its computational storage processor, solid-state drive, and data reading / writing method

Abstract

A storage system, its computational storage processor, solid-state drive, and data read / write method includes a solid-state drive (SSD) and a computational storage processor (CSP), and communication between the SSD and the CSP, and between the SSD, the CSP, and the outside of the storage system, is performed using a peer-to-peer communication protocol of a high-speed serial computer expansion bus (PCIe). The CSP is configured to receive a first operation command from the outside, and to generate a second operation command based on a flash memory address based on information included in the first operation command and locally maintained SSD resource information, and to send the second operation command to the SSD. After receiving the second operation command, the SSD is configured to exchange the data to be stored with the outside based on information included in the second operation command.

Description

[Technical Field]

[0001] This application claims priority to a Chinese patent application filed with the China Patent Office on August 17, 2022, bearing application number 202210985642.6 and entitled "Storage system and its computational storage processor, solid-state drive, and data reading and writing method," the contents of which are hereby incorporated by reference.

[0002] TECHNICAL FIELD The present disclosure relates to data storage technology, and more particularly to a storage system and its computational storage processor, solid-state drive, and method for reading and writing data. [Background technology]

[0003] Currently, some architectures in the field of computer storage use a CSP (Computational Storage Processor) as a front-end. However, with the development of the computational storage field, it is becoming increasingly necessary for object storage or key-value store commands received by storage devices to no longer be block address-based read and write commands. Therefore, when a CSP is used as a front-end, large amounts of data must pass through the CSP, resulting in increased latency and power consumption due to unnecessary data copies. To adapt to the development of the computational storage field, existing related devices can solve this problem by adding one or more CSPs. However, this solution is very cost-effective, and research into effective solutions is needed. Summary of the Invention

[0004] The following is a summary of the subject matter described in detail in the text, which is not intended to limit the scope of protection of the claims.

[0005] Some embodiments of the present application provide a storage system, which may include a solid-state drive (SSD) and a computational storage processor (CSP), wherein communication between the SSD and the CSP, and between the SSD, the CSP and an external device of the storage system, is performed using a peer-to-peer communication protocol of a high-speed serial computer expansion bus (PCIe) bus; The CSP is configured to receive a first operation command based on an external storage object, generate a second operation command based on a flash memory address based on information included in the first operation command and locally maintained SSD resource information, and send the second operation command to the SSD; the SSD is configured to exchange the data to be stored with an external device based on information included in the second operation command after receiving the second operation command; The first operation command includes a first write command and a first read command, and the second operation command includes a second write command and a second read command.

[0006] Some embodiments of the present application provide a computational storage processor (CSP) in a storage system, which may include a first high-speed serial computer expansion bus (PCIe) bus interface, a storage management system, and an information manager; the first high-speed serial computer expansion bus standard (PCIe) bus interface is connected to a PCIe bus and configured to communicate with a solid-state drive (SSD) in the storage system and with an external device to the storage system via the PCIe bus; the storage management system is configured to receive a first operation command based on an external storage object via a first PCIe bus interface, generate a second operation command based on a flash memory address based on information included in the first operation command and locally maintained SSD resource information, and send the second operation command to the SSD, the first operation command including a first write command and a first read command, and the second operation command including a second write command and a second read command; The information manager is configured to maintain SSD resource information in the storage system.

[0007] Some embodiments of the present application provide a solid-state drive (SSD) in a storage system, which may include a master chip and a flash memory chip, and the master chip may include a second high-speed serial computer expansion bus standard (PCIe) bus interface and a storage controller; the second high-speed serial computer expansion bus standard (PCIe) bus interface is connected to a PCIe bus and configured to communicate with a computational storage processor (CSP) in the storage system and with an external device to the storage system via the PCIe bus; The storage controller is configured to receive a second operation command based on the flash memory address sent by the CSP, and exchange the data to be stored with the outside based on the information contained in the second operation command, and the second operation command includes a second write command and a second read command.

[0008] Some embodiments of the present application provide a data writing method, which is applied to a computational storage processor (CSP) in a storage system, and the CSP communicates with a solid-state drive (SSD) in the storage system and with an external device via a high-speed serial computer expansion bus (PCIe) bus, the method comprising: receiving a first write command based on an external storage object; generating a second write command based on the flash memory address based on the information included in the first write command and locally maintained SSD resource information, and sending the second write command to the SSD.

[0009] Some embodiments of the present application provide a data writing method, which is applied to a solid-state drive (SSD) in a storage system, and the SSD communicates with a computational storage processor (CSP) in the storage system and with an external device via a high-speed serial computer expansion bus (PCIe) bus, and the method includes: receiving a second write command based on the flash memory address sent by the CSP, and analyzing the start source address of the data to be written this time and the flash memory address in the SSD included in the second write command; reading the data to be stored from outside based on the analyzed starting source address and writing it to the analyzed flash memory address.

[0010] Some embodiments of the present application provide a data reading method, the method being applied to a computational storage processor (CSP) in a storage system, the CSP communicating with a solid-state drive (SSD) in the storage system and with an external device via a high-speed serial computer expansion bus (PCIe) bus, the method comprising: receiving a first read command based on an external storage object; generating a second read command based on the flash memory address based on information included in the first read command and locally maintained SSD resource information, and sending the second read command to the SSD.

[0011] Some embodiments of the present application provide a data reading method, which is applied to a solid-state drive (SSD) in a storage system, and the SSD communicates with a computational storage processor (CSP) in the storage system and with an external device via a high-speed serial computer expansion bus (PCIe) bus, and the method includes: receiving a second read command based on the flash memory address sent by the CSP, and analyzing the flash memory address in the SSD of the data to be read this time and the start address of data reception included in the second read command; The method may include reading the data to be stored from the SSD based on the analyzed flash memory address, and writing it to the analyzed start address of external data reception.

[0012] Some embodiments of the present application provide a computational storage processor (CSP), which may include a third high-speed serial computer expansion bus standard (PCIe) bus interface and a first logic circuit assembly coupled to the third PCIe bus interface, the first logic circuit assembly configured to perform the data writing method according to the fourth aspect and the data reading method according to the sixth aspect.

[0013] Some embodiments of the present application further provide a master chip for a solid-state drive (SSD), comprising a fourth high-speed serial computer expansion bus (PCIe) bus interface and a second logic circuit assembly coupled to the fourth PCIe bus interface, the second logic circuit assembly configured to perform the data writing method according to the fifth aspect and the data reading method according to the seventh aspect.

[0014] Some embodiments of the present application provide a non-volatile storage medium having a computer program stored thereon, the computer program being configured to, when executed by a processor, perform the data writing methods described in the fourth and fifth aspects and the data reading methods described in the sixth and seventh aspects.

[0015] Additional features and advantages of the present disclosure will be set forth in the description which follows, and in part will be obvious from the description, or may be learned by the practice of the present disclosure. The objects and advantages of the present disclosure may be realized and obtained by the structure particularly pointed out in the description and drawings.

[0016] Other aspects will be understood after reading and understanding the drawings and detailed description. [Brief explanation of the drawings]

[0017] The drawings provide a further understanding of the technical solutions of the present disclosure, constitute a part of the specification, and are used to explain the technical solutions together with the embodiments of the present disclosure, but are not intended to limit the technical solutions.

[0018] [Figure 1] 1 is a structural schematic diagram of a storage system according to an embodiment of the present application; [Figure 2] FIG. 2 is a first configuration block diagram of a CSP in a storage system according to an embodiment of the present application. [Figure 3] FIG. 2 is a first configuration block diagram of an SSD in a storage system according to an embodiment of the present application. [Figure 4] 1 is a flowchart of a data writing method on the CSP side according to an embodiment of the present application. [Figure 5] 1 is a flowchart of a data writing method on an SSD side according to an embodiment of the present application. [Figure 6] 1 is a flowchart of a method for reading data on the CSP side according to an embodiment of the present application; [Figure 7] 1 is a flowchart of a method for reading data on an SSD according to an embodiment of the present application. [Figure 8] FIG. 10 is a second configuration block diagram of a CSP in a storage system according to an embodiment of the present application. [Figure 9] FIG. 2 is a second configuration block diagram of an SSD in a storage system according to an embodiment of the present application. DETAILED DESCRIPTION OF THE INVENTION

[0019] While multiple embodiments are described herein, the descriptions are illustrative and not limiting. Those skilled in the art will recognize that many more embodiments and implementations are possible within the scope of the embodiments described herein. While many possible combinations of features are shown in the drawings and discussed in specific embodiments, many other combinations of the disclosed features are possible. Unless otherwise specified, any feature or element of any embodiment may be used in combination with or substituted for any other feature or element in any other embodiment.

[0020] The present application contemplates the inclusion of combinations of features and elements known to those skilled in the art. The embodiments, features, and elements disclosed herein may be combined with any conventional features or elements to form a unique inventive solution defined by the claims. Any feature or element of any embodiment may be combined with features or elements from other inventive solutions to form another unique inventive solution defined by the claims. It is therefore understood that any feature shown or discussed herein may be implemented alone or in any suitable combination. Therefore, the embodiments are not limited except as limited by the appended claims and their equivalent substitutions. Various modifications and variations are possible within the scope of protection of the appended claims.

[0021] Furthermore, in describing exemplary embodiments, the specification may present a method and / or process as a particular sequence of steps. However, if the method or process does not depend on the particular order of steps described herein, the method or process should not be limited to that particular order of steps. As one of ordinary skill in the art will appreciate, other order of steps are possible. Thus, the order of features of steps described in the specification should not be construed as limitations on the claims. Also, claims to the method and / or process do not necessarily require the steps to be performed in the order described; as one of ordinary skill in the art will readily understand, these orders may be changed and still fall within the spirit and scope of the embodiments of the present application.

[0022] First aspect: Some embodiments of the present application provide a storage system, which may include a solid-state drive (SSD) and a computational storage processor (CSP), as shown in FIG. 1 , Communication can be performed between the SSD and the CSP, and between the SSD, the CSP and the outside of the storage system using a peer-to-peer communication protocol of a high-speed serial computer expansion bus standard (Peripheral Component Interconnect Express, PCIe), The CSP is configured to receive a first operation command based on an external storage object, generate a second operation command based on a flash memory address based on information included in the first operation command and locally maintained SSD resource information, and send the second operation command to the SSD; the SSD is configured to exchange the data to be stored with an external device based on information included in the second operation command after receiving the second operation command; The first operation command includes a first write command and a first read command, and the second operation command includes a second write command and a second read command.

[0023] In an exemplary embodiment of the present application, the outside of the storage system may include one or more requesters.

[0024] In an exemplary embodiment of the present application, the storage system may include a CSP and multiple SSDs, both of which access a PCIe bus and communicate via a peer-to-peer communication protocol.

[0025] In an exemplary embodiment of the present application, the CSP receives an external storage request, i.e., the above-mentioned first operation command (the first operation command may be a first write command or a first read command), via a PCIe bus, and the first operation command may include information related to the storage object. Based on the information related to the storage object and the resource information of each SSD to be stored, the CSP can perform processing such as assigning an SSD address to the data to be stored, or searching for the SSD address where the data to be stored is stored, and generates a corresponding second operation command (corresponding to the above-mentioned first operation command, the second operation command is a second write command or a second read command, i.e., if the first operation command is the first write command, the second operation command is a second write command, and if the first operation command is the first read command, the second operation command is a second read command) based on the processing result. Using the peer-to-peer communication mechanism of the PCIe bus, the CSP can send the generated second operation command to the assigned or searched SSD, which then exchanges data directly with the external requester (also referred to as the external data exchanger) based on the second operation command, without going through the CSP. Only the command is transmitted between the CSP and the SSD, so there is no need for a large amount of data to pass through the CSP, and the CSP does not need to copy unnecessary data. This significantly reduces the workload of the CSP and reduces latency and power consumption.

[0026] The following describes in detail the embodiments of the present application in two aspects, namely, data writing and data reading.

[0027] In an exemplary embodiment of the present application, if the first operation command sent from the outside is a first write command, the first write command may include, but is not limited to, an identifier of the storage object, data length information, and the starting source address of the entire data.

[0028] In an exemplary embodiment of the present application, the SSD resource information includes resource occupation information in the SSD, and after receiving a first write command, the CSP: Analyzing the storage target identifier, data length information, and start source address of the entire data included in the first write command; Allocating flash memory addresses in one or more SSDs to be used for writing the data to be stored based on the data length information and resource occupation information in the SSDs, and determining a start source address of a portion of the data to be written each time based on a start source address of the entire data to be stored and an offset of the portion of the data to be written each time in the entire data to be stored; One or more second write commands are generated for each SSD to which a flash memory address is assigned and sent to the SSD, and each second write command includes a starting source address of data to be written to the SSD at this time and a flash memory address to be used for data writing, thereby configuring the device to generate and send second operation commands based on the flash memory address to the SSD.

[0029] In exemplary embodiments of the present application, the flash memory address, i.e., any form of address provided by the SSD to the outside, may be, for example, the address of a flash memory page, a flash memory block, or a storage block, or may be an address segment of a flash memory page, a flash memory block, or a storage block. If the flash memory address is a storage block address, the SSD provides a logical address to the outside, and if the flash memory address is an address of a flash memory page or a flash memory block, the SSD provides an operation interface for a physical address to the outside. In the embodiments of the present application, the flash memory address is described as an address of a flash memory page or a flash memory block.

[0030] In an exemplary embodiment of the present application, the resource occupation information in the SSD may include, but is not limited to, addresses of all SSDs included in the storage system, flash memory pages in each SSD, at least one address of a flash memory block, resource occupation information of each flash memory page, identifiers and offsets of storage objects corresponding to each occupied flash memory page, etc.

[0031] In an exemplary embodiment of the present application, the CSP can determine flash memory addresses in one or more SSDs that can store the data of the storage object currently waiting to be stored based on the resource occupancy information, and assign one or more of the flash memory addresses to the storage object based on the data volume size of the storage object waiting to be stored.

[0032] In an exemplary embodiment of the present disclosure, the CSP can determine whether the data to be stored is large data or small data based on the data length information included in the first write command, and assign a corresponding flash memory address based on the determined data size. However, the embodiment of the present disclosure is not limited thereto. For example, the data to be stored may be divided into more types, or the data to be stored may not be divided into large data or small data.

[0033] In an exemplary embodiment of the present application, if the data length information is less than or equal to a predetermined first data amount threshold, the data may be determined as small data, and if the data length information is greater than the predetermined first data amount threshold, the data may be determined as large data.

[0034] In an exemplary embodiment of the present application, the first data amount threshold may be the storage data amount of one flash memory page, or other data amount of customization, and the specific value of the first data amount threshold is not limited here.

[0035] In an exemplary embodiment of the present application, if the data waiting to be written is small data, the CSP may allocate only one flash memory page to the data; if the data waiting to be written is large data, the CSP may allocate multiple flash memory pages to the data, or may allocate one or more flash memory blocks.

[0036] In an exemplary embodiment of the present application, when the CSP allocates at least one of a plurality of flash memory pages and a plurality of flash memory blocks to the large data, the plurality of flash memory pages may be flash memory pages with consecutive addresses, and the plurality of flash memory blocks may be flash memory blocks with consecutive addresses, thereby allowing the large data to be allocated one address segment with consecutive addresses.

[0037] In an exemplary embodiment of the present application, if the large amount of storage objects waiting to be written are all small data, one second write command may be generated each time one small data is written. In another embodiment, multiple small data that need to be continuously written may be merged and then written, that is, one second write command may be generated after merging, thereby reducing the workload of the storage system, improving storage system performance, and reducing power consumption.

[0038] In an exemplary embodiment of the present application, for example, after receiving a plurality of first write commands for a plurality of small data, the information contained in the plurality of first write commands is merged, and a second write command is generated and sent to the SSD based on the merged information and locally maintained SSD resource information, and then the SSD respectively retrieves each small data based on the corresponding starting source address and stores it in the assigned flash memory address.

[0039] In an exemplary embodiment of the present application, when the starting source addresses of multiple small data are consecutive, multiple consecutive flash memory addresses can be assigned to the multiple small data based on the arrangement order of the small data stored at the multiple starting source addresses, and the arrangement order of the small data stored at the multiple consecutive flash memory addresses is the same as the arrangement order of the small data stored at the multiple starting source addresses.

[0040] In an exemplary embodiment of the present application, when the storage object is allocated to multiple different SSDs, or allocated to different flash memory addresses in the same SSD, the amount of data required for each flash memory address can be obtained from an external requester and stored in the flash memory address. Therefore, based on the starting source address of the entire data of the storage object and the offset of the portion of the data of the storage object to be written each time in the entire data of the storage object, the starting source address of the portion of the data to be written each time is determined, and corresponding data is obtained and stored based on the starting source address of the portion of the data.

[0041] In an exemplary embodiment of the present application, when the CSP sends a second write command to the SSD, the CSP may send one second write command to each allocated flash memory address, or may send one second write command to multiple consecutive flash memory addresses (i.e., an address segment).

[0042] In an exemplary embodiment of the present application, the second write command may include, but is not limited to, an NVMe (Non-Volatile Memory express) write command.

[0043] NVMe is the mainstream SSD protocol on the market today. It is a transmission protocol based on the PCIe bus physical layer. Its characteristic is that it only transmits read and write commands and does not directly control data exchange. For example, a read command only contains the SSD's storage address and data receiving address, allowing the SSD to execute a large number of commands in its own order. When data transmission is required, it will transmit the data to the receiving address via the PCIe transmission mechanism.

[0044] In an exemplary embodiment of the present application, the SSD is configured to, after receiving a second write command, analyze a starting source address and a flash memory address included in the second write command, read data of a default size of the storage target from an external device based on the analyzed starting source address, and write the read data to the analyzed flash memory address, thereby exchanging the data of the storage target with an external device; or The second write command sent by the CSP further includes information on the amount of data to be written this time to the storage target, and when the SSD receives the second write command, it analyzes the start source address, flash memory address, and data amount information included in the second write command, and based on the analyzed start source address and data amount information, reads the corresponding amount of data from the storage target from the outside and writes the read data to the analyzed flash memory address, thereby exchanging the data of the storage target with the outside. The corresponding data amount is the data amount indicated by the data amount information.

[0045] In an exemplary embodiment of the present application, the second write command may include only the starting source address and flash memory address of the storage target, or may include both the starting source address and flash memory address of the storage target (e.g., the starting addresses of multiple flash memory pages or flash memory blocks) and information about the amount of data to be stored.

[0046] In an exemplary embodiment of the present application, when the second write command is an NVMe write command, the NVMe write command may be a normal NVMe command, including a starting source address of the storage object and a flash memory address, or the NVMe write command may be an extended NVMe command, including a starting source address of the storage object, data amount information and a flash memory address.

[0047] In an exemplary embodiment of the present application, the data amount information may include, but is not limited to, a total data capacity (eg, 32K, 64K, etc.), a required flash memory page, a total number of at least one of flash memory blocks, and the like.

[0048] In an exemplary embodiment of the present application, for example, for small data, the data can be stored in only one flash memory page, so one flash memory page is allocated to the small data, and the second write command only needs to include the starting source address of the data to be stored and the address of the flash memory page.

[0049] In an exemplary embodiment of the present application, for example, for large data, multiple flash memory pages are required to store the data, so multiple flash memory pages may be allocated to the large data and data amount information may be attached. For example, the total number of required flash memory pages may be attached, and the second write command may include the starting source address of the storage target, the flash memory address (the allocated multiple flash memory pages, at least one starting address of the flash memory block), and the data amount information.

[0050] In an exemplary embodiment of the present application, the size of each flash memory block may be 16kB, so that the starting address of the first flash memory block is equal to the original starting address, and the starting address of the second flash memory block is equal to the original starting address + 16kB, by which the starting address of any one flash memory block can be determined by analogy, and the same applies to the flash memory page, which will not be repeated here.

[0051] In an exemplary embodiment of the present application, if the first operation command sent from outside is a first read command, the first read command may include, but is not limited to, an identifier of the storage object and a start address of data reception.

[0052] In an exemplary embodiment of the present application, the SSD resource information includes a flash memory address in the SSD to which the data to be stored is to be written, and after receiving the first read command, the CSP: analyzing the identifier of the storage object and the start address of data reception included in the first read command; retrieving flash memory addresses in one or more SSDs to which the pre-stored data of the storage object is to be written based on the storage object identifier; One or more second read commands are generated for each SSD in which the data to be stored is stored and sent to the SSD, and each of the second read commands includes the flash memory address of the data to be read this time in the SSD and the start address of data reception, so that a second operation command based on the flash memory address is generated and sent to the SSD.

[0053] In an exemplary embodiment of the present application, the CSP can search for the flash memory address in the SSD of the data of the storage object to be read based on the identifier of the storage object included in the first read command, and can include the flash memory address and the start address of the external requester's data reception in a second read command and send it to the SSD.

[0054] In an exemplary embodiment of the present application, the data of the storage object waiting to be read may be large data or small data, and if the data of the storage object is large data and occupies at least one complete flash memory page or flash memory block (for example, there is waste due to complete occupancy or incomplete occupancy), the second read command may only include the address of at least one of the one or more flash memory pages or flash memory blocks, and the start address of data reception. If the storage object is large data and occupies at least one of a plurality of contiguous flash memory pages or flash memory blocks, the second read command may include the start address of at least one of the plurality of contiguous flash memory pages or flash memory blocks, and the data amount information may indicate the quantity of at least one of the flash memory pages or flash memory blocks, so the second read command may include the start address of at least one of the plurality of contiguous flash memory pages or flash memory blocks, the data amount information of the storage object, and the start address of data reception. If the storage object is small data and occupies only a portion of the space of one flash memory page, and another portion of the space of the flash memory page is occupied by other data, the second read command can include not only the address of the flash memory page, but also the offset of the flash memory address of the storage object to be read, so that the required data can be accurately read from the flash memory page based on the offset. Therefore, the second read command can include the address and offset of the flash memory page, and the start address of data reception.

[0055] In an exemplary embodiment of the present application, the second read command may include, but is not limited to, an NVMe read command.

[0056] In an exemplary embodiment of the present application, the second read command may be an NVMe read command, and the flash memory address included in the NVMe read command is the address of one flash memory page; or The second read command may be an extended NVMe write command, and the flash memory address included in the extended NVMe read command is an address segment consisting of addresses of multiple consecutive flash memory pages, and the address segment may be indicated by a starting address and data amount information.

[0057] In an exemplary embodiment of the present application, if the second read command is an NVMe read command, the NVMe read command may be a normal NVMe command, including the flash memory address to be stored and the start address of data reception, or the NVMe write command may be an extended NVMe command, including the flash memory address to be stored, data amount information and the start address of data reception.

[0058] In an exemplary embodiment of the present application, the data amount information may include, but is not limited to, a total data capacity (eg, 32K, 64K, etc.), a required flash memory page, a total number of at least one of flash memory blocks, and the like.

[0059] In an exemplary embodiment of the present application, the offset refers to the offset of the small data in one flash memory page relative to the flash memory address of the flash memory page, and the offset can be represented by two offset amounts.

[0060] In an exemplary embodiment of the present application, a small piece of data may be stored in only one flash memory page, may completely occupy one flash memory page, or may only occupy a portion of the flash memory page, with the other portion being occupied by other small pieces of data. Therefore, for small pieces of data that completely occupy one flash memory page, the second read command only needs to include the start address of the data to be stored and the address of the flash memory page. For small pieces of data that occupy one flash memory page together with other small pieces of data, the second read command may include the start address of the data to be stored, the address of the flash memory page, and an offset of the small piece of data, which is used by the SSD to extract required data from the read data based on the offset after reading the data from the flash memory page.

[0061] In an exemplary embodiment of the present application, for large data, since multiple flash memory pages are required to store the data, data amount information may be attached when reading the large data (e.g., the total number of flash memory pages to be stored is attached), and the second read command includes the start address of receiving the data to be stored, the flash memory address (one or more flash memory pages to be stored, at least one start address of the flash memory block), and the data amount information.

[0062] In an exemplary embodiment of the present application, the size of each flash memory block may be 16kB, so that the starting address of the first flash memory block is equal to the original starting address, and the starting address of the second flash memory block is equal to the original starting address + 16kB, by which the starting address of any one flash memory block can be determined by analogy, and the same applies to the flash memory page, which will not be repeated here.

[0063] In an exemplary embodiment of the present application, after receiving a second read command, the SSD is configured to analyze the flash memory address and the start address of data reception included in the second read command, read the data to be read this time from the SSD based on the analyzed flash memory address, and send the read data to the start address of data reception externally, thereby exchanging the data to be stored with the external device.

[0064] In an exemplary embodiment of the present application, for a second read command that includes only a flash memory address and a start address of data reception, the SSD can retrieve all data stored in at least one of the corresponding one or more flash memory pages or flash memory blocks from the SSD based on the flash memory address, and store them at the start address of data reception.

[0065] In an exemplary embodiment of the present application, the second read command sent by the CSP may further include information on the amount of data to be stored, and the SSD is configured, upon receiving the second read command, to analyze the flash memory address, the start address of data reception, and the data amount information included in the second read command, read the data waiting to be read this time from the SSD based on the analyzed flash memory address and the data amount information, and send the read data to the start address of data reception externally, thereby exchanging the data to be stored with the external device.

[0066] In an exemplary embodiment of the present application, for a second read command including a flash memory address, data amount information, and a start address of data reception, the SSD can retrieve all data stored in at least one of the corresponding multiple flash memory pages or flash memory blocks based on the flash memory address and data amount information, and store them at the start address of data reception.

[0067] In an exemplary embodiment of the present application, the second read command sent by the CSP may further include an offset of the flash memory address of the storage target, and the SSD is configured, upon receiving the second read command, to analyze the flash memory address and its offset and the start address of data reception included in the second read command, and read a portion of the data to be stored from one flash memory page based on the analyzed flash memory address and its offset, or read one page of data from one flash memory page based on the analyzed flash memory address, extract necessary data from the one page of data based on the offset, and write it to the start address of the external data reception, thereby exchanging the data to be stored with the outside.

[0068] In an exemplary embodiment of the present application, for a second read command including a flash memory address, an offset, and a start address of data reception, the SSD can retrieve all data stored in a corresponding flash memory page based on the flash memory address, extract required data from the read data based on the offset, and store the extracted data at the start address of data reception, or can read part of the data stored in a flash memory page based on the analyzed flash memory address and its offset.

[0069] In an exemplary embodiment of the present application, the SSD may be further configured to return a success response to the CSP after successfully exchanging the data to be stored with an external device based on information included in the second operation command; The CSP may further be configured to, after receiving success responses for all the sent second operation commands, return a success response to the sender of the first operation command and update the maintained SSD resource information.

[0070] Second aspect: An embodiment of the present application provides a computational storage processor in a storage system, which may include, as shown in FIG. 2, a first PCIe bus interface 11, a storage management system 12, and an information manager 13; the first PCIe bus interface 11 is connected to a PCIe bus and configured to communicate with a solid-state drive (SSD) in the storage system and with an external device to the storage system via the PCIe bus; the storage management system 12 is configured to receive a first operation command based on an external storage object via the first PCIe bus interface 11, generate a second operation command based on a flash memory address based on information included in the first operation command and locally maintained SSD resource information, and send the second operation command to the SSD, the first operation command including a first write command and a first read command, and the second operation command including a second write command and a second read command; The information manager 13 is configured to maintain SSD resource information in the storage system.

[0071] In an exemplary embodiment of the present application, the storage management system 12 of the CSP receives an external storage request, i.e., the first operation command (which may be a first write command or a first read command), via the first PCIe bus interface 11. The first operation command may include information related to the storage object. Based on the information related to the storage object and the resource information of each SSD stored in the information manager 13, the storage management system 12 can perform operations such as assigning an SSD address to the data to be stored or retrieving the SSD address stored in the data to be stored. Based on the processing result, the storage management system 12 generates a corresponding second operation command (which corresponds to the first operation command and is a second write command or a second read command). Using the peer-to-peer communication mechanism of the PCIe bus, the storage management system 12 can send the generated second operation command to the assigned or retrieved SSD, allowing the SSD to exchange data directly with the external requester based on the second operation command, without going through the CSP. Only commands are transmitted between the CSP and the SSD; no large amounts of data pass through the CSP, and the CSP does not need to copy unnecessary data. This significantly reduces the workload for CSPs, lowering latency and power consumption.

[0072] The following describes in detail the embodiments of the present application in two aspects: data writing and data reading.

[0073] In an exemplary embodiment of the present application, the storage management system 12 may include a write processor 121, which: receiving an external first write command through a first PCIe bus interface (11); analyzing a storage target identifier, data length information, and a start source address of the entire data included in the first write command; Allocate flash memory addresses in one or more SSDs to be used for writing the data to be stored based on the data length information and resource occupation information in the SSDs, and determine a start source address of a portion of the data to be written each time based on a start source address of the entire data to be stored and an offset of the portion of the data to be written each time in the entire data to be stored; For each SSD to which a flash memory address is assigned, one or more second write commands are generated and sent to the SSD, and each second write command can be configured to include the starting source address of the data to be written to the SSD this time and the flash memory address to be used for writing the data.

[0074] In exemplary embodiments of the present application, the flash memory address, i.e., any form of address provided by the SSD to the outside, may be, for example, the address of a flash memory page, a flash memory block, or a storage block, or may be an address segment of a flash memory page, a flash memory block, or a storage block. If the flash memory address is a storage block address, the SSD provides a logical address to the outside, and if the flash memory address is an address of a flash memory page or a flash memory block, the SSD provides an operation interface for a physical address to the outside. In the embodiments of the present application, the flash memory address is described as an address of a flash memory page or a flash memory block.

[0075] In an exemplary embodiment of the present application, if the first operation command sent from the external requester is a first write command, the first write command may include, but is not limited to, an identifier of the storage object, data length information, and a starting source address of the entire data.

[0076] In an exemplary embodiment of the present application, the resource occupation information in the SSD may include, but is not limited to, addresses of all SSDs included in the storage system, flash memory pages in each SSD, at least one address of a flash memory block, resource occupation information of each flash memory page, identifiers and offsets of storage objects corresponding to each occupied flash memory page, etc.

[0077] In an exemplary embodiment of the present application, the write processor 121 can determine flash memory addresses in one or more SSDs that can store the data of the storage object currently waiting to be stored based on the resource occupation information, and assign one or more of the flash memory addresses to the storage object based on the data volume size of the storage object waiting to be stored.

[0078] In an exemplary embodiment of the present application, the write processor 121 can determine whether the data to be stored is large data or small data based on the data length information included in the first write command, and assign a corresponding flash memory address based on the determined data size.

[0079] In an exemplary embodiment of the present application, if the data length information is equal to or less than a first predetermined data amount threshold, the data may be determined as small data, and if the data length information is greater than the first predetermined data amount threshold, the data may be determined as large data, where the first data amount threshold may be the storage data amount of one flash memory page or a customized data amount, and the specific value of the first data amount threshold is not limited.

[0080] In an exemplary embodiment of the present application, if the data waiting to be written is small data, the write processor 121 may allocate only one flash memory page to the data; if the data waiting to be written is large data, the write processor 121 may allocate multiple flash memory pages to the data, or may allocate one or more flash memory blocks.

[0081] In an exemplary embodiment of the present application, when the write processor 121 allocates at least one of a plurality of flash memory pages and a plurality of flash memory blocks to the large data, the plurality of flash memory pages may be flash memory pages with consecutive addresses, and the plurality of flash memory blocks may be flash memory blocks with consecutive addresses, thereby allowing the large data to be allocated one address segment with consecutive addresses.

[0082] In an exemplary embodiment of the present application, if the large amount of storage objects waiting to be written are all small data, one second write command may be generated each time one small data is written. In another embodiment, multiple small data that need to be continuously written may be merged and then written, that is, one second write command may be generated after merging, thereby reducing the workload of the storage system, improving storage system performance, and reducing power consumption.

[0083] In an exemplary embodiment of the present application, for example, after receiving a plurality of first write commands for a plurality of small data, the information contained in the plurality of first write commands is merged, and a second write command is generated and sent to the SSD based on the merged information and locally maintained SSD resource information, and then the SSD respectively retrieves each small data based on the corresponding starting source address and stores it in the assigned flash memory address.

[0084] In an exemplary embodiment of the present application, when the starting source addresses of multiple small data are consecutive, multiple consecutive flash memory addresses can be assigned to the multiple small data based on the arrangement order of the small data stored at the multiple starting source addresses, and the arrangement order of the small data stored at the multiple consecutive flash memory addresses is the same as the arrangement order of the small data stored at the multiple starting source addresses.

[0085] In an exemplary embodiment of the present application, when the storage object is allocated to multiple different SSDs, or allocated to different flash memory addresses in the same SSD, the amount of data required for each flash memory address can be obtained from an external requester and stored in the flash memory address. Therefore, based on the starting source address of the entire data of the storage object and the offset of the portion of the data of the storage object to be written each time in the entire data of the storage object, the starting source address of the portion of the data to be written each time is determined, and corresponding data is obtained and stored based on the starting source address of the portion of the data.

[0086] In an exemplary embodiment of the present application, when the write processor 121 sends the second write command to the SSD, it may send one second write command to each assigned flash memory address, or it may send one second write command to a group of consecutive flash memory addresses (i.e., an address segment).

[0087] In an exemplary embodiment of the present application, the second write command may include, but is not limited to, an NVMe write command.

[0088] In an exemplary embodiment of the present application, the flash memory address included in the second write command may be represented by an address of one flash memory page; When generating one or more second write commands for each SSD to which a flash memory address is assigned, the write processor 121 generates one second write command for each assigned flash memory page in the SSD, and includes the starting source address of the data to be written to the flash memory page and the address of the flash memory page.

[0089] In an exemplary embodiment of the present application, the flash memory address included in the second write command may be one address segment consisting of addresses of multiple consecutive flash memory pages, and the address segment is indicated by a start address and data amount information; When the write processor 121 generates one or more second write commands for each SSD to which a flash memory address is assigned, the write processor 121 may generate one second write command for each address segment in the SSD, and the second write command may include the starting source address of the data to be written to the address segment, and the starting address and data amount information of the address segment.

[0090] In an exemplary embodiment of the present application, the second write command may include only the starting source address and flash memory address of the storage target, or may include both the starting source address and flash memory address of the storage target (e.g., the starting addresses of multiple flash memory pages or flash memory blocks) and information about the amount of data to be stored.

[0091] In an exemplary embodiment of the present application, when the second write command is an NVMe write command, the NVMe write command may be a normal NVMe command, including a starting source address of the storage object and a flash memory address, or the NVMe write command may be an extended NVMe command, including a starting source address of the storage object, data amount information and a flash memory address.

[0092] In an exemplary embodiment of the present application, the data amount information may include, but is not limited to, a total data capacity (eg, 32K, 64K, etc.), a required flash memory page, a total number of at least one of flash memory blocks, and the like.

[0093] In an exemplary embodiment of the present application, for example, for small data, the data can be stored in only one flash memory page, so one flash memory page is allocated to the small data, and the second write command only needs to include the starting source address of the data to be stored and the address of the flash memory page.

[0094] In an exemplary embodiment of the present application, for example, for large data, multiple flash memory pages are required to store the data, so multiple flash memory pages may be allocated to the large data and data amount information may be attached. For example, the total number of required flash memory pages may be attached, and the second write command may include the starting source address of the storage target, the flash memory address (the allocated flash memory page, at least one starting address of the flash memory block), and the data amount information.

[0095] In an exemplary embodiment of the present application, the size of each flash memory block may be 16kB, so that the starting address of the first flash memory block is equal to the original starting address, and the starting address of the second flash memory block is equal to the original starting address + 16kB, by which the starting address of any one flash memory block can be determined by analogy, and the same applies to the flash memory page, which will not be repeated here.

[0096] In an exemplary embodiment of the present application, the write processor 121 is further configured to generate and send one or more second write commands to each SSD to which a flash memory address is assigned, and then, after receiving success responses for all the sent second write commands, return a success response to the sender of the first write command and notify the information manager 13 to record the identifier of the storage object and the flash memory address in the assigned SSD.

[0097] In an exemplary embodiment of the present application, the storage management system 12 may include a read processor 122, which: receiving an external first read command via a first PCIe bus interface (11), and analyzing an identifier of a storage object and a start address of data reception included in the first read command; retrieving flash memory addresses in one or more SSDs to which the pre-stored data of the storage object is to be written based on the identifier of the storage object; One or more second read commands are generated for each SSD in which the data to be stored is stored and sent to the SSD, and each second read command is set to include the flash memory address of the data to be read this time in the SSD and the start address of data reception.

[0098] In an exemplary embodiment of the present application, if the first operation command sent from the external requester is a first read command, the first read command may include, but is not limited to, an identifier of the storage object and a start address of data reception.

[0099] In an exemplary embodiment of the present application, the CSP can search for the flash memory address in the SSD of the data of the storage object to be read based on the identifier of the storage object included in the first read command, and can include the flash memory address and the start address of the external requester's data reception in a second read command and send it to the SSD.

[0100] In an exemplary embodiment of the present application, the data of the storage object waiting to be read may be large data or small data, and if the data of the storage object is large data and occupies at least one complete flash memory page or flash memory block (for example, there is waste due to complete occupancy or incomplete occupancy), the second read command may only include the address of at least one of the one or more flash memory pages or flash memory blocks, and the start address of data reception. If the storage object is large data and occupies at least one of a plurality of contiguous flash memory pages or flash memory blocks, the second read command may include the start address of at least one of the plurality of contiguous flash memory pages or flash memory blocks, and the data amount information may indicate the quantity of at least one of the flash memory pages or flash memory blocks, so the second read command may include the start address of at least one of the plurality of contiguous flash memory pages or flash memory blocks, the data amount information of the storage object, and the start address of data reception. If the storage object is small data and occupies only a portion of the space of one flash memory page, and another portion of the space of the flash memory page is occupied by other data, the second read command can include not only the address of the flash memory page, but also the offset of the data of the storage object waiting to be read, so that the required data can be accurately read from the flash memory page based on the offset. Therefore, the second read command can include the address, offset, and start address of the flash memory page.

[0101] In an exemplary embodiment of the present application, the second read command may include, but is not limited to, an NVMe read command.

[0102] In an exemplary embodiment of the present application, the second read command may be an NVMe read command, and the flash memory address included in the NVMe read command is the address of one flash memory page; or The second read command may be an extended NVMe write command, and the flash memory address included in the extended NVMe read command is an address segment consisting of addresses of one flash memory page or multiple consecutive flash memory pages, and the address segment can be indicated by a starting address and data amount information.

[0103] In an exemplary embodiment of the present application, if the second read command is an NVMe read command, the NVMe read command may be a normal NVMe command, including the flash memory address to be stored and the start address of data reception, or the NVMe write command may be an extended NVMe command, including the flash memory address to be stored, data amount information and the start address of data reception.

[0104] In an exemplary embodiment of the present application, the data amount information may include, but is not limited to, a total data capacity (eg, 32K, 64K, etc.), a required flash memory page, a total number of at least one of flash memory blocks, and the like.

[0105] In an exemplary embodiment of the present application, the offset refers to the offset of the small data in one flash memory page relative to the flash memory address of the flash memory page, and the offset can be represented by two offset amounts.

[0106] In an exemplary embodiment of the present application, a small piece of data may be stored in only one flash memory page, may completely occupy one flash memory page, or may only occupy a portion of the flash memory page, with the other portion being occupied by other small pieces of data. Therefore, for small pieces of data that completely occupy one flash memory page, the second read command only needs to include the start address of the data to be stored and the address of the flash memory page. For small pieces of data that occupy one flash memory page together with other small pieces of data, the second read command may include the start address of the data to be stored, the address of the flash memory page, and an offset of the small piece of data, which is used by the SSD to extract required data from the read data based on the offset after reading the data from the flash memory page.

[0107] In an exemplary embodiment of the present application, for large data, since storing the data requires multiple flash memory pages, data amount information may be attached when reading the large data (e.g., the total number of flash memory pages to be stored is attached), and the second read command includes the start address of receiving the data to be stored, the flash memory address (the start address of at least one of the flash memory pages, flash memory blocks to store the data), and the data amount information.

[0108] In an exemplary embodiment of the present application, the size of each flash memory block may be 16kB, so that the starting address of the first flash memory block is equal to the original starting address, and the starting address of the second flash memory block is equal to the original starting address + 16kB, by which the starting address of any one flash memory block can be determined by analogy, and the same applies to the flash memory page, which will not be repeated here.

[0109] In an exemplary embodiment of the present application, the flash memory address included in the second read command may be represented by an address of one flash memory page; When generating one or more second read commands for each SSD in which the data to be read this time is stored, the read processor 122 may generate one second read command for each allocated flash memory page in the SSD, and may include the address of the flash memory page to be read and the start address of data reception.

[0110] In an exemplary embodiment of the present application, for a second read command that includes only a flash memory address and a start address of data reception, the SSD can retrieve all data stored in at least one of the corresponding one or more flash memory pages or flash memory blocks from the SSD based on the flash memory address, and store them at the start address of data reception.

[0111] In an exemplary embodiment of the present application, the flash memory address included in the second read command may be represented by an address of one flash memory page; When generating one or more second read commands for each SSD in which the data to be read this time is stored, the read processor 122 may generate one second read command for each allocated flash memory page in the SSD, and may include the address of the flash memory page waiting to be read, the offset of the data waiting to be read this time relative to the address of the flash memory page, and the start address of data reception.

[0112] In an exemplary embodiment of the present application, for a second read command including a flash memory address, an offset, and a start address of data reception, the SSD can retrieve all data stored in a corresponding flash memory page based on the flash memory address, extract required data from the read data based on the offset, and store the extracted data at the start address of data reception, or read part of the data stored in a flash memory page based on the analyzed flash memory address and its offset.

[0113] In an exemplary embodiment of the present application, the flash memory address included in the second read command may be an address segment consisting of addresses of multiple consecutive flash memory pages, and the address segment is indicated by a start address and data amount information; When generating one or more second read commands for each SSD to which a flash memory address is assigned, the read processor 122 may generate one second read command for each address segment in the SSD, and may include the start address and data amount information of the address segment to be read, and the start address of data reception.

[0114] In an exemplary embodiment of the present application, for a second read command including a flash memory address, data amount information, and a start address of data reception, the SSD can retrieve all data stored in at least one of the corresponding multiple flash memory pages or flash memory blocks based on the flash memory address and data amount information, and store them at the start address of data reception.

[0115] Third aspect: An embodiment of the present application provides a solid state drive (SSD) in a storage system, which may include, as shown in FIG. 3, a master chip 21 and a flash memory chip 22, and the master chip 21 may include a second PCIe bus interface 211 and a storage controller 212; the second PCIe bus interface 211 is connected to a PCIe bus and configured to communicate with a CSP in the storage system and with an external device to the storage system via the PCIe bus; The storage controller 212 is configured to receive a second operation command based on the flash memory address sent by the CSP, and exchange the data to be stored with the outside based on the information contained in the second operation command, where the second operation command includes a second write command and a second read command.

[0116] In an exemplary embodiment of the present application, the CSP receives an external storage request, i.e., the first operation command, via the first PCIe bus interface 11. The first operation command may include information related to the storage object. Based on the information related to the storage object and the resource information of each SSD to be stored, the CSP can perform operations such as assigning an SSD address to the data to be stored or retrieving the SSD address stored in the data to be stored, and generates a corresponding second operation command based on the processing result. Using the peer-to-peer communication mechanism of the PCIe bus, the CSP can send the generated second operation command via the second PCIe bus interface 211 to the master chip 21 of the assigned or retrieved SSD. The storage controller 212 of the SSD master chip 21 then exchanges data directly with the external requester based on the second operation command, without going through the CSP. Only command transmission occurs between the CSP and the SSD, eliminating the need for large amounts of data to pass through the CSP and the need for the CSP to copy unnecessary data. This significantly reduces the CSP's workload and reduces latency and power consumption.

[0117] The following describes in detail the embodiments of the present application in two aspects: data writing and data reading.

[0118] In an exemplary embodiment of the present application, the storage controller 212 may include a write controller 2121, which: receiving the second write command, and analyzing a start source address of the data to be stored included in the second write command and a flash memory address at which the data is to be stored in the SSD; The data to be stored is read from the outside based on the analyzed starting source address, and the read data is written to the analyzed flash memory address.

[0119] In an exemplary embodiment of the present application, when the write processor 121 sends a second write command to the SSD, it may send one second write command to each assigned flash memory address, or it may send one second write command to a group of consecutive flash memory addresses (i.e., an address segment).

[0120] In an exemplary embodiment of the present application, the second write command may include, but is not limited to, an NVMe write command.

[0121] In exemplary embodiments of the present application, the flash memory address, i.e., any form of address provided by the SSD to the outside, may be, for example, the address of a flash memory page, a flash memory block, or a storage block, or may be an address segment of a flash memory page, a flash memory block, or a storage block. If the flash memory address is a storage block address, the SSD provides a logical address to the outside, and if the flash memory address is an address of a flash memory page or a flash memory block, the SSD provides an operation interface for a physical address to the outside. In the embodiments of the present application, the flash memory address is described as an address of a flash memory page or a flash memory block.

[0122] In an exemplary embodiment of the present application, the analyzed flash memory address may be the address of one flash memory page, and the write controller 2121 may be configured to read data of the size of one flash memory page of the storage target from outside based on the analyzed starting source address, and write the data to the analyzed address of the flash memory page; or The analyzed flash memory address may be an address segment indicated by a starting source address and data amount information, and the address segment may include addresses of multiple complete flash memory pages, and the write controller 2121 may be configured to read a corresponding amount of data of the storage target from outside based on the analyzed starting source address and data amount information, and write it to the analyzed address segment.

[0123] In an exemplary embodiment of the present application, the second write command may include only the starting source address and flash memory address of the storage target, or may include both the starting source address and flash memory address of the storage target (e.g., the starting addresses of multiple flash memory pages or flash memory blocks) and information about the amount of data to be stored.

[0124] In an exemplary embodiment of the present application, when the second write command is an NVMe write command, the NVMe write command may be a normal NVMe command, including a starting source address of the storage object and a flash memory address, or the NVMe write command may be an extended NVMe command, including a starting source address of the storage object, data amount information and a flash memory address.

[0125] In an exemplary embodiment of the present application, the data amount information may include, but is not limited to, a total data capacity (eg, 32K, 64K, etc.), a required flash memory page, a total number of at least one of flash memory blocks, and the like.

[0126] In an exemplary embodiment of the present application, for example, for small data, the data can be stored in only one flash memory page, so one flash memory page is allocated to the small data, and the second write command only needs to include the starting source address of the data to be stored and the address of the flash memory page.

[0127] In an exemplary embodiment of the present application, for example, for large data, multiple flash memory pages are required to store the data, so multiple flash memory pages may be allocated to the large data and data amount information may be attached. For example, the total number of required flash memory pages may be attached, and the second write command may include the starting source address of the storage target, the flash memory address (the allocated one or more flash memory pages, at least one starting address of the flash memory block), and the data amount information.

[0128] In an exemplary embodiment of the present application, the size of each flash memory block is 16kB, so the starting address of the first flash memory block is equal to the original starting address, and the starting address of the second flash memory block is equal to the original starting address + 16kB, and by analogy, the starting address of any one flash memory block can be determined, and the same applies to the flash memory page, so it will not be repeated here.

[0129] In an exemplary embodiment of the present application, the write controller 2121 is further configured to return a success response to the CSP after successfully receiving the second write command sent by the CSP and successfully writing data, and may return one success response after successfully writing data according to each second write command, or may return one success response after successfully writing data according to all second write commands.

[0130] In an exemplary embodiment of the present application, the storage controller 212 may include a read controller 2122, which: receiving the second read command, and analyzing a flash memory address in the SSD where the data to be stored is stored and a start address of data reception, the flash memory address being included in the second read command; The data to be stored is read from the SSD based on the analyzed flash memory address, and the read data is written to the start address of external data reception.

[0131] In an exemplary embodiment of the present application, the CSP can search for the flash memory address in the SSD of the data of the storage object to be read based on the identifier of the storage object included in the first read command, and can include the flash memory address and the start address of the external requester's data reception in a second read command and send it to the SSD.

[0132] In an exemplary embodiment of the present application, the data of the storage object waiting to be read may be large data or small data, and if the data of the storage object is large data and occupies at least one complete flash memory page or flash memory block (for example, there is waste due to complete occupancy or incomplete occupancy), the second read command may only include the address of at least one of the one or more flash memory pages or flash memory blocks, and the start address of data reception. If the storage object is large data and occupies at least one of a plurality of contiguous flash memory pages or flash memory blocks, the second read command may include the start address of at least one of the plurality of contiguous flash memory pages or flash memory blocks, and the data amount information may indicate the quantity of at least one of the flash memory pages or flash memory blocks, so the second read command may include the start address of at least one of the plurality of contiguous flash memory pages or flash memory blocks, the data amount information of the storage object, and the start address of data reception. If the storage object is small data and occupies only a portion of the space of one flash memory page, and another portion of the space of the flash memory page is occupied by other data, the second read command can include not only the address of the flash memory page, but also the offset of the data of the storage object waiting to be read, so that the required data can be accurately read from the flash memory page based on the offset. Therefore, the second read command can include the address, offset, and start address of the flash memory page.

[0133] In an exemplary embodiment of the present application, the second read command may include, but is not limited to, an NVMe read command.

[0134] In an exemplary embodiment of the present application, the second read command may be an NVMe read command, and the flash memory address included in the NVMe read command is the address of one flash memory page; or The second read command may be an extended NVMe write command, and the flash memory address included in the extended NVMe read command is an address segment consisting of addresses of multiple consecutive flash memory pages, and the address segment may be indicated by a starting address and data amount information.

[0135] In an exemplary embodiment of the present application, if the second read command is an NVMe read command, the NVMe read command may be a normal NVMe command, including the flash memory address to be stored and the start address of data reception, or the NVMe write command may be an extended NVMe command, including the flash memory address to be stored, data amount information and the start address of data reception.

[0136] In an exemplary embodiment of the present application, the data amount information may include, but is not limited to, a total data capacity (eg, 32K, 64K, etc.), a required flash memory page, a total number of at least one of flash memory blocks, and the like.

[0137] In an exemplary embodiment of the present application, the offset refers to the offset of the small data in one flash memory page relative to the flash memory address of the flash memory page, and the offset can be represented by two offset amounts.

[0138] In an exemplary embodiment of the present application, a small piece of data may be stored in only one flash memory page, may completely occupy one flash memory page, or may only occupy a portion of the flash memory page, with the other portion being occupied by other small pieces of data. Therefore, for small pieces of data that completely occupy one flash memory page, the second read command only needs to include the start address of the data to be stored and the address of the flash memory page. For small pieces of data that occupy one flash memory page together with other small pieces of data, the second read command may include the start address of the data to be stored, the address of the flash memory page, and an offset of the small piece of data, which is used by the SSD to extract required data from the read data based on the offset after reading the data from the flash memory page.

[0139] In an exemplary embodiment of the present application, for large data, since storing the data requires multiple flash memory pages, data amount information may be attached when reading the large data (e.g., the total number of flash memory pages to be stored is attached), and the second read command includes the start address of receiving the data to be stored, the flash memory address (the start address of at least one of the flash memory pages, flash memory blocks to store the data), and the data amount information.

[0140] In an exemplary embodiment of the present application, the size of each flash memory block may be 16kB, so that the starting address of the first flash memory block is equal to the original starting address, and the starting address of the second flash memory block is equal to the original starting address + 16kB, by which the starting address of any one flash memory block can be determined by analogy, and the same applies to the flash memory page, which will not be repeated here.

[0141] In an exemplary embodiment of the present application, the analyzed flash memory address may be the address of one flash memory page, and the read controller 2122 is configured to read the data to be stored from the SSD based on the analyzed address of the flash memory page and write it to the analyzed start address of data reception.

[0142] In an exemplary embodiment of the present application, for a second read command that includes only a flash memory address and a start address of data reception, the SSD can retrieve all data stored in at least one of the corresponding one or more flash memory pages or flash memory blocks from the SSD based on the flash memory address, and store them at the start address of data reception.

[0143] In an exemplary embodiment of the present application, the analyzed flash memory address may be one address segment indicated by a starting address and data amount information, and the read controller 2122 is configured to read the corresponding amount of data of the storage target from the SSD based on the analyzed starting address and data amount information, and write it to the analyzed starting address of the data reception.

[0144] In an exemplary embodiment of the present application, for a second read command including a flash memory address, data amount information, and a start address of data reception, the SSD can retrieve all data stored in at least one of the corresponding multiple flash memory pages or flash memory blocks based on the flash memory address and data amount information, and store them at the start address of data reception.

[0145] In an exemplary embodiment of the present application, the analyzed flash memory address may be the address of one flash memory page and the offset of the data to be read this time relative to the address of the flash memory page, and the read controller 2122 is configured to read one page of data from the SSD based on the analyzed address of the flash memory page, extract required data from the one page of data based on the offset, and write it to the analyzed start address of data reception.

[0146] In an exemplary embodiment of the present application, for a second read command including a flash memory address, an offset, and a start address of data reception, the SSD can retrieve all data stored in a corresponding flash memory page based on the flash memory address, extract required data from the read data based on the offset, and store the extracted data at the start address of data reception, or read part of the data stored in a flash memory page based on the analyzed flash memory address and its offset.

[0147] In an exemplary embodiment of the present application, the read controller 2122 is further configured to return a success response to the CSP after successfully receiving the second read command sent by the CSP and successfully reading the data.

[0148] Fourth aspect: An embodiment of the present application provides a data writing method, which is applied to a CSP in a storage system, and the CSP can communicate with an SSD in the storage system and with the outside of the storage system via a PCIe bus. As shown in FIG. 4, the method may include steps S101 to S102.

[0149] In step S101, a first write command based on an external storage object is received.

[0150] In an exemplary embodiment of the present application, the CSP communicates with an external requester based on a peer-to-peer communication mechanism via a PCIe bus to receive an external storage request, such as the first write command, which may include relevant information to be stored.

[0151] In the exemplary embodiment of the present application, the first write command may include, but is not limited to, a storage object identifier, data length information, and a starting source address of the entire data.

[0152] In step S102, a second write command based on a flash memory address is generated based on the information included in the first write command and the locally maintained SSD resource information, and is sent to the SSD.

[0153] In an exemplary embodiment of the present application, the SSD resource information may include resource occupation information in the SSD, which may include, but is not limited to, addresses of all SSDs included in the storage system, flash memory pages in each SSD, at least one address of a flash memory block, resource occupation information of each flash memory page, identifiers and offsets of storage objects corresponding to each occupied flash memory page, etc.

[0154] In an exemplary embodiment of the present application, the CSP can perform processes such as SSD address allocation for the data to be stored based on the related information of the storage object and the resource information of each SSD to be stored, and generate a corresponding second write command based on the processing result. Using the peer-to-peer communication mechanism of the PCIe bus, the CSP can send the generated second operation command to the assigned SSD, so that the SSD can exchange data directly with the external requester based on the second write command, without going through the CSP. Only the command is transmitted between the CSP and the SSD, so there is no large amount of data passing through the CSP, and the CSP does not need to copy unnecessary data. This significantly reduces the workload of the CSP and reduces latency and power consumption.

[0155] In an exemplary embodiment of the present application, generating and sending a second write command based on a flash memory address to the SSD based on information included in the first write command and locally maintained SSD resource information includes: Analyzing a storage target identifier, data length information, and a start source address of the entire data included in the first write command; Allocating flash memory addresses in one or more SSDs to be used for writing the data to be stored based on the data length information and resource occupation information in SSDs, and determining a start source address of a portion of the data to be written each time based on a start source address of the entire data and an offset in the entire data to be stored of the portion of the data to be written each time; This may include generating and sending one or more second write commands to each SSD to which the flash memory addresses are assigned, each second write command including a starting source address of data to be written to the SSD at this time and a flash memory address to be used for writing the data.

[0156] In exemplary embodiments of the present application, the flash memory address, i.e., any form of address provided by the SSD to the outside, may be, for example, the address of a flash memory page, a flash memory block, or a storage block, or may be an address segment of a flash memory page, a flash memory block, or a storage block. If the flash memory address is a storage block address, the SSD provides a logical address to the outside, and if the flash memory address is an address of a flash memory page or a flash memory block, the SSD provides an operation interface for a physical address to the outside. In the embodiments of the present application, the flash memory address is described as an address of a flash memory page or a flash memory block.

[0157] In an exemplary embodiment of the present application, the CSP can determine flash memory addresses in one or more SSDs that can store data of the storage object currently waiting to be stored based on the resource occupation information of the SSD, and assign one or more of the flash memory addresses to the storage object based on the data volume size of the storage object waiting to be stored.

[0158] In an exemplary embodiment of the present application, the CSP can first determine whether the data to be stored is large data or small data based on the data length information included in the first write command, and then assign a corresponding flash memory address based on the determined data size.

[0159] In an exemplary embodiment of the present application, if the data length information is less than or equal to a predetermined first data amount threshold, the data may be determined as small data, and if the data length information is greater than the predetermined first data amount threshold, the data may be determined as large data.

[0160] In an exemplary embodiment of the present application, the first data amount threshold may be the stored data amount of one flash memory page, or a customized data amount, where the specific value of the first data amount threshold is not limited.

[0161] In an exemplary embodiment of the present application, if the data waiting to be written is small data, the CSP may allocate only one flash memory page to the data; if the data waiting to be written is large data, the CSP may allocate multiple flash memory pages to the data, or may allocate one or more flash memory blocks.

[0162] In an exemplary embodiment of the present application, when the CSP allocates at least one of a plurality of flash memory pages and a plurality of flash memory blocks to the large data, the plurality of flash memory pages may be flash memory pages with consecutive addresses, and the plurality of flash memory blocks may be flash memory blocks with consecutive addresses, thereby allowing the large data to be allocated one address segment with consecutive addresses.

[0163] In an exemplary embodiment of the present application, if the large amount of storage objects waiting to be written are all small data, one second write command may be generated each time one small data is written. In another embodiment, multiple small data that need to be continuously written may be merged and then written, that is, one second write command may be generated after merging, thereby reducing the workload of the storage system, improving storage system performance, and reducing power consumption.

[0164] In an exemplary embodiment of the present application, for example, after receiving a plurality of first write commands for a plurality of small data, the information contained in the plurality of first write commands is merged, and a second write command is generated and sent to the SSD based on the merged information and locally maintained SSD resource information, and then the SSD respectively retrieves each small data based on the corresponding starting source address and stores it in the assigned flash memory address.

[0165] In an exemplary embodiment of the present application, when the starting source addresses of multiple small data are consecutive, multiple consecutive flash memory addresses can be assigned to the multiple small data based on the arrangement order of the small data stored at the multiple starting source addresses, and the arrangement order of the small data stored at the multiple consecutive flash memory addresses is the same as the arrangement order of the small data stored at the multiple starting source addresses.

[0166] In an exemplary embodiment of the present application, when the storage object is allocated to multiple different SSDs, or allocated to different flash memory addresses in the same SSD, the amount of data required for each flash memory address can be obtained from an external requester and stored in the flash memory address. Therefore, based on the starting source address of the entire data of the storage object and the offset of the portion of the data of the storage object to be written each time in the entire data of the storage object, the starting source address of the portion of the data to be written each time is determined, and corresponding data is obtained and stored based on the starting source address of the portion of the data.

[0167] In an exemplary embodiment of the present application, when the CSP sends a second write command to the SSD, the CSP may send one second write command to each assigned flash memory address, or may send one second write command to a group of consecutive flash memory addresses (i.e., an address segment).

[0168] In an exemplary embodiment of the present application, the second write command may include, but is not limited to, an NVMe write command.

[0169] In an exemplary embodiment of the present application, the second write command may be an NVMe write command, and the flash memory address included in the NVMe write command is the address of one flash memory page; or The second write command may be an extended NVMe write command, and the flash memory address included in the extended NVMe write command is an address segment consisting of addresses of multiple consecutive flash memory pages, and the address segment may be indicated by a starting address and data amount information.

[0170] In an exemplary embodiment of the present application, the second write command may include only the starting source address and flash memory address of the storage target, or may include both the starting source address and flash memory address of the storage target (e.g., the starting addresses of multiple flash memory pages or flash memory blocks) and information about the amount of data to be stored.

[0171] In an exemplary embodiment of the present application, when the second write command is an NVMe write command, the NVMe write command may be a normal NVMe command, including a starting source address of the storage object and a flash memory address, or the NVMe write command may be an extended NVMe command, including a starting source address of the storage object, data amount information and a flash memory address.

[0172] In an exemplary embodiment of the present application, the data amount information may include, but is not limited to, a total data capacity (eg, 32K, 64K, etc.), a required flash memory page, a total number of at least one of flash memory blocks, and the like.

[0173] In an exemplary embodiment of the present application, for example, for small data, the data can be stored in only one flash memory page, so one flash memory page is allocated to the small data, and the second write command only needs to include the starting source address of the data to be stored and the address of the flash memory page.

[0174] In an exemplary embodiment of the present application, for example, for large data, multiple flash memory pages are required to store the data, so multiple flash memory pages may be allocated to the large data and data amount information may be attached. For example, the total number of required flash memory pages may be attached, and the second write command may include the starting source address of the storage target, the flash memory address (the allocated one or more flash memory pages, at least one starting address of the flash memory block), and the data amount information.

[0175] In an exemplary embodiment of the present application, the size of each flash memory block is 16kB, so the starting address of the first flash memory block is equal to the original starting address, and the starting address of the second flash memory block is equal to the original starting address + 16kB, and by analogy, the starting address of any one flash memory block can be determined, and the same applies to the flash memory page, so it will not be repeated here.

[0176] In an exemplary embodiment of the present application, the CSP further receives a success response returned by the SSD after the SSD successfully receives the second write command and successfully writes the data.

[0177] Fifth aspect: An embodiment of the present application provides a data writing method, which is applied to an SSD in a storage system, and the SSD communicates with a CSP in the storage system and with the outside of the storage system via a PCIe bus. As shown in FIG. 5, the method may include steps S201 to S202.

[0178] In step S201, a second write command based on the flash memory address sent by the CSP is received, and the start source address of the data to be written this time and the flash memory address in the SSD included in the second write command are analyzed.

[0179] In an exemplary embodiment of the present application, when the SSD receives the second write command sent by the CSP, each flash memory address may receive one second write command, or a plurality of consecutive flash memory addresses (i.e., an address segment) may receive one second write command.

[0180] In an exemplary embodiment of the present application, the second write command may include, but is not limited to, an NVMe write command.

[0181] In an exemplary embodiment of the present application, the second write command may include only the starting source address and flash memory address of the storage target, or may include both the starting source address and flash memory address of the storage target (e.g., the starting addresses of multiple flash memory pages or flash memory blocks) and information about the amount of data to be stored.

[0182] In an exemplary embodiment of the present application, when the second write command is an NVMe write command, the NVMe write command may be a normal NVMe command, including a starting source address of the storage object and a flash memory address, or the NVMe write command may be an extended NVMe command, including a starting source address of the storage object, data amount information and a flash memory address.

[0183] In an exemplary embodiment of the present application, the data amount information may include, but is not limited to, a total data capacity (eg, 32K, 64K, etc.), a required flash memory page, a total number of at least one of flash memory blocks, and the like.

[0184] In an exemplary embodiment of the present application, for example, for small data, the data can be stored in only one flash memory page, so one flash memory page is allocated to the small data, and the second write command only needs to include the starting source address of the data to be stored and the address of the flash memory page.

[0185] In an exemplary embodiment of the present application, for example, for large data, multiple flash memory pages are required to store the data, so multiple flash memory pages may be allocated to the large data and data amount information may be attached. For example, the total number of required flash memory pages may be attached, and the second write command may include the starting source address of the storage target, the flash memory address (the allocated one or more flash memory pages, at least one starting address of the flash memory block), and the data amount information.

[0186] In an exemplary embodiment of the present application, for example, the size of each flash memory block is 16kB, so the starting address of the first flash memory block is equal to the original starting address, and the starting address of the second flash memory block is equal to the original starting address + 16kB, and by analogy, the starting address of any one flash memory block can be determined, and the same applies to the flash memory page, so it will not be repeated here.

[0187] In step S202, the data to be stored is read from outside based on the analyzed starting source address and written to the analyzed flash memory address.

[0188] In an exemplary embodiment of the present application, the second write command may be an NVMe write command, and the flash memory address included in the NVMe write command is the address of one flash memory page; Reading data of the storage target from outside based on the analyzed starting source address and writing it to the analyzed flash memory address includes reading data of the size of one flash memory page of the storage target from outside based on the analyzed starting source address and writing it to the analyzed address of the flash memory page.

[0189] In an exemplary embodiment of the present application, the second write command may be an extended NVMe write command, and the flash memory address included in the extended NVMe write command is an address segment consisting of addresses of multiple consecutive flash memory pages, and the address segment is indicated by a start address and data amount information; Reading the data of the storage target from outside based on the analyzed starting source address and writing it to the analyzed flash memory address includes reading a corresponding amount of data of the storage target from outside based on the analyzed starting source address and data amount information and writing it to the analyzed address segment.

[0190] In an exemplary embodiment of the present application, the SSD returns a success response to the CSP after successfully receiving the second write command sent by the CSP and successfully writing data, and may return a success response once after successfully writing data based on each second write command, or after successfully writing data based on all second write commands.

[0191] Sixth aspect: An embodiment of the present application provides a data reading method, which is applied to a CSP in a storage system, and the CSP communicates with an SSD in the storage system and with the outside of the storage system via a PCIe bus. As shown in FIG. 6, the method may include steps S301 to S302.

[0192] In step S301, a first read command based on an external storage object is received.

[0193] In an exemplary embodiment of the present application, the CSP communicates with an external requester based on a peer-to-peer communication mechanism via a PCIe bus to receive an external storage request, such as the above-mentioned first read command, which may include relevant information of the storage object to be read.

[0194] In the exemplary embodiment of the present application, the first read command may include, but is not limited to, a storage object identifier and a start address for receiving data.

[0195] In step S302, a second read command based on a flash memory address is generated based on the information contained in the first read command and the locally maintained SSD resource information, and sent to the SSD.

[0196] In an exemplary embodiment of the present application, the SSD resource information may include, but is not limited to, addresses of all SSDs included in the storage system, flash memory pages in each SSD, at least one address of a flash memory block, resource occupation information of each flash memory page, identifiers and offsets of storage objects corresponding to each occupied flash memory page, etc.

[0197] In an exemplary embodiment of the present application, the CSP can search for the flash memory address in the SSD of the data of the storage object to be read based on the identifier of the storage object included in the first read command, and can include the flash memory address and the start address of the external requester's data reception in a second read command and send it to the SSD.

[0198] In an exemplary embodiment of the present application, generating and sending a second read command based on a flash memory address to the SSD based on information included in the first read command and locally maintained SSD resource information includes: Analyzing a storage target identifier and a data reception start address included in the first read command; Searching for flash memory addresses in one or more SSDs to which the previously stored data to be read this time is written based on the identifier of the storage object; This may include generating one or more second read commands for each SSD in which the data to be read this time is stored and sending them to the SSD, and each second read command including the flash memory address of the data to be read this time in the SSD and the start address of data reception.

[0199] In an exemplary embodiment of the present application, the data of the storage object waiting to be read may be large data or small data, and if the data of the storage object is large data and occupies at least one complete flash memory page or flash memory block (for example, there is waste due to complete occupancy or incomplete occupancy), the second read command may only include the address of at least one of the one or more flash memory pages or flash memory blocks, and the start address of data reception. If the storage object is large data and occupies at least one of a plurality of contiguous flash memory pages or flash memory blocks, the second read command may include the start address of at least one of the plurality of contiguous flash memory pages or flash memory blocks, and the data amount information may indicate the quantity of at least one of the flash memory pages or flash memory blocks, so the second read command may include the start address of at least one of the plurality of contiguous flash memory pages or flash memory blocks, the data amount information of the storage object, and the start address of data reception. If the storage object is small data and occupies only a portion of the space of one flash memory page, and another portion of the space of the flash memory page is occupied by other data, the second read command can include not only the address of the flash memory page, but also the offset of the data of the storage object waiting to be read, so that the required data can be accurately read from the flash memory page based on the offset. Therefore, the second read command can include the address, offset, and start address of the flash memory page.

[0200] In an exemplary embodiment of the present application, when the CSP sends a second read command to the SSD, the CSP may send one second read command to each assigned flash memory address, or may send one second read command to a group of consecutive flash memory addresses (i.e., an address segment).

[0201] In an exemplary embodiment of the present application, the second read command may include, but is not limited to, an NVMe read command.

[0202] In an exemplary embodiment of the present application, the second read command may be an NVMe read command, and the flash memory address included in the NVMe read command is the address of one flash memory page; or The second read command may be an extended NVMe write command, and the flash memory address included in the extended NVMe read command is an address segment consisting of addresses of one flash memory page or multiple consecutive flash memory pages, and the address segment may be indicated by a starting address and data amount information.

[0203] In an exemplary embodiment of the present application, if the second read command is an NVMe read command, the NVMe read command may be a normal NVMe command, including the flash memory address to be stored and the start address of data reception, or the NVMe write command may be an extended NVMe command, including the flash memory address to be stored, data amount information and the start address of data reception.

[0204] In an exemplary embodiment of the present application, the data amount information may include, but is not limited to, a total data capacity (eg, 32K, 64K, etc.), a required flash memory page, a total number of at least one of flash memory blocks, and the like.

[0205] In an exemplary embodiment of the present application, the offset refers to the offset of the small data in one flash memory page relative to the flash memory address of the flash memory page, and the offset can be represented by two offset amounts.

[0206] In an exemplary embodiment of the present application, a small piece of data may be stored in only one flash memory page, may completely occupy one flash memory page, or may only occupy a portion of the flash memory page, with the other portion being occupied by other small pieces of data. Therefore, for small pieces of data that completely occupy one flash memory page, the second read command only needs to include the start address of the data to be stored and the address of the flash memory page. For small pieces of data that occupy one flash memory page together with other small pieces of data, the second read command may include the start address of the data to be stored, the address of the flash memory page, and an offset of the small piece of data, which is used by the SSD to extract required data from the read data based on the offset after reading the data from the flash memory page.

[0207] In an exemplary embodiment of the present application, for large data, since storing the data requires multiple flash memory pages, data amount information may be attached when reading the large data (e.g., the total number of flash memory pages to be stored is attached), and the second read command includes the start address of receiving the data to be stored, the flash memory address (the start address of at least one of the flash memory pages, flash memory blocks to store the data), and the data amount information.

[0208] In an exemplary embodiment of the present application, the size of each flash memory block may be 16kB, so that the starting address of the first flash memory block is equal to the original starting address, and the starting address of the second flash memory block is equal to the original starting address + 16kB, by which the starting address of any one flash memory block can be determined by analogy, and the same applies to the flash memory page, which will not be repeated here.

[0209] Seventh aspect: An embodiment of the present application provides a data reading method, which is applied to an SSD in a storage system, and the SSD communicates with a CSP in the storage system and with the outside of the storage system via a PCIe bus. As shown in FIG. 7, the method may include steps S401 to S402.

[0210] In step S401, a second read command based on the flash memory address sent by the CSP is received, and the flash memory address in the SSD of the data to be read this time and the start address of data reception included in the second read command are analyzed.

[0211] In an exemplary embodiment of the present application, the CSP can search for the flash memory address in the SSD of the data of the storage object to be read based on the identifier of the storage object contained in the first read command received from the external requester, and can include the flash memory address and the start address of data reception of the external requester in a second read command and send it to the SSD.

[0212] In an exemplary embodiment of the present application, the data of the storage object waiting to be read may be large data or small data, and if the data of the storage object is large data and occupies at least one complete flash memory page or flash memory block (for example, there is waste due to complete occupancy or incomplete occupancy), the second read command may only include the address of at least one of the one or more flash memory pages or flash memory blocks, and the start address of data reception. If the storage object is large data and occupies at least one of a plurality of contiguous flash memory pages or flash memory blocks, the second read command may include the start address of at least one of the plurality of contiguous flash memory pages or flash memory blocks, and the data amount information may indicate the quantity of at least one of the flash memory pages or flash memory blocks, so the second read command may include the start address of at least one of the plurality of contiguous flash memory pages or flash memory blocks, the data amount information of the storage object, and the start address of data reception. If the storage object is small data and occupies only a portion of the space of one flash memory page, and another portion of the space of the flash memory page is occupied by other data, the second read command can include not only the address of the flash memory page, but also the offset of the data of the storage object waiting to be read, so that the required data can be accurately read from the flash memory page based on the offset. Therefore, the second read command can include the address, offset, and start address of the flash memory page.

[0213] In an exemplary embodiment of the present application, when the CSP sends a second read command to the SSD, the CSP may send one second read command to each assigned flash memory address, or may send one second read command to a group of consecutive flash memory addresses (i.e., an address segment).

[0214] In an exemplary embodiment of the present application, the second read command may include, but is not limited to, an NVMe read command.

[0215] In an exemplary embodiment of the present application, the second read command may be an NVMe read command, and the flash memory address included in the NVMe read command is the address of one flash memory page; or The second read command may be an extended NVMe write command, and the flash memory address included in the extended NVMe read command is an address segment consisting of addresses of one flash memory page or multiple consecutive flash memory pages, and the address segment can be indicated by a starting address and data amount information.

[0216] In an exemplary embodiment of the present application, if the second read command is an NVMe read command, the NVMe read command may be a normal NVMe command, including the flash memory address to be stored and the start address of data reception, or the NVMe write command may be an extended NVMe command, including the flash memory address to be stored, data amount information and the start address of data reception.

[0217] In an exemplary embodiment of the present application, the data amount information may include, but is not limited to, a total data capacity (eg, 32K, 64K, etc.), a required flash memory page, a total number of at least one of flash memory blocks, and the like.

[0218] In an exemplary embodiment of the present application, the offset refers to the offset of the small data in one flash memory page relative to the flash memory address of the flash memory page, and the offset can be represented by two offset amounts.

[0219] In an exemplary embodiment of the present application, a small piece of data may be stored in only one flash memory page, may completely occupy one flash memory page, or may only occupy a portion of the flash memory page, with the other portion being occupied by other small pieces of data. Therefore, for small pieces of data that completely occupy one flash memory page, the second read command only needs to include the start address of the data to be stored and the address of the flash memory page. For small pieces of data that occupy one flash memory page together with other small pieces of data, the second read command may include the start address of the data to be stored, the address of the flash memory page, and an offset of the small piece of data, which is used by the SSD to extract required data from the read data based on the offset after reading the data from the flash memory page.

[0220] In an exemplary embodiment of the present application, for large data, since multiple flash memory pages are required to store the data, data amount information may be attached when reading the large data (e.g., the total number of flash memory pages to be stored is attached), and the second read command includes the start address of receiving the data to be stored, the flash memory address (one or more flash memory pages to be stored, at least one start address of the flash memory block), and the data amount information.

[0221] In an exemplary embodiment of the present application, the size of each flash memory block may be 16kB, so that the starting address of the first flash memory block is equal to the original starting address, and the starting address of the second flash memory block is equal to the original starting address + 16kB, by which the starting address of any one flash memory block can be determined by analogy, and the same applies to the flash memory page, which will not be repeated here.

[0222] In step S402, the data to be stored is read from the SSD based on the analyzed flash memory address, and written to the analyzed start address of external data reception.

[0223] In an exemplary embodiment of the present application, the second read command is an NVMe read command, and the flash memory address included in the NVMe read command is the address of one flash memory page or flash memory block; Reading the data to be stored from the SSD based on the analyzed flash memory address and writing it to the analyzed start address of external data reception may include reading all data of the corresponding flash memory page or flash memory block from the SSD based on the address of the analyzed flash memory page or flash memory block and writing it to the analyzed start address of external data reception.

[0224] In an exemplary embodiment of the present application, for a second read command that includes only a flash memory address and a start address of data reception, the SSD can retrieve all data stored in at least one of the corresponding one or more flash memory pages or flash memory blocks from the SSD based on the flash memory address, and store them at the start address of data reception.

[0225] In an exemplary embodiment of the present application, the second read command may be an extended NVMe read command, and the flash memory address included in the extended NVMe read command is an address segment consisting of addresses of multiple consecutive flash memory pages, and the address segment is indicated by a starting address and data amount information, and reading the data of the storage target from the SSD based on the analyzed flash memory address and writing it to the starting address of the analyzed external data reception may include reading a corresponding data amount of data of the storage target from the SSD based on the analyzed address segment and data amount information, and writing it to the starting address of the analyzed external data reception.

[0226] In an exemplary embodiment of the present application, for a second read command including a flash memory address, data amount information, and a start address of data reception, the SSD can retrieve all data stored in at least one of the corresponding multiple flash memory pages or flash memory blocks based on the flash memory address and data amount information, and store them at the start address of data reception.

[0227] In an exemplary embodiment of the present application, after the SSD successfully receives the second read command sent by the CSP and successfully reads the data, it returns a success response to the CSP.

[0228] Eighth aspect: In an embodiment of the present application, a CSP is provided, which may include a third PCIe bus interface 31 and a first logic circuit assembly 32 coupled to the third PCIe bus interface 31, as shown in FIG. 8, wherein the first logic circuit assembly 32 is configured to perform the data writing method described in the fourth aspect and the data reading method described in the sixth aspect.

[0229] In the exemplary embodiment of the present application, any of the embodiments in the fourth and sixth aspects is applicable to the content of the eighth aspect, and will not be described again here.

[0230] Ninth aspect: An embodiment of the present application further provides a master chip 21 of the SSD, which may include a fourth PCIe bus interface 41 and a second logic circuit assembly 42 coupled to the fourth PCIe bus interface 41, as shown in FIG. 9, wherein the second logic circuit assembly 42 is configured to perform the data writing method described in the fifth aspect and the data reading method described in the seventh aspect.

[0231] In the exemplary embodiments of the present application, any of the embodiments in the first to seventh aspects is applied to the content of the ninth aspect, and will not be described again here.

[0232] Tenth aspect: An embodiment of the present application provides a non-volatile storage medium having a computer program stored therein, the computer program, when executed by a processor, performing the data writing methods described in the fourth and fifth aspects and the data reading methods described in the sixth and seventh aspects.

[0233] In the exemplary embodiments of the present application, any of the embodiments in the first to seventh aspects are applied to the contents of the tenth aspect, and will not be described again here.

[0234] As will be understood by those skilled in the art, all or some of the steps in the methods, systems, and functional modules / units in the devices disclosed above can be implemented as software, firmware, hardware, or a suitable combination. In hardware embodiments, the division among the functional modules / units mentioned above does not necessarily correspond to the division among physical modules. For example, one physical module may have multiple functions, or one function or step may be performed cooperatively by several physical modules. Some or all of the modules may be implemented as software executed by a processor, such as a digital signal processor or microprocessor, or as hardware, or as an integrated circuit, such as a dedicated integrated circuit. Such software may be distributed on computer-readable media, which may include computer storage media (or non-transitory media) and communication media (or transitory media). As known to those skilled in the art, the term computer storage media includes volatile and non-volatile, removable and non-removable media implemented in any method or technology for storing information (e.g., computer-readable instructions, data structures, program modules, or other data). Computer storage media include, but are not limited to, RAM, ROM, EEPROM, flash memory or other memory technology, CD-ROM, digital versatile disk (DVD) or other optical disk storage, disk boxes, storage tape, disk storage or other magnetic storage devices, or any other medium used to store desired information and which can be accessed by a computer. Additionally, as known to those skilled in the art, communication media typically include computer-readable instructions, data structures, program modules, or other data in a modulated data signal such as a carrier or other transport mechanism, and may include any information delivery media.

Claims

1. A storage system comprising a solid-state drive (SSD) and a computational storage processor (CSP), wherein communication between the SSD and the CSP, and between the SSD, the CSP and an external device of the storage system, is performed using a peer-to-peer communication protocol of a high-speed serial computer expansion bus (PCIe) bus; The CSP is configured to receive a first operation command based on an external storage object, and generate a second operation command based on a flash memory address based on information included in the first operation command and locally maintained SSD resource information, and send the second operation command to the SSD; the SSD is configured to exchange the data to be stored with an external device based on information included in the second operation command after receiving the second operation command; The storage system, wherein the first operation command includes a first write command and a first read command, and the second operation command includes a second write command and a second read command.

2. The SSD resource information includes resource occupation information in the SSD; After receiving the first write command, the CSP: Analyzing a storage target identifier, data length information, and a start source address of the entire data included in the first write command; Allocating flash memory addresses in one or more SSDs to be used for writing the data to be stored based on the data length information and resource occupation information in the SSDs, and determining a start source address of a portion of the data to be written each time based on a start source address of the entire data and an offset in the entire data to be stored of the portion of the data to be written each time; 2. The storage system of claim 1, wherein one or more second write commands are generated and sent to each SSD to which a flash memory address is assigned, and each second write command includes a starting source address of data to be written to the SSD at this time and a flash memory address to be used for data writing, thereby generating and sending second operation commands based on the flash memory address to the SSD.

3. or, after receiving a second write command, the SSD is configured to analyze a starting source address and a flash memory address included in the second write command, read data of a default size of the storage target from an external device based on the analyzed starting source address, and write the read data to the analyzed flash memory address, thereby exchanging the data of the storage target with an external device; or 3. The storage system of claim 2, wherein the second write command sent by the CSP further includes information on the amount of data to be written this time to the storage target, and when the SSD receives the second write command, it analyzes the starting source address, flash memory address, and data amount information included in the second write command, and based on the analyzed starting source address and data amount information, reads data of the corresponding amount of data to be written to the storage target from the outside, and writes the read data to the analyzed flash memory address, thereby exchanging data of the storage target with the outside.

4. the SSD resource information includes a flash memory address in the SSD to which the data to be stored is written; After receiving the first read command, the CSP: Analyzing a storage object identifier and a data reception start address included in the first read command; retrieving flash memory addresses in one or more SSDs to which the pre-stored data of the storage object is to be written based on the storage object identifier; 2. The storage system of claim 1, wherein one or more second read commands are generated for each SSD in which the data to be stored is stored and sent to the SSD, and each of the second read commands includes a flash memory address of the data to be read this time in the SSD and a start address for receiving the data, thereby generating and sending a second operation command based on the flash memory address to the SSD.

5. After receiving the second read command, the SSD analyzes the flash memory address and the start address of the data reception included in the second read command, reads the data to be read this time from one or more SSDs based on the analyzed flash memory address, and transmits the read data to the start address of the data reception externally, thereby exchanging the data to be stored with an external device; or The second read command sent by the CSP further includes information about the amount of data to be stored, and when the SSD receives the second read command, it analyzes the flash memory address, the start address of the data reception, and the data amount information included in the second read command, reads the data to be read this time from the SSD based on the analyzed flash memory address and the data amount information, and transmits the read data to the start address of the data reception externally, thereby exchanging the data to be stored with an external device, or 5. The storage system according to claim 4, wherein the second read command sent by the CSP further includes an offset of the flash memory address to be stored, and when the SSD receives the second read command, it analyzes the flash memory address and its offset and the start address of the data reception included in the second read command, and based on the analyzed flash memory address and its offset, reads a portion of the data to be stored from one flash memory page and writes it to the external start address of the data reception, or reads all the data of the flash memory page, extracts necessary data from all the data based on the offset, and writes it to the external start address of the data reception, thereby exchanging the data to be stored with the external device.

6. The SSD is further configured to return a success response to the CSP after successfully exchanging the data to be stored with an external device based on information included in the second operation command; 2. The storage system of claim 1, wherein the CSP is further configured to, after receiving success responses for all of the transmitted second operation commands, return a success response to the sender of the first operation command and update the locally maintained SSD resource information.

7. a computational storage processor (CSP) in a storage system, the CSP including a first high-speed serial computer expansion bus (PCIe) bus interface, a storage management system, and an information manager; the first high-speed serial computer expansion bus standard (PCIe) bus interface is connected to a PCIe bus and configured to communicate with a solid-state drive (SSD) in the storage system and with an external device to the storage system via the PCIe bus; the storage management system is configured to receive a first operation command based on an external storage target via a first PCIe bus interface, generate a second operation command based on a flash memory address based on information included in the first operation command and locally maintained SSD resource information, and transmit the second operation command to the SSD, the first operation command including a first write command and a first read command, and the second operation command including a second write command and a second read command; The information manager is a computational storage processor (CSP) in a storage system configured to maintain SSD resource information in the storage system.

8. The SSD resource information includes resource occupation information in the SSD; The storage management system includes a write processor, the write processor comprising: receiving an external first write command via the first PCIe bus interface, and analyzing a storage target identifier, data length information, and a start source address of the entire data included in the first write command; Allocate flash memory addresses in one or more SSDs to be used for writing the data to be stored based on the data length information and resource occupation information in the SSDs, and determine a start source address of a portion of the data to be written each time based on a start source address of the entire data and an offset of the portion of the data to be stored each time in the entire data to be stored; The CSP of claim 7, wherein one or more second write commands are generated and sent to each SSD to which a flash memory address is assigned, and each second write command is configured to include a starting source address of data to be written to the SSD at this time and a flash memory address to be used for writing the data.

9. the flash memory address included in the second write command is indicated by the address of one flash memory page; 9. The CSP of claim 8, wherein when the write processor generates one or more second write commands for each SSD to which a flash memory address is assigned, the write processor generates one second write command for each assigned flash memory page in the SSD, and includes a starting source address of data to be written to the flash memory page and an address of the flash memory page.

10. the flash memory address included in the second write command is one address segment consisting of addresses of one flash memory page or a plurality of consecutive flash memory pages, and the address segment is indicated by a start address and data amount information; The CSP of claim 8, wherein when the write processor generates one or more second write commands for each SSD to which a flash memory address is assigned, the write processor generates one second write command for each address segment in the SSD, and includes a starting source address of the data to be written to the address segment, and a starting address and data amount information of the address segment.

11. 9. The CSP of claim 8, wherein the write processor is further configured to: generate and send one or more second write commands to each SSD to which a flash memory address is assigned, and then, after receiving success responses for all sent second write commands, return a success response to a sender of the first write command and notify the information manager to record the identifier of the storage target and the flash memory address in the assigned SSD.

12. the SSD resource information includes a flash memory address in the SSD to which the data to be stored is written; The storage management system includes a read processor, the read processor comprising: receiving an external first read command via the first PCIe bus interface, and analyzing a storage target identifier and a start address of data reception included in the first read command; retrieving flash memory addresses in one or more SSDs to which the pre-stored data of the storage object is to be written based on the identifier of the storage object; The CSP of claim 7, wherein one or more second read commands are generated for each SSD in which the data to be stored is stored and sent to the SSD, and each second read command is configured to include the flash memory address of the data to be read this time in the SSD and the start address of data reception.

13. the flash memory address included in the second read command is indicated by the address of one flash memory page; The CSP of claim 12, wherein when the read processor generates one or more second read commands for each SSD in which the data to be read this time is stored, the read processor generates one second read command for each allocated flash memory page in the SSD, and includes the address of the flash memory page to be read and the start address of data reception.

14. the flash memory address included in the second read command is indicated by the address of one flash memory page; 13. The CSP of claim 12, wherein when the read processor generates one or more second read commands for each SSD in which the data to be read this time is stored, the read processor generates one second read command for each allocated flash memory page in the SSD, and includes the address of the flash memory page to be read, the offset in the stored data of the flash memory page of the data to be read this time, and the start address of data reception.

15. the flash memory address included in the second read command is one address segment consisting of addresses of one flash memory page or a plurality of consecutive flash memory pages, and the address segment is indicated by a start address and data amount information; 13. The CSP of claim 12, wherein when the read processor generates one or more second read commands for each SSD to which a flash memory address is assigned, the read processor generates one second read command for each address segment in the SSD, and includes a starting address and data amount information of the address segment to be read, and a starting address of data reception.

16. A solid-state drive (SSD) in a storage system, comprising: a master chip and a flash memory chip, the master chip comprising a second high-speed serial computer expansion bus standard (PCIe) bus interface and a storage controller; the second high-speed serial computer expansion bus standard (PCIe) bus interface is connected to a PCIe bus and configured to communicate with a computational storage processor (CSP) in the storage system and with an external device to the storage system via the PCIe bus; A solid-state drive (SSD) in a storage system, wherein the storage controller is configured to receive a second operation command based on a flash memory address sent by the CSP, and exchange data to be stored with an external device based on information contained in the second operation command, and the second operation command includes a second write command and a second read command.

17. The storage controller includes a write controller, the write controller comprising: receiving the second write command, and analyzing a start source address of the data to be stored included in the second write command and a flash memory address at which the data is to be stored in the SSD; 17. The SSD of claim 16, configured to read the data to be stored externally based on the analyzed starting source address, and write the read data to the analyzed flash memory address.

18. The analyzed flash memory address is an address of one flash memory page, and the write controller is configured to read data of the size of one flash memory page of the storage target from outside based on the analyzed starting source address, and write the data to the analyzed address of the flash memory page; or 18. The SSD of claim 17, wherein the analyzed flash memory address is one address segment indicated by a starting address and data amount information, the address segment including addresses of multiple complete flash memory pages, and the write controller is configured to read data of a corresponding data amount of the storage target from outside based on the analyzed starting source address and data amount information, and write the data to the analyzed address segment.

19. The storage controller includes a read controller, the read controller comprising: receiving the second read command, and analyzing a flash memory address in the SSD where the data to be stored is stored and a start address of data reception, the flash memory address being included in the second read command; 17. The SSD of claim 16, configured to read the data to be stored from the SSD based on the analyzed flash memory address, and write the read data to an external data receiving start address.

20. The analyzed flash memory address is an address of one flash memory page, and the read controller is configured to read the data to be stored from the SSD based on the analyzed address of the flash memory page, and write the data to the analyzed start address of data reception; or 20. The SSD of claim 19, wherein the analyzed flash memory address is one address segment indicated by a starting address and data amount information, the address segment including addresses of multiple complete flash memory pages, and the read controller is configured to read a corresponding amount of data of the storage target from the SSD based on the analyzed starting address and data amount information, and write it to the analyzed starting address of the data reception.

21. 20. The SSD of claim 19, wherein the analyzed flash memory address is an address of one flash memory page and an offset relative to the address of the flash memory page of data to be read this time, and the read controller is configured to read a portion of the data stored in one flash memory page based on the analyzed flash memory address and its offset, or read all of the data of the flash memory page from the SSD based on the analyzed address of the flash memory page, extract necessary data from all of the read data based on the offset, and write it to the analyzed start address of data reception.

22. 1. A data writing method, applied to a computational storage processor (CSP) in a storage system, the CSP communicating with a solid-state drive (SSD) in the storage system and with the outside of the storage system via a high-speed serial computer expansion bus (PCIe) bus, the method comprising: receiving a first write command based on an external storage object; generating a second write command based on a flash memory address based on information included in the first write command and locally maintained SSD resource information, and sending the second write command to the SSD.

23. The SSD resource information includes resource occupation information in the SSD; generating a second write command based on a flash memory address and transmitting the second write command to the SSD based on information included in the first write command and locally maintained SSD resource information; Analyzing a storage target identifier, data length information, and a start source address of the entire data included in the first write command; Allocating flash memory addresses in one or more SSDs to be used for writing the data to be stored based on the data length information and resource occupation information in the SSDs, and determining a start source address of a portion of the data to be written each time based on a start source address of the entire data and an offset in the entire data to be stored of the portion of the data to be written each time; 23. The data writing method of claim 22, further comprising: generating and sending one or more second write commands to each SSD to which a flash memory address is assigned, each second write command including a starting source address of data to be written to the SSD at this time and a flash memory address to be used for writing the data.

24. The second write command is an NVMe write command, and the flash memory address included in the NVMe write command is the address of one flash memory page; or 24. The data write method of claim 23, wherein the second write command is an extended NVMe write command, and the flash memory address included in the extended NVMe write command is one address segment consisting of addresses of one flash memory page or multiple consecutive flash memory pages, and the address segment is indicated by a starting address and data amount information.

25. 1. A data writing method, applied to a solid-state drive (SSD) in a storage system, the SSD communicating with a computational storage processor (CSP) in the storage system and with the outside of the storage system via a high-speed serial computer expansion bus (PCIe) bus, the method comprising: receiving a second write command based on the flash memory address sent by the CSP, and analyzing a start source address of data to be written this time and a flash memory address in the SSD included in the second write command; reading the data to be stored from outside based on the analyzed starting source address and writing it to the analyzed flash memory address.

26. the second write command is an NVMe write command, and the flash memory address included in the NVMe write command is the address of one flash memory page; 26. The data writing method of claim 25, wherein reading the data to be stored from outside based on the analyzed starting source address and writing it to the analyzed flash memory address comprises reading data of a size of one flash memory page to be stored from outside based on the analyzed starting source address and writing it to the analyzed address of the flash memory page.

27. The second write command is an extended NVMe write command, and the flash memory address included in the extended NVMe write command is one address segment consisting of addresses of multiple consecutive flash memory pages, and the address segment is indicated by a start address and data amount information; 26. The data writing method of claim 25, wherein reading the data to be stored from outside based on the analyzed starting source address and writing it to the analyzed flash memory address comprises reading a corresponding amount of data to be stored from outside based on the analyzed starting source address and the data amount information and writing it to the analyzed address segment.

28. 1. A data reading method, applied to a computational storage processor (CSP) in a storage system, the CSP communicating with a solid-state drive (SSD) in the storage system and with the outside of the storage system via a high-speed serial computer expansion bus (PCIe) bus, the method comprising: receiving a first read command based on an external storage object; generating a second read command based on a flash memory address and sending the second read command to the SSD based on information included in the first read command and locally maintained SSD resource information.

29. the SSD resource information includes a flash memory address in the SSD to which the data to be stored is written; generating a second read command based on a flash memory address based on information included in the first read command and locally maintained SSD resource information and transmitting the second read command to the SSD; Analyzing a storage target identifier and a data reception start address included in the first read command; Searching for a flash memory address in the SSD to which the previously stored data to be read this time is to be written based on the identifier of the storage object; 29. The data reading method of claim 28, further comprising: generating one or more second read commands for each SSD in which the data to be read this time is stored and sending the second read commands to the SSD, each of the second read commands including the flash memory address of the data to be read this time in the SSD and the start address of data reception.

30. The second read command is an NVMe read command, and the flash memory address included in the NVMe read command is the address of one flash memory page; or 30. The data reading method of claim 29, wherein the second read command is an extended NVMe write command, and the flash memory address included in the extended NVMe read command is an address segment consisting of addresses of one flash memory page or multiple consecutive flash memory pages, and the address segment is indicated by a starting address and data amount information.

31. 1. A data reading method, applied to a solid-state drive (SSD) in a storage system, the SSD communicating with a computational storage processor (CSP) in the storage system and with the outside of the storage system via a high-speed serial computer expansion bus (PCIe) bus, the method comprising: receiving a second read command based on the flash memory address sent by the CSP, and analyzing the flash memory address in the SSD of the data to be read this time and the start address of data reception included in the second read command; A data reading method comprising: reading the data to be stored from the SSD based on the analyzed flash memory address, and writing the data to the analyzed external start address of data reception.

32. the second read command is an NVMe read command, and the flash memory address included in the NVMe read command is the address of one flash memory page; 32. The data reading method of claim 31, wherein reading the data to be stored from the SSD based on the analyzed flash memory address and writing it to the analyzed external starting address of the data reception comprises reading one page of data from the SSD based on the analyzed flash memory page address and writing it to the analyzed external starting address of the data reception.

33. The second read command is an extended NVMe read command, and the flash memory address included in the extended NVMe read command is one address segment consisting of addresses of multiple consecutive flash memory pages, and the address segment is indicated by a start address and data amount information; 32. The data reading method of claim 31, wherein reading the data to be stored from the SSD based on the analyzed flash memory address and writing it to the analyzed external start address of receiving the data comprises reading a corresponding amount of data of the data to be stored from the SSD based on the analyzed address segment and the data amount information and writing it to the analyzed external start address of receiving the data.

34. 31. A computational storage processor (CSP) comprising: a third high-speed serial computer expansion bus standard (PCIe) bus interface; and a first logic circuit assembly coupled to the third PCIe bus interface, the first logic circuit assembly configured to perform the data writing method of any one of claims 22 to 24 and the data reading method of any one of claims 28 to 30.

35. 36. A master chip for a solid-state drive (SSD), comprising: a fourth high-speed serial computer expansion bus standard (PCIe) bus interface; and a second logic circuit assembly coupled to the fourth PCIe bus interface, the second logic circuit assembly configured to perform the data writing method of any one of claims 25 to 27 and the data reading method of any one of claims 31 to 33.

36. A non-volatile storage medium in which a computer program is stored, and which, when executed by a processor, performs the data writing method according to any one of claims 22 to 27 and the data reading method according to any one of claims 28 to 33.

Images